TY - JOUR
AU - TAVINOR, GRANT
AB - ABSTRACT Virtual worlds are notable for their realism, both for the sense of psychological immersion they provide and the apparent potential for users to perform activities and interact with objects and people “within” such worlds. This article asks whether the concept of “photographic transparency,” introduced by Kendall Walton to account for the inherent realism of photography, can be extended to account for the realism of virtual media. Specifically, I investigate whether virtual media allow for a kind of prosthetic seeing that enables users to see the real world in potentially expanded ways. The complications for the thesis of “virtual transparency” arise from the analysis of the objects and egos involved in virtual seeing. These complications show that virtual reality as a medium is not transparent, even though token uses may be. I In an important but often contested paper, Kendall Walton argues that to understand a special sense of realism inherent in photography, we need to understand that “the invention of the camera gave us not just a new method of making pictures and not just pictures of a new kind: it gave us a new way of seeing” (1984, 251). Moreover, we should conclude that photographs are a “visual prosthesis” like mirrors, microscopes, and telescopes. Elaborating on this point, Walton notes that [m]irrors are aids to vision, allowing us to us to see things in circumstances in which we would not otherwise be able to; with their help we can see around corners. Telescopes and microscopes extend our visual powers in other ways, enabling us to see things that are too far away or too small to be seen with the naked eye. (251) Photography's specific prosthetic “contribution to seeing” is that it allows us to “see into the past.” To substantiate the special realism of this photographic prosthesis, Walton claims that photographs are “transparent” in that we “see the world through them.” Walton's claim of photographic transparency, which he admits may involve an “extension” of the ordinary usage of the term “see” (252) has been controversial, but that controversy is not my concern here. Rather, I want to investigate how virtual reality (VR) media, of the kind popularized by the commercial release of virtual reality headsets such as the Sony PS4 VR and the HTC Vive, might also be credibly seen as a “new contribution to seeing” or as a kind of visual prosthesis allowing users to see things they would not otherwise be able to. One might wonder whether Walton's account of photographic transparency can be adapted to the case of VR. If photographic transparency is the idea that viewers really see what is depicted in photographs, then “virtual transparency” is the position that viewers of VR—employing VR headsets—really see the objects depicted in that medium: that we “see the world through them.” Examples that make virtual transparency prima facie plausible are relatively easy to find. In Matthew Pan and Günter Niemeyer's work at Disney Research (2017), a user wearing a stereoscopic headset is able to catch a real ball that is tossed in her direction, based on the motion‐tracked ball's depicted movement in the VR world.1 In another version of the experiment, the VR representations include a predicted trajectory of the ball, and Pan and Niemeyer note that “[t]he predictive assistance visualizations effectively increases user's senses but can also alter the user's strategy in catching” (2017). It is very natural to say in this case that the VR acted as a visual prosthesis, allowing the user not only to see the ball (which was otherwise occluded by the headset) but also to see the trajectory of that ball (something the user would not otherwise see, but rather infer or anticipate). In addition to this prima facie plausibility, virtual transparency can be bolstered by the details of Walton's account of photographic transparency. For Walton, the transparency of a representational medium depends on “how we acquire information” from that medium (1984, 262). Taking paintings to stand in contrast to photographs in not being a transparent medium, Walton argues that typically what ends up depicted in a painting of a landscape depends on what the painter believes about that landscape, but what is depicted on a photograph of that landscape depends not on the beliefs of the photographer, but on the physical mechanism involved in the photographic process.2 Photographs are transparent because the counterfactual dependence of their depicted content on the scene depicted, is nonintentionally mediated during the production of the depiction. Now, VR media, at least in the case described above, seem to meet this condition, although again certain clarifications need to be made. The motion of the ball as depicted by the stereoscopic headset is counterfactually dependent on the motion of the real ball, and, moreover, this depends not on the beliefs of any agent who is generating the depiction, but on the algorithm that tracks, calculates, and displays the position of the real ball. This process is fallible—the algorithm may make a faulty prediction about the position of the ball—but it is not because of the intervention of false beliefs, but likely because of an error in the algorithm or the hardware supporting its performance (such as where mirrors, telescopes, and other transparent media may induce errors because of some mechanical fault). While Walton does not discuss VR, he does describe a pair of scenarios that further justifies the conclusion that VR media are transparent in his sense. He describes the case of Helen, whose optic nerves have been connected to a device that allows a doctor to “stimulate the optic nerves at will”: The doctor then stimulates Helen's nerves in ways corresponding to what he sees, with the result that she has “visual” experiences like ones she would have normally if she were using her own eyes. … Helen seems to be seeing things, and her visual experiences are caused by the things she seems to see. But she doesn't really see them. … It is only because differences in scenes make for differences in the doctor's beliefs that they make for differences in her visual experiences. (1984, 265) Walton then presents a second scenario that he takes to be a case of transparency: Contrast a patient who receives a double eye transplant or a patient who is fitted with artificial prosthetic eyes. This patient does see. He is not relying in the relevant manner on anyone's beliefs about the things he sees, although his visual experiences do depend on the work of the surgeon and on the donor of the transplanted eyes or the manufacturer of the prosthetic ones. (1984, 265) The Disney VR research corresponds more closely to the second scenario because VR's depictions are nonintentionally mediated, instead being automatically generated by the running of a computer program. And this is so even if the depictions indirectly depend on the intentions of those who manufactured the stereoscopic headset and wrote the code for the specific application. Indeed, it may simply be that VR in such cases could be conceived of as “artificial prosthetic eyes,” the complication being that in this instance they generate a sort of Rube‐Goldbergian redundancy because of the presence of a perfectly functional set of natural eyes.3 Walton's position has of course been criticized, and in one of these criticisms, we can also see how in at least one respect, the virtual transparency thesis may be easier to defend than photographic transparency. In their critique of Walton's position, Jonathan Cohen and Aaron Meskin argue that photographs fail to meet what they claim to be a necessary condition of genuine seeing: that the transaction between the viewer and picture provide the viewer with “egocentric spatial information” about the objects seen (2004, 198). For Cohen and Meskin “x sees y through a visual process z only if z carries information about the egocentric location of y with respect to x” (210). Cohen and Meskin claim that not only does the information in transparent media need to be counterfactually dependent on the scene depicted, as claimed by Walton, but that the information should also (at least partly) be of a specific kind. So, contrasting genuine cases of prosthetic seeing such as mirrors and microscopes with photographs, they claim that “mirrors are transparent in Walton's sense because mirrors carry egocentric spatial information about objects. In contrast, our view secures the desired conclusion that photography is not transparent, insofar as the visual process of looking at photographs fails to carry egocentric spatial information about their depicta” (203). But as we see above, VR as a form of depiction does seem capable of conveying to the viewer egocentric spatial information about the objects depicted because it is not just the motion of the ball that is depicted, but this movement in relation to the apparent spatial position of the viewer. It is precisely this depiction of the user's egocentric spatial relationship to the ball that allows the user to catch the ball in the Disney research. There is reason (both prima facie and theoretical) to count VR media as transparent in Walton's sense and in Cohen and Meskin's more demanding analysis of transparency. In the following I will mostly follow Cohen and Meskin's framing of the issue. Nevertheless, there is also reason to be hesitant of concluding VR to be a transparent, depictive medium or a case of prosthetic seeing. The complications I present here for the virtual transparency thesis arise from uncertainty about, first, what is seen in VR, and, second, who is doing the seeing. Because of these complications, the claim I defend in this article is that if virtual depictions are transparent at all, they are so in only a limited set of cases. In the next part of this article, I address the former issue of virtual objects. In part III, the idea that the objects seen in virtual worlds are of a distinctive ontological nature is considered and rejected. Part IV investigates the complications that the identity of apparent virtual egos adds to the issue and expresses some resulting doubts about Cohen and Meskin's account of the kind of information necessary to seeing. I finish this article by making some observations on the distinction between virtual realism and virtual fictionalism and the lessons learned here about the virtues of virtual representation in expanding our experiences of real and fictional worlds. II What is it that the user sees when he or she interacts with virtual reality media? A tempting immediate rejoinder to the virtual transparency thesis is that the user does not really see the ball, because there is a stereoscopic headset—a big lump of plastic and computer circuitry—blocking his or her vision of the ball. Instead, what the user sees is a pixelated animation rendered on a pair of stereoscopic screens, mere centimeters before his or her eyes. The latter claim is partially true: the stereoscopic image is what the user sees (and, given current technology, users typically cannot help also noticing it because of the pixelated appearance and limited visual field of the image). However, as Walton notes in defense of the transparency thesis, “to be transparent is not necessarily to be invisible. We see photographs themselves when we see through them” (1984, 252). This observation holds in other cases of prosthetic seeing: while one sees an image in a microscope eyepiece, one also sees a bacterium wiggling on a plate. In VR, the immediate cause of the viewer's perceptual experience is a stereoscopic screen, and so it may be a case of “indirect seeing,” but then this is just to reemphasize the point that, like a microscope, VR is a case of prosthetic rather than native seeing. But how is it possible to see a ball through a stereoscopic headset? Naturally the Disney researchers do not take up the philosophical issues inherent in their work, but the details of the case deserve a little more attention. In the experiment, using a motion tracking camera, a ball's position is tracked and displayed on the stereoscopic headset as an animated ball within a rudimentary virtual environment comprising a textured floor, basic lighting, and paddle‐like depictions of the user's hands (Pan and Niemeyer 2017, 1). The virtual world is also capable of depicting the ball's predicted trajectory and a target where the ball can be intercepted by the catcher. Using this visual information, the user can orientate his or her hands (depicted by the paddles) to achieve the task of catching the ball in virtual and real space. It is the informational transaction embodied here that gives credence to the interpretation of this as a case in which the user really sees the ball even under Cohen and Meskin's more demanding account of seeing. Drawing on the work of Fred Dretske, Cohen and Meskin analyze spatial information as relying on a “kind of (objective) probabilistic, counterfactual‐supporting connection between independent variables” (2004, 200). A telescope carries egocentric spatial information about objects such as Mars, and so meets this necessary condition of prosthetic seeing, because the nature of the images of Mars it produces are probabilistically dependent on the direction in which the telescope is pointed; moreover, this dependence is shown by the counterfactual that if Mars had been in a different position, a change in the orientation of the telescope would be required to produce its image. Photography fails this criterion because a change in position or orientation of a photograph will alter the egocentric location of what is depicted in the picture while leaving the image unchanged (Cohen and Meskin 2004, 204). The case of VR described above meets this condition because changes in the position of the ball are tracked by the technology, and this spatial information is used to produce corresponding changes in the images on the stereoscopic display depicting the ball.4 If the user turns his or her head, this change in visual orientation is also tracked, and the image of the ball will move on the stereoscopic display to reflect the change in user orientation. The virtual depiction of the ball is counterfactually dependent on the actual position of the ball with respect to the user, even though the nature of the dependence here is mediated by algorithms (described in the research) and not merely the physical shifts in orientation that preserve the dependence in the case of telescopes (and in cases of native seeing). If this VR instance counts as seeing, then, it turns out that prosthetic seeing may be preserved by algorithmic as well as purely physical or mechanical processes.5 That the user can physically interact with and catch the ball on visually receiving this spatial information surely adds a strong intuition that the ball is really seen. However, we need to be careful with how we interpret the cases and the kind of evidence they provide. In some instances, there is reason to doubt that this kind of virtually mediated task performance involves the user seeing the object with which he or she interacts. As noted, the Disney simulation can depict the trajectory of the ball and a simple target where the ball can be intercepted. In both instances, the simulation depicts egocentric spatial information about the ball, and this information can be used (in absence of the virtual ball) to catch the real ball. In the first instance, the user sees an image of the predicted trajectory of the ball and uses this image to anticipate the position of the ball in real space. In the second instance, the user simply places his or her hands in the position in which the algorithms predict the ball will arrive. Pan and Niemeyer observed that this “seems to allow the catcher's hands to reach the catch location much earlier prior to catching” showing that they did not need to infer the trajectory of the ball, but rather simply used the target to position their hands (2017, 2). The VR in this case can be conceived of not as a case of seeing the ball but as a visual prop that aids the performance of a specific task, or as Pan and Niemeyer put it, “tools to assist a user in catching [sic] ball” (2). Successful interaction does not establish that one sees the object, because seeing an object is only one way to gather egocentric spatial information about the position of the object. Thus, at most, the interaction in the case of the Disney VR counts as evidence that the transaction conveys egocentric spatial information, rather counting as an instance of seeing a ball.6 Cohen and Meskin argue that visual provision of egocentric spatial information may be a necessary condition of seeing, but it is not sufficient for seeing to occur. One other necessary condition for seeing is surely the provision of a visual presentation of the object itself. Walton notes that a light‐sensitive machine producing verbal descriptions of visual scenes might meet the necessary condition of being nonintentionally mediated but would fail to be a case of seeing (1984, 270). His analysis of this is that description fails to maintain the “similarity relations” crucial to perception (271). In his analysis of the moving image, Noël Carroll makes much the same claim when he notes that a mechanical “account of transparent seeing or seeing through pictures must be supplemented by the stipulation that the presentations in question preserve real similarity relations betwixt the photo and that of which it is a photo” (2006, 121). There are other cases where egocentric spatial information about an object is conveyed, but where the visual qualities of the depiction are in some way altered, potentially undermining the intuition that a user really sees the object depicted. For example, the physical qualities depicted of the apparent object or the identity of the object (what kind of thing it is) may easily be altered in VR. What if the ball is depicted as a baked potato being thrown to the user? One does not see the baked potato—there is no baked potato—but does one still see the ball? Does one see the ball in the guise of a baked potato? This example is a case of “augmented reality” (AR) in which objects in the real world are overlaid with further depictive content, often altering what it is that one “sees.” The additional depictive content of augmented reality can be real features of the world such as the predicted trajectory of the ball in the case of Disney VR or fantastical objects such as the creatures depicted in the augmented reality non‐VR game Pokemon Go (2016). While it might seem that augmented reality provides a challenge to virtual transparency (allowing users to see things that are “not there” either in the sense of being invisible to natural vision or of not existing), there are at least two reasons to think that augmented reality is not only consistent with virtual transparency, but that AR may even enrich our understanding of virtual visual prostheses. First, other cases of prosthetic seeing such as microscopes and telescopes also augment native seeing by allowing us to see things invisible to native vision. For example, an AR simulation of a ball that allows us to perceive the trajectory of a ball can be compared to a coronagraph that allows us to see the corona of the sun. Indeed, the augmentation of native abilities is important sense of “prosthetic” and can also be traced to its origin in the Greek word prostithenai: “to add.” Second, Cohen and Meskin's account of prosthetic seeing is designed to allow for cases of seeing where we see things other than they really are. Their theory is framed to avoid problems with doxastic accounts of seeing—where certain kinds of belief about the object seen are a necessary component of seeing—because they do not think that one must have true beliefs about the objects they see. We can see objects even when we are radically confused or deceived about their nature, and we do not have to go to the extent of VR to find such cases: someone throws you a potato, but it is in fact a ball that has been painted to appear like a potato. Nevertheless, you see the ball and catch it. Cohen and Meskin conclude that “[b]elief is fragile with respect to perturbations that leave seeing intact, so no doxastic state can be necessary for seeing” (2004, 199). Thus, in VR one does not have to believe that what one is seeing is a ball to actually see it: one could instead believe it to be a potato. III Other real or hypothetical cases of VR are less clear, however, and may genuinely motivate the argument that what is seen is not an object in the real world but an object of some other standing. What of cases where there is no real object corresponding to the ball—where, for example, a merely simulated rather than a real ball is “tossed” to the user? In this case, users, when they reach out to catch the ball, may discover that they have simply been deceived. But what do they see? It certainly seems that they receive egocentric spatial information, because they attempt to use this information, and they may do so successfully if the VR program allows them to catch the virtual ball (perhaps by depicting their paddle like hands closing around the ball). This situation is a case of deception of which it is not clear we are even tempted to describe as a case of prosthetic seeing, but such cases do not exhaust the situations in which there might be no real corresponding object in an apparent instance of seeing. In fact, by far the most popular use of VR—videogames—provokes similar issues. In VR videogames such as The London Heist (2016) on the PS4 console, the player can manipulate objects in a virtual world that may have no corresponding real existence. In one scene, a player is meeting a criminal boss in a smoky East London pub, and he or she can pick up a cigarette lighter and light the criminal boss's cigar by reaching out to it, utilizing the gestural control afforded by the tracked position of the PS4 controller in his or her hand. This situation could be characterized in terms of prosthetic seeing, as players do relate themselves in space to the cigarette lighter and the cigar to perform the task of lighting the cigar in the virtual world: the VR depiction of the event does convey spatial information about apparent objects, egos, and their spatial relationship. And this spatial relationship is probabilistically dependent on the player's movement: if the player moves his or her head, the depictions of the cigarette lighter and cigar change to reflect this movement. Still, even if we do quite naturally say in such situations that “the player sees the cigar,” we are reluctant to say of this and other similar videogame cases that they are instances of prosthetic seeing, because, most obviously, the objects depicted do not really exist. Some philosophers have been tempted, because of these kinds of examples, to claim that the objects (and, as we will find later, egos) depicted in VR worlds are of some unique kind. If it is not the apparent object—a cigar—that is really seen in VR, then what is seen? David Chalmers argues that the objects depicted are “digital objects, constituted by computational processes on a computer.”7 Chalmers calls his position “digitalism,” which he contrasts with the “virtual fictionalism” of some other accounts of virtuality, and it leads him to claim that “virtual reality is a sort of genuine reality, virtual objects are real objects, and what goes on in virtual reality is truly real” (7). Fundamental to virtual digitalism is the reasonable claim that the objects that we see in virtual reality are those objects that are the causal origins of our perceptual experiences. For Chalmers, this is not the screen, but “data structures, which are grounded in computational processes which are themselves grounded in physical processes on one or more computers” (7). Chalmers subsequently develops two arguments for the identity of virtual objects and data objects: the causal argument and the perceptual argument. The perceptual argument infers the identity of virtual objects with digital objects from the causal origin of our perceptual experiences in the case of virtual reality: (1) When using virtual reality, we perceive (only) virtual objects. (2) The objects we perceive are the causal basis of our perceptual experiences. (3) When using virtual reality, the causal bases of our perceptual experiences are digital objects. (Chalmers) The causal argument works similarly, and both claim that VR objects cannot be fictions because fictional objects are not available to play the required perceptual and causal roles we do observe in interactions with VR worlds. At first appearances, if what users of VR really see are digital objects, and if users are provided with egocentric spatial information about those objects, Chalmers's position could be a case of virtual transparency: users of VR really see the objects of their experiences; it is just that these are digital objects. But this becomes complicated if we probe further and ask precisely where these digital objects are located. In the perceptual and causal arguments, the perceptual object is identified with its cause, and so one option is to say that the VR object is located within the pixel array of the VR headset that the user is viewing, because it is this “object” that provides the proximate perceptual and causal basis of the user's visual experiences. Now, in one sense this is exactly right: the VR user does see the images on the stereoscopic screen (and this is partly because they do receive egocentric spatial information about those images). But this is obviously problematic if we conceive of these virtual objects as being “balls” or “cigars” or as being located such that one may catch or use them through their real physical gestures. As noted, Chalmers actually denies that the immediate perceptual object is an image on a stereoscopic screen when he says that “in typical VR, one need have no sense of seeing a screen, and it can perhaps be argued that one does not see a screen at all” (9). Chalmers instead identifies digital objects with the data structures that produce this pixel array. But, ignoring the unlikely claim that users do not see the screen, this is no better than identifying the object with the stereoscopic images: it is equally problematic to think that what one sees in a case like the Disney research is a data structure distributed on “physical processes on one or more computers” or that one catches or otherwise interacts with such objects. Unless it is our technical concern, we do not naturally describe the perceptual object in the Disney experiments as a pixel array (or a data structure); rather, the object is a ball that has been thrown toward the user. Moreover, the position of this object is not merely centimeters before the user's eyes (or distributed on the hardware of a computer) but is traveling across the room and into his or her waiting hands. The virtual transparency thesis, if it is to be worth anything at all, is about this object and not a pixel array, and hence virtual digitalism is not a promising account of virtual transparency as formulated above. A similar criticism holds for the cases with purely imagined objects: in VR videogames we naturally say that the player “sees and interacts with a cigar,” and not a pixel array on a stereoscopic headset or a data structure in a computer. Of course, in both realist and fictive VR cases, we now need to explain how such things are possible (that is, how a user can “see” real balls or nonexistent cigars through a stereoscopic headset and gesture toward and interact with these things) but Chalmers's analysis of both as involving the seeing of digital objects is unpromising. There are several related problems in Chalmers's analysis, and resolving these is needed to understand those cases in which VR may count as real prosthetic seeing. First is Chalmers's conflation of the depiction and the object depicted. In realistic cases, the object depicted is not a digital object; rather, it is a real thing. The image through which one sees this real object surely is a digital object (and we know just what kind of digital object it is in the case of stereoscopic headsets: a texture‐mapped wireframe animation rendered on a stereoscopic pixel array). But again, that one immediately sees an image does not rule out that one may also see through this image the object depicted. Curiously, Chalmers seems to agree with Walton's transparency thesis when he claims that “[i]t is widely accepted that when we look at a photograph or a film clip of Winston Churchill, we see Winston Churchill,” then uses this observation to back his digitalist position that we really see digital objects. Forgetting Chalmers's dubious claim about the consensus on the transparency thesis, surely the real lesson to be drawn from the example is that in some cases at least, viewers of virtual media see the objects depicted in virtual media and not only the images or data structures themselves. But Chalmers is barred from concluding this because, while he acknowledges that some virtual worlds are fictions (9), his simplistic conception of how fictions involve representations does not allow him to fully acknowledge the difference between, and compatibility of, virtual realism and virtual fictionalism. Because of the perceptual and causal arguments, for Chalmers, it does not make sense that one could see fictional things through virtual representations, because this would demand that one causally and perceptually encounters something that may not exist. Hence, what we must encounter in virtual reality is restricted to the digital object that is directly encountered. But this undermines not only virtual fictionalism but also virtual realism and virtual transparency (that is, the idea that we might see real things through virtual representations). Chalmers's conclusion relies on an undertheorized conception of how fictions employ images and other representations. Walton's prop‐based account of fiction has it that “a fictional truth consists in there being a prescription or mandate in some context to imagine something” (1990, 39). Crucial in this theory are representational props, which are “generators of fictional truths, things which, by virtue of their nature or existence, make propositions fictional” (37). Hence, objects that the appreciator really encounters make it fictional that there are objects that the appreciator encounters (or sees). Virtual images clearly do play this role in VR games such as the terrifying survival horror game Resident Evil VII: in virtue of seeing the stereoscopic image of an insane cannibal skulking at the end of a darkened hall, the players understand that it is fictional that an insane cannibal stands before them, and they may also infer that it is fictional that they are in danger. This account has the virtue, unlike virtual digitalism, of preserving the way in which players talk about their involvement in virtual worlds where the reference to images on stereoscopic headsets, data objects, or even virtual objects or worlds is notably absent: players talk, instead, of encountering and being terrified of insane cannibals.8 So, what is the object that is seen in cases of VR? To answer this question, we need a subtler analysis of the different uses of VR. In all cases, what is directly seen is the image on the stereoscopic headset (a digital object). But this image plays at least two distinctive roles in VR. In the case of Disney's experiments (and other cases of virtual realism) it may allow the user to see real objects—even those ordinarily invisible to native vision—and comprises a genuine case of prosthetic seeing. In the case of the virtual cannibals in Resident Evil VII, the player sees the image, and this image warrants that he or she imagines things to be true of a fictional world and his or her involvement with it. Augmented reality may be a combination of these. IV The complications for the virtual transparency thesis do not end here, as worries with the form or identity of the object seen are not the only problem facing the thesis. If the transmission of egocentric spatial information is necessary to prosthetic seeing, to be comfortable with applying that term to VR, we also need a good understanding of identity or nature of the ego to which visual information is conveyed in the case of VR. In ordinary seeing, we take this ego to be identical with the person seeing, but there are reasons to resist this identification in some cases of VR. In the case of the Disney research, at least, this identity seems clear: it is the person catching the ball because this is the person able to locate the object in the space in which he or she also exists. The ego in this case is given a minimal depictive embodiment, comprising a visuospatial orientation on the virtual world and a set of paddle‐like hands. In fact, Pan and Niemeyer note that the minimal visual representation of the user's virtual body does not impede performance of the task of ball catching, because users do not even look at their hands until just before catching the ball, instead using proprioception to position their hands (2017, 2). It is the visuospatial orientation on the virtual world and the spatial information inherent in that perspective that is necessary to the psychomotor task in which they are interested and that is crucial to the representation of the virtual ego. In the Disney experiments, users are in the physical proximity of the ball, and based on their VR mediated perceptual access to the world, use their real bodies to interact with the real world. The VR ego spatially coincides with the real ego (both defined with reference to the visuospatial orientation of the VR headset). But other situations can prise apart this coincidence of real and virtual egos. For example, in VR controlled drones, users may don a VR headset and view the world from the vantage point of a camera located on a drone that they fly. Here, users receive spatial information about objects in the environment, as evidenced by their ability to interact with these objects—but the informational relationship in this case is between a proxy VR ego that the users adopt and the world as orientated around this proxy visual orientation. That is, the ego comprises the position of the perceptual vantage point of the drone: moving their heads around, the users alter the spatial orientation of the virtual ego in a way that maintains a probabilistic dependence relationship between the virtual ego and the objects viewed, even while the users need not receive any spatial information about their own real relationship to the objects seen (indeed, conceivably the objects could be in a location to which the users are completely oblivious).9 Is this a case of virtual transparency or prosthetic seeing? Cohen and Meskin's necessary condition for seeing does not hold here because the egocentric spatial information holds only between the apparent positions of virtual object and virtual ego.10 If the user's position in the real world is changed—for example if the whole VR system which he or she uses is located in a vehicle, and this vehicle moves—the virtual depiction will not change to reflect this motion. In such cases of the spatial dislocation of the user from the virtual ego, we may be tempted to deny that the user really sees the objects that he or she interacts with. Again, the problem here may be that the ability to understand, infer, and interact with objects is not evidence itself that one sees these objects, because seeing is merely one means of conveying the information needed for such tasks. But this is the key fact about virtual reality prostheses: what they necessitate is not a genuine egocentric informational transaction between user and object but a transaction that conveys information about the apparent position and nature of the object and ego that supports the kind of interaction desired in the specific application of VR. Another option is to take this case as a counterexample to Cohen and Meskin's theory and deny the necessity of the egocentric spatial information condition for prosthetic seeing, and indeed my own intuitions lead me in this direction. This case is not so different from the Disney case, where we can see that there is also a virtual or proxy ego involved (that is, that of the apparent perceptual vantage point depicted on the VR headset). In the Disney case, it is just that this virtual ego coincides with the real spatial position of the user (at least as this is defined as his or her visual perspective on the world), and indeed the correspondence is carefully orchestrated so that the user can achieve the task of catching the real ball. But we could alter the situation—in ways that may be shortly within the realms of technology—to tease the real and virtual ego apart even in this case. Rather than having the user catch the ball with his or her own hands, we could augment the virtual ego by situating its visual perspective within a robot that can catch the ball based on feedback from a bodysuit that the user wears and that tracks the user's bodily movements. This is essentiality the aim of “telerobotics” (of which the VR piloted drone is a rudimentary instance). We can easily imagine such robot proxies that allow for an experientially replete virtual involvement, including haptic feedback to the user (similar to those depicted in the film Avatar). In such cases, there is no egocentric spatial informational transaction between the user and the object he or she encounters; rather the informational relationship involves the apparent perspective of the avatar and the object. But I am extremely tempted to say that this is the ultimate case of prosthetic seeing, allowing the user to see and interact with the world at a distance through an entirely prosthetic body: an embodied virtual ego. In videogame uses of VR, the notion of the VR ego takes on additional complications, and indeed this notion of a virtual ego is often especially rich in a fictive context. Take again, for example, Resident Evil VII; here the player is given an apparent virtual ego in the form of a character within a fictional world. The character—Ethan Winters—is given a backstory, personal qualities, and various abilities to act in the world of the game. The player “sees” and “interacts” in this world via this virtual ego. And there is a special sense of realism in such games: that players seem to feel a profound existential threat in such games is for some players a psychological obstacle to playing the games (Tavinor 2018, 157). This sense of realism, however, does not directly concern the notion of prosthetic seeing but seems to consist of a psychological fact about how VR affects its users. Another notorious effect of VR realism is the motion sickness potentially caused by the visual orientation of the VR world not matching with feedback from the vestibular system (Lawson 2014). As with Chalmers's digitalism and its reification of virtual objects, some writers have been tempted to reify the egos or identities apparent in virtual media. J. David Velleman argues that in online virtual worlds such as Second Life, the player's avatar “is a chimerical creature in which a fictional, virtual‐world body is joined to a literal, real‐world mind” (2008, 423). Moreover, a player of Second Life is “speaking the literal truth when he says of his avatar, ‘This is me,’” even where the actions seem to be fictional (407). While Velleman's position is somewhat vague, what does seem clear is that he interprets the perspective of the ego—the “virtual‐world body” inherent in virtual interactions—to be the real user's experiential perspective on the virtual world. While Second Life does not involve VR media narrowly defined, if Velleman's conclusions could be extended to VR, then at least one part of the informational transaction necessary to seeing might obtain in the interaction with virtual worlds. It is hard to see the justification of Velleman's claims about this chimerical, half‐real identity, however. Jon Robson and Aaron Meskin (2012) have argued that Velleman's resistance to identifying avatars and player‐characters with fictions can be avoided by a more careful analysis of the practices involved in fiction. They have also argued that videogames are a kind of “self‐involving interactive fiction” where the utterance that “This is me” should be interpreted as relating what is fictionally true of the player within the game world (2016). As fictive props, videogames warrant that things are fictionally true of the player, and in VR games, the means by which this is achieved is in the depiction of an apparent visuospatial orientation by the images in the stereoscopic headset. Inherent in the stereoscopic image of a VR headset, and especially in the way the images change corresponding to the movements of the headset, is that this depiction is the orientation of a viewpoint on the depicted world. In videogames, this apparent visuospatial orientation is often used to situate the player‐character within the fictional world of the game. Hence, VR fictive props of videogames allow for a particularly rich sense of spatial “self‐involvement”—while the player's position during playing Resident Evil VII is on the living room couch, the character's fictional position may be within a haunted house, because this is the fiction that is warranted by what is displayed on the headset. It is only fictionally true of players that they receive egocentric information about the objects they see and interact with in that haunted house. At most then, these are cases not of genuine seeing, but of “fictional seeing.” Such apparent visuospatial orientations on real or fictional worlds are not restricted to VR media. They have a more familiar existence in ordinary pictures that depict scenes from a specific position usually “outside” of the pictorial space. Many of these pictures are like videogames in suggesting that there is a viewer of the scene depicted, such as in Jan van Eyck's Arnolfini Portrait, where a mirror in the rear of the depicted scene suggests that the apparent spatial orientation of the picture's viewer is occupied by a viewing figure. While it has often inspired claims of comprising a metaphysical revolution (Heim 1994), I would argue that VR is best seen as an evolution in depictive technology similar to the development of linear perspective in paintings by artists such as Filippo Brunelleschi, though now utilizing the technology of computers to allow its users to manipulate their vantage point on the depicted content and even interact with this depicted content. In VR, the apparent perceptual orientation on the world implied by the stereoscopic image cannot be assumed to be identical with the user's own spatial orientation. First, in the cases of realistic uses, this visual perspective—which may be spatially related to the perceived object in a way that maintains probabilistic counterfactuals—may be spatially dislocated from the user. Second, in fictive cases, the virtual perspective may have no real location at all, but a merely fictional location. Both situations undermine the case for VR media as inherently a form of prosthetic seeing but also clarify those cases where prosthetic seeing may obtain. V The foregoing argument has repeatedly sketched a distinction between virtual realism and virtual fictionalism. Virtual realism comprises cases where VR is used to interact with real objects, explore real environments, and so on. It is in this case that we are tempted to say that the viewer really sees the object, and that VR is transparent. The prima facie plausibility can be bolstered by considering it in Walton's terms as involving the transaction of nonintentionally mediated information and Cohen and Meskin's more stringent claim. But as we have seen, virtual realism is a limited phenomenon, and even those cases in which it is credible are not without complications. Cohen and Meskin argue that the transaction of egocentric information is a necessary but not a sufficient condition for genuine seeing, and, thus, even if this condition is met in virtual realism, there may be other necessary conditions on seeing that virtual realism fails to meet (such as the “similarity relations” that appear in both Walton's and Carroll's descriptions of photographic realism). Second, there may be doubts about whether VR realism even meets their necessary condition because of the spatial dislocation of the real and virtual ego. But even if the claim of virtual transparency cannot be justified in cases such as the Disney research, the objects are certainly real, and VR media may be used to inspect and manipulate these objects, thus counting as a visual “tool” (Pan and Niemeyer 2017). Whether virtual realism is conceived of as a tool or a prosthesis comes close to splitting virtual hairs, and the potential dislocation of the virtual ego from the user's ego in some uses of VR could instead be interpreted as showing that Cohen and Meskin's egocentric spatial information condition is not necessary for prosthetic seeing. Hence, I am ambivalent about Cohen and Meskin's contribution: their account is useful in explaining the role that egocentric spatial information plays in many cases of VR, but it may stop short of acknowledging as prosthetic seeing some cases about which we may have just that intuition. If we find the ego‐related counterexamples above compelling, we have at least three options. We may decide that those cases are after all not instances of virtual transparency; to allow that they are, we could fall back to Walton's less demanding requirement that the informational transaction be a nonintentionally mediated bearer of similarity relations; or finally, we might seek some expansion or clarification of Cohen and Meskin's view that allows for its reconciliation with these potential counterexamples. Clearly more work is needed. Virtual fictionalism occurs in those cases where VR is used as a prop for warranting the imagination (that is, as warranting the generation of fictional truths). This then is one potential complication that needs to be set aside: VR media, in being capable of presenting fictions, will not be transparent for the obvious reason that there may not be anything to really see. In such cases, it is fictional that players see things, however.11 Virtual transparency, if it should exist, is limited to nonfictive cases of VR (or fictive cases involving nonfictive elements). Virtual fictionalism obviously leads to some fascinating gaming potential, particularly to new modes of interaction with games (Tavinor 2018). Augmented reality is an interesting case in which these two modes are mixed in that fictional depictive content is overlaid on realistic content. Augmented reality is not restricted to depicting fictional content, because it can also augment perception so that the user can “see” aspects of reality not normally perceived, as in the earlier noted variation on Disney's research where the technology is able to also display the trajectory of the ball. But despite the impressive media, augmented reality is not so different to other cases where fictional content is overlaid on real objects: trompe l'oeil paintings, for example. Introducing the distinction between virtual realism and fictionalism does much to clarify the potential confusion inherent in this issue by undermining the need for strongly metaphysical interpretations of virtual objects and egos as what is really seen or what is really doing the seeing. What the distinction also shows is that virtuality is not in conflict with either realistic depiction or fictive depiction and is instead a media device that can serve either purpose depending on the intentional context of its use. Virtual realism is intended to mediate, through virtual media, the real causal and perceptual interaction with the world. Virtual fictionalism is intended to mediate, through virtual media, a fictional causal and perceptual interaction with an imaginary world. The distinction between virtual realism and fictionalism also clarifies the kinds of uses to which virtual media may be put, and its aptness for those uses. First, in the case of virtual realism, the dislocation of the virtual ego (a depicted apparent perspective) allows for the safety of the user or perceptual access to a space that is not otherwise possible. In the Disney research, it curiously allows for a perceptual access that replicates an instance of native perceptual access (thus explaining its fruitful use as an illustration of VR transparency in exemplifying the functional correspondence between instances of real and virtual seeing). In the case of virtual fictionalism, the dislocation of the apparent ego into fictional space allows for exploration and adventures in fictional worlds. Augmented reality allows the user to “see” more of the environment perhaps in the realistic sense of depicting objective features of the world we cannot see with native vision or of the fictive sense of placing within the world fantastical elements that may engage the imagination. If VR counts as prosthetic seeing, this clearly depends on the context of its use.12 REFERENCES Carroll , Noël . 2006 . “ Defining the Moving Image .” In Philosophy of Film and Motion Pictures , edited by Noël Carroll and Jinhee Choi, 113 – 133 . Malden, MA : Blackwell . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Cohen , Jonathan , and Aaron Meskin. 2004 . “ On the Epistemic Value of Photographs .” The Journal of Aesthetics and Art Criticism 62 : 197 – 210 . Google Scholar Crossref Search ADS WorldCat Heim , Michael . 1994 . The Metaphysics of Virtual Reality . Oxford University Press . Google Scholar Crossref Search ADS Google Preview WorldCat COPAC Lawson , Ben D. 2014 . “ Motion Sickness Symptomatology and Origins .” In Handbook of Virtual Environments: Design, Implementation, and Applications , 2nd edition, edited by Kelly S. Hale and Kay M. Stanney, 531 – 599 . Boca Raton, FL : CRC Press . Google Scholar Crossref Search ADS Google Preview WorldCat COPAC Pan , Matthew , and Günter Niemeyer. 2017 . “ Catching a Real Ball in Virtual Reality .” Disney Research, March 18. https://www.disneyresearch.com/publication/catching-a-real-ball-in-virtual-reality/. Robson , Jon , and Aaron Meskin. 2012 . “ Videogames and the First Person .” In Mimesis: Metaphysics, Cognition, Pragmatics , edited by Gregory Currie, Petr Kot̓átko, and Martin Pokorný, 435 – 464 . London : College Publications . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Robson , Jon , and Aaron Meskin. 2016 . “ Video Games as Self‐Involving Interactive Fictions .” The Journal of Aesthetics and Art Criticism 74 : 165 – 177 . Google Scholar Crossref Search ADS WorldCat Tavinor , Grant . 2018 . “ Videogames and Virtual Media .” In The Aesthetics of Videogames , edited by Jon Robson and Grant Tavinor, 146 – 160 . New York : Routledge . Google Scholar Crossref Search ADS Google Preview WorldCat COPAC Velleman , J. David . 2008 . “ Bodies, Selves .” American Imago 65 : 405 – 426 . Google Scholar Crossref Search ADS WorldCat Walton , Kendall L. 1984 . “ Transparent Pictures: On the Nature of Photographic Realism .” Critical Inquiry 11 : 246 – 77 . Google Scholar Crossref Search ADS WorldCat Walton , Kendall L. . 1990 . Mimesis as Make Believe: On the Foundations of the Representational Arts . Harvard University Press . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC A video of the technology can be seen at “Catching a Real Ball in Virtual Reality,” Disney Research Hub: https://www.youtube.com/watch?v=Qxu_y8ABajQ. With certain caveats, such as, of course, that just which portion of the scene is captured by the photograph depends on the photographer's intentional states insofar as they decide which part of the scene to photograph. I would like to thank an anonymous reviewer for this journal for pointing out the aptness of this description of the Disney research. For those interested, Pan and Niemeyer used a “discrete time Unscented Kalman Filter (UKF) to filter position and velocity data of the ball generated by the motion capture system” (2017, 1). The role of algorithms in VR media here may also undermine their “epistemic value,” however. These algorithms must be formulated specifically to maintain the probabilistic relationship of interest here, and it is just as easy to formulate algorithms that destroy or distort the spatial (and other) properties of VR depictions. It should also be noted that interaction is not necessary for seeing: Cohen and Meskin's elaboration of Walton's example of the hall of mirrors in which one is confused about the position of a carnation is one in which the individual sees the object but is incapable of interacting with it (2004, 4). David Chalmers, “The Virtual and the Real.” Unpublished manuscript, NYU and Australian National University. Robson and Meskin argue that this is a virtue of videogame fictionalism more generally (2016, 168). This is part of the premise of Orson Scott Card's Ender's Game, where what appears to be a virtual reality game actually comprises a real and deadly interaction with an alien species. If the drone is being flown in the users’ location they might infer spatial information about their egocentric relationship to the objects seen; for example, if the drone is used to observe the users’ own position they may see the objects around them; but this kind of visual inference is different from the information transaction required by prosthetic seeing. Robson and Meskin (2016) argue that videogames are instances of “Self‐Involving Interactive Fictions,” or SIIFs. I think the account I have provided here is largely consistent with their analysis and indeed that their idea bolsters my own account in that virtual fictional egos allow for visuospatial self‐involvement in the fictional worlds of videogames and similar artifacts. I thank two anonymous reviewers of this journal for comments that helped improve the ideas presented here. © The American Society for Aesthetics This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model) © The American Society for Aesthetics
TI - On Virtual TransparencyTavinor On Virtual Transparency
JF - The Journal of Aesthetics and Art Criticism
DO - 10.1111/jaac.12626
DA - 2019-04-23
UR - https://www.deepdyve.com/lp/oxford-university-press/on-virtual-transparencytavinor-on-virtual-transparency-RFUMWiD0kr
SP - 145
EP - 156
VL - 77
IS - 2
DP - DeepDyve
ER -