 |
 |
| Figure 1.1 | Figure 1.2 |
Introduction
The Way of Looking at a Problem
The Disappearance of the "Picture"
Close-up on P I X E L
The Virtual Retinal Display
The Faith in the "Picture"
Living "Pictures"?
The Virtual Scientific Display
Introduction
Most research in medical visualization, especially virtual reality is done without a sufficient attention to semiotic and psychoanalytic theories of perception. This bears the risk of neglecting these theoretic devices" in understanding and developing the new visual syntax of virtuality. This text tries to develop a new visualization of the elements of pictorial perception - merely as a work of words and images - by reporting a study on P I X E L. (1)
First: What is a P I X E L ? Dating from the late 60s the term composes pic(ture) - in it's plural form: pix - and el(ements). It describes the so called "smallest element of an image that can be individually processed". That is: P I X E L are small parts of images on a digital basis. But who knows how they look? This simple question already causes epistemological trouble and needs more explanation.
The Way of Looking at a Problem>>
The study on P I X E L is a study on pictures and formulates its question historically. At the transition from analog recording technics to the digital signal-processing a potential arbitrariness of data-visualizations is already given. Changes in cultural and scientific discourses are taking place on this technological basis. P I X E L tries to find a way of looking at these processes by still looking at the pictures.
The picture is the problem and the problem is: you can't see a picture anymore by looking at something looking like a "picture". Concerning a computer-visualization, the variability of data (to be seen as variability of visualizations) is constitutive and not additional to the new technologies. It forms a critical moment of pictorial changes. In classical pictorial media like photography or film the information is embedded in the material basis and is (compared to the new possibilities) invariable. Only in a second step - "afterwards" - the manipulations of the recording can be done. This means that there are still steps to be "materially" recognized. First is a record, second is a manipulation of this record. Concerning the computed "picture" these two fundamental steps become one.
Figure 1.1 Figure 1.2
As a result of this technological novelty changes of "the picture" can be analyzed in perceptual and epistemological ways. New for visual perception is an incapability of distinction for the spectator: between "original" or "manipulated" parts of the picture, not because of skillful falsifications but because of a logical, technological impossibility. This loss of a material or materially defined "original" of the "picture" causes a new relation to the reference of the "picture", linked only by arbitrary algorithms with the reproduced object. Concerning scientific discourse this means nothing less than the technically induced development of a new definition for "fiction" or "document". These two terms are no visible and logical opposites any more. A new verbal differentiation has to be developed concerning invisible phenomena, even if these facts tend to escape from consciousness. The thesis is: they are simply overseen. (Figures 2.1, 2.2)
Already the habitual mouse-click transforms the idea of a "picture" by the experience of an access on the actual data-structure as a characteristics of a "picture". This "picture" becomes an animated, variable surface of a structure which does not appear visibly itself. So it is the basic data-structure which can be identified as not visible in the computed picture, even if the ease and quickness of commanded pictorial changes causes the impression of a visible access to the structure itself.
Figure 2.1 Figure 2.2
The Disappearance of the "Picture">>
Despite an overall, non-stop and infinite pictorial "flood" the main thesis of this paper is one of a contemporary disappearance of the "picture". This has to be unfolded.
Coming from the change of the picture we know and we know to read, it is the difference between subject and object, thing and representation which is not distinguishable any longer in the way it was before. A new conceptual and physical 3dimensionality, to be thought as a variable, reacting, visual and tactile 3dimensional structure includes the user. Especially in environments of virtual reality this concept becomes concrete. Just imagine to look at someone in a data-suit and with a Head Mounted Display, moving around "in" a completely different type of "picture".
The border between "man" and "picture" has been exceeded in an indistinguishable continuum of "information". The user is one of many data-producing factors in the computed "picture" and the idea of old analog pictorial technics is changing quickly. The possibilities of data-visualizations form the contemporary measure and imagination of "pictures". (Figures 3.1, 3.2)
If in this way a disappearance of the "picture" is maintained - disappearing in a continuum of "information" - a disappearance of "man" has to be stated too. Not longer being thought in the medium of analog pictorial technics, the subject of perception has to be revisited according to the new technological definitions of perception, memory and imagination - given by computed visualizations.
Figure 3.1 Figure 3.2
Close-up on P I X E Ls >>
Tiny as dots, photographic grains or points P I X E Ls tend to evade our eyes. In the state of historical documents P I X E Ls can still be seen as "stepped" unevenness in older computer graphics. To add this, P I X E Ls are regarded as a problem. P I X E Ls are concrete memories of a difference between representation and thing, as well as between "picture" on the computer-screens and its technological structure. In the new paradigm of a continuum of information the visibility of P I X E Ls disturbs the accompanying concept of "reality". Trying to eliminate the P I X E Ls there is the attempt to undergo their visibility by multiplying their quantity.
Beyond the border of human visual physiology P I X E Ls disappear out of sight. If approaching very close to the computer display with your eyes you can determine your personal distance, just showing you the border between visibility and non-visibility. At this distance you can still notice yourself as this historical subject of perception by seeing a picture and at the same time the limits of your visual perception. You can see the point where visual hallucination starts, when the differences between elements disappear to form one unified figure with a "realistic" or "photographic" impression. Please notice this point when you get close to this desire. (Figures 4.1, 4.2)
Let us now imagine a special case. Your eyes are still very close to the computer screen. A medical visualization of a telepresentic environment takes place. You know that a robotic device is operating with surgical instruments miles away and according to your gestures. So you can influence the "picture" on the screen by moving your hands. You can see changes of P I X E Ls close to your eyes. But can you see what you are doing? Are these your gestures or are these moving P I X E Ls ?
Figure 4.1 Figure 4.2
The telepresentic end of the "picture" is executed by the "realization" of the "picture", but the cinematographic paradigm of documentation is still the same. Representation and visual control of executed commands become one thing. Is this "life" as were for instance the CNN news of the gulf war? You would be the reporter, you would be the camera, you would be the bombs and you would be the pilot. But first of all you would be a spectator of "pictures". Perhaps you can see this by looking as closely as I suggested at the material surface of imaging technology, by looking at the P I X E Ls ?
The question is how the new telepresentic and virtual reality visualizations change the structure of literal and visual representation itself. New relationships between word, imagination and thing are being installed. Is there a new category of "intelligent pictures" starting a basic symbolic shift? "Intelligent pictures" restructure the borders between language and action, analysis and invention, reality and simulation. The possible interfaces between visualizations on the screen and any robotic apparatus changes the relations between fiction and facts.
But what does this mean to you looking so close at the P I X E Ls ? What will happen if even the P I X E Ls will disappear from your sight?
The Virtual Retinal Display >>
An contemporary example of a new visualization system, presented on the "VR World '95" in Stuttgart, FRG, can try to imagine a future of P I X E Ls. Developed by the Human Interface Technology Laboratory of the University of Washington, Seattle, the Virtual Retinal Display could replace the screens and LCDs and CRTs of Head Mounted Displays.
A stereoscopic laser microscanner stimulates directly the light receptor cells in the human eye to produce the images physically "inside" - onto the retina. Modulated laser beams scan the calculated images P I X E L by P I X E L at a high rate, synchronized with the eyeballs' motion in real time. Can a future Virtual Retinal Display in fact "paint realities directly on the retina", as Thomas A. Furness confirms? (2) In any case: this would be a new type of painting.
The desire to go beyond paintings, to bring paintings to live continues the tradition of the 18th century, when the upper class cultivated the art of re-posing the figurative composition of paintings with living models as "tableau vivants", until the place was taken by the new imaging technics photography and film.
But still every picture - painting, tableau vivant, photo or film - needs you. There is the need of a spectator. In the case of the Virtual Retinal Display this spectator must be very small to fit in the eyeball, because this is the only place where the laser projections onto the retina could be regarded as a sort of "painting", probably more resembling a motion picture. Or is the tiny spectator supposed to be seated on the other side of the retina, in the brain? In the visual cortex or other parts? No such place of the human conscience has yet been found and the "linguistic turn" in psychoanalysis teaches that it is impossible to find - if looking for a visible representation of the human identity in the human. So the question is still urgent: Who is watching the so called "realities on the retina"? How can we imagine ourselves beyond the idea of a visible, concrete image - beyond an identification with the mirror image giving us the "reality" of "ourselves" as unified figure? (Figures 5.1, 5.2)
Perhaps one possible solution could be a paradoxical identification with the borderline between visibility and disappearing visibility. Please think back to the point of appearing and disappearing P I X E Ls in your perception according to the distance to the screen. This point can be characterized by a pulsation of visibility to be felt.
Figure 5.1 Figure 5.2
But the P I X E Ls of the Virtual Retinal Display have no distance to the retina any more and could even become invisible (in advanced models). Perhaps they could become indistinguishable from the neurons of the retina itself. Then this device could be named a retinal "picture" implant, not only eliminating the "picture" but the sight itself. Blindness is the goal, a blindness of the dreams when you can see without your eyes. The ultimate sight in the vision of the Virtual Retinal Display would be one where no eyes are needed any more and "pictures" are triggered neuronally in the brain.
The Faith in the "Picture">>
Beyond the P I X E Ls there is an area of faith in the "pictures" left. It is known as the faith in visual representation, based on visible similarity. The new possibilities of computed "pictures" support this faith (even if similarity is accidental). Traces of this faith can be found as well in scientific as in popular texts. What can be read out of these is a fascination radiating from the new digital pictorialism. The attraction of the popular and scientific formulations of an "information- age" under the sign of an universal programmability derives essentially from this fascination. Fascination forms the "pictorial" basis of the universal metaphor "information".
When there are "information stores of life", "algorithms of consciousness" or "secret codes of life" to be cracked, the main subject is always "information". It seems that this "information" just to be conceptualized has an essential need of "pictures", precisely a need of faith in the "picture".
The arbitrary figure of a visualization would be confused with the "true picture" of an information-structure. The attempt to read a computer-visualization like a new type of photographic document derives from the idea that the computer could reproduce not only an "outer" similarity but an "inner" mathematical structure from which the outer appearance is logically developed. This desire for such a reproduction in a "heightened", "true" form believes in a "picture" which neither a photography nor a computer-visualization ever could be: the reproductive identity. (Figures 6.1, 6.2)
Figure 6.1 Figure 6.2
Living "Pictures"?>>
The most spectacular form of realization of a "picture" is maintained in the contemporary assertion of "life" concerning computer-visualizations. Not only to the programs of "Artificial Life" this "life" is attested, but theorists like Peter Weibel call every interactive visualization, every "intelligent picture" in medicine as well as in computer-games: "living". A multitude of interpretations of this term is possible. The most extreme interpretation believes in "living" computer-programs, the most historical remembers the so called "living pictures" in the begin of our century: the film. For the early cinematographic inventions - known as "bioscope", "biographe" or "lebendes Bild" - have been "alive" as well as the data- visualizations in our days. These "living", animated photographs had one major technical and epistemological subject: to exceed the speed of visual physiology by creating a mechanical induced visual hallucination with a frequency of 16 frames per second. "Life" was in this way asserted to an illusion which forms still the base of television sets and computer-screens. Just the frequencies have changed since.
The smallest element of the cinematographic pictorial illusion lies in between. It lies between two frames, showing almost the same picture. But at this borderline between two elements - where we could try to have a look at the represented movement - there is nothing but a tiny line. The mystery of the "living" cinematographic pictures, the mystery of this "life" is not represented in the picture itself.
And what about the P I X E Ls in computer-visualizations? Can they be "alive" as the "cellular automata" of Artificial Life are regarded as? Can they be a record of "life" and being "alive" as well, as the fascination supposes? This question seems to depend on the point of view. If the cinematographic picture really is a "living" one, then P I X E Ls live in the same way. Because even if your eyes approach very close to the screen to be able to distinguish these smallest elements you 'll see a P I X E L only half of the time - appearing and disappearing according to the frequency of the display. This may be the technological and physiological proof that "reality" is neither identical with "visibility" nor with "information", but has to do a lot with physiological hallucinations and beliefs.
The Virtual Scientific Display>>
Do you remember that the "Virtual Retinal Display" was supposed to "paint realities directly on the retina"? Of course these "realities" must be the P I X E Ls. But are P I X E Ls still to be thought as visible tiny irregularities? Or are we just able to notice appearing and disappearing signs of something?
The main question is: Has a P I X E L to be considered as visible or as not visible? At least according to the definition P I X E L could be the name of pictorial elements, which do not appear visibly in the picture themselves. Concerning the Virtual Retinal Display, this device stimulates the sensible cells of the retina P I X E L by P I X E L (corresponding to the encoded information) to cause a pictorial perception. A paradoxical status has to be stated. The P I X E Ls appear definitely neither as visible nor not-visible. From this point of view the Virtual Retinal Display can help to imagine and analyze a category of elements in between: this is the category of invisible elements of the picture. These invisible elements must be the same: of the picture and of the perception, "outside" and "inside", beyond figurative perception and structuring perception itself. So the function of the Virtual Retinal Display shows structural equivalencies to the mechanisms of dream production and perception.
As Jacques Lacan asserted in his "Discours de Rome" 1953 "the unconscious does not function instinctually or symbolically (in the traditional Freudian sense) but, rather, linguistically (3), he names these invisible elements: signifiers. (Figures 7.1, 7.2)
The structural analogy between dream perception and Virtual Retinal Display, unconscious and language can help as well to answer the question how to imagine ourselves beyond the identification with this tiny person inside of the brain watching the "realities on the retina". Experiments to observe and imagine the P I X E Ls lead directly to the paradox of the invisible as the third element of the picture.
Figure 7.1 Figure 7.2
P I X E L , when no longer conceptualized as an identification of "picture" and "information", introduces a model as well for the subject of perception. "Individually processed" this subject is neither image nor information sequence of neuronal impulses, but the difference in between.
Notes
ad 1) The study P I X E L has been carried out in cooperation with the interdisciplinary group of scientists from the University of Hamburg, directed by Prof. Dr. Marianne Schuller: "Medizin - Ästhetik", comprising media sciences, literary studies and psychoanalysis.
>>
ad 2)
The quotation of Thomas A. Furness III is reported by Howard Rheingold in: Virtual Reality, New York 1991, p. 194. >>
ad 3)
Jacques Lacan, "The Function of Language in Psychoanalysis", The Language of the Self, trans. A. Wilden (Baltimore, Md.: Johns Hopkins Press, 1968), p. 35. >>
published in:
Medicine Meets Virtual Reality:
Health Care in the Information Age,
H. Sieburg e+al. (Eds.),
Amsterdam, Oxford, Tokyo, Washington DC 1995,
S.681-689
