 |
Spatio-temporal blended identitiesby Alvaro Cassinelli, 2008~ |
 |
|
In the film “A Scanner Darkly” [1] (Richard Linklater, 2006) based on the novel of Philip K. Dick, special agents use a disguise that constantly morph their appearance: identity protection is achieved not by wearing a mask, but by constantly changing recognition cues, such as color of the hair, eyebrows, shape of the face, etc, all taken from a huge database of people. In other words, annonimity is not achieved by using an avatar or an “average” face, but by actually oversaturating the cognitive process of recognition. Today this technology could apply to video-conferencing for instance, using images extracted automatically from the web. This is an interesting observation: one can remember a face (even if it is fake mask), but it is impossible to remember a thousand faces. In fact, the approach has a parallel in the domain of secure telecommunications: using a unique carrier not only makes the system more sensitive to interference and jamming, but simplifies eveasdroping. In a sense, spatio-temporal blending of faces is reminiscent of frequency hopping or spread-spectrum techniques [2]. A face-to-face conversation provides two information channels: one is the face itself that reveals the identity of the speaker; the other channel is the sequence of gestures that the speaker does, providing additional non-verbal information. Here, we want to secure the identity, i.e. avoid using a particular carrier frequency, while letting the partner to recognize the gestures. A particular face correspond to a single carrier frequency, and the gestures modulate that carrier. “Frequency hopping” means transmitting the information contained in the “modulation” (i.e., the sequence of gestures), over a broad range of hopping frequencies (i.e., different faces). |
|
 |
 |
 |
I wondered then what would be the “hopping frequency threshold” at which one could still have the feeling that we are switching from one (known or unknown) face to another; and what would be the percentage of the face that, while modified, would render recognition impossible (a threshold measured in percentage of change/second, and “change” having both a spatial and temporal component). I think there are serious interesting experiments to do in this area. In the following, I show some preliminary experiments done with the “khronos projector” interface. Here is an example: [WMV-6MB].
You can try by yourself below using a Processing applet. It’s an eerie feeling… Try it by fixating the face for a while, while changing the flipping speed. Why fixation is needed? I don’t know (this process of fixation reminds me of the old tradition of scrying in mirrors in a half-lited room: the scrying end up seeing morphing faces and “visions”). Perhaps it enables to concentrate all the power of observation on the visual changes taking place before the eyes, and never attributing these changes to modifications of the observer position.
|
|
Informal observations
The following are some preliminary observations as well as explanatory hypothesis obtained by just playing with the speed of transition between two whole faces, i.e. having a change rate defined only in the temporal domain (in the future I plan to study the effects of mosaicing these):
- The most uncanny feeling (perhaps the treshold at which we don’t know if it is the same or two different faces) is obtained when fixing the face(s) without moving the eyes, and for a frequency of change of about 3.5 to 4 Hz. At this frequency of change, the mind seems unable to decide between two conflicting hypotesis: that there are two faces alternating in time, or that we are seeing one face continuously morphing (a sort of “beta motion” or “filling in” for our mind face detector?)
- For this effect to be strong, there must be some clear relation between the two faces (some kind of anchoring percept). In my experiments, this was the location of the eyes (I arranged the images so that the pupils always coincide). In this situation, our visual system is somehow mislead into thinking that there is a single face that keeps transforming. Our visual system it trying perhaps to minimize the differences between these images, and interpret them as a trasformation in space or some natural deformation of the flesh – this is what I thought it may be happening with the Khronos Projector. However, when this transformation is extreme (as it is the case when juxtaposing in time two very different faces), the mind does not know what to make of it, and we get the feeling that something really wrong is taking place – something “scary” or uncanny… Indeed, there is a sort of “uncanny valey” effect here, a threshold of discomfort. I would like to corroborate this theory using galvanic-skin response, or if it were possible, fMRI.
 |
“Also, the result of visualizing simple gestures using the Khronos projector can be strikingly weird, probably because our visual system is fed with information that almost make sense – but for some subtle, continuous disturbance on the temporal coordinates of each pixel. The result of this reconstruction effort is that sometimes the deformation is no longer interpreted as the result of an isometric rotation or translation of the whole subject, but instead as a tremendously deformed gesture. There are some interesting things to explore here concerning the emotional response to such “impossible” gestures, and perhaps the Khronos Projector can effectively be used to explore some cognitive aspects concerning our visual system (for instance, how motion is integrated in the process of face recognition). ” [3] |
Some interesting experiments to do in the future may involve:
- Alternatively flipping a single face horizontally. It is well known that this will make oneself very aware of dis-symmetries of the face for which we have adapted. But by flipping them, the brain may try to “unify” these (but at a certain threshold of speed, they will perhaps appear more marked, not less). An hypothesis to verify (in fact, a rapid test seems to give the opposite result – check the applet below). Perhaps precisely because we know it is the same person: so any change is interpreted a priory as a head rotation or something of the sort.
 |
 |
|
| “In 1930 William E. Benton patented the “Duality Mirror,” which showed how faces would look with two perfectly symmetrical right or left halves.” (from [4]). |
(click on the image to launch applet) |
-
Prepare a set of images with an intermediate stack of slow morphing faces between them, and see if the speed threshold of uncanniness changes.
-
Try with a larger set of faces. Do the mind make out the differences, or instead the similarities? Will the perception converge to an “eigen-face” or on the contrary will we retain just the most salient differences?
References
- “A Scanner Darkly“, by Richard Linklater (2006), adapted from the novel of the same name by Philip K. Dick.
- Frequency-hopping spread spectrum (FHSS) is a method of transmitting radio signals by rapidly switching a carrier among many frequency channels, using a pseudorandom sequence known to both transmitter and receiver (from Wikipedia). Of course, this is just an analogy: there is no “known sequence” here, but this is unnecessary, as the observer is capable of “listening” to all the “frequencies” at the same time (i.e., she or he is able to recognize any face as a face, and then “demodulate” the gesture, even though he cannot identify these faces).
- The Khronos Projector web page, 2005
- “Mirror, Mirror: a history of human love affair with reflection”, by Mark Pendergrast.
 |
Exhibition History
|