EARLIDS is a wearable device enabling the semi-voluntary control of auditory gain. Artificial “earlids” represent to the ears what natural eyelids are to the eyes: a fast and efficient reflex mechanism for protecting delicate sensory organs. Externally, the device presents itself as an ordinary pair of closed headphones, but hidden under each ear-cup we find EMG electrodes that monitor the contraction of the temporal and masseter muscles. When they contract or relax (consciously or unconsciously), the external sound being picked by binaural microphones placed on each side of the head will be greatly attenuated – or greatly amplified depending on the mode of operation. Applications may range from instant hearing protection without requiring the use of hands (in particular for people having to work at the boundary of environments with notably different sound levels such as night clubs), to the generation of highly personal acoustic experiences rendered possible by the manipulation of the environmental acoustic material.
[supsystic-gallery id=69]
INTRODUCTION
The motivation for this experiment comes from the casual observation of small children performing a similar ’experiment’, virtually always with a sense of awe. I remembered myself being scolded at least once without given a chance of explaining why I would enjoy placing my hands around my ears to hear the voice of the teacher change as if heard through a sea-shell, or rhythmically closing and opening the ear canals so as to brutally slice her (angry) words. The experience of a mute world that goes unnoticed to anyone but oneself is highly disturbing (the technique is routinely used in cinema as a way to render a scene dream-like, and as such relating more closely to one’s own subjective world). There is a curious asymmetry with the sense of vision, perhaps precisely because we are endowed with eyelids and are thus habituated to the experience (for a related experiment, see ’To blink or not to blink’ paper in this workshop). That such tampering of the senses alter the way the world is naturally perceived is obvious; autistic children and children with certain mental illnesses do something similar in order to self-stimulate and perhaps bring a certain regularity to their sensory input (the so called “stimming” behaviour that can include self talk, rubbing, jumping, but also tapping ears).
[supsystic-gallery id=71]
EARLIDS AS A SOUND MANIPULATION INSTRUMENT?
The system proposed here explores the possibility of a highly individual auditory experience thanks to conscious or unconscious muscle tension and facial expression. Instead of using such information in an attempt to generate sound for others to hear [1], we are interested in manipulating sound by the user of the interface, and for that user only. It is interesting to note that the body and its sensory organs always modulate the external sound field in one way or another. The ear transfer function is determined from a variety of parameters including the shape of the outer ear and internal hearing organs, head structure, middle ear muscle activity, etc. Neural low and high level processing also come into play to shape the final perceptual experience. However, if such subtle chain of processing is invisible and seemingly beyond the control of the person’s mind, it is precisely because in normal conditions auditory sensor and motor integration produces the illusion of an objective, coherent, stable acoustic world that seems ’out there’. The attempt behind this project is to amplify the effects of this unconscious processing to a level such that it will become readily apparent to the user that they are playing a crucial role in shaping the experience. We can then expect that, before the user re-learns the new artificial auditory sensory-motor contingencies, they will have the opportunity of consciously and playfully manipulate the raw-material constituted by the environmental sound. The importance given to this sensory-motor feed-back makes this work depart from closely related experiments on “audio augmentation” such as [2] or [3]. In [4] a system is described capable of detecting discrete facial movements (e.g. sticking of the tongue) using infrared sensors embedded on the speaker’s earbud. However, the system is used as an mp3 remote controller and it is not clear if it could be used to perform continuous modulation of audio parameters.
EARLIDS AS A PROTECTIVE DEVICE
The human body presents a number of unconscious reflexes to protect the sensory organs from overly intense stimuli. The iris contracts as a way to limit the amount of energy that reaches the retina, but when this is not sufficient, eyelids provide a very efficient luminous barrier. Several muscles including muscles in the middle ear but also the masseter muscles contract naturally in human beings when exposed to loud sounds (“jaw acoustic reflex” [5]). This natural reflex decreases the transmission of vibrational energy to the cochlea, thus protecting the inner ear, but artificial earlids could dramatically increase the hearing dynamic range without the need of tapping ears with our hands.
[supsystic-gallery id=72]
EARLY PROTOTYPE
In the early prototype presented here (see Fig.1-b,c,d), sound is first blocked almost completely by circumaural (ear-cup) headphones; sound is picked by bin-aural microphones and fed to a MAX/MSP patch on a laptop computer. The patch modulates the gain before redirecting the sound stream towards the headphones, using as a control signal the output of a custom made EMG detector based on an INA128 instrumentation amplifier. Integrating noise canceling technology, it would be possible to create a prototype in the shape of ear-buds or in-ear ’earlids’ (Fig.1-a).
[supsystic-gallery id=73]
DISCUSSION
Sensual experiences can be brought about by the design of complex stimuli, or alternatively by tampering with the senses themselves. The first case is pervasive, and may well be the subject of a study in the Arts. The second case has, depending on the context, either positive or negative connotations. Tiredness, illness, and drugs can, as Huxley would put it, lower the efficiency of the “cerebral valve” so much that “biologically speaking useless material flows into consciousness from out there”, forcing the individual into close contact with naked reality [6].
Music is an example of a complex stimuli that requires specialized knowledge and training to produce. The skilled musician knows something about our senses, but indirectly: they know how one would react to each particular auditory stimulation. Complex, fined tuned machines called “musical instruments” are routinely designed to produce these sounds. On the other hand, devices that alter normal perception may bring extremely powerful sensual experiences not requiring the design of complex external stimuli; instead, by tampering with the senses, an otherwise crude flow of data (light, sound, etc) can be experienced as something enjoyable or of transcendental value. Molecular “devices” – drugs – have been used since time immemorial; more recently, electricity [7], magnetic fields [8], strobe lights [9] and more general audio-visual installations have been capable of producing similar effects in both controlled and uncontrolled environments for the purpose of scientific research, metaphysical inquiry or as means of pure recreational experience. Whether this sort of tampering of the senses is “deceiving” the soul, and is therefore of less intrinsic value than “real” art is a legitimate question, in particular if one standpoint is that art is something that can be somehow developed and perfected by an artist that is, to a certain extent, an actor and not a passive spectator of the whole experience. At any rate, the development of an art require skills, and offers the opportunity to explore perception in a systematic way. In this regard, and extending the concept of the EARLIDS, I propose to discuss a form of musical expression that requires skills and can be improved over an entire life, but whose concrete expression is intrinsically difficult to share – just like a highly personal drug-induced experience. While it is true that virtually all musical instruments bond to the body of the musician in more or less inextricable ways, the performance itself is an audible product for all to share, even though the sound itself may be relatively different for the performer an their audience (certain traditional instruments such as the ’jaw harp’ of Turkish tribes produce sounds that may be quite different from the point of view of the performer, because of strong sound bone transmission for instance), but both the externally audible performance and the subjective experience of the musician relate in a coherent way. Learning these differences suffices to cope with this disparity, and as a result the experience becomes interpersonal. In contrast, an “entacoustic musician” would be able to discuss at length a particular technique, but they could not directly share the resulting sensual experience.
CONCLUSION AND FURTHER RESEARCH
This paper describes the principles of the EARLID, and introduces a prototype of the device, but most experimental research is yet to be done. If the device proves interesting, other entacoustic phenomena not necessarily originating in the ear may be amplified and integrated in the feedback loop. These may include heartbeat (to regulate tempo, for example), breathing, blood flow and reflow, etc. Again, these sounds may seem barely controllable (visceral functions are controlled by the autonomic nervous system without conscious intervention), but as the system purposely distorts the normal sensory-motor flow, it may force introspection and bring about some form of conscious control after some training (this of course is at the core of biofeedback techniques). Other applications of the system can be imagined, ranging from controlling the volume of an mp3 player or a mobile phone to more sophisticated functions such as control of the hearing gain as a function of the psychological state of the person (anxiety, relaxation, anger). A concrete example would be a system that detect stress, and reduces the auditory input in an attempt to soothe the person.
ACKNOWLEDGEMENTS
The author would like to thank Daito Manabe, Carson Reynolds and Stephane Perrin for interesting discussions related to this work.
REFERENCES
1. Manabe D. et al: Myoelectic sensors triggering sound, www.youtube.com/watch?v=r27KdzCgHT4.
2. Mueller, F. and Karau M.: Transparent Hearing, CHI 2002 Extended Abstracts on Human Factors in Computing Systems, Minneapolis, 20 – 25 April 2002.
3. Watanabe, J. et al.: Visual Resonator: Interface for Interactive Cocktail Party Phenomenon, CHI2006 Extended Abstracts pp. 1505-1510.
4. Taniguchi, K.: Mimi Switch, www.engadget.com/tag/Kazuhiro+Taniguchi/
5. Deriu, F. et al.: Origin of sound-evoked EMG responses in human masseter muscles, April 1, 2007, The Journal of Physiology, 580, 195-209.
6. Huxley, A.: The Doors of Perception and Heaven and Hell, 1954, 1956, Harper & Brothers
7. Penfield, W.: The Mystery of the Mind : A Critical Study of Consciousness and the Human Brain. Princeton University Press, 1975.
8. Khamsi, R: Electrical brainstorms busted as source of ghosts, BioEd Online, (2004)
9. Geiger, J: The Chapel of Extreme Experience: A Short History of Stroboscopic Light and the Dream Machine, (2003)
Copyright is held by the author/owner(s). ISMAR’10 Workshop – Devices that Alter Perception (DAP 2010), October 13th, 2010.
This position paper is not an official publication of ISMAR’10.