Recent developments in computer vision hardware have enabled (and popularised) the use of gestures as well as full body posture as a form of input control in commercial gaming applications. However, the computer screen remains the place where the eyes must be placed at all times. Freeing graphics from that rectangular cage is a hot topic in Spatial Augmented Reality (SAR): using static or dynamic projection mapping and ‘smart projectors’, it is possible to recruit any surface in the surrounding for displaying the game’s graphics. The present work introduces an original interaction paradigm building on kinetic interfaces and SAR: in ‘Skin Games’ the body acts simultaneously as the controller and as the (wildly deformable) projection surface on which to display the game’s output.
Skin Games takes the concept of ‘immersion’ to a whole new physical level: you are literally covered by the game. The fact that the user needs to contort to see the graphics on her own body is perceived here as an interesting inherent feature of the Skin Games paradigm. Technologically speaking, there are different possible ways to materialize the concept. One approach is to actually wear the screen; this is technically feasible as demonstrated in numerous art/entertainment projects. A wearable, full body display can easily integrate kinetic sensors (accelerometers, gyros), biosensors or simple switches thus providing appropriate input, as demonstrated in our Light Arrays project. A different approach consist on using external tracking and projection hardware. In the case of a traditional projector/camera setup, this requires relatively sophisticated hardware and significant computer power, because the system must realize real-time dynamic projection mapping while maintaining minimum delay and spatial mismatch. While we are working on this technology and setup (see for instance “Real-time Shape Measurement of a Moving/Deforming Object“), the prototype demonstrated here uses the Laser Sensing Display technology developed in our lab.
More information:
Cassinelli, A., Angesleva, J., Watanabe, Y., Frasca, G., Ishikawa, M.: Skin Games, Proceedings of the ACM international conference on Interactive Tabletops and Surfaces (ITS’12), (Cambridge, MA, 2012/11/11-14), pp: 323-326