Everything squeaks when pressed , stretched or deformed (audible sounds or ultrasounds). The concept behind this project is to create controllers out of everything – by listening and analysing their sound signature. A machine learning algorithm can recognise which controlled is being actuated, and what gesture is being done (even when controllers are “actuated” simultaneously – see video).
References
安宅 佑樹，Alvaro Cassinelli， 渡辺 義浩， 石川 正俊： 特徴音による無線無電源インタフェースの実現 (Acoustic Phidgets: untethered passive controllers from acoustic signatures), 第20回日本バーチャルリアリティ学会大会論文集, Virtual Reality Society of Japan (VRSJ2015), 11C-6 (2015)
Webpage at Ishikawa Laboratory