Gesture sonification for enhancing agency: an exploratory study on healthy participants

Felix Schoeller ^1,2* Parham Ashur ³ Joseph Larralde ^4* Clement Le Couedic ^5* Rajeev Mylapalli ⁶ Karthikeyan Krishnanandan ^4* Anna Ciaunica ^7,8* Adam Linson ⁹ Mark Daniel Miller ¹⁰ Nicco Reggente ² Vladimir Adrien ^11,12,13*

• ¹ Massachusetts Institute of Technology, Cambridge, Massachusetts, United States
• ² Institute of Advanced Consciousness Studies, Santa Monica, California, United States
• ³ École de Management de Lyon, Écully, Rhône-Alpes, France
• ⁴ Other, Paris, France
• ⁵ Aura Healthcare, Paris, France
• ⁶ Learning planet institue, Paris, France
• ⁷ Center for Philosophy of Sciences at the University of Lisbon, Lisbon, Portugal
• ⁸ Institute of Cognitive Neuroscience, Faculty of Brain Sciences, University College London, London, England, United Kingdom
• ⁹ School of Computing & Communications, Open University, Edimbourg, United Kingdom
• ¹⁰ Hokkaido University, Sapporo, Hokkaidō, Japan
• ¹¹ Assistance Publique Hopitaux De Paris, Paris, France
• ¹² Université Sorbonne Paris Nord, Villetaneuse, Île-de-France, France
• ¹³ INSERM U1266 Institut de Psychiatrie et Neurosciences de Paris, Paris, Île-de-France, France

Background: Body awareness (BA) and proprioception, which are essential components of the sense of agency (SA), are often altered in various mental disorders such as posttraumatic stress disorder (PTSD). However, the relationship between BA, proprioception, and SA, as well as the methods to manipulate them, remain unclear. This study explored using real-time gesture sonification (GS), i.e. wearable technology transforming body movements into sounds, to enhance proprioception, BA, and thus the SA. Methods: In this within-subjects design, 17 healthy adults (mean age = 25.5 years) with varying dance expertise (novice, amateur, expert) improvised movements to match sounds with and without auditory feedback from motion sensors on wrists/ankles modulated by their gestures. BA, immersion, pleasure, and self-efficacy were measured. Results: Sonification significantly increased body awareness, reward, and immersion (all p<0.05). Conclusions: GS can enhance BA and the SA, pleasure, and control during physical activity. This highlights potential mental health applications, such as agency-based therapies for PTSD. Manipulating bodily perception could improve symptoms and embodiment. Further research should replicate this in clinical populations and explore neurocognitive mechanisms.