Editorial: Electrical stimulation for immersive virtual and augmented reality

Lopes, Pedro

doi:10.3389/frvir.2022.1000054

EDITORIAL article

Front. Virtual Real., 03 November 2022

Sec. Technologies for VR

Volume 3 - 2022 | https://doi.org/10.3389/frvir.2022.1000054

This article is part of the Research Topic Electrical Stimulation for Immersive Virtual and Augmented Reality View all 5 articles

Editorial: Electrical stimulation for immersive virtual and augmented reality

Pedro Lopes*

The University of Chicago, Chicago, IL, United States

Editorial on the Research Topic
Electrical stimulation for immersive virtual and augmented reality

Introduction

Electrical stimulation has a long tradition in medicine and neuroscience, where it is typically used to understand the human body by triggering neurons using electricity. A canonical example of electrical stimulation in medicine is muscle rehabilitation, in which clinicians attach electrodes to a patient’s muscle (e.g., in the legs) and connect these to a medical stimulator. Currents running through these electrodes induce an involuntary muscle contraction on the muscles, assisting the patient to rehabilitate a weak muscle (e.g., muscles that were weak because the patient wore a cast due to a fractured leg). Many such electrical stimulation approaches exist in neuroscience and medicine, including peripheral nerve stimulation, brain stimulation, and so forth.

The key question behind our Research Topic is not understanding the value that these techniques have for medicine, but rather “what value does electrical stimulation have outside of medicine”?

Electrical stimulation can revolutionize human-computer interfaces

We focus on the inventive potential of electrical stimulation for user interfaces. In these interactive systems, the electrical stimulator is controlled by the interface, rather than by a clinician. As such, the interface can now deliver electrical impulses, in real time, to the user’s body. In turn, these electrically-generated sensations allow the user interface to communicate back to the user by means of rich bodily sensations, which include sense of touch, forces, and more, all of which are typically denoted as haptic sensations. This allows interfaces to go beyond the traditional audiovisual modalities. For example, using electrical stimulation, one can engineer an interactive device that assists users in learning sign language (Nith et al., 2021) or playing musical instruments (Akifumi et al., 2021), which instead of relying on visual cues to indicate the correct hand poses, makes use of electrical stimulation of the user’s muscles can directly pose their finger muscles in the correct poses.

Electrical stimulation vs. mechanical stimulation

Over the past decades, this approach of electrically stimulating the user shook the fields of human-computer interaction by challenging the way a device can render haptic sensations. Electrical stimulation offers an alternative to previous haptic techniques based on creating the physical effect on the user. For instance, the traditional way to re-create the sense of force required to move a user’s body (e.g., move their fingers to play piano) was to have an actuator that mechanically pushed against the user’s body. Similarly, to create the sense of heat, interfaces made use of actuators that would heat up the user’s skin. Instead, interfaces based on electrical stimulation work inside out. Rather than creating the physical effect (heat or force), electrical stimulation creates the internal effect—it internally triggers the nerve that would otherwise be triggered by the externally applied heat or force. The advantage is that this changes the hardware required to generate haptic sensations, replacing motors and other mechanical actuators with electrodes and stimulators. What is more remarkable is that swapping mechanical components for electrical components has a powerful consequence: it advances miniaturization of interactive devices. As a type of electronic devices, electrical stimulators are easier to miniaturize than mechanical devices—as an analogy, we invite the reader to contrast how much computers have shrunk from the size of an entire room to the size of a wrist-worn watch in just 60 years, while mechanical actuators, such as a car’s motor, have not undergone the same dramatic size reduction. This size reduction is a key advantage of electrical stimulation when applied to immersive technologies (virtual or augmented reality), since these are all based on wearable devices (head-mounted displays) and feature untethered users freely moving around (Nagai et al., 2015; Lopes et al., 2017; Lopes et al., 2018).

Electrical stimulation enables new output modalities

Since interactive devices based on electrical stimulation can internally cause sensations in their user’s body, they can, in turn, induce a variety of interesting physical responses that the interface can now leverage as output modalities, including: 1) tactile sensations [electrotactile stimulation (SmartTouch, 2022)], 2) force sensations [electrical muscle stimulation (Farbiz et al., 2007)], 3) increased friction sensations [electrovibration (Shultz et al., 2015)]; 4) balance sensations [galvanic vestibular stimulation (Maeda et al., 2005; Aoyama et al., 2013; De Maio et al., 1079)]; and, 5) taste sensations [electrical stimulation of taste receptors in the tongue (Nakamura and Miyashita, 2011; Sakurai, 2016)]. These five interactive applications have been the frontline of electrical stimulation outside the realm of medicine/science. In all these interactive applications, the switch from mechanical to electrical stimulation allowed to create more wearable and portable devices.

Furthermore, emergent types of electrical stimulation also hold potential to enable exciting new modalities for interactive devices, including sensations not easily achieved (or even possible) with external actuators, such as: 6) smell sensations [electrical olfactory bulb stimulation (Hariri et al., 2016) or electrical trigeminal stimulation (Brooks et al., 2021; Aoyama et al., 2021)]; 7) temperature sensations [electrical stimulation of the skin nerves responsible for temperature sensing (Saito et al., 2021)]; 8) goosebump-like sensations [using electrostatic stimulation to move the hairs on the skin (Fukushima and Kajimoto, 2012)]; 9) brain stimulation [using transcranial direct current stimulation to create sensory illusions (Škola and Liarokapis, 2019)]; 10) tendon stimulation (Takahashi and Kajimoto, 2021; Takahashi et al., 2022); or even 11) retinal stimulation (Higuchi et al., 2017).

Overview of our research topic

In this Research Topic, our contributing authors explore electrical stimulation in three ways: 1) technical challenges for closed-loop stimulation; 2) rendering more realistic sensations in virtual worlds; and 3) transforming existing sensations—all of these have implications to improving the user’s experience in immersive technologies.

First, in the technical solution space, Hosono et al., demonstrates how to close the loop on muscle stimulation using infrared optical sensing to determine the level of a muscular contraction; the resulting setup is compatible with electrical muscle stimulation and still results in a small form factor, ideal for immersive applications. Next, Nunez et al. and Shell et al. explore combining electrical stimulation with other haptic actuators to improve the haptic realism of virtual interactions. Finally, Kaji et al. take electrical stimulation one step further by exploring waveforms that enable enhancing saltiness sensation of foods. Taken together, these examples of electrical stimulation in human-computer interfaces demonstrate how profound the shift from mechanical to electrical can be.

Author contributions

The author confirms being the sole contributor of this work and has approved it for publication.

Funding

This work was supported by NSF grants 2047189. Any opinions, findings, and conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of any funding agencies.

Acknowledgments

Moreover, we’d like to thank our collaborators Jas Brooks, Yudai Tanaka, Akifumi Takahashi and Jasmine Lu for guidance on the references cited in this editorial. Special thanks to Eva McCord for helping to edit the article. Finally, to the train conductors in Chicago’s Metra line, since this article was entirely written during train commutes.

Conflict of interest

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Akifumi, T., Jas, B., Hiroyuki, K., and Pedro, L. (2021). “Increasing electrical muscle stimulation’s dexterity by means of back of the hand actuation,” in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI’ 21) (New York, NY, United States: Association for Computing Machinery), 216, 1–12. doi:10.1145/3411764.3445761