While one of the most common delivery mechanisms for virtual reality, augmented reality, and mixed reality experiences has been a head-mounted display (HMD), an HMD is not the only mechanism for delivering such experiences. With this special issue of Frontiers, we open the space to consider the broad perceptual integration of mixed physical and virtual entities that co-exist and interact in situ (not simply in the visual field) as part of realizing the vision of “being there” in the augmented/mixed reality space. We are interested in hearing about inventive solutions utilizing, for example, handheld, stereo-capable tablets, stereo-capable tracked projectors and displays, unobtrusive tracking solutions, Internet of Things (IoT) or other integrated devices, physical props, and even real world objects. We are interested in prototypes, commercial products, and applications.

University of Southern California

University of Central Florida

University of North Carolina at Chapel Hill

Frontiers in Virtual Reality

Technologies for VR

Virtual Reality, Augmented Reality and Mixed Reality - Beyond Head Mounted Displays

The recent rise in popularity of head-mounted displays (HMDs) for immersion into virtual reality has resulted in demand for new ways to interact with virtual objects. Most solutions utilize generic controllers for interaction within virtual environments and provide limited haptic feedback. We describe the construction and implementation of an ambulatory (allowing walking) haptic feedback stylus with primary use in computer-aided design. Our stylus is a modified 3D Systems Touch force-feedback arm mounted on a wearable platform carried in front of a user. The wearable harness also holds a full-sized laptop, which drives the Meta Quest 2 HMD that is also worn by the user. This design provides six degrees-of-freedom without tethered limitations, while ensuring a high precision of force-feedback from virtual interaction. Our solution also provides an experience wherein a mobile user can explore different haptic feedback simulations and create, arrange, and deform general shapes.

The recent rise in popularity of head-mounted displays (HMDs) for immersion into virtual reality has resulted in demand for new ways to interact with virtual objects. Most solutions utilize generic controllers for interaction within virtual environments and provide limited haptic feedback. We describe the construction and implementation of an ambulatory (allowing walking) haptic feedback stylus with primary use in computer-aided design. Our stylus is a modified 3D Systems Touch force-feedback arm mounted on a wearable platform carried in front of a user. The wearable harness also holds a full-sized laptop, which drives the Meta Quest 2 HMD that is also worn by the user. This design provides six degrees-of-freedom without tethered limitations, while ensuring a high precision of force-feedback from virtual interaction. Our solution also provides an experience wherein a mobile user can explore different haptic feedback simulations and create, arrange, and deform general shapes.

Development of a wearable force-feedback mechanism for free-range haptic immersive experience

AR Drum Circle, is an augmented reality (AR) platform we developed to facilitate a collaborative remote drumming experience. We explore the effects of virtual avatars that are rendered in a player’s view to provide the joy and sensation of co-present music creation. AR Drum Circle uses a head mounted AR display, providing players with visual effects to assist drummers in coordinating musical ideas while simultaneously using a latency-optimized remote collaboration service (JackTrip). AR Drum Circle helps overcome barriers in music collaboration introduced when using remote collaboration tools on their own, which typically do not support real-time video, lack spatial information, and volume control between players, all of which are important for in-person drum circles and music collaboration. This paper presents the results of several investigations: (1) analysis of an in-depth, free-form response survey of drumming communication provided by a small group of expert drummers, (2) observations and analysis of 20 videos of live drum circles, (3) a case study using the AR Drum Circle application in pairs of remotely located participants, and (4) a collaborative drumming case study using a popular networked conferencing application (Zoom). Findings suggest that substantial communicative information including facial expressions, hand-tracking, and eye-contact is lost when using AR alone. Findings also indicate that the AR Drum Circle application can make players less self-conscious and more willing to participate while playing drums improvisationally with others.

AR Drum Circle, is an augmented reality (AR) platform we developed to facilitate a collaborative remote drumming experience. We explore the effects of virtual avatars that are rendered in a player&#x2019;s view to provide the joy and sensation of co-present music creation. AR Drum Circle uses a head mounted AR display, providing players with visual effects to assist drummers in coordinating musical ideas while simultaneously using a latency-optimized remote collaboration service (JackTrip). AR Drum Circle helps overcome barriers in music collaboration introduced when using remote collaboration tools on their own, which typically do not support real-time video, lack spatial information, and volume control between players, all of which are important for in-person drum circles and music collaboration. This paper presents the results of several investigations: (1) analysis of an in-depth, free-form response survey of drumming communication provided by a small group of expert drummers, (2) observations and analysis of 20 videos of live drum circles, (3) a case study using the AR Drum Circle application in pairs of remotely located participants, and (4) a collaborative drumming case study using a popular networked conferencing application (Zoom). Findings suggest that substantial communicative information including facial expressions, hand-tracking, and eye-contact is lost when using AR alone. Findings also indicate that the AR Drum Circle application can make players less self-conscious and more willing to participate while playing drums improvisationally with others.

AR Drum Circle: Real-Time Collaborative Drumming in AR

The sensation of self-motion is essential in many virtual reality applications, from entertainment to training, such as flying and driving simulators. If the common approach used in amusement parks is to actuate the seats with cumbersome systems, multisensory integration can also be leveraged to get rich effects from lightweight solutions. In this paper, we introduce a novel approach called the “Kinesthetic HMD”: actuating a head-mounted display with force feedback in order to provide sensations of self-motion. We discuss its design considerations and demonstrate an augmented flight simulator use case with a proof-of-concept prototype. We conducted a user study assessing our approach’s ability to enhance self-motion sensations. Taken together, our results show that our Kinesthetic HMD provides significantly stronger and more egocentric sensations than a visual-only self-motion experience. Thus, by providing congruent vestibular and proprioceptive cues related to balance and self-motion, the Kinesthetic HMD represents a promising approach for a variety of virtual reality applications in which motion sensations are prominent.

The sensation of self-motion is essential in many virtual reality applications, from entertainment to training, such as flying and driving simulators. If the common approach used in amusement parks is to actuate the seats with cumbersome systems, multisensory integration can also be leveraged to get rich effects from lightweight solutions. In this paper, we introduce a novel approach called the &#x201c;Kinesthetic HMD&#x201d;: actuating a head-mounted display with force feedback in order to provide sensations of self-motion. We discuss its design considerations and demonstrate an augmented flight simulator use case with a proof-of-concept prototype. We conducted a user study assessing our approach&#x2019;s ability to enhance self-motion sensations. Taken together, our results show that our Kinesthetic HMD provides significantly stronger and more egocentric sensations than a visual-only self-motion experience. Thus, by providing congruent vestibular and proprioceptive cues related to balance and self-motion, the Kinesthetic HMD represents a promising approach for a variety of virtual reality applications in which motion sensations are prominent.