Editorial: Remote XR user studies

Editorial on the Research Topic
Remote XR user studies

Introduction

Extended reality (XR) technology, which includes virtual reality (VR), augmented reality (AR), and augmented virtuality (AV), allows people to interact with content in new ways. For example, in the case of VR, users can enter a new reality to practice training, reduce stress, or play games. In AR people are able to augment the real world with the virtual world, providing a fusion of the digital and physical. Studies into the efficacy of such environments have typically been conducted in controlled, specialized research laboratories with co-present participants, researchers, and equipment. When the COVID-19 pandemic rendered in-person research unsafe, many XR researchers turned to the internet to source participants to complete user studies and experiments at a distance. Whilst this had already become common practice in HCI research on computer based applications or smartphone apps, little has been published on conducting remote XR user studies.

As the trend toward remote and hybrid work grows, it is likely that XR research will increasingly be conducted online. Additionally, the increasing popularity of commercial XR hardware at home and at work means that remote participants will become more accessible, less costly and time consuming to involve in research studies, and researchers will be able to reach larger and more diverse populations of participants. This Research Topic of Frontiers in Virtual Reality presents a collection of issues faced and lessons learnt from a variety of different XR studies from health-based applications, location based experiences, and learning environments.

This Research Topic starts with Ratcliffe and Tokarchuk's paper on “The potential of remote XR experimentation: defining benefits and limitations through expert survey and case study” which provides an engaging summary of the issues listed above, as well as the state of remote XR research both before the COVID pandemic and today. The paper surveys contemporary XR researchers to understand potential benefits and limitations of remote XR experimentation and contextualizes these benefits and limitations through a case study of fully encapsulated XR experiments. The paper also provides practical guidance for researchers designing unmoderated remote XR studies. Lastly, the authors propose steps the XR research community can take to evaluate the validity of fully encapsulated research, in which experiments occur without researcher oversight, compared to un-encapsulated remote research and lab research.

Further exploring challenges and opportunities of remote XR user studies, the paper “Lessons learned running distributed and remote mixed reality experiments,” (Steed et al.) provides a review of experiences conducting four distributed and remote Mixed Reality (MR) experiments. The work provides particularly insightful views into technical issues that arose and how they were addressed, as well as an in-depth presentation of new tools such as Ubiq, a remote XR toolkit, to support remote XR experimentation.

The paper “The reality of remote extended reality research: Practical case studies and taxonomy” (Kroma et al.), also provides case studies (six) while considering what worked well and what did not in a variety of different remote scenarios. These include larger participant pools, wider inclusion and easier facilitation of longitudinal studies. However, it also goes beyond this, to provide preliminary taxonomies to create such studies in a systematic and easy-to-follow manner, encouraging other researchers to move into sharing practical data-informed proposals and guidelines.

“Mitigation strategies for participant non-attendance in VR remote collaborative experiments” (Bovo et al.) explores the issue of participant non-attendance during remote XR experimentation through the creation of a metric for measuring the cost of participant non-attendance. They use this metric to explore different recruitment strategies, and using evidence that a personal connection between researcher and participant reduces non-attendance, find evidence that a remote recruitment process that includes a short pre-experiment online meeting between researcher and participant boosts attendance. While having further implication, this is discussed through the lens of a series of collaborative online experiments, for which the cost of non-attendance is high.

In the realm of augmented reality, “Remote evaluation of augmented reality interaction with personal health information” (Shaer et al.) presents an evaluation of two studies conducted using an augmented reality personal health information system. It demonstrates not only the potential for AR interventions in personal health, but contributes to remote XR user study methodology by examining their experiences with these home based remote user studies and proposing a new framework, DICRAs for design choices of remote AR studies.

Additionally, “Developing a play-anywhere handheld AR storytelling app using remote data collection” (Raeburn et al.) presents an in-depth case study into the development and of remote AR experiments for AR story-telling. Across five studies, Raeburn et al.'s research covers the feasibility and testing approaches of these remote AR experiences, while reflecting and analyzing a second iteration of improvements. Usefully, they also provide a comparison between these “remote” use-cases, in which participants could use the AR application anywhere, and a site-specific use-case, in which the AR activity takes place in a pre-arranged location.

Finishing off the Research Topic is “Remote iVR for nutrition education: From design to evaluation” (Sajjadi et al.), which offers a practical example of remote XR user study in practice. It looks at the use of a VR application which allows users to manipulate virtual food items and test hypotheses about portion size control in VR. Results show an improved participants' knowledge of portion size control, especially in interactive condition compared with passive condition. This suggests that the level of interactivity in a VR study may affect the perceived learning experience of users. Overall, this study demonstrates the feasibility of conducting remote VR experiments using complex and interactive health-related apps.

This collection provides theoretical and empirical evidence regarding the current benefits and limitations of remote XR research, demonstrated through numerous case studies. Perhaps the more intriguing aspect of this collection is how these limitations can be avoided, and the benefits maximized. Potential solutions are offered, including fully encapsulating experiments, leveraging purpose-specific toolkits, encouraging greater collaboration and laser-focusing on individual solutions to the specific issues that could affect remote XR user studies.

Author contributions

LT, JR, MH, and LM drafted the editorial. All authors conceived and edited the final version. All authors contributed to the article and approved the submitted version.

Funding

JR and NB-K were funded by the EPSRC and AHRC Centre for Doctoral Training in Media and Arts Technology (EP/L01632X/1). FO was funded by the National Science Foundation (NSF 2106590, NSF 2016714, NSF 2037417, and NSF 1948254).

Conflict of interest

MH was employed by Google. LM was employed by Accenture Song.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.