Jiayan Zhao
Bernhard E. Riecke
Jonathan W. Kelly
Jeanine Stefanucci
Alexander Klippel- 1Laboratory of Geo-Information Science and Remote Sensing, Wageningen University & Research, Wageningen, Netherlands
- 2iSpaceLab, School of Interactive Arts and Technology, Simon Fraser University, Burnaby, BC, Canada
- 3Department of Psychology, Iowa State University, Ames, IA, United States
- 4Department of Psychology, The University of Utah, Salt Lake City, UT, United States
Editorial on the Research Topic
Human spatial perception, cognition, and behaviour in extended reality
The concept of eXtended Reality (XR) has recently become a staple of the mainstream discussion of technological innovation. XR is an umbrella term for digital technologies simulating sensory experiences in real or imagined environments, such as Augmented Reality (AR) and Virtual Reality (VR). These technologies replace or augment the physical world with a mediated version of what designers want the world to look like. To that end, XR refers to a wide range of hardware and software platforms, from partial sensory inputs to fully immersive bodysuits, to make users believe they are in another environment or interacting with something or someone that does not exist in reality.
Its inherent spatiality, the freedom of design, and full control over experiences make XR ideally suited for investigating spatial perception and cognition. Technologies that use space as input and output, such as motion controllers and position sensors, fall under spatial computing. At its core, spatial computing defines how we explore and interact with our surroundings (Pangilinan et al., 2019). In an XR environment, the user’s position and relationship to objects within that environment are synchronized to maintain a unified experience between different modalities (virtual-real or virtual-virtual). The rich visual-spatial cues that are accessible from an egocentric, embodied perspective offer a source of stimulation for users to think spatially and guide their actions. By replicating real-life scenarios and sensory input, XR has the potential to support, facilitate, or develop spatial thinking (i.e., thinking in, about, and with space) in a comparable manner to everyday activities but with the freedom of infinite realities.
XR environments are highly flexible and programmable, opening up research opportunities for the development of novel methods to understand human cognition and behaviour. It should be noted that the quality of the experience does not rest on whether it follows the physical laws of the real world. A digital representation that is entirely different from the physical one may still be perceived to be plausible as long as its spatial features and object movements and interactions comply with internally consistent rules that are reasonable for the presented scene (referred to as coherence by Skarbez et al., 2021).
Experiencing XR content with a high level of coherence can lead to a strong sense of presence, or the illusion of being physically present in the mediated environment. Sense of presence is an important characteristic of XR and has been studied by many researchers. Wirth et al. (2007) propose a two-step process for the formation of spatial presence, during which the construction of a mental model of the mediated space is advocated as a precondition for presence to emerge. The construction of mental models includes the detection and comprehension of spatial cues directly from the simulated environment, and users’ mental capacities are bound by mediated stimuli rather than physical reality. Wirth’s framework provides a theoretical basis for the importance of understanding how users perceive, reason, and memorize space in XR.
The goal of this Research Topic is to bring together different perspectives that address research questions related to human spatial perception, cognition, and behaviour in XR. The papers in this Research Topic centre around four main themes: Ecological validity of spatial perception and cognition; Embodiment and spatial behaviour; Locomotion and cybersickness; and XR as a design (and research) tool.
Ecological validity. Considering behavioural research, one primary application of XR is to mimic real-world settings through realistic visual representations and sensory feedback. However, reproducing every aspect of reality is not yet possible. One question of concern is: To what extent can XR accurately convey the spatial properties of physical space like distance and object dimensions? It has been shown that distances are frequently underestimated in virtual compared to physical environments (Creem-Regehr et al., 2023), and there seems to be no exception even with modern head-mounted displays (HMDs). This challenge is evidenced by a study of this Research Topic evaluating distance perception in Oculus Quest 1 and 2 (Kelly et al.). The paper by Juliano et al. finds that also the visual processing of actions directed toward objects differs between real and virtual environments.
Embodiment. Human beings have evolved to understand the world with our bodies (Wilson, 2002). The recent advancement of position tracking in XR allows users to bring their bodies into experiences as instruments to naturally deal with space and movement. This often means a direct exploration of 3D environments through an egocentric perspective at normal eye level. Performing navigation tasks with such a natural interface has the potential to improve the validity of assessing spatial abilities and skills in laboratory settings, as demonstrated by a study focusing on spatial perspective taking in ambulatory VR (He et al.). Furthermore, XR technologies allow users to be embodied in new ways—examples include experiencing a virtual body different from their own, as varied in body weight (Wolf et al.) or identity (Kenneth et al.), and viewing an avatar from different spatial perspectives for imitation (Barhorst-Cates et al.). These novel manipulations of embodied cues help investigate the relationship between the body and our experience of virtual worlds.
Locomotion and cybersickness. Although modern HMDs provide full motion tracking capabilities, locomotion in virtual environments is still a challenging issue due to the fact that users are usually constrained by the physical space. As a compromise, some designers choose joystick steering or teleportation to realize translational movements (Zhao et al., 2020). However, the discrepancy between virtual and physical motion cues can cause cybersickness (Saredakis et al., 2020) and disorientation (Cherep et al., 2020), leading to frustrating user experiences and decreased spatial learning. In this Research Topic, Ang and Quarles provide a comprehensive review of the methods researchers and practitioners have adopted to reduce cybersickness. The use of environmental features like boundaries to maintain spatial orientation during VR teleportation has also attracted the attention of researchers (Kelly et al.).
XR as a design tool. XR technologies have been widely used in psychological research related to spatial cognition for simulating navigation and decision making scenarios. The programmability and controllability of XR provide a tool by which we can understand how users may want future applications to be designed. For example, utilizing VR to simulate experiences that would be possible only through a certain type of media, we can prototype and evaluate the effectiveness of an idealized user interface that is not constrained by hardware resources. Areas of interest for researchers are the design and allocation of visual cues on mobile maps (Cheng et al.) or next-generation AR headsets (Stefanucci et al.; Gardony et al.) to aid spatial and situational awareness. In the context of a discussion on XR for research, the development of frameworks for conducting VR experiments should incorporate a series of core components, such as analysability and reproducibility, to support researchers with different methodological and technical needs (Grübel).
In summary, this Research Topic explored the role of XR in spatial perception, cognition, and behaviour and vice versa. XR offers a plethora of opportunities to reveal new or unexplored dimensions of human interaction with environments. The inherently spatial nature of XR underscores the need for future research to take into account spatial cognition principles and implications when developing XR-enabled experiences. We hope that the Research Topic will inspire more studies along these lines and further scientific and practical interests at the important intersection of spatial cognition and XR.
Author contributions
JZ drafted the manuscript and all authors were involved in revising the manuscript. All authors contributed to the article and approved the submitted version.
Acknowledgments
The authors thank the paper contributors and reviewers for making this Research Topic a success.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Cherep, L. A., Lim, A. F., Kelly, J. W., Acharya, D., Velasco, A., Bustamante, E., et al. (2020). Spatial cognitive implications of teleporting through virtual environments. J. Exp. Psychol. 26, 480–492. doi:10.1037/xap0000263
Creem-Regehr, S., Stefanucci, J. K., and Bodenheimer, B. (2023). Perceiving distance in virtual reality: Theoretical insights from contemporary technologies. Philosophical Trans. R. Soc. Lond. 378, 20210456. doi:10.1098/rstb.2021.0456
Pangilinan, E., Lukas, S., and Mohan, V. (2019). in Creating augmented and virtual realities: Theory and practice for next-generation spatial computing. Editors E. Pangilinan, S. Lukas, and V. Mohan (Sebastopol, CA: O'Reilly Media).
Saredakis, D., Szpak, A., Birckhead, B., Keage, H. A. D., Rizzo, A., and Loetscher, T. (2020). Factors associated with virtual reality sickness in head-mounted displays: A systematic review and meta-analysis. Front. Hum. Neurosci. 14, 96. doi:10.3389/fnhum.2020.00096
Skarbez, R., Smith, M., and Whitton, M. C. (2021). Revisiting milgram and kishino's reality-virtuality continuum. Front. Virtual Real 2, 647997. doi:10.3389/frvir.2021.647997
Wilson, M. (2002). Six views of embodied cognition. Psychonomic Bull. Rev. 9, 625–636. doi:10.3758/BF03196322
Wirth, W., Hartmann, T., Böcking, S., Vorderer, P., Klimmt, C., Schramm, H., et al. (2007). A process model of the formation of spatial presence experiences. Media Psychol. 9, 493–525. doi:10.1080/15213260701283079
Keywords: spatial cognition, virtual reality, immersive technologies, presence, virtual environments, spatial thinking
Citation: Zhao J, Riecke BE, Kelly JW, Stefanucci J and Klippel A (2023) Editorial: Human spatial perception, cognition, and behaviour in extended reality. Front. Virtual Real. 4:1257230. doi: 10.3389/frvir.2023.1257230
Received: 12 July 2023; Accepted: 13 July 2023;
Published: 31 July 2023.
Edited and reviewed by:
Stephen Palmisano, University of Wollongong, AustraliaCopyright © 2023 Zhao, Riecke, Kelly, Stefanucci and Klippel. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Jiayan Zhao, jiayan.zhao@wur.nl