Exploring the Perceived Realism of XR Experiences: Unveiling the Impact of Cutting-Edge Simulation Tools and their Interplay with Human and Contextual Factors

Mincheol Shin^1*David C. Jeong²Tamara Makana Chock³

• ¹Tilburg School of Humanities and Digital Sciences, Tilburg University, Tilburg, Netherlands
• ²Department of Communication, Santa Clara University,, Santa Clara, CA, United States
• ³S.I. Newhouse School of Public Communications, Syracuse University, Syracuse, New York, United States

Among the various human and contextual factors, the article by Mal et al. (this volume) suggests that congruence and plausibility could play decisive roles. Supporting previous studies that emphasize the role of coherence (Skarbez, 2016;Skarbez et al., 2017), the authors found that incongruence between the realism of the self-avatar and other social actors in a virtual environment can significantly impact the perception of self-location and self-identification, whereas congruence between the realism of the self-avatar and the user's physical body enhances perceived virtual body ownership and self-identification. These findings emphasize that the referential frame users hold in mind is important in determining the perceived realism of virtual experiences, as they evaluate the discrepancies between the realism of virtual actors (or objects) and their expectations-formed based on previous experiences and knowledge-at a perceptual level, as suggested by Slater et al. (2022).In alignment with these findings, the article by Pouke et al. (this volume) provides another perspective on how coherence influences the perceived realism of physics in virtual environments. By conducting two creative experiments in which the scale of participants (normal vs. reduced) and the physics of virtual models (true vs. movie) are varied based on the virtual characters of a scaled-down robot and a regular-sized cat, the authors found that a key contextual factor-whether the familiarity of a virtual character or expectations regarding its identity match its physics (rigid-body dynamics)-can determine the perceived realism of a virtual character's physics. The findings of this study suggest that the contextual coherence between users' beliefs about a virtual character's identity and its body dynamics, formed based on prior experiences, could act as a perceptual ruler in determining the perceived realism of XR experiences.At the individual level, the article by Kim et al. (this volume) demonstrates that enhancing the realism of virtual objects via stereoscopic visualization improves the reaching performance of participants in a virtual environment-but only for young participants aged 22 to 26, compared to older participants aged 69 to 88. This finding highlights the importance of considering individual differences in perceptual ability, stemming from physical limitations or previous experiences, when understanding how task performance and behavioral realism are influenced by perceptual differences at the individual level.From a technical standpoint, previous literature on XR technologies has predominantly focused on enhancing the perceived realism of virtual experiences through visual and auditory feedback. However, the article by Li et al. (this volume) suggests that perceived realism could be further enhanced by incorporating tactile feedback via a wearable haptic system with kinesthetic feedback. Although several technical challenges must be addressed before this technology becomes commercially viable, the article underscores the importance of integrating multi-sensory feedback to enhance the perceived realism of virtual experiences.Finally, the article by Hameed et al. (this volume) provides a holistic framework supporting the idea that human and contextual factors interact with system (i.e., technical) factors in determining the quality of virtual experiences-namely, realism constructs such as presence, plausibility, and embodiment. By defining virtual experiences through a fivedimensional taxonomy of immersivity, interactivity, explorability, plausibility, and believability, the authors offer a comprehensive perspective on the system, human, and contextual factors that influence the quality of users' virtual experiences, as well as methodological approaches for measuring and assessing them. Notably, this article also underscores the importance of considering plausibility and coherence (i.e., believability) when assessing the quality of virtual experiences, particularly in terms of perceived realism.The articles collected for this special issue offer several pieces of food for thought that could attract future researchers for further investigation. First of all, now that we recognize the importance of contextual factors in determining the perceived realism of virtual experiences, we are left with the unanswered question of whether the cognitive and emotional processes underlying the use of various XR technologies (e.g., virtual reality [VR], augmented reality [AR], and mixed reality [MR]) will differ depending on their structural features, as hinted at by recent studies (Shin et al., 2024;Shin & Onderdijk, 2024). For instance, the fact that VR structurally differs from AR and MR-eliciting a sense of "being there" in another virtual place, whereas AR and MR create the perception of a virtual object "being in my place"-might prime users with different expectations for their experiences. If so, what would serve as the individual and contextual reference points for VR, AR, and MR in determining the perceived realism of users' experiences and relevant cognitive and emotional processes? Furthermore, to what extent might the technical implementation of multi-sensory feedback via state-of-the-art technologies, such as wearable haptic systems and digital scent technology (i.e., olfactory haptics), interact with human and contextual factors? Addressing these research questions may provide valuable insights into how emerging XR technologies can be leveraged to create plausible virtual experiences and how they influence subsequent cognitive and emotional processing across different XR modalities.