Skip to main content

PERSPECTIVE article

Front. Virtual Real., 30 August 2024
Sec. Virtual Reality and Human Behaviour
This article is part of the Research Topic Exploring the Perceived Realism of XR Experiences: Unveiling the Impact of Cutting-Edge Simulation Tools and their Interplay with Human and Contextual Factors View all 3 articles

A holistic quality taxonomy for virtual reality experiences

  • 1Department of Electronic Systems, Norwegian University of Science and Technology (NTNU), Trondheim, Norway
  • 2Quality and Usability Lab, Technische Universität Berlin (TUB), Trondheim, Germany

The rapid advancement of virtual reality (VR) technology has brought many immersive experiences, each designed to transport users into captivating virtual worlds. While these experiences aim to provide a sense of presence and engagement, the factors contributing to a truly immersive experience are often complex and multifaceted. Existing scholarship has predominantly focused on specific aspects of user experience, such as psychological factors (e.g., sense of presence), emotional factors (e.g., enjoyment), or design-related factors (e.g., interface usability). This fragmented approach has impeded a comprehensive understanding of the overall quality of VR experiences. To address this, we propose a multidimensional taxonomy encompassing five essential qualities: immersivity, interactivity, explorability, plausibility, and believability. The framework aims to disentangle the complex, interrelated facets shaping VR experiences for a more systematic evaluation. Immersivity refers to the subjective sense of presence and “being there” in a virtual environment. Interactivity denotes the ability to interact with virtual objects, promoting engagement dynamically. Explorability refers to users’ freedom to navigate and discover new elements. Plausibility examines the logical congruence of the virtual environment’s rules and behaviors. Finally, believability relates to the world-building and internal coherence of the VR world. This taxonomy provides a structured approach to look at VR experiences holistically, assessing the interplay of these facets to facilitate a more objective, comprehensive evaluation, capturing the multidimensional nature of VR experiences. In summary, our proposed taxonomy marks an essential step toward systematic VR evaluation, providing guidance for researchers and highlighting factors integral to VR quality.

1 Introduction

Since the arrival of immersive VR, our media consumption has evolved beyond the confines of rectangular screens to fully immersive interactive environments, further transforming viewers into active participants. From the early days of bulky, wired prototypes to the current lightweight, standalone systems, VR has taken user engagement and immersion to unprecedented levels–building immensely on the interactivity once limited to digital games. The rise of social VR has further expanded the boundaries of these virtual worlds, allowing multiple users to inhabit and interact within shared spaces simultaneously. At the same time, this media convergence has blurred the lines between conventional telecommunications and human-computer interaction (HCI) systems, presenting unique opportunities and challenges for both fields (Perkis et al., 2020). Comprehensively evaluating the quality of these complex VR experiences remains a significant challenge. Existing approaches often focus on specific aspects of the experience, such as technical performance metrics or user-centric aspects, without considering the holistic interplay of factors that shape the overall VR experience (Keighrey et al., 2017). Quality assessments in VR must account for the system, the content or application, and the user’s overall experience, all of which are closely intertwined with the user’s psychological state.

1.1 Existing approaches and limitations

Over the years, numerous influential frameworks have laid the groundwork for understanding the unique characteristics of VR experiences. These frameworks have highlighted the multifaceted nature of VR, encompassing various dimensions. For example, the sense of being present in a virtual environment has been a central focus of many VR frameworks. Slater and Wilbur (1997) differentiated between immersion as an objective description of the technology and presence as the subjective experience of being in a virtual environment. This distinction has been widely adopted in subsequent research (McMahan, 2013; Witmer and Singer, 1998; Nilsson et al., 2016). A virtual world being immersive in its life-likeness reflected by the extent of its inclusiveness, extensiveness, surroundedness, and vividness (Slater, 2018). Separately, Steuer Steuer (1992) identified interactivity as a key component of VR, defining it as the extent to which users can modify the form and content of a mediated environment in real-time. Bowman and Hodges (1999); Mütterlein (2018); Mütterlein and Hess (2017) further emphasized the importance of interaction and responsiveness, noting that it enables users to actively engage with and influence the virtual environment. Another critical aspect closely recognized with interactivity is the extent to which a virtually mediated environment is explorable, highlighting the significance of users’ ability to freely navigate and discover the virtual world (Boletsis, 2017; Flotyński et al., 2017; Flotyński et al., 2018). There is also a strong position held vis-a-vis the narrative/transportation capabilities of a VR experience in terms of the strength of its stories and characters (Van Laer et al., 2019). Slater and Wilbur refer to it as plot Slater and Wilbur (1997) whereas narrative immersion is common within game and literary studies (Arsenault, 2005; Ryan, 2015). Magnenat-Thalmann et al. (2005) and Luciani et al. (2004) have emphasized the importance of believable virtual characters and environments, arguing that it is crucial for maintaining user engagement and ensuring a suspension of disbelief. Slater et al. (2009); Slater et al. (2022); Skarbez (2016); Skarbez (2020) have written on coherence and plausibility of the mediated world in whether it behaves in a manner consistent with the user’s expectations based on their experiences in the real world.

Despite the valuable insights these frameworks provide, existing VR quality assessment approaches often have limitations. Many studies have relied heavily on self-report inventories and questionnaires that focus primarily on presence and immersion (Kim and Biocca, 1997; Lombard et al., 2009; Witmer and Singer, 1998; Lessiter et al., 2001; Schubert et al., 2001) or emotional responses related to user experience (engagement, enjoyment, usability, challenge, etc.) (Jennett et al., 2008; Laugwitz et al., 2008; Brockmyer et al., 2009; IJsselsteijn et al., 2013; Takatalo et al., 2008). While these measures provide useful information, they may not fully capture the complex interplay between the various technical, experiential, and psychological factors that shape VR experiences (Cummings and Bailenson, 2016). Psychological measures in VR focus on broader mental processes, traits, and states, while experiential measures emphasize the immediate, subjective qualities of user interaction within the virtual environment. While psychological measures provide insights into overall mental factors, experiential measures capture the unique, in-the-moment aspects of VR engagement. Moreover, many existing approaches have been criticized for their lack of standardization and inconsistency in terminology and measurement (Cummings and Bailenson, 2016; Skarbez, 2016). This fragmentation has made it challenging to compare results across studies and has hindered the development of a comprehensive understanding of VR quality.

1.2 The need for a broad taxonomy

To properly evaluate immersive VR experiences, it is crucial to consider their technology-related attributes using objective metrics and quantitative methods. At the same time, its vital to also focus on the design-related issues, functionality, and comfort of the system using qualitative heuristic evaluation and usability testing methods. However, the most critical aspect of VR lies in the subjective feelings and experiences of the user while immersed in the virtual environment. These experiential qualities extend beyond pragmatic considerations to include hedonic aspects such as aesthetics, enjoyment, and meaning.

A comprehensive assessment of VR experiences is therefore not a result of system capabilities and product qualities alone but intrinsically linked to the psycho-phenomenological dimensions it elicits–namely, presence, immersion, and embodiment. Without them, there is no VR. Additionally, just like the feeling of “being there” is crucial to delivering this illusion. No less important is its “sense of ‘trueness and genuineness”–its authenticity–that offers credence to the illusion (Hameed and Perkis, 2024).

Therefore, cross-examining these interconnected factors is essential to evaluate the overall user-perceived experience (Held and Durlach, 1992; Moller et al., 2009; Hameed et al., 2019). To address the limitations of existing approaches and provide a more holistic perspective on VR evaluation, we propose a five-dimensional taxonomy for VR experiences (Figure 1). The taxonomy aims to disentangle the often-conflated concepts and provide a structured framework for assessing the quality of VR experiences.

Immersivity captures the subjective sense of being transported into a VE, characterized by a feeling of “being there.” and made possible by various system aspects.

Interactivity refers to the ability of users to interact with the VE and influence their experience, resulting in a sense of control and engagement.

Explorability captures the ease and degree of freedom users can navigate and discover new elements within the VE.

Plausibility refers to the congruence of the VE, its rules and interactions, aligning with user expectations and cognitive models.

Believability refers to the extent to which the VE appears realistic and coherent, giving it a sense of authenticity and acceptance.

Figure 1
www.frontiersin.org

Figure 1. A taxonomy for QoS and QoE and inter-relational factors of IMEx.

The value of this broad taxonomy lies in its ability to bridge the gap between theoretical constructs and practical implications. By encompassing various dimensions, from system-level factors to human-centric features, as well as physiological and psychological measures, this hierarchical structure enables a holistic understanding of VR quality aspects and facilitates informed evaluations. It is important to note that our taxonomy builds upon the extensive scholarship available in the field, presenting a novel interpretation of these concepts that integrate existing research while offering a fresh perspective on the interplay between different aspects of VR experiences. As such, We present our taxonomy as a living framework, acknowledging that VR technology and user experiences are continually evolving. This living aspect ensures that our taxonomy remains relevant and comprehensive as new insights and developments emerge.

2 Factors influencing VR quality

It is worth noting that each of the five abstractions highlighted in this taxonomy, see Table 1, are simultaneously both quality elements (technical or system-related attributes that influence quality) and quality features (perceptual characteristics or dimensions that are relevant for quality judgment) (Csikszentmihalyi et al., 2014; Jekosch, 2005). These abstractions were selected and refined based on a comprehensive review and synthesis of key concepts and frameworks, ensuring a balanced and well-informed taxonomy carefully considering the high-level influence factors (IF) that affect each abstraction, namely, system, user, and context (Perkis et al., 2020). By systematically examining the interplay between these factors and their impact on the five abstractions, we aim to achieve a robust framework for assessing VR quality.

System IF: VR systems largely depend on their hardware and software components. Display and graphics quality, tracking accuracy, latency, network stability, and overall system reliability are critical factors that directly impact the sensory elements of the VR experience, ultimately contributing to the psycho-phenomenological effects experienced by the user (Gobbetti and Scateni, 1998). In multiplayer VR environments, additional factors such as interoperability, compatibility, and scalability of the operating system and software can significantly influence the quality of the experience (Yin et al., 2021). Moreover, the content, rendering, and audio quality play a vital role in creating rich and engaging VR experiences (Sherman and Craig, 2018). The design, usability, comfort, and safety of interfaces and devices also significantly shape the overall user experience.

Human IF: User characteristics, both static and dynamic, profoundly impact the perception and experience of VR. Static factors include demographic attributes such as age, gender, and physical abilities (Jerald, 2015). For example, older users may require simpler interfaces or slower-paced experiences, while experienced gamers prefer more complex and challenging content (Mütterlein and Hess, 2017). Dynamic factors encompass the user’s prior experience with VR technologies, spatial ability, and gaming experience, which can influence presence, engagement, dissociation, and learnability (Jurnet et al., 2005; Möller et al., 2013). Other dynamic, physiological factors, such as visual acuity and susceptibility to motion sickness, also play a significant role in determining the quality of the VR experience. Additionally, the user’s current mood, stress level, mental fatigue, and imaginative faculties can all shape their perception and response to the various illusions presented within the VR medium (Howard and Van Zandt, 2021).

Context IF: The context in which a VR experience occurs can significantly impact its quality. Environmental conditions can affect the performance of VR hardware and the user’s comfort level. Factors such as room size, obstacles, ambient light, and noise can influence tracking accuracy, immersion, and safety (Falahee et al., 2000; Chang et al., 2020). The purpose of the VR application, whether it be entertainment, education, training, or therapy, is a unique determinant that highlights the compositional nature of the experience. For instance, a VR game may prioritize low latency and high graphical fidelity, while a VR therapy session may emphasize user comfort and safety more. The length and frequency of VR sessions can also impact user comfort and safety, making it an important consideration in the overall evaluation of VR quality.

Table 1
www.frontiersin.org

Table 1. Technical factors and corresponding user-centric features for each of the five abstractions.

2.1 Assessments of VR quality

Selecting appropriate assessment methods is crucial for comprehensively evaluating the quality of VR experiences. A combination of VR-specific methods, such as presence questionnaires and physiological measures, and more general HCI-related methods, like usability tests and user experience evaluations, is necessary to capture the unique aspects of VR while also considering established usability and user experience principles. A brief overview thus follows.

Self-Reported Measures: involve asking users to provide subjective feedback on their VR experiences. These assessments often utilize questionnaires or surveys to gauge user perceptions and emotions regarding presence, involvement, engrossment, realism, etc. Some widely used self-reported measures include the MEC-SPQ (Vorderer et al., 2004), PQ-ITQ (Witmer and Singer, 1998), TPI (Lombard et al., 2009), ITC-SOPI (Lessiter et al., 2001), among many. Subjective measures that capture users’ enjoyment and satisfaction (Bradley and Lang, 1994) are also regularly employed.

Physiological Measures: are objective measures that capture users’ physical reactions to the VR experience, providing insights into their engagement and potential discomfort. These include Electroencephalography (EEG) for brain activity (Baumgartner et al., 2006), Electrooculography (EOG) for eye movements, Electrodermal Activity (EDA) for skin conductance, Heart Rate Variability (HRV) (Salgado et al., 2018; Egan et al., 2016), and more recently, fMRI scans have gained popularity (Hoffman et al., 2003; Taube et al., 2013). Eye tracking systems are incorporated to monitor eye movements, revealing user focus, attention patterns, and gaze fixations within the VR environment (Clay et al., 2019; Rappa et al., 2022).

Performance Measures: evaluate users’ ability to complete tasks or achieve goals within the VR environment. These metrics can be objective or subjective, depending on the specific task or activity (Hameed et al., 2023; Voigt-Antons et al., 2020).

Usability and Task Load Measures: assess how easy and intuitive a VR experience is to use, considering factors like task completion, error rates, time on task, and user satisfaction. Standard usability metrics include System Usability Scale (SUS) (Brooke et al., 1996), NASA-TLX (Hart and Staveland, 1988), and After Scenario Questionnaire (ASQ) (Lewis, 1991), among others.

3 Towards a quality taxonomy

In this section, we break down each abstraction into technical factors responsible for achieving it and the human-centric factors that describe its effects. Further, we provide an assessment methodology for each of the five abstractions.

3.1 Immersivity

The extent to which a user feels surrounded by and present inside a virtual environment. It is critical in drawing users into the virtual world, creating a sense of presence and engagement. Several factors affect immersivity, including the sensory modalities available, their fidelity, and their vividness. Representational fidelity involves conveying a sense of place through sensory and symbolic cues.

3.1.1 Technical factors

1. Visual fidelity: Visual quality and visual realism are achieved through high-resolution displays, wide Field-of-View (FoV), and advanced rendering techniques. A VR headset provides an angular extent of the observable world. A wider FoV gives a better feeling of surroundness (Slater and Wilbur, 1997), enhancing immersion (LaValle, 2016). High-resolution displays with dense pixels (pixels per degree) contribute to richness and vividness that enable perceptual absorption (Sheridan, 2016; Steuer, 1992).

2. Tracking: a system’s ability to track a user’s position and orientation. Tracking allows real-time synchronization of users’ movements in the real world with those in the virtual world. Jitter-free motion tracking that covers multiple degrees of freedom, the precision of gesture recognition, and accurate perspective and auditory inputs relative to the mediated world contribute to embodied immersion (McMahan, 2013; Kilteni et al., 2012). Full-body tracking yields the maximum possible embodiment.

3. Persistence, Latency, and Refresh Rates: High persistence causes motion blur, whereas high latency causes lag and makes the experience feel unnatural. Similarly, lower refresh rates lead to motion sickness. A higher refresh rate makes an image feel smoother. Therefore, lowering latency and persistence while aiming for higher frame rates is desirable for reducing discomfort motion sickness, or nausea (Chang et al., 2020; Saredakis et al., 2020) – all of which can result in break-in-presence (Slater, 2002).

4. Audio fidelity: Surround ambionic sound and binaural audio sufficiently improves experience in virtual worlds. Sound sources at 360° around the user, replicating natural sound waves, contribute to aural immersion (Spors et al., 2013). Adding sound effects and ambient sounds further improves the user’s sense of envelopment (Rumsey, 2002).

5. Headset Types: Depending on their purpose, some VR headsets can be tethered to offer premium performance, while others offer standalone or wearable for versatile everyday use or hands-free work. In VR headsets, for example, visual occlusion and reduced local ambient noise (with headphones) enforce isolation (Witmer and Singer, 1998). In general, heavy devices with hanging wires and loose fits can lead to distraction (Held and Durlach, 1992). Lightweight HMDs reduce discomfort and motion sickness (Howard and Van Zandt, 2021).

3.1.2 Human-centric factors

1. Presence: A sensation of being enveloped by the multisensory representation of the virtual world delivered via high-fidelity displays contributing to a stronger subjective feeling of “being there,” or presence, in the VE (Arsenault, 2005). Users experiencing presence in virtual environments report feeling genuinely transported to another reality. They describe a profound sense of immersion where the digital world becomes their primary reality, evoking authentic emotional and physiological responses. This psychological state often leads to users forgetting their physical surroundings, fully engaging with virtual objects and characters as if they were tangible and present (Slater et al., 2022; McMahan, 2013).

2. Attention: In virtual experiences, users often lose track of time, and external distractions fade away. Their focus narrows to the virtual world, with a heightened awareness of its details and events. This can be affected by factors such as the level of engagement, the novelty of the environment, and the presence of distractions (Murray et al., 2007).

3. Sense of embodiment: in virtual reality often describes feeling as if their virtual body is theirs. This feeling increases with the accuracy of the head tracking, body tracking, and motion capture systems to represent the user’s movements and position in the virtual world accurately (Kilteni et al., 2012; Kilteni et al., 2015). They report intuitive control over their virtual form, with movements feeling natural and responsive. This deep connection to their virtual self can lead to altered perceptions of their physical body, sometimes resulting in surprising sensations when returning to the real world (Peck et al., 2013; Slater and Sanchez-Vives, 2014).

3.1.3 Assessments

Objective:

Hardware Quality: resolution, sound system capabilities, number of pixels per inch (PPI)

Image quality metrics: Evaluate sharpness, contrast, and color fidelity using standardized tools like PSNR and SSIM (Sun et al., 2017).

Measure and analyze frame rate data to detect drops and ensure smooth motion.

FoV measurement: Utilize specialized equipment to measure the actual FoV provided by the HMD (Cutting, 1997; Masnadi et al., 2021).

Latency measurement: Employ tools to measure the delay between user actions and visual feedback to identify potential motion sickness triggers (Raaen and Kjellmo, 2015).

Sound spatialization accuracy: Assess the accuracy of Head-related transfer function (HRTF) implementation using objective metrics such as signal-to-noise ratio (SNR) and frequency response and distortion measures to assess the accuracy of sound localization and rendering in the VE (Serafin et al., 2018; Murphy and Neff, 2011).

Behavioral:

Use subjective scales and interviews to understand users’ sense of “presence” beyond just a yes/no answer. Analyze eye-tracking data to see where users focus and explore attention patterns (Li et al., 2021).

Analysis of user engagement and time spent with virtual objects and environments to understand how their behavior reflects the level of immersion and identify factors that enhance or detract from presence (Hameed and Perkis, 2021; Simone et al., 2006).

Psychological:

Assessments of spatial awareness: Employ open-ended questionnaires to gather in-depth feedback from users on the effectivity of system immersion (Servotte et al., 2020; Nilsson et al., 2016).

User feedback on feeling “presence”: using validated questionnaires or surveys to collect subjective ratings from users regarding their immersion and presence in the virtual world (Schwind et al., 2019; Schubert et al., 2001).

Focus groups and interviews: Conduct facilitated discussions with users to explore their perceptions of presence and the overall sense of being transported into the virtual world.

Post-experience evaluations: Ask users to reflect on their immersive experiences through journaling, storytelling, or creative expression.

Physiological:

Presence-related physiological responses (e.g., eye pupil dilation, increased heart rate, etc.) (Phillips et al., 2012).

Functional magnetic resonance imaging (fMRI) to study brain activity associated with presence (Hoffman et al., 2003)

Eye-tracking and gaze analysis to measure user’s attention patterns, focus, and engagement with specific elements of the virtual environment (Li et al., 2021; Clay et al., 2019).

Correlate physiological measures with subjective ratings on presence to validate their significance.

3.2 Interactivity

The degree to which the user can interact with the virtual world and influence its events. Being able to influence the VR experience is essential for fostering engagement and allowing active participation in the virtual world. Intuitive and responsive input devices that enable various actions for a natural and enjoyable experience also enhance a sense of agency or the feeling of being in control of the experience.

3.2.1 Technical factors

1. Intuitiveness and Responsiveness: The input devices’ ease of use and responsiveness enable natural interactions and afford various types of actions that the user can perform in the virtual world. Time delays between user input and system response (interaction latency) or the speed and efficiency of the system to handle data exchange (throughput) bear a significant influence on user experience and satisfaction (Elbamby et al., 2018; Brunnström et al., 2020). Responsive inputs, user-friendly interfaces, and interactivity features that meet or exceed the user’s expectations yield an overall positive experience (Wu et al., 2010).

2. Input Modality: Various modalities like gaze, laser, and hand-tracking offer different interaction styles (Harms, 2019). Positive experience depends on matching the modality of choice, the task at hand, the challenge level, and user-centered factors (Holderied, 2017; Fontaine, 1992). Integrating natural gestures and movements to interact with the virtual environment enhances immersion and engagement.

3. Device and Interface Appropriateness: primary concerns of usability, aesthetics, utility, etc., highlight that the ergonomics and functionality of devices and UIs significantly influence user experience (Moller et al., 2009; Rebelo et al., 2012). An intuitive control scheme that matches the user’s expectations based on their real-world experience ensures the naturalness of interactions (Bowman et al., 2012; Liou et al., 2017). Poor interface quality (Witmer and Singer, 1998), mismatches, and unfamiliarity can adversely affect performance and lead to unfulfillment and dissatisfaction.

3.2.2 Human-centric factors

1. User agency and control The user’s perception of control over their actions and outcomes in the virtual world fosters a sense of ownership and empowerment. Better tracking, intuitive controls, and timely feedback from the virtual world in response to interactions help users understand the consequences of their actions and motivate further engagement (Sheridan et al., 1992). This heightened sense of control often leads to increased emotional investment and motivation to explore, as users feel their actions have meaningful consequences within the virtual space. Thus improving the overall sense of agency (Kilteni et al., 2012).

2. Ease of Interaction: Simple and intuitive control mechanisms reduce cognitive load and allow users to focus on exploring and enjoying the experience. Balancing the complexity of interactions with user capabilities avoids overwhelming users and does not hinder engagement. Considering physical constraints, learning abilities, and potential fatigue ensures interaction methods are comfortable and user interfaces remain spatial, consistent, and accessible for extended VR sessions (Pallavicini et al., 2020).

3. Cognitive Adaptability: As discussed, optimal engagement and satisfaction is achieved when interactions align closely with user abilities. The balance between user skills and interaction complexity can lead to increased motivation, prolonged engagement, and a marked improvement in mental flexibility and problem-solving skills (Velichkovsky et al., 2017; Ryan et al., 2006). This fluidity and progress-reward cycle encourages a state of flow characterized by a blurring of time, intense focus and absorption into the activities performed inside the mediated environment (Csikszentmihalyi et al., 2014).

3.2.3 Assessments

Objective:

Controller responsiveness: Measure the latency between user input and the corresponding virtual action. Use eye-tracking and motion capture techniques to capture accurate input timings and interactions’ overall smoothness and fluidity (Seibert and Shafer, 2018; Masurovsky et al., 2020).

Interaction latency: Measure the delay between user input and the corresponding response from the virtual environment. Use event markers or synchronization protocols to capture accurate response timings (Brunnström et al., 2020).

Behavioral:

Completion Speed: Measure users’ time to learn, perform, and complete tasks using standardized tasks and time-to-completion metrics. Gather subjective ratings from users regarding the efficiency and speed of completing tasks in the virtual environment (Patel et al., 2006).

Interaction Accuracy: Use task-based performance metrics to assess the efficiency and precision of user interactions with virtual objects and environments. Use methods like time to target and error rates to assess performance (Yang et al., 2019).

Integration of natural gestures: Measure the accuracy and effectiveness of natural gesture recognition using objective metrics such as recognition rate and error rates (Seibert and Shafer, 2018; Kangas et al., 2022).

Analyze user navigation patterns and preferences to identify areas of interest and potential points of frustration in the interactive environment.

Observe users’ interactions with virtual objects and entities to assess their understanding of the interaction mechanics and their ability to manipulate the virtual world. Actions, expressions, and verbalizations can be documented to gain insights into their perceived usability and engagement.

Psychological:

Sense of agency: Evaluate the user’s perception of control over their actions and the outcomes in the virtual world using subjective questionnaires and qualitative interviews (Kangas et al., 2022).

Challenge and motivation: Assess the level of challenge and motivation provided by the tasks and interactions in the virtual world using subjective questionnaires and qualitative interviews (Winkler et al., 2020)

Employ validated questionnaires to assess user satisfaction, including their perceived effectiveness, efficiency, and overall satisfaction with interactive elements through realness assessments, user preference surveys, and emotional response indicators (e.g., joy, frustration) (Fagan et al., 2012).

Investigate the role of cognitive load and mental effort in mediating the relationship between interactive elements and user satisfaction.

Analyze the user’s subjective feedback on the overall intuitiveness, efficiency of menus and controls, and ease of use of the interactive user interface.

Physiological:

Correlate physiological measures, such as muscle tension and heart rate variability, with user performance and perceived satisfaction during interactions (Chen et al., 2015).

Study the impact of interaction complexity and challenge on physiological responses, such as Stress levels during challenging interactions (Bergström et al., 2016).

Cognitive load and workload assessment: Employ eye-tracking, brain imaging, or other physiological measures to assess the mental workloads required to comprehend the interactions inside the virtual world Galais et al. (2019).

3.3 Explorability

This refers to the ease and freedom of movement with which users can navigate and discover new content within the virtual environment. This aspect is influenced by factors such as degrees of freedom (DoF) for movement, locomotion techniques, and wayfinding or pathfinding options. Other factors like map design, level of complexity, and the overall layout of the virtual world also provide users with opportunities to explore and discover the virtual world.

3.3.1 Technical factors

1. Degrees of Freedom (DoF): Higher DoF supports directional and positional tracking, which allows more intuitive and realistic movement. Higher DoF and effective navigation systems contribute to a feeling of freedom, autonomy, and the ability to explore (Bowman et al., 2012). It also decreases disorientation and VR sickness (Fernandes and Feiner, 2016).

2. Spatial Resolution and Loading Times: The detail and size of the explorable environment and the time it takes to load new areas or features are essential to meeting the user’s natural desire to explore and discover the environment, which can significantly fulfill the user’s need for curiosity.

3. Navigation: Efficacy of user interface elements that aid exploration, e.g., effective wayfinding (Peck et al., 2011). This remains an unexplored avenue as far as QoE studies are concerned. It consists of wayfinding (the mental component) and travel (the motoric component) (Goldstein and Brockmole, 2016). Recent research on neural mechanisms in VR showed that the two are intimately tied together, and their coordination had a profound impact on presence, user performance, and occurrence of sickness (Taube et al., 2013). Effective navigation, techniques that improve spatial awareness and understanding, and the ease with which users can mentally map the virtual environment contribute to a more fulfilling experience.

4. Locomotion: Techniques currently used are motion-based, room-scale-based, slide-teleportation, and arm-swinging (Boletsis, 2017; Bowman et al., 1997). The possibility of continuous, unhindered exploration adds to the degree of movement perception and perceived naturalness of VR spaces (Chen et al., 2013; Slater et al., 1995). Incoherence between tracking and the displayed image induces vection and motion sickness in users (Riecke and Feuereissen, 2012).

3.3.2 Human-centric factors

1. Sense of expansiveness: Virtual worlds that allow spatial exploration and afford users free navigation of the virtual world encourage a sense of discovery. This expansiveness can trigger a profound psychological shift, temporarily liberating users from the constraints of their physical reality and inspiring feelings of awe and excitement. The feeling can be likened to encountering the unknown and unraveling something new, evoking memories of real-world adventures (Slater and Wilbur, 1997).

2. Spatial awareness and understanding: Users with a clear understanding of the virtual environment’s layout and landmarks are more likely to feel confident and motivated to explore further. Being able to form mental maps and emotional connections to specific locations, similar to how we develop attachments to real-world places, increases spatial understanding, leading to a comforting sense of belonging (Rahimi et al., 2018; Hameed and Perkis, 2018).

3. Curiosity and intrigue: An environment that encourages users to explore and seek out new things and spaces actively foment a sense of wonder and curiosity within the virtual world. Making every virtual world corner feel potentially accessible can be deeply engaging, often causing users to lose track of time as they become absorbed in their explorations (Quesnel and Riecke, 2018).

3.3.3 Assessments

Objective:

Environment rendering speed: Measure the virtual environment’s frame rate and overall rendering performance. Use benchmark tools or performance monitoring software to capture metrics (Hsu et al., 2017).

Benchmark loading times: Measure the time required to load virtual environments or map data. Use time measurement tools to capture loading times accurately. Compare loading times on different hardware platforms to assess accessibility and ensure a smooth user experience with varying tech capabilities (Lee and Oh, 2013).

Incorporating metrics related to the overall speed and responsiveness of the virtual environment, such as frame rate and object loading times. Measure level of detail (LOD) streaming efficiency for how effectively the system manages rendering detailed objects only when they are visually relevant, ensuring smooth exploration without performance dips (Heidrich et al., 2020).

Behavioral:

Navigation efficiency analysis: Assess user’s satisfaction with exploration by measuring their ease of navigating, number of areas explored, and time spent in the virtual space. Game metrics could be correlated to subjective data acquired post-experience (Taube et al., 2013; Drogemuller et al., 2018).

Task Analysis: Use task-based metrics to evaluate the efficiency of task performance using various locomotion techniques (Buttussi and Chittaro, 2019).

Analyze dwell time in different areas: Observe how long users spend in specific locations, revealing areas of high interest and potential points for experiential focus (Howie and Gilardi, 2021).

Track user’s exploration paths and patterns to identify areas of interest and areas that may pose challenges or distractions (Rummukainen et al., 2017).

Observe user’s interactions with the environment, including their use of navigational aids and their response to unexpected events or obstacles.

Non-linear exploration: Assess the flexibility of non-linear exploration by observing game-based map logs correlated to subjective questionnaires and qualitative observations.

Psychological:

Employ questionnaires to assess users’ perception of cognitive mapping and their ability to navigate and orient themselves in the virtual environment (Costa et al., 2018).

Measuring sense of agency and control: Evaluate user’s perception of freedom and control during exploration, including their ability to make decisions and influence the course of the experience (Murray et al., 2007).

Investigate the influence of spatial memory, cognitive mapping, navigation strategies, and prior experience on the user’s ability to explore effectively (Carbonell-Carrera and Saorin, 2017).

User satisfaction: Apply qualitative questionnaires and surveys to measure user satisfaction with the ease of use of navigation UI and overall exploration experience.

Incorporate emotional scales into questionnaires to assess users’ excitement, curiosity, and engagement during exploration (Felnhofer et al., 2015).

Investigate the influence of emotions on exploration preferences, such as the tendency to seek out new challenges or avoid areas that induce anxiety or discomfort.

Physiological:

Measure stress levels during challenging or disorienting exploration tasks.

Track heart rate variability during exploration tasks: Analyze changes in heart rate variability to understand users’ levels of arousal and engagement during different exploration phases (discovery, navigation, challenge) (Malińska et al., 2015).

Measure skin conductance responses to discoveries: Track changes in skin conductance to identify moments of surprise, excitement, or wonder related to exploration findings (Caldas et al., 2020).

Analyze facial expressions during exploration tasks: Use facial expression recognition to detect and categorize emotions like joy, curiosity, or frustration while exploring different areas.

3.4 Plausibility

The extent to which a VR system can logically explain and remain consistent with real-world principles. It refers to the degree to which the VR environment and its contents exhibit logical congruence, follow common sense, and align with user expectations. Plausibility operates at the syntactic level and reflects in logical consistency, adheres to real-world principles, and feels rational and explainable (Hameed and Perkis, 2024).

3.4.1 Technical factors

1. Perceptual Constancy: Consistency in object appearance despite varying environmental and contextual conditions. Some common constancies are size, shape, position, color, and lightness (Jerald, 2015; Coren et al., 2004). Disruptions, distortions, glitches, etc., can affect our perception of the object in terms of its shape and size, its position in space, its whiteness intensity, and the color of its surface. Stable geometries and optimized models maintain immersion and an overall positive experience (Lessiter et al., 2001; Schubert et al., 2001).

2. Aliasing and Sampling: It is desirable to reduce visual artifacts (such as jagged edges or pixelated textures) that disrupt the visual continuity of the experience. Lower aliasing and perceptual constancy contribute to the credibility of the virtual environment.

3. Physics consistency: For VR experiences, emulating real-world scenarios or fictional scenarios in real-world settings requires the physics engine to behave realistically regarding gravity, collisions, kinematics, and materials. Consistency of simulated physical interactions using realistic physics engines enhances authenticity (Hummel et al., 2012); uniformity in rules and logical cause-and-effect chains across the virtual environment improves the feeling of presence.

3.4.2 Human-centric factors

1. Perceived congruence: A virtual world will bring on a sense of comfort when elements within it, including objects and behaviors, behave in a manner that feels consistent, natural, predictable, and generally makes sense to the user. This congruence allows users to seamlessly apply their real-world knowledge. Congruence may carry the processing of physical and relational information reflected in matching the logic, physical behaviors, and limitations within a virtual experience to real-world principles (Skarbez, 2016; Hameed and Perkis, 2024). Incongruent features and erratic behaviors can break presence and disrupt the authenticity of the virtual world.

2. Alignment and prior knowledge: Our personal experiences and understanding of the world shape our perception of what’s plausible. When a system’s behavior aligns with user expectations and draws on their existing mental models, even if not identical to the real world, it reinforces a sense of alignment (Rauschnabel et al., 2022). Users may express surprise and delight when virtual worlds resonate with their experiences and knowledge. Concerning perceptual modalities, incongruence and mismatches may lead to a loss of spatial awareness, feel jarring, and break plausibility (Perkis et al., 2020; Rahimi et al., 2018).

3. Cognitive dissonance: Encountering unexpected or illogical elements or experiencing inconsistencies in the virtual world can create discomfort and undermine plausibility. This dissonance can create a mental struggle as users attempt to reconcile conflicting information, potentially leading to a diminished sense of presence and enjoyment in the virtual environment (Sutcliffe et al., 2019).

3.4.3 Assessments

Objective:

Object interaction consistency: Analyze if objects react realistically to various interactions (pushing, hitting, throwing) based on their physical properties and material types using quality metrics to evaluate 3D models and physics simulations, etc (Rogers et al., 2019; Sutcliffe and Gault, 2004).

Benchmark logical contradictions: Evaluate how well the system adheres to established rules and cause-and-effect relationships within the defined world logic. Examine against physical rules and check sensory alignment (Sutcliffe and Gault, 2004).

Simulation Assessments: Assess the accuracy and realism of physics simulations in the virtual environment using metrics like collision detection, object interactions, and gravity behavior to evaluate simulation accuracy (Lin et al., 2016).

Behavioral:

Analyze user adaptation to plausible vs. implausible scenarios: Observe how users adapt their behavior and decision-making to situations that align with or contradict their understanding of the world’s rules (Cavazza et al., 2007).

Track user trust in the virtual world based on interaction outcomes: Investigate how users’ trust in the world’s consistency evolves based on their experiences with interactions and consequences (Kim et al., 2017).

Affordance Testing: Track object interaction frequency and accuracy in identifying how users interact with specific objects and analyze control mechanics to identify instances of affordance mistakes (Regia-Corte et al., 2013).

Psychological:

Use questionnaires assessing realness and adherence to expectations in terms of real-world physics principles, object permanence, and predictability of interactions. Evaluate how realistically users perceive the virtual world, its logical consistency, and how well it aligns with their prior expectations and understanding of similar environments (Ogawa et al., 2019; Paes et al., 2023).

Employ self-reported measures for emotional responses to implausible or nonsensical events. Investigate how users’ emotional engagement varies depending on different levels of world plausibility and situations that break logic or violate their expectations, potentially leading to frustration, amusement, or confusion (Cavazza et al., 2007).

Assess the overall pleasantness and engagement of the virtual experience using questionnaires and surveys. Gather user feedback on their positive and negative affective responses to the features and elements within the VR environment.

Physiological:

Measure confusion-related brain activity: Utilize EEG to identify moments of cognitive confusion or disbelief triggered by implausible events or inconsistencies within the virtual world (Wan et al., 2021).

Track changes in muscle tension during unexpected occurrences: Analyze changes in muscle tension as users encounter situations that violate their expectations or understanding of the world’s logic, reflecting potential surprise or unease (Kim et al., 2021).

3.5 Believability

A user-centric aspect that refers to users’ likelihood of accepting the world on offer, whether emulated or fictional. It goes beyond mere visual realism and taps into the world-building made possible within the VR system, evoking the user’s emotions, senses, and overall engagement with the virtual world. Believability is semantic and reflects the genuineness of the depicted world in its subtle details and nuanced attention to its world-building. The extent to which a system can build such a world or deliver such an internally coherent experience would result in a “suspension of disbelief” on the user’s part to willingly accept the virtual world (Hameed and Perkis, 2024).

3.5.1 Technical factors

1. Visual Representation: Stimuli with appropriate render quality, simulation, and effects, etc. (Skarbez, 2016; Slater et al., 2022). Physically based rendering, materials, and textures that perform efficiently. Low-resolution visuals or unrealistic visual cues can break the illusion and hinder believability. Higher fidelity in asset geometry and resolution improves perceived realism (Gibson and Mirtich, 1997).

2. Audio Synchronization: When sounds’ timing and spatial location are based on virtual distance and location (Guastavino et al., 2007) of the user from the source. Accurate sound rendering enhances the aural authenticity of the experience.

3. Internal coherence and consistency: All elements within the virtual world, from physics and interactions to character behaviors and story logic, should be consistent and make sense within the established setting and rules. Similarly, narrative and stylistic cohesion should be present. Inconsistencies in cause-and-effect relationships or illogical elements can pull users out of the experience and damage believability (Lepecq et al., 2009).

4. Atmospherics and randomness: Details that reflect real-world experiences, like environmental imperfections, object interactions, nuanced reactions, and character animations, greatly enhance the feeling of naturalness within the environment (Loomis, 2016). Attention to detail within virtual worlds sparks curiosity and motivates users to seek out new things–involvement.

5. Scenario Logic: Complexity and perceived realism of scripted events or narratives in the virtual world remain logically consistent. At a high level, this refers to the extent to which a virtual world behaves reasonably or has predictability (Skarbez et al., 2020) – in that, a user anticipates what will come next (Llobera et al., 2013). The extent to which the virtual world engages the user’s reasoning, skills, and decision-making can heighten their commitment to the world. This results in cognitive absorption and the sensation of time flying by (Murray et al., 2007). The sense that actions and experiences within the virtual world have value or significance adds to their meaningfulness, especially if the users’ “expectations, attitude, and attention are aligned with the actual VR experience (Beckhaus and Lindeman, 2011). Experiences that simulate a user’s imagination also produce emotions.

3.5.2 Human-centric factors

1. Suspension of disbelief: Users’ willingness to embrace the virtual environment as “real,” despite their logical understanding that it is not. This psychological state allows users to engage more deeply with the experience, often leading to genuine emotional responses and a heightened sense of presence within the virtual world. Engaging storytelling, immersive visuals and audio, and a lack of technical glitches all suspend disbelief and enhance believability (Karhulahti, 2012).

2. Narrative Immersion and Emotional Responsiveness: When users feel emotionally invested in the characters, story, or situations within the VR world, they are more likely to believe in it and suspend their disbelief. Creating relatable characters, meaningful interactions, and engaging narratives results in users losing track of time and their physical surroundings (Ryan, 2001). It yields emotional responsiveness tapping into fundamental psychological processes that evoke authentic feelings (Rollings and Adams, 2003).

3. Social Presence: Effective social interactions often result in a genuine sense of connection with other entities, whether agents or representing real people (Arsenault, 2005). Social interactions in virtual worlds enable immersive storytelling experiences that can facilitate learning about sensitive topics in a more engaging and empathetic manner (Murray et al., 2007). The resulting sense of social presence can lead to the formation of meaningful relationships and collaborative experiences within the virtual space, enhancing the overall feeling of being part of a living, responsive world (Slater, 2018; Jerald, 2015).

4. Prior VR experiences: Users with extensive VR experience might have a more nuanced appreciation for virtual experiences, with their expectations and perceptions shaped by past encounters. Some users may possess a heightened sensitivity to both the subtleties and shortcomings of virtual environments, while others may be easily moved by the wonder and novelty of the virtual worlds (Jurnet et al., 2005). Individuals with vivid imaginations and susceptibility to suggestion are also open to fantastical elements and more accepting of realistic and fictional VR experiences (Gilbert, 2016).

3.5.3 Assessments

Objective:

Benchmark assets quality: Utilize industry standards and benchmarks to evaluate the fluidity and naturalness of assets and animations and the emotional expressiveness of character animations (Nehmé et al., 2020b; Gillies and Spanlang, 2010).

Benchmark environment quality: Measure the size and complexity of the virtual world using metrics such as the number of environments, objects, and paths to explore. Evaluate the level of dynamism and change in the virtual world using objective metrics such as the number of events, changes in the environment, and adaptation to user actions (Ragan et al., 2015).

Assess the level of detail in the virtual environments using objective metrics such as texture resolution, object density, and background complexity (Nehmé et al., 2020a; Nehmé et al., 2020b).

Benchmark environmental audio: Evaluate the use of sound effects and ambient sounds to enhance the believability of the virtual world using objective measures like sound levels and frequency spectrum to assess sound quality (Geronazzo et al., 2018).

Behavioral:

Track user choices and reactions in response to character actions: Observe how users react to and adjust their behavior based on the actions and behaviors of virtual characters (Lahiri et al., 2012).

Analyze user decisions driven by their perception of the world’s rules: Investigate how users’ understanding of the world’s logic and cause-and-effect relationships informs their choices and actions within the narrative (Badia et al., 2018).

Psychological:

Narrative Absorption and Presence: Assessing the believability of the story and narrative elements using validated instruments. Assess the coherence, consistency, and emotional impact of the story and narrative elements in the VR experience. Use questionnaires or surveys to gather user feedback on narrative and fictional immersion (Pianzola et al., 2019).

Narrative and character analysis: Analyze the story elements, character development, and overall narrative structure to assess their believability and emotional impact. Evaluate characters’ realism, personality, and emotional depth in the virtual world through user feedback and analysis of behavioral patterns (Roth and Koenitz, 2016).

Use self-report measures to analyze how users connect with, empathize with, and react emotionally to different characters and their actions. Explore how users’ emotions evolve throughout the story, culminating in their reactions to the narrative’s resolution. This can be cross-referenced against a facial expression analysis to understand the users’ emotional responses.

Physiological:

Measure startle responses to unexpected events: Track physiological changes (e.g., heart rate spikes) in response to surprising or suspenseful moments within the narrative, indicating effective use of dramatic tension (Troxler et al., 2018).

Analyze electrodermal activity (EDA) during emotional scenes: Track changes in skin conductance to understand users’ emotional responses to character interactions, plot twists, and other emotionally charged moments (Li et al., 2015).

4 Discussion

This paper has tried to identify five fundamental abstractions embodying VR experiences (see Figure 1). The proposed taxonomy provides a comprehensive framework for analyzing and evaluating the quality of VR experiences. While our taxonomy aims to be comprehensive, we recognize the importance of continuous improvement. As a living framework, it will be periodically reviewed and updated as a living framework to incorporate new findings and address emerging challenges in the VR landscape.

We provided an overview of factors influencing quality perceptions and established connections between technical system capabilities and user-centric considerations. The system-based dimensions, visual, haptic, audio, etc., are interrelated and work together to create an immersive and enjoyable VR experience. High-quality visuals with a wide FoV, sharp clarity, and accurate color reproduction enhance the sense of presence and draw users into the virtual world. Realistic interactions ground users in the VE, encourage a sense of agency, and make it seem credible. Similarly, spatialized sound effects complement the visual and haptic elements, making the experience engaging and adding to their realness. The human-centric features, usability, and enjoyment directly impact the user’s satisfaction with the VR experience. Presence, the feeling of being physically present in the virtual environment, is crucial for achieving a sense of escapism and its authenticity to sustain a prolonged engagement with the virtual world. Usability refers to the VR system’s ease of use and intuitiveness, including navigation, interaction, and application controls. Whereas enjoyment encompasses the fun factor, motivation to continue using VR, and emotional engagement with the virtual experiences.

4.1 Why differentiation matters

VR experiences are complex and multifaceted, and a single metric or criterion cannot fully capture the richness and nuance of these experiences. Differentiating between the five abstractions allows for a more granular understanding of each aspect and its contribution to the overall quality of the experience. The framework is meant to enable practical and detailed evaluations that are compatible with various implementations. The taxonomy is adaptable to a wide range of cases. For example, to understand this in terms of application, we find that entertainment-focused VR experiences like a virtual museum tour may place a high priority on immersivity, believability, and explorability to create captivating and engaging virtual worlds (Carrozzino and Bergamasco, 2010; Styliani et al., 2009). Interactivity would also be crucial to enable users to influence the narrative or gameplay actively. In contrast, VR training simulations for high-risk professions (e.g., firefighting, surgery) would likely prioritize plausibility and believability to ensure the virtual environment accurately reflects real-world conditions and consequences (Schmid Mast et al., 2018; Bergström et al., 2017). Immersivity and interactivity would also be essential for trainees to feel fully present and able to practice critical skills. Explorability may be less relevant if the simulation aims to recreate specific scenarios.

Importantly, the five dimensions in this taxonomy allude to the dyadic interplay between presence and authenticity. We consider authenticity a key factor of quality perception complementary to the feeling of presence. Authenticity is often overlooked since most conceptualizations focus on system-driven immersion and realism in the form of the fidelity and richness of the mediated environment. Equivalently, authenticity speaks to the trueness and genuineness of that mediated environment (Hameed and Perkis, 2024; Gilbert, 2016). In this context, immersivity and interactivity are more closely related to presence, as they contribute to the user’s sense of being physically present in the virtual environment. On the other hand, plausibility and believability are more closely tied to authenticity, as they ensure that the virtual world is consistent, coherent, and true to its intended purpose. Explorability lies at the intersection of presence and authenticity, as it enables users to actively engage with and discover the genuineness of the virtual world while maintaining a sense of presence.

We consider some examples of how differentiating between the five fundamental abstractions can help us generate better-quality models.

4.1.1 Believability vs. interactivity

A VR experience can be highly realistic visually but lack in perceived realism if the user cannot interact with the virtual world meaningfully. For example, a virtual setting with stunning graphics and textural detail may not remain believable if the user cannot pick up objects or interact with other characters (Hameed and Perkis, 2021).

4.1.2 Immersivity vs. plausibility

A VR experience can be fully immersive but lacks plausibility if the user encounters inconsistencies or contradictions within the virtual world. For example, a user with a high-performance HMD may feel fully immersed in a VE. Still, if the world’s physics are unrealistic or the characters behave in ways inconsistent with human behavior, the experience may be inauthentic (Rovira et al., 2009).

4.1.3 Explorability vs. plausibility

A VR experience can be highly explorable but lacks plausibility if the user cannot predict the consequences of their actions. For example, a virtual world may be large and expansive. Still, suppose the user cannot understand how the world works or cannot predict how their actions will affect the world. In that case, they may not explore it effectively and find little motivation to discover hidden elements or influence the narrative. (Irshad et al., 2021).

4.1.4 Immersivity vs. interactivity

Not all VR experiences involve high interactivity or explorability levels. 360° videos are VR experiences where you remain a passive observer, which are excellent examples of this distinction. This difference is crucial when evaluating VR experiences and justifies differentiating between purely immersive (passive or active) and interactive (minimal or high) experiences within the proposed taxonomy.

4.2 Interdependencies and potential challenges

The five abstractions proposed in the taxonomy are inherently interconnected, with potential interdependencies and trade-offs between them. For example, enhancing immersivity through highly realistic visuals and spatialized audio may come at the cost of reduced interactivity due to increased computational demands or latency issues. However, immersivity is a highly subjective experience that may be influenced by individual differences and contextual factors. Developing standardized, objective measures of immersivity that account for these variations remains challenging for evaluators. Similarly, assessing interactivity may require a combination of objective measures (e.g., tracking user inputs and response times) and subjective evaluations of the intuitiveness and naturalness of interactions. Additionally, defining thresholds or benchmarks for “high” or “low” interactivity may be context-dependent.

One can prioritize explorability by creating vast, open-ended virtual worlds, which could undermine plausibility if the virtual environment lacks coherence or consistently fails to adhere to logical rules. But at the same time, explorability introduces measurement challenges related to quantifying the extent and freedom of navigation and the ease of discovering new content. Developing metrics that capture the breadth and depth of exploration opportunities could be complex. Whereas, plausibility and believability assessments may require a deeper understanding of user expectations, mental models, and cultural backgrounds, as perceptions of logicalness and realness can vary significantly across individuals and contexts.

The interdependencies between abstractions have the potential to create synergistic effects. For instance, high levels of interactivity and explorability may contribute to heightened immersivity and a stronger sense of presence within the virtual environment. Assessment strategies must involve triangulating multiple data sources (subjective ratings, behavioral data, physiological markers) and developing standardized testing protocols or benchmark scenarios for comparative evaluations.

4.3 The benefits of a broad taxonomy

We highlighted that overall VR quality arises from the optimal configuration of the mentioned abstractions. Their prioritization will vary on the purpose of the application, i.e., entertainment, training, education, therapy, or social connection. Evaluating VR experiences, therefore, requires examining the combination of factors most suited to the experience objectives and target users. There are no absolute thresholds, only permutations aligned to aims. We hope this taxonomy will achieve the following:

1. Enhanced Evaluation Capability: The taxonomy provides a clear framework for identifying and evaluating the various factors contributing to VR experiences, allowing for a more comprehensive and objective assessment of VR systems.

2. Improved Design Guidance: By disentangling the often-conflated aspects, the taxonomy helps researchers understand specific elements that influence various quality aspects.

3. More Comparable Evaluations: The taxonomy provides a common language for discussing and evaluating VR experiences, facilitating more consistent and comparable assessments across different studies and platforms.

4. Enhanced User Understanding: The taxonomy helps researchers gain a deeper understanding of the subjective factors that influence user experiences in VR, bridging the gap between purely quantitative and user-centered studies.

In conclusion, this research proposes a comprehensive and multifaceted approach to evaluating the quality of VR experiences, considering both technical and human-centered aspects. The perceived quality of which is determined by the delicate balance between the technology’s capabilities and the user’s expectations. As VR technologies advance and mature, this framework serves as a guide for creating experiences tailored to human perception, cognition, and needs. The taxonomy presented in this research emphasizes the importance of examining the interdependencies between various factors and their cumulative impact on the user’s perception of quality. With the increasing adoption and diversification of VR applications, the ability to deconstruct experiences and identify the key drivers of positive user experiences within VR will become increasingly valuable. Our approach aims to maintain a balanced and comprehensive perspective by considering diverse viewpoints based on the accumulated knowledge in the field. We encourage future research to further validate and extend the proposed framework through empirical studies and cross-validation with other taxonomies.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Author contributions

AH: Writing–review and editing, Writing–original draft, Visualization, Methodology, Conceptualization. SM: Writing–review and editing, Supervision, Methodology. AP: Writing–review and editing, Supervision, Methodology.

Funding

The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.

Acknowledgments

This text used Grammarly® AI Writing Assistant to optimize language clarity and improve sentence structure and grammatical correctness. It was solely used to improve readability and refine overall writing mechanics. Some regular prompts used were “improve it,” “make it clearer,” “sound fluent,” “rewrite it,” “make it objective,” and “clean up notes,” amongst others. AI was not used to replace vital authoring tasks such as producing scientific, pedagogic, or research insights or drawing scientific conclusions. All generated work has been carefully reviewed to avoid output that can be incorrect, incomplete, or biased. The author is ultimately responsible and accountable for the contents of the work.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Arsenault, D. (2005). “Dark waters: spotlight on immersion,” in GAMEON-NA international conference (eurosis), 50–52.

Google Scholar

Baumgartner, T., Valko, L., Esslen, M., and Jäncke, L. (2006). Neural correlate of spatial presence in an arousing and noninteractive virtual reality: an eeg and psychophysiology study. Cyber. Psychology. Behav. 9, 30–45. doi:10.1089/cpb.2006.9.30

PubMed Abstract | CrossRef Full Text | Google Scholar

Beckhaus, S., and Lindeman, R. W. (2011). “Experiential fidelity: leveraging the mind to improve the vr experience,” in Virtual realities: dagstuhl seminar 2008 (Springer), 39–49.

CrossRef Full Text | Google Scholar

Bergström, I., Azevedo, S., Papiotis, P., Saldanha, N., and Slater, M. (2017). The plausibility of a string quartet performance in virtual reality. IEEE Trans. Vis. Comput. Graph. 23, 1352–1359. doi:10.1109/tvcg.2017.2657138

PubMed Abstract | CrossRef Full Text | Google Scholar

Bergström, I., Kilteni, K., and Slater, M. (2016). First-person perspective virtual body posture influences stress: a virtual reality body ownership study. PLoS one 11, e0148060. doi:10.1371/journal.pone.0148060

PubMed Abstract | CrossRef Full Text | Google Scholar

Boletsis, C. (2017). The new era of virtual reality locomotion: a systematic literature review of techniques and a proposed typology. Multimodal. Technol. Interact. 1, 24. doi:10.3390/mti1040024

CrossRef Full Text | Google Scholar

Bowman, D. A., and Hodges, L. F. (1999). Formalizing the design, evaluation, and application of interaction techniques for immersive virtual environments. J. Vis. Lang. and Comput. 10, 37–53. doi:10.1006/jvlc.1998.0111

CrossRef Full Text | Google Scholar

Bowman, D. A., Koller, D., and Hodges, L. F. (1997). “Travel in immersive virtual environments: an evaluation of viewpoint motion control techniques,” in Proceedings of IEEE 1997 annual international symposium on virtual reality (IEEE), 45–52.

Google Scholar

Bowman, D. A., McMahan, R. P., and Ragan, E. D. (2012). Questioning naturalism in 3d user interfaces. Commun. ACM 55, 78–88. doi:10.1145/2330667.2330687

CrossRef Full Text | Google Scholar

Bradley, M. M., and Lang, P. J. (1994). Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. psychiatry 25, 49–59. doi:10.1016/0005-7916(94)90063-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Brockmyer, J. H., Fox, C. M., Curtiss, K. A., McBroom, E., Burkhart, K. M., and Pidruzny, J. N. (2009). The development of the game engagement questionnaire: a measure of engagement in video game-playing. J. Exp. Soc. Psychol. 45, 624–634. doi:10.1016/j.jesp.2009.02.016

CrossRef Full Text | Google Scholar

Brooke, J. (1996). Sus-a quick and dirty usability scale. Usability. Eval. industry 189, 4–7. doi:10.1201/9781498710411-35

CrossRef Full Text | Google Scholar

Brunnström, K., Dima, E., Qureshi, T., Johanson, M., Andersson, M., and Sjöström, M. (2020). Latency impact on quality of experience in a virtual reality simulator for remote control of machines. Signal Process. Image Commun. 89, 116005. doi:10.1016/j.image.2020.116005

CrossRef Full Text | Google Scholar

Buttussi, F., and Chittaro, L. (2019). Locomotion in place in virtual reality: a comparative evaluation of joystick, teleport, and leaning. IEEE Trans. Vis. Comput. Graph. 27, 125–136. doi:10.1109/tvcg.2019.2928304

PubMed Abstract | CrossRef Full Text | Google Scholar

Caldas, O. I., Aviles, O. F., and Rodriguez-Guerrero, C. (2020). “A simplified method for online extraction of skin conductance features: a pilot study on an immersive virtual-reality-based motor task,” in 2020 42nd annual international conference of the IEEE engineering in medicine and biology society (EMBC) (IEEE), 3747–3750.

PubMed Abstract | CrossRef Full Text | Google Scholar

Carbonell-Carrera, C., and Saorin, J. L. (2017). Virtual learning environments to enhance spatial orientation. Eurasia J. Math. Sci. Technol. Educ. 14, 709–719. doi:10.12973/ejmste/79171

CrossRef Full Text | Google Scholar

Carrozzino, M., and Bergamasco, M. (2010). Beyond virtual museums: experiencing immersive virtual reality in real museums. J. Cult. Herit. 11, 452–458. doi:10.1016/j.culher.2010.04.001

CrossRef Full Text | Google Scholar

Cavazza, M., Lugrin, J.-L., and Buehner, M. (2007). Causal perception in virtual reality and its implications for presence factors. Presence 16, 623–642. doi:10.1162/pres.16.6.623

CrossRef Full Text | Google Scholar

Chang, E., Kim, H. T., and Yoo, B. (2020). Virtual reality sickness: a review of causes and measurements. Int. J. Human–Computer Interact. 36, 1658–1682. doi:10.1080/10447318.2020.1778351

CrossRef Full Text | Google Scholar

Chen, K. B., Ponto, K., Tredinnick, R. D., and Radwin, R. G. (2015). Virtual exertions: evoking the sense of exerting forces in virtual reality using gestures and muscle activity. Hum. factors 57, 658–673. doi:10.1177/0018720814562231

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, W., Plancoulaine, A., Férey, N., Touraine, D., Nelson, J., and Bourdot, P. (2013). “6dof navigation in virtual worlds: comparison of joystick-based and head-controlled paradigms,” in Proceedings of the 19th ACM symposium on virtual reality software and technology, 111–114.

Google Scholar

Clay, V., König, P., and Koenig, S. (2019). Eye tracking in virtual reality. J. Eye Mov. Res. 12. doi:10.16910/jemr.12.1.3

CrossRef Full Text | Google Scholar

Coren, S., Ward, L. M., and Enns, J. T. (2004). Sensation and perception. (Hoboken, NJ: John Wiley and Sons).

Google Scholar

Costa, R. Q. M. D., Pompeu, J. E., Mello, D. D. D., Moretto, E., Rodrigues, F. Z., Santos, M. D. d., et al. (2018). Two new virtual reality tasks for the assessment of spatial orientation preliminary results of tolerability, sense of presence and usability. Dementia and neuropsychologia 12, 196–204. doi:10.1590/1980-57642018dn12-020013

PubMed Abstract | CrossRef Full Text | Google Scholar

Csikszentmihalyi, M., Larson, R., et al. (2014) Flow and the foundations of positive psychology, 10. Springer.

Google Scholar

Cummings, J. J., and Bailenson, J. N. (2016). How immersive is enough? a meta-analysis of the effect of immersive technology on user presence. Media. Psychol. 19, 272–309. doi:10.1080/15213269.2015.1015740

CrossRef Full Text | Google Scholar

Cutting, J. E. (1997). How the eye measures reality and virtual reality. Behav. Res. Methods. Instrum. Comput. 29, 27–36. doi:10.3758/bf03200563

CrossRef Full Text | Google Scholar

Drogemuller, A., Cunningham, A., Walsh, J., Cordeil, M., Ross, W., and Thomas, B. (2018). “Evaluating navigation techniques for 3d graph visualizations in virtual reality,” in 2018 International symposium on big data visual and immersive analytics (BDVA) (IEEE), 1–10.

CrossRef Full Text | Google Scholar

Egan, D., Brennan, S., Barrett, J., Qiao, Y., Timmerer, C., and Murray, N. (2016). “An evaluation of heart rate and electrodermal activity as an objective qoe evaluation method for immersive virtual reality environments,” in 2016 eighth international conference on quality of multimedia experience (QoMEX) (IEEE), 1–6.

CrossRef Full Text | Google Scholar

Elbamby, M. S., Perfecto, C., Bennis, M., and Doppler, K. (2018). Toward low-latency and ultra-reliable virtual reality. IEEE Netw. 32, 78–84. doi:10.1109/mnet.2018.1700268

CrossRef Full Text | Google Scholar

Fagan, M., Kilmon, C., and Pandey, V. (2012). Exploring the adoption of a virtual reality simulation: the role of perceived ease of use, perceived usefulness and personal innovativeness. Campus-Wide Inf. Syst. 29, 117–127. doi:10.1108/10650741211212368

CrossRef Full Text | Google Scholar

Falahee, M., Latham, K., and Geelhoed, E. (2000). “Safety and comfort of eyeglass displays,” in International symposium on handheld and ubiquitous computing (Springer), 236–247.

CrossRef Full Text | Google Scholar

Felnhofer, A., Kothgassner, O. D., Schmidt, M., Heinzle, A.-K., Beutl, L., Hlavacs, H., et al. (2015). Is virtual reality emotionally arousing? investigating five emotion inducing virtual park scenarios. Int. J. Human. Computer Stud. 82, 48–56. doi:10.1016/j.ijhcs.2015.05.004

CrossRef Full Text | Google Scholar

Fernandes, A. S., and Feiner, S. K. (2016). “Combating vr sickness through subtle dynamic field-of-view modification,” in 2016 IEEE symposium on 3D user interfaces (3DUI) (IEEE), 201–210.

CrossRef Full Text | Google Scholar

Flotyński, J., Krzyszkowski, M., and Walczak, K. (2017). “Semantic composition of 3d content behavior for explorable virtual reality applications,” in Proceedings 14 virtual reality and augmented reality: 14th EuroVR international conference, EuroVR 2017, laval, France, december 12–14, 2017,(Springer), 3–23.

Google Scholar

Flotyński, J., Nowak, A., and Walczak, K. (2018). “Explorable representation of interaction in vr/ar environments,” in Augmented reality, virtual reality, and computer graphics: 5th international conference, AVR 2018, otranto, Italy, june 24–27, 2018, proceedings, Part II 5 (Springer), 589–609.

Google Scholar

Fontaine, G. (1992). The experience of a sense of presence in intercultural and international encounters. Presence. Teleoperators. Virtual. Environ. 1, 482–490. doi:10.1162/pres.1992.1.4.482

CrossRef Full Text | Google Scholar

Galais, T., Delmas, A., and Alonso, R. (2019). “Natural interaction in virtual reality: impact on the cognitive load,” in Adjunct proceedings of the 31st conference on l’Interaction homme-machine, 1–9.

Google Scholar

Geronazzo, M., Sikström, E., Kleimola, J., Avanzini, F., De Götzen, A., and Serafin, S. (2018). “The impact of an accurate vertical localization with hrtfs on short explorations of immersive virtual reality scenarios,” in 2018 IEEE international symposium on mixed and augmented reality (ISMAR) (IEEE), 90–97.

CrossRef Full Text | Google Scholar

Gibson, S. F., and Mirtich, B. (1997) “A survey of deformable modeling in computer graphics,”. Technical report, Mitsubishi Electric Research Laboratories. Cambridge, MA: Mitsubishi Electric Information Technology Center America.

Google Scholar

Gilbert, S. B. (2016). Perceived realism of virtual environments depends on authenticity. Presence 25, 322–324. doi:10.1162/pres_a_00276

CrossRef Full Text | Google Scholar

Gillies, M., and Spanlang, B. (2010). Comparing and evaluating real time character engines for virtual environments. Presence 19, 95–117. doi:10.1162/pres.19.2.95

CrossRef Full Text | Google Scholar

Gobbetti, E., and Scateni, R. (1998) “Virtual reality: past, present, and future,” in Virtual environments in clinical psychology and neuroscience: methods and techniques in advanced patient-therapist interaction.

Google Scholar

Goldstein, E. B., and Brockmole, J. (2016). Sensation and perception. Belmont, CA: Cengage Learning.

Google Scholar

Guastavino, C., Larcher, V., Catusseau, G., and Boussard, P. (2007). Spatial audio quality evaluation: comparing transaural, ambisonics and stereo. Atlanta, GA: Georgia Institute of Technology, Georgia Tech Library.

Google Scholar

Hameed, A., Irshad, S., and Perkis, A. (2019). “Towards a quality framework for immersive media experiences: a holistic approach,” in Interactive storytelling: 12th international conference on interactive digital storytelling, ICIDS 2019, little cottonwood canyon, UT, USA, november 19–22, 2019, proceedings 12 (Springer), 389–394.

CrossRef Full Text | Google Scholar

Hameed, A., Möller, S., and Perkis, A. (2023). How good are virtual hands? influences of input modality on motor tasks in virtual reality. J. Environ. Psychol. 92, 102137. doi:10.1016/j.jenvp.2023.102137

CrossRef Full Text | Google Scholar

Hameed, A., and Perkis, A. (2018). “Spatial storytelling: finding interdisciplinary immersion,” in Interactive storytelling: 11th international conference on interactive digital storytelling, ICIDS 2018, Dublin, Ireland, december 5–8, 2018, proceedings 11 (Springer), 323–332.

CrossRef Full Text | Google Scholar

Hameed, A., and Perkis, A. (2021). A subjective and behavioral assessment of affordances in virtual architectural walkthroughs. Appl. Sci. 11, 7846. doi:10.3390/app11177846

CrossRef Full Text | Google Scholar

Hameed, A., and Perkis, A. (2024). Authenticity and presence: defining perceived quality in vr experiences. Front. Psychol. 15, 1291650. doi:10.3389/fpsyg.2024.1291650

PubMed Abstract | CrossRef Full Text | Google Scholar

Harms, P. (2019). “Vr interaction modalities for the evaluation of technical device prototypes,” in Human-computer interaction–INTERACT 2019: 17th IFIP TC 13 international conference, paphos, Cyprus, september 2–6, 2019, proceedings, Part IV 17 (Springer), 416–435.

CrossRef Full Text | Google Scholar

Hart, S. G., and Staveland, L. E. (1988) “Development of NASA-TLX (task load index): results of empirical and theoretical research,” in Development of NASA-TLX (task load index): results of empirical and theoretical research, 52. Elsevier, 139–183. doi:10.1016/s0166-4115(08)62386-9

CrossRef Full Text | Google Scholar

Heidrich, D., Wohlan, A., and Schaller, M. (2020). “Perceived speed, frustration and enjoyment of interactive and passive loading scenarios in virtual reality,” in International conference on human-computer interaction (Springer), 343–355.

CrossRef Full Text | Google Scholar

Held, R., and Durlach, N. (1992). Telepresence. presence: teleoperatorsand virtual environments. Presence. (Camb). 1 (1), 109–112. doi:10.1162/pres.1992.1.1.109

CrossRef Full Text | Google Scholar

Hoffman, H. G., Richards, T., Coda, B., Richards, A., and Sharar, S. R. (2003). The illusion of presence in immersive virtual reality during an fmri brain scan. Cyberpsychology and Behav. 6, 127–131. doi:10.1089/109493103321640310

PubMed Abstract | CrossRef Full Text | Google Scholar

Holderied, H. (2017). Evaluation of interaction concepts in virtual reality applications.

Google Scholar

Howard, M. C., and Van Zandt, E. C. (2021). A meta-analysis of the virtual reality problem: unequal effects of virtual reality sickness across individual differences. Virtual Real. 25, 1221–1246. doi:10.1007/s10055-021-00524-3

CrossRef Full Text | Google Scholar

Howie, S., and Gilardi, M. (2021). Virtual observations: a software tool for contextual observation and assessment of user’s actions in virtual reality. Virtual Real. 25, 447–460. doi:10.1007/s10055-020-00463-5

CrossRef Full Text | Google Scholar

Hsu, C.-F., Chen, A., Hsu, C.-H., Huang, C.-Y., Lei, C.-L., and Chen, K.-T. (2017). “Is foveated rendering perceivable in virtual reality? exploring the efficiency and consistency of quality assessment methods,” in Proceedings of the 25th ACM international conference on multimedia, 55–63.

Google Scholar

Hummel, J., Wolff, R., Stein, T., Gerndt, A., and Kuhlen, T. (2012). “An evaluation of open source physics engines for use in virtual reality assembly simulations,” in Advances in visual computing: 8th international symposium, ISVC 2012, rethymnon, crete, Greece, july 16-18, 2012, revised selected papers, Part II 8 (Springer), 346–357.

Google Scholar

Badia, S. B., Quintero, L. V., Cameirao, M. S., Chirico, A., Triberti, S., Cipresso, P., et al. (2018). Toward emotionally adaptive virtual reality for mental health applications. IEEE J. Biomed. health Inf. 23, 1877–1887. doi:10.1109/JBHI.2018.2878846

CrossRef Full Text | Google Scholar

Ijsselsteijn, W. A., De Kort, Y. A., and Poels, K. (2013). The game experience questionnaire.

Google Scholar

Irshad, S., Perkis, A., and Azam, W. (2021). Wayfinding in virtual reality serious game: an exploratory study in the context of user perceived experiences. Appl. Sci. 11, 7822. doi:10.3390/app11177822

CrossRef Full Text | Google Scholar

Jekosch, U. (2005). Voice and speech quality perception: assessment and evaluation. Springer Science and Business Media.

Google Scholar

Jennett, C., Cox, A. L., Cairns, P., Dhoparee, S., Epps, A., Tijs, T., et al. (2008). Measuring and defining the experience of immersion in games. Int. J. human-computer Stud. 66, 641–661. doi:10.1016/j.ijhcs.2008.04.004

CrossRef Full Text | Google Scholar

Jerald, J. (2015). The VR book: human-centered design for virtual reality. Morgan and Claypool. New York, NY: ACM Books.

Google Scholar

Jurnet, I. A., Beciu, C. C., and Maldonado, J. G. (2005). “Individual differences in the sense of presence,” in Proceedings of presence 2005: the 8th international workshop on presence (College London London: University), 133–142.

Google Scholar

Kangas, J., Kumar, S. K., Mehtonen, H., Järnstedt, J., and Raisamo, R. (2022). Trade-off between task accuracy, task completion time and naturalness for direct object manipulation in virtual reality. Multimodal Technol. Interact. 6, 6. doi:10.3390/mti6010006

CrossRef Full Text | Google Scholar

Karhulahti, V.-M. (2012). “Suspending virtual disbelief: a perspective on narrative coherence,” in Interactive storytelling: 5th international conference, ICIDS 2012, san sebastián, Spain, november 12-15, 2012. Proceedings 5 (Springer), 1–17.

CrossRef Full Text | Google Scholar

Keighrey, C., Flynn, R., Murray, S., and Murray, N. (2017). “A qoe evaluation of immersive augmented and virtual reality speech and language assessment applications,” in 2017 ninth international conference on quality of multimedia experience (QoMEX) (IEEE), 1–6.

Google Scholar

Kilteni, K., Groten, R., and Slater, M. (2012). The sense of embodiment in virtual reality. Presence Teleoperators Virtual Environ. 21, 373–387. doi:10.1162/pres_a_00124

CrossRef Full Text | Google Scholar

Kilteni, K., Maselli, A., Kording, K. P., and Slater, M. (2015). Over my fake body: body ownership illusions for studying the multisensory basis of own-body perception. Front. Hum. Neurosci. 9, 141. doi:10.3389/fnhum.2015.00141

PubMed Abstract | CrossRef Full Text | Google Scholar

Kim, H., Kim, D. J., Kim, S., Chung, W. H., Park, K. A., Kim, J. D., et al. (2021). Effect of virtual reality on stress reduction and change of physiological parameters including heart rate variability in people with high stress: an open randomized crossover trial. Front. psychiatry 12, 614539. doi:10.3389/fpsyt.2021.614539

PubMed Abstract | CrossRef Full Text | Google Scholar

Kim, K., Maloney, D., Bruder, G., Bailenson, J. N., and Welch, G. F. (2017). The effects of virtual human’s spatial and behavioral coherence with physical objects on social presence in ar. Comput. Animat. Virtual Worlds 28, e1771. doi:10.1002/cav.1771

CrossRef Full Text | Google Scholar

Kim, T., and Biocca, F. (1997). Telepresence via television: two dimensions of telepresence may have different connections to memory and persuasion. J. computer-mediated Commun. 3, 0. doi:10.1111/j.1083-6101.1997.tb00073.x

CrossRef Full Text | Google Scholar

Lahiri, U., Bekele, E., Dohrmann, E., Warren, Z., and Sarkar, N. (2012). Design of a virtual reality based adaptive response technology for children with autism. IEEE Trans. Neural Syst. Rehabilitation Eng. 21, 55–64. doi:10.1109/tnsre.2012.2218618

PubMed Abstract | CrossRef Full Text | Google Scholar

Laugwitz, B., Held, T., and Schrepp, M. (2008). “Construction and evaluation of a user experience questionnaire,” in HCI and usability for education and work: 4th symposium of the workgroup human-computer interaction and usability engineering of the Austrian computer society, USAB 2008, graz, Austria, november 20-21, 2008. Proceedings 4 (Springer), 63–76.

Google Scholar

LaValle, S. (2016). Virtual reality

Google Scholar

Lee, K.-W., and Oh, M.-K. (2013). Skill of improving quality and loading speed of virtual reality software. Int. J. Multimedia. Ubiquitous Eng. 8, 179–186. doi:10.14257/ijmue.2013.8.5.17

CrossRef Full Text | Google Scholar

Lepecq, J.-C., Bringoux, L., Pergandi, J.-M., Coyle, T., and Mestre, D. (2009). Afforded actions as a behavioral assessment of physical presence in virtual environments. Virtual Real. 13, 141–151. doi:10.1007/s10055-009-0118-1

CrossRef Full Text | Google Scholar

Lessiter, J., Freeman, J., Keogh, E., and Davidoff, J. (2001). A cross-media presence questionnaire: the itc-sense of presence inventory. Presence Teleoperators and Virtual Environ. 10, 282–297. doi:10.1162/105474601300343612

CrossRef Full Text | Google Scholar

Lewis, J. R. (1991). Psychometric evaluation of an after-scenario questionnaire for computer usability studies: the asq. ACM Sigchi Bull. 23, 78–81. doi:10.1145/122672.122692

CrossRef Full Text | Google Scholar

Li, F., Lee, C.-H., Feng, S., Trappey, A., and Gilani, F. (2021). “Prospective on eye-tracking-based studies in immersive virtual reality,” in 2021 IEEE 24th international conference on computer supported cooperative work in design (CSCWD) (IEEE), 861–866.

CrossRef Full Text | Google Scholar

Li, Y., Elmaghraby, A. S., El-Baz, A., and Sokhadze, E. M. (2015). “Using physiological signal analysis to design affective vr games,” in 2015 IEEE international symposium on signal processing and information technology (ISSPIT) (IEEE), 57–62.

CrossRef Full Text | Google Scholar

Lin, J., Guo, X., Shao, J., Jiang, C., Zhu, Y., and Zhu, S.-C. (2016). “A virtual reality platform for dynamic human-scene interaction,” in SIGGRAPH ASIA 2016 virtual reality meets physical reality: modelling and simulating virtual humans and environments, 1–4.

CrossRef Full Text | Google Scholar

Liou, H.-H., Yang, S. J., Chen, S. Y., and Tarng, W. (2017). The influences of the 2d image-based augmented reality and virtual reality on student learning. J. Educ. Technol. Soc. 20, 110–121.

Google Scholar

Llobera, J., Blom, K. J., and Slater, M. (2013). Telling stories within immersive virtual environments. Leonardo 46, 471–476. doi:10.1162/leon_a_00643

CrossRef Full Text | Google Scholar

Lombard, M., Ditton, T. B., and Weinstein, L. (2009). “Measuring presence: the temple presence inventory,” in Proceedings of the 12th annual international workshop on presence, 1–15.

Google Scholar

Loomis, J. M. (2016). Presence in virtual reality and everyday life: immersion within a world of representation. PRESENCE Teleoperators and Virtual Environ. 25, 169–174. doi:10.1162/pres_a_00255

CrossRef Full Text | Google Scholar

Luciani, A., Urma, D., Marlière, S., and Chevrier, J. (2004). Presence: the sense of believability of inaccessible worlds. Comput. and Graph. 28, 509–517. doi:10.1016/j.cag.2004.04.006

CrossRef Full Text | Google Scholar

Magnenat-Thalmann, N., Kim, H., Egges, A., and Garchery, S. (2005). “Believability and interaction in virtual worlds,” in 11th international multimedia modelling conference (IEEE), 2–9.

Google Scholar

Malińska, M., Zużewicz, K., Bugajska, J., and Grabowski, A. (2015). Heart rate variability (hrv) during virtual reality immersion. Int. J. Occup. Saf. Ergonomics 21, 47–54. doi:10.1080/10803548.2015.1017964

CrossRef Full Text | Google Scholar

Masnadi, S., Pfeil, K. P., Sera-Josef, J.-V. T., and LaViola, J. J. (2021). “Field of view effect on distance perception in virtual reality,” in 2021 ieee conference on virtual reality and 3d user interfaces abstracts and workshops (vrw) (IEEE), 542–543.

CrossRef Full Text | Google Scholar

Masurovsky, A., Chojecki, P., Runde, D., Lafci, M., Przewozny, D., and Gaebler, M. (2020). Controller-free hand tracking for grab-and-place tasks in immersive virtual reality: design elements and their empirical study. Multimodal. Technol. Interact. 4, 91. doi:10.3390/mti4040091

CrossRef Full Text | Google Scholar

McMahan, A. (2013). Immersion, engagement @inproceedingsarsenault2005dark, title=Dark waters: spotlight on immersion, author=Arsenault, Dominic, booktitle=GAMEON-NA International Conference, pages=50–52, year=2005, organization=Eurosis and presence. Video. Game. Theory. Read. 67, 86. doi:10.4324/9780203700457-10

CrossRef Full Text | Google Scholar

Moller, S., Engelbrecht, K.-P., Kuhnel, C., Wechsung, I., and Weiss, B. (2009). “A taxonomy of quality of service and quality of experience of multimodal human-machine interaction,” in 2009 international workshop on quality of multimedia experience (IEEE), 7–12.

CrossRef Full Text | Google Scholar

Möller, S., Schmidt, S., and Beyer, J. (2013). “Gaming taxonomy: an overview of concepts and evaluation methods for computer gaming qoe,” in 2013 fifth international Workshop on Quality of multimedia experience (QoMEX) (IEEE), 236–241.

Google Scholar

Murphy, D., and Neff, F. (2011). “Spatial sound for computer games and virtual reality,” in Game sound technology and player interaction: Concepts and developments (IGI Global), 287–312.

Google Scholar

Murray, C. D., Fox, J., and Pettifer, S. (2007). Absorption, dissociation, locus of control and presence in virtual reality. Comput. Hum. Behav. 23, 1347–1354. doi:10.1016/j.chb.2004.12.010

CrossRef Full Text | Google Scholar

Mütterlein, J. (2018). The three pillars of virtual reality? investigating the roles of immersion. presence, interactivity. doi:10.24251/HICSS.2018.174

CrossRef Full Text | Google Scholar

Mütterlein, J., and Hess, T. (2017). Immersion, presence, interactivity: towards a joint understanding of factors influencing virtual reality acceptance and use

Google Scholar

Nehmé, Y., Dupont, F., Farrugia, J.-P., Le Callet, P., and Lavoué, G. (2020a). Visual quality of 3d meshes with diffuse colors in virtual reality: subjective and objective evaluation. IEEE Trans. Vis. Comput. Graph. 27, 2202–2219. doi:10.1109/tvcg.2020.3036153

CrossRef Full Text | Google Scholar

Nehmé, Y., Farrugia, J.-P., Dupont, F., Callet, P. L., and Lavoué, G. (2020b). Comparison of subjective methods for quality assessment of 3d graphics in virtual reality. ACM Trans. Appl. Percept. TAP 18, 1–23. doi:10.1145/3427931

CrossRef Full Text | Google Scholar

Nilsson, N. C., Nordahl, R., and Serafin, S. (2016). Immersion revisited: a review of existing definitions of immersion and their relation to different theories of presence. Hum. Technol. 12, 108–134. doi:10.17011/ht/urn.201611174652

CrossRef Full Text | Google Scholar

Ogawa, N., Narumi, T., and Hirose, M. (2019). “Virtual hand realism affects object size perception in body-based scaling,” in 2019 IEEE conference on virtual reality and 3D user interfaces (VR) (IEEE), 519–528.

CrossRef Full Text | Google Scholar

Paes, D., Irizarry, J., Billinghurst, M., and Pujoni, D. (2023). Investigating the relationship between three-dimensional perception and presence in virtual reality-reconstructed architecture. Appl. Ergon. 109, 103953. doi:10.1016/j.apergo.2022.103953

PubMed Abstract | CrossRef Full Text | Google Scholar

Pallavicini, F., Pepe, A., Ferrari, A., Garcea, G., Zanacchi, A., and Mantovani, F. (2020). What is the relationship among positive emotions, sense of presence, and ease of interaction in virtual reality systems? an on-site evaluation of a commercial virtual experience. Presence 27, 183–201. doi:10.1162/pres_a_00325

CrossRef Full Text | Google Scholar

Patel, K., Bailenson, J. N., Hack-Jung, S., Diankov, R., and Bajcsy, R. (2006). “The effects of fully immersive virtual reality on the learning of physical tasks,” in Proceedings of the 9th annual international workshop on presence, Ohio, USA, 87–94.

Google Scholar

Peck, T. C., Fuchs, H., and Whitton, M. C. (2011). The design and evaluation of a large-scale real-walking locomotion interface. IEEE Trans. Vis. Comput. Graph. 18, 1053–1067. doi:10.1109/tvcg.2011.289

PubMed Abstract | CrossRef Full Text | Google Scholar

Peck, T. C., Seinfeld, S., Aglioti, S. M., and Slater, M. (2013). Putting yourself in the skin of a black avatar reduces implicit racial bias. Conscious. cognition 22, 779–787. doi:10.1016/j.concog.2013.04.016

PubMed Abstract | CrossRef Full Text | Google Scholar

Perkis, A., Timmerer, C., Baraković, S., Husić, J. B., Bech, S., Bosse, S., et al. (2020). Qualinet white paper on definitions of immersive media experience (imex). arXiv preprint arXiv:2007.07032.

Google Scholar

Phillips, L., Interrante, V., Kaeding, M., Ries, B., and Anderson, L. (2012). Correlations between physiological response, gait, personality, and presence in immersive virtual environments. Presence 21, 119–141. doi:10.1162/pres_a_00100

CrossRef Full Text | Google Scholar

Pianzola, F., Bálint, K., and Weller, J. (2019). Virtual reality as a tool for promoting reading via enhanced narrative absorption and empathy. Sci. Study Literature 9, 163–194. doi:10.1075/ssol.19013.pia

CrossRef Full Text | Google Scholar

Quesnel, D., and Riecke, B. E. (2018). Are you awed yet? how virtual reality gives us awe and goose bumps. Front. Psychol. 9, 2158. doi:10.3389/fpsyg.2018.02158

PubMed Abstract | CrossRef Full Text | Google Scholar

Raaen, K., and Kjellmo, I. (2015). “Measuring latency in virtual reality systems,” in Entertainment computing-ICEC 2015: 14th international conference, ICEC 2015, trondheim, Norway, september 29-ocotober 2, 2015, proceedings 14 (Springer), 457–462.

CrossRef Full Text | Google Scholar

Ragan, E. D., Bowman, D. A., Kopper, R., Stinson, C., Scerbo, S., and McMahan, R. P. (2015). Effects of field of view and visual complexity on virtual reality training effectiveness for a visual scanning task. IEEE Trans. Vis. Comput. Graph. 21, 794–807. doi:10.1109/tvcg.2015.2403312

PubMed Abstract | CrossRef Full Text | Google Scholar

Rahimi, K., Banigan, C., and Ragan, E. D. (2018). Scene transitions and teleportation in virtual reality and the implications for spatial awareness and sickness. IEEE Trans. Vis. Comput. Graph. 26, 2273–2287. doi:10.1109/TVCG.2018.2884468

PubMed Abstract | CrossRef Full Text | Google Scholar

Rappa, N. A., Ledger, S., Teo, T., Wai Wong, K., Power, B., and Hilliard, B. (2022). The use of eye tracking technology to explore learning and performance within virtual reality and mixed reality settings: a scoping review. Interact. Learn. Environ. 30, 1338–1350. doi:10.1080/10494820.2019.1702560

CrossRef Full Text | Google Scholar

Rauschnabel, P. A., Felix, R., Hinsch, C., Shahab, H., and Alt, F. (2022). What is xr? towards a framework for augmented and virtual reality. Comput. Hum. Behav. 133, 107289. doi:10.1016/j.chb.2022.107289

CrossRef Full Text | Google Scholar

Rebelo, F., Noriega, P., Duarte, E., and Soares, M. (2012). Using virtual reality to assess user experience. Hum. Factors 54, 964–982. doi:10.1177/0018720812465006

PubMed Abstract | CrossRef Full Text | Google Scholar

Regia-Corte, T., Marchal, M., Cirio, G., and Lécuyer, A. (2013). Perceiving affordances in virtual reality: influence of person and environmental properties in perception of standing on virtual grounds. Virtual Real. 17, 17–28. doi:10.1007/s10055-012-0216-3

CrossRef Full Text | Google Scholar

Riecke, B. E., and Feuereissen, D. (2012). “To move or not to move: can active control and user-driven motion cueing enhance self-motion perception (vection) in virtual reality?,” in Proceedings of the ACM symposium on applied perception, 17–24.

Google Scholar

Rogers, K., Funke, J., Frommel, J., Stamm, S., and Weber, M. (2019). “Exploring interaction fidelity in virtual reality: object manipulation and whole-body movements,” in Proceedings of the 2019 CHI conference on human factors in computing systems, 1–14.

Google Scholar

Rollings, A., and Adams, E. (2003). Andrew rollings and ernest adams on game design (new riders).

Google Scholar

Roth, C., and Koenitz, H. (2016). “Evaluating the user experience of interactive digital narrative,” in Proceedings of the 1st international workshop on multimedia alternate realities, 31–36.

CrossRef Full Text | Google Scholar

Rovira, A., Swapp, D., Spanlang, B., and Slater, M. (2009). The use of virtual reality in the study of people’s responses to violent incidents. Front. Behav. Neurosci. 3, 59. doi:10.3389/neuro.08.059.2009

PubMed Abstract | CrossRef Full Text | Google Scholar

Rummukainen, O., Schlecht, S., Plinge, A., and Habets, E. A. (2017) “Evaluating binaural reproduction systems from behavioral patterns in a virtual reality—a case study with impaired binaural cues and tracking latency,” in Audio engineering society convention, 143. New York, NY: Audio Engineering Society.

Google Scholar

Rumsey, F. (2002). Spatial quality evaluation for reproduced sound: terminology, meaning, and a scene-based paradigm. J. Audio Eng. Soc. 50, 651–666.

Google Scholar

Ryan, M.-L. (2001). Narrative as virtual reality. Immersion Interactivity Literature, 357–359.

Google Scholar

Ryan, M.-L. (2015). Narrative as virtual reality 2: revisiting immersion and interactivity in literature and electronic media. Baltimore, MD: JHU, Hopkins Press.

Google Scholar

Ryan, R. M., Rigby, C. S., and Przybylski, A. (2006). The motivational pull of video games: a self-determination theory approach. Motivation. Emot. 30, 344–360. doi:10.1007/s11031-006-9051-8

CrossRef Full Text | Google Scholar

Salgado, D. P., Martins, F. R., Rodrigues, T. B., Keighrey, C., Flynn, R., Naves, E. L. M., et al. (2018). “A qoe assessment method based on eda, heart rate and eeg of a virtual reality assistive technology system,” in Proceedings of the 9th ACM multimedia systems conference, 517–520.

CrossRef Full Text | Google Scholar

Saredakis, D., Szpak, A., Birckhead, B., Keage, H. A., Rizzo, A., and Loetscher, T. (2020). Factors associated with virtual reality sickness in head-mounted displays: a systematic review and meta-analysis. Front. Hum. Neurosci. 14, 96. doi:10.3389/fnhum.2020.00096

PubMed Abstract | CrossRef Full Text | Google Scholar

Schmid Mast, M., Kleinlogel, E. P., Tur, B., and Bachmann, M. (2018). The future of interpersonal skills development: immersive virtual reality training with virtual humans. Hum. Resour. Dev. Q. 29, 125–141. doi:10.1002/hrdq.21307

CrossRef Full Text | Google Scholar

Schubert, T., Friedmann, F., and Regenbrecht, H. (2001). The experience of presence: factor analytic insights. Presence. Teleoperators. Virtual. Environ. 10, 266–281. doi:10.1162/105474601300343603

CrossRef Full Text | Google Scholar

Schwind, V., Knierim, P., Haas, N., and Henze, N. (2019). “Using presence questionnaires in virtual reality,” in Proceedings of the 2019 CHI conference on human factors in computing systems, 1–12.

CrossRef Full Text | Google Scholar

Seibert, J., and Shafer, D. M. (2018). Control mapping in virtual reality: effects on spatial presence and controller naturalness. Virtual Real. 22, 79–88. doi:10.1007/s10055-017-0316-1

CrossRef Full Text | Google Scholar

Serafin, S., Geronazzo, M., Erkut, C., Nilsson, N. C., and Nordahl, R. (2018). Sonic interactions in virtual reality: state of the art, current challenges, and future directions. IEEE Comput. Graph. Appl. 38, 31–43. doi:10.1109/mcg.2018.193142628

PubMed Abstract | CrossRef Full Text | Google Scholar

Servotte, J.-C., Goosse, M., Campbell, S. H., Dardenne, N., Pilote, B., Simoneau, I. L., et al. (2020). Virtual reality experience: immersion, sense of presence, and cybersickness. Clin. Simul. Nurs. 38, 35–43. doi:10.1016/j.ecns.2019.09.006

CrossRef Full Text | Google Scholar

Sheridan, T. B. (2016). Recollections on presence beginnings, and some challenges for augmented and virtual reality. Presence Teleoperators Virtual Environ. 25, 75–77. doi:10.1162/pres_e_00247

CrossRef Full Text | Google Scholar

Sheridan, T. B. (1992). Musings on telepresence and virtual presence. Presence Teleoperators Virtual Environ. 1, 120–126. doi:10.1162/pres.1992.1.1.120

CrossRef Full Text | Google Scholar

Sherman, W. R., and Craig, A. B. (2018). Understanding virtual reality: interface, application, and design. (San Francisco, CA: Morgan Kaufmann).

Google Scholar

Simone, L. K., Schultheis, M. T., Rebimbas, J., and Millis, S. R. (2006). Head-mounted displays for clinical virtual reality applications: pitfalls in understanding user behavior while using technology. Cyber. Psychology. Behav. 9, 591–602. doi:10.1089/cpb.2006.9.591

PubMed Abstract | CrossRef Full Text | Google Scholar

Skarbez, R. (2016). Plausibility illusion in virtual environments.

Google Scholar

Skarbez, R., Brooks, F. P., and Whitton, M. C. (2020). Immersion and coherence: research agenda and early results. IEEE Trans. Vis. Comput. Graph. 27, 3839–3850. doi:10.1109/tvcg.2020.2983701

PubMed Abstract | CrossRef Full Text | Google Scholar

Slater, M. (2002). Presence and the sixth sense. Presence 11, 435–439. doi:10.1162/105474602760204327

CrossRef Full Text | Google Scholar

Slater, M. (2009). Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical Trans. R. Soc. B Biol. Sci. 364, 3549–3557. doi:10.1098/rstb.2009.0138

PubMed Abstract | CrossRef Full Text | Google Scholar

Slater, M. (2018). Immersion and the illusion of presence in virtual reality. Br. J. Psychol. 109, 431–433. doi:10.1111/bjop.12305

PubMed Abstract | CrossRef Full Text | Google Scholar

Slater, M., Banakou, D., Beacco, A., Gallego, J., Macia-Varela, F., and Oliva, R. (2022). A separate reality: an update on place illusion and plausibility in virtual reality. Front. Virtual Real. 3, 914392. doi:10.3389/frvir.2022.914392

CrossRef Full Text | Google Scholar

Slater, M., and Sanchez-Vives, M. V. (2014). Transcending the self in immersive virtual reality. Computer 47, 24–30. doi:10.1109/mc.2014.198

CrossRef Full Text | Google Scholar

Slater, M., Usoh, M., and Steed, A. (1995). Taking steps: the influence of a walking technique on presence in virtual reality. ACM Trans. Computer. Human Interact. (TOCHI) 2, 201–219. doi:10.1145/210079.210084

CrossRef Full Text | Google Scholar

Slater, M., and Wilbur, S. (1997). A framework for immersive virtual environments (five): speculations on the role of presence in virtual environments. Presence Teleoperators and Virtual Environ. 6, 603–616. doi:10.1162/pres.1997.6.6.603

CrossRef Full Text | Google Scholar

Spors, S., Wierstorf, H., Raake, A., Melchior, F., Frank, M., and Zotter, F. (2013). Spatial sound with loudspeakers and its perception: a review of the current state. Proc. IEEE 101, 1920–1938. doi:10.1109/jproc.2013.2264784

CrossRef Full Text | Google Scholar

Steuer, J. (1992). Defining virtual reality: dimensions determining telepresence. J. Commun. 42, 73–93. doi:10.1111/j.1460-2466.1992.tb00812.x

CrossRef Full Text | Google Scholar

Styliani, S., Fotis, L., Kostas, K., and Petros, P. (2009). Virtual museums, a survey and some issues for consideration. J. Cult. Herit. 10, 520–528. doi:10.1016/j.culher.2009.03.003

CrossRef Full Text | Google Scholar

Sun, W., Gu, K., Zhai, G., Ma, S., Lin, W., and Le Calle, P. (2017). “Cviqd: subjective quality evaluation of compressed virtual reality images,” in 2017 IEEE international conference on image processing (ICIP) (IEEE), 3450–3454.

CrossRef Full Text | Google Scholar

Sutcliffe, A., and Gault, B. (2004). Heuristic evaluation of virtual reality applications. Interact. Comput. 16, 831–849. doi:10.1016/j.intcom.2004.05.001

CrossRef Full Text | Google Scholar

Sutcliffe, A. G., Poullis, C., Gregoriades, A., Katsouri, I., Tzanavari, A., and Herakleous, K. (2019). Reflecting on the design process for virtual reality applications. Int. J. Human. Computer Interact. 35, 168–179. doi:10.1080/10447318.2018.1443898

CrossRef Full Text | Google Scholar

Takatalo, J., Nyman, G., and Laaksonen, L. (2008). Components of human experience in virtual environments. Comput. Hum. Behav. 24, 1–15. doi:10.1016/j.chb.2006.11.003

CrossRef Full Text | Google Scholar

Taube, J. S., Valerio, S., and Yoder, R. M. (2013). Is navigation in virtual reality with fmri really navigation? J. cognitive Neurosci. 25, 1008–1019. doi:10.1162/jocn_a_00386

PubMed Abstract | CrossRef Full Text | Google Scholar

Troxler, M., Qurashi, S., Tjon, D., Gao, H., and Rombout, L. E. (2018). “The virtual hero: the influence of narrative on affect and presence in a vr game,” in AfCAI.

Google Scholar

Van Laer, T., Feiereisen, S., and Visconti, L. M. (2019). Storytelling in the digital era: a meta-analysis of relevant moderators of the narrative transportation effect. J. Bus. Res. 96, 135–146. doi:10.1016/j.jbusres.2018.10.053

CrossRef Full Text | Google Scholar

Velichkovsky, B. B., Gusev, A. N., Kremlev, A. E., and Grigorovich, S. S. (2017). “Cognitive control influences the sense of presence in virtual environments with different immersion levels,” in Augmented reality, virtual reality, and computer graphics: 4th international conference, AVR 2017, ugento, Italy, june 12-15, 2017, proceedings, Part I 4 (Springer), 3–16.

CrossRef Full Text | Google Scholar

Voigt-Antons, J.-N., Kojic, T., Ali, D., and Möller, S. (2020). “Influence of hand tracking as a way of interaction in virtual reality on user experience,” in 2020 twelfth international conference on quality of multimedia experience (QoMEX) (IEEE), 1–4.

CrossRef Full Text | Google Scholar

Vorderer, P., Wirth, W., Gouveia, F. R., Biocca, F., Saari, T., Jäncke, L., et al. (2004). Mec spatial presence questionnaire. Retrieved Sept.

Google Scholar

Wan, B., Wang, Q., Su, K., Dong, C., Song, W., and Pang, M. (2021). Measuring the impacts of virtual reality games on cognitive ability using eeg signals and game performance data. IEEE Access 9, 18326–18344. doi:10.1109/access.2021.3053621

CrossRef Full Text | Google Scholar

Winkler, P., Stiens, P., Rauh, N., Franke, T., and Krems, J. (2020). How latency, action modality and display modality influence the sense of agency: a virtual reality study. Virtual Real. 24, 411–422. doi:10.1007/s10055-019-00403-y

CrossRef Full Text | Google Scholar

Witmer, B. G., and Singer, M. J. (1998). Measuring presence in virtual environments: a presence questionnaire. Presence 7, 225–240. doi:10.1162/105474698565686

CrossRef Full Text | Google Scholar

Wu, W., Arefin, A., Huang, Z., Agarwal, P., Shi, S., Rivas, R., et al. (2010). “i’m the jedi!-a case study of user experience in 3d tele-immersive gaming,” in 2010 IEEE international Symposium on multimedia (IEEE), 220–227.

Google Scholar

Yang, L., Huang, J., Feng, T., Hong-An, W., and Guo-Zhong, D. (2019). Gesture interaction in virtual reality. Virtual Real. Intelligent. Hardw. 1, 84–112. doi:10.3724/sp.j.2096-5796.2018.0006

CrossRef Full Text | Google Scholar

Yin, K., He, Z., Xiong, J., Zou, J., Li, K., and Wu, S.-T. (2021). Virtual reality and augmented reality displays: advances and future perspectives. J. Phys. Photonics. 3, 022010. doi:10.1088/2515-7647/abf02e

CrossRef Full Text | Google Scholar

Keywords: virtual reality (VR), user experience, quality of experience (QoE), presence, immersion, authenticity, plausibility, quality modeling and assessment

Citation: Hameed A, Möller S and Perkis A (2024) A holistic quality taxonomy for virtual reality experiences. Front. Virtual Real. 5:1434016. doi: 10.3389/frvir.2024.1434016

Received: 16 May 2024; Accepted: 13 August 2024;
Published: 30 August 2024.

Edited by:

Mincheol Shin, Tilburg University, Netherlands

Reviewed by:

Jan de Wit, Tilburg University, Netherlands
Minjin Rheu, Loyola University Chicago, United States

Copyright © 2024 Hameed, Möller and Perkis. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Asim Hameed, asim.hameed@ntnu.no

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.