Direct comparison of virtual reality and 2D delivery on sense of presence, emotional and physiological outcome measures

Kuhne, Caroline; Kecelioglu, Eda D.; Maltby, Steven; Hood, Rebecca J.; Knott, Brendon; Ditton, Elizabeth; Walker, Frederick Rohan; Kluge, Murielle G.

doi:10.3389/frvir.2023.1211001

ORIGINAL RESEARCH article

Front. Virtual Real., 17 August 2023

Sec. Virtual Reality and Human Behaviour

Volume 4 - 2023 | https://doi.org/10.3389/frvir.2023.1211001

Direct comparison of virtual reality and 2D delivery on sense of presence, emotional and physiological outcome measures

Caroline Kuhne^1,2

Eda D. Kecelioglu^1,2

Steven Maltby^1,2

Rebecca J. Hood^2,3

Brendon Knott^1,4

Elizabeth Ditton^1,3,5

Frederick Rohan Walker^1,2^†

Murielle G. Kluge^1,2*^†

¹Centre for Advanced Training Systems, College of Health, Medicine and Wellbeing, The University of Newcastle, Callaghan, NSW, Australia
²School of Biomedical Sciences and Pharmacy, College of Health, Medicine and Wellbeing, The University of Newcastle, Callaghan, NSW, Australia
³Hunter Medical Research Institute, New Lambton Heights, NSW, Australia
⁴Contextual Interventions, Newcastle, NSW, Australia
⁵School of Medicine and Public Health, College of Health, Medicine and Wellbeing, The University of Newcastle, Callaghan, NSW, Australia

Introduction: Virtual-reality (VR) technology has, over the last decade, quickly expanded from gaming into other sectors including training, education, and wellness. One of the most popular justifications for the use of VR over 2D is increased immersion and engagement. However, very little fundamental research has been produced evaluating the comparative impact of immersive VR on the user’s cognitive, physiological, and emotional state.

Methods: A within-subject cross-over study design was used to directly compare VR and 2D screen delivery of different subject matter content. Both physiological and self-report data were collected for scenes containing calming nature environments, aggressive social confrontations, and neutral content.

Results: Compared to 2D, the VR delivery resulted in a higher sense of presence, higher ratings of engagement, fun, and privacy. Confrontational scenes were rated as more tense whilst calming scenes were rated as more relaxing when presented in VR compared to 2D. Physiological data indicated that the scenes promoted overall states of arousal and relaxation in accordance with the scene subject matter (both VR and 2D). However, heart rate (HR) and galvanic skin response (GSR) were consistently higher throughout the VR delivery condition compared to 2D, including responses during scenes of neutral and calming subject matter.

Discussion: This discrepancy between emotional and physiological responses for calming and neutral content in VR suggest an elevated arousal response driven by VR immersion that is independent of the emotional and physiological responses to the subject matter itself. These findings have important implications for those looking to develop and utilize VR technology as a training and educational tool as they provide insights into the impact of immersion on the user.

Introduction

Virtual reality (VR) is a rapidly developing technology which is expanding from a gaming context into wellness applications, training, and education across multiple sectors. VR provides an immersive experience by using a head-mounted display (HMD), containing a fully digital environment or 360 image/video content (Slater and Sanchez-Vives, 2016). Interactions with digital elements are typically facilitated via hand-held controllers or hand motions. One of the most popular arguments for the use of VR technology is its ability to generate a more immersive and engaging experience compared to a 2D computer screen (Baños et al., 2004; Freina and Ott, 2015). While this argument has further been used to hype the use of VR for educational and wellness interventions, real-world outcomes and impacts of VR applications and tools have been highly variable. This suggests there is a knowledge gap in understanding the relationship between how users experience the immersive VR and how the technology impacts learning, training, and relaxation. Little fundamental research has been undertaken to evaluate and understand how elements of immersion and engagement in VR impact the user’s experience compared to traditional or 2D delivery modalities. As a result, the justification for the use of VR over 2D applications for education, training and wellness are largely based on an untested assumption that higher levels of immersion are always beneficial for most intended use cases (Cho et al., 2002; Wiederhold and Wiederhold, 2008; Riches et al., 2021).

Real-world training outcomes and efficacy of VR applications vary greatly within the literature (Grassini et al., 2020; Wu et al., 2020; Riches et al., 2021; Arpaia et al., 2022; Ma et al., 2022). In some cases, immersive and engaging qualities of VR applications have been linked to positive educational performance, including knowledge retention. However, directly compared to their 2D or real-world counterparts, many VR training applications underperform, show no improvement in learning, or fail to translate skills into a real-world setting, despite being subjectively rated as immersive and engaging (Richards and Taylor, 2015; Jensen and Konradsen, 2018; Makransky et al., 2019; Johnson et al., 2020; López Chávez et al., 2020; Barrett et al., 2022; Houzangbe et al., 2022). Similar observations of inconsistent impacts are made for VR applications aimed at improving human wellbeing using immersive nature content to generate relaxation and recovery (Frost et al., 2022; Hejtmánek et al., 2022). In general, the most consistent benefit of immersive VR appears to be an increase in motivation and engagement rather than improvements in learning or skill acquisition (Jensen and Konradsen, 2018).

To better understand the impact of virtual worlds used outside of the gaming context, more basic research is needed to understand the interplay between subject matter, immersion, and sense of presence, and how they impact the user’s experience. VR is likely to impact learning processes as the immersive and engaging qualities of the technology itself modulate emotional and physiological states, along with the interest of the user, independent of the content and subject matter (Ai-Lim Lee et al., 2010; Antley and Slater, 2010; Cadet and Chainay, 2020). Connecting a unique level of immersion to changes in engagement, emotional and physiological states is complicated by the reliance on self-report instruments and ambiguous terminology used in this space (Berkman and Akan, 2019).

In the context of spatial awareness, the terms immersion, sense of presence, privacy and engagement, whilst inherently linked, are distinct concepts despite being commonly used interchangeably (Diemer et al., 2015). The term immersion is defined as the objective number of interactions within an environment, whilst the term “sense of presence” describes the subjective “feeling” of being within a space (Schubert et al., 2001). VR, by way of using head and arm movement, is therefore more immersive than a 2D screen and a mouse, as the objective level at which a person’s perceptual system is engaged is higher (Slater and Wilbur, 1997). Sense of presence is interpreted to be a subjective measure of a user’s experience and can therefore be seen as the subjective consequence of immersion (Kim and Biocca, 2018). Whilst higher levels of immersion do not always generate a stronger sense of presence content delivered in VR is, typically rated higher on presence than 2D delivery (Lemmens et al., 2022). Privacy is interpreted differently amongst different fields of research. Here, privacy is defined as a feeling of isolation within a public or personal space (Eghbali et al., 2019). Engagement is a term used to describe the user’s actual level of interaction with the environment, measured as the amount of head movements to observe the environment or controller clicks used during the experience, but can also be used to describe the emotional “involvement” with an environment (Huang et al., 2021). Of note, these constructs are all connected; increased options for interaction (immersion) promotes actual engagement with the environment, and if the user experiences increased levels of engagement, they will also experience an increased sense of presence, which in turn contributes to a sense of privacy (Diemer et al., 2015; Allcoat and Von Mühlenen, 2018; Huang et al., 2021). The specific distinction and terminology in this space represents a particular challenge for self-report measures, as the general public is not aware of the distinction between these terms and will consequently use them interchangeably. In particular, immersion is frequently used by the general public to describe a high sense of involvement with a specific subject matter, topic, or activity.

Very little objective data is available on how sense of presence and immersion affect the emotional and physiological state of the trainee within the virtual environment and across different subject matter content. Studies collecting physiological data including heart rate (HR), respiratory rate (RR) and galvanic skin response (GSR) focus primarily on comparing changes during VR exposure to a non-VR baseline and therefore fail to deliver a comparison to 2D. There is robust evidence to show that immersive VR is able to elicit a heightened emotional and physiological state in response to provocative content compared to baseline (Tian et al., 2021). However, these changes may be connected to the intensity of the subject matter. Directly comparing VR and 2D exposure, Lemmens et al. (2022) demonstrated an increased emotional responses (i.e., fear), coupled with a physiological response [Increased HR, and reduced heart rate variability (HRV)] during a horror survival game but not a first-person shooter game. However, there are very few studies that directly compare physiological responses in VR and different delivery modalities as outlined above. Physiological data collected during relaxing VR experiences are even less common, and the results are largely inconsistent. Physiological outcome measures (HR and skin conductance) showed no difference between VR and 2D exposure to nature videos after a stress-inducing task, despite the VR being perceived to be more calming and relaxing (Knaust et al., 2022).

The varying and inconsistent outcomes reported for different VR applications suggests a knowledge gap on how the VR specific user experience impacts the learning process or otherwise intended objectives (wellness, relaxation, focus, etc.). To address this gap, this study takes an intentional step back into basic research to explore fundamental elements of the psychophysiological users experience in VR by direct comparison to a 2D screen delivery. This comparison was chosen as the benefits and limitations of either VR and 2D screen delivery are consistently weighed up against each other, both in the literature and the commercial space. Whilst studies have evaluated individual elements of the VR user experience, very few studies provide a comprehensive comparison between VR and 2D delivery, collecting both emotional and physiological response data across different subject matter content. As such this exploratory study aimed to gain much needed fundamental insights on the interplay between subject matter, sense of presence, and the psychophysiological user’s experience by using a systematic approach to compare intentionally arousing and relaxing scenes side by side. We report both objective data on multiple physiological output measures (HR, RR, GSR) as well as self-report on sense of presence, engagement, and emotional responses. Understanding the emotional and physiological responses to VR compared to 2D delivery will inform how VR technology can be used effectively within a training, learning and wellness context to consistently address the intended goals and outcomes.

Methods

Ethics statement

Research activities were reviewed and approved by The University of Newcastle Human Research Ethics Committee [Study reference: H-2021-0177].

Study design and procedure

A within-subject crossover study design was used to compare physiological and emotional responses between two exposure conditions, 2D screen and VR delivery (Figure 1). Participants passively viewed the same content (6 scenes) on an Oculus Quest 1 headset in the VR condition and Alienware m17 R5 17.3” laptop with a computer mouse in the 2D condition. Physiological data was recorded throughout the entire study session including baseline measurements, both exposure conditions and survey response periods. Surveys were administered before and after each exposure condition. Baseline measurements were recorded for 3 min before each exposure condition whilst participants listened to an educational podcast episode (The Dirt on Houseplants. In Every Little Thing. Lichtman, F. (Host). Published 16 Nov 2020, on Gimlet Media, 0–3.30 min, 3.30–7 min). Participants’ movements were constantly recorded throughout study procedures via camera (logitech, C922 PRO HD Stream Webcam). Participants were made specifically aware of the recording and the researchers’ presence throughout the session.

FIGURE 1

FIGURE 1. Within subject crossover study design. All participants were exposed to both conditions (VR/2D) through a randomised cross over design. Surveys were completed before and after each exposure, with a final survey comparing the participants’ experience with VR/2D. Physiological biometric data was collected throughout the experiment.

Study participants and recruitment

A total of 82 healthy participants were recruited using a convenience sample from staff and students at The University of Newcastle, Australia, and volunteers from the surrounding area. Participants were screened for contraindicated conditions for the use of VR (such as motion or cyber sickness susceptibility, epilepsy, mobility impairments, claustrophobia, post-traumatic stress disorder, or un-managed anxiety and depression) prior to giving consent. One participant reported a medicated heart condition and respiratory dysfunction but was included in the data analysis. Three participants were removed from analyses due to environmental factors (session was conducted using an incorrect protocol, power tools were being used within the neighboring lab, VR malfunction) affecting the validity of their data, leaving a total of 79 participants (59 female/20 male), aged 18–52 years (mean = 24.29 ± 7.82 years) in the study.

The sequence of whether participants were first exposed to VR vs. 2D was randomly assigned. To ensure there were no systematic differences, randomization was checked for key factors including, but not limited to age, gender, and existing VR experience (yes/no). No difference between groups was found on any factor (see Supplementary Table S1).

VR and 2D intervention

A single 360-degree mp4 file was used for both delivery modalities, played using the default Oculus TV application in VR, and the Windows 10 Home Movies and TV application in 2D. The total length of each exposure condition was 8 min, with each scene being 80 s separated by a 5 s black fade. Scene order was not randomised. Sony ZX110 On-Ear Headphones were used to play sound at ∼80 decibels (corresponding to max volume in the Oculus headset) during both exposure conditions. All scenes and audio clips were taken and modified from the Performance Edge VR stress management training platform (Kluge et al., 2021).

All scenarios were passive with individual scenes contained different subject matter chosen based on their potential to deliver a neutral (1x; climbing), calming (2x; outer space, beach), or confrontational/tense (2x; pub, carpark) experience (Table 1). To acclimatise the participants to either condition or account for movement that occurs during headset setup, an acclimatising scene (1x; still 360 images of a forest with congruent sounds) was included at the beginning of each exposure.

TABLE 1

TABLE 1. Scene content and delivery in VR and 2D. Forest: Lauter Waterfall by Andreas Mischok, https://polyhaven.com/a/lauter_waterfall, licensed under CC0 1.0. Outer Space: Pro-Lighting: Skies (reprinted from Blender Guru via Blender Market, under Blender Market Standard Royalty Free License)

Measures and data collection

Self-report instruments

Self-report data was collected using specifically designed pre- and post-exposure surveys (a full list of questions can be found in Supplementary Tables S2, S3). Pre-exposure questions aimed to identify experiences, exposure, and expectations towards VR technology. Post-exposure surveys included a validated scale to collect presence and questions on the emotional responses to the content.

The I-group Presence Questionnaire (IPQ: Schubert et al., 2001) English was used to measure the subjective experience of being present in both environments after each exposure condition. The IPQ is a 14-item questionnaire made up of three subscales (Spatial Presence, Involvement, and Experienced Realism) and single items measuring general presence. All items were answered using a 7-point scale. Responses were averaged within their subscales, with higher scores representing a greater sense of presence (items Spatial Presence 2, Involvement 3, and Realism 1 were reverse scored). The IPQ English version is found to be a reliable measure with adequate psychometric properties (α > 0.68 for all subscales).

Privacy was assessed using a single item five-point Likert scale on the perceived level of privacy (0 = Not at all to 4 = very private).

Emotional responses to individual scenes were assessed after each exposure condition using a 7-point semantic differential scale, ranging from tense (−3) to relaxing (3). Participants were asked questions directly comparing both exposure conditions using a 5-point Likert scale (0 = Strongly disagree to 4 = Strongly agree) on constructs relating to presence (awareness of surrounding, enjoyment, and engagement) and emotional states. Four open-ended questions were included where participants reported the positive and negative aspects for both the VR and 2D experience.

Physiological data collection

Electrocardiogram (ECG), chest expansion and GSR were recorded using the Equivital Life Monitor Belt (EquiVital, Hidalgo, United Kingdom) and LabChart 8.0 software (ADInstruments, Australia). Chest belts were fitted and worn under participants’ clothing, positioned in line with the bottom of the pectoral muscles. MLA1010 disposable electrodes, positioned on the left palm, connected to Equivital GSR sensors were connected to the Equivital sensor electronics module (SEM).

SEM data was sent via Bluetooth connection to a computer and recorded using the LabChart software. Real-time digital filtering was applied to minimise noise and movement artefacts. Based on visual inspection 6 ECG and 5 respiratory signals were excluded from analysis. Three participants were identified as GSR non-responders and consequently excluded from analysis. GSR traces were assessed for movement artefacts and where relevant, sections were manually excluded.

Biometric data for specific scenes was removed where 1) participants reported recognising the actor within the scene (bar scene 4x, climbing 1x), 2) if there was a technology failure (forest scene 1x), or 3) if more than 60% of the trace from a scene contained artefacts (beach 1x).

Physiological measures HR (beats per min; bpm) and RR (breaths per min; BPM) were calculated via peak detection algorithm using the derivate of data points after low pass filtering and were updated after each peak detection (Supplementary Figure S1). Average HR, RR and GSR were calculated for both 3 min baseline periods and each of the 6 scenarios. Two seconds at the beginning and the end of each scene were excluded from data extraction to accommodate responses due to scene changes.

Statistical analysis

Statistical analysis was performed using JASP v0.16.3 and R version 4.2.1 and graphed using Prism v.8 (Graphpad, USA).

To assess whether randomisation to 2D delivery or VR delivery first was successful and ensuring there were no meaningful differences between the 2D- or VR-first groups, fisher and chi-square tests were used for responses with categorical dependent variables (i.e., Gender, experience with VR), and two-tailed independent t-tests for dependent variables that were continuous/could be treated as continuous (i.e., Age). Since no differences were found, results are included in the Supplementary Table S1.

Self-report data was analysed question by question. Responses to open-ended questions were qualitatively assessed by two researchers independently, followed by a discussion identifying nodes. A third researcher sorted, and coded responses into the identified nodes. Scores for the IPQ were calculated as per protocol. In depth analysis was done using Hoteling’s T2 multivariate test with the three subscales of the IPQ as dependent variables. Single item responses post VR vs. post 2D were analysed with the paired two-tailed Wilcoxon-signed rank test. Binomial tests were computed for single item questions that directly compare VR and 2D within one question (i.e., Which experience affected your emotional state more?).

All data was tested for relevant assumptions (including normality, outlier presence, homogeneity of variance, and sphericity). Outliers were removed if identified as sampling errors, otherwise outliers were considered to reflect natural variation in the data. Analysis was performed on data sets including and excluding outliers using non-parametric and parametric analysis, respectively. Unless specifically mentioned non-parametric analysis is reported in the main body of the manuscript due to normality and outlier violations, and outcome differences are reported in Supplementary Table S4.

Physiological data was analysed using paired two-tailed Wilcoxon signed-rank comparing overall physiological response across the entire experiment, and each scene type by category (i.e., tense or calm scenes) between VR and 2D delivery modalities.

Change in HR, GSR and RR for each scene was analysed using two-way repeated measures ANOVAs. In all tests the assumption of sphericity was violated, so the Greenhouse-Geisser sphericity correction was applied. Repeated Measures ANOVA analysis excludes outliers to satisfy test assumptions.

All data is presented as mean difference (M_Diff) ± SD with the addition of median difference (Mdn_Diff) where non-parametric tests are applied. p-values, 95% CI, relevant effect sizes and Bayes factors (BF) are reported. p-values <0.05 are reported as indicators of significance. However, due to the exploratory nature of this study, a focus is placed on reporting effect sizes and a BF₁₀ > 3 is considered as decisive evidence in favour of a difference or effect (See Supplementary Table S5 for a guide to interpreting Bayes Factors). Rank-Biserial correlation (r_B) is used for paired Wilcoxon-signed rank tests, Cohens’s d (d) is used for paired t-test, Omega squared (ω2) is used for RM ANOVAs and Cohen’s g (g) is used for Binomial tests.

Results

Comparison of presence, privacy, and engagement

Participants reported a higher level of sense of presence during the VR condition compared to 2D on all components of the IPQ including the single item assessing general presence [M_Diff ± SD = 2.20 ± 1.49, Mdn_Diff = 2.50, 95% CI (2.00, 3.00), r_B = 1], sub-scales for Spatial Presence [M_Diff ± SD = 2.01 ± 1.1, 95% CI (1.76, 2.26), d = 1.81], Involvement [M_Diff ± SD = 1.9 ± 1.27, 95% CI (1.62, 2.19), d = 1.50], and Realism [M_Diff ± SD = 1.43 ± 1.49, 95% CI (1.23, 1.63), d = 0.99]; all p < .001, all BF₁₀ > 100; Figure 2A).On a scale from 0 (low) to 6 (high) VR was rated above 3.5 on all subscales, whilst the 2D exposure was consistently rated below 3. Participants perceived VR to be significantly more private compared to the 2D delivery [M_Diff ± SD = 0.94 ± 1.18, Mdn_Diff: = 1.50, p < .001, 95% CI (1.00, 1.50), r_B = 0.84, BF₁₀ > 100; Figure 2B]. Further, in direct comparison questions participants reported being significantly more aware of their surroundings during the 2D experience than during the VR experience [M_Diff ± SD = 1.76 ± 1.09, Mdn_Diff: = 2.00, p < .001, 95% CI (2.00, 2.00), r_B = 0.99, BF₁₀ > 100; Figure 2C]. With respect to engagement, 91.1% of participants found the VR experience to be more fun, and 84.8% stated it to be more engaging than in 2D (agree/strongly agree; Figure 2D). Taken together, decisive evidence with high effect sizes are found in support of a difference between exposure conditions for all question items and scales used to assess perceived levels of presence, privacy, and engagement.

FIGURE 2

FIGURE 2. Higher sense of presence during VR exposure. (A) I-group presence questionnaire (IPQ) administered after each 2D and VR exposure condition, transformed to a scale ranging from 0 to 6, with negative scores from 0 to 3, and positive scores from 4 to 6. The IPQ identifies the participants’ sense of presence within each virtual environment based on three subscales: Spatial Presence, Involvement, Realism, and a single item on General Presence. (B) Perceived privacy, rated after each exposure. “Please rate the level of privacy you felt during the experience?” Response made on scale from 0 = not at all private to 4 = very private. (C) Direct comparison Likert scale question after both conditions “I was fully aware of my surroundings during the VR (/2D) experience” (0 = strongly disagree to 4 = strongly agree). (D) Direct comparison Likert scale questions after both conditions “The VR delivery was more fun than the 2D delivery “and “I felt more engaged with the delivery of scenes using a PC and 2D screen compared to VR” (Reverse scored) (0 = strongly disagree to 4 = strongly agree). N = 79, ***p < 0.001 and BF₁₀ > 100.

Quantitative outcomes identified a general preference and higher levels of engagement during the VR exposure condition. In the open-ended question regarding beneficial elements of VR, out of 79 participants, 45 (57%) mentioned feeling more involved in the VR scenes, and 44 (56%) specifically mentioned that VR was more immersive. When asked about negative elements, 20 (25%) participants reported the headset to be uncomfortable and 21 (27%) stated they did not like having a restricted field of view due to being seated and connected to biometrics sensors, which limited their ability to move their head and fully explore the 360 environments in VR. The latter theme was mirrored in the questions regarding positive elements of the 2D delivery, with many stating that the 2D was easier to navigate whilst being seated due to interactions with the mouse. 34 (43%) of participants specifically mentioned that they were aware of their surroundings and the video camera during the 2D delivery and that the delivery was boring (21 = 27%).

Representative quotes:

Negative component of VR: “My span of movement was limited due to the fact that I had to stay seated”.

Positive component of 2D: “I was able to see the whole 360 degrees aground me easily, which made it easier to interact with”.

Comparison of physiological responses

Physiological data was collected as average HR (bpm), GSR (µS) and RR (BPM) during the entire session. This included both 3 min baseline periods and both VR and 2D delivery conditions, including all scenarios within. No difference in mean baseline was recorded for any outcome measure (Figure 3A). Physiological measures are reported and analysed i) for the entire delivery condition (VR and 2D), ii) grouped into calming and tense (involving a confrontation) subject matter scenes (Figure 3B) and iii) scene-by scene, as a change from baseline (Figure 3C).

FIGURE 3

FIGURE 3. Increased Physiological responses in VR. (A) Average heart rate (HR in beats per min = bpm), galvanic skin conductance (GSR in µS) and respiratory rate (RR in breaths per min = BPM) collected during baseline period prior to VR or 2D exposure condition. Two-tailed, non-parametric Wilcoxon matched pairs signed rank test, n = 74. All data points presented. (B) Change in average HR, GSR and RR from baseline for both VR and 2D exposure condition, categorised into overall exposure duration including scene changes (n = 74), tense (average carpark and pub; n = 74) and calming (average space and beach; n = x) scenes. Two-tailed non-parametric Wilcoxon matched pairs signed rank test for each category. (C) Change in average HR, GSR and RR from baseline for both VR and 2D exposure conditions for all scenes in sequence. RM one-way ANOVA, n = 73–76 per group. Presented as mean ± SEM, p, 0.05*, p < 0.01**, p < 0.001***, outliers removed.

Heart rate (HR)

For the total duration of the exposure and all scenes, mean HR was higher in the VR condition compared to 2D [M_Diff ± SD = 1.21 ± 3.53 bpm, Mdn_Diff = 1.03 bpm, p < 0.01, 95% CI (0.33, 1.78), r_B = 0.37 BF₁₀ = 14.99; Figure 3B top panel]. Grouped into subject matter areas, decisive evidence was found to support a higher mean HR in the confrontational scenes in VR compared to 2D [M_Diff ± SD = 1.55 ± 4.09 bpm, Mdn_Diff = 1.38 bpm, p < 0.001, 95% CI (0.55, 2.30), r_B = 0.43, BF₁₀ = 24.17]. No difference in mean HR across conditions was recorded for the calming scenes.

Changes between scenes resulted in a scene-specific increase or decrease of HR, the direction which was aligned across both exposure conditions. Analysis of individual scenes showed decisive evidence in support of a main effect of exposure condition (F_1,66 = 8.605, p < 0.01, ω2 = 0.04, BF₁₀ = 9.277) and main effect of scene (F_{3.241,213.926} = 12.637, p < 0.001, ω2 = 0.05, BF₁₀ > 100). Evidence supporting an interaction between exposure condition and scene was minimal with an inconclusive Bayes factor and trivial effect size (F_{4.022,265.242} = 2.657, p = 0.03, ω2 = 0.003, BF₁₀ = 2.04). Mean HR change from baseline was significantly higher in VR compared to the 2D delivery condition for the climbing scene, which was intended to be neutral (M_Diff ± SD = 2.02 ± 4.80 bpm, p < 0.001) and both intentionally tense scenes (pub M_Diff ± SD = 1.51 ± 4.87 bpm, p < 0.01; carpark M_Diff ± SD = 1.71 ± 4.27 BPM, p < 0.01). No difference in HR can be found for the acclimatisation, forest scene, or for either of the intended calming scenes (Figure 3C, top panel).

Of note, increases in HR compared to baseline were only recorded for scenes when delivered in VR, specifically the climbing and carpark scenarios, whilst the mean HR did not exceed baseline levels throughout any of the scenes when delivered in 2D (Figure 3C top panel).

Galvanic skin response (GSR)

GSR was elevated from baseline throughout both VR and 2D delivery conditions (Figure 3C middle panel). Like HR, mean GSR was higher throughout the entire VR condition compared to 2D (M_Diff ±SD = 1.01 ± 4.33 µS, Mdn_Diff = 1.07 µS, p = 0.02, 95% CI [0.17,2.07], r_B = 0.31, BF₁₀ = 2.14). Whilst the p-value suggest a possible difference between VR and 2D delivery for the pooled tense scenes the small effect size is small (r_B = 0.28) and the BF (BF₁₀ = 0.88) was inconclusive (M_Diff ± SD = 0.92 ± 5.07 µS, Mdn_Diff = 1.03 µS, p = 0.038, 95% CI [0.07, 2.12]; Figure 3B, middle panel). No difference between VR and 2D delivery was recorded for the pooled calming scenes. Scene-to scene analysis showed that mean GSR was consistently higher during the VR exposure for all individual scenes compared to their delivery in 2D, with a possible main effect of environment (F_1,53 = 5.897, p = 0.019, ω2 = 0.05, BF₁₀ = 2.512) and evidence supporting a scene effect (F_{2.426,128.562} = 19.899, p < 0.001, ω2 = 0.09, BF₁₀ > 100; Figure 3C middle panel). No interaction between environment and scene was found (p = 0.128, BF₁₀ = 0.684). There was a difference between VR and 2D for the acclimatisation forest scene (M_Diff ± SD = 0.95 ± 2.95, p < 0.029), the climbing scene (M_Diff±SD = 1.49 ± 4.28, p < 0.001), and the Pub scene (M_Diff±SD = 1.03 ± 5.15, p = 0.019). Average GSR levels increased for the first three scenes and decreased in the later three scenes, and this trend was similar across both exposure conditions.

Respiratory rate (RR)

In contrast to the previous physiological measures, overall mean RR was lower in VR compared to 2D. Whilst the p-value (p = 0.034) suggest a possible difference between overall VR and 2D delivery the inconclusive BF of 0.62 and the small effect size (r_B = −0.29) provide little supportive evidence for this difference (M_Diff±SD = −0.59 BPM±3.01, Mdn_Diff = −0.76 BPM, 95% CI [-1.41,-0.04]; Figure 3B, bottom panel). Whilst calming scenes resulted in a reduction in RR and tense scenes increased RR, no evidence supporting a difference between VR and 2D was found when grouped (tense M_Diff±SD = −0.49 ± 3.3 BPM, Mdn_Diff = −0.66 BPM, p = 0.09, BF₁₀ = 0.38; calm M_Diff ± SD = −0.21 ± 3.59 BPM, Mdn_Diff = −0.40 BPM, p = 0.42, BF₁₀ = 0.15; Figure 3B bottom panel). Repeated measures analysis of all individual scenes showed a main effect of scene (F_{3.699,221.928} = 0.841, p = <0.001, ω2 = 0.24, BF₁₀ > 100) however no main effect of environment (p = 0.363, BF₁₀ = 0.198; Figure 3C bottom panel), and no interaction between environment and scene (p = 0.323, BF₁₀ = 0.032, Figure 3C bottom panel). A lower RR in the VR condition compared to 2D based on the p-value was only recorded for the intended neutral climbing scene (M_Diff±SD = −1.45 ± 3.41 BPM, p < 0.001).

Comparison of emotional responses

When asked to directly compare the VR and 2D exposure conditions, 72.2% (n = 57) of participants stated that their overall emotional state was more affected in VR than 2D, whilst only 5.0% (n = 4) indicated they had a stronger emotional response in 2D. 22.8% (n = 18) of participants responded with “equally”, “neither” or “unsure” (Figure 4A, left panel). When asked to compare emotional responses for intended calming and tense scenes, 69.6% (n = 55) of participants reported that the intended tense scenes were more uncomfortable in VR, and 74.7% (n = 59) reported that the intended calm scenes were more relaxing in VR (Figure 4A, middle and right panel). A significant preference to select VR as having a stronger effect on emotional responses compared to all other response options combined was reported (all p < 0.001, all g = 0.20,—0.25, all BF₁₀ > 100). This is true for ratings of the overall exposure condition (VR 95% CI [0.61, 0.82], Other 95% CI [0.18, 0.39]) and grouped into specifically tense (VR 95% CI [0.58, 0.80], Other 95% CI [0.21, 0.42]) and calming content (VR 95% CI [0.64, 0.84], Other 95% CI [0.16, 0.36]), Figure 4A).

FIGURE 4

FIGURE 4. VR exposure elicits greater emotional responses. (A) Single item questions assessing participants perceived emotional responses directly comparing 2D and VR delivery modes administered after both exposures. Question related to emotional state (left panel), perceived discomfort during tense scenes (e.g., pub and carpark; middle panel) and perceived relaxation during calming scenes (e.g., outer space and beach; right panel). (B) Individual scenes rated on semantic differential scale from tense to relaxed, after each 2D or VR exposure. N = 79, p < 0.001 and BF₁₀ > 5**, p < 0.001 and BF₁₀ > 100***.

Participants rated their emotional responses to individual scenes after each exposure condition on a scale from tense to relaxed (−3 to 3). Intentionally calming scenes (outer space and beach), as well as the acclimatisation scene (forest 360 still image), and the neutral scene (climbing gym) were rated as being relaxing (scores from 0 to 3) whilst the confrontational scenes (pub and car park) were rated as being tense (scores from -3-0) across both exposure conditions, VR and 2D (Figure 4B).

Intentionally confrontational scenes (pub and carpark) were perceived as significantly more tense (lower score) when delivered in VR compared to 2D (Pub: M_Diff±SD = −0.29 ± 0.85, Mdn_Diff = −1, p < 0.01, 95% CI [-1.00, −0.00], r_B = −0.48, BF₁₀ = 6.65; Carpark: M_Diff ± SD = −0.46 ± 0.96, Mdn_Diff = −1, p < 0.001, 95% CI [-1.00, −1.00], r_B = −0.68, BF₁₀ > 100). Both the beach and forest scene were rated as being significantly more relaxing in VR compared to 2D (Beach: M_Diff±SD = 0.39 ± 1.08, Mdn_Diff = 1, p < 0.01, 95% CI [0.00, 1.00], r_B = 0.50, BF₁₀ = 22.54; Forest: M_Diff±SD = 0.30 ± 0.94, Mdn_Diff = 1, p < 0.01, 95% CI [0.00, 1.00], r_B = 0.51, BF₁₀ = 9.49). Evidence to support these outcomes is high. No difference between VR and 2D was found for the outer space and the neutral climbing scene.

An interesting theme emerged from the open-ended responses. Whilst overall VR was rated more positively, one of the most-commonly reported negative aspects of VR was the increased ‘stress’ response to the confrontational scenes (23/79 responses), which was conversely perceived as a positive element for 2D delivery. 30 (38%) of participants stated that 2D was pleasant as it was not as emotionally intense during the confrontational scenes. 14 people indicated that a positive aspect of the VR experience was that the intended calm scenes supported relaxation.

Representative quotes:

Negative elements of VR: “During confronting scenes, I felt extremely close to the person, and felt like I could not escape the situation. Very immersed—but so immersed I felt paranoid about being in the argument.”

Positive elements of 2D: “The negative scenarios such as the bar [pub] and the carpark did not feel as real and therefore as stressful—as I was more aware of my real-life physical surroundings.”

Discussion

This study aimed to provide valuable insights on how immersion and sense of presence provided by VR affects the emotional and physiological state of users, across different subject matter content, and compared to 2D screen delivery. In the first instance and in alignment with the existing literature we confirmed that increased immersion provided by the VR headset resulted in a higher sense of presence compared to 2D (Figure 2A; [Lemmens et al., 2022]). Further, VR delivery was rated higher on all elements and constructs commonly connected to immersion, including engagement, fun, and privacy (Figures 2B–D).

The commonly used justification for the use of VR over 2D is the beneficial impact of immersion and sense of presence to support and amplify the emotional and physiological responses intended by different subject matter scenes. A general assumption is made that relaxing content would be more relaxing, and in turn, provocative content would be more tense when experienced in VR. The outcomes of this study do not support this, as VR did not enhance the relaxation response to calming content compared to the 2D delivery. Instead, mean HR and mean GSR were higher throughout the entire VR exposure compared to the 2D condition across all scenes. Our findings support the idea that higher levels of immersion and sense of presence in VR increase the user’s state of physiological arousal in addition to and irrespective of the emotional and physiological response to the subject matter itself. Further, physiological outputs did not align directly with self-reported perceived emotional effects for relaxing and neutral content, suggesting that users may not be aware of the impact a higher level of immersion or sense of presence has on their physiological state.

The physiological impact of immersion and sense of presence

VR has gained popularity for its utility in stress research, stress inoculation training and in exposure therapy for specific phobias due to its ability to elicit a controllable and consistent stress response (Rizzo et al., 2014; Notzon et al., 2015; Maples-Keller et al., 2017). Our findings support a benefit of VR over a 2D delivery for these use cases, where an elevated arousal response is intentional and desired.

The side-by-side comparison of 2D and VR for two scenes containing intentionally provocative content in the form of a verbal confrontation using 360 videos (pub and carpark) demonstrated that VR exacerbated both the emotional and physiological responses to provocative content compared to a 2D screen delivery. Strong differences were found for perceived feelings of tension, mean HR and mean GSR (Figures 3, 4). Participants specifically commented on the intensity of these scenes in VR in open-format survey responses. Importantly, our results suggest only a moderate arousal response in 2D, as HR during confrontation content never exceeded baseline levels in 2D whilst VR resulted in an elevated HR (Figure 3B, top panel). These findings add objective, physiological data to the results of a recent study which linked higher levels of presence during a simulated confrontation in VR with higher ratings of perceived anxiety compared to 2D delivery (Dickinson et al., 2021).

Whilst an elevated response to confrontational scenes in VR was expected, physiological arousal appeared to be elevated for all scenes during the VR exposure, including those of neutral and relaxing subject matter. HR and GSR signals in both delivery modalities tracked in parallel across all scenes, matching the responses to the subject matter, with consistently higher mean values during the VR exposure (Figure 3C). Higher mean HR and mean GSR were recorded not only for the intentionally tense scenes, but also the neutral and acclimatising scenes at the start of the VR exposure condition.

Based on the physiological responses it seems unlikely that immersive VR simply amplifies the emotional and physiological responses of the specific subject matter as higher levels of immersion in VR result in an increase in physiological arousal that is independent of the subject matter. For confrontational content, the arousal triggered by the VR delivery is therefore compounded to the arousal caused by the content, resulting in an overall heightened stress response. Concurrently, the inherent elevation of physiological measures in response to immersive VR, does not mean that content delivered in VR is not relaxing, but rather that the relaxation promoted by the subject matter may need to also overcome the physiological arousal triggered by heighted immersion and the observed physiological measures reflect a combination of these opposing effects.

Relaxation is a psycho-physiological response that opposes the stress response and the arousal of the central nervous system, which is associated with changes in physiological variables including the reduction of HR, RR and GSR (Benson, 1983; Teplan et al., 2014; Park et al., 2017). Two specifically calming scenes were included in this study containing elements from relaxation techniques, mindfulness, and the display of natural environments, all of which have a strong evidence base to promote relaxation (Pascoe et al., 2017; Tsutsumi et al., 2017; Song et al., 2018; Zaccaro et al., 2018; Toussaint et al., 2021). Both exposure conditions (VR and 2D) promoted a state of relaxation, measured as i) a marked reduction in HR, RR compared to baseline levels, ii) reduction in GSR compared to the previous scene (Figure 3C) as well as iii) self-reported ratings of relaxation (Figure 4B). However, when comparing exposure conditions, immersive VR did not enhance any of the physiological outcome measures compared to 2D. In fact, mean HR and mean GSR were lower during the 2D delivery and not VR, albeit not significantly (Figures 4B, C). These results are especially interesting given that VR was consistently reported to be more relaxing than the 2D delivery modality, particularly for the beach scene (Figure 3). As such, our results align with existing studies that show no improvement in objective measures of relaxation in VR despite consistent self-report of improved relaxation (Frost et al., 2022; Knaust et al., 2022).

An opposing impact of relaxing content and immersion on the nervous system provides an explanation for these seemingly conflicting observations between self-report and physiological data. One of the first comprehensive studies to specifically investigate immersion and sense of presence showed that the delivery of nature content using a “high” immersive (larger) 2D screen resulted in a greater HR recovery and reduction in skin conductance after a stress-inducing task, compared to the delivery of the same content on a “low” immersive (smaller) 2D screen (de Kort et al., 2006). Surprisingly, these findings did not extrapolate to VR, as demonstrated in a similar study comparing a VR and 2D screen delivery after stress (Knaust et al., 2022). In line with our own results, physiological outcome measures (HR and GSR) showed no difference compared to the 2D exposure. The study comparing different 2D screen sizes showed no differences in sense of presence or emotional responses across the two screen sizes which suggest no or only minimal physiological arousal in response to higher levels of immersion when using a larger 2D screen as opposed to a VR headset.

There are multiple factors that may explain an increased physiological response to VR during the relaxation scenes. Novelty of the content and unfamiliarity with the technology is likely to impact the user experience by elevating physiological responses (Ochadleus et al., 2023). Novelty to the scene content has been controlled for by using a cross-over study design, however we do propose that the novelty of the immersive experience and the scene change may be a contributing factor to the observation. As Physiological responses to a stress or startling stimulus occur more rapidly (on a scale of 1–6 s) than the slower and prolonged changes associated with relaxation and recovery (typically occurring on scales of 10 s—1 min) (Benson, 1983; Boucsein, 2012; Widjaja et al., 2013). As the individual scene exposure in this study was relatively short (90 s) a prolonged exposure within a calming VR environment may be required to overcome the initial arousal response to the scene change. Further research is warranted to investigate whether the arousal associated with VR immersion diminishes with a prolonged exposure duration. To investigate the potential impact of familiarity with VR technology a post hoc analysis was performed to compare responses of those who had indicated previous experience with VR technology and those who had not. No group differences were found on the I group presence scale, GSR and HR comparing VR and 2D differences (Supplementary Figure S2). Previous research has shown that experience levels with VR can impact responses to the technology including sense of presence (Liu et al., 2020; Maneuvrier et al., 2020). However, these studies are often conducted without a comparator and fail to discuss the observed impact of technology familiarity on responses to other technology delivered content including a 2D screen.

Cybersickness during VR is a common occurrence and has important consequences for physiological measures (Guna et al., 2019). In this study however, cybersickness appears to be unlikely the main contributor of the increased physiological response within the VR delivery condition. Whist the elevated arousal response is observed in direct comparison to the 2D delivery, absolute physiological responses are moderate in both VR and 2D for all physiological output measures (HR, GSR and RR). Further, arousal responses within the VR condition of this study reduce over time (Figure 3C). A physiological response to cybersickness would increase with prolonged exposure, not decrease as is the case here. GSR is particularly susceptible to cybersickness and did not increase from start to finish for any single participant. Further a self-report in the debrief indicated that no participant “felt” the effects of cybersickness. Whilst indirect evidence seems to suggest a negligible impact of cybersickness on the participants, no structured evaluation of cybersickness was performed and can therefore not be fully excluded as a contributing factor to the observation. This should be a consideration of future investigations.

Of the three physiological output measures assessed, HR and GSR appear to be more susceptible to the changes caused by the immersive VR delivery modality. No difference in RR was observed between VR and 2D delivery overall, despite scene-specific changes occurring in line with the intended scene content (Figure 3C, bottom panel). Although the respiratory pattern is influenced by emotional stimuli (Masaoka and Homma, 1997), RR is also the only physiological measure that is under voluntary control and might therefore not be a suitable indicator of immersion/sense of presence. In contrast, as GSR is commonly used to index anticipated and appraised environmental demands (Dawson et al., 2017), it is not surprising that this measure appears to be the most consistent measure of the physiological impact of immersion.

Implications for VR training applications

Our results are particularly relevant for VR applications and areas in which higher immersive modalities are intended to facilitate a calming environment rather than generate a state of heightened arousal. This may include specifically relaxation, mindfulness apps, but also training applications aimed to focus the user’s attention. Within a learning context it is therefore particularly important to be aware of and consider any additional physiological arousal added by the delivery modality as arousal is an important aspect in many learning theories. Memory in particular is impacted by arousal, with the view that an optimal state of arousal supports different task and learning processes (Eysenck, 1976; Malmberg et al., 2022). Whilst the exact influence of emotional factors on human learning and memory remains unclear, both physiological responses and emotional states have been documented to impact cognitive processing, including attention, memory, and reasoning (Ai-Lim Lee et al., 2010; Tyng et al., 2017). This is particularly relevant given the discrepancy between percieved emotional and physiological responses to the different delivery modalities and scene content reported in this study. Participants reported feeling calmer and more relaxed within the VR condition compared to 2D which was in strong contrast to their concomitant physiological outcomes, which showed no evidence of an enhanced relaxation response in VR. An even bigger discrepancy was reported for the “calming” acclimatisation and “neutral” climbing scenes, both of which were rated as being more relaxing than tense yet displayed a physiological profile indicating a heightened arousal state. In particular, the climbing scenario in VR resulted in a physiological stress response, measured as increased HR compared to baseline, which was higher than that of the confrontational scenes (Figure 4B). This discrepancy between perceived emotional and physiological states complicates the underlying learning theory for VR applications.

We conclude that users themselves seem to not be aware of the physiological impact that immersion and/or sense of presence has on them. This is an important finding as it may provide an explanation for why so many VR teaching tools do not improve learning outcomes, despite the perception of being engaging. The elevated physiological state may manifest subjectively as excitement, fun, and engagement while also negatively impacting attention and memory. This is supported by Cao et al. (2021), showing a heightened cortical activation during VR delivery compared to 2D monitors connected to no improvement in memory recall Cao et al. (2021). More and more studies have identified that VR can serve as a distraction, introducing unnecessary cognitive load and longer reaction times (Roettl and Terlutter, 2018; López Chávez et al., 2020).

Taken together, immersive modalities and a stronger sense of presence may not always be desirable for all teaching goals and learning processes. In line with existing learning theories and teaching frameworks, the intended learning objectives must inform the choice of delivery modality, considering the arousal impacts of the technology on the trainee.

Study limitations

Multiple factors including demographics, environment and content novelty can modulate the emotional and physiological response to VR and presence (Liu et al., 2020; Servotte et al., 2020; Lorenz et al., 2023). Whilst the cross-over study design and primary intent to compare VR and 2D delivery controls some environmental factors and content novelty, limitations remain due to the convenience sample of our population. A post hoc analysis of response differences to I-presence, differences in HR and GSR show no difference between gender, age and previous experience with VR (Supplementary Figure S2). However, it is important to note that the study population contains 74.68% females, which have shown a stronger sense of presence in VR compared to men and may therefore exacerbate our results (Grassini and Laumann, 2020). Study participants are largely in their 20s, with 50% indicating previous experience/exposure to VR technology. Having a population that contains a range of experience with the technology represents the general population and reduces the modulation of results due to technology novelty alone. To ensure stability of physiological readings and reduce artefacts, participants were required to be seated throughout the experiment, and were limited in moving their upper torso restricting the field of view in VR (unable to turn through 360-degrees). As such, the differences in sense of presence found between 2D and VR in the current study are likely to be underestimated. Restricting movement and therefore immersion due to biometric data collection or motion sickness is a common limitation found in VR research (Cao et al., 2021). Further, the room temperatures during the study commonly fell below 23 degrees, which is likely to reduce skin conductance over time. However, due to the within-subject nature of the experiment, the impact of external factors on the GSR readings is unlikely to impact the results regarding the primary intent to compare VR and 2D delivery. A final limitation of the study is related to the sequence of subject matter scenes. There was no randomization of scene order due to budget constraints. Fatigue effects and reduced arousal responses within each VR/2D sequence that may occur, particularly in later scenes, potentially reduce the observed differences between VR and 2D delivery. Secondly, alternating between tense and calming scenes complicates the comparison and interpretation of physiological responses to relaxing content, since the starting point of arousal is different between groups (VR vs. 2D). As is evident in Figure 3C the physiological changes in response to intentionally tense content in 2D was modest compared to the highest arousal response in VR. A higher level of arousal in the VR condition prior to entering a relaxing scene, which can also be seen as a recovery period, Future research could disentangle these effects by altering the scene sequence or including baseline scenes between relaxing and confrontational content. Nevertheless, this study has provided a comprehensive overview of how physiological and self-reported responses are affected by different subject matter content in VR and 2D, as was intended.

Conclusion

Taken together our findings provide valuable fundamental insights into the emotional and physiological responses to immersive VR in direct comparison to 2D across different subject matter content. Basic research investigating fundamental principles of the user experience in this space is scarce, despite the ever-growing number of VR applications and tools. An elevated physiological response to immersion, independent to the subject matter and perceived emotional responses, has important implications for existing and future VR use cases. Those looking to integrate VR technology to support education, training or relaxation need to ensure that the immersive qualities do not distract from the intended outcomes or dominate a desired physiological state.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.

Ethics statement

The studies involving humans were approved by The University of Newcastle Human Research Ethics Committee [Study reference: H-2021-0177]. The studies were conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation in this study was provided by the participants’ legal guardians/next of kin.

Author contributions

MGK and FRW contributed to conception and design of the study. CK, LD, BK, and EK were responsible for recruitment. CK, RH, and EK were involved in data collection and the organization of raw data. Data extraction and analysis was performed by CK, EK, and MGK. CK performed the statistical analysis. SM, MGK, CK, and FRW were involved in the interpretation of findings. MGK wrote the first draft of the manuscript. SM, CK, and EK wrote sections of the manuscript. All authors contributed to the article and approved the submitted version.

Acknowledgments

We would like to acknowledge the software developers at JumpGate for creating a mp4 video file of existing VR content that was used in the study.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frvir.2023.1211001/full#supplementary-material

References

Ai-Lim Lee, E., Wong, K. W., and Fung, C. C. (2010). How does desktop virtual reality enhance learning outcomes? A structural equation modeling approach. Comput. Educ. 55 (4), 1424–1442. doi:10.1016/j.compedu.2010.06.006