John Desnoyers-Stewart
Margherita Bergamo Meneghini
Ekaterina R. Stepanova
Bernhard E. Riecke- 1School of Interactive Arts and Technology, Simon Fraser University, Burnaby, BC, Canada
- 2Doctoral School of Aesthetics, Sciences and Technology of the Arts, Université Paris 8, Paris, France
- 3Compagnie Voix, Paris, France
Despite being an embodied medium, virtual reality (VR) prioritizes vision and sound over the other senses. While touch has been demonstrated to foster a sense of presence and embodiment, most haptic research in VR focuses on uncanny vibration motors or limited experiences of touch with simple props. Meanwhile, immersive performances such as Eve 3.0 incorporate performer-facilitated touch in novel ways to evoke a complete and social experience of human touch in VR. In response, we conducted a mixed-methods study to investigate the experience of performer-facilitated touch in a 360° video segment from the immersive performance Eve 3.0. Using a 3 × 2 factorial design, we compared touch from a diary prop and performer in festival and laboratory settings. We found that performer-facilitated touch increased realistic behaviours and questionnaire measures of social presence, embodiment, and tactile realism. The setting also had a significant effect with festival participants demonstrating significantly more behaviours indicating presence, particularly in the no-touch condition. Participant descriptions reveal that in addition to touch, a rich narrative and vivid visuals of social interaction were just as important in immersing participants in the experience and making them feel present. We find that participant experiences are a co-creation situated at the intersection of artefact and context that require a willing suspension of disbelief. The authentic setting and performance artefact afforded a deep understanding of the rich and complex experience of human touch in immersive performance.
1 Introduction
While virtual reality (VR) is positioned as a technology that can afford highly embodied experiences that rely on the integration of multiple senses (Gallace et al., 2012), most VR experiences focus solely on vision and sound. Touch is an important sense in the embodiment of virtual bodies (Petkova and Ehrsson, 2008; Slater et al., 2009; Kilteni et al., 2012) and their presence in virtual environments (Slater, 2009), as well as the social presence of other bodies (van Erp and Toet, 2015). While some VR systems do indeed engage the tactile sense, they typically use vibrohaptic motors and fall far short (Gallace and Girondini, 2022) of reproducing the rich experience of touch through which we ordinarily encounter the world (Field, 2014). As such, we need to explore how rich, authentic experiences of touch in VR may afford a stronger sense of embodiment and presence.
The incredible potential of synchronously stimulating touch and vision to alter bodily boundaries was first demonstrated by Botvinick and Cohen (1998). In their study, a rubber hand is positioned in front of the subject and their actual hand is hidden. The synchronous touch of the hidden physical hand along with visible touch of the rubber hand leads people to perceive the hand as part of their body. This mediation of the body through multisensory stimuli was demonstrated to work with video (IJsselsteijn et al., 2006) and applied to the body of a mannequin in video-based VR (Petkova and Ehrsson, 2008). Slater et al. (2009, 2010) further extended this, demonstrating that touch could elicit body ownership over computer-generated avatars. Body ownership is one of the components of embodiment in VR, which also includes self-location and agency (Kilteni et al., 2012). Self-location, or the sense of where oneself is located, and agency, have also been linked to synchronous visuo-tactile stimulus (Kilteni et al., 2012). Agency is distinct from body ownership and is closely related to visuomotor synchrony; however, it can even occur with an immovable rubber hand (Kilteni et al., 2012).
Body ownership and presence are closely related, and synchronous visuo-tactile stimuli may by extension elicit a sense of presence in VR (Slater et al., 2009). Spatial presence, defined as the “sense of being there” in a virtual environment, has often been the focus of research into the phenomenology of presence (Slater, 2009). However, we should also consider social presence, “the sense of being together” (Biocca et al., 2003), and object presence, “the subjective experience that a particular object exists in a user’s environment” (Stevens et al., 2002). Embodiment and spatial presence are not required for social presence (De Greef and Ijsselsteijn, 2001; Lee, 2004). However, embodiment is likely linked to social presence, particularly in media where users are represented by avatars (Biocca, 1997). While there are many ways to measure spatial presence (Souza et al., 2021) and social presence (Oh et al., 2018), few VR studies consider object presence. Yet, the incorporation of physical objects increases the perceived realism of the virtual environment (Hoffman, 1998; Zhang et al., 2022; Felip et al., 2023). For synchronous tactile stimuli to affect the sense of presence in a virtual environment, it must act through specific tactile objects perceived in the virtual environment (Fulkerson, 2013). From this, we can expect synchronous touch to have a positive impact on embodiment, as well as spatial, social, and object presence.
However, the state of the art in VR tends to lack any form of touch beyond controllers. Research on social touch in VR typically employs vibrohaptics rather than direct human contact (Gallace and Spence, 2010; van Erp and Toet, 2015). Touch, especially social touch, is a complex sense deeply rooted in gender, context, and culture (Gallace and Spence, 2010; Fulkerson, 2013; Field, 2014). Even studies that facilitate touch through objects typically use pens, wands, and knives rather than skin-to-skin human contact (e.g., Petkova and Ehrsson, 2008; Slater et al., 2009). Conclusions around the incorporation of touch into VR are thus being made based on a low fidelity expression of non-human touch (e.g., Hoppe et al., 2020; Yarosh et al., 2022). Simply visually displacing controllers in VR outperformed the typical vibrohaptic paradigm (Rietzler et al., 2018), suggesting a requirement for a significant shift in haptic research.
In contrast, there are many innovative solutions arising in immersive dance and theatre. To capture the full richness of touch in VR, immersive performances, such as The Machine to be Another (Bertrand et al., 2014), Eve: dance is an unplaceable place (Bergamo Meneghini, 2019), Draw me Closer (Wilson, 2020), and Delirious Departures (Joris and Vandebroeck, 2022), incorporate performer-facilitated touch. These aim to produce a rich experience of tactility that enables a more immersive, embodied experience and stronger sense of physical and social presence. While immersive performance leverages the extensive knowledge of artists to support a compelling embodied experience, these innovative solutions are rarely evaluated systematically. A deeper understanding of the experience of touch in these performances can guide haptic research in VR by grounding it in real, physical touch. Research into experiences of performer-facilitated physical touch can encourage much more authentic perspectives on touch that integrate their rich physical and social dimensions.
One rare example is The Machine to Be Another, an embodied VR system that allows participants to swap bodies in VR through synchronous movement and touch (Bertrand et al., 2014). Collaço de Oliveira et al. (2016) found that The Machine to Be Another significantly increased the sense of presence and body agency compared to Second Life. However, this increased presence could be attributed to anything from the use of a live camera feed to the incorporation of touch and the lack of a bodily representation in the Second Life condition. Cebolla et al. (2019) evaluated the sense of embodiment as a result of visuo-haptic synchrony in the Machine to be Another. However, the level of embodiment was compared with an arbitrary score of 1 out of 7 as being “non-embodied” rather than to a specific control condition. Neither study investigates what elements might have contributed to the presence or embodiment measured.
Given that much of the innovation in performer-facilitated touch occurs in public performance spaces, there is particular value in exploring them in their authentic context. In the laboratory, the experience may become substantially different (Rogers and Marshall, 2017). While there are numerous investigations of performances in situ at festivals, these tend to focus on individual reflexive accounts (e.g., Popat, 2016; Wilson, 2020) or qualitative studies of the complete experience of a limited number of participants (e.g., Rostami et al., 2018; Jégo and Bergamo Meneghini, 2020). However, systematic and quantitative studies are rare. To better understand the real-world effects and generalizability of in-laboratory findings, research is required that investigates performances in an ecologically valid festival context and compares them to a lab setting.
1.1 Our research
To address this research gap and facilitate knowledge transfer between art and science, we investigated how a physical prop and performer-facilitated touch affect the experience of immersive performance with a controlled study across laboratory and festival settings. We use a segment from Eve 3.0,1 a multi-user VR performance where a performer facilitates moments of touch, as seen in Figure 1. The performer synchronizes their performance on stage with the choreography executed by characters seen in VR, touching participants at key moments and encouraging them to dance and become performers themselves. Participants also pick up a physical diary prop placed next to them when they see their virtual body reach for the diary. After hearing an excerpt narrated from the diary, it is then ripped from their hands in sync with seeing this action in VR. Later, the diary is thrown towards the participant in the 360° video but not physically. When exhibiting this performance, we often noticed that participants tried to catch the diary at that moment, something that we would not ordinarily expect to see with a 360° video. We hypothesized that added tactility may have invoked a higher level of presence that was expressed through this realistic behaviour (Slater, 2009). This behaviour was particularly interesting as it appeared to be a reflex that the participants did without thinking, often followed by laughter at their realization of what had happened.
FIGURE 1. Eve 3.0 as seen during a performance. The performer kneels in sync with the virtual character. Following the music, she touches the immersant’s hand in time with the visuals.
1.2 Objectives and research questions
Inspired by this observation, we designed a convergent mixed-methods study (Creswell and Creswell, 2018) comparing three versions of the experience—A (all/full touch): the full performer-facilitated experience, B (book/diary only): using only the physical diary prop, and C (control/no touch): a typical audio-visual 360° video experience without touch. To understand the effects of performer-facilitated touch, we investigate the following research questions:
1.2.1 Quantitative research questions
How does the incorporation of a physical prop and performer-facilitated touch influence:
• the alignment of participants’ behaviour with the VR experience?
• the sense of presence (social, object, and spatial)?
• the sense of embodiment?
• participants’ affective state (valence and arousal)?
• the sense of connection to the virtual character?
We analysed behaviours indicative of embodiment and presence, such as moving hands along with the body in VR and attempting to catch the diary. Through validated questionnaires, we measured self-reported affective valence and arousal, embodiment, presence, immersion, and the participants’ sense of connection to the main character.
1.2.1.1 Hypotheses
We anticipated that the performer-facilitated full-touch condition (A) would support a stronger sense of embodiment and presence, a stronger emotional response, and a deeper sense of connection to the VR character than the no-touch control condition (C). Similarly, we expected more people to attempt to catch the diary and move along with the visuals seen in the headset in the full-touch condition. As for the diary-only condition (B), we anticipated all variables to fall somewhere between A and C. We also anticipated the festival setting would affect our results compared to the laboratory setting due to differences in demographics, disposition, and context. Our hypotheses were as follows:
• H1: Setting (1.1) along with physical touch of a prop alone (1.2) and prop and performer (1.3), each increases the probability that participants will engage in behaviour that reflects presence in the virtual experience, such as trying to catch a diary thrown in the 360° video.
• H2: Setting (2.1) along with physical touch from a prop alone (2.2) and prop and performer (2.3), each increases spatial, social, and object presence, immersion, tactile realism, embodiment, social connection, and valence and arousal.
1.2.2 Qualitative research questions
We also asked open-ended questions to gain a deeper qualitative understanding of the rich complexity of participant experiences. Our qualitative research questions were
• What aspects of the experience do participants describe as contributing to their experience of touch, embodiment, and presence in Eve 3.0?
• Why do participants try to catch the diary, or not, when it is thrown towards them in Eve 3.0?
1.2.3 Mixed research question
The qualitative findings are used to explain the quantitative results, connecting them to the behavioural and questionnaire measures to develop a holistic understanding and triangulate the data:
• How do participants’ descriptions of their experience align and explain the behavioural and questionnaire outcomes?
2 Materials and methods
2.1 Artefact: Eve 3.0 performance 360° video segment “Dear Diary”
The full version of Eve 3.0 is performed in front of an audience where six participants are brought on stage and each sees a different story in VR. Each story follows a character as they detail their struggle with anxiety, addiction, depression, paranoia, obsession, and jealousy. Participants are given a VR headset and asked to align their bodies with the one seen in a stereoscopic 360° video. Throughout the 360° video, the performer makes contact with the participants in sync with the visuals, encouraging them to become immersed and to move and dance. The full performance continues with a motion-captured dance, with each dance reflecting movements inspired by each character’s struggles, and ends with an open, improvised dance encouraged through lively pass-through visuals.
Rather than being designed specifically for this study, we take an unmodified segment of a complete artwork, created “for art’s sake.” The performance has been minimally adjusted to stay true to the authentic artefact while enabling quantitative analysis. We focused on the 360° video as it offered an opportunity to investigate the behaviours regularly seen in public performances. It also provided an easily repeatable segment that could be run under different conditions simultaneously. The video segment is 9 min long. Moments of touch are synchronized with a subtle audio cue in the soundtrack allowing the researchers to consistently facilitate the moments of contact.
We specifically used the 360° video featuring Désirée, who struggles with addiction, as this story had the most consistently relatable narrative and best audio recording quality. The narrative is presented in English for this study.
2.1.1 Narrative and key moments of touch in “Dear Diary”
In the 360° video, participants embody the character Eve. This performance includes not only the physical moments of touch but also invites participants to participate in a rich narrative, as seen in Figure 2.
FIGURE 2. Narrative structure and moments of touch in the 360° video. Only 2 and 3 are enacted in the diary-only condition, while none are in the no-touch condition. No actual diary is ever thrown in 7.
2.1.2 Hardware
Due to hardware availability, different headsets were used. Three Oculus Go headsets were used in the festival setting with Showtime VR2 to synchronize the videos. Two Meta Quest 2s and one Meta Quest 1 were used in the laboratory setting, synchronized using custom Eve 3.0 software. The Quest 1 was alternated between conditions each session to avoid affecting the results for any one condition. A Meta Quest 2 was used for all convenience samples. All headsets were used to view the same three degrees of freedom, stereoscopic, 360° video, minimizing any differences in the hardware. While the resolutions and refresh rates differ between the headsets, the limiting factor was the video quality (4096 × 4096 @ 60 fps) which, at a 90° field of view (FOV), produces a 1024 × 1024 stereoscopic image.
2.2 Participants
This study was approved by the Simon Fraser University Research Ethics Board (#30001617) and the Université Paris 8 Ethics Committee (#CE-P8-2023-06-3). Participants gave their informed consent and were allowed to withdraw from the study until their data were de-identified. Participants could ask questions about the research and performance after completing the study and sign up to receive the study results.
2.2.1 Recruitment
By conducting our research in both a laboratory and festival setting, we aimed to improve ecological validity, recruit a more diverse sample, and investigate whether laboratory studies are representative of a real-world application. 108 participants were recruited through three methods: 1) through a performance at the 2023 Cinedans festival in Amsterdam3 (festival, n = 50); 2) through Simon Fraser University’s School of Interactive Arts Research Participation System in Surrey, BC, Canada (lab, n = 47); and 3) through convenience sampling (convenience, n = 11). Convenience sampled participants were recruited in a wide variety of locations during visits to labs and universities to ensure a more representative sample (Calgary, AB, Canada, n = 4; Portland, OR, United States, n = 1; Vienna, Austria, n = 2; Santiago de Compostela, Spain, n = 3; Vancouver, BC, Canada, n = 1). Laboratory and convenience participants were combined for analysis due to similarities in context and the small number of convenience participants.
2.2.2 Exclusions
Seven additional participants were excluded from the study. Three participants in the festival setting did not complete the optional survey and provide consent. Three convenience participants were excluded due to asynchronous physical touch. One laboratory participant was excluded because they removed the headset during critical moments of the experience.
2.2.3 Demographics
Seventy-two (66.7%) participants were identified as women, 35 (32.4%) as men, and one (0.926%) as gender-fluid. Furthermore, 70.7% of laboratory participants identified as women compared to 62.0% of festival participants. The participants were between 18 and 74 years of age and had a wide range of VR experience, from none (n = 16) to some with over 100 times using VR (n = 5) and 360° video (n = 2). Participants had a wide range of movement practice experience, from 0 to 40 years (mean = 6.72 years). Their current movement practice frequency varied from none (n = 30) to daily (n = 9). Movement practice was defined in the questionnaire to include informal and professional practices such as dance, tai chi, capoeira, or yoga but left open to interpretation by participants; for more details, see Figure 3.
FIGURE 3. Histograms of participant demographics.
2.3 Experimental design and procedure
The study follows a 2 × 3 factorial between-subjects design. The independent variables are different levels of physical touch (A: full touch, B: diary only, and C: no touch), and setting (festival and laboratory). The overall structure is shown in Figure 4A. Each session lasted 30 min that included setup, the 9-min 360° video, and 15 min of questionnaires.
FIGURE 4. Study design and settings showing the layout where participants were seated. The conditions were routinely seated in the same location from left to right: full touch, diary-only, and no touch. (A) Overview of study design showing when each measure was collected. (B) Festival setting at Cinedans 2023 in Amsterdam, Netherlands. The facilitator helps the full touch participant to stand up. Festival-goers can be seen lounging at a cafe in the background. (C) Lab setting at Simon Fraser University in Surrey, Canada. Participants are lined up at the beginning of the study in the black box laboratory. The diaries have not been placed yet.
2.3.1 Settings
Participants in the festival setting attended Cinedans in Amsterdam alongside other performances, films, and VR experiences. The experience was included in the festival guide and online, and participants could walk up to participate in the experience. They could try the experience without participating in the study. The context can be seen in Figure 4B. In the laboratory setting, participants signed up in advance and came to a black box studio laboratory at a scheduled time. The laboratory setting can be seen in Figure 4C. Convenience participants primarily engaged with the experience in university lab similar to the one seen here.
2.3.2 Condition assignment and pre-experience questionnaire
Upon entering the lab or visiting the festival booth, participants were assigned a condition and participant number. We sought to stratify participants across conditions by assigning them sequentially to the next condition pseudo-randomly based on different alternating factors such as positioning from left to right or right to left to reduce possible bias in selecting the condition. When fewer than three participants were present, we alternated conditions to maintain a balanced number of participants in each condition. Participants were given their participant number and asked to complete a digital consent form and demographic survey on their smartphone or a researcher-provided laptop.
2.3.3 Conditions
Upon completing the demographic survey, participants were asked to sit in one of three chairs set up in a line to allow the researcher to facilitate multiple experiences simultaneously. The same chair position was used for each condition to ensure consistent video analysis, as shown in Figure 4. Similar participant numbers were run through each condition in each setting and overall: full touch (A) = 35 (festival = 18, lab = 17); diary only (B) = 38 (festival = 17, lab = 21); no touch (C) = 35 (festival = 15, lab 20). Discrepancies in participant numbers primarily arose due to exclusions.
Each participant was asked to sit in a chair with a low table or chair to their right. Once all participants were comfortably seated and ready with their headset on, the facilitator started the experience. The facilitator asked participants to line themselves up with what they saw to ensure their body was aligned. The exact diary seen in the 360° video was then carefully placed aligned with its visual location next to participants in the diary-only (B) and full-touch (A) conditions. The control condition (C) was provided with no prop, of no touch, and simply watched the 360° video. In the diary-only condition (B), only a physical diary was used. In the full-touch condition (A), a diary was provided, and the facilitator touched the participants in sync with the characters in the 360° video to produce the complete experience as designed.
Participants viewed the 360° video segment simultaneously allowing for consistent facilitation video analysis using the music to identify the moment in the experience. The facilitator performed six moments of touch shown in Figure 2. In the diary-only condition, the diary was placed next to the participant and then removed from their hands at the right moment during the experience. When a participant did not reach over to pick up the diary or struggled to find it, the facilitator brought it closer or placed it in their hands.
2.3.4 Video recording and behavioural measures
The entire experience was video-recorded for analysis of any significant behaviours. Facilitators also recorded in a notebook whether participants attempted to catch the diary and any interesting behaviours to include in the analysis. Such behaviours that align with what is experienced in VR are indicative of presence (Slater, 2009) and object presence (Reiner and Hecht, 2009).
2.3.5 Post-experience questionnaire
After completing the VR experience, participants’ headsets were removed. Participants were asked not to remark on the experience or ask questions until they completed the survey. The survey was conducted in a public festival setting, so emphasis was placed on keeping the survey concise for reliable completion. With a 93.9% completion rate, we struck a good balance. The full list of questions is shown in Table 1 and included in the Supplementary Material.
TABLE 1. Questions from the post-experience questionnaire.
A continuous Affect Grid (Russell et al., 1989) was used to measure valence and arousal upon completing the experience. While there are many different questionnaires for measuring presence (Souza et al., 2021), we chose the Temple Presence Inventory (TPI) for its adaptability and inclusion of social presence and immersion (Lombard et al., 2009). Select items from the TPI (Lombard, 2013) were used to measure spatial and social presence (active social and parasocial dimensions), as well as immersion and perceptual realism. Items were selected to fit the research questions of the study and ensure they were sensible, given the constraints of the 360° video. For example, we did not ask “How often did it feel as if someone you saw/heard in the environment was talking directly to you?” because there is only internal dialogue. We excluded the social richness and social realism dimensions because the narrative was relatively dreamlike and the experience was not interactive.
We used a single-item embodiment question based on Peck and Gonzalez-Franco (2021) and a single-item object presence measure based on Schubert et al. (2001). Similar to Stevens et al. (2002), we rephrased the question to refer to the object rather than environment presence. For each question we asked participants to refer to a specific instance for consistency. Participants then identified their connection to the main character using the Inclusion of Other in Self (IOS) scale (Aron et al., 1992).
Finally, participants filled out questions about technical and touch quality before answering four open-ended questions. Namely, the open-ended questions inquired about participants’ memory and reasoning for trying to catch the diary; the most significant moment; and their experience of the diary being thrown towards them. All Likert-scale questions are on a 7-point rating scale, except the quality questions which were out of 5.
2.4 Analysis
JMP Statistical Discovery Software4 was used to analyse quantitative data. Behaviour coding was combined with questionnaire results by the participant number in Microsoft Excel and imported into JMP. Qualitative analysis was conducted using NVivo.5
2.4.1 Behaviour coding and analysis
Each researcher recorded perceived attempts to catch the diary at the end of each session. Additional interesting behaviours were noted through observation and selected for coding before the quantitative analysis began. In total, we observed whether participants
• reached for the diary (Figure 2.2),
• stood up (Figure 2.4),
• followed hand movements seen in VR (Figure 5A),
• backed away at a moment when the children came close while fighting over the diary (Figure 5B),
• tracked the diary as it was thrown around (Figure 5C),
• tried to catch the diary when it was thrown to them (Figure 2.7), and
• reached to help the girl as she stood up (Figure 2).
FIGURE 5. Moments in the 360° video that we coded for participant behaviours in addition to Figure 2. (A) The children lead the participant in a dance, while their representation in the 360° video follows along. Followed Hands was coded for participants following along with the movements this section. (B) The children fight over a diary right in front of the participant. Many participants were seen Backing Away at this moment, especially if they were standing. (C) The children pass the diary to each other. We noticed almost every single participant Tracking the Diary as it was Thrown Around with their head in this moment, suggesting it was the centre of their attention.
These behaviours were identified to indicate that participants were deeply engaged or feeling a strong sense of presence as suggested by Slater (2009). Behavioural measures have been previously used to measure object presence (Reiner and Hecht, 2009) and demonstrate the social presence of avatars (Bailenson et al., 2003). The video data were independently coded by the first two authors and then reviewed to ensure consensus for the final coding. Participants were also asked whether they tried to catch the diary. We reviewed any video data where there was a mismatch between our code and participant perceptions.
To analyse the relationship between these behaviours and the different settings and conditions, we ran a full-factorial logistic regression for each behaviour. In cases where a significant main effect was found, we conducted a post hoc χ2 (chi-squared) test between each level. We used likelihood ratio χ2 for main effects and interactions, while post hoc tests are Wald-based odds ratio χ2 tests.
2.4.2 Quantitative questionnaire responses
Two-way ANOVA was used to analyse all Likert-scale questionnaire responses. According to Harpe (2015), Likert-scale items can be analysed as continuous data provided that conditions are met, such as having at least five numerical response categories. This also aligns with how the Temple Presence Inventory (Lombard et al., 2009), IOS (Aron et al., 1992), and Affect Grid (Russell et al., 1989) are typically analysed to facilitate meta-analysis. ANOVA is also highly robust to violations of assumptions that include skewness and non-normality (Carifio and Perla, 2008). Tukey’s honest significance test was used for post hoc tests.
We also conducted correlation analyses between specific behaviours and questionnaire results. We used Pearson correlations for relationships between questionnaire answers. Pearson correlations are robust against violations of normality and can be used with Likert-data (Carifio and Perla, 2008). We used point-biserial correlation to identify relationships between dichotomous behaviour coding and questionnaire measures (Field, 2013).
2.4.3 Qualitative questionnaire responses
The open-ended survey questions were analysed through template analysis (King, 2012). The first three authors reviewed all qualitative data and made notes on prominent themes and possible codes for the template. Then, the researchers discussed their observations to derive a draft template. This template was informed by the research questions, knowledge of relevant phenomena (i.e., embodiment, presence, etc.), and observations from the data collection and familiarization stages. Then, the second and third authors divided the components of the template and individually coded the data, revising the template as necessary. The first author reviewed the final coding and contributed to the overarching themes. Finally, the first author connected quotes to relevant quantitative data to understand how participants’ interpretations connected to our measurements.
2.4.3.1 Researcher reflexivity
Qualitative approaches acknowledge the key role that the researcher’s background and expertise play in analysis (Creswell and Poth, 2018). To support the transparency of our qualitative analysis, we present brief statements from each researcher who engaged with the qualitative data outlining the lens through which we engage with participants’ descriptions.
John Desnoyers-Stewart is an artist/researcher with a background in engineering who has exhibited over 40 interactive immersive performances and installations since 2018. His practice-based research investigates underexplored possibilities of VR to support embodiment, social connection, and self-expression. For Eve 3.0, he contributed to the experience design, developed the software, and created the real-time graphics.
Margherita Bergamo Meneghini is a dancer and choreographer who led the creation of Eve 3.0, building upon the successes of Eve: dance is an unplaceable place (2019). Her research is oriented towards audience participation, applying elements from dance movement therapy in combination with immersive technologies.
Ekaterina R. Stepanova is a cognitive scientist and a VR researcher, with over 8 years of research experience. Informed by embodied cognition, phenomenology, and social psychology, her work explores how VR can foster a felt experience of connection.
3 Results
Our results are presented below. In line with Cumming (2014) and APA recommendations, we report 95% confidence intervals for all quantitative measurements [in square brackets]. Marginal significance levels are also analysed to reduce dichotomous reliance on the p-value in evaluating results and focus on the story told by the measurements and effect size (Cumming, 2011). We begin with behavioural measures, questionnaire measures, and correlation results, followed finally by qualitative themes. Raw questionnaire data and coded video data are included in Supplementary Material. Three participants’ data have been removed from the data set as they did not consent to share their data.
3.1 Summary of quantitative findings
Given the complexity of the data presented below (Figures 7, 9, 10), we present an overview of the significant main effects found in this study in Figure 6. Condition had a significant effect on all behaviours except reaching for the diary, while the setting had an effect on all but backing away. Age, movement experience, and frequency were all significantly higher in the festival setting. Condition significantly affected parasocial presence, tactile realism, embodiment, and touch quality, while setting significantly affected social connection (IOS), valence, arousal, and technical quality.
FIGURE 6. Overview of main effects (dashed line: 0.05 < p < 0.10, *p < 0.05, **p < 0.01, ***p < 0.001). Bold text indicates a medium effect size or greater. Colour indicates which independent variable had a significant effect. Elements in brackets are theorized but not measured.
3.2 Behavioural measures
As shown in Table 2, the logistic regressions demonstrated significant main effects for both condition and setting across almost all behaviours. The only exceptions were reaching for the diary, which only showed a marginally significant effect for condition, and backing away, which was not significantly affected by the setting. Detailed descriptive statistics and post hoc test results for each section can be seen in Figure 7. Unless noted otherwise, logistic regression p-values are based on likelihood ratio χ2 tests while post hoc tests are Wald-based odds ratio χ2 tests with one degree of freedom.
TABLE 2. Table of results from logistic regressions on key observed behaviours.
FIGURE 7. Frequency of observed behaviours. Error bars show the 95% confidence interval. Bars show significant pairwise comparisons (*p < 0.05, **p < 0.01, ***p < 0.001). The top row is separated by setting, while the bottom is the total.
3.2.1 Reaching for the diary
We compared only two conditions for reaching for the diary because up to this moment, the no-touch and diary-only conditions were identical. No touch and diary only are combined into “No Touch Before Diary” and compared to “Touch Before Diary” (full touch). A single touch on the participant’s shoulder led to a marginally significant increase in participants reaching to pick up the diary. As seen in Figure 7, this had a marginally significant effect on festival participants (χ2 = 3.50, p = 0.061) but not on lab participants. Overall, festival participants were significantly more likely to reach for the diary (66.0% [52.2%, 77.6%]) than laboratory participants (38.6% [27.1%, 51.6%]).
3.2.2 Standing up
We did not conduct inferential tests for standing up because participants in the full-touch condition were physically compelled to stand up. However, as shown in Figure 7, significantly more festival participants (n = 7, 46.7%) stood up unprompted in the no-touch condition compared to only a single (5%) laboratory participant (χ2 = 5.98, p = 0.014).
3.2.3 Following hands
In all, 100% of full touch participants were seen following the hands seen in the 360° video (Figure 5A). While the facilitator pushed full touch participants’ hands up as they let go in line with the video, we only coded instances where participants continued to move their hands or did so unprompted. There was a large (w > 0.5) effect for condition and medium (w > 0.3) effect for setting (Cohen, 1988). While laboratory participants steadily increased from one condition to the next (Figure 7), festival participants were significantly more likely to follow the hands in the no-touch condition compared to laboratory participants (χ2 = 8.52, p = 0.0035).
3.2.4 Backing up
In all, 63.6% (n = 35) of participants who were standing (n = 55) stepped back from the children fighting over the diary (Figure 5B), regardless of the condition. We also included seated participants who visibly shifted their weight back in their chair (n = 6). As shown in Figure 7, there was a medium effect for condition with full touch participants being 3.619 [1.35, 9.62] times more likely to back away than diary only participants (χ2 = 9.11, p = 0.0025) and 6.67 [2.26, 19.7] times more than no touch participants (χ2 = 10.39, p = 0.0013). More festival participants (46.0% [33.0%, 59.6%]) exhibited this behaviour than laboratory participants (31.0% [20.6%, 43.8%]); however, we found no significant difference (χ2 = 0.71, p = 0.398).
3.2.5 Tracking the diary as it is thrown around
A total of 105 (97.2%) participants were observed tracking the diary with their heads as it was thrown around in the video (Figure 5C). The three participants who did not follow the diary were in the no-touch condition in the laboratory. We did not conduct a logistic regression given the prevalence of this behaviour.
3.2.6 Trying to catch the diary
We found a significant but small (w > 0.1) effect for condition and condition × setting on trying to catch the diary (Table 2). We also found a significant medium effect of setting (w > 0.3). As shown in Figure 7, festival participants (58.0% [44.2%, 70.6%]) were much more likely to try catching the diary than laboratory participants (27.6% [17.8%, 40.2%]). This was particularly notable in the no-touch condition where participants were 28.5 [2.97, 273] times more likely to try catching the diary in the festival setting than in the laboratory (χ2 = 8.43, p = 0.0037). Overall, participants in the full-touch condition were 5.41 [1.43, 20.5] times more likely to try catching the diary than in the no-touch setting (χ2 = 6.18, p = 0.013). In the laboratory setting, participants in the diary-only (χ2 = 4.00, p = 0.046) and full-touch (χ2 = 6.20, p = 0.013) conditions were significantly more likely to try catching the diary than those in the no-touch condition. Meanwhile, in the festival setting, we found a marginally significant increase from diary-only condition to full touch (χ2 = 3.31, p = 0.069).
An additional 17 participants thought they tried to catch the diary despite not physically responding (16.5% of total), while two tried to catch the diary but thought they had not (1.94%), However, most participants’ perceptions aligned with the observed behaviour (n = 84, 81.6%).
3.2.7 Helping the girl stand up
Both condition and setting significantly affected the proportion of participants who reached to help the girl stand up (Table 2). Participants in the full-touch condition were significantly more likely to reach and help than in the diary-only (χ2 = 3.90, p = 0.048) or no-touch (χ2 = 7.29, p = 0.007) conditions (Figure 7). Participants in the festival setting (64.0% [50.1%, 75.9%]) were much more likely to reach out and help than laboratory participants (25.9% [16.3%, 38.4%]) (χ2 = 8.43, p = 0.0037). While no interaction was found, Figure 7 clearly shows that festival participants were significantly more likely to reach out to help than laboratory participants in both the no-touch condition (χ2 = 8.43, 0.0037) and the full-touch condition (χ2 = 6.01, p = 0.0142).
3.2.8 Total key behaviours
We added the total number of key behaviours exhibited by each participant to investigate the overall difference between conditions. This included reaching for the diary, following hands, backing away, trying to catch the diary, and reaching to help (n = 5). We left out standing up since it was physically compelled in the full-touch condition and tracking the diary since nearly every participant engaged in this behaviour.
A two-way ANOVA revealed significant main effects for condition and setting with no interaction (Table 2). There was a strong effect for condition
3.2.9 Behavioural hypotheses findings
These results support H1.1 that setting affects the probability of all realistic behaviours except backing away. H1.2 was not supported as we found only one significant result (trying to catch in the lab) for diary only compared to no touch. The data does however support H1.3 that full touch that includes the diary and performer-facilitated touch increases the probability of all realistic behaviours compared to no touch and using the diary only. Two key patterns are visible in nearly all of the graphs in Figure 7:
1. In the laboratory setting, there is a steady increase in realistic behaviour with additional touch starting from almost none in the no-touch condition.
2. In the festival setting, there is an elevated baseline of realistic behaviour for the no-touch condition. The diary-only condition typically performed worst.
We explore possible explanations in the Discussion section.
3.3 Questionnaire measures
As shown in Table 3, two-way ANOVAs showed significant main effects for condition on parasocial presence, tactile realism, embodiment, and touch quality, while setting significantly affected social connection (IOS), valence, arousal, and technical quality. We found no significant results for condition or setting on object presence. Detailed descriptive statistics and post hoc test results for each section can be seen in Figure 9. Unless noted otherwise, post hoc tests are Tukey’s honest significance tests.
TABLE 3. Results from 2-way ANOVAs on questionnaire results.
3.3.1 Demographics
We compared demographics between settings to identify any significant differences. Gender, VR use, and 360° video use were not significantly different. A contingency analysis showed that festival participants (M = 33) were significantly older than laboratory participants (M = 25) [χ2(5) = 24.2, p < 0.0002]. Festival participants (median = monthly) had a more frequent movement practice than lab participants (median
3.3.2 Temple Presence Inventory
As shown in Table 3, there was a significant main effect of condition on the parasocial presence (actor within medium) and tactile perceptual realism dimensions of the TPI. Post hoc tests revealed a marginally significant increase parasocial presence for the full-touch condition compared to diary only (p = 0.090) and no touch (p = 0.059). There was a medium effect on tactile realism which was significantly higher for the full-touch condition than was for both the diary-only (p = 0.019) and no-touch (p = 0.020) conditions. Interestingly, the pattern seen in the behavioural results was reversed for perceptual realism. At the festival, perceptual realism increased steadily with the increase in touch, while in the laboratory, the diary-only condition is rated the lowest; for details, see Figure 8. Condition had a marginally significant effect on the immersion (engagement) dimension but post hoc tests were non-significant.
FIGURE 8. Score of Likert scale questions from the Temple Presence Inventory. Error bars show the 95% confidence interval. Bars show significant pairwise comparisons (*p < 0.05, **p < 0.01, ***p < 0.001). p-values specified for marginal results. The top row is separated by setting, while the bottom is the total.
Setting had a marginally significant effect on the parasocial and active social dimensions. The mean parasocial score was higher at the festival (5.72 [5.36, 6.08]) than the laboratory (5.37 [5.05, 5.69]). The mean active social score was also higher at the festival (4.94 [4.48, 5.40]) than the laboratory (4.03 [3.59, 4.48]). All other TPI results were non-significant.
3.3.3 Embodiment
We found a medium main effect for condition and a medium interaction effect of condition × setting on embodiment. Overall, the full-touch condition was significantly higher than the no-touch condition (p = 0.022). In the laboratory, embodiment was significantly higher in the full-touch condition than in the diary-only (p = 0.038) and no-touch (p = 0.042) conditions. The diary-only condition also showed significantly higher embodiment at the festival than the lab (p = 0.027). This result is particularly interesting, given the inverted relationship with behavioural measures in the festival setting; for details, see Figure 9.
FIGURE 9. Score of Likert scale questions and Affect Grid from the questionnaire. Error bars show the 95% confidence interval. Bars show significant pairwise comparisons (*p < 0.05, **p < 0.01, ***p < 0.001). The top row is separated by setting, while the bottom is the total.
3.3.4 Social connection
The Inclusion of Other in Self (IOS) scale showed a significant but small main effect of setting (Table 3). Festival participants’ IOS score towards the main character in the 360° video was higher on average at 3.56 [3.02, 4.10] than at 3.29 [2.83, 3.755] for laboratory participants (p = 0.0367). There was also a marginally significant interaction of setting × condition; however, post hoc tests did not reveal any significant differences between specific conditions and settings. This marginally significant interaction may be explained by the contrast between the steady increase in IOS scores across conditions in the laboratory and the steady decrease in the festival setting, as seen in Figure 9.
3.3.5 Affect
There was a medium effect of condition on valence, with festival participants reporting a more positive valence (M = 0.749 [0.698, 0.800]) than laboratory participants (M = 0.611 [0.542, 0.681]). We also found a marginally significant difference between the festival and laboratory settings in the no-touch condition (p = 0.0534). Laboratory participants reported a significant but subtly higher level of arousal (M = 0.570 [0.514, 0.626]) than festival participants (M = 0.531 [0.472, 0.590]). A marginal interaction effect was detected for arousal, but we found no results from post hoc tests. However, as shown in Figure 9, most of the difference between settings can be attributed to the no-touch setting where festival participants were generally more relaxed. Taken together, festival participants reported a more relaxed and positive affect in the no-touch condition, aligning closely with laboratory participants in other conditions.
3.3.6 Quality of experience
We found a medium effect of setting on the perceived technical quality and a significant but almost undetectable interaction effect between condition and setting. Festival participants rated the technical quality significantly higher (M = 4.10 [3.91, 4.29]) than laboratory participants (M = 3.96 [3.75, 4.18]). This small difference can be entirely attributed to the no-touch condition which was higher at the festival than in the laboratory, as seen in Figure 10 with marginal significance (p = 0.075).
FIGURE 10. Score of touch and technical quality scores from the questionnaire. Error bars show the 95% confidence interval. Bars show significant pairwise comparisons (*p < 0.05, **p < 0.01, ***p < 0.001). The top row is separated by setting, while the bottom is the total.
For touch quality, we removed the no-touch condition since judgements of touch quality would be baseless. We found a strong effect of condition on touch quality with full touch scoring significantly higher than the diary-only condition (p = 0.0001), as shown in Figure 10. We found no significant difference between the lab setting (M = 3.82 [3.47, 4.16]) and festival (M = 3.92 [3.57, 4.27]). Touch quality may have been worse in the diary-only condition for two reasons:
1. There was only one moment of touch for participants to base their judgement of tactile quality on.
2. Participants perceived other moments of visual touch as missing tactility, and thus a lower quality.
3.3.7 Questionnaire hypothesis findings
The results partially support H2.1 and H2.3 that setting and the full-touch condition increase the scores of some questionnaire-based indicators of presence, while others were inconclusive. The full-touch condition incorporating a prop and performer increased participants’ sense of parasocial presence (actor within medium), tactile realism, and embodiment. The festival setting significantly increased participants’ sense of social connection (IOS), valence, and arousal. The data does not support H2.2 as we found no measurable difference between the no-touch and diary-only conditions.
3.4 Correlation results
To better understand how participants’ questionnaire results correlated with their behaviour, we ran a multivariate correlation analysis between select behavioural and questionnaire measures. All notable correlations were positive unless noted otherwise.
3.4.1 Prior experience
We investigated correlations of reaching for the diary with participants’ prior VR and 360° use as this was the first element to diverge from ordinary VR and 360° video. As such, their behaviour up to this point should correlate with their prior experience. We ran a contingency analysis that showed that reaching for the diary increased from 36.5% of participants who had never used 360° video to 66.7% of participants who had used it more than three times [χ2(5) = 12.3, p = 0.031]. However, there was no significant correlation between overall VR use and reaching for the diary [χ2(5) = 6.26, p = 0.282]. This suggests that participants with more 360° video, but not VR experience, were more likely to reach for the diary.
3.4.2 Behaviour correlations with questionnaire results
Point-biserial regression was used to identify correlations between dichotomous behaviour variables and questionnaire results. The correlation results are summarized in Table 4. p-values can be misleading when analysing correlations (Cumming, 2011), and effect size is a significantly better indicator (Field, 2013). Therefore, we present only the effect size with confidence intervals and look for at least a weak (r > 0.1) correlation whose confidence interval does not cross zero.
TABLE 4. Correlations between behaviours and questionnaire results.
Backing away was weakly correlated with spatial presence but showed no other clear correlations. Trying to catch the diary was moderately (r > 0.3) correlated with object presence. It also showed weak correlations with the TPI active dimension of social presence and participants’ valence. Reaching to help the girl stand up showed the strongest correlation results. We found moderate correlations between reaching to help and both object presence and valence. There were weak correlations between reaching to help and all dimensions of the TPI except tactile realism. Embodiment was also weakly correlated with reaching to help. While the 95% confidence interval for IOS just barely crosses zero, there may be a weak correlation present here as well. In addition to the results shown in Table 4, following along with the hands in the 360° video (Followed Hands) was weakly correlated with embodiment (r(106) = 0.212[0.023, 0.387]). The total behaviour count was also moderately correlated with object presence (r(106) = 0.296[0.112, 0.460]) and valence (0.323[0.141, 0.484]).
3.4.3 Questionnaire result correlations
Pearson’s r was used to identify correlations between questionnaire results, as shown in Table 5. We did not assess correlations between the TPI dimensions, as this was already addressed in the development of the questionnaire (Lombard et al., 2009). Embodiment was strongly correlated (r > 0.5) with tactile realism and moderately correlated with the spatial, parasocial presence, and immersion dimensions of the TPI but not with active social presence. Object presence was strongly correlated with spatial presence, weakly correlated with active social presence, and moderately correlated with the other TPI dimensions. Embodiment and object presence were also moderately correlated. The IOS results were moderately correlated with embodiment, object presence, and both the spatial and tactile dimensions of the TPI. IOS also showed weak correlations with the parasocial and immersion dimensions but not with active social. We also found a correlation between the TPI active social dimension, based on whether participants smiled, and valence (r(106) = 0.292 [0.107, 0.457]). Although this correlation was weak, it was the strongest correlation of any variable with valence.
TABLE 5. Correlations between questionnaire results.
3.5 Qualitative themes
Here, we present the key themes identified in participants’ qualitative responses (Section 2.4.3). We include exemplary quotes with additional quotes in the Supplementary Material. Relevant quantitative measures provide context to illustrate how qualitative themes relate to quantitative results.
Participants described their experience in VR, reactions to feeling touch, and other significant moments. By considering participants’ reflections, we can understand what felt important for them and interpret which elements of the VR experience may have elicited, supported, or inhibited aspects of their phenomenological experience. The themes identified tell a nuanced story about each participant’s unique experience that contributed to their sense of presence in VR. These themes are interwoven with the notions of embodiment, touch, emotions, and the social and narrative components of the experience. We outline the themes through three intertwined facets of the complex experience of presence: physical touch supports embodiment, object, and spatial presence (Section 3.5.1); eye contact, proximity, and engagement support social presence (Section 3.5.2); and narrative immerses participants in the story (Section 3.5.3). We then consider participants’ descriptions of the diary, which fuses all three facets into a rich experience.
We refer to participants by code numbers. The first two letters indicate the location:
• CD: Cinedans Festival, Amsterdam, Netherlands
• SF: Simon Fraser University, Surrey, Canada
• CA: University of Calgary, Canada
• SC: University of Santiago de Compostela, Spain
• PL: Portland, OR, United States.
The two-digit code indicates the session number which included up to three participants. The last letter indicates the condition: A = all/full touch, B = book/diary only, and C = control/no touch. Relevant quantitative measures are indicated by the following codes:
• SP: Spatial Presence (1–7)
• SoP: Social Presence (Parasocial) (1–7)
• SoA: Social Presence (Active) (1–7)
• EI: Engagement/Immersion (1–7)
• TR: Tactile Realism (1–7)
• Em: Embodiment (1–7)
• OP: Object Presence (1–7)
• IOS: Social Connection (1–7)
• Ar: Arousal (0–1.0)
• Va: Valence (0–1.0)
• TeQ: Technical Quality (1–5)
• ToQ: Touch Quality (1–5).
For example, a participant in the third session at Cinedans in the diary-only condition with a tactile realism score of 6 would be indicated as (CD03B—TR: 6).
3.5.1 Physical touch supports embodiment, and spatial and object presence
Participants described how physical touch contributed to spatial presence, embodiment, and object presence. Picking up the physical diary supported participants’ sense of object presence which increased their spatial presence. “I reached out to touch the books that were placed next to me, they were actually there which made me think that I’m fully immersed in the VR environment” (SF07B—TR: 6, OP: 7, SP: 6). Interestingly, several participants described this moment of presence as surreal: “The most surreal part of the experience was when I was physically interacting with the objects that were in the environment which made me really feel like I was there” (SF10A—TR: 7, OP: 7, SP: 6.6). The merging of the physical and virtual worlds through touch felt not only surprisingly realistic but also uncannily impossible and therefore surreal.
Physical presence often had a rich social and emotional component. For instance, CA03A described how the touch they felt elicited a complex emotional response: “The two boys touching me on the shoulder. It was a strange moment, it seemed equal parts threatening and kind” (SoA: 7, SoP: 7, EI: 7). Physical touch stimulated a further desire to interact with the environment physically. Participants described a desire to touch, to be touched, to interact with the diary, and to take part in the action seen in VR: “…each time the people in the environment approached me …I felt a not insignificant desire to physically respond and interact in kind” (PL01B).
Inconsistent touch took some participants out of the experience. Some participants in the diary-only condition developed an expectation for physical touch that went unfulfilled, impeding their sense of presence: “Later the girl held my hands but in reality I didn’t feel anyone grabbing my hands which really reduced my immersion” (SF07B—ToQ: 2). However, SF07B still rated their spatial and social presence highly, stating that “the visuals were extremely realistic” (SP: 6, SoP: 7, EI: 5).
While the diary supported presence, when immersants reached for it and did not “catch” it, it sometimes made them more aware of the mediation of their experience: “My first response was to reach out and grab it, but after realizing it was not real, I laughed” (SF01B—EI: 7). These are examples of bifurcation, a simultaneous awareness of two parallel dimensions (Morie, 2007).
3.5.2 Eye contact, proximity, and engagement support social presence
A crucial component of presence is the social presence of other people. In the 360° video, characters acknowledge the presence of the participant and invite them to join in. This created some of the most compelling moments as a majority of the participants reported a strong reaction to the characters making eye contact or getting close to them: “…each time a person looked directly at me or approached me. That elicited the strongest feelings for me. It was eerie how much it felt like they were looking directly at me” (CA04C—SoP: 5, SoA: 5). Eye contact and proximity were the most commonly mentioned significant moments of social presence.
3.5.2.1 Eye contact acknowledges immersant’s presence
Eye contact produced a spectrum of reactions, with many participants recognizing the pivotal role it played in social presence: “The eye contact was powerful throughout the experience and contributed a lot to the sense that I was part of the experience, rather than an observer” (SC01B—SoP: 6.5, SoA: 6). This led participants to feel the social presence of others and being noticed by them: “I felt like the people are actually there and are noticing me” (SF07B). Proximity and touch seemed to compound this experience. Eye contact sometimes felt “intimate” (SC01B) and even “uncomfortable” (SF06A) and could elicit a sense of “awkwardness” (SF06B), “stress” (CA01B), or “anxiety” (SF08B). While eye contact sometimes felt intense and uncomfortable, this intensified the experience, making participants more engaged and immersed: “The direct eye contact and abstract movements made me feel uneasy but even more immersed” (SF01B).
3.5.2.2 Proxemics pushed people away but made social presence palpable
Another contributing factor to social presence was the proximity of the characters to participants. Being directly approached stimulated a strong sense of being together with the characters: “I thought that I was watching a film … but after the character approached me I realized I was part of it too” (SF12C—SoP: 6). A particular moment when all of the characters get close to the participant often led to an embodied response from participants trying to create more distance, as seen in Section 3.2. “Having people in the video get close to my space. I instinctively would react to move out of the way” (SF19B—SoP: 5.5). Similarly to the direct eye contact, moments of close interpersonal proximity were sometimes associated with fear: “I think that it was interesting to see how other people started getting closer to me … It felt like a threat mentally so my hands were a bit shaking” (SF13A—SoP: 5.5). Participants sometimes tried to remove themselves from the experience to overcome this discomfort: “ … I had to close my eyes as consciously I knew it’s just video but I wanted to move away from them …” (SF11C—SoP: 2.5, SoA: 2, EI: 5).
The discomfort of intense eye contact was mitigated by positive facial expressions: “When the two boys came up close and danced, I still wanted to take a step back, but their attitude was so jovial that it really made me smile” (CA03A—SoP: 7, SoA: 6). Characters smiling pulled participants into the experience with many reporting a desire to smile back: “The moments where characters smiled or clearly addressed me with an expression or gesture made me feel strongly included in the action” (CD12A—SoP: 7, SoA: 5, EI: 6). Strong emotional reactions also supported the feeling of presence: “The first person who made eye contact changed my feeling of observing the scene to being really present. It’s quite intense” (SC01A—SP: 4.4, SoP: 5.5, SoA: 5). This also demonstrates the dynamic nature of the experience, changing from one moment to the next. Participants felt that the sustained eye contact was very realistic, while sometimes too much: “the main character was looking right at my eyes … It gave me a bit of anxiety because it felt too realistic” (SF08B—SoP: 6.5, SoA: 5).
3.5.2.3 Included through an invitation to participate
When the characters invited participants to join, it deepened participants’ sense of presence, engagement, and embodiment: “When the two girls helped me get up and come dance with them. It felt like I was actually there and included. It changed from watching to (inter)acting” (CD06C—SP: 6.6, Em: 6). However, the sense of social presence was most strongly supported by participants’ own desire to participate: “I wanted to move and dance with the people in the VR, I wanted to smile back and respond” (CA01B—SoP: 6.5, SoA: 6).
Eye contact and proximity elicited diverging feelings of either intrusiveness or inclusiveness. Intrusiveness happened when the VR characters approached directly and did not engage the participant, who remained an observer: “Whenever the characters looked directly at me I felt as if I was intruding their space. Over time that changed and they started to include me …” (SF05A). Inclusiveness occurred when participants were invited to participate, finding their place in the narrative (e.g., CD06C). Characters smiling created a more positive disposition: “When they came close with the two of them, smiling, that felt nice and included” (CD07B—SoP: 5.5, SoA: 5, Va: 0.69).
When eye contact and proximity led to a feeling of inclusiveness, participants reported a sense of social connection: “Dancing with the characters. I felt connected to them and felt like they can actually see me” (SF17A—SoP: 5.5, IOS: 1). This supported immersion into the virtual world: “It felt like I was being led into another world, one that those kids are seeing and feeling” (SF15B—SP: 6.4, SoP: 5, EI: 5, IOS: 6).
3.5.3 Narrative immerses participants through role play and a willing suspension of agency
Participants willingly suspended their disbelief and sought their role in the narrative: “The set-up makes me try to play a role and kind of obey the role” (CD13B—EI: 6). Participants often pondered about the appropriateness of their interpretation and actions: “It made me wonder what my role in this narrative was supposed to be” (CA03A—EI: 7). They often followed what was expected from their role, taking the cues from the virtual body provided to them: “I think I was trying to match my apparent hands” (PL02A—EI: 4, Em: 2). This willing engagement immersed participants without necessarily evoking a strong feeling of embodiment: “The way I had to follow my own arms was a strong experience, I was trying to imitate myself” (CD03C—EI: 6, Em: 3). By moving along with what they saw in VR, participants became more involved and immersed in the story: “Arm waves made me feel that I’m in same dimension” (CD09B—SoP: 5.5, EI: 5, Em: 5).
3.5.3.1 A willing suspension of agency: allowing the virtual body to lead you
Since the experience was a 360° video and could not respond to participants’ actions, it might be assumed that participants feel little to no agency. However, many participants’ descriptions indicate the contrary, due to their willing suspension of agency akin to the concept of willingly suspending disbelief (Coleridge, 1817). Participants engaged with the narrative by suspending their agency and submitting to what their virtual body was doing, gaining an embodied involvement in the story: “it felt more like the VR body was controlling my actual body than the other way around” (CD06C—EI: 6, Em: 6). Participants often felt surprised by how the experience pulled them in: “I was intrigued by how I … was trying to move my body simultaneously with the virtual body” (CD04A—EI: 4, Em: 2). However, participants were sometimes frustrated by the lack of agency in the 360° video: “…the experience was wanting me to move when I wanted to stay still and forcing me to stay still when I wanted to move …” (CA05B—EI: 5, Em: 5).
3.5.3.2 Identities shared and split
Participants felt like they were a character playing a role in the story, performing the actions they expected of their character. Some participants felt like they were a different virtual character: “I suppose I didn’t really associate myself in the experience with my own identity. I got thinking about whether the children were seeing someone male or female, young or old” (CA04C—EI: 6). Other participants retained their own identity in the virtual space. This discrepancy sometimes inhibited participants’ experiences as they felt out of place in the story: “When two girls came very close to me it felt awkward, because of the age difference … Like I wasn’t the right character for the role, maybe?” (CD02A—SoP: 5, EI: 5). In addition to age, male participants CA05B and PL01B were particularly uncomfortable because of the gender interaction as they felt they should not be interacting with a young girl in such a seemingly intimate encounter in a park.
3.5.4 The physical diary fuses touch, narrative, and social significance
The diary was central to the experience. It combined the physical, social, and narrative elements, all intertwined through a single prop.
The diary’s virtual and physical existence caused participants to reflect on what was real or not: “The most surreal part of the experience was when I was physically interacting with the objects that were in the environment which made me really feel like I was there” (SF10A—SP: 6.6, TR: 7, Em: 7, OP: 7). Participants stated that during their experience in VR, it felt “real” (SF01B), but in retrospect, they knew it could not be: “[I’m] not sure to what extent I am trying to justify trying to catch the book after the fact. It did make me laugh that I tried to catch it” (CA06A—OP: 6).
3.5.5 Subjective explanations for trying to catch the diary
Participants’ reactions to the diary being thrown towards them exemplify how the physical, social, and narrative components of the experience come together to produce an observable physical response. Reflecting on their experience, participants provided a range of explanations for why they think they tried to catch the diary ranging from a simple reflex to reasons based on the meaning embedded in the diary.
3.5.5.1 Because it is an instinctive response
Many participants stated that they attempted to catch the diary reflexively: “It was automatic. I will always try to catch something if it is thrown to me!” (SC01B—OP: 7). Surprised by this reflex, participants realized how immersed they were: “Instinctively I felt I had to catch the book, that made me realise how much I felt [immersed] into the environment” (CD13A—EI: 6, OP: 7).
3.5.5.2 Because it was expected
Others anticipated the diary to be thrown to them narratively: “I thought they’re going to throw me the book so I should be ready to catch it” (CD16B—TR: 7). The physical interaction with the diary in the full-touch condition set the expectation that a physical diary might be thrown: “…since I had a physical diary to pick up at the beginning, I thought that … I would get to touch it physically” (SF10A—TR: 7, OP: 7). For some participants, their inability to catch the diary imbued with emotions: “I felt slightly disappointed when I didn’t catch the book as if I truly missed it” (SF09A—OP: 7).
3.5.5.3 Because I was immersed
Some participants attributed their actions to feeling immersed in the experience: “[The] very immersive experience led to my natural bodily reflexes to act immediately” (SF14A—EI: 6, OP: 6). Many explain that it “felt real.” Social presence was also a factor: “Because I felt like … I was [one] of them, so I tried catching the diary …” (CD18A—SoP: 6.5, SoA: 7, OP: 6). Conversely, participants who did not try to catch the diary felt like spectators watching a film. “[I] didn’t feel like it was my own body but watching as in the same perspective” (SF16B—Em: 3, OP: 4).
3.5.5.4 Because the diary was significant
The need to catch the diary was often rooted in participants’ relationship to it. Some felt that the diary was theirs, while others felt they should not touch it since it belonged to the girl. For instance, SF17A describes a sense of personal connection to the diary: “I had the urge to grab the diary like it was mine” (OP: 7). Conversely, CD12A felt that it was inappropriate for them to grab someone else’s diary: “I wanted to pick it up–but … it wasn’t for me to touch” (SoP: 7, OP: 7). Participants felt that the narrative importance of the diary made them try to catch it: “I tried to catch the diary because … it was something important” (SF15B—OP: 6). The story filled the diary with significance: “Through the narration … we understand the weight of what is inside the book” (CD12A).
3.5.5.5 Because I felt empathy for the girl
Some participants felt a sense of emotional connection to the main character and wanted to help her: “I felt a lot of love toward her and wanted to find a way to help” (CA04C—IOS: 3). They expressed a physical urge to help: “I felt an overwhelming sense to intervene/empathy for the main character” (CD12A—IOS: 2), or at least to comfort her: “I felt bad for the main girl because everyone was throwing her stuff around. I wanted to give her a hug” (CD16C—IOS: 6). Participants who felt empathy did not necessarily feel connected to the girl sometimes, as indicated by low IOS results. Those who did feel connected mentioned that this feeling was fostered by the intimate thoughts shared from the diary: “It felt like the main character was telling everything to me that was in the diary and her thoughts, that made me feel super connected with her” (SF08A—SoP: 7, IOS: 6).
4 Discussion
This study systematically explored how performer-facilitated touch in controlled laboratory and ecological festival settings affected participants’ presence in VR. We found that both setting and touch impacted participants’ experiences of presence and embodiment in a 360° video performance, Eve 3.0, as demonstrated through behaviour and self-report. Here, we interpret the quantitative and qualitative results and how they relate, discussing the underlying factors contributing to our findings. We discuss the values and challenges of conducting research in the wild and embracing the complexity and diversity of experiences that arise. We conclude with research directions towards a deeper understanding of physical human touch in immersive experiences.
4.1 Festival setting and performer-facilitated touch increase realistic behaviour, presence, and embodiment
As hypothesized, performer-facilitated touch increased the probability of realistic behaviour in response to a 360° video. As Slater (2009) claims, this realistic behaviour reflects a stronger sense of presence in VR, suggesting that touch indeed enhances presence, aligning with previous research (e.g., IJsselsteijn et al., 2006; Petkova and Ehrsson, 2008; Slater et al., 2009; for a review, see Souza et al., 2021). Previous research showed that touching a physical prop increased presence and realism in VR (Hoffman, 1998; Zhang et al., 2022; Felip et al., 2023). Unlike these studies, we did not find a significant difference between the no-touch control and diary-only conditions. However, we did see a significant increase from diary only to full touch in all behaviours. As hypothesized, we observed a steady increase between conditions in the laboratory setting, suggesting that touching a single prop likely lies somewhere between experiences with no touch and those incorporating multiple moments of human touch.
Interestingly, participants were more likely to engage in all of the observed behaviours in the festival setting, particularly in the no-touch condition. While the laboratory results suggest an incremental benefit with increased touch, the festival results suggest that consistency is more important. The physical diary might set up an expectation for future touch that is then unfulfilled, reducing presence. This difference may relate to different expectations. Festival participants came to the dance festival to enjoy it. Thus, they were likely eager to engage and follow along. Festival participants in the full-touch condition were also more likely to reach for the diary after a single shoulder tap, suggesting that they may have been more attentive to touch. On the other hand, laboratory participants, mostly university students pursuing a technical degree, may have anticipated an innovative technological solution. Touching the diary therefore incrementally enhanced their experience over 360° video.
4.1.1 Touch leads to complex experiences that support realistic behaviours resulting in complex experiences
In line with Souza et al. (2021), we found behavioural measures were more reliable than questionnaires. However, as Souza et al. identified, it can be challenging to understand behaviour without additional measures. We found that performer-facilitated touch increased participants’ sense of social presence, tactile realism, and embodiment. These results make clear sense as the performer’s touch should feel realistic and mostly facilitate social touch between different characters and participants’ bodies. The lack of results around spatial presence and immersion is reasonable as their high scores likely represented a ceiling effect from the realistic visuals and engaging narrative. We were surprised by the lack of results for object presence, given the clear effects on tactile realism and embodiment. However, this may have resulted from the diary’s narrative significance, leading its presence to be high across all conditions and reinforcing the social dimension of presence. This combination of questionnaire results along with the increased behaviours at the festival indicates that the experiences evoked by touch were complex.
To unpack the complexity of participants’ experiences, we conducted a correlation analysis. The moderate correlation between object presence and trying to catch the diary indicates that this behaviour coincided with a high level of object presence, regardless of what led to that level of object presence. This suggests that the reflex to catch the diary might be a good indicator of object presence. While backing away was weakly correlated with spatial presence, this was more likely an indicator of participants restoring a comfortable level of proximity to the characters in line with the equilibrium theory (Argyle and Dean, 1965; Bailenson et al., 2001). While backing away may indicate social presence, some participants were instead excited to be included in the circle.
Reaching to help correlated weakly with most measures of presence, suggesting that this behaviour might indicate overall presence. While this gesture might indicate a connection to the person being helped, there was no clear correlation for IOS, suggesting participants instead followed their virtual body. Meanwhile, the correlation between following hands and embodiment suggests that following hand movements led to embodiment, or vice versa. While we cannot infer causation, participants’ dispositions and experiences likely led to realistic behaviours that in turn reinforced their experience through a feedback loop. This is perhaps most clearly demonstrated by the moderate correlation of valence with several behaviours. Happier participants may have been more likely to willingly suspend their disbelief and engage, creating a feedback loop that reinforced that willing engagement.
Strong correlations between questionnaire results suggest that the different measures of presence all were closely interrelated. While these correlations are not enough to unpack a detailed model of their relationships, we can begin to see how deeply interwoven each aspect is. For example, tactile realism may facilitate object presence and embodiment and thereby extend to spatial presence. These aspects also seem to be modulated by other factors such as social dimensions that are deeply rooted in the participants’ cultural context and past experiences.
4.1.2 Triangulating methods towards a multi-faceted understanding
To better understand participants’ experiences, we triangulated our quantitative and qualitative results. Generally, participants’ descriptions align with questionnaires while capturing additional significant elements such as eye contact and proximity. They give insights into what might be happening behind the numbers and unpack the unfolding of the experience over time, giving specificity to what moments led to that experience. Qualitative results suggested that immersion may have been influenced by the rich narrative more than by touch. They also showed that eye contact and proximity with characters led to social presence.
We also found conflicts that indicate complexity or issues with interpreting questions. For example, some participants reported a low sense of embodiment despite following the hands suggesting that embodiment required more than aligning with the body. We also found that participants who mentioned empathy and connection gave highly variable IOS scores for social connection. The clearest conflict was between participants’ perception of catching the diary versus the observed behaviour. In all, 18.5% of participants either misremembered (n = 14) or could not remember (n = 6) whether they tried to catch the diary. This demonstrates the utility of behavioural measures to corroborate with participant’s experiences or overcome fallible memories (Henry et al., 1994).
4.2 Richness of experience
Our qualitative results suggest that beyond touch, many components contributed to an overall richness of experience, allowing participants to become immersed. While Slater and Wilbur (1997) described immersion as a property of technology, Ermi and Mäyrä (2005) proposed it as the confluence between design elements, and Vidyarthi (2012) posited it as an active co-creation in the mind of the spectator. Our findings align with Ermi and Mäyrä’s (2005) sensory and imaginative dimensions of immersion. A combination of sensory and narrative richness led participants to become immersed and feel present.
4.2.1 Richness of tactility: physical interaction
In the full-touch condition, this experience evoked rich tactility comparable to physical reality. Participants experienced moments of touch completely in intricate detail, making the experience feel real. It encouraged participants to become immersed and feel present in the environment along with the objects and people they touched and saw.
4.2.2 Richness of visuals: social interaction
The compelling visual richness of Eve 3.0 was captured in participants’ descriptions, high questionnaire scores, and surprisingly realistic behaviours in the no-touch condition. Even without touch, the visual experience was compelling, but together they produced an experience that drew participants in deeper. Participants spoke to the power of eye contact and proximity, suggesting that the rich visuals evoked an intimate social presence that overwhelmed some as they stepped back to restore a comfortable level of proximity (Argyle and Dean, 1965; Bailenson et al., 2003).
4.2.3 Richness of story: meaningful engagement
Revealing the contents of a personal diary, Eve 3.0 presented a rich and emotional story. The action of the characters is equally rich, as they dance in a story that unfolds through layered movement and keeps participants engaged. Beyond touch and visuals, rich narrative elements are critical to immersing participants in the story world and might amplify the presence facilitated through the senses. The diary’s focal point in the story ties the rich imaginative and sensory elements of immersion into a single object, leading participants to reach out to catch it for a variety of reasons.
4.3 A willing suspension of disbelief based on individual and experience, context, and content
The rich tactile, visual, and narrative elements of Eve 3.0 led to a variety of experiences and behaviours from participants. Participants’ experiences and behaviours are situated by their past, dispositions, memories, context, expectations, and intentions. The outcome is the result of the interaction between the VR artefact and participants’ active role in willingly suspending their disbelief and engaging in meaning-making within the narrative as they co-construct the experience. The settings were a confluence of different cultures, contexts, expectations, and more; all of which played a role in deciding their resulting experience and behaviour.
While our study focused on how tactility could support the sense of presence, participants often described willingly “playing a role.” Role-playing might be a useful conceptualization of embodiment in VR that recognizes participants’ awareness (Quaglia and Holecek, 2018) and agency over such experiences, and how attention can shift between physical and virtual or even be experienced simultaneously (Morie, 2007).
4.4 Meaningful discomfort
While many experiences were positive, participants also described discomfort. Discomfort was often associated with a lack of agency or social discomfort. Some felt that their body did not represent them. Other participants mentioned discomfort around interacting with school-age children. Indeed, giving participants a clearer role to play from the outset might have helped, for example, telling them who they were and why they were there. We do indeed introduce participants to their character “Eve” in the full performance.
While physical touch from an unseen facilitator might be expected to increase discomfort, it seems that this novel experience drew participants in. Participants became more immersed in their role, and no participant reported discomfort from the performer’s touch itself. Elements that did make some participants uncomfortable also drew others into the experience. For example, while eye contact and proximity felt intense, it increased immersion and connected participants to the characters. Discomfort around agency left participants feeling trapped, wanting to help but unable to do so. However, these moments were vital to the story, and built tension that contributed to a meaningful experience. This is a challenge to balance in immersive storytelling, especially with pre-recorded 360° video. Discomfort is not inherently bad and can be vital to producing a meaningful experience (Benford et al., 2012).
4.5 Reciprocal benefits of bringing art and science together: authentic experience, authentic setting, and authentic results
Rooting research in artistic performance allows us to capture the richness and authenticity of an experience intended for real world use. This means that the visual and narrative quality were very high, with a focus on rich storytelling rather than manipulating a particular variable. This ensures a complete experience that best represents how the technology is used outside of the laboratory, especially in an ecological setting like the dance festival. While this can lead to noisier data that can be challenging to interpret, the results we do find are made more meaningful by their authenticity. Moreover, unexpected results are often the most interesting. As we see here, this can lead to more questions than answers, and carefully designed laboratory-based research is required to reciprocally inform such studies. Nonetheless, we demonstrated how a festival can provide a relatively controlled setting that invites a vastly different and potentially more representative demographic.
Bringing art and science together also leads to more ethical research through direct benefits to participants. Research can unfortunately treat participants as subjects rather than collaborators, sometimes even causing harm to evoke a behavioural response (e.g., Petkova and Ehrsson, 2008). By contrast, participants thanked us for their experience and shared profound moments, leaving us feeling elevated and enthusiastic. Despite not being required to participate in the study, the minimal dropout rate of 2.6% (n = 3) speaks of participants’ enthusiastic participation.
4.6 Limitations: it’s complicated
While conducting our research using an authentic experience in an ecologically valid setting led to rich and interesting results, it also limits how our results can be interpreted. Our settings included many inseparable elements such as demographics, context, facilitator, and headsets used. As a result, it is challenging to identify what differences between the settings may have contributed to the results. However, we simplified and aligned our procedures to limit the impact of any differences beyond the context and participant demographics. As with most VR studies, participants are not necessarily representative of the general population. However, by studying two decidedly different demographics and contexts, our study captures just how much the results can be affected.
As with any VR study, it is difficult to measure complex phenomenological experiences that vary over time through post-experience questionnaires. Limited research indeed shows that embodiment changes dynamically (Keenaghan et al., 2020). For example, some participants described negative experiences in one moment, followed by positive ones in another. It becomes unclear which moment(s) that participants’ scores reflect, even when we ask them to reflect on a specific one. Our questionnaires were also limited by a lack of standardized questions that fit a 360° video experience. However, behavioural measures and qualitative responses helped overcome this.
4.7 Future work and research directions
Most systems for social touch fall far short of the richness of human touch (Gallace and Girondini, 2022). Human touch is inextricably imbued with sociocultural meanings, not only making social touch a more potent experience but also introducing a broad range in experiences, depending on how participants perceive such touch. As we found in Section 3.5.3, even touching the diary was imbued with rich social significance through the narrative. Future research should continue exploring the interplay between physical tactility and sociocultural meaning, especially in mediated VR experiences.
While questionnaires are used commonly in VR research, as we found here, they have many limitations. They face challenges around interpretations of language used in the survey, have limited comparability across different media, and are particularly limited in capturing the dynamic experience of presence or even a singular past moment. As suggested by Souza et al. (2021), behavioural measures can be more reliable and may offer a way to analyse presence over time. However, they are still best supported by measures that give insights into participants’ interpretations. To reduce recall errors and misinterpretation while digging deeper into the connections between behaviour and experience, we recommend a combination of behavioural observation and micro-phenomenology (Petitmengin et al., 2018). Micro-phenomenology is based on an explication interview and allows deeper access to an authentic singular past experience through the process of evocation. With a highly trained interviewer, it affords a fine-grained exploration of the elements of a very short singular experience. Simultaneous behavioural observations can capture the experience over time and help corroborate and interpret the interview results. This approach could capture the rich temporal nature of presence, measuring presence while also keeping the rich diversity of individual experiences in view.
Here, we began to unpack the complexity of the experience of presence. From our correlation analysis and qualitative results, it seems that many aspects of the phenomenology of VR that are often studied in isolation (e.g., embodiment, spatial presence, social presence) are deeply intertwined. Mediation analysis could help unpack the deeper structure of how these elements are intertwined, keeping in mind that this structure likely depends on many factors embedded in each individual, technology, and media artefact. For this reason, it is also clear that more research is required that establishes a reciprocal relationship between art and science, with each contributing to a deeper and richer understanding of the phenomenology of VR experiences.
4.8 A rich experience of real human touch
This study uncovered how tactile cues and human interaction contribute to a more compelling embodied immersive experience. With the growing use of performer-facilitated touch in immersive performance, it provides a timely guide to artists and researchers alike to better understand how incorporating physical human touch can affect embodiment and presence in the immersive performance. In addition, we hope that our use of pre-recorded 360° video opens other researchers to the potential of an often overlooked technology. Our participants had deep experiences of immersion and presence that real-time computer graphics cannot yet replicate. Even in a pre-recorded 360° video, the incorporation of physical touch is deeply interwoven with other considerations such as the individual experience, the context in which it is viewed, and the sensory and narrative richness of the experience beyond touch.
We hope this study can inform the exploration of novel solutions that root virtual experiences of tactility in the full richness offered by human touch. VR research surrounding touch, especially social touch, should consider as its basis, a full physical experience of touch in its entirety rather than being constrained to what is currently possible with haptic technology. This can help identify important gaps in research that often perpetuate significant assumptions about touch, a sorely under-investigated and misunderstood sense (Gallace and Spence, 2010). As mixed reality becomes more prevalent, this approach also gains importance as our physical and virtual worlds begin to blend into one. Research into real human touch will continue to guide VR towards rich and meaningful experiences.
Data availability statement
The original contributions presented in the study are included in the article/Supplementary Material; further inquiries can be directed to the corresponding author.
Ethics statement
The studies involving humans were approved by the Simon Fraser University Research Ethics Board and Université Paris 8 Ethics Committee. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individuals for the publication of any potentially identifiable images or data included in this article.
Author contributions
JD-S: conceptualization, data curation, formal analysis, investigation, methodology, project administration, software, visualization, writing–original draft, and writing–review and editing. MB: conceptualization, formal analysis, investigation, resources, writing–original draft, and writing–review and editing. ES: conceptualization, data curation, formal analysis, investigation, methodology, writing–original draft, and writing–review and editing. BR: conceptualization, methodology, resources, supervision, and writing–review and editing.
Funding
The authors declare financial support was received for the research, authorship, and/or publication of this article. This research was supported in part by the Social Science and Humanities Research Council of Canada (R632343, R640228). The artefact we researched, Eve 3.0, benefited from state aid managed by the French National Research Agency under the Future Investments program bearing the reference ANR-17-EURE-0008. The artwork was produced with the financial assistance of the European Union. Open access publication fees funded by Simon Fraser University.
Acknowledgments
The authors thank Daniel González-Franco for contributing the VR interaction concept and supporting the filming of the 360° video. They also thank Veronica Boniotti for concept development, scriptwriting, and stage assistance and Mark Lee for production and scriptwriting. They especially thank the whole production team, especially the 360° video dancers: Maude Doublet-Viguié, David Fleury, Alexi Mondet, Lola Montaut, Charline Regnier, Mélissa Vaurin, and Camille Nerry. Eve 3.0 was produced by Compagnie Voix with support from Digital Rise, the Seesaw Project, and Laval Virtual Center. Research venues were provided by Cinedans Amsterdam, Simon Fraser University, University of Calgary, Intangible Realities lab 2.0 at the University of Santiago de Compostela, and more. For full credits, visit compagnievoix.com/en/projects/creation/eve-3.
Conflict of interest
The performance artefact studied here was conceived by the second author (MB) and produced by Compagnie Voix, a non-profit production association that MB belongs to as artistic director and choreographer. Compagnie Voix, along with the first (JD-S) and second (MB) authors, may receive royalties and other compensation for public performances of Eve 3.0. However, this experimental performance was conceived to investigate new forms of immersive performance, and we do not feel that fair compensation for performances has affected the results presented here. We present a transparent and honest account of our findings in the interest of sharing knowledge across art and science.
The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors, and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Author disclaimer
The views expressed herein can in no way be taken to reflect the official opinion of the European Union.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frvir.2023.1336581/full#supplementary-material
Footnotes
1See a trailer of a performance at vimeo.com/834217436; for more details, visit compagnievoix.com/en/projects/creation/eve-3.
References
Argyle, M., and Dean, J. (1965). Eye-contact, distance and affiliation. Sociometry 28, 289–304. doi:10.2307/2786027
Aron, A., Aron, E. N., and Smollan, D. (1992). Inclusion of other in the self scale and the structure of interpersonal closeness. J. Personality Soc. Psychol. 63, 596–612. doi:10.1037/0022-3514.63.4.596
Bailenson, J. N., Blascovich, J., Beall, A. C., and Loomis, J. M. (2001). Equilibrium theory revisited: mutual gaze and personal space in virtual environments. Presence Teleoperators Virtual Environ. 10, 583–598. doi:10.1162/105474601753272844
Bailenson, J. N., Blascovich, J., Beall, A. C., and Loomis, J. M. (2003). Interpersonal distance in immersive virtual environments. Personality Soc. Psychol. Bull. 29, 819–833. doi:10.1177/0146167203029007002
Benford, S., Greenhalgh, C., Giannachi, G., Walker, B., Marshall, J., and Rodden, T. (2012). “Uncomfortable interactions,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (New York, NY, USA: Association for Computing Machinery), CHI ’12, New York, NY, USA, May 5 - 10, 2012, 2005–2014. doi:10.1145/2207676.2208347
Bergamo Meneghini, M. (2019). “Performance “Eve, dance is an unplaceable place”,” in IEEE GEM 2019, New Haven, CT, 18-21 June 2019, 6.
Bertrand, P., Gonzalez-Franco, D., Cherene, C., and Pointeau, A. (2014). ““the machine to be another”: embodiment performance to promote empathy among individuals,” in AISB 2014 50th Annual Convention of the AISB, London, UK, 1st-4th April 2014. 5.
Biocca, F. (1997). “The cyborg’s dilemma: embodiment in virtual environments,” in Proceedings Second International Conference on Cognitive Technology Humanizing the Information Age (Aizu-Wakamatsu City, Japan: IEEE Comput. Soc), 12–26, Aizu-Wakamatsu City, Japan, 25–28 August 1997. doi:10.1109/CT.1997.617676
Biocca, F., Harms, C., and Burgoon, J. K. (2003). Toward a more robust theory and measure of social presence: review and suggested criteria. Presence Teleoperators Virtual Environ. 12, 456–480. doi:10.1162/105474603322761270
Botvinick, M., and Cohen, J. (1998). Rubber hands ‘feel’ touch that eyes see. Nature 391, 756. doi:10.1038/35784
Carifio, J., and Perla, R. (2008). Resolving the 50-year debate around using and misusing Likert scales. Med. Educ. 42, 1150–1152. doi:10.1111/j.1365-2923.2008.03172.x
Cebolla, A., Herrero, R., Ventura, S., Miragall, M., Bellosta-Batalla, M., Llorens, R., et al. (2019). Putting oneself in the body of others: a pilot study on the efficacy of an embodied virtual reality system to generate self-compassion. Front. Psychol. 10, 1521. doi:10.3389/fpsyg.2019.01521
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. 2. New York: Routledge. doi:10.4324/9780203771587
Coleridge, S. T. (1817). “Chapter XIV,” in Biographia literaria (Chicago, Illinois: Poetry Foundation).
Collaço de Oliveira, E., Bertrand, P., Lesur, M. E. R., Palomo, P., Demarzo, M., Cebolla, A., et al. (2016). “Virtual body swap: a new feasible tool to Be explored in health and education,” in 2016 XVIII Symposium on Virtual and Augmented Reality (SVR). 81–89, Gramado, Brazil, June 21 2016 to June 24 2016. doi:10.1109/SVR.2016.23
Creswell, J. W., and Creswell, J. D. (2018). Research design: qualitative, quantitative, and mixed methods approaches. 5. Thousand Oaks, California: SAGE Publications, Inc.
Creswell, J. W., and Poth, C. N. (2018). Qualitative inquiry & research design: choosing among five approaches. 4. Thousand Oaks: SAGE Publications, Inc.
Cumming, G. (2011). Understanding the new statistics: effect sizes, confidence intervals, and meta-analysis. New York: Routledge. doi:10.4324/9780203807002
Cumming, G. (2014). The new statistics: why and how. Psychol. Sci. 25, 7–29. doi:10.1177/0956797613504966
De Greef, P., and Ijsselsteijn, W. A. (2001). Social presence in a home tele-application. CyberPsychology Behav. 4, 307–315. doi:10.1089/109493101300117974
Ermi, L., and Mäyrä, F. (2005). “Fundamental components of the gameplay experience: analysing immersion,” in Changing views: worlds in play. Editors S. de Castell, and J. Jenson (Vancouver, Canada: DiGRA), 14.
Felip, F., Galán, J., Contero, M., and García-García, C. (2023). Touch matters: the impact of physical contact on haptic product perception in virtual reality. Appl. Sci. 13, 2649. doi:10.3390/app13042649
Field, A. P. (2013). Discovering statistics using IBM SPSS statistics: and sex and drugs and rock ’n’ roll/andy Field. 4. Thousand Oaks, California: Sage.
Fulkerson, M. (2013). The first sense: a philosophical study of human touch. Cambridge, UNITED STATES: MIT Press.
Gallace, A., and Girondini, M. (2022). Social touch in virtual reality. Curr. Opin. Behav. Sci. 43, 249–254. doi:10.1016/j.cobeha.2021.11.006
Gallace, A., Ngo, M. K., Sulaitis, J., and Spence, C. (2012). “Multisensory presence in virtual reality: possibilities & limitations,” in Multiple sensorial media advances and applications: new developments in MulSeMedia (IGI global). 1–38.
Gallace, A., and Spence, C. (2010). The science of interpersonal touch: an overview. Neurosci. Biobehav. Rev. 34, 246–259. doi:10.1016/j.neubiorev.2008.10.004
Harpe, S. E. (2015). How to analyze Likert and other rating scale data. Curr. Pharm. Teach. Learn. 7, 836–850. doi:10.1016/j.cptl.2015.08.001
Henry, B., Moffitt, T. E., Caspi, A., Langley, J., and Silva, P. A. (1994). On the “remembrance of things past”: a longitudinal evaluation of the retrospective method. Psychol. Assess. 6, 92–101. doi:10.1037/1040-3590.6.2.92
Hoffman, H. (1998). “Physically touching virtual objects using tactile augmentation enhances the realism of virtual environments,” in Proceedings of the IEEE 1998 Virtual Reality Annual International Symposium (Cat. No.98CB36180) (Atlanta, GA, USA: IEEE Comput. Soc), 59–63, Atlanta, GA, USA, March 1998. doi:10.1109/VRAIS.1998.658423
Hoppe, M., Rossmy, B., Neumann, D. P., Streuber, S., Schmidt, A., and Machulla, T.-K. (2020). “A human touch: social touch increases the perceived human-likeness of agents in virtual reality,” in Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (New York, NY, USA: Association for Computing Machinery), CHI ’20, 1–11, New York, NY, USA, April 25-30, 2020. doi:10.1145/3313831.3376719
Ijsselsteijn, W. A., de Kort, Y. A. W., and Haans, A. (2006). Is this my hand I see before me? The rubber hand illusion in reality, virtual reality, and mixed reality. Presence Teleoperators Virtual Environ. 15, 455–464. doi:10.1162/pres.15.4.455
Jégo, J.-F., and Bergamo Meneghini, M. (2020). “Let’s resonate: how to elicit improvisation and letting go in interactive digital art,” in Proceedings of the 7th International Conference on Movement and Computing (Jersey City/Virtual NJ USA: ACM), Jersey City, NJ, USA, July 15-17, 2020, 1–8. doi:10.1145/3401956.3404194
Joris, E., and Vandebroeck, I. (2022). “Delirious Departures,” in ACM SIGGRAPH 2022 immersive pavilion (New York, NY, USA: Association for Computing Machinery). SIGGRAPH ’22, 1–2. doi:10.1145/3532834.3536222
Keenaghan, S., Bowles, L., Crawfurd, G., Thurlbeck, S., Kentridge, R. W., and Cowie, D. (2020). My body until proven otherwise: exploring the time course of the full body illusion. Conscious. cognition 78, 102882. doi:10.1016/j.concog.2020.102882
Kilteni, K., Groten, R., and Slater, M. (2012). The sense of embodiment in virtual reality. Presence Teleoperators Virtual Environ. 21, 373–387. doi:10.1162/PRES_a_00124
King, N. (2012). “Qualitative organizational research: core methods and current challenges,” in Qualitative organizational research: core Methods and current challenges (55 City Road: SAGE Publications, Inc.), 426–450. doi:10.4135/9781526435620
Lee, K. M. (2004). Presence, explicated. Commun. Theory 14, 27–50. doi:10.1111/j.1468-2885.2004.tb00302.x
Lombard, M., Ditton, T. B., and Weinstein, L. (2009). “Measuring presence: the Temple presence inventory,” in Proceedings of the 12th annual international workshop on presence. 15, Valencia, Spain, 12-15 May 2015.
Miles, J., and Shevlin, M. (2001). Applying regression & correlation: a guide for students and researchers. London; Thousand Oaks, Calif: Sage Publications.
Morie, J. F. (2007). Performing in (virtual) spaces: embodiment and being in virtual environments. Int. J. Perform. Arts Digital Media 3, 123–138. doi:10.1386/padm.3.2-3.123_1
Oh, C. S., Bailenson, J. N., and Welch, G. F. (2018). A systematic review of social presence: definition, antecedents, and implications. Front. Robotics AI 5, 114. doi:10.3389/frobt.2018.00114
Peck, T. C., and Gonzalez-Franco, M. (2021). Avatar embodiment. A standardized questionnaire. Front. Virtual Real. 1. doi:10.3389/frvir.2020.575943
Petitmengin, C., Remillieux, A., and Valenzuela-Moguillansky, C. (2018). Discovering the structures of lived experience. Phenomenology Cognitive Sci. 18, 691–730. doi:10.1007/s11097-018-9597-4
Petkova, V. I., and Ehrsson, H. H. (2008). If I were you: perceptual illusion of body swapping. PLOS ONE 3, e3832. doi:10.1371/journal.pone.0003832
Popat, S. (2016). Missing in action: embodied experience and virtual reality. Theatre J. Baltim. 68, 357–378. doi:10.1353/tj.2016.0071
Quaglia, J. T., and Holecek, A. (2018). “Lucid virtual dreaming: antecedents and consequents of virtual lucidity during virtual threat,” in 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (Tuebingen/Reutlingen, Germany: IEEE), Reutlingen, Germany, 18-22 March 2018, 1–72. doi:10.1109/VR.2018.8446546
Reiner, M., and Hecht, D. (2009). Behavioral indications of object-presence in haptic virtual environments. CyberPsychology Behav. 12, 183–186. doi:10.1089/cpb.2008.0141
Rietzler, M., Geiselhart, F., Frommel, J., and Rukzio, E. (2018). “Conveying the perception of kinesthetic feedback in virtual reality using state-of-the-art hardware,” in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’18 (Montreal QC, Canada: ACM Press), Montreal, QC, Canada, April 21 - 26, 2018, 1–13. doi:10.1145/3173574.3174034
Rogers, Y., and Marshall, P. (2017). Research in the wild. No. 37 in synthesis lectures on human-centred informatics (morgan & claypool).
Rostami, A., Rossitto, C., and Waern, A. (2018). “Frictional Realities: enabling immersion in mixed-reality performances,” in Proceedings of the 2018 ACM International Conference on Interactive Experiences for TV and Online Video - TVX ’18 (SEOUL, Republic of Korea: ACM Press), SEOUL, Republic of Korea, June 26-28, 2018, 15–27. doi:10.1145/3210825.3210827
Russell, J., Weiss, A., and Mendelsohn, G. (1989). Affect grid: a single-item scale of pleasure and arousal. J. Personality Soc. Psychol. 57, 493–502. doi:10.1037/0022-3514.57.3.493
Schubert, T., Friedmann, F., and Regenbrecht, H. (2001). The experience of presence: factor analytic insights. Presence Teleoperators Virtual Environ. 10, 266–281. doi:10.1162/105474601300343603
Slater, M. (2009). Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical Trans. R. Soc. B Biol. Sci. 364, 3549–3557. doi:10.1098/rstb.2009.0138
Slater, M., Pérez Marcos, D., Ehrsson, H., and Sanchez-Vives, M. V. (2009). Inducing illusory ownership of a virtual body. Front. Neurosci. 3, 214–220. doi:10.3389/neuro.01.029.2009
Slater, M., Spanlang, B., Sanchez-Vives, M. V., and Blanke, O. (2010). First person experience of body transfer in virtual reality. PLOS ONE 5, e10564. doi:10.1371/journal.pone.0010564
Slater, M., and Wilbur, S. (1997). A framework for immersive virtual environments (FIVE): speculations on the role of presence in virtual environments. Presence Teleoperators Virtual Environ. 6, 603–616. doi:10.1162/pres.1997.6.6.603
Souza, V., Maciel, A., Nedel, L., and Kopper, R. (2021). Measuring presence in virtual environments: a survey. ACM Comput. Surv. 54, 1–37. 1–163:37. doi:10.1145/3466817
Stevens, B., Jerrams-Smith, J., Heathcote, D., and Callear, D. (2002). Putting the virtual into reality: assessing object-presence with projection-augmented models. Presence Teleoperators Virtual Environ. 11, 79–92. doi:10.1162/105474602317343677
van Erp, J. B. F., and Toet, A. (2015). Social touch in Human–Computer interaction”. Front. Digital Humanit. 2. doi:10.3389/fdigh.2015.00002
Vidyarthi, J. (2012). Sonic cradle: evoking mindfulness through ‘immersive’ interaction design. Surrey, BC, Canada: Master’s thesis, Simon Fraser University.
Wilson, H. R. (2020). New ways of seeing, feeling, being: intimate encounters in virtual reality performance. Int. J. Perform. Arts Digital Media 16, 114–133. doi:10.1080/14794713.2020.1770531
Yarosh, S., Wang, X., and Yao, Y. (2022). Perceptions of visual and multimodal symbolic mediated social touch: role of technology modality, relationship, and task emotional salience. Int. J. Human-Computer Stud. 159, 102757. doi:10.1016/j.ijhcs.2021.102757
Zhang, Y., Wang, Y., and Liu, Q. (2022). “Touch the history in virtuality: combine passive haptic with 360° videos in history learning,” in 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). 824–825, Christchurch, New Zealand, March 12 2022 to March 16 2022. doi:10.1109/VRW55335.2022.00263
Keywords: virtual reality, touch, performance, behaviour, presence, embodiment, social touch, research in the wild
Citation: Desnoyers-Stewart J, Bergamo Meneghini M, Stepanova ER and Riecke BE (2024) Real human touch: performer-facilitated touch enhances presence and embodiment in immersive performance. Front. Virtual Real. 4:1336581. doi: 10.3389/frvir.2023.1336581
Received: 11 November 2023; Accepted: 29 December 2023;
Published: 13 February 2024.
Edited by:
Domna Banakou, New York University Abu Dhabi, United Arab EmiratesReviewed by:
Merel M. Jung, Tilburg University, NetherlandsMarco Fyfe Pietro Gillies, Goldsmiths University of London, United Kingdom
Copyright © 2024 Desnoyers-Stewart, Bergamo Meneghini, Stepanova and Riecke. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: John Desnoyers-Stewart, desnoyer@sfu.ca