Carolin Wienrich
Philipp Komma
Stephanie Vogt
Marc E. Latoschik- 1Human-Technology-Systems, Julius Maximilian University of Würzburg, Würzburg, Germany
- 2Human-Computer Interaction, Julius Maximilian University of Würzburg, Würzburg, Germany
Plenty of theories, models, measures, and investigations target the understanding of virtual presence, i.e., the sense of presence in immersive Virtual Reality (VR). Other varieties of the so-called eXtended Realities (XR), e.g., Augmented and Mixed Reality (AR and MR) incorporate immersive features to a lesser degree and continuously combine spatial cues from the real physical space and the simulated virtual space. This blurred separation questions the applicability of the accumulated knowledge about the similarities of virtual presence and presence occurring in other varieties of XR, and corresponding outcomes. The present work bridges this gap by analyzing the construct of presence in mixed realities (MR). To achieve this, the following presents (1) a short review of definitions, dimensions, and measurements of presence in VR, and (2) the state of the art views on MR. Additionally, we (3) derived a working definition of MR, extending the Milgram continuum. This definition is based on entities reaching from real to virtual manifestations at one time point. Entities possess different degrees of referential power, determining the selection of the frame of reference. Furthermore, we (4) identified three research desiderata, including research questions about the frame of reference, the corresponding dimension of transportation, and the dimension of realism in MR. Mainly the relationship between the main aspects of virtual presence of immersive VR, i.e., the place-illusion, and the plausibility-illusion, and of the referential power of MR entities are discussed regarding the concept, measures, and design of presence in MR. Finally, (5) we suggested an experimental setup to reveal the research heuristic behind experiments investigating presence in MR. The present work contributes to the theories and the meaning of and approaches to simulate and measure presence in MR. We hypothesize that research about essential underlying factors determining user experience (UX) in MR simulations and experiences is still in its infancy and hopes this article provides an encouraging starting point to tackle related questions.
Introduction
The construct of presence is strongly linked with user experience in Virtual Reality (VR) (e.g., Skarbez et al., 2018). The feeling of being there (i.e., virtual spatial presence) can probably be considered as a so-called hygiene factor (Wienrich and Gramlich, 2020). To a certain extent, it might be necessary to allow other VR potentials to become effective. It might be conceptualized similarly to the role of pragmatic quality within the field of user experience (Hassenzahl, Diefenbach, and Göritz, 2010). The emergence of presence is often determined by the allocation of attention to the virtual environment and the occlusion of the physical environment. Wirth et al. (2007) define this allocation/occlusion process as reference setting, using the (virtual or physical) environment as referential cues. Persons have a sense of virtual presence if attention is allocated to immersive factors, and the virtual environment is chosen as the primary reference frame. Under the umbrella of this understanding, plenty of models, measures, and investigations target the understanding of presence in virtual reality (e.g., Slater, 2009). However, virtual reality is only one possible variant of the mixed reality continuum (Milgram and Kishino, 1994). Other variants have less immersive features (at least considering the inclusive dimension of the Inclusive-Extensive-Surrounding-Vivid characteristics, short IESV-characteristics (Skarbez and Whitton, 2017), see below) and continuously interfere with the real physical space. Those variants that allow for fluent transitions between virtual and physical realities might alter the idea of reference setting using different referential cues as the environment (the space). The occlusion of the physical world cannot not be such a crucial criterion inducing a sense of presence beyond full immersive VR. Hence, questions arise about the interplay between virtual and physical referential cues and their consequences for corresponding outcomes (e.g., the sense of transportation, the sense of realism). The present work aims to analyze the construct of presence and the interplay of virtual and physical referential cues in the context of mixed realities (MR). To achieve this goal, we present 1) a short review of definitions, dimensions, and measurements of presence in VR, and 2) the state of the art views on MR. Furthermore, we 3) derive a working definition of MR implying new conceptual ideas for the reference setting beyond full immersive VR and environmental cues. In our opinion, in order to discuss spatial presence in MR, it must be assumed that environmental entities are detached as anchor cues. Based on these new ideas, we 4) identify three research desiderata, including research questions about the referential power of entities occurring in MR, the corresponding dimension of transportation (i.e., place-illusion in VR), and the dimension of realism (i.e., plausibility-illusion in VR) in MR. Finally, we 5) suggest an experimental setup to reveal the research heuristic behind experiments investigating presence in MR. In sum, the current work presents an alternate conceptual idea of reference setting in MR, which raises the question: Does spatial presence in MR refer to the sense of being anywhere (space-related, inside-out) or to the sense of being with something (object-related, outside-in) somewhere? The following presents a research heuristic to investigate the idea and resulting question. Since presence is the most investigated construct evaluating VR experiences, we raise no claim to completeness. Our considerations are limited to the subconstruct of virtual presence and current debates about the spatial component of presence in VR. Nevertheless, we hope to encourage a discussion of the meaning, measurement, and design of spatial presence in MR. We suggest that empirical studies that are described as paradigmatic convey the questions into a coherent set of assumptions, measurements, and useful design suggestions for further research.
Theoretical Background
Recent Views on Spatial Presence in Virtual Reality
Definition of Presence in the Sense of Spatial Presence in Virtual Reality
This paper considers the construct of presence in the context of technologically mediated realities such as VR, augmented reality (AR), augmented virtuality (AV), and MR. In general, presence refers to subjective perceptions and feelings occurring in those realities by different immersive factors. Many sub-constructs refer to the term of presence, such as social presence (e.g., Lee, 2004), co-presence (e.g., Slater, 1999), story presence (e.g., Brown et al., 2003), cognitive presence (e.g., Nunez and Blake, 2001), relational presence (e.g., Maguire and Connaughton, 2006), and spatial presence (e.g., Lee et al., 2004). Since considering all sub-constructs is beyond the scope, the present paper focuses on spatial-related definitions of virtual presence.
Almost all definitions of spatial presence refer to the spatial context in which the term should be used. Gibson (1966) wording of the experience of presence as " [...] the sense of being in an environment” has been the basis for the definition of spatial presence as “being there” (in Steuer, 1992, p.75). Sheridan (1992) referred for the first time to spatial presence - not a real place but a virtual presence - as “feeling like you are present in the environment generated by the computer” (Sheridan, 1992). Minsky (1980) directed the discussion to the subjective sensation and termed spatial presence as “the sense of being there,” referring to the most common use today (Skarbez et al., 2017, 2018). Spatial presence has also often been related to the sense of transportation (Lombard and Ditton, 1997). The researchers distinguish between three different types of transportation. Firstly, when the user is transported to another place, secondly, the transportation of another place and objects to the user, and thirdly, two or more communicators are transported to a shared place (Lombard and Ditton, 1997). In VR, spatial presence has been mainly connected to the first type of transportation, i.e., self-transportation. Diverse authors described the “being there” aspect of spatial presence with different words such as the illusion of non-mediation (Lombard and Ditton, 1997) or the place-illusion (Slater, 2009). In the current article, those spatial-related definitions of presence in VR are named virtual presence, i.e., the feeling of being in an environment generated by an immersive computer system. In contrast, spatial presence encompasses a wider meaning and refers to the sense of being in an environment, including artificial, semi-artificial, and real.
Further, immersion, or rather immersion factors, defined as objective system factors, influence the place-illusion. Primarily, immersion is described by the following characteristics (Slater and Wilbur, 1997; Skarbez et al., 2017):
• Inclusive (I) indicates the extent to which physical reality is shut out.
• Extensive (E) indicates the range of sensory modalities accommodated.
• Surrounding (S) indicates the extent to which this virtual reality is panoramic rather than limited to a narrow field.
• Vivid (V) indicates the resolution, fidelity, and variety of energy simulated within a particular modality.
All variants of MR can differ in the characteristics of extensiveness, surrounding, or vividness. However, only the variant of VR possesses the characteristic of inclusiveness. Consequently, when discussing the relation between immersion and virtual presence, the relationship between inclusiveness and virtual presence is meant.
For more discussion about different uses of the term immersion, see, e.g., (Skarbez et al., 2017; Wienrich and Gramlich, 2020).
The second aspect of spatial presence concerns the plausibility of the experience in VR. While the suspension of disbelief describes a general willingness to accept objects or events that are not physically real (Slater and Usoh, 1993), the plausibility-illusion refers to " […] the illusion that what is happening is real (even though you know that it is not real)." (Slater, 2009). In this view, virtual presence is defined as being there plus–the sense of being in the virtual world and (plus) feeling that the events are plausible within this world (Skarbez et al., 2017). The plausibility is often connected to the perceived action possibilities (Wirth et al., 2007) or the richness of interaction (Schubert, 2009). Skarbez et al. (2017) introduced coherence as a set of reasonable circumstances that influences the plausibility illusion. The authors have seen it as parallel to the role of immersion for the place-illusion in VR.
Models of the Emergence of Presence in Virtual Reality
Besides diverse approaches of definition, different models try to answer how the sense of presence emerges. Two-pole models assume that a VR application user always feels present in one of two environments, either the real environment or the virtual environment (Biocca, 2003). From the real to the virtual environment, movements on the spectrum are explained by increasing immersive factors of the virtual environment and the amount of attention paid to those factors, such as the field of view or the VR’s interactivity. The same is true the other way around, as sudden interruptions, and distractions from the real world can cause breaks in presence (BIP; Slater and Steed, 2000). Similarly, Wirth et al. (2007) described a two-level model. Different factors contribute to the construction of a primary egocentric reference frame (PERF). Persons have a sense of virtual presence if attention is allocated to immersive factors, and the virtual environment is chosen as the primary reference frame. Further, personal factors such as involvement and the suspension of disbelief impact the attentional shift (i.e., the shift of reference frame) from the real to the virtual world. Wirth et al. (2007) conceived the emergence of virtual presence as a binary sensation, although they, at least theoretically, argued for the possibility of consecutive sensations of virtual presence. Other models added a third pole by integrating the mental imagery space (Biocca, 2003). This pole represents fictitious environments created by imagination. According to this model, virtual presence is a continuous state influenced by the position on the three-dimensional spectrum. Thus, some emerging models regard virtual presence sensation as an all-or-nothing principle, others as a continuous state (for a detailed discussion, see Nunez, 2007). Latoschik and Wienrich (2021) proposed an alternative theoretical model describing how XR experiences, including the many variants of presence, emerge. Their model integrates plausibility (Slater, 2009; Skarbez et al., 2017) and coherence (Skarbez et al., 2017) as much more central states or conditions during an XR exposure. They further argue that “there is no plausibility illusion but merely plausibility” with plausibility being defined “as a state or condition during an XR experience that subjectively results from the evaluation of any information processed by [...] sensory, perceptual, and cognitive layers” (Latoschik and Wienrich, 2021). Hence, in their view there are no illusions of the different qualia but just qualia and states. However, in the scope of the current paper we adhere to the widely used illusion terminology in harmonization with the current literature.
Measuring the Dimension of Spatial Presence in Virtual Reality
Different researchers have proposed different operationalizations and related measure methods for presence in VR. However, probably the most common way to capture presence is by post-experience-questionnaires (PEQ, also referred to as post-immersion-questionnaire). PEQ are self-report questionnaires that are answered following a VR experience (e.g., Insko, 2003; Skarbez et al., 2017). The most commonly used questionnaires based on their citations (Schwind et al., 2019) are the presence questionnaire (PQ), the immersive tendencies questionnaire (ITQ; Witmer and Singer, 1998), the SUS presence questionnaires (SUS; Slater et al., 1998; Usoh et al., 2000, 1999), the igroup presence questionnaire (IPQ; Schubert et al., 2001; Schubert, 2003), and the ITCSense of presence inventory (ITC-SOPI; Lessiter et al., 2001). Although each questionnaire refers to slightly different scales, three dimensions are essential: immersion, transportation, and realism (Lombard and Ditton, 1997). As stated above, immersion is defined as objective system factors influencing the place-illusion (e.g., Skarbez et al., 2017; Wienrich and Gramlich, 2020). Thus, the dimension of transportation and realism is more interesting for the present work.
The Dimension of Transportation
The sense of transportation refers to the crucial dimension of virtual presence (Lombard and Ditton, 1997). All PEQ included transportation questions assessing the sense of “being there” (Steuer, 1992). Thus, transportation is closely linked to the place-illusion in VR. Although Lombard and Ditton (1997) distinguished between three different types of transportation, spatial-related definition presence in VR has been mainly connected to the first type, i.e., self-transportation to another place.
For example, the current version of the SUS by Slater et al. (2000) includes six questions, which can be answered on a 7-point Likert scale. The dimension of transportation is, e.g., measured with the “sense of being in the […] space” (Slater et al., 2000) and with the question, whether the user “think [s] of the [...] space more as images that [he] saw, or more as somewhere that [he] visited.” (Slater et al., 2000).
The Dimension of Realism
Following Skarbez and colleagues (Skarbez et al., 2017), realism refers to fidelity, including physical, functional, and psychological sub-categories (see Alexander et al., 2005). It depends on the consistency of the virtual experience. Thus, realism does not describe how well the virtual environment or experience resemble the physical reality, but the effectiveness of the plausibility-illusion. When objects, entities, events, or actions make sense in the place accepted as real (i.e., place-illusion), users would indicate a high sense of realism. “Coherence can be thought of as a superset of realism or fidelity.” (Skarbez et al., 2017, p.6). As immersion determines the place-illusion and the sense of transportation, coherence refers to a set of reasonable circumstances that influences the plausibility-illusion and the sense of realism. For example, the PQ by Witmer and Singer (1998) measures the dimension of realism on behalf of seven questions. e.g., by determining the degree, the user felt “confused or disoriented at the beginning of breaks or at the end of the experimental session” or “how well [the user] could identify sounds” during the experience (Witmer and Singer, 1998). According to the different approach of Latoschik and Wienrich (2021), they extended the significance of coherence. Within their model of XR experiences, coherence activations occur on every level of information processing, including sensory, perceptual, and cognitive, and they elicit all qualities of experiences solely based on the respective cues, e.g., including the place-illusion. However, in the current paper’s scope, we adhere to the widely used terminology in harmonization with the current literature.
Summary
In sum, presence in VR is mainly defined by two aspects - the self-transportation to another place (i.e., place-illusion, sense of being there) and the feeling that events at this place are real (plausibility-illusion, sense of being there plus). Focussing on technological-mediated experiences, most emerging models suggest two poles of the place-illusion–the real environment and the virtual environment (e.g., Slater and Steed, 2000). Others define the poles as two possible manifestations of the primary egocentric reference frame (PERF, Wirth et al., 2007) within which events and interactions become plausible. Most authors assume that the sense of being in one of the environments follows an all-or-nothing principle. At the same time, others conceive it as a continuous state with more or less presence in one of the environments (e.g., Biocca, 2003). Measurements focus on the transportation dimension assessing the effectiveness of the place-illusion, and the realism dimension assessing the plausibility-illusion effectiveness. The present contribution revisited this knowledge corpus for virtual experiences, allowing for a fluent transition between virtual and physical realities, such as many MR headsets with see-through functions or MR applications defined as continuous transitions between the poles (Milgram and Colquhoun, 1999). We particularly question the current view on the all-or nothing principle of reference setting in MR variants where users perceive virtual and physical cues simultaneously. Before discussion of a modified referential cue model, recent views on MR are presented to show why it is important to rethink the concept of presence beyond full immersive VR experiences.
Recent Views on Mixed Realities
There is no unified definition of MR (Speicher and Nebeling, 2019). The most well-known and most cited definition in academic research is that of Milgram and Kishino (1994). The authors formulate MR as “a particular subset of Virtual Reality (VR) related technologies that involve the merging of real and virtual worlds somewhere along the “virtuality continuum” which connects completely real environments to completely virtual ones.”. Another key feature of MR environments, according to Milgram and Kishino (1994), was that in “a Mixed Reality environment [...] objects are presented together within a single display”. Finally, the authors themselves defined their primary work as: "non-exhaustive examples of existing display systems in which real objects and virtual objects are displayed together.”. Milgram and Colquhoun (1999) reiterate the original notion of Milgram and Kishino (1994) by explicitly excluding the extrema on this continuum (“completely real” and “completely virtual”) from their MR definition. MR stated as such, therefore only includes environments where real and virtual content is being mixed. Further, Milgram and Colquhoun (1999) extend the conceptualization of MR by showing the mixed reality combination space, presenting different combinations of real and virtual content within the same display. They firstly show how different types of hybrid displays may be mapped onto the MR combination space, but also go on to “extend the concept somewhat” by describing a journey along with the combination space within a single application. They never explicitly state such a transition as an MR application, but use it as an example of why the terms “AR” and “AV” cannot always clearly distinguish different states on the virtuality continuum and formulate the need for the broader definition of MR.
Others conceived of MR as even broader by including non-technological mediated phenomena such as realities perceived in dreams or drug experiences. Examples are seen in the work of Hillstead (2017), Mann (2002), and Mann and Nnlf (1994). However, these ideas are out of the scope of this paper because the present considerations focus on technic-mediated perceptions of reality.
More recently, Speicher et al. (2019) analyzed the usage of MR in academia. Through expert interviews and a literature review, they tried to gather different views on the term MR. Researchers used the definition of Milgram and Kishino (1994) most widely. Other conceptualizations included MR either as a synonym for AR, a strong AR, a combination of AR and VR, a type of collaboration, or an alignment of environments. The authors concluded that MR could mean many different things, depending on the context. Further, they recognized the need for researchers to clearly describe their understanding of the term MR within the context of their work. For that purpose, Speicher et al. (2019) described five main MR dimensions: 1) the number of environments (one or many), 2) the number of users (one or many), 3) the level of immersion (not, partially, entirely), 4) the level of virtuality (not, partially, entirely), 5) the degree of interaction (implicit, explicit). In addition, they introduced input and output as lower-level dimensions. Although Speicher and colleague’s work contributes to the clarity of MR in academic research, transitions (combinations of real and virtual content) are not explicitly discussed. However, as introduced by Billinghurst et al. (2001), transitional interfaces or the MR headset’s see-through function cannot be classified with the dimensions. Transitional interfaces allow the interpolation between physical real and virtual environments, and they meet the definition of MR applications as introduced by Milgram and Colquhoun (1999). In addition, the dimensions remain unclear in their descriptions of space and interactions in MR, metaphors being essential for discussions about the meaning of the place-illusion and plausibility-illusion in MR.
Spatial Presence in Mixed Realities
Only a few scientific publications address the intersection between spatial presence and MR. Wagner et al. (2009), for example, compared three different MR applications. The applications, MapLense, a mobile AR system, TimeWarp, an augmented reality game, and MR Tent, were not classified by the authors. Schaik et al. (2004) examined a collaborative MR application, i.e., Dessert Rain. The authors argued that evaluating these MR applications using the standard methods established in VR scenarios (see above) is not necessarily meaningful. Despite the recommendation to use other factors such as ecological and cultural factors, they did not provide any solution to conceive spatial presence in MR.
Moreover, the work of Schaik et al. (2004) and Wagner et al. (2009), despite their scientific contributions, demonstrated, again, a problem in the research of MR applications. There is no unified definition of the term MR, and because of this scientific investigation collected under the term may vary considerably in content and or scope. Also, contributions such as that of Billinghurst et al. (2001) are mostly hidden to researchers because they lack MR as a label. The understanding of the term MR remains unclear, and the investigated MR applications incomplete. For example, to the author’s best knowledge, the spatial presence was not the object of research for interfaces allowing the interpolation between physical real and virtual environments as defined by Milgram and Colquhoun (1999).
Outline of Present Contribution
The emergence of virtual presence is often determined by the allocation of attention to the virtual environment and the occlusion of the physical environment. Wirth et al. (2007) define this allocation/occlusion process as reference setting using the (virtual or physical) environment as referential cues. However, virtual reality is only one possible variant of the mixed reality continuum (Milgram and Kishino, 1994). Other variants have less immersive features and continuously interfere with the real physical space. Those variants that allow for fluent transitions between virtual and physical realities question the interplay between virtual and physical referential cues and their consequences for corresponding outcomes (e.g., the sense of transportation, the sense of realism). The present work aims to analyze the construct of presence and the interplay of virtual and physical referential cues in the context of MR. In the following, we derive a working definition of MR implying new conceptional ideas for the reference setting beyond full immersive VR and environmental cues. In our opinion, detaching from environmental entities as anchor cues is the essential assumption to discuss the spatial presence in MR. Based on these new ideas, we identify three research desiderata, including research questions about the referential power of entities occurring in MR, the corresponding dimension of transportation (i.e., place-illusion in VR), and the dimension of realism (i.e., plausibility-illusion in VR) in MR. Finally, we suggest an experimental setup to reveal the research heuristic behind experiments investigating presence in MR. In sum, the current work presents an alternative conceptional idea of reference setting in MR which raises the question: Does spatial presence in MR refer to the sense of being anywhere (space-related, inside-out) or to the sense of being with something (object-related, outside-in) somewhere? It also presents a research heuristic to investigate the idea and resulting question.
Working Definition of Mixed Realities
Scope
Similarly to Mann (2002) and Hillstead (2017), the scope of the present work refers to the role of spatial presence concerning their fourth area, i.e., computer-generated realities. The role of spatial presence in the first order, second-order, or mediated realities such as the sense of spatial presence during dreams, reading books, or watching movies are outside of the scope.
Defining the Modified Reality-Virtuality-Continuum
In the following of the present work, the working definition of MR is considered as concrete computer-generated virtual reality traversing the modified Reality-Virtuality-Continuum (short mRVC). Similar to the continuum defined by Milgram and Colquhoun (1999), the poles of the mRVC are defined as natural or physical reality (R) and virtuality (V). While the poles R and V represent abstract and theoretical forms of reality, the positions between them refer to concrete forms of reality illusions (Figure 1). Each of these concrete forms of reality illusion defines a mixed reality illusion at one time point. MR must include a merging of real and virtual qualities.
FIGURE 1. The upper panel illustrates the modified Virtual-Reality-Continuum (mRVC). The continuum is based on Milgram and Colquhoun (1999) continuum with the addition of real and virtual entities possessing a certain degree of referential power. The bottom panel illustrates that entities can become referential cues determining, in turn, the selection of the referential frame and the emergence of presence in MR.
In addition to Milgram and colleague’s definitions, firstly, each reality illusion includes entities reaching from real to virtual manifestations at one time point. Entities can refer to spatial environments (i.e., space-based) or concrete objects (i.e., object-based). Secondly, referential cues determine the selection of the reference frame, e.g., allocentric vs egocentric and inside-out vs outside-in. Cues are stemming from the presented entities themselves but also from introduction texts or other framing experiences. We only refer to referential cues stemming from the entities presented as real or virtual manifestations in an MR experience. Thus, spaces can be entities and can serve as cues simultaneously. In previous views, for example, virtual objects represented in virtual spaces refer to VR. Real objects represented in virtual spaces refer to an AV and virtual objects in real space to an AR. Based on the views about virtual presence presented above, the sense of virtual spatial presence has been implicitly anchored in the virtual space. Thus, the virtual spatial environmental entity (space-based) is the cue determining the frame of reference. However, assuming the spatial environmental entity is the cue determining the frame of reference reduces convincibility in MR (Figure 1). Why should the virtual environment in AV serve more likely as an anchor cue than the real object user interacting with? Thus, the view presented here overcomes the distinction of environments and objects by introducing entities possessing different degrees of referential power.
The referential power is defined as a weight indicating the probability of each entity or a class of entities (such as the spatial environment) to be selected as a referential cue. These probabilities are not independent of each other but are context-sensitive, and are a result of a given spatial configuration of entities and the location of the user relative to these entities, i.e., her current perspective she has on the entities. For example, an egocentric outside-in view of an entity that does not convey any self-location - imagine a manipulation of a CAD object of an engine part in front of the user - will most likely not elicit a space-based sensation of a being-there. In contrast, simple entities resembling floor tiles, geometrically arranged in congruence with a user’s perceived floor will most likely elicit a space-based sensation. When multiple entities are perceivable by the user, the question becomes how their referential power compares, how strong the respective cues are in a relation, how congruent they are to each other, and–in the case of the various XR variants defined by the mRVC, the ratio between the real physical and the simulated entities and their relative coverage of a user’s field of view.
It follows from the preceding that spatial environmental entities and objects theoretically can serve as referential anchor cues but that the actual effect is dependent on a combination of various conditions as given for a specific scene and entity configuration. In our opinion, detaching from environmental entities as anchor cues is the essential assumption to discuss spatial presence in MR. From this point of view, we discuss in the following subsection how we can operationalize the degree of real or virtual entity manifestations of MR experiences as one possibility to determine objectively the position of the mRVC. Subsequently, we discuss three research desiderata addressing the referential power of entities.
Defining Positions on the mRVC
Although the authors agree with Milgram and Colquhoun’s (1999, p. 8) assertion that " [...] determining whether an image should be considered augmented reality or augmented virtuality is also not necessarily a matter of simply summating the respective areas of real and virtual images […]", the mRVC visualized areas defined by the amount of real and virtual picture portions at one time point (Figure 2, upper panel). Figure 2 (lower panel) shows a stepwise interpolation of Varjo demonstration on the mRVC (Varjo, 2019). The mRVC uses a naive percent-visualization of real and virtual picture portions, with 100% referring to reality. A higher portion of cues engendering the perception of real entities leads to a more left-side position on the mRVC. In contrast, cues engendering the perception of virtual entities lead to a more right-side position on the mRVC. Notably, experiences refer to a continuous stream of feelings, thoughts, and actions (Kahneman et al., 1999). Hence, this position finding does not reflect user’s experiences but helps to define and classify reality illusion and thus support systematic testing. Further, such a definition cannot capture subjective attentional allocation to particular areas of interest (e.g., interaction areas). Nevertheless, the possibility to define and classify reality illusions might be one crucial step in enabling a systematic investigation of the entity’s referential power and spatial presence in MR.
FIGURE 2. Illustrates the determination of reality (blue) and virtuality (red) portions on the mRVC being useful for experimental setups testing assumptions about the spatial presence in MR.
In the following, we consider the relationship between the main aspects of presence in VR, i.e., the place-illusion, and the plausibility-illusion, and the entities occurring in the mRVC, i.e., space-based and object-based, to discuss the concept, measures, and design of spatial presence in MR by considering the referential power of the different entities.
Research Desiderata and Research Questions About Spatial Presence in Mixed Realities
Considering the adaptation of spatial presence for MR, three research desiderata arise concerning: the reference frame (short: RD1), the corresponding dimension of transportation (short: RD2), and the dimension of reality in MR (short: RD3). Each desideratum includes three classes of research questions; questions concerning the construct (short: C-RQ), the ones concerning the measurement (short: M-RQ), and the ones about the design challenges (short: D-RQ). Notably, the desiderata are theoretically derived from knowledge about the spatial presence in VR presented earlier. Further, the desiderata and questions are based on the working definition of MR given above. Of course, other approaches might also be possible. Thus, we raise no claim to completeness but hope to encourage discussion of the meaning of spatial presence in MR.
RD1: Research Desiderata Concerning the Reference Frame in Mixed Realities
More or less directly, considerations about the presence in VR insert the virtual space as an anchor or reference. Often, the real environment serves as a kind of comparison-reference that should enable users to report their virtual experience (e.g., in the IPQ the user is asked whether “the virtual world seemed more realistic than the real world.”, Schubert et al., 2016). Nevertheless, the virtual and real environments are considered as opponents–only one can be the only (or at least the dominant) reference to judge the experience’s plausibility. Others framed the selection of reference as an attentional allocation process. If the virtual environment obtains (more) attention, users feel (more) present in it. In contrast, MR allows for a fluent transition between virtual and physical realities questioning the view of opponent realities. On the other hand, overcoming the distinction between environments and objects (by introducing entities possessing different degrees of referential power) leads to the RD1, including the following research questions:
C-RQ 1.1: Do users have/need a dominant reference frame in MR experience?
C-RQ 1.2: Which entity determines the reference frame in MR experience?
M-RQ 1.1: How can it be measured whether users insert or alternate a reference frame during the MR experience?
M-RQ 1.2: How can it be measured which entity determines the reference frame during the MR experience?
D-RQ 1.1: Which MR experiences should offer a dominant reference frame, which an alternating one?
D-RQ 1.2: Which design implications result from a dominant or alternating (entity) reference frame?
The authors identify three views on the RD1 described in the following.
1) One view to answer the research questions might be the view of Wirth et al. (2007). The authors assumed the place-illusion, i.e., the self-positioning in a real or virtual environment, determines the primary egocentric reference frame (PERF). Following this assumption, the user’s positioning in a real space would lead up to the user’s perception as part of the real space (space-PERF = R). In contrast, positioning the user in a virtual space would lead up to the user’s perception as part of the virtual space (space-PERF = V). However, in mixed realities, besides the self-positioning, the positioning of the objects further determines the perception of the illusion. Consequently, the positioning of objects in the real space would also lead up to the perception that they are part of the real place (left side of the mRVC, see Figure 1). In contrast, the positioning of objects in the virtual space would lead up to the perception that they are part of the virtual place (right side of the mRVC, see Figure 1). Thus, one view might be that an MR illusion emerges by integrating virtual objects into the real space (space-PERF = R). Then the user accepts the virtual objects as a part of the real space. In contrast, another MR illusion emerges by integrating real objects into the virtual space (space-PERF = V). Then the user accepts the real objects as a part of the virtual space. In this view, space-based entities would possess more referential power, become more likely to be referential cues, and determine the PERF more likely than object-based entities.
2) Another view to answer the research questions is that the object-based entities would possess more referential power, become more likely to be referential cues, and determine the PERF more likely than space-based entities. Then space-based entities are only an additional cue that indicates the concrete reality (i.e., position on the mRVC). Real object-based entities would then lead up to the user’s perception as part of the real place (object-PERF = R). Some MR-illusion emerges by integrating virtual spaces into the object-R-PERF. Then the user accepts the virtual space as a part of the real object-based - place-illusion. In contrast, virtual objects would lead up to the user’s perception as part of the virtual place (object-PERF = V). The MR illusion then emerges by integrating a real space into the object-V-PERF. Then the user accepts the real space as a part of the virtual object-based-place-illusion.
Both views would assume one dominant reference frame, either the quality of space-based or object-based entity. The former view would result in similar considerations about MR measures and design as it is in VR. In contrast, the latter view would result in new considerations about measures and design in MR. However, established spatial-related presence measures in VR assess transportation dimension and ask more or less after the degree of being in the experienced VR. In our opinion, transportation plays a unique role in experiencing MR (see 4.2). Instead of referring to the sense of transportation, we would argue for including direct questions about the reference frame or rather about the referential power of entities. One operationalization could be to assess the user’s expectations or breaks in expectations relating to the place-illusion emerged by a real or virtual space or the place spanned by the object-based entities. Those questions change the balance when a real and a virtual (space-based, object-based) PERF would make a huge difference.
It would be essential for design to know if users form a PERF and if the space-based or the object-based entities determine the reference frame. Then the MR application should include easy to perceive cues indicating clearly to a (space, object) R-PERF or a (space, object) V-PERF. For example, if users form a space-R-PERF, objects should cast a shadow in a way that would be plausible under real light conditions.
3) A third view of answering the research questions might be that users alter the PERF continuously or just do not have one. Then probably, the amount of real and virtual picture portions during the experience determine the position on the mRVC. In this view, space could be seen as the sum of its objects. The higher the number of virtual entities (including space as one cue), the more likely the user experiences a position on the right side of the mRVC (Figure 1). Then users would expect either real or virtual entity behavior and would not experience breaks by transitions. Measuring the absence of an experience is challenging (see below). For design, in contrast, this possibility enables much openness.
RD2: Research Desiderata Concerning the Dimension of Transportation in Mixed Realities
The dimension of transportation refers to the place-illusion in VR (i.e., the sense of being there, Steuer, 1992). Although Lombard and Ditton (1997) distinguished between three different types of transportation, spatial presence in VR has been mainly connected to the first type, i.e., self-transportation to another place. However, in MR, self-transportation might only be one aspect of transportation (egocentric inside-out). The second type, addressing the transportation of another place and objects to the user, probably draws more attention to objects-based entities (egocentric outside-in). The sense of spatial presence might also be determined by the feeling of sharing the MR space with virtual or real objects emphasizing the RD2, including the following research questions:
C-RQ 2.1: Does the sense of spatial presence in MR include the first and the second type of transportation?
C-RQ 2.2: Does the sense of transportation in MR refer to a change of reference frame?
M-RQ 2.1: How can the first and the second type of transportation be measured?
M-RQ 2.2: How can the relation of transportation and a change of reference frame be measured?
D-RQ 2.1: If necessary, how can techniques that enable the first or second type of transportation be designed?
D-RQ 2.2: If necessary, how can techniques that support a change of reference frame be designed?
D-RQ 2.3: Are similar transport metaphors as used in VR appropriate in MR?
The authors detected three views on the RD2 described in the following.
1) The established self-transportation probably refers to the views regarding the rest frame described above. In VR, transportation is closely related to a change of space within the VR experience (e.g., from virtual Europe to virtual Africa) or between the real and the virtual space (e.g., at the beginning of an experience when putting on the head-mounted display (HMD)). In MR, the sense of transportation can be similarly described when users select one dominant spatial (space-) reference frame. Then metaphors indicating a change of space would be similarly appropriate (e.g., portals). Thus, similar questions measuring spatial transportation might also be valid.
2) However, a reference framed by object-based entities draws attention to techniques that transport others to the user. Including the second type of transportation might refer to the feeling of sharing the room with objects. If this dimension is relevant for spatial presence in MR, measures should also draw attention to the other transportations. Questions asking for the sense of being might be supplemented by questions asking for the being with object-based entities or even space-related ones. Similarly, design considerations could think about transition techniques of object-based entities (e.g., fading in objects, replaying objects). Those techniques should correspond to the context to support a fluent experience. Vice versa, the affordance of the transportation technique might influence the set of the reference frame (if users need/have one). Thus, the affordance of transportation techniques might offer a vast design space for MR experiences. The time course of transportations might also be exciting in MR. Affordances of transportation at the beginning of the experience (when the user is forming expectations) might be different from transportations during the experiences (when users update expectations).
3) When users do not form a reference frame, the transportation dimension might be doubted in a general sense. Then, transportation (i.e., the place-illusion) might not be an appropriate metaphor for the user’s experiences in MR, and consideration of new metaphors would probably be necessary.
RD3: Research Desiderata Concerning the Dimension of Realism in Mixed Realities
The dimension of realism refers to the plausibility-illusion (i.e., the sense of being there plus; Skarbez et al., 2017). In VR, plausibility addresses mainly objects, entities, and events that make sense in the place accepted as real (PERF). MR includes incongruences between the space-based and object-based entities by definition emphasizing the RD3, including the following research questions:
C-RQ 3.1: What can plausibility mean in MR defined by incongruences between the space-based and the object-based entities?
C-RQ 3.2: Which user expectancies or sets of (in-)coherence shape the plausibility-illusion in MR?
M-RQ 3.1: How can the plausibility-illusion (realism dimension) be measured in MR?
M-RQ 3.2: How can the presence of an (in-)coherent experience be measured in MR?
D-RQ 3.1: Which implications arise for the plausibility design in MR?
D-RQ 3.2: Which implications arise for an (in-)coherent design in MR?
The authors identify the following three views on the RD3.
1) The sense of plausibility probably refers to the views regarding the rest frame described above. In VR, the plausibility-illusion is closely related to the extent to which an entity makes sense in the place that has been accepted as real (PERF). In MR, the sense of plausibility can be similarly described when users select one dominant spatial (space-) reference frame. Then incongruent entities (i.e., virtual objects in a real space or real objects in a virtual space) would be perceived as plausible if they behave coherently to the dominant place-illusion (i.e., the selected space-PERF). For example, in AR, virtual objects should fall downwards and not hang in the air. Measures and design considerations might be similar to corresponding considerations in VR.
2) In contrast, when object-based entities determine the reference frame, plausibility should be evaluated relative to the object-based entity. Then spaces would be perceived as plausible if they behave coherently to the dominant object-reference (i.e., the selected PERF). Measures should probably assess coherence and plausibility relations between the object-based entities. Designers might draw attention to those relations when the goal is a fluent and coherent experience, even during transitions on the mRVC.
3) Suppose users do not need or have any reference frame. In that case, plausibility might be considered in a wider way or even in a different way. Particularly incoherent sets of circumstances might define the plausibility of MR. Measurements and design considerations should also reflect such a definition.
In sum, in MR, object-based entities might play an additional role in MR to the space-based entities in VR. Similar to in VR experiences, when users form a dominant reference frame, the question arises as to whether the space-based or the object-based entities determine it. Then, questions around the dimensions of transportation and realism could either be similar to VR or draw more attention to the object-based entity. Finally, users might not need or have a dominant reference frame or alternate it continuously. Then the construct of transportation could be doubted in general. The dimension of realism might also change since exceptionally incoherent sets of circumstances and not coherent sets might define the plausibility of MR.
The research desiderata and corresponding considerations lead up to the final question: Does spatial presence in MR refer to the sense of being anywhere (space-related, inside-out) or to the sense of being with something (object-related, outside-in) somewhere?
Experimental Setups to Test the Research Questions
Hitherto, the discussion reflected spatial presence in MR. Diverse experimental setups could find answers to the research questions. Presenting one experiment for each question goes beyond the scope of the present article. Thus, we present one example to reveal the research heuristic behind such experiments. We hope that empirical studies convey the questions into a coherent set of assumptions, measurements, and useful design suggestions.
Control for Confounding Factors
Since immersion, defined as objective system factors, influences the place-illusion in VR (e.g., Skarbez et al., 2017; Wienrich and Gramlich, 2020), it might also play a significant role in MR (except the characteristic of inclusiveness). One good practice in VR research is to hold confounding immersive factors constant such as the kind of display. For example, when researchers want to investigate the sense of agency in dependency of the virtual embodiment, the same HMD should be worn in all embodiment conditions to control undesirable impacts of the HMD. Similarly, experiments investigating merging real and virtual entities in MR should follow the same good practice. Thus, if researchers explore the feeling of spatial presence in MR, the display properties of the system should remain the same over the application runtime to avoid confounding immersive factors. Currently, there is only one way to use the same system to represent both fully immersive virtual environments, as well as the real environments, and each possible merging in between. This possibility is the use of see-through HMDs, i.e., fully immersive VR glasses, which are able to display the real environment through cameras.
Paradigmatic Investigation of RD1–The Reference Frame in Mixed Realities
The questions about the reference frame in MR might be the most significant one of our considerations presented above. Hence, we present a paradigmatic experiment concerning the first research desideratum in the following.
The task of participants might be to search and merge objects by color. For that, they have to move around, find objects, and bring them to an object with the same color. During the experiment, participants would experience at least six transitions on the mRVC. Each transition reflects a systematic variation, either of a space-based or object-based entity (independent variable). To control for order-effects, the order of transitions would be balanced. One order of transitions is shown in Table 1. Before and after each transition, questions including the set, or the change of reference frame would be assessed (first-order dependent variable). Notably, the question must be assessed without switching off the HMD due to the above-described reasons. In addition, the sense of transportation (i.e., sense of place-illusion) or the sense of realism (i.e., sense of plausibility-illusion) could be assessed (second-order dependent variables).
TABLE 1. Shows one order of transitions of the paradigmatic experiment.
The results of the experiments would bring primary answers to C-RQ 1.1 (Do users have/need a dominant reference frame in MR experience?) and C-RQ 1.2 (Does the space-based or object-based entities determine the reference frame in MR experiences?). Furthermore, the results would indicate how the sense of transportation (i.e., place-illusion in VR) or realism (i.e., plausibility-illusion in VR) are evaluated in MR.
Discussion
Aim of the Present Considerations
Plenty of considerations, models, measures, and investigations target the understanding of the sense of presence in VR. However, full-immersive virtual reality is only one possible variant of the mixed reality continuum (Milgram and Kishino, 1994; Milgram and Colquhoun, 1999). Other variants have less immersive features (are less inclusive) and continuously interfere with the real physical space questioning the applicability of the accumulated knowledge about virtual presence and corresponding outcomes. The current work presents an alternative conceptional idea of reference setting in MR which raises the question: Does spatial presence in MR refer to the sense of being anywhere (space-related, inside-out) or to the sense of being with something (object-related, outside-in) somewhere? It also presents a research heuristic to investigate the idea and resulting question.
Contribution of the Present Considerations
To achieve the aim, we 1) presented a short review of definitions, dimensions, and measurements of presence in VR, and 2) presented the state of the art views on MR. Furthermore, we 3) derived a working definition of MR implying new conceptional ideas for the reference setting beyond full immersive VR and environmental cues. In our opinion, detaching from environmental entities as anchor cues is the essential assumption to discuss the spatial presence in MR. Based on these new ideas, we 4) identify three research desiderata, including research questions about the referential power of entities occurring in MR, the corresponding dimension of transportation (i.e., place-illusion in VR), and the dimension of realism (i.e., plausibility-illusion in VR) in MR. Finally, we 5) suggest an experimental setup to reveal the research heuristic behind experiments investigating presence in MR.
Working Definition of Mixed Realities
The working definition of MR is considered as concrete computer-generated Virtual Reality traversing the modified Reality-Virtuality-Continuum (short mRVC). Similar to the continuum defined by Milgram and Colquhoun (1999), the poles of the mRVC are defined as natural or physical reality (R), and virtuality (V). The positions between them refer to concrete forms of reality illusions (Figure 1). Each of these concrete forms of reality illusion defines a mixed reality illusion at one time point. Each reality illusion includes entities reaching from real to virtual manifestations at one time point. Entities can refer to spatial environments (i.e., space-based) or concrete objects (i.e., object-based). In addition, referential cues stemming from the presented entities themselves determine the selection of the reference frame. Our view overcomes the distinction of environments and objects by introducing entities possessing different degrees of referential power. It follows from the foregoing that spatial environmental entities and objects theoretically can serve as referential anchor cues. (Figure 1). Moreover, the mRVC uses a naive percent-visualization referring to the portion of real or virtual pictures (Figure 2). The amount classification of real and virtual picture portions might be supportive for systematic testing.
Research Desiderata and Research Questions About Spatial Presence in Mixed Realities
According to the inclusion of the space-based and object-based entities possessing referential power in the definition of MR, the question arises as to whether the spatial presence in MR refers to the sense of being anywhere (space) or to the sense of being with something (object) somewhere?
Three research desiderata underpin this question: RD1 concerns the reference frame, RD2 regards the dimension of transportation (i.e., place-illusion in VR), and RD3 refers to the realism dimension (i.e., plausibility-illusion in VR). Each desideratum includes three classes of research questions, C-RQ concerns the construct, M-RQ regards the measurements, and D-RQ refers to design challenges.
The authors detected three views on the RD1. Each further influenced the views on transportation and realism. The first view assumed the place-illusion, i.e., the self-positioning in a real or virtual environment, determines the PERF. Following this assumption, the user’s positioning in a real space or virtual space determines the PERF selection. Then users would accept the real or virtual objects as a part of the space selected as space-PERF. The second view assumed that not the space-based entities, but the object-based entities determined the PERF. Then users would accept the real or virtual spaces as a part of the place selected as object-PERF. While both views assume one dominant reference frame, either space or the object, the third view assumed that users alter the PERF continuously or just do not have one. Then, space could be seen as the sum of its objects. In order to measure this, we would argue for including direct questions about the reference frame and the referential power of entities. One operationalization could be to assess the user’s expectations or breaks in expectations relating to the place-illusion emerging by a real or virtual space, or the place spanned by the object-based entities. Those questions turn the balance when a real and a virtual PERF would make a huge difference. Results might be essential for design. When users form a dominant PERF, either by space or objects, MR applications should include easy to percept cues indicating clearly to a (space, object-) R-PERF or a (space, object-) V-PERF. In the case of no dominant PERF, many possibilities occur for the design that might be different from VR design.
Similarly, three views for the dimension of transportation (i.e., place-illusion in VR) and realism (i.e., plausibility-illusion in VR) are discussed. In MR, the sense of transportation can be similarly described to VR when users select one dominant spatial reference (space-PERF). Metaphors indicating a change of place would be similarly appropriate (e.g., portals). Similar questions measuring spatial transportation might also be valid. However, a reference framed by the object-based entities draws attention to techniques that transport others to the user. Measures should also draw attention to the other-transportations. Similarly, design considerations could involve transition techniques of the different entities occurring in MR. For both views, the affordance of the transportation technique, and vice versa, might influence the set of the reference frame. Thus, the affordance of transportation techniques might offer a vast design space for MR experiences. In addition, the time course of transportations might also be an exciting topic for MR design. In contrast, the third view doubted the existence of a transportation dimension in the case users do not form a dominant reference frame. Thus, transportation (i.e., the place-illusion in VR) might not be an appropriate metaphor for the user’s experiences in MR, and consideration of new metaphors would probably be necessary.
In MR, the sense of plausibility can be similarly described to VR when users select one dominant space-based reference frame. Then even incongruent entities (i.e., virtual objects in a real spaces or real objects in virtual spaces) could be perceived as plausible if they behave coherently with the dominant space-PERF. In contrast, when object-based entities determine the reference frame, plausibility should be evaluated relative to them. Then spaces would be perceived as plausible if they behave coherently to the dominant object-PERF. We argued for measures and designs similar to VR in the former case and supplementing the assessment and design of coherence and plausibility relations between the entities in the latter. Suppose users do not need or have any frame of reference. In that case, the dimension of realism (i.e., plausibility-illusion in VR) might also change since exceptionally incoherent sets of circumstances and incoherent sets might define the plausibility of MR.
In sum, the present work contributes to the debate about what learnings and knowledge about presence and corresponding outcomes collected in the context of VR can be gainful for research in MR. The research questions considered guiding questions for experimental setups resulting in substantiation of spatial presence in MR. One paradigmatic experiment is described to illustrate a possible research heuristic for future work.
Limitations and Future Work
Five main limitations characterize the present contribution.
1) The first limitation refers to the focus on spatial presence. As mentioned above, presence is a broad construct, including, besides spatial presence (focus here), social presence (e.g., Lee, 2004), or cognitive presence (Nunez and Blake, 2001), for example. Furthermore, this paper considers spatial presence in the context of technologically mediated realities such as VR, AR, AV, and MR and excludes senses caused by dreams or drugs. Consequently, the discussions are restricted to the conceptional focus set by current debates of spatial presence in VR. The place-illusion and plausibility-illusion (Slater, 2009; Skarbez et al., 2017) and corresponding dimensions (Lombard and Ditton, 1997) combined with the idea of a primary (space-based) egocentric reference frame (Wirth et al., 2007) guided the presented research questions. Thus, scrutinizing current views on presence in VR was out of the scope here. Future work should examine other sub-constructs that refer to presence to consider their application for evaluating MR experiences. Similarly, revisited views on the concept of presence, such as the view of Latoschik and Wienrich (2021) should be incorporated in the future.
2) Second, the present considerations are limited to the working definition of MR. The working definition is based on the current scientific view on MR (e.g., Milgram and Kishino, 1994; Milgram and Colquhoun, 1999; Speicher et al., 2019). Other definitions, such as industrial views (Microsoft, 2020), were not in the scope of the present paper. In addition, Milgram and Colquhoun (1999) extended their definition of MR in the form of transitions between reality and virtuality (i.e., mixed reality combination spaces). This extension opens up the MR view, particularly for those allowing for transitions, such as MR headsets’ see-through function. A more radical interpretation of the transitions might be the inclusion of transition, i.e., interaction between the respective reality illusions on the continuum, as a necessary part of the reality form, which is called MR. Thus, MR is not an umbrella term but a specific type of reality similar to and consisting of AR and AV. MR’s distinct feature would be an interpolation between the extreme reality and virtuality on the continuum. An MR application is one in which within the runtime of one application and within one single display, the user traverses from reality to virtuality or vice versa at least once by stepping through instances of R, AR, AV, and VR frames. If you stopped the application at any point in time, the frame could be described as either R, AR, AV, or VR. Nevertheless, researchers need a concrete definition of MR and future work should examine whether the present considerations are valid for different MR definitions or dimensions, as introduced recently by Speicher and colleagues (2019), for example.
3) The present considerations include single-user MR applications. However, in the future, MR applications will probably allow for multi-user functions. Questions about joint reference frames and anchors will then arise. Moreover, MR applications allowing personalized viewing content, such as glasses showing personalized advertisements in the supermarket, cause questions about different reference frames for different users within the same application. Future work should examine whether the present considerations are valid and incorporate different views for those applications.
4) Although the present work considers implications for measuring and designing MR experience, the focus is laid on the conceptional implications. In addition, the research heuristic is only described by one paradigmatic experiment. Thus, empirical and practical validity is limited. In our opinion, implications for measures and design result from conceptional considerations, and practical research needs a starting point. However, future studies should focus on empirical data and practical significance.
5) Finally, the present work assumed that MR experiences aim for fluent experiences. Thus, plausibility can be linked to coherence (Skarbez et al., 2017). The zeitgeist focuses on intuitive use and technical usage without effort (Hartson and Pyla, 2012). However, intuitive use can also lead to uncritical usages and the psychological risks of misunderstanding technical devices and their power (Long and Magerko, 2020). The design of safety-critical systems already includes so-called intentional frictions or desirable difficulties (Druckman and Bjork, 1994). Thus, when using MR in safety-critical contexts (e.g., hospitals) disfluent experiences might also have relevant applications. Future work should examine plausibility and the link to coherence for those applications.
Conclusion
The present article analyzed the construct of spatial presence in MR. It presented an alternative conceptional idea of reference setting in MR, which raises the question: Does spatial presence in MR refer to the sense of being anywhere (space-related, inside-out) or to the sense of being with something (object-related, outside-in) somewhere? The current work also presented a research heuristic to investigate the idea and resulting question. Considerations about implications for the concept, the measurement, and the design of spatial presence in MR are encouraged. We hope further that empirical studies, described paradigmatically, convey the questions into a coherent set of assumptions, measurements, and valuable design suggestions. The construct of virtual presence is strongly linked with user experience in Virtual Reality (e.g., Skarbez et al., 2018). The feeling of being there (i.e., virtual spatial presence) can probably be considered as so-called hygiene factors (Wienrich and Gramlich, 2020). To a certain extent, it might be necessary to allow other VR potentials to become effective. It might be conceptualized similarly to the role of pragmatic quality within the field of user experience (Hassenzahl et al., 2010). For MR experiences, the research about essential underlying factors, hygiene factors, determining the user experience is still in its infancy. The present considerations might be a promising starting point.
Data Availability Statement
The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.
Author Contributions
CW was the main idea giver, made the theoretical concept, made the empirical concept, was the main writer PK was an idea giver, contributed to the theoretical and empirical concept SV supported in writing, contributed to the empirical concept ML contributed mainly to the idea and to the theoretical concept and supported writing a lot.
Funding
This publication was supported by the Open-Access Publication Fund of the University of Wuerzburg.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Alexander, A. L., Brunyé, T., Sidman, J., and Weil, S. A. (2005). From Gaming to Training: A Review of Studies on Fidelity, Immersion, Presence, and Buy-In and Their Effects on Transfer in Pc-Based Simulations and Games. DARWARS Train. Impact Group 5, 1–14.
Billinghurst, M., Kato, H., and Poupyrev, I. (2001). The Magicbook-Moving Seamlessly between Reality and Virtuality. IEEE Comp. Graphics Appl. 21 (3), 6–8. doi:10.1109/38.920621
Biocca, F. (2003). “Can We Resolve the Book, the Physical Reality, and the Dream State Problems? from the Two-Pole to a Three-Pole Model of Shifts in Presence,” in EU Future and Emerging Technologies, Presence Initiative Meeting (Venice, Italy: Citeseer), 13–69. doi:10.1162/105474603322761270
Brown, S., Ladeira, I., Winterbottom, C., and Blake, E. (2003). “The Effects of Mediation in a Storytelling Virtual Environment,” in International Conference on Virtual Storytelling (Springer), 102–111. doi:10.1007/978-3-540-40014-1_13
Druckman, D. E., and Bjork, R. A. (1994). Learning, Remembering, Believing: Enhancing Human Performance. Washington, DC: National Academy Press.
Gibson, J. J. (1966). The Senses Considered as Perceptual Systems. Washington: American Psychological Association.
Hartson, R., and Pyla, P. S. (2012). The UX Book: Process and Guidelines for Ensuring a Quality User Experience. Elsevier.
Hassenzahl, M., Diefenbach, S., and Göritz, A. (2010). Needs, Affect, and Interactive Products - Facets of User Experience. Interacting Comput. 22 (5), 353–362. doi:10.1016/j.intcom.2010.04.002
Hillstead, A. (2017). Simulating Psychedelic Therapy through Mediated Reality. Wellington: Victoria University of Wellington.
Insko, B. E. (2003). Measuring Presence: Subjective, Behavioral and Physiological Methods. Washington: American Psychological Association.
Latoschik, M. E., and Wienrich, C. (2021). Coherence and Plausibility, Not Presence?! Pivotal Conditions for XR Experiences and Effects, a Novel Model. New York: ArXiv Preprint.
Lee, K. M. (2004). Presence, Explicated. Commun. Theor. 14 (1), 27–50. doi:10.1111/j.1468-2885.2004.tb00302.x
Lee, S., Kim, G. J., Rizzo, A., and Park, H. (2004). “Formation of Spatial Presence: By Form or Content?,” in Proceedings of the 7th International Workshop on Presence, 20–27.
Lessiter, J., Freeman, J., Keogh, E., and Davidoff, J. (2001). A Cross-media Presence Questionnaire: The ITC-Sense of Presence Inventory. Presence: Teleoperators & Virtual Environments 10 (3), 282–297. doi:10.1162/105474601300343612
Lombard, M., and Ditton, T. (1997). At the Heart of it All: The Concept of Presence. J. Computer-Mediated Commun. 3 (2), JCMC321. doi:10.1111/j.1083-6101.1997.tb00072.x
Long, D., and Magerko, B. (2020). “What Is AI Literacy? Competencies and Design Considerations,” in Conference on Human Factors in Computing Systems - Proceedings (Honolulu: Association for Computing Machinery), 1–16. doi:10.1145/3313831.3376727
Maguire, K. C., and Connaughton, S. L. (2006). “Relational Presence and Distanced Interdependent Relationships,” in The Ninth International Workshop on Presence (Cleveland, OH: PRESENCE2006).
Mann, S. (2002). Mediated Reality with Implementations for Everyday Life. Cambridge, MA: MIT Press journal PRESENCE, 1.
Milgram, P., and Colquhoun, H. (1999). A Taxonomy of Real and Virtual World Display Integration. Mixed Reality: Merging Real and Virtual Worlds 1, 1–26. doi:10.1007/978-3-642-87512-0_1
Milgram, P., and Kishino, F. (1994). A Taxonomy of Mixed Reality Visual Displays. IEICE Trans. Inf. Syst. E77-D (12), 1. 10.1.1.102.4646
Nunez, D. (2007). A Capacity Limited, Cognitive Constructionist Model of Virtual Presence. Town: University of Cape.
Nunez, D., and Blake, E. (2001). “Cognitive Presence as a Unified Concept of Virtual Reality Effectiveness,” in Proceedings of the 1st International Conference on Computer Graphics, Virtual Reality and Visualisation, 115–118. doi:10.1145/513867.513892
Schubert, T., Friedmann, F., and Regenbrecht, H. (2001). The Experience of Presence: Factor Analytic Insights. Presence: Teleoperators & Virtual Environments 10 (3), 266–281. doi:10.1162/105474601300343603
Schubert, T. W. (2009). A New conception of Spatial Presence: Once Again, with Feeling. Commun. Theor. 19 (2), 161–187. doi:10.1111/j.1468-2885.2009.01340.x
Schubert, T. W. (2003). The Sense of Presence in Virtual Environments:. Z. für Medienpsychologie 15 (2), 69–71. doi:10.1026//1617-6383.15.2.69
Schwind, V., Knierim, P., Haas, N., and Henze, N. (2019). “Using Presence Questionnaires in Virtual Reality,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 1–12. doi:10.1145/3290605.3300590
Sheridan, T. B. (1992). Musings on Telepresence and Virtual Presence. Presence: Teleoperators & Virtual Environments 1 (1), 120–126. doi:10.1162/pres.1992.1.1.120
Skarbez, R., Brooks, F. P., and Whitton, M. C. (2018). “Immersion and Coherence in a Stressful Virtual Environment,” in Proceedings of the 24th Acm Symposium on Virtual Reality Software and Technology, 1–11. doi:10.1145/3281505.3281530
Skarbez, R., Brooks, Jr., F. P., and Whitton, M. C. (2018). A Survey of Presence and Related Concepts. ACM Comput. Surv. 50, 1–39. doi:10.1145/3134301
Slater, M. (1999). “Co-presence as an Amplifier of Emotion,” in 2nd International Workshop on Presence (Colcester, England: University of Essex).
Slater, M., McCarthy, J., and Maringelli, F. (1998). The Influence of Body Movement on Subjective Presence in Virtual Environments. Hum. Factors 40 (3), 469–477. doi:10.1518/001872098779591368
Slater, M. (2009). Place Illusion and Plausibility Can lead to Realistic Behaviour in Immersive Virtual Environments. Phil. Trans. R. Soc. B 364 (1535), 3549–3557. doi:10.1098/rstb.2009.0138
Slater, M., Sadagic, A., Usoh, M., and Schroeder, R. (2000). Small-group Behavior in a Virtual and Real Environment: A Comparative Study. Presence: Teleoperators & Virtual Environments 9 (1), 37–51. doi:10.1162/105474600566600
Slater, M., and Steed, A. (2000). A Virtual Presence Counter. Presence: Teleoperators & Virtual Environments 9 (5), 413–434. doi:10.1162/105474600566925
Slater, M., and Usoh, M. (1993). Simulating Peripheral Vision in Immersive Virtual Environments. Comput. Graphics 17 (6), 643–653. doi:10.1016/0097-8493(93)90113-n
Slater, M., and Wilbur, S. (1997). A Framework for Immersive Virtual Environments (FIVE): Speculations on the Role of Presence in Virtual Environments. Presence: Teleoperators & Virtual Environments 6 (6), 603–616. doi:10.1162/pres.1997.6.6.603
Speicher, M., Hall, B. D., and Nebeling, M. (2019). “What Is Mixed Reality?,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 1–15.
Steuer, J. (1992). Defining Virtual Reality: Dimensions Determining Telepresence. J. Commun. 42 (4), 73–93. doi:10.1111/j.1460-2466.1992.tb00812.x
Usoh, M., Arthur, K., Whitton, M. C., Bastos, R., Steed, A., Slater, M., et al. (1999). “Walking> Walking-In-Place> Flying, in Virtual Environments,” in Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques, 359–364. doi:10.1145/311535.311589
Usoh, M., Catena, E., Arman, S., and Slater, M. (2000). Using Presence Questionnaires in Reality. Presence: Teleoperators & Virtual Environments 9 (5), 497–503. doi:10.1162/105474600566989
Van Schaik, P., Turnbull, T., Van Wersch, A., and Drummond, S. (2004). Presence within a Mixed Reality Environment. Cyberpsychol Behav. 7 (5), 540–552. doi:10.1089/1094931042403145
Wagner, I., Broll, W., Jacucci, G., Kuutii, K., McCall, R., Morrison, A., et al. (2009). On the Role of Presence in Mixed Reality. Presence: Teleoperators and Virtual Environments 18 (4), 249–276. doi:10.1162/pres.18.4.249
Wienrich, C., and Gramlich, J. (2020). appRaiseVR - an Evaluation Framework for Immersive Experiences. I-Com 19 (2), 103–121. doi:10.1515/icom-2020-0008
Wirth, W., Hartmann, T., Böcking, S., Vorderer, P., Klimmt, C., Schramm, H., et al. (2007). A Process Model of the Formation of Spatial Presence Experiences. Media Psychol. 9 (3), 493–525. doi:10.1080/15213260701283079
Keywords: mixed reality, virtual-reality-continuum, spatial presence, place-illusion, plausibility-illusion, transportation, realism
Citation: Wienrich C, Komma P, Vogt S and Latoschik ME (2021) Spatial Presence in Mixed Realities–Considerations About the Concept, Measures, Design, and Experiments. Front. Virtual Real. 2:694315. doi: 10.3389/frvir.2021.694315
Received: 12 April 2021; Accepted: 28 September 2021;
Published: 25 October 2021.
Edited by:
Richard Skarbez, La Trobe University, AustraliaReviewed by:
Thomas W. Schubert, University of Oslo, NorwayFlorian Daiber, German Research Center for Artificial Intelligence (DFKI), Germany
Copyright © 2021 Wienrich, Komma, Vogt and Latoschik. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Carolin Wienrich, carolin.wienrich@uni-wuerzburg.de