What Does “Asymmetric VR” Mean? A Directed Content Analysis of Co-Located Use of VR by Users on Reddit

Technology is often framed in terms of space and time of use, such that a mobile phone is used to either send asynchronous messages or host synchronous conversations with remote others, while a classroom smartboard supports co-located, synchronous learning. As the technology becomes more immersive, the applicability of frameworks such as the time/place matrix becomes less clear. This study attempts to provide clarity by applying the Composite framework for Asymmetric VR (CAVR) to online forum descriptions of the use of immersive virtual reality (VR) in co-located groups. A related framework, Roles of Technology, is also explored; however, the authors argue the framework must be expanded before application of it beyond mobile technology. To better understand one possible solution to co-located VR’s isolation problem, a directed content analysis was conducted, exploring the discussion of co-located and asymmetric VR use on various subreddits. As a result, 11 patterns of co-located use of VR, including 8 which specify asymmetric VR designs, are identified. The researchers update the dimensions of CAVR according to these results, compare CAVR to another nascent framework, and offer suggestions for future work and applicability to practice. This work is intended to help guide future creation and research of asymmetric VR experiences through the deconstruction of existing asymmetric VR experiences to their key parts via the application of CAVR.

Introduction

Immersive virtual reality (VR) can be an isolating experience. Once the head-mounted display (HMD) is on, the physical world and other people around the user disappear, replaced by a digital world which, when externally visible, is most commonly shown to outsiders via a stationary 2D screen. However, just presenting the image of the virtual world without providing means of interaction inhibits the ability of those around a VR user to collaborate with them, despite their physical proximity. While VR creators in industry have focused on supplying outside awareness of the eye movements of the VR user as a way to make the experience “more social” (D. Brown, 2021), the authors of this paper seek to understand how current VR users cope with this isolation problem by exploring the discussion surrounding asymmetric VR, or the pairing of a single HMD with other interaction modalities. This effort aspires to validate a framework of asymmetric VR (Ouverson and Gilbert, 2021), through directed content analysis of posts from VR and gaming subreddits which center the co-located use of VR, including asymmetric VR experiences.

Researchers have been interested in asymmetries since as early as 1963, when Kenneth Arrow published work on the uncertainty and information asymmetry driving market behavior in health care. While research on information asymmetry has continued in management science (Bergh et al., 2019), economics (Basak, 2016), and social psychology (Sohrab et al., 2015), the concept of asymmetry has seen little attention in human-computer interaction and related, technology-centric fields. The attention it has seen comes from the disciplines of media spaces (Voida et al., 2008), ludology (Harris et al., 2016; Harris and Hancock, 2019) and more recently, from collaborative use of VR (Olin et al., 2020; Ouverson and Gilbert, 2021; Karaosmanoglu et al., 2021; Rogers et al., 2021).

In prior work, the disciplinary silos surrounding research related to asymmetric VR were broken down and their ideas merged to form the Composite framework for Asymmetric VR (CAVR) (Ouverson and Gilbert, 2021). Resulting from a scoping review of academic works which leveraged asymmetric VR, 18 foundational framing papers were identified and organized into a testable framework. Additionally, that work proposed cursory relationships between non-asymmetric frameworks and CAVR, including an update to the correlation between the level of asymmetry, the direction of dependence, and the timing of each member’s tasks (Ouverson and Gilbert, 2021; Thomsen et al., 2019). While the CAVR effort (Ouverson and Gilbert, 2021) charted a course for the framework’s relationship to other frameworks commonly used in asymmetric VR research, a validation of the findings of that scoping review remains.

The primary purpose of the present study was to identify the ways co-located users interact with VR and one-another while using immersive HMDs. The present work leveraged CAVR in a directed content analysis of posts and their comments on VR forums frequented by technology early adopters. The discussions (posts and comments) were coded for relevance, asymmetry per the dimensions of CAVR (Ouverson and Gilbert, 2021) and Roles of Technology (Olsson et al., 2019). This study served as a first step to validating the prior work and further sought to understand the patterns of asymmetry relevant to co-located VR use. The authors believe using patterns of asymmetry and symmetry of the five dimensions of CAVR to be more appropriate for discussing the differences between nuanced asymmetric VR experiences than using a scale of low to high asymmetry, which has thus far been common in studies of mixed reality and VR (Thomsen et al., 2019). These patterns of asymmetry will be formally discussed in the results section of this paper. Further, the results of this study are expected to aid in the design of innovative solutions to VR’s isolation problem.

The remainder of this paper is organized as follows: First, background for the frameworks used in coding is given. The materials and methods of the directed content analysis are then delineated, including the training methods and codebook leveraged in the analysis. Next, observations pertaining to the dataset and the patterns of asymmetry are presented and contextualized using the verbiage of the various subreddit discussions. Finally, a discussion of the study’s findings and limitations, a recommendation of future research areas, and the applicability of the findings to design of VR experiences close out the paper.

Background

In 2019, Ens and others (Ens et al., 2019) noted that mixed reality instantiated a new kind of groupware, a form of collaborative digital workspaces commonly studied in the field of Computer Supported Cooperative Work (CSCW). In their scoping review, they identified some trends in the ways people had begun using VR and some other device to collaborate, and they proposed an update to the way in which we understand groupware. While it was typical that the early years of research around group technology use followed groups in the same location to account for network limitations and experimental (read as: sometimes uncooperative) software, this trend was not seen for asymmetric VR, a form of mixed reality which merges non-immersive displays with immersive HMDs to facilitate access to a shared workspace.

Asymmetric VR is a newer phenomenon, especially when centered on the co-located group use of VR, entering the academic lexicon in 2017¹ (Gugenheimer et al., 2017). For the general public, the rise of this descriptor seems to have coincided with the release of the Oculus Development Kit (DK) and the creation of Keep Talking and Nobody Explodes (KTaNE), and it refers solely to the use of VR by a co-located group of players (Ouverson and Gilbert, 2021). Still, most of the academic attention has followed the application of a VR headset and a 2D interface in remote experiences, looking forward to the future of using VR to guide another person through a repair, for example. In doing so, the wisdom and creativity of the early adopters who have been trying to introduce naturally single-user hardware to their friends and family is largely ignored.

As identified in earlier work, the newness of the idea of Asymmetric VR has led to a dearth of accurate descriptions of the experience, its interaction paradigms, and its relationship to established frameworks (Ouverson and Gilbert, 2021). Following the lead of Johansen, who wrote in his 1988 chapter on groupware, “A good rule of thumb in the exploration of emerging technologies is to start with the users, even if there are only a few of them around,” (Johansen, 1988, p. 12) the researchers of the present work turned to Reddit to find accounts of VR use by early adopters.

As identified in previous studies, Reddit is a popular forum for the anonymous discussion of topics (De Choudhury et al., 2016). Additionally, Reddit is known as a source of technologically-savvy individuals, such as VR innovators (the first 2.5%) and VR early adopters (the next 13.5% in the theory of the diffusion of innovations; E. Rogers, 2003); therefore, the site was mined for discussions of co-located VR use by groups of people. This work seeks to validate the application of key frameworks (Roles of Technology (Olsson et al., 2019) and CAVR (Ouverson and Gilbert, 2021)) to the use of VR in social settings involving a single HMD. First is an overview of a framework, Roles of Technology (Olsson et al., 2019), which was identified as relating to CAVR (Ouverson and Gilbert, 2021), and a brief introduction to a recently published framework of asymmetric interaction in VR (K. Rogers et al., 2021). Following that is a summary of CAVR (Ouverson and Gilbert, 2021), the framework which served as the basis for the content analysis described in the materials and methods section of this paper.

The Roles of Technology

For VR, early work is suggestive of an end goal to use the technology to connect with remote others in a more naturalistic way, hence the focus on telepresence (Kraus and Kibsgaard, 2015) and the exclamation of co-located studies as a way of dealing with networking limitations (Ens et al., 2019). One symptom of this is that immersive VR has an isolation problem; that is, the use of the headset obstructs the senses of the VR user and does not natively support awareness for any bystanders to the interaction (Shao, 2019; Ouverson and Gilbert, 2021). This isolation occurs in the use of other technology, as well. For example, as smartphones have become more ubiquitous in daily life, researchers have noted a social phenomenon, phubbing, wherein a smartphone user snubs a co-located interactant in favor of checking their mobile device to engage with remote others (Olsson et al., 2019). This phenomenon, Olsson et al. (Olsson et al., 2019) claim, is proof that not only is commonplace technology built primarily for connecting remote users but also that the isolating designs are disruptive to ongoing social interaction and therefore need more attention.

Following a scoping review of literature in top HCI journals and conferences, Olsson and others (Olsson et al., 2019) determined that the attention of research on co-located interaction with technology is quite limited. Indeed, a majority of the studies which followed co-located use in a recent review of collaborative VR did so to ease networking requirements rather than as an intrinsically valuable research paradigm (Ens et al., 2019). Noting the dominance of work surrounding technologies which enable users to interact across distances, Olsson et al. (2019) describe the need to design technology which aims to enhance the interactions of people within a single space by inviting, facilitating, or enabling social interaction. It is expected that the design recommendations and roles of technology delineated by Olsson et al. (2019) would apply to the co-located use of VR.

According to Olsson and others (2019), remote use technology development has focused on enabling the interactions of users across distances, while co-located use technology should be designed with an overall goal of enhancing the interactions of people within a single space. Within the general area of “enhancement,” they identify three levels of the role of technology in interactions: inviting, facilitating, and enabling roles (Olsson et al., 2019). These levels were developed after a scoping review of studies on the use of mobile devices in co-located interactions but not the use of a VR headset in the presence of others. The authors of the present work recognize a need to understand how current users of VR respond to and interact with asymmetric VR and a need for a proposed solution to the “isolation problem,” which is already manifesting in and interfering with mainstream VR adoption. However, because the Roles of Technology framework was created for the use of mobile phones, applicability to the co-located use of asymmetric VR may be limited.

Frameworks of Asymmetric Virtual Reality

Attention to asymmetric VR and mixed reality has been expanding, and so too has the desire to guide research in the area. Both Ouverson and Gilbert. (2021) and Rogers et al. (2021) have proposed frameworks to help in this effort, and both are introduced herein.

In their systematic review of 25 asymmetric VR studies, Rogers and others (2021) use an a priori framework to make sense of prior research on asymmetric VR. This framework includes social and technological roots, but firmly plants itself in the study of games. Rogers et al. (2021) include Harris et al. (2016) framework of asymmetric gameplay (Harris et al., 2016) as a core component of their framework. In addition, they also incorporate the original MDA aesthetics categories, social asymmetry (including group flow), and the framework of shared control (Hunicke et al., 2004). Lastly, the conclusions Rogers et al. (2021) draw about the necessary changes to their a priori framework mirror the decisions which led to the framework utilized herein (i.e., CAVR), including that the Harris categories of ability and challenge are largely similar while the goal/responsibility category may need to be separated, and the idea that a balance (or imbalance) of power is extant in asymmetric VR designs. While it is not used in the content analysis, the Rogers et al. framework (2021) will be considered in the discussion of the results and implications for further design of asymmetric VR experiences.

Seeing as asymmetric VR largely consists of video games or gamified experiences (for examples of non-game applications, see Ibayashi et al., 2015; Menezes, 2020), establishing a framework rooted in the study of games, or ludology, is not inappropriate. As the technology supporting asymmetric VR advances and more workflows are designed to support the use of extended reality (including VR, mixed reality, and asymmetric VR), adoption across industries will continue to grow (Fillmore and Storr, 2020). And as the world continues to cope with the COVID-19 pandemic, there is an expectation that a larger percentage of the global workforce will be remote, even as companies open their physical offices, leading to an increase in hybrid teams (Mckendrick, 2021; Neeley, 2021). In other words, while asymmetric VR started gaining traction as a way share a single-user piece of technology (i.e., an HMD) with a group, its ability to connect folks to a single shared space using different digital means while, in the best-designed scenarios, also matching each interactant’s technological experience to their required tasks, indicates a need to incorporate research from multiple disciplines. More about this idea will be covered in the Implications for Practice section.

As such, the present work builds on CAVR, a framework of asymmetric VR which incorporates theory from CSCW, organizational psychology, and ludology, among other disciplines, seeking to merge sociological and technological perspectives of asymmetry. CAVR consists of five dimensions of asymmetry: transportation, spatial co-presence, informational richness, team interdependence, and balance of power (Ouverson and Gilbert, 2021). Table 1 delimits the definitions of each of these dimensions. The first three dimensions are very explicitly about the technology used in the experience, while the remaining two dimensions are proposed as having more to do with the interpersonal experience within the context of asymmetric VR. Team interdependence was derived from work centering on teamwork in virtual spaces (Marlow et al., 2016) and the element of dependence which had been previously established as existing outside the experience of asymmetry—though directly related to it (Harris and Hancock, 2019). Balance of power, the other interpersonal dimension of CAVR, deals in the control of information within the space.

TABLE 1

TABLE 1. Definitions of the five CAVR dimensions of asymmetry (Ouverson and Gilbert, 2021). Give space as per author request Tables 1, 3, 4, and 5 are a little compact, which negatively impacts readability. Could the rows be spaced a bit further from one another?

Materials and Methods

The primary goal of this study was to identify the ways co-located users in the real world (as opposed to in lab settings) interact with VR and one-another. Reddit was identified as a mediator for discussion around topics often centering niche and emerging ideas or phenomena, and thus a good way to observe VR use by way of early adopters of the technology communicating with one-another within the forum site. The researcher’s perspective bloomed from a desire to understand the naturally-evolving and innovative uses of VR within co-located groups.

As explained below and summarized in Table 3 (at the end of section 3), the research process occurred over roughly 10 steps. The data were coded in concordance with a type of qualitative content analysis called directed content analysis (Hsieh and Shannon, 2005) while data collection and analysis followed the qualitative research tradition (Creswell and Poth, 2018), moving through subsequently more abstraction until the data are interpreted and findings are accounted. As a result, the present work will extend CAVR (Ouverson and Gilbert, 2021) and provide insights to the design of experiences that will overcome VR’s stigma as an isolating technology. The role of the present section is to provide a high-level overview of the dataset and establish a background for the observations and categories which are presented in the results section of this paper.

Directed Content Analysis Methodology

This study followed the directed content analysis methodology described by Hsieh and Shannon. (2005), which requires the use of predetermined codes to tag a corpus of data, typically as a way to extend or refine an existing theory. As with other kinds of content analysis, the coders use a codebook to apply meaning and structure (or codes) to unstructured data. Using the recommended approach of highlighting relevant information and subsequently applying codes after extensive training with the codebook (Hsieh and Shannon, 2005), the research team produced a dataset of conversations (i.e., a corpus), broken into paragraphs, and coded according to their relevance to the research question. Unlike quantitative content analysis methodologies, the frequency of the codes will not be leveraged in answering the research question. Instead, the data will be synthesized and the relationships among the data will be described, per qualitative methodology styles of data analysis (Creswell and Poth, 2018).

Data Collection

We identified posts on Reddit related to co-located use of a single VR headset, specifically using the Reddit API to scrape posts and comments for the keywords “asymmetric VR” and “asymVR.” Pilot searches included “party VR,” but the phrase did not reveal any new, relevant content and was excluded from collection. These initial keywords were chosen based on previous literature about the topic of interest.

To aid in the creation of the codebook (with the intention to ensure thoroughness and appropriateness), the first author read each of the initial posts and comments (hereafter collectively referred to as discussions) and noted any new definitions or terms while considering the fit of the data to the anticipated codes. This process, noted in Table 3 as Step 1, resulted in the definition of terms for ease of understanding during coding as well as the identification of additional search terms, as shown in Table 2. Thus, the first author conducted an additional search and produced a larger, more comprehensive corpus.

TABLE 2

TABLE 2. Extended keywords or key phrases list, which was used in gathering data from Reddit via API.

Reddit’s API was used to collect the data, as described in De Choudhury et al. (2016), using the RedditExtractoR package (Rivera, 2019) to query Reddit per Step 2 of Table 3. The original set of data consisted of 12,450 comments on 279 posts between January 2014 and July 2020. RStudio (RStudio Team, 2021) was used to facilitate the cleaning of the data according to the date of the post (no data before 2014) and the relevance of the post. Relevance was determined based on whether the post discussed use of VR by more than one player; other posts were designated as false positives. As noted in Step 3 of Table 3, the removal of false positives and of entries which were written before 2014 (when the Oculus DK 2 was released), resulted in a corpus of 260 posts with 7,462 comments between January 2014 and July 2020.

TABLE 3

TABLE 3. The qualitative research process summarized.

The first round of cleaning surfaced a corpus of discussions which included at least one mention of VR. The second round (Step 4 of Table 3) was focused on each discussion’s relevance to asymmetric or co-located VR more specifically. To be included, the discussions were required to 1) relate to VR, 2) relate to asymmetric use of VR, either co-located or remote, 3) not be the result of a post calling for playtesters, 4) not be the result of posts requiring respondents to share what they liked about a video of gameplay in exchange for free access to the game, 5) not be the result of a post asking either a gameplay or a development question, and 6) include comments which discussed the original post. The full corpus consisted of 179 posts and 4,552 comments and can be found in the online repository, linked at the end of this paper.

The codebook was drafted by the first author and described the coding process, code definitions, and the levels of each code in the coding scheme. Additionally, definitions of terms which had been identified during data immersion (Step 1, Table 3) as necessary to the interpretation of the content were included at the end of the codebook. Identifiers which allowed the re-tracing of the data, specifically the URL, comment structure, and paragraph number, were also defined and listed as the first steps of the coding process. These identifiers acted as fail-safes to allow recovery in case of accidental shuffling of the corpus items.

In creating the codebook, the first author developed the codes and tested the interpretation of those codes with the middle two authors. All three worked independently to code a pilot sample of the data (Step 5, Table 3). After discussing inconsistencies in the coder’s interpretations, changes were made to the descriptions and levels of the codes to aid in further analysis. These three authors met weekly to discuss changes made to the codebook and identify challenges in the process before the codebook was finalized and coding could begin. As identified by Neuendorf. (2017), content analysis is an iterative process, and her suggestions and sample codebook served as the basis for this endeavor. The initial codebook, but not the gathered definitions, is included in Supplementary Appendix SA1.

Paragraphs, defined as “complete ideas” since written works on internet forums may or may not adhere to English grammar rules, formed the unit of analysis. To increase the trustworthiness of the coding process, the data were unitized prior to coding, and the researchers assigned relevance (irrelevant, social VR—meaning multiple headsets, or asymmetric VR—meaning a single headset) to the units before coding (Hsieh and Shannon, 2005). (An example of this unitization process is depicted in Figure 1.) Coders were asked to mark content that was not determinable as relevant with a 99, and only the content which addressed asymmetric or co-located VR use (as opposed to content in which the socialization is uniquely centered in the virtual space) was fully coded and analyzed.

FIGURE 1

FIGURE 1. An example of the unitization process for a single post: (A) Posts matching specific keywords and phrases, such as this one, were scraped using RedditExtractoR; (B) The scraped Reddit posts and associated data were compiled into a single spreadsheet; (C) The corpus was restructured so as to support human coding, and the data were unitized into paragraphs with one per row; (D) Manual coding was performed by trained researchers, who first identified the relevance of the entry. The full text of these images is available as part of the appendices and Supplementary Materials.

Each unit was coded according to the five dimensions of CAVR, resulting in a number of dimensional forms. A “dimensional form” was the unique grouping of symmetry and asymmetry as determined by the initial coding of the data. These forms were sorted by the similarities of their descriptions to generate the patterns of asymmetry, which will be examined in the results section of this paper.

During training, it was determined that only two of the authors would continue coding due to their levels of understanding of the content area. In directed content analysis, it is important that the coders are well informed about the content with which they are engaging, which is why iterative training is emphasized. As such, while three individuals worked to create the codebook, only two worked to code the corpus.

After about 15% of the data were coded, an inter-rater reliability (IRR) analysis was performed for each of the code categories (Step 7, Table 3). Utilizing the R package irr (Gamer et al., 2012), IRR was measured as iota (ι) or Conger’s Fleiss exact kappa coefficient (κ), depending on the number of categories per code (Conger, 1980; Janson and Olsson, 2004). For both ι and κ, the upper limit is 1, indicating perfect agreement. The lower limit is −1/(number of judges—1), or −1 in this case with two judges. Per Landis and Koch. (1977), a value >0.6 represents “substantial agreement” and a value >0.8 represents “almost perfect” agreement. Therefore, the first measured IRR was not yet satisfactory for codes corresponding to the CAVR dimensions (ι = 0.418) nor for the codes pertaining to the Roles of Technology (κ = 0.111).

Because the IRR indicated less than “substantial agreement,” the coders were retrained and the codebook was adjusted (Step 8, Table 3). The first author facilitated discussion of the coding inconsistencies, aimed at reaching a consensus in the understanding of the codes, but not necessarily a consensus in the data (as per the recommendation of Krippendorff, 2019). For example, if the coders could agree on the thought processes behind the coding decision, even if they did not agree on the code assignment, the code descriptions and decisions remained unchanged.

After this discussion, the coders adjusted their work on this 15%, IRR was remeasured to be κ = 0.113 for the Roles of Technology and ι = 0.788 for the CAVR dimensions. The decision was made to drop Role of Technology from the analysis, and the coders worked independently to complete the coding of the corpus (Step 9, Table 3). The final codebook is included in Supplementary Appendix SB1.

A random purposive sample (Collins, 2010) was leveraged during coding and analysis, as the corpus was purposefully identified and a randomly-selected subset of posts and their comments were then coded (Step 6, Table 3). The sampling resulted in a dataset of 73 posts and 2,925 comments unitized to 3,694 paragraphs. While the first author examined the posts for relevance prior to the coding of the data, the decision to unitize at the level of paragraph called for an additional data cleaning step. Therefore, the coders additionally flagged the data for relevance before moving forward with coding. As a result, 736 paragraphs were classified as relevant and fully coded by two coders. This sample is provided in Supplementary Table S1.

Analysis Method

Following the qualitative tradition, data analysis began in parallel with data collection and reduction (Creswell and Poth, 2018). As such, analysis began with observations of the dataset. These observations included a consideration of the ways in which redditors refer to the phenomenon as well as the apparent contexts of use associated with each described experience.

After analyzing the IRR associated with both frameworks, the decision was made to move forward with only an analysis of CAVR, rather than focusing on both CAVR (Ouverson and Gilbert, 2021) and the Roles of Technology (Olsson et al., 2019). Thus, we focused on the following research question: What types of asymmetry are present in co-located VR use? To answer this question, the corpus was coded according to the CAVR dimensions of asymmetry, which were identified in a scoping literature review based partly on work in ludology and media spaces (Ouverson and Gilbert, 2021). The coders examined each relevant paragraph and determined which, if any, CAVR dimensions of asymmetry were present. Coders were instructed to leave blanks when they could not determine whether or not there was a specific asymmetry for a dimension.

It should be noted that an asymmetric VR experience may include symmetries in one or more of the CAVR dimensions, and that the symmetry or asymmetry identified by coders is meant to relate to the intention of the design and perceived experience of that design, as reported on VR and gaming subreddits. Therefore, while the experience may involve a movement through symmetry and asymmetry in each CAVR dimension, coding examined moments-in-time and reported perceptions; as such, a single game may belong to multiple patterns. Also, mentions of “symmetry” may or may not reflect a truly symmetric manifestation of a given dimension.

After the corpus was coded, the first author again examined the content to understand the experiences represented by each dimensional form, or unique combination of CAVR dimension asymmetries and symmetries (Step 10, Table 3). Still adhering to the qualitative research tradition, the first author began an iterative process of analyzing the content associated with each dimensional form. Each dimensional form (or grouping of codes per CAVR dimension) was given a description, associated to the content with which it was related during the initial coding stage.

After categorizing the dimensional forms, the first author grouped them according to the similarity of their descriptions, and the distinct pattern categories were named. When a dimensional form fit more than one pattern, or the descriptions were not distinctive enough to facilitate sorting, the content of the post or comment was examined to determine, first, whether the coding was accurate. If the coding was accurate, the first author used the content associated to the dimensional form to determine which pattern best fit each unit. As a result, multiple dimensional forms are not uniquely allocated to individual patterns (see Supplementary Table S2). The final patterns are named and discussed in the following section, along with a qualitative description of the sample and examples of experiences coded as having symmetric or asymmetric CAVR dimensions (i.e., transportation, spatial co-presence, informational richness, team interdependence, and balance of power).

Results

First, observations from the initial data exploration are presented. Then, the patterns of asymmetry, as well as a description of the differences between these patterns, are delineated, answering the paper’s research question. The paper concludes with a discussion of the results, limitations, and future directions, as well as a summary of the implications for the design of VR experiences which are meant to be used in co-located groups.

Asymmetric Virtual Reality Through Redditor’s Eyes

In the initial review of the data, it quickly became clear that there are a multitude of ways to refer to the idea of using VR to play a game with a non-VR user. For example, “asymmetrical multiplayer,” “asymmetric VR,” “asym VR,” (also spelled asymm and assym), “asymmetrical multi,” “flat + VR,” “couch co-op VR,” and “pancake multiplayer,” all refer to the use of VR and some other interaction paradigm in a co-located group. For remote play, especially on posts which occurred prior to March 2020, a distinction from asymmetric VR (which is traditionally co-located) was made: “cross-platform online multiplayer” or “cross-play” for short. As will be discussed later, the COVID-19 pandemic blurred the line between cross-play and asymmetric multiplayer VR titles.

The asymmetric VR games that seemed to be held in highest esteem leveraged the unique affordances of the technology needed to play VR. At minimum, most asymmetric VR games utilize the “social screen,” or the screen not in use by the VR user, to present a second view. The social screen can be a television, a desktop computer monitor, or even a smartphone screen. Often the social screen allows onlookers to interact with the virtual space using game controllers, the keyboard and mouse, or through an interface on their mobile device. However, not every game must use technology in addition to the VR headset in order to be “asymmetric VR.” For a case in this point, KTaNE uses only the VR headset and a printed document. The enjoyment comes from navigating the asymmetry of information, for the onlookers by learning the specifics of the bomb seen by the VR player and for the VR player by learning the steps needed to diffuse said bomb which are listed in the onlooker’s paper manual.

One of the more opaque descriptions in the corpus was that of games which capitalize on short-duration experiences to facilitate the rotation of the VR headset among group members. Posts describing these games used the following words: “local party hotseat,” “hotseat local multiplayer,” “pass-and-play,” and “hotseat multiplayer.” While these sorts of games often were asymmetrical in nature, the terms used to describe them seemed to change depending on the intended purpose for the gaming session. Additionally, the category included games which are single player but either offer an engaging onlooker experience (using the social screen) or a local leaderboard for score competition. Because of the current effort’s focus on both co-located and asymmetric games, hotseat multiplayer games remain in the analysis.

An understanding of the settings for asymmetric and other varieties of co-located multiplayer VR games emerged from the initial reading of the posts. These included casual play of VR, VR demos to people who do not yet own a VR headset, and social gatherings centered around the use of VR. For such use cases, asymmetric VR is an ideal way to keep people engaged in an otherwise solitary experience. As put by one redditor, you are “no longer in a helmet that hides you from the world, others can enter it, too.”

CAVR was designed for use in classifying and understanding asymmetric experiences, specifically asymmetric VR in a co-located group (Ouverson and Gilbert, 2021). This content analysis started more broadly, focusing on the co-located use of VR and examining the asymmetry inherent in those co-located interactions. Due to the onset of the COVID-19 pandemic in early 2020, an interesting line of conversation emerged in the corpus in which redditors sought ways to enjoy asymmetric VR during mandatory lockdowns, even though these games were meant to be played in a single room with co-located others. Therefore, while the exclusionary and inclusionary criteria described in Section 3.2 were adhered to, no additional exclusions were made for comments around online use of asymmetric VR. The four observed contexts—online, casual, party, and demonstrations (or demos), are shown in Figure 2 and defined, below.

FIGURE 2

FIGURE 2. Visual examples of the identified settings: (A) Online, indicating the social interaction surrounding use of a single VR HMD happens over a network connection and may be displayed on the monitor; (B) Casual, indicating the “everyday use” of VR; (C) Party, indicating VR use in a social setting as part of an event dedicated to using VR, especially with friends or colleagues; (D) Demonstration, indicating the introduction of VR to new user(s) in public or private settings. Photos used with permission from Author 1 per CC-BY-NC-ND.

Online use of asymmetric VR games includes the use of third-party software to mimic the existence of a shared space using virtual means. These experiences are different from “social VR,” in which the socialization takes place in a single virtual space, because 1) only a single VR headset is required and 2) the players do not meet in a shared virtual space in the way they might in social VR games. Instead, players meet in a set of separate virtual spaces which may be merged only by communication or via other game mechanics.

Casual use is best understood as simply using VR in mundane circumstances. Often, casual use of asymmetric VR happens with roommates or family members as a way to share the experience with them so as to not exclude them from the experience. In Figure 2B, this is represented by the first author and her cat who often feels excluded from VR experiences, likely because there is not a proper way to mediate the interspecies social interactions.

Party settings of asymmetric VR use are those in which a group of friends, colleagues, or even family members who do not live together attend a social function for the express purpose of using VR. Asymmetric VR, in party settings, is a way to keep external users engaged with one another and the individual in the HMD so that no one is (and ideally, no one feels) excluded from the group. Demos are similar to party settings because both likely include folks who are new to VR and coalesce out of a desire to share VR with others. However, the two settings are different because 1) demos are solely for the purpose of using VR, making socialization of the group a secondary goal, and 2) they may include acquaintances or strangers in the interaction, which may occur either privately or publicly.

Asymmetric VR is recognized as a legitimate way to expand the reach of VR in the consumer market. As innovations continue, those which leverage the unique abilities of the technology will prove, in the author’s opinion, most viable. The next section describes the results of the content analysis with regards to the dimensions of CAVR and the patterns of asymmetry present in the corpus.

Patterns of Asymmetry

From the coded data, an answer to the research question, what are the types of asymmetry that are present in co-located use of VR, emerged. Examined individually, each dimension of asymmetry from CAVR is present for co-located VR use, depending on the design of the game. Thirteen general patterns of those dimensions of asymmetry were noted, and 12 of those patterns were meaningful (one pattern was indicative of content lacking enough specificity to be useful and is excluded from further analysis and discussion). These 12 meaningful patterns of asymmetry are named and defined in Table 4, and all but the Online pattern of asymmetry were identified as representing co-located use, thus answering the research question.

TABLE 4

TABLE 4. Patterns of asymmetry, identified by content analysis of Reddit posts, and their definitions. Shaded rows are excluded from further analysis.

While 11 of the patterns represent co-located experiences, only eight of the named patterns depict asymmetric VR experiences in adequate detail for discussion. These patterns are 1) Dueling, 2) Cooperative Eye-Spy, 3) Navigator and Pilot, 4) Boss vs Horde, 5) Hide and Seek, 6) Game Mastering, 7) Teleguidance, and 8) showcase. These eight, in addition to the Pass-and-Play pattern (which may use asymmetric or symmetric technological access to the virtual space) and the more generic patterns, Spectating and Asymmetric VR, are the focus of the remainder of this paper.

In Table 4, each pattern is associated with symmetry or asymmetry of the CAVR dimensions depending on what each dimension was rated during the coding phase of the content analysis. The unique combination of ratings makes up the dimensional forms, which were categorized and interpreted (Step 10, Table 3) to generate the patterns of asymmetry. In the text that follows, “symmetric” and “asymmetric” are used in a similar way, describing each dimension’s coding outcomes for the examples pulled from Reddit. For instance, a few of the patterns are quite similar to one another and experience categorization may be variable depending on the viewpoint and granularity of the examination. As discussed later, future work examining the integration of other frameworks may increase our ability to differentiate and clearly discuss the experiences contained in the general term, “asymmetric VR.”

It is noteworthy that transportation is the differentiator between what is and is not asymmetric VR, according to the present definition. Particularly for studies of asymmetric VR (Gugenheimer et al., 2018; Thomsen et al., 2019), it is not uncommon to describe asymmetry on a scale from high to low, which results in lost nuance and may encourage a bias toward asymmetry as negative, which is not always the case. When the content was not rich with description of the experience, the coders identified asymmetries in transportation, meaning it was clear there was different technology in use. With more detail present, balance of power was also coded, indicating differences in control of the experience for the VR user and those using other kinds of interaction.

At each traditional level of asymmetry (high, medium, low) there may be multiple patterns of asymmetry. For example, highly asymmetric experiences of VR include both games which do not require asymmetric transportation to interact in the shared space, such as those aligning with showcase or Pass-and-Play patterns, and games which require asymmetric transportation, such as those which follow the Teleguidance and Navigator and Pilot patterns. At lower levels of asymmetry, the nuance increases, as discussed in the following paragraphs.

Game Mastering, Hide and Seek, Boss vs Horde, and Cooperative Eye-Spy all describe experiences which may also be referred to using the less specific descriptor “medium asymmetry.” Similarly, low levels of asymmetry correspond to five identified patterns: Spectating, Dueling, Boss vs Horde, Teleguidance and Online. These categorizations should be considered fluid, as design choices and user actions can influence the “level” of asymmetry; however, such categorization is meant more as an indicator of the problems with referring to asymmetric experiences on a simple scale of high to low. The authors of the present work believe the multidimensional CAVR framework is more useful in categorizing and representing the nuanced patterns of asymmetry associated with asymmetric VR games, especially for those experiences which are traditionally identified as having lower levels of asymmetry.

The language used previously to refer to asymmetric VR is not specific enough to facilitate discussions of the nuance of such experiences. Designed asymmetric VR experiences may incorporate a variety of asymmetries and symmetries in each of the five CAVR dimensions to manifest unique experiences and solve different needs or leverage different advantages. When researchers (and designers) limit our discussion of asymmetry to a single dimension scaled from high to low, we ignore the complexity of the experiences. In fact, we may only be including the design patterns associated with the base Asymmetric VR pattern or Spectating pattern of asymmetry. To aid in the application of CAVR to future discussions around asymmetric VR experiences and their design, the following sections will exhibit quotes which correspond to asymmetry and symmetry in each dimension, as well as a description of each dimension based on the categorization given in the coding phase of the content analysis.

Transportation

The CAVR dimension of transportation describes the interaction metaphors supported by the technology which allow access to the mixed-reality space, i.e., the extent to which the technology transports the user to a virtual space. As mentioned above, this dimension is the crux of the generic term “asymmetric VR.” Previous definitions, including the one given at the beginning of this paper, directly point to an asymmetry of transportation alone. For a VR experience to be asymmetric, there must be at least the possibility for asymmetric transportation. Thus, at a minimum, asymmetric VR experiences are those for which transportation is not required to be symmetric.

When VR experiences are designed to be asymmetric, transportation is asymmetric. A good example of this came from a comment by redditor NateBGamer: “So my mind is blown how much fun these asymmetric games can be, and I truly believe these sort of games, done properly, will carry [VR] forward. One + player on the [PC], one player in [VR].” Asymmetric VR, or at least those asymmetric VR experiences “done properly,” appear to provide different modes of transportation to the shared space. However, that transportation does not need to be explicitly mediated by technology, as described in the next section.

Asymmetric VR experiences are not the only possibility for a co-located group of interested players. Regarding the co-located use of a VR experience not designed to be asymmetric, redditor Kalazor wrote, “I’ve found that if I’m just hanging out with one friend, it’s pretty easy to get them to try out the Rift, but it’s hard when a group of people are over since no one wants to separate themselves from the group to try it.” Without proper mediation, the virtual space stays separate from the physical one, and to enter it is to become isolated from anyone in your vicinity.

Spatial Co-presence

The CAVR dimension of spatial co-presence describes the mediated access to each member of a group, meaning the extent to which the technology provides each participant with access to others in the shared space. The original definition of this dimension called out an associated range from unmediated to mediated spatial co-presence. The present work instead looked at whether the content described a symmetric or asymmetric experience, which means spatial co-presence was either a mix of mediated and unmediated (i.e., asymmetric) or it was consistently mediated or unmediated for all members of the group (i.e., symmetric).

When looking at the dimension in this way, some of the granularity was lost in favor of simplicity, but themes were still detectable. For many of the posts, spatial co-presence was asymmetric, but when it was symmetric, either the host of the gathering or the game designers made specific choices to make it that way. Spatial co-presence, like transportation, is a dimension that is closely tied to the design of the experience (e.g., the hardware and software choices). Because the range of spatial co-presence is originally more about whether the interactant’s access to one another is mediated than about the asymmetry of the design, more quotes from the data are presented here than in other sections (two for asymmetric, one for symmetric and mediated, and one for symmetric and unmediated).

Regarding an experience with asymmetric spatial co-presence, in response to a question about mirroring the VR interface for external players of one game, redditor jtaosmsou remarked, “this would break the main part of the game. You have to explain what you see to your friends. And that is the funniest part. If all can see what you see it would be way [too] easy.” Another redditor, Lettuphant, offered the following about asymmetric spatial co-presence:

This is an excellent game, especially to show off how unique VR is to fellow nerds: Play on one side and you experience a first person sneak-“em-up,” but then put on the headset and see the whole map as a doll’s house with a strong sense of presence. It’s a quick way to get a “Whoa” from seasoned gamers who haven’t bought into VR.

Regarding symmetric spatial co-presence, redditor dhaupert recalled the impact of their alterations to a game to provide symmetry of spatial co-presence:

The second biggest hit of the night was VNAF (Vive nights at Freddy’s-the fan remake of Five nights at Freddy’s). Most of the kids noped right out of playing with the headset but watched and helped from the PC side as the bravest took it on. Even though it is a one player, they had almost as much fun watching and helping. One kid actually leaped when the animatronic appeared next to him-a literal jump scare!

For another redditor techies_9001, there was a desire for more mediation than was initially provided. Their perspective highlights symmetric unmediated spatial copresence, as they wrote, “quick question, having the headset on makes it sometimes harder to hear someone from the outside. Is there some kind of solution for local voice chat?”

While spatial co-presence could be considered a designed dimension, it is still affected by psychological factors. As put by one anonymous redditor, “Because one player is not in front of a computer screen or inside a headset doesn’t mean they are not in the game. Games are inside your imagination, they don’t require a computer or any devices. The second player is clearly also playing.”

Informational Richness

Informational richness has to do with extent to which the technology delivers and captures information about the mixed-reality space which is valuable to the interaction for each member of the group. This is one of the CAVR dimensions which is more related to the design of the experience but is altered by the presentation choices of the host. For example, some asymmetric VR experiences provide an asymmetry of informational richness, giving different players access to different levels of detail or types of information. Other experiences are set up by the host of the social gathering or the owner of the VR headset to add symmetric elements of informational richness, wherein the social screen broadcasts a mirrored view of the VR player’s perceived environment regardless of whether the game is designed to support multi-player interaction or not.

In the corpus, redditors expressed a desire for asymmetric informational richness. Redditor Ossius, when anticipating the release of the Oculus CV1, expressed that they were excited to show their brother the experience. In writing, they wondered, “since the PC monitor will be more or less free, or projecting what he sees in VR, couldn’t the monitor be used to display a different user interface?”

Redditor dhaupert, during a review of the VR birthday party they threw for their teenaged son, pointed to the impacts of symmetric informational richness of a specific style of game on the out-of-VR experience, writing “Never did play any of the FPS shooters, Robo Recall, or anything of that sort because the only one player games that were selected were the ones that were fun enough to watch. RR is an awesome game, and I knew they’d all love it, but really only fun for the player.” Informational richness can be symmetric and result in an enjoyable game, as was the case for VNAF (mentioned in the quote by dhaupert in the previous section). However, symmetric informational richness must be paired with a cinematic or exciting game, one that is “fun enough to watch.”

Team Interdependence

Team interdependence has to do with the alignment of the different goals each member of a group brings to the experience. It is important to remember that while the overall objectives may well be the same (e.g., beat the other team, get to safety), the goals can still be different for each member (e.g., prevent the other team from retrieving an object, defuse a bomb by describing it to another person and following their instructions). While team interdependence can be influenced by the design choices that create a virtual experience, this dimension of CAVR is more likely to be influenced by the behavioral and social variations of the group involved in the experience. To reduce the complexity, only the design intentions, rather than the individual member choices, will be considered in the following discussion.

When considering the possible tie of this dimension to collaborative/cooperative or competitive experiences (as proposed by K. Rogers et al., 2021), consider the following: Passive spectator games tend to be both cooperative and symmetric, while those which promote active use of technology support a broader range of asymmetry and competition or cooperation. For example, asymmetric team interdependence is found both in games designed to pit group members against one another in a competition in which each side has different kinds of goals and in games designed to encourage members who control different components of the shared space to work cooperatively toward a shared objective. A game’s competitive or cooperative design therefore depends on more than just symmetric or asymmetric alignment of interactant goals. While symmetric goals tend to facilitate cooperative experiences, symmetric team interdependence can also mean that every group member is trying to achieve the same thing wherein one member’s achievement is not compatible with any other member achieving their goal. One such game, Priest vs Poltergeist, as described by redditor spammaleros, is “an action-packed duel between the mighty Poltergeist (VR player) and a battle-hardened Priest (PC player).”

Holding constant the cooperation of the experience, the difference between symmetric and asymmetric team interdependence is more describable. Redditor Spenceanator, writing about an experience with asymmetric team interdependence, shared that they were working on an asymmetric game in which, “one player is attempting to fend off [asteroids] and aliens, while the other player in VR is attempting to keep the ship up and running.” In this case, the goals are different—one player protects the ship, and one player keeps the ship going—but the experience is cooperative and the overall objective is the same. Another cooperative experience which featured symmetric team interdependence was described by redditor Th3irdEye:

I couldn’t remember the name but I’m pretty sure this is what I played at PAX East last year. Me and my buddy took turns using the Rift and using the regular screen. Regular screen guy warns the Rift user about obstacles and enemies that are invisible to the Rift user and directs them through the level. Pretty cool idea and it was fun to try and talk my friend around a pit only to have them walk right into it.

Balance of Power

Balance of power refers to the different degrees of control over information in an experience, which is influenced by the group member’s roles and levels of experience. As such, balance of power is expected to relate to each actor’s level of ability as well as the creator’s design choices. Most of the content analyzed exemplified the design choices side of the dimension, and—perhaps symptomatic of the differences between VR and the other technologies used in the experiences—it was much more common to find a design with an asymmetric balance of power than a symmetric one.

Symmetric balance of power is difficult to maintain, which may explain why it is less common for asymmetric VR experiences. Still, some developers try to provide VR and PC cross-play which, unless specifically designed to support distinct roles for the different technologies, necessitates a symmetrically designed reach of control (which is tightly tied to the balance of power) to be perceived as “fair” by the players of the game. One popular game, Payday 2, was given a glowing review by redditor MatteAce specifically because the developers were actively working to allow VR and PC players to play together online. They wrote, “if you played Payday before you’ll get the exact same game you’re used to - but in VR. You can play online with other people and your non-VR friends and all in all it's a blast.” As they clarify in a later comment, “both “flat” version and VR version can play together, however, while the VR version is in beta only those who are playing on the beta branch can play together. When the VR version gets fully released everyone will play together automatically.”

Games with an asymmetric balance of power seem to be the most desirable, as multiple redditors surface ideas for experiences in which the VR and PC players have different control over the experience:

You could also make a game where the VR player is a ghost, allowing him to clip through walls and the screen player has to escape.—Ossius.

How about the Monitor player looks for pieces of a rescue pod on an abandoned spaceship using a map. They tell the VR player where the parts are and can see all the enemies that VR player will run into so it’s their job to assist like a Rally Car.—Ghs2.

The currently extant asymmetric VR experiences often provide an asymmetric balance of power to their players. In Mass Exodus, “the VR player is a giant robot and the PC player is a tiny factory robot trying to escape the warehouse. The VR player can pick up the pc robot and drop them in a giant machine grinder if they spot them among the NPC machines,” as explained by redditor boredguy12. For video games, developers are already seeing the need to cater to the individual strengths of the technology for which they are creating experiences and to offer diverse experience options to reach a broader gaming audience (Harris et al., 2016; Harris and Hancock, 2019).

Discussion and Conclusion

The previous section presented observations about the dataset, as collected during the data immersion phase of the content analysis, and the overall results of the content analysis, including the definition of patterns of asymmetry and examples from the sample of the CAVR dimensions. What follows is a discussion of the patterns and dimensions of asymmetry, as derived from CAVR, and how these findings coordinate with Rogers et al. (2021) “best fit” framework for asymmetric gameplay in multiplayer VR. Then, limitations and future directions are explored. The paper concludes with implications for practice, meant to aid both VR creators and researchers.

Patterns of Asymmetry in Co-located Use of Virtual Reality

When considering the patterns of asymmetry, it should be noted that the terms may not correspond to their original definitions. For instance, Teleguidance originates from terms (e.g., telepresence, teleoperation) which describe the use of technology to remotely interact with someone or something else to offer aid or accomplish a task (Ens et al., 2019; Kraus and Kibsgaard, 2015). While this is certainly an application of the pattern identified, it was also apparent when redditors discussed instructing or guiding a player through interactions with the virtual space while having access only to the physical space or even keeping VR users from inadvertently colliding with the physical space. Redditor Bacon_00 explains, “I had people slamming into my desk, punching my couch, and I had to get up and guide people back into the safe zone numerous times. It was kind of ridiculous.”

A few of the patterns, including Navigator and Pilot and Cooperative Eye-Spy, are quite similar. The differences between these two patterns are in the symmetries they employ. Cooperative Eye-Spy necessitates asymmetries in spatial co-presence, as the VR user’s interaction with external others is not mediated by the technology, but the external others have access to the shared space by way of the social screen. On the other hand, Navigator and Pilot necessitates symmetric spatial co-presence, as both roles must have mediated access to the shared space to accomplish the shared objective (regardless of the interactants asymmetric goals).

While each of the patterns is ascribed a name, the names are not meant to signify the complete range of possible experiences. In fact, the patterns are easily translatable outside of entertainment. A psychotherapy use case as depicted by Menezes. (2020) offers an example. They describe a shared space system for use in exposure therapy, which is used most commonly for treating phobias through habituation to a target stimulus—or incremental exposure to the thing for which the patient is experiencing disruptive fear. The shared space system, in this case, consists of an augmented-reality-enabled mobile tablet, which the therapist uses to craft an exposure experience that scales with the patient’s ability to cope with the presence of the fear stimulus object without using avoidance techniques (e.g., closing their eyes or removing the headset). For the patient, these objects are visualized in 3D as part of a virtual environment, which they experience using a VR headset. Thus, the use case includes asymmetries in transportation, spatial co-presence, informational richness, and balance of power, but not necessarily team interdependence, corresponding with the pattern Game Mastering.

One perhaps obvious question to arise from this work is whether the patterns of asymmetry should replace the CAVR dimensions. While the authors are not opposed to the idea of referring to asymmetric VR experiences as the patterns they embody, they suggest the CAVR dimensions will be more useful as a scaffold for building and understanding unique asymmetric VR experiences than will a fixed list of patterns. As such, the authors propose an update to the CAVR definitions of the dimensions, based on the above work, and intend for the patterns to serve as a guide for understanding and creating asymmetric VR experiences.

The Composite Framework for Asymmetric Virtual Reality Dimensions of Asymmetry

When designing this study, the authors chose to simplify the application of the CAVR dimensions (Ouverson and Gilbert, 2021) to binary response options (eventually extended with the addition of a non-response option to avoid arbitrary “default” assignment during coding). While applying the codes, it became clear that transportation, which is explicitly about the hardware systems used in the interaction (and the differences between them), is the default expression of asymmetric VR. When a person discusses “asymmetric VR,” they often are only communicating the idea of using different interfaces to play a single game with another person (or a group of people), which calls to mind the work in cross-device computing (Olin et al., 2020). Thus, transportation is only affected by the design choices of the developer or development team.

Spatial co-presence is the first of the CAVR dimensions which are influenced both by the design of the experience and by the choices and personalities of the interactants, counter to original assumptions (Ouverson and Gilbert, 2021). The dimension (which was originally defined with a range of unmediated to mediated) was coded according to asymmetry, which meant there was no distinction between experiences in which spatial co-presence was fully mediated and fully un-mediated, in effect reducing the dimension to the category of “shared space,” which Rogers et al. (2021) include in their review as part of social asymmetry. This dimension (as it was defined in the content analysis) is possibly least affected by interactant experience, as only intentional choices may tip an asymmetric experience (in which the VR player’s interactions with the virtual space are displayed on a social screen without providing further interaction modalities to external others) to a symmetrical one (in which the VR player and external others are equally unaware of each other’s spaces).

Informational richness is distinctly centered on the design of the interface, yet an interactant can choose to do with the presented information what they wish. In the original definition, informational richness was related mainly to the usefulness of the information available given the goals of the team or group (Marlow et al., 2016), thus relating to the personal choice aspect of the interaction. In the present study, it was clear that design choices were largely responsible for how rich the information was (and, by extension, how useful it would be to the interaction), but interactants could decide to limit the usefulness of the information, for example, by removing spatial audio cues from the VR player or removing access to the virtual space for external players. In non-game settings, this may be realized as delays in visual updates in the virtual world as the external user manipulates an artifact with keyboard and mouse, or perhaps as the seemingly unavoidable task of teaching a new user to navigate the menus without visual information of their location in the menu.

Team interdependence can be influenced by specific design choices, yet it is inextricably linked to the behavioral and social choices of the interactants. For example, if an asymmetric VR game follows the Spectating design pattern, the team interdependence may be symmetric or asymmetric, depending on whether the players decide to involve themselves, for example, by video recording the VR player from the external point-of-view. This act, in itself, may cause discomfort, and as noted in prior work (Benford et al., 2012; Rogers et al., 2021), attention to ethical considerations such as consent is imperative, especially in situations with imbalances of power (i.e., asymmetric balances of power).

Balance of power is least influenced by the experience’s design. When created, experiences are defined with roles, which can have different levels of control over information, yet the player’s levels of familiarity with the genre of game, the controls, and even their unique abilities may impact the realized balance of power. In the corpus, a large component of this dimension was the difference in the controls of the users. Table 5 restates the dimensions and their definitions, and further summarizes the expected experiences which relate to each dimension, based on the results of the content analysis.

TABLE 5

TABLE 5. Updated definitions and suggested experiences of the five CAVR dimensions of asymmetry.

When applying the CAVR framework, it became clear how other areas of game design (as suggested by K. Rogers et al., 2021) may be useful in understanding the described experiences. For example, there was a desire to include information about the timing of the experience (whether access to the shared space was synchronous or not), which was originally excluded from CAVR (Ouverson and Gilbert, 2021), as well as the locus of control (whether each interactant had control of a single component or many components within the shared space). The “best fit” framework proposed by Rogers and others (2021) offers extensions to the CAVR framework in such ways, as explored in the following section.

Discussion of These Results, Composite Framework for Asymmetric Virtual Reality, and the Rogers “Best Fit” Framework

While Rogers and others (2021) have created a framework which complements and strengthens CAVR (Ouverson and Gilbert, 2021), some key differences remain. One major way in which CAVR differs from Rogers et al. best-fit framework is in the treatment of the term “interface.” Rogers et al. (2021) recognize interface as the display medium through which the players interact with the game, rather than the more formal definition, which includes both the output and input mechanics (Dix et al., 2004). Second, while CAVR (Ouverson and Gilbert, 2021) does not exclude competitive games, even while considering the dependence, or team interdependence, of an experience, Rogers et al. (2021) work suggests that dependence may need further consideration, as the way dependence works in competitive games is fundamentally different than in collaborative games. Despite the lack of competitive applications of asymmetric VR beyond game-based experiences, future work around this topic may be worthwhile.

Some categories identified in the Rogers et al. (2021), namely shared control and social asymmetry, either are implicitly present or are expected to be highly related to dimensions already present in CAVR. Indeed, these categories describe components of the Reddit content which was not discernible using CAVR alone. Shared control, for instance, is expected to be related to the informational richness and balance of power dimensions, which are highly related to the interactant’s in-experience loci of control. Social asymmetry is also already presented as spatial co-presence, which may even support the extension of the category suggested by Rogers et al. (2021).

Both frameworks emphasize the applicability of asymmetric VR to collaboration and teamwork both in and outside of video games. While neither framework is firmly rooted in literature on group interaction, CAVR appears to draw more from literature in organizational psychology and workspace awareness than does the Rogers framework. These were added to the scoping review after being identified as a gap in the identified asymmetric VR literature, which tends to focus on the digital interactions and tangible experience rather than the group interaction facilitated by the technology (Ouverson and Gilbert, 2021). When considering the application of theory developed in game settings to the facilitation of collaboration in team settings, it is important to understand and incorporate the findings from studies of virtual teams and computer-mediated communication, as was done in the conceptualization of CAVR (Ouverson and Gilbert, 2021).

To summarize, we see Rogers et al. (2021) framework as a complement to CAVR. The questions that could not be answered in their systematic review of the literature on asymmetric VR (K. Rogers et al., 2021) have answers within CAVR (Ouverson and Gilbert, 2021) and the results of the present study. In addition to this work’s treatment of team interdependence, referred to as “dependence” by Rogers et al. (2021), the relationship between spatial co-presence and social asymmetry may be explained by a nesting of the dimensions, wherein social asymmetry exists within the concept of spatial co-presence or vice versa. Lastly, the patterns of asymmetry may be clarified through attention to the concept of shared control (Rogers et al., 2021), whereas the Rogers et al. “best fit” framework would benefit from more attention to group use of and collaboration via virtual experiences. More work should be done to unify the two frameworks so as to guide future research and design of asymmetric VR and mixed reality experiences.

Limitations

In the present work, the authors had intended to pursue an additional research question relating the asymmetry present in VR games to the role of technology each experience served. Due to the persistently low IRR associated with coding the Roles of Technology, the decision was made to drop the variable from the analysis. While the roles may be adequately defined for describing mobile technology, more work is needed before they can be applied to virtual reality and other emerging technology used in social or group situations. It is also possible that the descriptions of technology use on Reddit were more focused on the technology experience rather than the social experience, complicating the coder’s abilities to distinguish between the enhancement roles of technology for each described VR experience.

The present work did not consider spatial co-presence in the way the dimension was defined for CAVR (Ouverson and Gilbert, 2021). As such, there may be additional nuance to the showcase, Teleguidance, Navigator and Pilot, Cooperative Eye-Spy, and Dueling patterns of asymmetry. While care was taken to represent the range of unmediated to mediated spatial co-presence when giving examples of content representative of the CAVR dimension, future work which more systematically examines the dimension through the lens of the interaction metaphor present in each of these patterns would both strengthen CAVR and the design recommendations around asymmetric VR.

Future Directions

Perhaps the most tangible work which should be pursued in the future is an expansion of the present study. Future work could either use the codebook in Supplementary Appendix SB1 to analyze more of the content in the full corpus (and beyond) or could follow the methods described in this paper to create a new codebook including variables from the Roles of Technology (Olsson et al., 2019) and Rogers et al. (2021) “best fit” frameworks against which the entire dataset could be compared. It is imperative that such an expansion include additional coder training and consensus building around the definitions of the framework(s). While not mandatory, researchers should consider expanding the search criteria to include keywords which center the use of VR among a group of people and take care to avoid limiting the use to one form (e.g., asymmetric VR). Also, the exclusion criteria should be refined so that more posts and comments relating to the demonstration of VR technology to an audience can be gathered and analyzed for a holistic view of the perceptions of those who have interacted with VR in a group format.

Another way the present work could be extended is through reflection on the items which were coded as being unclear. In the present paper these items were removed from the analysis, as many of the items flagged in this way were deemed irrelevant, yet it is common practice to explore those categories to determine if a new code or category is present within the work.

One aspect of the examined discussions that was new to the authors was the remote use of games originally designed for co-located use. In the corpus, online settings appeared in response to limitations in technology (one game intended to use motion capture suits) or in travel (i.e., because of public health safety measures enacted in response to the COVID pandemic). Because these insights arose as part of the observations of the first author, and not as part of a broader, more strategic content analysis, more work is needed which focuses on the use of experiences which were designed for co-located use in remote socialization contexts.

While it is suggested that asymmetric VR may be a ripe opportunity for increasing the accessibility of the technology (Ouverson and Gilbert, 2021; Rogers et al., 2021), the authors are not aware of research to date that has centered the opinions and experiences of those living with disabilities, although some attention has been given to user testing of VR (though not asymmetric VR) with older adults (J. A. Brown, 2019; Bruun-Pedersen et al., 2016). An important avenue of research includes documenting the ways in which asymmetric and similar VR experiences may include more people but also testing these designs with actual disabled people to better understand their needs.

Overall, attention to the human experience is important. While the aim of the present paper was to categorize the designed experiences based on the descriptions of those who have encountered them, a future direction of this work is to analyze the interactions and patterns of asymmetry within asymmetric VR games. Previous research has used autoethnography to explore the privacy aids in social VR games, documenting the interaction patterns present in those existing applications (McVeigh-Schultz et al., 2018; McVeigh-Schultz et al., 2019). In addition to noting the ways asymmetric VR games mediate interaction, work could examine the engagement and enjoyment factors such as flow (Kaye, 2016) which result from the interaction, or the impacts of these experiences on rapport building (Bradner et al., 2005; Nardi, 2005) and other precursors to collaboration and teamwork (Steinmacher et al., 2010; Marlow et al., 2016).

Implications for Practice

The most important thing to remember when designing an asymmetric experience for VR is that building out two similar experiences and relying on the differences in the interfaces to carry the player experiences will not suffice. As such, we suggest moving beyond the basic concept of asymmetry as transportation to make use of the unique affordances of the incorporated technology, as outlined in this paper and in the work of Rogers et al. (2021). Relatedly, the roles for the VR and non-VR players should be built around the unique affordances of the technology, for example, in considering how each incorporated interface supports full-body direct manipulation or abstracted manipulation, such as through keyboard and mouse or touch screen controllers. Note, as well that the role taken by a VR player in one experience may be experienced differently than the same role when given to an external player, thus affecting the patterns of asymmetry of a game or experience.

There are certainly direct implications for game design; however, we find it more interesting to consider the design of asymmetric VR experiences including and also extending beyond games. Consider that at what have been called “lower levels of asymmetry,” meaning the experience uses high-fidelity forms of interaction, such as by incorporating augmented reality HMDs and phone-based experiences, it can be tempting to design symmetric balances of power, informational richness, and team interdependence while fully mediating spatial co-presence for both technologies. However, in doing so the unique properties of the mobile device or of augmented reality as a paradigm are lost, and the incentive to engage as the VR- or non-VR-interactant is lowered.

Take for example Dollhouse VR, conceptualized by Ibayashi et al. (2015), in the context of this framework. The asymmetry present may well be considered “lower,” as described in the previous paragraph, as the non-VR user accesses the virtual world via a large touchscreen device known as a multi-touch table. Using the touchscreen in Dollhouse VR, architectural or interior designers bring VR-user attention to various targets of interest, facilitating communication between all involved while also increasing the immersion of the VR user, who may not be as accustomed to blueprints or top-down 2D drawings. As such, the technology supporting asymmetric VR in Dollhouse VR was chosen to complement the tasks involved in the specific workflow: showing a design to another person, such as a client or a coworker, prior to putting that design into 3D, for the purpose of getting feedback or alignment. The client or coworker can experience the space in scale while still interacting with the designers on the outside of the virtual space. Meanwhile, the designers (one or more) can collaborate using the touchscreen to ask contextual questions or explore alternatives. Due to the presence of asymmetries in transportation, balance of power, and symmetric access to the shared space (spatial co-presence), the experience is most indicative of the Navigator and Pilot pattern. The author’s expectation is that other Navigator and Pilot experiences may be used to enhance the present design of Dollhouse VR, or, similarly, that future experiences that fall into this category may find beneficial design patterns in the design of Dollhouse VR.

As suggested by the surge of interest in remote play versions of asymmetric VR games, many are coping with pandemic-related inconveniences by finding ways to play games meant for co-located use even when being together is not feasible. For enterprise applications of asymmetric VR (such as Dollhouse VR), the ability of the experience to function despite the location of the interactants is and will continue to be important. Even this is not as simple as introducing a remote version of the experience. For example, Dollhouse VR would require additional considerations around the mediation of communication between remote interactants, and further decisions would need to be made about how and whether the designers should still be represented in the shared space.

While exploring new methods of mediating and enhancing the co-located interaction surrounding VR, remember that balance of power is difficult to maintain. While some patterns of asymmetry do not require asymmetric balance of power, none require symmetry in this dimension. Instead, focus on granting different and unique abilities which may correspond to the transportation method in use by each role in the experience. It seems more important that the unique affordances are supported, and that the shared space is sufficiently accessible, whether literally through an interface or metaphorically through establishing a shared narrative.

Data Availability Statement

The original contributions presented in the study are publicly available. This data can be found here: https://figshare.com/articles/dataset/Co-located_or_asymmetric_VR_reddit_posts_from_2014-2020/16547415/1.

Ethics Statement

Written informed consent was obtained from the individual(s), and minor(s)' legal guardian/next of kin, for the publication of any potentially identifiable images or data included in this article.

Author Contributions

Conceptualization, KO and SG; data curation, KO; methodology, KO; validation, KO, EO and CS; formal analysis, KO; investigation, KO, EO and CS; resources, SG and the IMSE department; writing—original draft preparation, KO; writing—review and editing, KO, CS and SG; visualization, KO; supervision, SG; project administration, KO. All authors have agreed to the published version of the manuscript.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frvir.2021.765881/full#supplementary-material

Footnotes

¹The asymmetric use of VR has been studied for longer (see (Benford et al., 1998; Grasset et al., 2005; Simon et al., 2005; Steed et al., 2012)) but under different names (e.g., mixed reality, mixed-space collaboration, single display groupware, or transitional collaboration).

References

Albæk Thomsen, L., Nilsson, N. C., Nordahl, R., and Lohmann, B. (2019). Asymmetric Collaboration in Virtual Reality. Lom 12 (20), 1–28. doi:10.7146/lom.v12i20.109391

CrossRef Full Text | Google Scholar

Basak, D. (2016). Incomplete Information and Delay in Bargaining (Issue September). NY(US): New York University. PhD thesis.

Google Scholar

Benford, S., Greenhalgh, C., Giannachi, G., Walker, B., Marshall, J., and Rodden, T. (2012). “Uncomfortable Interactions’, in Conference on Human Factors in Computing Systems - Proceedings, 2005–2014, Austin, Texas, USA, May 5 - 10, 2012. doi:10.1145/2207676.2208347

CrossRef Full Text | Google Scholar

Benford, S., Greenhalgh, C., Reynard, G., Brown, C., and Koleva, B. (1998). Understanding and Constructing Shared Spaces with Mixed-Reality Boundaries. ACM Trans. Comput.-Hum. Interact. 5 (3), 185–223. doi:10.1145/292834.292836

CrossRef Full Text | Google Scholar

Bergh, D. D., Ketchen, D. J., Orlandi, I., Heugens, P. P. M. A. R., and Boyd, B. K. (2019). Information Asymmetry in Management Research: Past Accomplishments and Future Opportunities. J. Manage. 45 (1), 122–158. doi:10.1177/0149206318798026

CrossRef Full Text | Google Scholar

Bradner, E., Mark, G., and Hertel, T. D. (2005). Team Size and Technology Fit: Participation, Awareness,and Rapport in Distributed Teams. IEEE Trans. Profess. Commun. 48 (1), 68–77. doi:10.1109/TPC.2005.84329910.1109/tpc.2004.843299

CrossRef Full Text | Google Scholar

Brown, D. (2021). Nobody Can See Your Eyes when You Wear a VR Headset. Facebook Wants to Change that. The Washington Post. Available at: https://www.washingtonpost.com/technology/2021/08/13/facebook-vr-headset-eyes/?utm_campaign.

Google Scholar

Brown, J. A. (2019). An Exploration of Virtual Reality Use and Application Among Older Adult Populations. Gerontol. Geriatr. Med. 5, 233372141988528. doi:10.1177/2333721419885287

PubMed Abstract | CrossRef Full Text | Google Scholar

Bruun-Pedersen, J. R., Serafin, S., and Kofoed, L. B. (2016). Going outside while Staying inside - Exercise Motivation with Immersive vs. Non-immersive Recreational Virtual Environment Augmentation for Older Adult Nursing Home Residents. 2016 IEEE Int. Conf. Healthc. Inform. (Ichi), 216–226. doi:10.1109/ICHI.2016.31

CrossRef Full Text | Google Scholar

Collins, K. M. T. (2010). “Advanced Sampling Designs in Mixed Research: Current Practices and Emerging Trends in the Social and Behavioral Science,” in SAGE Handbook of Mixed Methods in Social & Behavioral Research (Second. Editors A. Tashakkori, and C. Teddlie (NY, US: SAGE Publications Inc), 353–378.

Google Scholar

Conger, A. J. (1980). Integration and Generalization of Kappas for Multiple Raters. Psychol. Bull. 88 (2), 322–328. doi:10.1037//0033-2909.88.2.322

CrossRef Full Text | Google Scholar

Creswell, J. W., and Poth, C. N. (2018). Qualitative Inquiry & Research Design: Choosing Among Five Approaches (Fourth). NY, US: SAGE Publications Inc.

Google Scholar

De Choudhury, M., Kiciman, E., Dredze, M., Coppersmith, G., and Kumar, M. (2016). Discovering Shifts to Suicidal Ideation from Mental Health Content in Social media. Conf. Hum. Factors Comput. Syst. - Proc., 2098–2110. doi:10.1145/2858036.2858207

CrossRef Full Text | Google Scholar

Dix, A., Finlay, J., Abowd, G. D., and Beale, R. (2004). “The Interaction,” in Human-Computer Interaction. Third (London, UK: Pearson Education Limited), 123–163.

Google Scholar

Ens, B., Lanir, J., Tang, A., Bateman, S., Lee, G., Piumsomboon, T., et al. (2019). Revisiting Collaboration through Mixed Reality: The Evolution of Groupware. Int. J. Human-Computer Stud. 131 (February), 81–98. doi:10.1016/j.ijhcs.2019.05.011

CrossRef Full Text | Google Scholar

Fillmore, H., and Storr, T. (2020). “AR and VR in the Workplace: Extended Reality Reimagines How Work Is Done,” in Expert Insights. (Armonk, NY: IBM). Available at: https://www.ibm.com/thought-leadership/institute-business-value/report/ar-vr-workplace.

Google Scholar

Gamer, M., Lemon, J., Fellows, I., and Singh, P. (2012). Package ‘ Irr ’ (0.84.1. CRAN. Available at: https://cran.r-project.org/web/packages/irr/irr.pdf.

Google Scholar

Grasset, R., Lamb, P., and Billinghurst, M. (2005). “Evaluation of Mixed-Space Collaboration”, in Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR’05), Vienna, Austria, October 5-8, 2005, 90–99. doi:10.1109/ISMAR.2005.30

CrossRef Full Text | Google Scholar

Gugenheimer, J., Stemasov, E., Frommel, J., and Rukzio, E. (2017). “ShareVR: Enabling Co-Located Experiences for Virtual Reality between HMD and Non-HMD Users”, in Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, NY, US, May 6-11, 2017, 4021–4033. doi:10.1145/3025453.3025683

CrossRef Full Text | Google Scholar

Gugenheimer, J., Stemasov, E., Sareen, H., and Rukzio, E. (2018). “FaceDisplay: Towards Asymmetric Multi-User Interaction for Nomadic Virtual Reality”, in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’18, NY,USA, April 21-26, 2018. doi:10.1145/3173574.3173628

CrossRef Full Text | Google Scholar

Harris, J., Hancock, M., and Scott, S. D. (2016). “Leveraging Asymmetries in Multiplayer Games”, in Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts - CHI PLAY Companion ’16, Austin, Texas, USA, October 16-19, 2016, 350–361. doi:10.1145/2967934.2968113

CrossRef Full Text | Google Scholar

Harris, J., and Hancock, M. (2019). “To Asymmetry and beyond!”, in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI ’19, Glasgow, Scotland, Uk, May 4-9, 2019, 1–12. doi:10.1145/3290605.3300239

CrossRef Full Text | Google Scholar

Hsieh, H.-F., and Shannon, S. E. (2005). Three Approaches to Qualitative Content Analysis. Qual. Health Res. 15 (9), 1277–1288. doi:10.1177/1049732305276687

PubMed Abstract | CrossRef Full Text | Google Scholar

Hunicke, R., Leblanc, M., and Zubek, R. (2004). MDA: A Formal Approach to Game Design and Game Research. AAAI, AAAI Workshop - Technical Report, WS-04-04, 1–5.

Google Scholar

Ibayashi, H., Sugiura, Y., Sakamoto, D., Miyata, N., Tada, M., Okuma, T., et al. (2015). Dollhouse VR: A multi-view, multi-user collaborative design workspace with VR technology. SIGGRAPH Asia 2015 Posters on - SA 15, 2–3. doi:10.1145/2820926.2820948

CrossRef Full Text | Google Scholar

Janson, H., and Olsson, U. (2004). A Measure of Agreement for Interval or Nominal Multivariate Observations by Different Sets of Judges. Educ. Psychol. Meas. 64 (1), 62–70. doi:10.1177/0013164403260195

CrossRef Full Text | Google Scholar

Johansen, R. (1988). “Current User Approaches to Groupware,” in Groupware: Computer Support for Business Teams. Editors E. M. Rogers, and F. Williams (NY, USA: Free Press), 12–44.

Google Scholar

Karaosmanoglu, S., Rogers, K., Wolf, D., Rukzio, E., Steinicke, F., and Nacke, L. E. (2021). “Feels like Team Spirit: Biometric and Strategic Interdependence in Asymmetric Multiplayer VR Games”, in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, Yokohama Japan, May 08-13, 2021. doi:10.1145/3411764.3445492

CrossRef Full Text | Google Scholar

Kaye, L. K. (2016). Exploring Flow Experiences in Cooperative Digital Gaming Contexts. Comput. Hum. Behav. 55, 286–291. doi:10.1016/j.chb.2015.09.023

CrossRef Full Text | Google Scholar

Kraus, M., and Kibsgaard, M. (2015). “A Classification of Human-To-Human Communication during the Use of Immersive Teleoperation Interfaces”, in Proceedings of the 2015 Virtual Reality International Conference on ZZZ - VRIC ’15, Laval France, April 08-10, 2015, 1–8. doi:10.1145/2806173.2806198

CrossRef Full Text | Google Scholar

Krippendorff, K. (2019). Content Analysis: An Introduction To its Methodology (Fourth). NY, US: SAGE Publications Inc.

Google Scholar

Landis, J. R., and Koch, G. G. (1977). The Measurement of Observer Agreement for Categorical Data. Biometrics 33 (1), 159. doi:10.2307/2529310

PubMed Abstract | CrossRef Full Text | Google Scholar

Marlow, S. L., Lacerenza, C. N., and Salas, E. (2017). Communication in Virtual Teams: a Conceptual Framework and Research Agenda. Hum. Resource Manage. Rev. 27 (4), 575–589. doi:10.1016/j.hrmr.2016.12.005

CrossRef Full Text | Google Scholar

Mckendrick, J. (2021). Business Leaders Are Changing Their Minds about Hybrid and Remote Work. Forbes. Available at: https://www.forbes.com/sites/joemckendrick/2021/11/07/business-leaders-are-changing-their-minds-about-hybrid-and-remote-work/?sh=37b3aa03d4c8.

Google Scholar

McVeigh-Schultz, J., Kolesnichenko, A., and Isbister, K. (2019. Shaping Pro-social Interaction in VR. 1–12. doi:10.1145/3290605.3300794

CrossRef Full Text | Google Scholar

McVeigh-Schultz, J., Márquez Segura, E., Merrill, N., and Isbister, K. (2018). What's it Mean to "Be Social" in VR? in Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility, DIS, Baltimore, Maryland, USA, October 20-November 1, 2018, 18, 289–294. doi:10.1145/3197391.3205451

CrossRef Full Text | Google Scholar

Nardi, B. A. (2005). Beyond Bandwidth: Dimensions of Connection in Interpersonal Communication. Comput. Supported Coop. Work 14 (2), 91–130. doi:10.1007/s10606-004-8127-9

CrossRef Full Text | Google Scholar

Neeley, T. (2021). 12 Questions about Hybrid Work, Answered. Harv. Business Rev. Available at: https://hbr.org/2021/09/12-questions-about-hybrid-work-answered.

Google Scholar

Neuendorf, K. A. (2017). The Content Analysis Guidebook. Second. NY,USA, Los Angeles, CA: SAGE Publications, Inc.

Google Scholar

Olin, P. A., Issa, A. M., Feuchtner, T., and Grønbæk, K. (2020). Designing for Heterogeneous Cross-Device Collaboration and Social Interaction in Virtual Reality. ACM Int. Conf. Proceeding Ser., 112–127. doi:10.1145/3441000.3441070

CrossRef Full Text | Google Scholar

Olsson, T., Jarusriboonchai, P., Woźniak, P., Paasovaara, S., Väänänen, K., and Lucero, A. (2019). Technologies for Enhancing Collocated Social Interaction: Review of Design Solutions and Approaches. Comput. Supported Coop. Work 29, 29–83. doi:10.1007/s10606-019-09345-0

CrossRef Full Text | Google Scholar

Ouverson, K. M., and Gilbert, S. B. (2021). A Composite Framework of Co-located Asymmetric Virtual Reality. Proc. ACM Hum.-Comput. Interact. 5 (CSCW1), 1–20. doi:10.1145/3449079

CrossRef Full Text | Google Scholar

Patrão, B., Menezes, P., and Gonçalves, N. (2020). Augmented Shared Spaces: An Application for Exposure Psychotherapy. Int. J. Onl. Eng. 16 (4), 43–50. doi:10.3991/ijoe.v16i04.11876

CrossRef Full Text | Google Scholar

Rivera, I. (2019). Package ‘ RedditExtractoR ’ (2.1.5. CRAN. Available at: https://cran.r-project.org/web/packages/RedditExtractoR/RedditExtractoR.pdf.

Google Scholar

Rogers, E. (2003). The Diffusion Of Innovations (Fifth). NY, US: Free Press. doi:10.1007/s10460-007-9072-2

CrossRef Full Text | Google Scholar

Rogers, K., Karaosmanoglu, S., Wolf, D., Steinicke, F., and Nacke, L. E. (2021). A Best-Fit Framework and Systematic Review of Asymmetric Gameplay in Multiplayer Virtual Reality Games. Front. Virtual Real. 2 (July), 1–23. doi:10.3389/frvir.2021.694660

CrossRef Full Text | Google Scholar

RStudio Team (2021). RStudio: Integrated Development For R (1.4.1106). RStudio, PBC, Available at: http://www.rstudio.com/.

Google Scholar

Shao, Y. (2019). RABBOT - Exploring Shared Awareness in Virtual Reality. Ontario: OCAD University.

Google Scholar

Simon, A., Dressler, A., Krüger, H.-P., Scholz, S., and Wind, J. (20052005). Interaction and Co-located Collaboration in Large Projection-Based Virtual Environments. IFIP Conf. Human-Computer Interaction, INTERACT 3585 LNCS, 364–376. doi:10.1007/11555261_31

CrossRef Full Text | Google Scholar

Sohrab, S. G., Waller, M. J., and Kaplan, S. (2015). Exploring the Hidden-Profile Paradigm. Small Group Res. 46 (5), 489–535. doi:10.1177/1046496415599068

CrossRef Full Text | Google Scholar

Steed, A., Steptoe, W., Oyekoya, W., Pece, F., Weyrich, T., Kautz, J., et al. (2012). Beaming: An Asymmetric Telepresence System. IEEE Comput. Grap. Appl. 32 (6), 10–17. doi:10.1109/MCG.2012.110

PubMed Abstract | CrossRef Full Text | Google Scholar

Steinmacher, I., Chaves, A. P., and Gerosa, M. A. (2010). Awareness Support in Global Software Development: A Systematic Review Based on the 3C Collaboration Model. Lecture Notes Comp. Sci.6257, 185–201. doi:10.1007/978-3-642-15714-1_15

CrossRef Full Text | Google Scholar

Voida, A., Voida, S., Greenberg, S., and He, H. A. (2008). “Asymmetry in media Spaces”, in Proceedings of the ACM Conference on Computer Supported Cooperative Work, CSCW, San Diego, CA, USA, November 8-12, 2008, 313–322. doi:10.1145/1460563.1460615

CrossRef Full Text | Google Scholar