Reality Stems From Modality: Stereotype Threat Effects of a STEM Game in Augmented and Virtual Reality

Ratan, Rabindra; Boumis, Josephine K.; Kuang, Sarah; Gambino, Andrew; Huang, Kuo-Ting

doi:10.3389/frvir.2021.636643

ORIGINAL RESEARCH article

Front. Virtual Real., 12 August 2021

Sec. Virtual Reality and Human Behaviour

Volume 2 - 2021 | https://doi.org/10.3389/frvir.2021.636643

This article is part of the Research Topic Presence and Beyond: Evaluating User Experience in AR/MR/VR View all 16 articles

Reality Stems From Modality: Stereotype Threat Effects of a STEM Game in Augmented and Virtual Reality

Andrew Gambino⁴

¹Department of Media and Information, Michigan State University, East Lansing, MI, United States
²Department of Communication, University of South Florida, Tampa, FL, United States
³Department of Psychology, Michigan State University, East Lansing, MI, United States
⁴College of Communications, Pennsylvania State University, State College, PA, United States
⁵Department of Journalism, Ball State University, Muncie, IN, United States

This study examined the relationship between stereotype threat, game modality (augmented reality, virtual reality), and stereotypic beliefs about STEM fields. Results of a 2 [modality] x 2 [stereotype threat] factorial, between-subjects experiment with women participants (N = 64) suggest that gender stereotypes primed before playing the STEM game in AR induced stereotype threat, but induced stereotype reactance in VR. Specifically, for participants who played in AR, the stereotype-reinforcing prompt (compared to a counter-stereotype prompt) was associated with worse STEM-game performance, which mediated an increase in stereotypical beliefs about women in STEM. Conversely, for participants who played in VR, the stereotype-reinforcing prompt was associated with better STEM-game performance and more positive (i.e., counter-stereotypic) beliefs about women in STEM, though without mediation. These findings support the claim that stereotypes triggered in a STEM-gaming context have the potential to reinforce stereotypes in STEM fields. Researchers and practitioners should consider the implication that VR is potentially more male-stereotyped than AR, while AR makes stereotyped identity characteristics more accessible than VR.

Introduction

Although the percentage of women in science, technology, engineering, math (STEM) fields has been increasing worldwide (Wiest et al., 2017), gender inequality in STEM fields is still a major problem in the U.S., with women only representing 24% of the STEM workforce (Noonan, 2017). Organizations and programs that encourage more students, especially girls, toward STEM careers often utilize activities related to video games (Jenson et al., 2007; Collette, 2013), which act as an entry point to STEM thinking (e.g., design, programming; Giammarco et al., 2015). The present research extends an understanding of the relationship between video games and STEM to the understudied context of augmented reality (AR) and virtual reality (VR) gaming. Given their immersive nature, AR and VR are potentially better at facilitating STEM-relevant skills, such as spatial rotation ability (Spence and Feng, 2010; Granic et al., 2014), compared to traditional (e.g., flatscreen) gaming modalities. Although AR and VR both provide immersive experiences that can enhance learning outcomes, they are also fundamentally different in the ways that they present and allow users to interact with educational information, which may influence learning outcomes. Building from a recent finding that AR leads to better retention of auditory content, while VR is better for visual content (Huang et al., 2019), the present research compares VR and AR modalities with a common facet of video games and STEM fields: gender¹ stereotypes.

Just as STEM fields are male-dominated, male-catering, and often hostile environments for women (Smith et al., 2013), women also receive more negative commentary in gaming contexts than men, regardless of skill (Kuznekoff and Rose, 2013). They also tend to be underrepresented in games as characters (Behm-Morawitz and Mastro, 2009), often depicted as weak, dependent (“damsels in distress”), and sexually objectified (Dill and Thill, 2007; Near, 2013), all of which creates an unwelcoming climate for women players. Despite the fact that women represent 45% of U.S. video game players (Entertainment Software Association, 2018), women are less likely to identify themselves as “gamers” and report less video game use than men (Crawford and Gosling, 2005). This further fuels the stereotype that women are not “real gamers” who prefer more casual and mobile games focused on fantasy and completion rather than action and competition (Hartmann and Klimmt, 2006; Yee, 2017).

Such gender stereotypes have a harmful, self-reinforcing effect (Kaye and Pennington, 2016; Shen et al., 2016) that potentially influences performance and attitudes not only in video game contexts but also in STEM contexts. Just as gender stereotypes introduced to children in the home (e.g., by parents) have been found to affect girls’ beliefs about self-efficacy in STEM fields (Gunderson et al., 2012), gender stereotypes propagated through video games may have a similar effect, especially given that video games often serve as a gateway to STEM learning (Giammarco et al., 2015).

Combining these threads, this research examines the potential for gender stereotypes introduced in an AR or VR STEM-gaming context to influence stereotypic beliefs about women in STEM fields. This study is one of the first (Fordham et al., 2020) to examine the relationship between stereotypes in a STEM-gaming context and gender-stereotypic beliefs about STEM fields. This study also contributes a novel examination of differences in the outcomes of gender stereotypes between AR and VR in educational gaming. Such differences potentially relate to varying stereotypical associations with these two technologies. This study has implications for theoretical understandings of stereotypes in STEM-gaming contexts as well as for practical implications related to the development of AR and VR video games, particularly those that intend to promote equitable learning outcomes.

Stereotype Threat and Stereotype Reactance

In order to understand the effects of stereotypes in gaming contexts more deeply, we delve into concepts that explain how people respond to stereotypes, namely, stereotype threat and stereotype reactance. Stereotype threat occurs when individuals respond to a subtle reminder of a negative stereotype by conforming behaviorally to the stereotype (Steele, 1997). For example, a woman is more likely to perform poorly on a math test after being told that the test tends to yield gender differences—a subtle reminder of the stereotype—compared to being told that the test yields no gender differences (Spencer et al., 1999). This phenomenon occurs because subtle introductions of stereotypes have potent effects on a subconscious level (Nguyen and Ryan, 2008), leading to increased anxiety, arousal, and other factors that are often imperceivable to the individual (Shapiro and Neuberg, 2007). The phenomenon has been studied and replicated in many different contexts, from cognitive performance to physical activity to workplace performance (Azzarito and Harrison, 2008; Nguyen and Ryan, 2008; Pennington et al., 2016).

In contrast, stereotype reactance—derived from the theory of psychological reactance (Brehm, 1966)—occurs when individuals respond to a blatant reminder of a stereotype by behaving in ways that disconfirm the stereotype (Kray et al., 2004). For example, women were more effective in a negotiation task after being explicitly told that masculine traits are associated with negotiation success—an overt reminder of the stereotype—compared to receiving an implicit reminder of this stereotype. This phenomenon occurs when the recognition of a threat to individual freedom triggers anger and other negative emotions that lead the individual to attempt to assert their freedom (Miron and Brehm, 2006) and in a sense, resist stereotype threat (Pennington et al., 2016).

To summarize, when people experience stereotype threat, they conform to negative stereotypes about their social group. When people experience stereotype reactance, they act in ways that contradict the stereotype. The likelihood of whether someone experiences stereotype threat or reactance depends on whether the stereotype is triggered subtly or overtly, respectively. Thus, in the context of video games, when gender stereotypes are communicated subtly, they may have harmful effects (e.g., stereotype-consistent beliefs) through stereotype threat, but when made overt, these stereotypes may lead to stereotype reactance (e.g., counter-stereotypic beliefs).

Stereotypes, Video Games and STEM

The potential that stereotype threats in video game contexts influence performance and stereotypic beliefs—both within and beyond the gaming context—has been examined in multiple studies. In one experiment, a stereotype threat that prompted gaming ability led to women underperforming compared to men (Kaye and Pennington, 2016). In another study, when women participants who strongly identified as gamers were exposed to stereotype threat (i.e., a male-dominated leaderboard), they performed worse at a puzzle-platform game and reported lower self-confidence than women with weaker gamer identities (Vermeulen et al., 2016). These studies support the idea that gender-related stereotype threats prompted in the context of educational video games influences behavior.

Given the potential connection between video game contexts and STEM fields, it is important to examine the relationship between gender stereotypes, gaming performance, and gendered-stereotypic beliefs about women in STEM fields. In one study (Fordham et al., 2020), women participants who read an article stating that men are more skilled than women at video games (compared to a women-are-as-good-as-men article) performed worse in a first-person-shooter game and rated STEM fields as better suited for men than women. Further, participants in that study who were led to believe that the opponent was a man compared to a woman also rated STEM fields as better suited for men than women. These findings are consistent with the notion that stereotype threats in the context of video gaming impact women, illustrating the link between stereotypes in video game contexts and in STEM fields.

In a second study (Fordham et al., 2020), participants were presented either a hateful nonsexist or sexist message, prompting experiences of threat and sexism. Afterward, they customized a video game character that would represent either the shooter game’s story or themselves, prompting self-concept, which was expected to increase the recognition of the stereotype threat and thereby trigger reactance. The condition that made the stereotype most blatant—a sexist message plus a self-representing avatar—led to more positive beliefs about women in STEM fields, supporting the expectation of stereotype reactance. In contrast, the condition that made the stereotype subtle—a sexist message plus a game-representing avatar—led to more negative beliefs about women in STEM fields, consistent with stereotype threat. This study supports the notion that stereotype threat and stereotype reactance outcomes in video game contexts—reflected by stereotypic beliefs about women in STEM fields—depend on the extent to which the gender and gaming stereotype is made subtle or overt.

Together, these findings suggest that reminders of gender stereotypes in gaming contexts may influence game performance and the endorsement of gender stereotypes in STEM fields, but the blatancy of the reminder influences whether the stereotype has negative (stereotype threat) or positive (stereotype reactance) outcomes. In this largely understudied context of video games and STEM stereotypes, it is difficult to predict whether a stereotype reminder is blatant enough to exceed the threshold of stereotype reactance. If the prompt is below this threshold, then we would expect it to cause stereotype threat; namely, subsequent performance and beliefs would be aligned with the stereotype. If the prompt is overt enough to trigger stereotype reactance, then we would expect subsequent performance and beliefs to contradict the stereotype. In either case, we expect that people who have been prompted with gender stereotypes prior to playing a STEM game would exhibit different performance compared to people who receive a counter-stereotypic prompt. However, because we do not know the extent to which the stereotype prompt will be perceived as subtle or overt, we pose a non-directional research question.

Research Question 1: Do gender stereotypes prompted in a STEM-gaming context influence 1) STEM-game performance, and 2) stereotypic beliefs about women in STEM fields?

Stereotypes, Augmented/Virtual Reality, and STEM

Augmented reality and virtual reality are becoming more commonplace in education contexts (Wu et al., 2013; Merchant et al., 2014). AR typically incorporates digital images into a physical space with real world objects while VR isolates the user from the physical environment and immerses them in a new virtual world (Milgram et al., 1995). In light of new advances, researchers have examined AR and VR effects on education and learning (Wu et al., 2013; Merchant et al., 2014). For example, VR and AR have been shown to differentially impact student learning (Huang et al., 2019). While such research provides valuable insights into the potential for these technologies to be integrated into educational contexts, few have examined how differences between these technologies relate to social factors, such as gender and stereotypes. There is a broad field of research on gender gaps in internet and computer use (Hargittai and Shafer, 2006; van Deursen and van Dijk, 2014) suggesting that gendered differences in self-efficacy and other attitudes about technology are associated with digital skills (Correll, 2001; Beckwith et al., 2007; Huffman et al., 2013). However, little if any research has examined such gaps with respect to AR and VR in educational contexts. Still, previous research on these two modalities can be synthesized to argue that AR and VR likely differ in the extent to which they are associated with gender stereotypes.

VR is Potentially More Gender-Stereotyped than AR

VR is potentially oriented toward and thus associated with men more than women, thereby reinforcing gender stereotypes related to this technology. Studies suggest men have an advantage over women in VR contexts, particularly regarding susceptibility to cybersickness, spatial tasks, and cognitive performance (Larson et al., 1999; Terlecki and Newcombe, 2005; Munafo et al., 2017). The aptly named study Virtual Reality Is Sexist: But It Does Not Have to Be (Stanney et al., 2020) found that incorrect interpupillary distance (IPD) fit of head-mounted VR devices contributes to sex differences in cybersickness, suggesting that VR devices have not been designed to sufficiently consider women users. Although a systematic review found conflicting evidence of sex differences in cybersickness (Grassini and Laumann, 2020)—and other sex-associated technology-use differences such as spatial rotation skills-have been found to dissipate after practice (Rodán et al., 2016; Spence and Feng, 2010), cultural assumptions and stereotypes about this technology have persisted. Men are more likely to own and intend to own consumer VR headsets (Clement, 2021), possibly due to the growing availability and appeal of virtual reality video games (Foxman et al., 2020; Kosa et al., 2020), and men have been found to enjoy and intend to play virtual reality games more than women (Um, 2020).

In contrast, AR technologies are far less gender-stereotyped. The arguably most widely adopted AR game to date—Pokémon GO—has been popular across gender and age classifications (Serino et al., 2016), with one study receiving a higher response rate of women (58%) than men players from a sample of over 600 respondents recruited on internet forums. Further, in comparison to the offerings of VR games that are oriented toward men, AR games span a range of genres (Tan and Soh, 2010). AR is also being readily adopted outside of video gaming in more gender-balanced contexts, such as in social media and marketing (Bin Mohd Nasir, 2015). Together, AR seems to be evolving into a technology that is more balanced across genders and media applications than VR.

VR Masks Self-Identity Cues More than AR

In contrast with the argument that VR tends to be more male-stereotyped than AR, stereotype-triggering cues may become less salient during a VR than an AR task because VR is more likely to mask identity cues to a greater extent than AR (assuming there are no self-representing avatars in either scenario). In other words, VR isolates the individual from most reminders of the outside world, but when using AR, the individual can still see elements of the outside world. One such element of the outside world is the individual’s own body, which is hidden from the user in VR, but is a direct representation of the individual’s non-digital self in AR. Hence, in AR, users are more likely to be reminded of their own identity characteristics, including gender. In the context of gender stereotypes, VR users (compared to AR users) are less likely to be reminded of their own gender, making them less susceptible to gender-related stereotypes.

Do Stereotypes Effects Differ between VR and AR?

The previous two sections argue that VR might be stereotyped as male more than AR, but VR might also mask self-identity characteristics more than AR. These differences may influence the effects of a gender stereotype prompt within a STEM-gaming context (e.g., reading an article stating that women perform worse with digital technology than men compared to an article saying they perform equivalently). As described earlier, if a stereotype is presented in a subtle way, it is more likely to lead to stereotype threat than if it is presented in a blatant way, which potentially leads to stereotype reactance.

Differences between VR and AR in gender-stereotype associations and in the salience of self-identity characteristics may influence perceptions of a stereotype prompt as being subtle or blatant, thereby influencing whether there is a stereotype threat or stereotype reactance response. If VR is associated with gender stereotypes to a greater extent than AR is, a gender-stereotypic prompt in a VR context could make the gender stereotype more blatant than it would be in an AR context. If this prompt then exceeds the threshold to be perceived as blatant—which would be more likely in VR than AR—then it would lead to stereotype reactance in VR and stereotype threat in AR. At the same time, if VR masks identity cues to a greater extent than AR does, a gender-stereotypic prompt in a VR context might be perceived as more subtle than it would be in an AR context, where users are more likely already primed with a gender-associated self-concept. If this prompt then exceeds the threshold to be perceived as blatant—which would be more likely in AR than VR—then it would lead to stereotype reactance in AR and stereotype threat in VR. Hence, because we do not know the extent to which a stereotype prompt will be perceived as subtle or overt, we pose the following open-ended question.

Research Question 2: Is the effect of gender-stereotype prompts in a STEM-gaming context on 1) STEM-game performance and 2) beliefs about women in STEM fields is moderated by modality (i.e., AR or VR)?

Performance as Mediator of Effects on Stereotype-Consistent Beliefs

Up until this point, effects on STEM-gaming performance and gender-stereotypic beliefs about STEM fields have been discussed as separate outcomes of gaming-related stereotype reminders, but the two outcomes may be related. If performance is interpreted as an indicator of ability, then negative performance in the STEM-gaming context may lead individuals to believe that they are not as well-suited for related contexts, such as STEM fields. Thus, the harmful effect of gaming-related stereotype threat on beliefs about women in STEM fields—reflected by attitudes about how well women perform relative to men in those specific fields (e.g., science, technology, etc.) might be mediated by the harmful effect of gaming-related stereotype threat on STEM-gaming performance. Taking the previous expectation that modality moderates the effect of stereotype prompts, we hypothesize the following moderated mediation relationship.

Hypothesis 1: The effect of gender-stereotype prompts on beliefs about women in STEM fields—moderated by gaming modality—is mediated by STEM-game performance.

Effects on Non-STEM Fields?

One final consideration is whether the effects of stereotype threats in a STEM-gaming context is restricted to STEM fields or if they extend into non-STEM fields as well. The arguments here suggest that stereotypical associations of gaming contexts relate more to STEM fields than non-STEM fields given the stronger technical focus of both video games and STEM. However, it is possible that the effect of stereotype threat in a STEM-gaming context also extends into non-STEM fields through other associations, such as changes in mood, self-efficacy, or motivation. Therefore, we are interested to see if the effect of the gender-stereotype prompts extends into participants’ beliefs concerning how well women perform relative to men in non-STEM fields (e.g., English, Language, Education, and Humanities). In order to put this logic up against the claim that the phenomenon is unique to STEM fields, we pose the following question.

Research Question 3: Do the effects of stereotype threat on beliefs about women in STEM fields also extend into non-STEM fields?

Materials and Methods

Participants

The current study builds off the same dataset as a previously published paper (Huang et al., 2019), which focused on different aspects of the dataset (i.e., spatial presence and knowledge-retention differences between AR and VR, but not gender stereotypes). To explore the given hypotheses, the current study consists of a between-subjects design with participants randomly assigned to one of the conditions in the 2 (modality: AR or VR) x 2 (stereotype prompt: stereotype-reinforcing or counter-stereotype). Participants from a large Midwestern university took part in an Institutional Review Board approved study. Through an interdepartmental research subject pool, 109 participants, with an average age of 20.5 years old (SD = 1.61), were recruited for extra credit. Given the focus of the study on gendered stereotype threat and STEM-related beliefs, we chose to include only women participants that passed the manipulation check in the subsequent analyses (n = 64).

Materials

The current study was conducted in a 10′ x 8′ segmented office space in order to reduce distractions. The office contained beige curtains, beige walls, and an empty desk with only a desktop computer. The desktop computer contained the survey that was already projected on the screen when the participant entered.

The Solar System–Space Museum mobile app, developed by ZeeMelApps (available at https://play.google.com/store/apps/details?id=com.zeemel.spacemuseumvr), was displayed on a Samsung S4 smartphone. This app allowed for the digital and auditory content to be presented in both VR and AR modes while still providing the same amount of information. The app showed three-dimensional visual representations of the solar system complimented by auditory commentary and information about the objects shown in the app such as planets and galaxies. In the AR mode, participants viewed the physical environment around them with a non-interactive layer on top of the portrayed image of the solar system digital content. Participants in this AR mode held the smartphone in their hands. In the VR mode, participants wore a Mattel VR Viewmaster phone-based VR headset. The solar system digital content in the VR mode was displayed in front of a white background.

Although the navigation screens were different in the two modes, researchers explained how to select the same option in both modes to participants to reduce discrepancies. Specifically, in the AR mode participants touched the “solar system” option on the screen, whereas in the VR mode participants used gaze selection to highlight “solar system” and then selected the highlighted option by pressing on a button on the top of the VR headset. To increase consistency between the two modes, participants were told to stand up and were explicitly told they were able to move in 360° to view the digital content in both modes. Participants were additionally reminded in the AR mode to hold the smartphone in front of their faces. The audio content transmitted through the smartphone’s built-in speakers was identical in each mode.

Procedure

After signing a consent form which provided a brief description of the research purpose (i.e., to understand how different people experience mixed reality games), participants were directed to a desk that projected a survey on the screen. Participants completed a pre-test questionnaire that contained questions measuring their solar system knowledge, demographics, and were randomly prompted to read one of two short articles that presented an abstract from what appeared to be a published research article. The fictitious article abstracts, derived by the research team by making minor changes to an actually published abstract (Shen et al., 2016), contained a stereotype-reinforcing or a counter-stereotype statement about women’s performance and participation in digital technologies. The stereotype-reinforcing article (see Supplementary Appendix A) claimed men advanced faster than women in technology use, whereas the counter-stereotype article (see Supplementary Appendix B) said men and women advance equally. At the end of the survey, participants answered a question about the study presented in the article as a manipulation check.

At the conclusion of the pre-survey, participants were instructed on the screen to tell the research assistant that they had completed the survey. The research assistant then explained how to use the solar system application (either AR or VR, depending on condition) and to pay attention to the information presented on the app. The research assistant then situated the participant with the technology, asked for questions, and gave instructions on how to start the app. After the participant selected the “solar system” option on the app, visual and auditory information played on the app for under 5 min. This time allowed for the “solar system” option to completely display the entirety of the visual and audio information. Participants then played a short game in which they destroyed incoming asteroids by moving their heads to aim and shoot (within the Solar System app) or by using their finger on the phone screen for 2 min—in order to reinforce the gaming-related nature of the context—and then, completed the post-survey, starting with items on performance, then spatial presence, and concluded with items on gender-stereotypic beliefs about STEM and non-STEM fields.

Measures

Performance was measured using an original scale developed for this specific context. Past research created similar indices to gauge performance by using learning outcomes in both AR (Lin et al., 2013) and VR (Kockro et al., 2015). Participants were aware that their performance assessment would be based on information presented in the game. Therefore, they were asked multiple choice questions about the information provided by the mobile app either through a spoken voice (e.g., “When did the solar system form?”) or visually on the screen (e.g., “What was the color of Neptune?”). Given that learning efficacy for auditory and visual information differs between AR and VR (Huang et al., 2019), we included an even mix of information provided through visual and auditory channels in our analysis. In order to ensure a sufficient variance in performance between participants, items were only retained if more than 30% of participants answered correctly. In other words, the items which 70% or more participants incorrectly were deemed too difficult—potentially leading to a floor effect and noise in the data—so they were not included in the composite metric, thereby ensuring that this metric had a sufficient level of variance to reflect a signal in the data. Our final measure of performance included five audio questions and five visual questions. Participants’ performance score was computed as the proportion of questions answered correctly (M = 0.60, SD = 0.21).

Gender-stereotypic beliefs about STEM fields was derived from ratings for each of five fields—Science, Computer Science, Engineering, Mathematics, and Video Game Design—in response to the question, “please rate how much you think men or women are better at the following.” Responses were coded on a 100-point (unnumbered) sliding scale anchored by “WOMEN are much better” and “MEN are much better”, with higher scores indicating more positive beliefs about women. A composite score was generated from the mean response across the five fields, measured in both the posttest (α = 0.86; M = 41.89, SD = 13.62)—which was a primary dependent variable of interest—and pretest (α = 0.88; M = 41.75, SD = 15.90) as a covariate used to control for the potential influence or pre-existing gender-stereotypic beliefs about STEM fields.

Gender-stereotypic beliefs about non-STEM fields was derived from responses to the same question as in the STEM-fields measure, but with respect to these fields: Humanities, English, Education, and Language. A composite score was generated from the mean response across the five fields, measured in both the posttest (α = 0.89; M = 64.75, SD = 15.46) and pretest (α = 0.88; M = 65.36, SD = 15.81), as with the previous measure. The questionnaire interspersed the STEM and non-STEM items in order to help mask the study purpose.

Spatial presence—the perceptual illusion of physically being in a mediated space (Biocca, 1997; Lombard and Ditton, 1997)—differs between AR and VR and this may be an important cause of differences in the outcomes of using these media modalities (Riva et al., 2016). One study found that using the VR (fully mediated) compared to AR (digital overlay onto a physical environment) mode of the same educational application led to more spatial presence, which mediated the effect on application-related task performance (Huang et al., 2019). Hence, spatial presence was included as a covariate in order to control for its potential effect on learning performance and stereotypic attitudes in this context. This was measured with five items from a revised version of an immersive virtual technologies scale on a 7-point Likert scale (Fox et al., 2009). Example items include “To what extent did it feel like you visited another place?” and “To what extent did you feel surrounded by the virtual world?”. A composite measure was constructed from a mean of these scores (α = 0.88).

Space Knowledge was measured on the pretest with ten multiple choice questions regarding general knowledge about our solar system (e.g., “Is Earth larger or smaller than most of the other planets?”, “Which are the gas giants?”, and “True or false: the Sun’s gravity is the strongest gravity in the solar system.”) Each participant was given a point for each right answer, then these points were added together for a final space knowledge score (M = 5.38, SD = 1.54). This measure was included as a covariate given the potential that space knowledge prior to the study would influence participants ability on the main performance measure.

Results

Manipulation Check and Pretest Equivalence

A manipulation check was employed to ensure participants read and understood the stereotype-reinforcing or counter-stereotype article through answering the question, “According to the article, women advance ____ in skill level as men.” Out of 81 participants, 17 answered incorrectly, likely because they did not carefully read or understand the abstract, which was in academic language. Given that it was essential for participants to understand the article, those who failed the manipulation check were removed from the analysis, leaving 64 participants.

To confirm that scores for pretest variables did not differ between condition, we conducted a multivariate analysis of variance (MANOVA) test with stereotype condition and AR/VR condition as the fixed factors and the three pretest outcomes of interest (gender-stereotypic beliefs about STEM fields; gender-stereotypic beliefs about non-STEM fields; space knowledge) as the dependent variables. Neither the main nor interaction effects were found to be significant for the omnibus multivariate test (all p values over 0.680) nor for the individual between-subjects tests (all p values over 0.234), suggesting that random assignment to condition was successful.

Analysis of Covariance Tests on STEM-Game Performance

In order to examine RQ1a (Do gender stereotypes prompted in a STEM-gaming context influence STEM-game performance?) and RQ2a (Is the effect of gender-stereotype prompts on STEM-game performance moderated by modality?), we conducted an analysis of covariance (ANCOVA) with stereotype condition and AR/VR condition as the manipulated independent variables, spatial presence and space knowledge as covariates, and participant performance as the outcome variable. No main effects of the manipulated independent variables were significant, but a significant interaction effect was found for stereotype threat by AR/VR condition, F (1, 63) = 12.05, p < 0.001, η_p² = 0.17 (see Figure 1). For participants in the AR condition, the stereotype-reinforcing article hindered STEM-game performance (M = 0.51, SE = 0.05) compared to counter-stereotype article (M = 0.65, SE = 0.05), but for participants in the VR condition, the stereotype-reinforcing article was associated with better performance (M = 0.72, SE = 0.05) compared to counter-stereotype article (M = 0.53, SE = 0.05).

FIGURE 1

FIGURE 1. Impact of stereotype threat prompt article and modality on performance (means and confidence intervals).

To probe this interaction further, two simple effects tests were conducted. Analyzing only participants in the AR condition, those who received the stereotype-reinforcing article exhibited significantly worse performance (M = 0.54, S.E. = 0.05) than those who received the counter-stereotype article (M = 0.75, S.E. = 0.05), F (1, 29) = 10.00, p = 0.004, η_p² = 0.28, consistent with stereotype threat. Analyzing only participants in the VR condition, the difference approached significance, F (1, 33) = 3.55, p = 0.069, η_p² = 0.11, with those who received the stereotype-reinforcing article exhibiting better performance (M = 0.50, S.E. = 0.05) than those who received the counter-stereotype article (M = 0.65, S.E. = 0.05).

As a final probe of this interaction, two additional simple effects tests were conducted with splits in the opposite direction. Analyzing only participants in the stereotype-reinforcing condition, the difference approached significance, F (1, 31) = 4.02, p = 0.055, η_p² = 0.13, with those who used VR exhibiting better performance (M = 0.68, S.E. = 0.05) than those who used AR (M = 0.52, S.E. = 0.05). The difference among participants in the counter-stereotype condition was significant, F (1, 31) = 8.16, p = 0.008, η_p² = 0.23, with those who used VR exhibiting significantly worse performance (M = 0.50, S.E. = 0.05) than those who used AR (M = 0.72, S.E. = 0.05).

Therefore, these results inform RQ1a, suggesting that gender stereotypes do indeed influence STEM-game performance. Modality was found to moderate the effect of gender stereotype on performance. According to the simple-effects tests, the stereotype-reinforcing article hindered performance in AR (consistent with stereotype threat) but increased performance in VR (consistent with stereotype reactance, albeit approaching significance). Further, for participants who read the counter-stereotype article, VR was associated with significantly worse performance than AR, while for participants who read the stereotype-reinforcing article, VR was associated with better performance than AR (albeit approaching significance). Together, these results inform RQ2a, suggesting that the stereotype-reinforcing prompt led to stereotype threat in AR and stereotype reactance in VR.

Analysis of Covariance Tests on Gender-Stereotypic Beliefs About STEM

In order to examine RQ1b (Do gender stereotypes prompted in a STEM-gaming context influence gender-stereotypic beliefs about STEM?) and RQ2b (Is the effect of gender-stereotype prompts on beliefs about women in STEM fields moderated by modality?), we conducted a repeated measures ANCOVA with stereotype condition and AR/VR condition as the manipulated independent variables, spatial presence and space knowledge as covariates, and change in gender-stereotypic beliefs about STEM from pretest to post-test as the outcome variable. No main effects of the manipulated independent variables were found significant, but a nearly significant interaction effect was found for stereotype threat by AR/VR condition, F (1, 58) = 4.00, p = 0.051, η_p² = 0.06 (see Figure 2). For participants in the AR condition, the stereotype-reinforcing article was associated with a negative change in beliefs about women in STEM fields (M = −1.95, SE = 2.56), while the counter-stereotype article was associated with a positive change in beliefs (M = 1.28, SE = 2.41). Conversely, for participants in the VR condition, the stereotype-reinforcing article was associated with a positive change (counter-stereotypic) in beliefs about women in STEM fields (M = 3.65, SE = 2.25), while the counter-stereotype article was associated with a negative change in beliefs (M = −2.51, SE = 2.45).

FIGURE 2

FIGURE 2. Impact of stereotype threat prompt article and modality on change in gender-stereotypic beliefs about STEM fields (means and confidence intervals).

To probe this interaction further, two simple effects tests were conducted. Analyzing only participants in the AR condition, no significant difference was found between the stereotype-reinforcing article and counter-stereotype article conditions. Analyzing only participants in the VR condition, there was a significant difference, F (1, 30) = 4.25, p = 0.048, η_p² = 0.12, with those who received the stereotype-reinforcing article exhibiting more positive change (counter-stereotypic) in beliefs about women in STEM fields (M = 4.15, SE = 1.97) than those who received the counter-stereotype article (M = −1.93, SE = 1.97).

As a final probe of this interaction, two additional simple effects tests were conducted with splits in the opposite direction. Analyzing only participants in the stereotype-reinforcing condition, there was a significant difference, F (1, 28) = 4.30, p = 0.048, η_p² = 0.13, with those used VR exhibiting more positive change (counter-stereotypic) in beliefs about women in STEM fields (M = 3.20, SE = 1.70) than those in the AR condition (M = −2.17, SE = 1.82). Analyzing only participants in the counter-stereotype condition, no significant difference was found between the stereotype-reinforcing article and counter-stereotype article conditions.

These results inform RQ1b, with gender stereotypes prompted in the STEM-gaming context found to influence gender-stereotypic beliefs about STEM. Further, modality was found to moderate the effect of gender stereotypes in the STEM-gaming context on gender-stereotypic beliefs about STEM. According to the simple-effects test, the stereotype-reinforcing article was associated with more counter-stereotypic beliefs in VR (consistent with stereotype reactance), though no difference was found in AR. Further, for participants who received the stereotype-reinforcing article, VR was associated with more counter-stereotypic beliefs than AR. Together, these results inform RQ2b and suggest that the gender-stereotype prompt induced stereotype reactance for participants in the VR condition.

Analysis of Covariance Test for RQ3

To examine RQ3 we conducted a repeated measures ANCOVA with stereotype condition and AR/VR condition as the manipulated independent variables, spatial presence and space knowledge as covariates, and change in gender-stereotypic beliefs about non-STEM fields from pretest to post-test as the outcome variable. No significant main effects nor interaction effects were found. These findings inform RQ3 (Do the effects of stereotype threat on beliefs about women in STEM fields also extend into non-STEM fields?), providing no evidence that gender stereotypes triggered in the STEM-gaming context influence perceptions of non-STEM fields.

Ordinary Least Squares Regression Path Analysis

Finally, we tested the expectation for moderated mediation in H1 (The effect of gender-stereotype prompts on beliefs about women in STEM fields—moderated by gaming modality—is mediated by STEM-game performance) by using ordinary least squares regression path analysis to perform a moderated mediation analysis (Hayes PROCESS, Model 7, 10,000 bootstrapped samples) with stereotypic beliefs about women in STEM (post-test) as the outcome, stereotype condition as the predictor, STEM-game modality as the moderator, STEM-game performance as the mediator, and stereotypic beliefs about women in STEM (at pretest), preknowledge, and spatial presence as covariates. The index of moderated mediation was significant [B = −4.12, LLCI: −8.60, ULCI −0.87], with STEM-game performance significantly mediating the effect of stereotype condition on stereotypic beliefs about women in STEM in the AR condition [B = −2.59, LLCI: −5.43, ULCI −0.53], but not in the VR condition [B = 1.53, LLCI: −0.19, ULCI 4.25]. In other words, for participants in the AR condition, the gender-stereotype prompt was associated with more negative attitudes about women in STEM fields and this effect was mediated by a reduction in STEM-game performance, but there was no such mediation effect for participants in the VR condition. These results provide partial support for H1.

Discussion

In response to the potential link between gender disparity in video games and STEM fields, the present study examined whether gender stereotypes in a STEM-game influences gender-stereotypic views of STEM fields and whether such influence differs depending on gaming modality (i.e., AR vs. VR). Results suggest that gender stereotypes prompted before playing the STEM game—through an article reinforcing or countering gender stereotypes about gaming ability—influenced game performance and STEM beliefs in a direction consistent with stereotype threat in AR and with stereotype reactance in VR. Specifically, for participants who played in AR, the stereotype-reinforcing prompt (compared to the counter-stereotype prompt) was associated with worse game performance and more stereotype-consistent beliefs about women in STEM (albeit with the stereotype-consistent beliefs finding approaching significance in the simple-effects test). Conversely, for participants who played in VR, the stereotype-reinforcing prompt was associated with better STEM-game performance and more counter-stereotypic beliefs about women in STEM (albeit with the performance finding approaching significance). Further, the effect of the stereotype-reinforcing prompt on beliefs about women in STEM was mediated by game performance, though only in AR. In contrast, the stereotype-reinforcing prompt was not found to influence beliefs about non-STEM fields, likely because these fields are not stereotyped in the same way as video games and STEM fields. Altogether, these findings support the argument that stereotypes triggered in a STEM-gaming context have the potential to reinforce stereotypes in STEM fields. Further, VR games appear to be more likely than AR games to cause gendered stereotype threat in the absence of additional stereotype prompts, but AR games seem more likely to cause such threat than VR when additional stereotype reminders are present.

The differential effects found between AR and VR modality potentially results from the distinction between stereotype threat and stereotype reactance. For participants who played the game in AR, the stereotype-reinforcing (compared to counter-stereotype) prompt was associated with worse game performance (significant for both interaction and simple effects) and more negative beliefs about women in STEM fields (approaching significance for interaction effect). This aligns with stereotype threat: being reminded of the stereotype that women have lower gaming ability led these participants to conform to the stereotype through a variety of potential mechanisms (e.g., increased cognitive load, lower self-efficacy, etc.). In contrast, for participants who played the game in VR, the stereotype-reinforcing (compared to counter-stereotype) prompt was associated with better game performance (approaching significance in the simple-effects test) and more positive beliefs about women in STEM fields. This might have occurred due to stereotype reactance—the phenomenon that when a stereotype is made explicitly salient to a member of the stereotyped group, they resist and actively attempt to counteract it. In this case, VR might be more male-stereotyped than AR. Thus, the stereotype-reinforcing prompt with VR-based gameplay was like a double-dosage of the stereotype reminder to which the participants would have been more likely to experience stereotype reactance. In other words, the stereotype-reinforcing prompt was relatively subtle in the AR condition (leading to stereotype threat) and relatively explicit in the VR condition (leading to stereotype reactance).

The finding of mediation for AR users (i.e., stereotype-related article —> game performance —> stereotypic attitudes) supports the argument that game performance may serve as an attitude-reinforcing mechanism that fuels the vicious cycle of stereotypes in video game and STEM contexts. If performance is interpreted as an indicator of ability, then performing poorly in the gaming context may signal to individuals that they will also perform poorly in related contexts, such as STEM fields. Hence, for women, when stereotype threat hindered performance, it also reinforced stereotypic attitudes about women in STEM fields. Although this logic seems sound, future research should be used to confirm the pattern and delve deeper into the mechanisms, especially given that the pattern was only found for AR and not VR users. For example, although not supported by the present study, the argument leading up to RQ2 may relate to this question. Namely, in AR, users can still see elements of the outside world, including themselves, while VR isolates the individual from most reminders of the outside world. Hence, AR might communicate reminders of personal identity characteristics—such as gender—more than VR, potentially complementing other subtle stereotype reminders that trigger stereotype threat, such as game performance.

The finding that the stereotype-reinforcing prompt induced stereotype threat in AR (not VR) notwithstanding, the present results suggest that VR might be more likely to cause stereotype threat in general educational contexts where overt gender stereotype reminders are less common. This somewhat counterintuitive inference builds from the argument that VR is more male-stereotyped than AR—given gender differences in cybersickness, consumer adoption (Bin Mohd Nasir, 2015; Clement, 2021), and gaming (Serino et al., 2016; Um, 2020). Consistent with this logic, in the present study, the stereotype-reinforcing prompt in the male-stereotyped VR condition made gender stereotypes explicit, thereby triggering stereotype reactance. However, without such a stereotype-reinforcing prompt, the stereotypical association of the VR context alone seemed to be sufficient to trigger stereotype threat. Consistent with these two points, the simple effects tests found that for participants who received the stereotype-reinforcing prompt, participants in VR (compared to AR) exhibited higher STEM-game performance (approaching significance) and counterstereotypic STEM attitudes (significant). In contrast, for participants who received the counter-stereotype prompt, STEM-game performance was lower and counterstereotypic STEM attitudes were higher in VR than AR (considering the nearly significant interaction effect for the latter). One possible interpretation is that in the absence of a stereotype-consistent prompt, women who play a STEM game in VR are likely to perform worse and endorse more gender-stereotypic attitudes about STEM fields compared to those who play the same game in AR because VR is more male stereotyped (at baseline) than AR. An alternative (unexpected) interpretation is that the counter-stereotypical prompt causes AR users to experience stereotype boost, an improvement in performance after being exposed to a positive generalization about a personal social group (Shih et al., 2012), because identity cues such as gender are more salient in AR compared to VR. In other words, participants in AR were more likely than those in VR to be reminded of their gender identity—because VR literally occludes the user’s view of their own body—and thus experience a (counter-stereotype) boost after reading an article saying that women are as strong as men at video games.

Implications

This study offers two fundamental implications: 1) VR is potentially more male-stereotyped than AR; 2) AR makes personal identity cues more accessible than VR. Regarding #1, an important practical implication is that educators implementing VR in learning contexts should consider that women and girls may perceive this technology to be less inclusive of them than men and boys do. Of course, these perceptions can only change through exposure, so instructors and educators should not shy away from using VR, but instead should actively work to mitigate stereotype threat by encouraging equal use of VR by women and girls and otherwise working to dispel any gender stereotypes about the technology. Regarding #2, practitioners should recognize that female users are likely more susceptible to stereotype threat when using AR compared to VR and thus should be careful to avoid any subtle stereotype cues about gender that might arise in the context.

Technology designers could consider these implications as well. Studies suggest that VR has a great potential to influence stereotypes and implicit biases through embodiment in avatars (Peck et al., 2013; Banakou et al., 2016; Christofi and Michael-Grigoriou, 2017; Farmer and Maister, 2017). Avatars can also likely be implemented into AR (e.g., seeing a digital filter on your body when you look in a real mirror) to similar effect, though the research on this is limited due to the increased technological complexity of developing functional products. In any case, avatars (and avatar design options/guidelines) can be designed to deemphasize stereotyped identity characteristics and this may help mitigate stereotype threat (Fordham et al., 2020). Designers and practitioners should consider such potential effects and implement avatars in VR and AR in deliberate ways that will minimize negative effects of stereotypes.

Limitations and Future Directions

Some important limitations of this study should be noted. Most notably, only women participants were included in the analysis due to the study’s focus on gender-related stereotype threat. However, men’s beliefs about women in STEM fields might also be influenced by reminders of gender stereotypes in gaming contexts. Future studies should include men participants given this potential effect of gender stereotyping in gaming contexts reinforcing gender stereotypes in STEM for men as well as women.

The sample size for this study was quite small, resulting in the study being somewhat underpowered. However, in studies of virtual reality, smaller sample sizes are not uncommon (Cummings and Bailenson, 2016). Further, according to a sensitivity analysis using G*Power (Faul et al., 2007) with this study’s characteristics (i.e., sample size, degrees of freedom, number of covariates) as inputs, the study was able to detect outcomes with medium effect sizes (f = 0.458), and nearly all of the effects identified were in this range or larger. While future research should certainly use sample sizes that can provide sufficient power to detect smaller effects, this caveat should not detract from the reliability of the present findings.

The AR and VR technology utilized in this study was based on a mobile-phone platform in order to maintain consistency in the content between the experimental conditions; however, there are far more advanced AR and VR systems on the market that future research should explore. Further, the study relied on a single, education gaming context and this game was not particularly interactive. Hence, the results might have limited generalizability outside of educational gaming and with other game genres. Future research should compare AR and VR in other gaming (and also non-gaming) contexts. This is especially important because previous research has found some gender differences in gaming genre preferences (Greenberg et al., 2010), but these trends might be changing (Wohn et al., 2020) and stereotypes about gender differences in gaming ability are often inconsistent with reality (Shen et al., 2016; Ratan et al., 2020).

Finally, future research on this topic could add value with younger participants, implicit measures in addition to self-report, measures of social identities related to being a gamer (which might mediate effects of stereotype threat), assessments and controls of the participants’ previous exposure to AR and VR technology, and qualitative methods (e.g., interviews) to better understand how women and men perceive the differences between AR and VR.

Conclusion

The present study adds to the growing evidence that just as video games have been touted as a means of improving gender equality in STEM fields, gender stereotypes in STEM-gaming contexts may contribute to gender inequity in STEM fields, which then reinforces the deleterious stereotypes across STEM-related contexts. Future research should continue to examine the factors that both fuel and could be harnessed to break this vicious cycle, such as game modality.

Data Availability Statement

The datasets presented in this article are not readily available due to the consent process used at the time of data collection. Requests to access the datasets should be directed to rar@msu.edu.

Ethics Statement

The study involved human participants and was reviewed and approved by the Institutional Review Board at Michigan State University. The participants provided their informed consent to participate in this study.

Author Contributions

RR: Conceptualization; Data curation; Formal analysis; Funding acquisition; Methodology; Project administration; Supervision; Roles/Writing – original draft; Writing – review and; editing JB: Conceptualization; Data collection; Formal analysis; Methodology; Project administration; Supervision; Roles/Writing – original draft; Writing – review and; editing SK: Formal analysis; Roles/Writing – original draft; Writing – review and; editing AG: Formal analysis; Writing – review and; editing K-TH: Conceptualization; Data collection; Methodology; Project administration; Supervision; Writing – review and; editing.

Funding

This research was partially supported by the Honors College at Michigan State University. The funding came in the form of a gift and the funders were not involved in the study design in any way. We would also like to thank the AT&T endowment to the Media and Information Department at MSU for supporting Dr. Ratan’s AT&T Scholar position.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frvir.2021.636643/full#supplementary-material

Footnotes

¹Although that gender refers to non-binary identity characteristics in some contexts (Bem, 1981; Ansara and Berger, 2016; Lips, 2017), this paper focuses on gender-majority groups (i.e., men and women), consistent with video game literature (Kuznekoff and Rose, 2013; Kaye and Pennington, 2016; Shen et al., 2016) and previous research on gender-stereotypes (e.g., design, programming; Hargittai and Shafer, 2006; Nguyen and Ryan, 2008; Giammarco et al., 2015).

References

Ansara, Y. G., and Berger, I. (2016). Cisgenderism. in The Wiley Blackwell Encyclopedia of Gender and Sexuality Studies. Editors N. Naples, R. C. Hoogland, M. Wickramasinghe, and W. C. A. Wong (Oxford: Blackwell), 137–160. doi:10.1002/9781118663219.wbegss426