Evidence of a Relationship Between Mental Rotation Skills and Performance in a 3D Puzzle Game

Spatial reasoning is an ability that people utilize on a daily basis, that has also been linked to performance in Science, Technology, Engineering, and Mathematics (STEM) fields. However, there are very few widely available opportunities to train spatial reasoning skills that have been proven to be effective tools. As a first step in the validation process, this study sought to establish whether performance on a measure of intrinsic and dynamic spatial reasoning ability was related to performance within Optica, a mobile puzzle game. To investigate this relationship, 168 middle school students participated in a within-subjects study over three days. The results of this study have been promising, as our analysis indicated that there was a significant relationship between the number of levels completed in Optica and score on the Purdue Spatial Visualization Test—Revised: Visualization of Rotations (Revised PSVT: R).

Introduction

People use spatial reasoning on a daily basis to accomplish a variety of tasks, such as navigating a city with a map, packing their bag for a vacation, or playing a video game. Spatial reasoning skills are not only important for everyday activities, but evidence suggests that there is a relationship to academic success in the fields of Science, Technology, Engineering, and Mathematics (STEM). Specifically, stronger performance on the Purdue Spatial Visualization Test—Revised: Visualization of Rotations (Revised PSVT: R) was found to be associated with higher rates of academic success among first-year engineering students at a large midwestern university in the United States (Maeda et al., 2013). In addition, they were able to more accurately predict academic outcomes when considering a student's performance on the Revised PSVT: R in addition to the Verbal and Mathematical portions of the Scholastic Aptitude Test (SAT) than when using the SAT scores alone. Several longitudinal studies (Shea et al., 2001; Wai et al., 2009) that were tasked with identifying intellectually gifted youth have also found evidence that stronger spatial reasoning ability was associated with more success in STEM fields.

Despite its common usage and relevance to education, there is no consensus on the definition of spatial reasoning. In one of the first meta-analyses on the topic, Linn and Petersen (1985) suggested that spatial reasoning is a person's “skill in representing, transforming, generating and recalling symbolic non-linguistic information” (p. 1482). In a more recent meta-analysis of spatial reasoning training, Uttal et al. (2013) present a more precise definition of spatial reasoning, which is built on a theoretical classification system. In their definition, there are two dimensions of spatial reasoning tasks: (1) intrinsic vs. extrinsic and (2) static vs. dynamic (Figure 1). Within the first dimension, intrinsic spatial reasoning refers to a specific object and how its parts are arranged. For example, our ability to discriminate between a phone and a television would be considered intrinsic. The extrinsic component of spatial reasoning refers to the relationship between objects. For instance, comprehension of where buildings are in relation to each other in a city is a common task that is dependent on extrinsic spatial reasoning. The second dimension of spatial reasoning (static vs. dynamic) refers to movement and manipulation. Static tasks are those that involve analyzing or using static spatial information, such as reading a map or analyzing an object. Dynamic spatial reasoning specifies that we have an ability to comprehend the movement of objects or the manipulation of objects. For example, our ability to determine how an object's movement will change its relationship to the objects around it is considered to be a dynamic task.

FIGURE 1

Figure 1. Screenshots of an advanced puzzle in Optica with a rotation and one color.

This classification system is advantageous to previous definitions of spatial reasoning because it allows for research on the unique components of the construct that are theory-driven, instead psychometrics-focused like many other tests of intelligence. Previously identified sub-components of spatial reasoning can also be consolidated and mapped onto this 2 × 2 classification system along with their respective instruments that have been developed to assess proficiency in each of these areas. For example, Revised PSVT: R (Maeda et al., 2013) primarily focuses on assessing an individual's capacity to mentally rotate objects, which could be re-mapped onto a person's intrinsic and dynamic spatial reasoning ability.

There is also strong evidence that spatial reasoning skills are receptive to training according to a recent meta-analysis (Uttal et al., 2013), and several other studies (Keehner et al., 2006; Wright et al., 2008). Sorby (2009) has shown evidence that middle-school students who receive spatial reasoning skills training alongside a normal curriculum has a positive academic effect. Specifically, they found that middle-school women who took part in additional spatial skills training enrolled in more advanced math and science classes than their counterparts in a comparison group that did not receive spatial skills training. The skills gained during training also transfer to novel spatial tasks that were not directly trained. Despite a renewed interest in developing training for spatial reasoning abilities, there are relatively few training opportunities available in comparison to those available for mathematical, verbal, or written skills.

Tetris (1989) and Blockout (1991) have been used to train spatial reasoning skills in the past to varying levels of success (Okagaki and Frensch, 1994; De Lisi and Cammarano, 1996; De Lisi and Wolford, 2002; Sims and Mayer, 2002; Feng et al., 2007; Cherney, 2008). However, the studies that were more successful often trained their participants for longer periods of time. These studies also employed a high degree of similarity between the training environment and the assessment of spatial reasoning skills. These characteristics of successful training were also found to be beneficial in the extensive study of spatial reasoning training by Wright et al. (2008). Thus, instead of developing new formal training for spatial reasoning from scratch, a more efficient and effective way to help people develop these skills would be to adapt video games as training tools. This method would be advantageous as modern video games are readily accessible via mobile devices, and the medium is as effective at training spatial reasoning skills as formal courses or cognitive tasks (Uttal et al., 2013).

As the first step in determining whether a video game is a viable training environment, the current study will investigate whether performance on the Revised PSVT: R (Maeda et al., 2013) is related to performance while playing Optica, a video game where the goal of the player is to create a path connecting all visible tiles on a 3D object to solve a variety of puzzles (Optica, 2018). We believe that there will be a strong relationship between performance on the Revised PSVT: R and Optica because they both require players to utilize intrinsic and dynamic spatial reasoning abilities. In Optica, the player interacts with a single object and is not required to evaluate a relationship between objects, which suggests that they will need to utilize intrinsic spatial reasoning. To solve more advanced puzzles in the game, players must manipulate the perspective from which they are viewing the object to create an appropriate path. This type of manipulation of the 3D object in the game aligns with dynamic spatial reasoning because they are actively moving the object to see it from different perspectives. Based on the literature review, we have developed the following hypotheses. Null Hypothesis (H₀): There is either a lack of or a negative relationship between performance in Optica and the Revised PSVT: R. Alternate Hypothesis (H_a): There is a positive relationship between performance in Optica and the Revised PSVT: R.

Materials and Methods

This study employs a within-subjects design. Participants completed all research tasks during three science class meetings. Each class meeting lasted ~50 min. Activities were presented by the first author. On the first day of class, each student completed the Revised PSVT: R on a computer. During the next day of class, students completed the tutorial for Optica and worked on the first twenty levels of the game. On the third day, students were asked to work on levels 21 through 40. If the students completed all of the requested levels on either day of class they were given free time to use their computer.

Participants

One-hundred and sixty-eight students participated in this study who were enrolled in a seventh-grade science class at a racially and ethnically diverse middle-school in the midwestern region of the United States of America. A breakdown of their demographics can be found in Table 1. Data from 28 students was dropped from the analysis because they did not spend enough time (<5-min) working on the Revised PSVT: R. The decision to drop their data from our analysis was based on a previous study performed by Yoon and Mann (2017) that used the Revised PSVT: R to explore the spatial ability of undergraduate students. Thus, the final sample for this study was 140 students.

TABLE 1

Table 1. Description of student performance on the Revised PSVT: R and Optica.

The game was played as a part of students' normal science instruction, so all students played for the allotted time. At the school site, teachers are incorporating board games and digital games in their science instruction that support scientific practices and increase knowledge of scientific content. This game was played because it has the potential to increase students' spatial reasoning ability. For the study, parental consent was obtained by sending home information sheets about the study. Parents signed and returned the form if they did not want their students to participate in the study activities which were the pre-post assessments. This consent method was approved because no identifying information was collected from students beyond their student identification (ID) numbers which were never associated with their names or any personal information.

Purdue Spatial Visualization Test—Revised: Visualization of Rotations

The Revised PSVT: R is a spatial reasoning assessment focused on measuring a person's ability to mentally rotate objects (Maeda et al., 2013). In each of the thirty items, “the respondents' task is to mentally rotate an object in the same direction as indicated visually in the instructions, and then to select an answer from among five possible options” (Maeda et al., 2013, p. 766). The Revised PSVT: R was administered by converting the paper version of the assessment into an online form. Prior to working on the assessment, directions were read to the participants by the researcher and two examples were reviewed. This task was chosen for this study because it has been found to have strong reliability, validity, and regular use with middle school students (Sorby, 2009; Maeda and Yoon, 2013). Previous studies have found gender differences while using this assessment (Maeda and Yoon, 2013) and other spatial reasoning assessments (Ganley et al., 2014; Lauer et al., 2019). To determine whether there were systematic differences in spatial reasoning between genders on the Revised PSVT: R among our sample, students were asked to self-identify their gender on a demographic survey after completing the Revised PSVT: R.

Optica

A 3D puzzle game named Optica was used in this study. The player's goal is to draw a line that touches all of the tiles on a 3D-grid-object between two colored tiles. In more advanced levels, the player may need to draw lines between multiple colored tiles and/or rotate the object to complete the puzzle (Figure 1). Unity Analytics was used to gather information about player behavior in Optica (completion of levels, time to level completion, etc.).

Results

The demographics of participants are presented in Table 1. Descriptive statistics, including means and standard deviations (SD) on the Revised PSVT: R scores, completion time on the Revised PSVT: R, and number of levels completed in Optica are presented by participants' characteristics, including age, gender, race/ethnicity, and frequency of video game play. Table 2 describes the correlations between performance on the Revised PSVT: R, Optica, and age. Score on the Revised PSVT: R was found to be correlated with time to complete the assessment (r = 0.41, p < 0.001). In addition, the number of levels completed in Optica was significantly correlated with score on the Revised PSVT: R (r = 0.27, p < 0.01).

TABLE 2

Table 2. Pearson correlations between regression variables.

Given the positive correlations between performance in Optica and the Revised PSVT: R we analyzed the data further with a linear regression. Results of the linear regression indicated that there was a significant relationship between the number of levels completed in Optica and score on the Revised PSVT: R [F_{(9, 125)} = 2.28, p < 0.05, R² = 0.079]. The individual predictors were examined further and indicated that score on the Revised PSVT: R (t = 3.32, p < 0.01) and monthly video game play (t = −2.78, p < 0.01) was significant, but age, gender, and other reported video game play frequencies (daily, weekly, yearly, and never) were not significant predictors in the model. The positive relationship between the number of levels completed in Optica and score on the Revised PSVT: R can be seen in Figure 2. Additionally, the impact of video game play frequency can be viewed in Figure 3. Each of the individual plots in Figure 3 show the relationship between the number of levels completed and score on the Revised PSVT: R among students who reported playing video games at the same frequency (daily, weekly, monthly, yearly, and never).

FIGURE 2

Figure 2. Plot of the relationship between performance on Optica and the Revised PSVT:R.

FIGURE 3

Figure 3. Plot of the relationship between performance on Optica, Revised PSVT:R, and frequency of video game play.

Discussion

The null hypothesis has been rejected based on evidence of a positive relationship between performance on the Revised PSVT: R and Optica. Thus, according to Uttal's classification, playing Optica may require the player to use intrinsic and dynamic spatial reasoning ability to solve the puzzles in the game. Given the relationship between these abilities seen in our experiment, Optica may also be a viable environment for training intrinsic and dynamic spatial reasoning skills for use in other scenarios such as solving a Rubik's cube, playing a sport, or using a tool to repair a computer. However, experimental research will be necessary to determine if playing Optica has a direct impact on intrinsic and dynamic spatial reasoning skills. In addition, future experimental research will be necessary to rule out any confounding variables that were not measured in this study.

Future Work

Proper studies of the development of spatial skill need to exhibit five characteristics according to Wright et al. (2008). First, the component processes that are necessary to perform mental rotation (visual encoding, object rotation, object comparison, response; Cooper and Shepard, 1973) must be individually evaluated to determine which aspects of mental rotation are transferring to novel contexts. Second, test-retest effects must be avoided by using novel stimuli prior to and after training. Third, transfer to a separate spatial reasoning skill should be assessed. Fourth, skills in a non-spatial domain should be assessed as a control condition. The authors suggest a verbal assessment since these skills are thought to be distinct (Halpern, 1997). Fifth, training should be intensive, so that transfer effects can be measured. Intensive training seems to be around 400 min (6–7 h), because in their 2008 study, Wright and her colleagues instructed their participants to spend 15–20 min per day training their mental rotation skills for 21 consecutive days.

These recommendations will be used as the primary guide to plan out future research regarding Optica. In addition, the ability levels of students in other domains should be evaluated alongside spatial reasoning ability because it is likely that proficiency with other abilities are necessary to solve the puzzles in Optica such as aspects of problem solving and non-cognitive skills such as persistence. Shute et al. (2015) study suggested that performance in Portal 2, a complex puzzle game, is mediated by a person's spatial reasoning ability, problem solving skills, and persistence. Thus, if Optica were to be developed into a tool for training intrinsic and dynamic spatial reasoning ability, additional research would need to be performed to identify the impact of other cognitive and non-cognitive abilities on performance in the game.

Ultimately, this research will help the field by equipping it with a novel paradigm for evaluating intrinsic and dynamic spatial reasoning ability that is theoretically grounded. It will also provide the public with spatial reasoning skills training that is easily accessible, fun, and supported by methodical research.

Data Availability

The raw data supporting the conclusions of this manuscript will be made available by the authors, without undue reservation, to any qualified researcher. Requests to access the datasets should be directed to NV, dmFubTAwMzQmI3gwMDA0MDt1bW4uZWR1.

Ethics Statement

This work was conducted under approval from the University of Minnesota's Institutional Review Board (Study # 00003087).

Author Contributions

NV, KV, and EK contributed to the conception and design of the study. MG, NM, and FF assisted in data collection by designing the analytics events for the game. NV wrote the first draft of the manuscript and performed the statistical analyses. All authors contributed to manuscript revision, read, and approved the submitted version.

Conflict of Interest Statement

Several authors own the companies that are selling the game (GLITCH and Graveck), Optica, including NV, EK, MG, and NM.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This work could not have been completed without the support of the school, teachers, and students who participated in this research.

References

Blockout [Computer software]. (1991). Redwood City, CA: Electronic arts.

Google Scholar

Cherney, I. D. (2008). Mom, let me play more computer games: they improve my mental rotation skills. Sex Roles 59, 776–786. doi: 10.1007/s11199-008-9498-z

CrossRef Full Text | Google Scholar

Cooper, L. A., and Shepard, R. N. (1973). Chronometric studies of the rotation of mental images. Visual Inform. Process. 1973, 75–176. doi: 10.1016/B978-0-12-170150-5.50009-3

CrossRef Full Text | Google Scholar

De Lisi, R., and Cammarano, D. M. (1996). Computer experience and gender differences in undergraduate mental rotation performance. Comput. Hum. Behav. 12, 351–361. doi: 10.1016/0747-5632(96)00013-1

CrossRef Full Text | Google Scholar

De Lisi, R., and Wolford, J. L. (2002). Improving children's mental rotation accuracy with computer game playing. J. Genet. Psychol. 163, 272–282. doi: 10.1080/00221320209598683

PubMed Abstract | CrossRef Full Text | Google Scholar

Feng, J., Spence, I., and Pratt, J. (2007). Playing an action video game reduces gender differences in spatial cognition. Psychol. Sci. 18, 850–855. doi: 10.1111/j.1467-9280.2007.01990.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Ganley, C. M., Vasilyeva, M., and Dulaney, A. (2014). Spatial ability mediates the gender difference in middle school students' science performance. Child Dev. 85, 1419–1432. doi: 10.1111/cdev.12230

PubMed Abstract | CrossRef Full Text | Google Scholar

Halpern, D. F. (1997). Sex differences in intelligence: implications for education. Am. Psychol. 52:1091. doi: 10.1037//0003-066X.52.10.1091

PubMed Abstract | CrossRef Full Text | Google Scholar

Keehner, M., Lippa, Y., Montello, D. R., Tendick, F., and Hegarty, M. (2006). Learning a spatial skill for surgery: how the contributions of abilities change with practice. Appl. Cogn. Psychol. 20, 487–503. doi: 10.1002/acp.1198

CrossRef Full Text | Google Scholar

Lauer, J. E., Yhang, E., and Lourenco, S. F. (2019). The development of gender differences in spatial reasoning: a meta-analytic review. Psychol. Bull. 145, 537–565. doi: 10.1037/bul0000191

PubMed Abstract | CrossRef Full Text | Google Scholar

Linn, M. C., and Petersen, A. C. (1985). Emergence and characterization of sex differences in spatial ability: a meta-analysis. Child Dev. 1985, 1479–1498. doi: 10.2307/1130467

CrossRef Full Text | Google Scholar

Maeda, Y., and Yoon, S. Y. (2013). A meta-analysis on gender differences in mental rotation ability measured by the Purdue spatial visualization tests: Visualization of rotations (PSVT: R). Educ. Psychol. Rev. 25, 69–94. doi: 10.1007/s10648-012-9215-x

CrossRef Full Text | Google Scholar

Maeda, Y., Yoon, S. Y., Kim-Kang, G., and Imbrie, P. K. (2013). Psychometric properties of the revised PSVT: R for measuring first year engineering students' spatial ability. Int. J. Eng. Educ. 29, 763–776.

Google Scholar

Okagaki, L., and Frensch, P. A. (1994). Effects of video game playing on measures of spatial performance: gender effects in late adolescence. J. Appl. Dev. Psychol. 15, 33–58. doi: 10.1016/0193-3973(94)90005-1

CrossRef Full Text | Google Scholar

Optica [Computer Software]. (2018). Minneapolis, MN: Graveck. Available online at: https://opticagame.com/ (accessed March 26, 2019)

Google Scholar

Shea, D. L., Lubinski, D., and Benbow, C. P. (2001). Importance of assessing spatial ability in intellectually talented young adolescents: a 20-year longitudinal study. J. Educ. Psychol. 93:604. doi: 10.1037//0022-0663.93.3.604

CrossRef Full Text | Google Scholar

Shute, V. J., Ventura, M., and Ke, F. (2015). The power of play: the effects of Portal 2 and Lumosity on cognitive and noncognitive skills. Comput. Educ. 80, 58–67. doi: 10.1016/j.compedu.2014.08.013

CrossRef Full Text | Google Scholar

Sims, V. K., and Mayer, R. E. (2002). Domain specificity of spatial expertise: the case of video game players. Appl. Cogn. Psychol. 16, 97–115. doi: 10.1002/acp.759

CrossRef Full Text | Google Scholar

Sorby, S. A. (2009). Educational research in developing 3-D spatial skills for engineering students. Int. J. Sci. Educ. 31, 459–480. doi: 10.1080/09500690802595839

CrossRef Full Text | Google Scholar

Tetris [Computer software]. (1989). Redmond, WA: Nintendo of America.

Google Scholar

Uttal, D. H., Meadow, N. G., Tipton, E., Hand, L. L., Alden, A. R., Warren, C., and Newcombe, N. S. (2013). The malleability of spatial skills: a meta-analysis of training studies. Psychol. Bull. 139, 352–402. doi: 10.1037/a0028446

PubMed Abstract | CrossRef Full Text | Google Scholar

Wai, J., Lubinski, D., and Benbow, C. P. (2009). Spatial ability for STEM domains: aligning over 50 years of cumulative psychological knowledge solidifies its importance. J. Educ. Psychol. 101:817. doi: 10.1037/a0016127

CrossRef Full Text | Google Scholar

Wright, R., Thompson, W. L., Ganis, G., Newcombe, N. S., and Kosslyn, S. M. (2008). Training generalized spatial skills. Psychonom. Bull. Rev. 15, 763–771. doi: 10.3758/PBR.15.4.763

PubMed Abstract | CrossRef Full Text | Google Scholar

Yoon, S. Y., and Mann, E. L. (2017). Exploring the spatial ability of undergraduate students: association with gender, STEM majors, and gifted program membership. Gifted Child Quart. 61, 313–327. doi: 10.1177/0016986217722614

CrossRef Full Text | Google Scholar