Does self-paced learning in mobile flood protection unit construction in virtual reality have advantages over traditional measures?

Introduction: The study explores the application of virtual reality (VR) in university education, specifically within the context of civil engineering. It aims to investigate the potential of an immersive virtual lab employing self-paced learning for teaching complex tasks. The focus is on the construction of a Mobile Flood Protection Unit (MFPU), traditionally taught through written instructions or video tutorials.

Methods: An experiment was conducted involving 48 students who were divided into two groups. One group learned to build an MFPU using a VR tutorial, while the other group used a traditional instructional video. The effectiveness of these teaching tools was assessed based on factual and procedural knowledge transfer. Additionally, students' personal perceptions regarding the use of VR software were evaluated.

Results: The findings indicated a positive effect on factual knowledge transfer when using VR. Moreover, students expressed favorable perceptions towards utilizing VR as a learning tool.

Discussion: The study suggests that VR can enhance factual knowledge acquisition and is well-received by students in educational settings. However, it also highlights the need for further research to better understand its impact on procedural knowledge gain. Future studies could explore long-term effects and different applications within various fields of education.

1 Introduction

Practical exercises that incorporate authentic experiences and high levels of student engagement provide valuable additions to the curriculum in science, technology, engineering, and mathematics (STEM) fields (Gya and Bjune, 2021). However, these exercises can present significant challenges in higher education due to the limited availability of laboratory space for all students (dela Cruz and Mendoza, 2018). Additionally, there is often a lack of access to modern equipment and instruments required to conduct laboratory experiments (Kamińska et al., 2017). Exercises in real laboratories within STEM fields can require substantial resources (Achuthan and Murali, 2015). An example in the domain of civil engineering is the construction of a Mobile Flood Protection Unit (MFPU), which is the focus of this study. The assembly process takes students between 10 and 25 min to complete. Supervisors are also necessary to ensure student safety when handling heavy and cumbersome metal components. The construction of an MFPU is dependent on location, as it requires suitable bases in the floor. Furthermore, the components occupy significant space and involve material costs of approximately 3680€ for a wall section with realistic dimensions. However, these costs are not particularly critical in our case since the equipment is relatively robust and reusable. Regardless, despite these challenges, it is essential for aspiring civil engineers to acquire hands-on experience with flood protection measures in a realistic setting. One approach to mitigate or reduce costs of such an exercise is through the use of virtual laboratories (Achuthan and Murali, 2015). Virtual laboratories have been implemented across various STEM disciplines (Gya and Bjune, 2021; Sellberg et al., 2024), including chemistry, automotive training, and medicine (Ferrell et al., 2019; Xu and Wang, 2022; Mahling et al., 2023).

Virtual reality (VR) offers a cost-effective and particularly valuable implementation by providing immersive and adaptable virtual labs that can positively impact engineering education (Srinivasa et al., 2020; Alnagrat et al., 2022). Immersive virtual environments refer to virtual experiences perceived through natural sensorimotor contingencies (Slater and Sanchez-Vives, 2016), often described as the feeling of truly being present. In this context, immersive virtual labs are defined as virtual recreations of laboratories that share this quality.

This study evaluates the effectiveness of an immersive virtual lab in VR as a supplementary tool during a laboratory exercise in civil engineering. It compares the use of VR with an instructional video to determine which method is more effective in conveying factual and procedural knowledge about the laboratory experiment. In this work, VR refers to immersive virtual experiences which are experienced using head-mounted devices (HMDs). Those are typically paired with two handheld controlling devices as, for example, described by Steuer (1995). With this kind of VR technology, users wear a piece of hardware with two high resolution displays and integrated speakers on their head, allowing users to perceive the virtual world with stereoscopic vision and directional audio. The users movements are tracked at their head and both hands and are mapped to some kind of virtual representation of the users body, the avatar. Both, movement and rotation are tracked in three degrees of freedom, totaling in six degrees of freedom. This way, users can use their bodies and hands to physically engage with the virtual environment. This technology is characterized by a high degree of immersion and interaction which enables strong feelings of presence, i.e., the impression of really being in the virtual world and agency, the feeling of being in control of ones own actions (Slater and Sanchez-Vives, 2016; Synofzik et al., 2008; Makransky and Petersen, 2021). This illusion can cause users to interact just as natural with the virtual environment as they would with a physical counterpart in reality (Slater, 2009).

Advancements in the field of VR have significantly enhanced its accessibility and user-friendliness, fostering a growing interest in VR (Radianti et al., 2020; Lee, 2021). This surge in interest aligns with the United Nation’s Sustainable Development Goals by promoting innovative and accessible educational approaches. VR, in particular, holds potential for enriching teaching and learning experiences (Radianti et al., 2020; Hamilton et al., 2021), especially in environments that allow users to engage in meaningful interactions with the virtual environment (Southgate, 2020). Studies have shown positive student preferences towards VR-based learning (Maiero et al., 2023). Further research has indicated beneficial effects of VR on student motivation (Krokos et al., 2019; Mouatt et al., 2020; Jiang and Fryer, 2024) and additional evidence is suggesting improved learning outcomes and an overall learning experience (Radianti et al., 2020; Bermejo et al., 2023).

This study aims to evaluate the benefits of using a VR based self-paced learning tutorial in the context of a laboratory exercise in civil engineering. It was conducted with students of the RWTH Aachen University in Germany. During the experiment, students learned how to construct an MFPU. MFPUs are used to prevent humanitarian and financial damages caused by floodings induced by water bodies (Fekete, 2020; Subramanian et al., 2022). Techniques for mitigating environmental damages are an integral part of the civil engineering studies. As such, constructing an MFPU is taught at the RWTH Aachen University in the course of the study module Flood Risk Management.

The effects of using VR in the preparation for the construction of an MFPU in comparison to traditional teaching media such as videos were examined in this work. The students were randomly assigned to two experimental conditions. The first group engaged in VR training for the construction of the MFPU, while the second group utilized an instructional video for their preparation. Following these preparatory activities, both groups completed an online assessment to evaluate their factual knowledge prior to the hands-on task. Subsequently, all participants were required to physically construct the MFPU on-site. Performance metrics used for assessing procedural knowledge included the time taken to complete the construction and the number and type of errors made during the process.

Based on this background, the following two research hypotheses were formulated and examined.

H1 Students trained with VR achieve better quiz scores about building the MFPU than those who received video training.

H2 Students trained with VR perform better in building the MFPU compared to students trained with the teaching video.

In Section 2, the theoretical context for this work will be outlined. Different studies examining the effects of immersive learning effects on procedural tasks like laboratory exercises are presented. It is also argued why VR is a suitable method for conveying the knowledge needed to construct the MFPU. The methodology of this study is illustrated in Section 3. It defines learning objectives and describes the teaching media that was used for the experiment. In Section 4 the results of the experiment are reported. Section 5 contains a discussion of these results, summarizes lessons that were learned during this study and informs future research on immersive virtual laboratories in VR.

2 Related work

2.1 Background

The Constructive Alignment model, as proposed by Biggs and Tang (2011), is a widely utilized framework for course design. This model emphasizes the alignment of learning objectives, assessment methods, and instructional activities to enhance educational effectiveness. According to this approach, assessments are selected based on the predefined learning objectives, and subsequently, teaching activities are designed to optimally prepare students for these assessments. The revised taxonomy of Bloom (Anderson and Krathwohl, 2001) is commonly used for categorizing learning outcomes and levels of understanding. The learning objectives addressed in this study encompass both factual and procedural knowledge according to this taxonomy (Anderson and Krathwohl, 2001). Factual knowledge is commonly assessed through quizzes (Yang et al., 2021), while procedural knowledge is typically evaluated based on students’ performance in executing the relevant tasks (Jongbloed et al., 2024).

To achieve the learning objectives of this intervention, it is essential to identify which teaching activities most effectively support the acquisition of factual and procedural knowledge. Markant et al. (2016) explored the advantages of active learning over traditional passive approaches in general educational contexts. The authors argue that the freedom to choose information and control the pace of information intake can enhance knowledge retention compared to passive methods that lack such autonomy (Markant et al., 2016). However, Markant et al. (2016) also note that a complete absence of control elements can lead to diminished knowledge acquisition. Learning environments where the pace is tailored to the learners rather than the instructors are referred to as self-paced (Kumar et al., 2010). Gya and Bjune (2021) emphasize the positive impact of practical exercises on student learning and motivation. They attribute this effect to increased sensory and cognitive engagement during these activities and highlight the multimedia effect, which suggests that utilizing various brain regions enhances knowledge retention more effectively than comparable passive methods (Gya and Bjune, 2021).

2.2 Laboratories and VR

One medium that has recently gained prominence for active learning approaches is VR (Srinivasa et al., 2020; Gloy et al., 2021; Makransky et al., 2021; Kuhne et al., 2023). The comparative advantages of VR over non-immersive methods have been well-documented in recent years. For instance, Srinivasa et al. (2020) investigated the effectiveness of VR-supported preparatory measures for work in real mechanical engineering laboratories. The study compared VR with conventional approaches such as reading laboratory notes or watching instructional videos and found that the VR group demonstrated improved knowledge retention, self-efficacy, and engagement. Additionally, they noted the high cost-effectiveness of VR training compared to physical equipment (Srinivasa et al., 2020).

In another study, Kuhne et al. (2023) examined the effects of content intake delivered via VR and non-immersive screens on both physiological responses and self-reports across different subjects. They reported that calming and confrontational content had stronger arousing and calming effects, respectively, when experienced through VR compared to 2D content (Kuhne et al., 2023). This was attributed to the higher level of immersion provided by VR, which was also associated with a heightened sense of presence. Furthermore, participants indicated greater enjoyment, engagement, and a sense of privacy in the VR condition (Kuhne et al., 2023).

Gloy et al. (2021) conducted a study comparing VR tutorials with traditional textbooks in terms of factual knowledge retention for anatomy studies. Their experiment revealed that students who learned anatomy using VR had higher memory retention than those using textbooks. They suggested that the greater success of VR could be due to higher degrees of presence and agency experienced by learners (Gloy et al., 2021).

2.3 Immersive learning

Presence and agency are integral components of the Cognitive Affective Model of Immersive Learning (CAMIL), as introduced by Makransky and Petersen (2021). This framework provides guidance for the pedagogical design of VR teaching scenarios, emphasizing methods that enhance presence or agency within virtual environments (Makransky and Petersen, 2021). Similarly, Johnson-Glenberg (2018) presents design principles for successful embodied learning experiences, which also focus on increasing presence and agency. According to Makransky and Petersen (2021), various factors regulated by presence and agency–such as interest, motivation, self-efficacy, embodiment, cognitive load, and self-regulation–contribute to different learning outcomes in line with Anderson and Krathwohl (2001). Increased situational interest, intrinsic motivation, self-efficacy, embodiment, and self-regulation, along with reduced cognitive load, can positively impact learning outcomes as predicted by CAMIL (Makransky and Petersen, 2021). Additionally, several external factors influence the model, including usability, social influences, and individual differences such as age, susceptibility to cyber sickness, working memory capacity, personality traits, predisposition towards absorption, and spatial ability (Makransky and Petersen, 2021).

Makransky et al. (2021) caution that merely using VR as a medium does not necessarily lead to greater gains in declarative or procedural knowledge compared to less immersive media. Their experiments compared video and VR interventions and found that generative teaching methods provided particular benefits for VR training, such as enacting the subject with real props after the intervention (Makransky et al., 2021). While their study did not show a higher knowledge gain for VR interventions compared to video, they highlighted the significant positive motivational effects of VR (Makransky et al., 2021).

According to Hamilton et al. (2021), most procedural tasks seem to be benefiting from using VR. However, their literature review revealed that the application of acquired knowledge to real-world scenarios was underrepresented in the studies they analyzed (Hamilton et al., 2021).

This provides us with a good opportunity for research and motivates our study, as our experiment focuses on transferring the knowledge gained through VR to the real world. In this way, the knowledge gained is supplemented by concrete enactment in practice following the VR intervention.

3 Methodology

3.1 Learning objectives and assessment methods

The proposed VR tutorial was implemented to supplement the lecture Flood Risk Management at our university. The lecture raises awareness for flood risks and informs students about historical flood events, hazard and risk maps, hydraulic models among other topics including flood protection. As such, the construction of an MFPU is taught and the understanding of it is viewed as basic requirement to pass the respective exam. The following learning objectives for this topic were defined prior to our experiment.

LO1 Students understand how the MFPU works, they can name the parts and explain their function as part of the whole.

LO2 Students are able to build the MFPU in a team. They know the correct order of execution of individual steps and avoid mistakes.

LO1 and LO2 entail factual and procedural knowledge across the lowest three levels of understanding in accordance with the revised taxonomy of Bloom (Anderson and Krathwohl, 2001).

Based on this, it was chosen to use an online quiz to assess the success in reaching LO1. The universities learning management system Moodle offers to include quizzes in the online course material seamlessly and was therefore used in this study. A questionnaire with 10 items was designed, focusing on different aspects of the MFPU construction on various levels of Bloom’s taxonomy. It includes different questions (Multiple Choice and Ordering Elements) about the sequence of work steps, purpose, errors or good practises in the execution of individual steps or the naming of certain components. A full list of questions can be found in the Supplementary Material.

For the assessment of LO2, it was chosen to measure the performance of the students while constructing the MFPU in the real world. In order to achieve a measurable assessment, the time taken to construct the MFPU was recorded while also counting various types of errors that were made during the construction. This resulted in a two-dimensional evaluation of the student’s performance. To construct the MPFU, the students visited the university and built the MFPU under supervision. The components are heavy and unwieldy, so that construction cannot be carried out safely on an individual basis. Hence it was necessary to request the students to form groups of three in order to construct the MFPU. The groups were formed freely. The error types were defined prior to the experiment and considered several aspects of constructing the MPFU like procedural mistakes such as wrong order of work steps or improper execution, carelessness and/or dangerous behaviour. The list of error types is contained in Table 2.

3.2 The teaching media

Before the VR tutorial, an instructional video was used to convey basic knowledge about the construction of an MFPU. The video is available for unlimited access in Moodle and Youtube and briefly depicts all the necessary working steps in the correct order. For further context, textual explanations for the individual steps are shown as the construction progresses. (Faculty of Civil Engineering RWTH Aachen University, 2019).

The virtual environment replicates the Mobile Flood Protection Unit type HW-B100K by IBS Technik GmbH, which is also available in the real world laboratory of our university. The MFPU is composed of aluminum dam beams or logs, heavyweight central support beams, so called posts or columns, a pressing tool, anchor plates and rubber seals (IBS Technics GmbH, 2015). The MFPU design schematic is depicted in Figure 1, for a more detailed description see Table 1. Figure 2 illustrates some exemplary worksteps during the construction of the MFPU, with their virtual equivalents depicted in Figure 3.

Figure 1

Figure 1. Design schematics for the MFPU.

Table 1

Table 1. Construction Components of the MFPU type HW-B100K (IBS Technics GmbH, 2015).

Figure 2

Figure 2. Example work steps during the construction of the MFPU: (A) Assembly of the column on the anchor plate, (B) Lubrication of the column to allow a smooth insertion of dam beams, (C) Insertion of a dam beam.

Figure 3

Figure 3. Screenshots of the MFPU assembly in VR.

The VR environment was implemented in the avatar-based immersive VR-platform MyScore (Berkaoui et al., 2022; Chandra et al., 2023) and is inspired by urban river areas like the city of Cologne, Germany, featuring generic urban characteristics as well as a larger river flowing next to the construction site. Figure 4 illustrates the appearance of the environment. The design is simple to avoid distractions.

Figure 4

Figure 4. An overview of the VR training environment.

The VR scenario involves several features for an enhanced user experience and promoting effective training. Users embody a virtual avatar that can be visually adjusted to match their own appearance. This aims to increase the sense of embodiment and presence within the virtual environment while also maintaining a feeling of realism. It is also supposed to ease the engagement with others fostering collaborative use-cases, as researched by Chandra et al. (2024). Further emphasizing this, spatial audio is utilized as means for communication which creates a three-dimensional audio experience enabling users to discern the location and direction of sounds made by other speaking users.

To assist in the training process, several multimedia instructions are integrated into the scenario. Users can access a written user manual, tooltips at the respective components that indicate the next necessary step, and a video tutorial that demonstrates the correct steps and procedures required for assembling the MFPU. Once a work step is completed correctly, the tooltips for that step disappear and reveal tips for the next step. If desired, teacher-based instructions can be provided by joining with another avatar to support the students.

When entering the scenario for the first time, all virtual components of the MFPU–such as the automatic drill, bolts, posts, dam beams, etc., –are placed next to the construction site, ready to be used. While it is crucial for students to learn the correct sequential order of assembling the MFPU, the training scenario allows them flexibility to explore alternative approaches. This dynamic method enables learning from mistakes with the goal of providing valuable experience. For instance, inserting a dam beam without first placing a rubber ground seal is highly undesirable because water would continue to leak through the bottom gap. However, this mistake could occur in real-life situations if students are not properly informed. When noticing a severe mistake during the construction, users have the opportunity to reset the tutorial and start over again.

The MFPU VR training scenario has been designed such that assembly follows a non-deterministic process, allowing students to plan and execute working steps in their own order. This also provides a higher degree of agency, which is expected to have a positive effect on learning outcomes. Some errors will result in immediate implicit feedback (e.g., improper lubrication of a post will make seamless insertion of dam beams impossible), while other mistakes show their consequences once construction is finished–mirroring how errors in construction would manifest in reality.

To test the correctness of their work at the end of the tutorial, the users can trigger a simulated flood, which will result in a flood wave approaching the MFPU. If the MFPU has been constructed adequately, it will remain stable once the flood wave hits, while falling apart if certain or too many mistakes were made. One example for this would be to place the posts and dam beams in the right positions while not tightening the fastening screws at the posts. The process can be started over to allow students to improve on their mistakes, if necessary.

3.3 Participants

99 civil engineering students were enrolled in the Flood Risk Management elective lecture at RWTH Aachen University, Germany. They were randomly divided into two groups of approximately equal size (VR: 49 students, Video: 50 students). Participation in our study was stipulated as an admission requirement for the final examination in the module handbook. When choosing the module, students therefore committed to participating in the study, provided they wanted to take the exam and were not disqualified for valid reasons (such as severe visual impairment or similar). Of these students, 48 (VR: 24 students, video: 24 students) ultimately completed our study, taking the online quiz and constructing the MFPU on site. Of these, 22 were male and 26 female. The age of the students was not recorded.

3.4 Experimental procedure

The experiment consisted of three stages: a training stage, a quiz stage, and a construction stage. During the first stage, the training stage, participants prepared for assembling the MFPU using our learning media. One group, referred to as the video group, was provided with a training video detailed in Section 3.2. This video was produced prior to the experiment and made available on Moodle and YouTube for unlimited access. The second group, known as the VR group, learned how to build the MFPU in a VR training environment. To do so, they visited the university and were temporarily handed out a VR headset. A supervisor was assigned to ensure that the student was able to use the VR headset properly and to make sure that they felt well during the trial. To simulate the same kind of unlimited learning situation, we aimed to minimize restrictions on the use of the VR tutorial. However, due to practical considerations, we imposed a 45-min time limit for the VR trial (including preparation time), which was exceeded by only one student. During this experiment phase, students in the VR group used a Meta Quest 3 VR headset and touch controllers to interact with the virtual environment. Some example worksteps of constructing the virtual MFPU can be seen in Figure 3.

To gather additional information about external factors influencing the efficacy of the VR training scenario, a brief survey was conducted with students in the VR group. The survey assessed how students felt during their time in virtual reality and their opinions on the application’s performance. Additionally, students were asked to provide a self-estimation of their confidence in constructing an MFPU after the VR training. This survey was administered to all participants who learned in the VR scenario immediately after using it. Since this stage of the experiment still included students who dropped out before the quiz stage, the number of participants was initially 33, which then reduced to 24. This usability questionnaire was conducted anonymously to encourage honest responses.

In the second stage of the experiment, both groups of students took an online quiz about how to construct the MFPU. Each question was weighted equally, with one point per question, allowing students to score a maximum of 10 points. To investigate H1, the scores of both groups were compared by analyzing and comparing the descriptive statistics of each group’s distribution. An independent samples $t$-test was conducted to determine the statistical significance of the results. Additionally, we examined the individual questions and compared the scores achieved by each group, as well as their standard deviations. We were only interested in one experimental condition, which was the teaching method used. Hence, descriptive statistics where chosen to evaluate the results because it was sufficient for our particular focus and allows a comprehensible interpretation of our data.

In the final stage of the experiment, students were required to form teams of three. Students were allowed to form their own teams, but those who were unable to do so were assigned teams by the administration as optimally as possible. Each team was composed exclusively of students from either the video group or the VR group. The teams then came to the university and built the MFPU under supervision, without any additional help or information other than necessary safety instructions. The time taken by each team to build the MFPU was measured, and the different error types, as referenced in Table 2, were counted. To compare both groups and answer H2, descriptive statistics of the construction times for each group were analyzed for similar reasons as above. Additionally, we calculated the average number of errors per team and error type along with their respective standard deviations.

Table 2

Table 2. Error types assessed during the construction of the MFPU.

4 Results

The following section presents the results of the experiment and evaluates whether hypotheses H1 and H2 are supported. To analyze H1, we collected individual scores from students in both the video and VR group. Descriptive statistics and an independent samples $t$-test were utilized to assess the overall performance of the students and determine the statistical significance of the findings. A similar methodology was applied to test H2, focusing on construction time and error counts. We measured the average time each team took to build the MFPU, along with the number and types of errors made during construction. Additionally, we calculated and assessed the average error count per team as well as the standard deviation for each error type. No severe issues were reported during the experimental stages.

4.1 Factual knowledge

In the online quiz, both groups of students had to answer 10 questions about constructing the MFPU, each question weighted equally with one point for a correct answer. The test results were evaluated for $n=48$ students, equally distributed between the two conditional groups. The results indicate that students in the VR group generally achieved higher scores than those who used the learning video. A detailed comparison is presented in Figure 5.

Figure 5

Figure 5. Comparison of quiz results of the VR group $(n_{VR}=24,{\bar{x}}_{VR}=8.76,S{D}_{VR}=1.85,M_{VR}=10)$ and the video group $(n_{\mathit{V\; ideo}}=23,{\bar{x}}_{\mathit{V\; ideo}}=7.68,S{D}_{\mathit{V\; ideo}}=1.97,M_{\mathit{V\; ideo}}=8)$. One outlier (score = 0.2) was removed from the video group.

An independent samples $t$-test was conducted to determine the likelihood that the differences between the results of both groups were due to chance. The resulting $t$-statistic $t(46)=2.22$ and $p$-value $p=.03$ indicated a statistically significant difference between the results of the two groups.

Further analysis of the scores per question revealed that the VR group performed at least as well as the video group on every question, and outperformed the video group on most questions (all except Q3 and Q4, where the performance was equal). These detailed results are illustrated in Figure 6.

Figure 6

Figure 6. Comparison of the average scores per question and group.

As shown in Figure 6, the standard deviation in student scores per question was lower for the VR group in every question except Q3 and Q4, where they were equal. Overall, the standard deviation in the VR group was lower.

4.2 Procedural knowledge

To evaluate H2, we observed the time required by students from both groups to construct the MFPU according to the specifications. The students were organized into teams of three, which significantly reduced the sample size. A total of $n=17$ teams were recorded for the experiment. The detailed results are presented in Figure 7.

Figure 7

Figure 7. Comparison of real world construction time per team in the VR group $(n_{VR}=9,{\bar{x}}_{VR}=1057.44,S{D}_{VR}=273.67,M_{VR}=950,Mi{n}_{VR}=735,Ma{x}_{VR}=1580)$ and video group $(n_{\mathit{V\; ideo}}=8,{\bar{x}}_{\mathit{V\; ideo}}=1101.75,S{D}_{\mathit{V\; ideo}}=171.64,M_{\mathit{V\; ideo}}=1145,Mi{n}_{\mathit{V\; ideo}}=870,Ma{x}_{\mathit{V\; ideo}}=1270)$.

The measurements indicate that the mean construction time in the VR group was slightly lower by a margin of 4% relative to the video group. The difference in medians is even larger, at 17%, indicating an increase in construction speed by roughly one-sixth for the faster half of the VR teams compared to the faster half of the video teams. Additionally, a lower standard deviation was observed. However, these findings are not statistically significant, as indicated by an independent samples $t$-test with $t(15)=-0.39$ and $p=.70$. An evaluation of the errors made during construction is depicted in Figure 8.

Figure 8

Figure 8. Comparison of the average number of errors per type, team and group. (A) Average number of errors (mean) and (B) its standard deviation.

While the VR group tends to have a lower error count per type, except for E8, neither group consistently outperforms the other in terms of the total number of errors made per type. The same observation applies to the standard deviation, as shown in Figure 8. The average number of mistakes (median) per team was two in the VR group and 2.5 in the video group. This concludes the findings for the procedural assessments.

In the following, the results of the survey conducted with the VR group during the training stage of the experiment are presented. This questionnaire was completed by $n=34$ students. Over 84% of the participants reported being first-time users of VR. The students were asked to answer the following questions by assigning rates of agreement between 0% and 100%.

Q1 How intuitive was the handling?

Q2 How clear (easy) to use was the application?

Q3 How close to reality was the application?

Q4 How well did the application perform?

Q5 Could you now build a MFPU in the real world?

The results of the survey are depicted in Figure 9.

Figure 9

Figure 9. The survey results of the VR usability in building the MFPU for the VR group by VR experience.

In addition, qualitative feedback was collected. Participants were asked to report both positive and negative aspects of the VR intervention. Common themes in the positive feedback included high engagement and enjoyment, intuitive use, and a high perceived degree of realism. One student mentioned that the training was more realistic than they had expected. Another noted that the training could be conducted from anywhere without the need for physical equipment. Additionally, a participant remarked that the VR experience was unique and would help them retain knowledge better than reading or watching a video. Some students viewed the training as a good introduction to the topic.

On the negative side, a recurring piece of feedback was that some students experienced cybersickness during the trial. Others reported unrealistic behavior of certain components and bugs, such as screws that could not be tightened properly. Further participants mentioned control issues, such as having to approach components too closely to interact with them or generally dissatisfying or imprecise controls. Another student reported connection issues.

The students were also asked to provide general suggestions on how the VR tutorial could be improved. Several ideas for the didactical design were proposed. One suggestion was to allow more preparatory time for beginners. Another student wished for the ability to reset individual work steps if mistakes are noticed, rather than resetting the entire tutorial. The participants also made design suggestions. For instance, it was recommended to create a more detailed and realistic flood simulation. Other quality of life improvements were suggested, such as adding extra screws in case some are lost, allowing users to carry more than two screws at once, and enhancing the visual design for a more sophisticated appearance.

5 Discussion

5.1 Data interpretation

In this section, we assess the research hypotheses postulated in Section 1 using the results presented in Section 4. The online quiz results indicate that VR positively impacts factual knowledge gain about construction processes, as demonstrated in this study. The VR group not only performed better overall (Figure 5), but also consistently outperformed the video group (Figure 6). Therefore, our data supports H1. Additionally, the lower standard deviation in scores (Figure 6) suggests more consistent learning success within the VR group. Thus, the tutorial appears to be an effective tool for conveying factual knowledge and achieving LO1, as defined in Section 3.

Aligned with the literature reviewed in Section 2, one contributing factor could be the high motivation and interest reported by students after using VR and the variety of instructional methods employed, supporting Gya and Bjune (2021). Furthermore, students controlled their tutorial pace and cognitive load during the experiment by using VR independently, which corresponds with Markant et al. (2016). Students managed both the pace and order of assembling MFPU parts while receiving immediate feedback on correct execution, enhancing their self-efficacy. These findings align with the CAMIL model (Makransky and Petersen, 2021), explaining why the VR group achieved better quiz results than the video group. Another positive factor could be that constructing MFPU requires spatial understanding–a field where VR shows promise according to Hamilton et al. (2021).

Regarding H2 and procedural knowledge gain, our data is less conclusive. The differences from the third experimental stage are statistically insignificant. While indicating better outcomes for the VR group–such as shorter construction times (Figure 7) and generally fewer errors (Figure 8)–it remains unclear if these improvements stem from teaching methods or other factors like physical strength or team coordination unrelated to teaching methods due to a small sample size. Hence, we cannot confirm nor refute H2 based on this study’s data. This limitation affects generalizability and indicates further studies with larger participant groups are needed for more reliable results. Both teaching methods were comparably effective for achieving LO2 based on error counts. All teams successfully constructed MFPU onsite after preparation with an average error count of two to three per team being minor for a first time construction in the real world. While our data was inconclusive, other scientists reported positive effects of VR in similar use cases under certain conditions (Srinivasa et al., 2020). Srinivasa et al. (2020) used VR to prepare students for a real world exercise and reported higher learning gains for students who used VR first to prepare for the real exercise compared to those who started in the real world right away. Makransky et al. (2021) found that VR benefits particularly from generative learning activities like real world enactment being included in the process, as was done in this experiment.

Figure 9 presents the quantitative feedback provided by VR participants immediately following the virtual training session. The results indicate a generally positive reception, consistent with findings from other research on educational VR content (Radianti et al., 2020). Participants gave high ratings for ease of use, tutorial intuitiveness, perceived realism, and performance, suggesting overall satisfaction and high engagement with the VR experience. This coincides with findings of other researchers such as Srinivasa et al. (2020) or Makransky et al. (2021). However, when asked to assess their ability to construct an MFPU in the real world post-training (Q5), the responses were more varied, with median values below 50% for both experienced and first-time users. This suggests that certain key aspects of MFPU setup may not have been adequately covered in the VR tutorial. More in-depth questioning in future studies could provide further insights into this specific result. Additionally, the results indicate lower usability ratings for first-time VR users compared to experienced users, which has to be expected.

5.2 Limitations and lessons learned

The following methodological issues and limitations are acknowledged. The participant selection exhibited biases, as the study exclusively involved engineering students who are likely to have a high openness for technology. While this study primarily evaluates VR-based lab training in the STEM field, it is important to note this bias. Additionally, only students who completed the entire study were included in the evaluation, leading to survivorship bias and loss of data from participants who engaged with VR but did not continue. Self-selection bias was also present due to students forming their own groups in the construction stage; random group assignments would be beneficial for future studies.

Further limitations arise with data acquisition. The usability questionnaire used was neither validated nor pre-tested and relied on multiple-choice questions and simple tasks, which may restrict generalizability and depth of data. A stronger focus on flow, presence, and other established VR metrics would enhance comparability with other research.

The subject–setting up an MFPU–presents challenges such as heavy components requiring group work. These conditions, although aggravating, are necessary for a realistic experience. This complicates the measurement of individual performance and reduces the sample size, which should be recognized as a limitation. To improve future research, an increase in sample size should be sought to enhance data quality.

Student feedback also suggests that bugs and control issues were inhibiting the tutorial’s success for some individuals. These issues should be addressed to maximize the utility of the presented VR tutorial by minimizing learner irritation. Furthermore, the impact of cybersickness on our results has gone unnoticed. Future studies should aim to evaluate this effect, and improvements should be made to counteract cybersickness during training. Accessibility modes limiting the number of “moving pixels” on displays, such as using rest frames, could be useful in this regard.

5.3 Conclusion and outlook

This study presents an immersive virtual tutorial for constructing the MFPU, which can aid in preparing students for its real-world construction. The tutorial utilizes an embodied training experience and was coupled with a real-world enactment phase in our experiment, aligning with research by Makransky et al. (2021). By reducing the amount of resources and time needed for training compared to a real-world experiment, it addresses the issues described by dela Cruz and Mendoza (2018) and Kamińska et al. (2017). The study successfully addressed research hypothesis H1 but failed to confirm H2.

The presented tutorial aims to make practical experiences in engineering studies more accessible, even under time and resource constraints. As noted by Gya and Bjune (2021), lab experiences are an integral part of STEM studies. The knowledge conveyed in this study was factual and procedural. Several dimensions of presence and agency were considered in the design process of this tutorial to positively influence factual and procedural knowledge gain (Makransky and Petersen, 2021). The VR scenario also implemented multimedia instructions for improved knowledge retention and allows users a self-paced approach, guiding but not predetermining the process of construction. The results presented in Section 4 provide evidence that the VR tutorial proves to be advantageous over the traditional teaching video, at least for the factual knowledge assessed during the quiz stage of this experiment. Factual knowledge can contribute significantly to preparing for real lab work. It has been shown that multimedia approaches, such as training with safety instruction videos, can be beneficial in laboratory scenarios (Pekdağ, 2020). Based on this, our results suggest that VR offers a promising perspective for further improvement in this area. Proper safety instructions require time, often limited in laboratory settings (Achuthan and Murali, 2015). Making them available outside the lab using VR technology could help save time for conducting actual experiments.

However, the widespread availability of VR as a technology presents challenges, particularly in emerging and developing countries. It is essential to consider how VR training can be made accessible in these regions. The tutorial provided is compatible with smartphones and PCs, and additional measures like VR cardboards can facilitate low-cost access to VR. Still, it is important to note that these alternatives may not fully replicate the immersive experience provided by a high-quality Head-Mounted Display with controllers. This limitation should be carefully considered. It also has to be noted that an increased usage of VR can lead to a dependency on this technology. As right now there is a very limited amount of players in the industry of VR hardware, this should not go unnoticed. Concerns about data privacy and security are of importance.

While implementing VR in higher education may incur initial costs due to necessary investments, it is important to recognize that the hardware is highly reusable. Additionally, the increased student interest and engagement that practical activities and VR can generate should be considered when evaluating these expenses, especially in comparison to more passive traditional learning media such as videos.

Further improvements to the presented VR tutorial have been motivated by this study. Although few technical difficulties were experienced during our study, this could be due to the study population. VR as a medium still faces certain weaknesses like accessibility, cybersickness, novelty effect, general technical issues, availability, and content scarcity. While tutorials like ours can contribute educational content in VR, issues like cybersickness and general accessibility remain unaddressed. To maximize the educational benefits of VR training, implementing strategies to mitigate cybersickness and address the initial novelty effect of using VR technology could further improve the learning experience (Garrido et al., 2022; Miguel-Alonso et al., 2024).

Learning Experience Design (LXD) offers a promising data-driven methodology for improving digital educational methods while including learners in the process (Schmidt and Huang, 2021). Our study provides valuable student data that can be used to further improve the tutorial. Based on common error types observed during the construction phase, future scenario improvements could be tailored specifically to teach avoidance of these errors. The design suggestions made by the students are additional and valuable feedback that should be considered in future developments. However, it is also crucial to directly include students in the design process to develop a tutorial that truly understands its learners (Schmidt and Huang 2021). According to Schmidt and Huang (2021), LXD emphasizes user experience from the learner’s perspective by incorporating theoretical elements from learning design and technology, human-computer interaction, and user experience theories. LXD adapts methods from user experience design better suited for educational contexts aiming to foster empathetic understanding of learners’ socio-cultural backgrounds and environments where they engage socially-mediated meaning-making processes (Schmidt and Huang, 2021).

Another aspect of improving the VR tutorial involves incorporating intelligent tutoring systems (Guo et al., 2021), which can cater to the individual needs of different learners. Adapting the virtual environment to users could facilitate self-paced learning experiences that address individual strengths and weaknesses, thereby offering potential benefits for learners (Wang et al., 2024). Additionally, adaptive assistant systems could be utilized to help with control issues, thereby increasing the general accessibility of VR technology.

Data availability statement

The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found below: https://cloud.lfi.rwth-aachen.de/d/15b8e0adc7674c2eb68f/. The authors will also provide the data upon request.

Ethics statement

Ethical approval was not required for the studies involving humans because because the subjects doesn’t need any ethical concern in the VR approach. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author contributions

PQ: Conceptualization, Data curation, Formal Analysis, Investigation, Resources, Software, Visualization, Writing–original draft, Writing–review and editing. RC: Conceptualization, Investigation, Methodology, Resources, Software, Visualization, Writing–original draft, Writing–review and editing. DB: Writing–review and editing. KC: Software, Writing–review and editing. HN: Funding acquisition, Methodology, Project administration, Resources, Supervision, Writing–review and editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This research was funded by the German Ministry of Education and Research and the German Academic Exchange Service, grant number 57516716. The APC was funded by the RWTH Aachen University.

Acknowledgments

The MyScore project is acknowledged to be made possible through the funding support of the German Federal Ministry of Education and Research and the German Academic Exchange Service. The authors used GPT-4o for spell and grammar checking.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frvir.2024.1447288/full#supplementary-material

Abbreviations

STEM, Science, Technology, Engineering and Mathematics; VR, Virtual Reality; HMD, Head-Mounted Device; MFPU, Mobile Flood Protection Unit; CAMIL, Cognitive Affective Model of Immersive Learning; LXD, Learning Experience Design.

References

Achuthan, K., and Murali, S. S. (2015). A comparative study of educational laboratories from cost and learning effectiveness perspective. Springer International Publishing, 143–153. doi:10.1007/978-3-319-18473-9_15

CrossRef Full Text | Google Scholar

Alnagrat, A. J. A., Ismail, R. C., and Idrus, S. Z. S. (2022). The opportunities and challenges in virtual reality for virtual laboratories. Innovative Teach. Learn. J. 6, 73–89. doi:10.11113/itlj.v6.91

CrossRef Full Text | Google Scholar

Anderson, L. W., and Krathwohl, D. R. (2001). A taxonomy for learning, teaching and assessing: a revision of Bloom’s taxonomy of educational objectives: complete edition. Boston, MA: Addison Wesley Longman, Inc.

Google Scholar

Berkaoui, D., Chandra, R., and Castermans, K. (2022). Myscore–avatar-based teaching and learning

Google Scholar

Bermejo, B., Juiz, C., Cortes, D., Oskam, J., Moilanen, T., Loijas, J., et al. (2023). Ar/vr teaching-learning experiences in higher education institutions (hei): a systematic literature review. Inf. (MDPI) 10, 45. doi:10.3390/informatics10020045

CrossRef Full Text | Google Scholar

Biggs, J., and Tang, C. (2011). Teaching for quality learning at university: what the student does. 4 edn. Maidenhead: Open University Press.

Google Scholar

Chandra, R., Castermans, K., Berkaoui, D., Querl, P., and Heribert, N. (2024). Creating realistic human avatars for social virtual environments using photographic inputs. J. Inf. Syst. Inf. 6, 2110–2129. doi:10.51519/journalisi.v6i3.842

CrossRef Full Text | Google Scholar

Chandra, R. L., Berkaoui, D., Castermans, K., and Nacken, H. (2023). “Utilizing virtual reality in higher education marketing through open-source and open-educational software,” in Proceedings of the 2023 7th international conference on big data and internet of things.

Google Scholar

dela Cruz, D. R., and Mendoza, D. M. M. (2018). “Design and development of virtual laboratory: a solution to the problem of laboratory setup and management of pneumatic courses in bulacan state university college of engineering,” in 2018 IEEE games, entertainment, media conference (GEM) (IEEE), 1–23. doi:10.1109/gem.2018.8516467

CrossRef Full Text | Google Scholar

Faculty of Civil Engineering RWTH Aachen University (2019). How to build a mobile flood protection wall. Aachen, Germany: Faculty of Civil Engineering RWTH Aachen University. Available at: https://www.youtube.com/watch?v=quQ3lCXV0TY (Accessed May 21, 2024).

Google Scholar

Fekete, A. (2020). Critical infrastructure cascading effects. disaster resilience assessment for floods affecting city of cologne and rhein-erft-kreis. J. Flood Risk Manag. 13, e312600. doi:10.1111/jfr3.12600

CrossRef Full Text | Google Scholar

Ferrell, J. B., Campbell, J. P., McCarthy, D. R., McKay, K. T., Hensinger, M., Srinivasan, R., et al. (2019). Chemical exploration with virtual reality in organic teaching laboratories. J. Chem. Educ. 96, 1961–1966. doi:10.1021/acs.jchemed.9b00036

CrossRef Full Text | Google Scholar

Garrido, L. E., Frías-Hiciano, M., Moreno-Jiménez, M., Cruz, G. N., García-Batista, Z. E., Guerra-Peña, K., et al. (2022). Focusing on cybersickness: pervasiveness, latent trajectories, susceptibility, and effects on the virtual reality experience. Virtual Real. 26, 1347–1371. doi:10.1007/s10055-022-00636-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Gloy, K., Weyhe, P., Nerenz, E., Kaluschke, M., Uslar, V., Zachmann, G., et al. (2021). Immersive anatomy atlas: learning factual medical knowledge in a virtual reality environment. Anat. Sci. Educ. 15, 360–368. doi:10.1002/ase.2095

PubMed Abstract | CrossRef Full Text | Google Scholar

Guo, L., Wang, D., Gu, F., Li, Y., Wang, Y., and Zhou, R. (2021). Evolution and trends in intelligent tutoring systems research: a multidisciplinary and scientometric view. Asia Pac. Educ. Rev. 22, 441–461. doi:10.1007/s12564-021-09697-7

CrossRef Full Text | Google Scholar

Gya, R., and Bjune, A. E. (2021). Taking practical learning in stem education home: examples from do-it-yourself experiments in plant biology. Ecol. Evol. 11, 3481–3487. doi:10.1002/ece3.7207

PubMed Abstract | CrossRef Full Text | Google Scholar

Hamilton, D., McKechnie, J., Edgerton, E., and Wilson, C. (2021). Immersive virtual reality as a pedagogical tool in education: a systematic literature review of quantitative learning outcomes and experimental design. J. Comput. Educ. 8, 1–32. doi:10.1007/s40692-020-00169-2

CrossRef Full Text | Google Scholar

IBS Technics GmbH (2015). Product catalogue – HW-B100k/HW-W100K. Thierhaupten, Germany: IBS Technics GmbH. Available at: https://www.flood-defenses.com/fileadmin/documents/Produktkatalog_100K_System_-_Englisch.pdf (Accessed May 21, 2024).

Google Scholar

Jiang, J., and Fryer, L. K. (2024). The effect of virtual reality learning on students’ motivation: a scoping review. J. Comput. Assisted Learn. 40, 360–373. doi:10.1111/jcal.12885

CrossRef Full Text | Google Scholar

Johnson-Glenberg, M. C. (2018). Immersive vr and education: embodied design principles that include gesture and hand controls. Front. Robotics AI 5, 81. doi:10.3389/frobt.2018.00081

PubMed Abstract | CrossRef Full Text | Google Scholar

Jongbloed, J., Chaker, R., and Lavoué, E. (2024). Immersive procedural training in virtual reality: a systematic literature review. Comput. and Educ. 221, 105124. doi:10.1016/j.compedu.2024.105124

CrossRef Full Text | Google Scholar

Kamińska, D., Sapiński, T., Aitken, N., Rocca, A. D., Barańska, M., and Wietsma, R. (2017). Virtual reality as a new trend in mechanical and electrical engineering education. Open Phys. 15, 936–941. doi:10.1515/phys-2017-0114

CrossRef Full Text | Google Scholar

Krokos, E., Plaisant, C., and Varshney, A. (2019). Virtual memory palaces: immersion aids recall. Virtual Real. 23, 1–15. doi:10.1007/s10055-018-0346-3

CrossRef Full Text | Google Scholar

Kuhne, C., Kecelioglu, E. D., Maltby, S., Hood, R. J., Knott, B., Ditton, E., et al. (2023). Direct comparison of virtual reality and 2d delivery on sense of presence, emotional and physiological outcome measures. Front. Virtual Real. 4. doi:10.3389/frvir.2023.1211001

CrossRef Full Text | Google Scholar

Kumar, M., Packer, B., Koller, D., Mohta, A., and Khurana, N. (2010). “Cushing syndrome in an infant due to cortisol secreting adrenal pheochromocytoma: a rare association,”. J. Pediatr. Endocrinol. Metab. Editors J. Lafferty, C. Williams, J. Shawe-Taylor, R. Zemel, and A. Culotta, 23, 621–625. doi:10.1515/jpem.2010.102

PubMed Abstract | CrossRef Full Text | Google Scholar

Lee, J. Y. (2021). A study on metaverse hype for sustainable growth. Int. J. Adv. smart convergence 10, 72–80.

Google Scholar

Mahling, M., Wunderlich, R., Steiner, D., Gorgati, E., Festl-Wietek, T., and Herrmann-Werner, A. (2023). Virtual reality for emergency medicine training in medical school: prospective, large-cohort implementation study. J. Med. Internet Res. 25, e43649. doi:10.2196/43649

PubMed Abstract | CrossRef Full Text | Google Scholar

Maiero, J., Fehling, C. D., and Müser, S. (2023). “Does higher education need virtual reality? a survey,” in 2023 11th international conference on information and education technology (ICIET) (IEEE), 6–13.

CrossRef Full Text | Google Scholar

Makransky, G., Andreasen, N. K., Baceviciute, S., and Mayer, R. E. (2021). Immersive virtual reality increases liking but not learning with a science simulation and generative learning strategies promote learning in immersive virtual reality. J. Educ. Psychol. 113, 719–735. doi:10.1037/edu0000473

CrossRef Full Text | Google Scholar

Makransky, G., and Petersen, G. B. (2021). The cognitive affective model of immersive learning (camil): a theoretical research-based model of learning in immersive virtual reality. Educ. Psychol. Rev. 33, 937–958. doi:10.1007/s10648-020-09586-2

CrossRef Full Text | Google Scholar

Markant, D. B., Ruggeri, A., Gureckis, T. M., and Xu, F. (2016). Enhanced memory as a common effect of active learning. Mind, Brain, Educ. 10, 142–152. doi:10.1111/mbe.12117

CrossRef Full Text | Google Scholar

Miguel-Alonso, I., Checa, D., Guillen-Sanz, H., and Bustillo, A. (2024). Evaluation of the novelty effect in immersive virtual reality learning experiences. Virtual Real. 28, 27. doi:10.1007/s10055-023-00926-5

CrossRef Full Text | Google Scholar

Mouatt, B., Smith, A. E., Mellow, M. L., Parfitt, G., Smith, R. T., and Stanton, T. R. (2020). The use of virtual reality to influence motivation, affect, enjoyment, and engagement during exercise: a scoping review. Front. Virtual Real. 1, 564664. doi:10.3389/frvir.2020.564664

CrossRef Full Text | Google Scholar

Pekdağ, B. (2020). Video-based instruction on safety rules in the chemistry laboratory: its effect on student achievement. Chem. Educ. Res. Pract. 21, 953–968. doi:10.1039/d0rp00088d

CrossRef Full Text | Google Scholar

Radianti, J., Majchrzak, T. A., Fromm, J., and Wohlgenannt, I. (2020). A systematic review of immersive virtual reality applications for higher education: design elements, lessons learned, and research agenda. Comput. and Educ. 147, 103778. doi:10.1016/j.compedu.2019.103778

CrossRef Full Text | Google Scholar

Schmidt, M., and Huang, R. (2021). Defining learning experience design: voices from the field of learning design and technology. TechTrends 66, 141–158. doi:10.1007/s11528-021-00656-y

CrossRef Full Text | Google Scholar

Sellberg, C., Nazari, Z., and Solberg, M. (2024). Virtual laboratories in stem higher education: a scoping review. Nordic J. Syst. Rev. Educ. 2. doi:10.23865/njsre.v2.5766

CrossRef Full Text | Google Scholar

Slater, M. (2009). Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical Trans. R. Soc. B Biol. Sci. 364, 3549–3557. doi:10.1098/rstb.2009.0138

PubMed Abstract | CrossRef Full Text | Google Scholar

Slater, M., and Sanchez-Vives, M. V. (2016). Enhancing our lives with immersive virtual reality. Front. Robotics AI 3. doi:10.3389/frobt.2016.00074

CrossRef Full Text | Google Scholar

Southgate, E. (2020). “Conceptualising embodiment through virtual reality for education,” in 2020 6th international conference of the immersive learning research network (ilrn) (IEEE), 38–45.

CrossRef Full Text | Google Scholar

Srinivasa, A. R., Jha, R., Ozkan, T., and Wang, Z. (2020). Virtual reality and its role in improving student knowledge, self-efficacy, and attitude in the materials testing laboratory. Int. J. Mech. Eng. Educ. 49, 382–409. doi:10.1177/0306419019898824

CrossRef Full Text | Google Scholar

Steuer, J. (1995). Defining virtual reality: Dimensions determining telepresence. J. Commun. 42 (4), 73–93. doi:10.1111/j.1460-2466.1992.tb00812.x

CrossRef Full Text | Google Scholar

Subramanian, S., Höreth, K., Sänger, N., et al. (2022). Catchment, streams and sewers: strengthening flood resilience in bonn

Google Scholar

Synofzik, M., Vosgerau, G., and Newen, A. (2008). Beyond the comparator model: a multifactorial two-step account of agency. Conscious. Cognition 17, 219–239. doi:10.1016/j.concog.2007.03.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, X., Huang, R. T., Sommer, M., Pei, B., Shidfar, P., Rehman, M. S., et al. (2024). The efficacy of artificial intelligence-enabled adaptive learning systems from 2010 to 2022 on learner outcomes: a meta-analysis. J. Educ. Comput. Res. 62, 1568–1603. doi:10.1177/07356331241240459

CrossRef Full Text | Google Scholar

Xu, X., and Wang, F. (2022). Engineering lab in immersive vr—an embodied approach to training wafer preparation. J. Educ. Comput. Res. 60, 455–480. doi:10.1177/07356331211036492

CrossRef Full Text | Google Scholar

Yang, C., Luo, L., Vadillo, M. A., Yu, R., and Shanks, D. R. (2021). Testing (quizzing) boosts classroom learning: a systematic and meta-analytic review. Psychol. Bull. 147, 399–435. doi:10.1037/bul0000309

PubMed Abstract | CrossRef Full Text | Google Scholar