Robin Horst
Ramtin Naraghi-Taghi-Off
Linda Rau
Ralf Doerner- Visualization Laboratory, Department of Design, Computer Science, Media, RheinMain University of Applied Sciences, Wiesbaden, Germany
Virtual Reality (VR) systems and VR content are complex, and their creation can mainly be conducted by experts in VR and related areas. That makes the use of VR challenging for experts from other domains, such as educators. In this paper, we build up on existing work and investigate the VR nugget concept—small self-contained VR systems that are built from educational design patterns. Particularly, we extend this concept and introduce structured authoring processes based on VR nuggets that show how standalone, combinable, and reusable VR software can serve as a meta-level guide for various VR applications and involve educators as domain experts. We conduct a user study with VR Forge, a VR-nugget-based authoring software tool, to draw conclusions on how pattern-based VR authoring tools should be designed to support domain experts. We compare our results with those of a related study of the VR nugget tool IN Tiles. Based on the comparative results of usability and hedonic quality measures, both anecdotal evidence and statistically significant results support the concept’s potential for VR authoring conducted by practitioners who are not experts in VR. We derive the recommendation that the design of a VR-nugget-based authoring environment will benefit from using both immersive and desktop user interface (UI) technologies and that the authoring workflow will need authors to frequently alternate between the technologies. We state findings and lessons learned from the development and the studies. We contribute insights in developing reusable and use-case-specific VR content and tools and propose authoring processes that focus on the tasks and goals of domain experts as the primary authoring role within educational VR development. Finally, the relevance of VR-nugget-based authoring is supported by anecdotal evidence gathered from over 3 years of investigating and applying it within three educational institutions and a company providing education services.
1 Introduction
Virtual Reality (VR) is a technology that can support educational purposes. Educators can provide spatial-related information for learners. Learners can explore the information interactively within a virtual world. However, the application of VR for teaching involves challenges that prevent VR from being used more widely. The advent of low-cost, high-fidelity VR hardware has reduced one significant challenge—the costs (Coburn et al., 2017). However, while we could observe within the last years that such systems are getting accessible for educators within various institutions such as universities, the creation of content for the systems is another significant challenge that remains and still prevents VR from a broader use in this domain.
The creation of learning content by splitting it into short and adaptable units [so-called learning nuggets (Hug, 2005; Bailey et al., 2006)] that deliver pragmatic and clearly presented learning knowledge is one significant sub-class of E-Learning. Despite ongoing research, developing (authoring) VR systems remains a difficulty for educators (Ashtari et al., 2020). Content development processes, for example, typically entail a slew of responsibilities and authors with a variety of backgrounds (Grimm et al., 2002; Abawi and Dörner, 2004). Furthermore, recent research (Ashtari et al., 2020) has revealed that novice authors have difficulty starting VR creation from scratch. The authors may benefit from the provision of templates or other ready-made software as a starting point.
In this paper, we explore the design of software tools that can support practitioners who are not experts in the authoring of VR experiences. We build up on existing work on the VR nugget concept (Horst and thesis, 2021)—small VR systems that convey concise educational VR experiences based on high-level design patterns from an educator’s perspective. We use the term “concise” within this paper to describe a comparatively short duration of the educational VR experiences with respect to the overall learning experience. The basic idea is inspired by concepts of the educational domain (Microlearning), where a course’s content is structured into multiple elementary and self-contained learning nuggets.
We make the following contributions:
• We analyze and summarize the current state of VR nuggets and their authoring.
• We investigate the user roles relating to VR nuggets and introduce authoring processes differentiating between different phases.
• We conduct a user study with an implementation of the VR Forge authoring tool (Horst and Dörner, 2019a) and compare our results with the ones of IN Tiles’s (Horst et al., 2020). Both authoring tools are based on VR nuggets. We state conclusions and lessons learned from the user studies and also based on researching and applying VR nuggets for over 3 years within three educational institutions and a company providing education services.
The paper is organized as follows. We review related work in the next section and thereafter we state work on VR nuggets that preceded this paper. Then, we present our concepts relating to VR nuggets and their authoring. Finally, we share experiences and discuss lessons learned from our practice and our evaluation before we conclude and give an outlook on future work.
2 Related Work—Virtual Reality Authoring and Tools
With a view to VR and its authoring, even decades after the first VR research, VR authoring and content creation still involve challenges (Green and Jacob, 1991; Billinghurst et al., 1997; Steed, 2008; Barbosa Dias et al., 2010; Wingrave and LaViola, 2010; Gerken et al., 2013; Dörner et al., 2014; Dörner et al., 2015; Ashtari et al., 2020). Current examples are early prototyping and testing of VR applications, the translation of design guidelines to VR, and the lack of concrete design guidelines Ashtari et al. (2020). Such challenges may even be more severe when domain experts (i.e., experts in domains other than computer science, where a VR experience shall be applied) are involved within the development. A recent study on different author backgrounds, conducted by Ashtari et al., 2020, concludes that domain experts were able to use VR authoring tools (e.g., game engines) but did not feel confident that they could create a VR experience from scratch. Domain experts stated that they wanted to use existing examples as a starting point as they did not know where or how to start. Recent work on authoring challenges that layperson authors have to face by Krauß et al., 2021 emphasizes that novel authoring tools should enable authors to create VR prototypes quickly and easily. Furthermore, they conclude that tools for domain experts should focus on tasks and goals from a practitioners perspective rather than providing complex feature-bound functionalities. Another aspect they pointed out was that all three dimensions should be utilized during the authoring process, referring to the notion of including immersive authoring methods (Lee et al., 2004).
Kemanji 2020 proposes a systematic implementation approach for VR training applications targeted at developers at the implementation phase of the development process. They also describe how they integrate experts from the application domain of the VR application. They are involved in two tasks: 1) defining object behavior and 2) defining training attributes. Both tasks are performed with VR developers who understand what behavior can be simulated and tracked. These processes are run through each time a new VR application is developed. Kemanji 2020 also proposes a tool for developers that supports them in developing parametrized VR training systems for cognitive intensive training tasks. However, the domain expert’s involvement remains advisory, and programmers still implement the VR environment and object behaviors individually.
Work by Adão et al., 2018 proposes a rapid prototyping tool that allows authors to create 360°-video-based immersive environments. The underlying videos can be complemented with static content (e.g., 3D models, texts, and sounds). Time-dependent behavior and user-interaction-dependent behavior can optionally be chosen for each of the content objects. Overall, Adão et al., 2018 decide to use content-specific parameters to adjust the 360°-video-based immersive environments depending on the type of content. Meyer and Pfeiffer 2020 provide a perspective on what is called VR-based digital reusable learning objects to include reusable content components within an authoring tool that enables domain experts to create 360°-video VR environments for healthcare training purposes. However, the authoring tool and the definition of the reusable objects were not yet introduced and remains future work.
Wang et al. (2017) propose a framework for developing VR experiences for the health domain. In their work, they state that the use of object-oriented application frameworks (Fayad and Schmidt, 1997) that combine domain-specific design patterns within software components reduce the cost and improve the quality of domain-specific software. They utilize a definition of frameworks from Johnson (1997), who defines this software technology as “reusable design of all or part of a system that is represented by a set of abstract classes and the way their instances interact.” However, similar to D-Flow (Geijtenbeek et al., 2011), their system provides authors with low-level components to create a VR system. More abstract patterns, for example, on a level where a component may represent a specific use-case, are not investigated.
Finally, creating VR experiences with game engine tools [e.g., Unity Technologies (2021b) and Unreal Engine Epic Games, Inc (2021)] has already become “the standard for AR/VR (development)” (Nebeling and Speicher, 2018) but still requires expert knowledge in programming and designing (Nebeling and Speicher, 2018). These requirements are barriers for non-technical designers and authors from outside the field of information technology (IT) (Conway et al., 2000; MacIntyre et al., 2004), so that game engine tools aim towards programming- and design-experienced users (Nebeling and Speicher, 2018). There are also tools that incorporate features similar to game engines and target a non-programmer and non-designer audience. For example, Alice (Pausch et al., 1995) at its current version (Alice 3) incorporates features such as a WYSIWYG scene editor, allowing authors visual access to properties of scene objects apart from code representations. Alice is a tool that targets teachers and students as authors for creating interactive 3D programs. It was originally designed for the rapid prototyping of VR. However, Alice developed into an educational programming environment that allows authors to create animations and 3D models using drag and drop actions. VR development with Alice will need a more specific setup of Alice or the Unity game engine. Another example tool for intuitive programming and scripting is Scratch (Maloney et al., 2004). Scratch utilizes visual programming aspects to let “young people” (children and teenagers from 8 to 16 years) learn programming. The programming aspects are visually represented with affordances that guide the authors during the development and ensure that only compatible programming blocks can be attached to each other. Scratch does not focus on VR development and generally only supports 2D game development. However, through advanced mathematical transformations (e.g., matrix transformations, 3D projections, or ray casting), 3D and VR projects are feasible but not suitable for domain experts or young adolescents. Current tools and technologies such as A-Frame (Marcos et al., 2021) and WebXR (W3C, 2021) seem promising for future authoring efforts targeting VR experience for the web. Such work also complements established game engine tools such as Unity. Unity currently does not target creating VR experiences made for web deployment.
Besides serving as tools to create VR experiences directly, game engines can also serve as a foundation to build novel VR authoring tools from them. This also became a trend in recent years, which is reflected in the number of scientific publications about developing game engine based authoring tools. Several of the earlier described tools also utilize game engines, precisely Unity [e.g., (Roth, 2011; Takala, 2014; Adão et al., 2018; Park and Han, 2018; Arrighi and Mougenot, 2019; Coelho et al., 2019; Gasques et al., 2019)]. Similar to tools that were not built with game engines, the tools resulting from game engines can differ in various aspects. One example is the area of application, such as authoring VR applications for museums (Mateevitsi et al., 2008). Another example would be the type of VR hardware that is supported, such as supporting the specific CAVE (Cruz-Neira et al., 1993) environment [e.g., (Jacobson and Lewis, 2005; Lugrin et al., 2012; Takala, 2014)] or supporting more consumer-oriented hardware such as HTC Vive or Oculus Rift head-mounted displays (HMDs) [e.g., (Park and Han, 2018; Arrighi and Mougenot, 2019; Coelho et al., 2019)].
Overall, our related work analysis has shown that novel VR authoring tools for end-user authors are demanded to be “[…] simple tools developed based on tasks and goals” (Krauß et al., 2021) instead of providing a low-level set of more complex and feature-bound functionalities. Particularly, such authoring tools should be designed with specific authoring roles and end-user authors in mind (Ashtari et al., 2020; Krauß et al., 2021), including immersive authoring methodologies in novel authoring tools intended for domain expert authors (Krauß et al., 2021), and providing tools for early- to mid-level prototypes (Ashtari et al., 2020). A lack of facile tools that include the authors’ knowledge background was noticed (Conole and Fill, 2005). However, generally, the studies have also shown that domain experts were willing to use VR authoring tools to create VR experiences by themselves. With a view to the learning domain, the alignment of VR content to particular learning goals is still challenging—the involvement of educational concepts within VR authoring concepts is sill scarce (Jensen and Konradsen, 2018; Kwon, 2019; Fransson et al., 2020).
3 Previous Work—Learning Nuggets and Virtual Reality Nuggets
Microlearning (Hug, 2005) is an educational concept where small learning units present easily understandable chunks of knowledge. There exists different terminology in the learning domain for units that a learning structure consists of, for example, learning nuggets (Bailey et al., 2006), learning objects (Wiley, 2000), micro-units (Hug, 2005), micro-content (Souza and do Amaral, 2014), or bite-sized learning chunks Armstrong and Sadler-Smith (2008). Some of them implicitly address the size of such a unit. Polsani 2003 describes that a larger curricular structure can be built from elementary micro-nuggets by combining them. Bannan-Ritland et al. (2002) describe nuggets on a micro-level as fundamental building blocks that can be used and combined for populating higher-level learning frameworks such as tutorials. Combining micro-nuggets [e.g., building a sequence by concatenating them (Wiley, 2000)] will create more complex learning structures on a macro or meso level (Hug, 2005) (illustrated in Figure 1). Furthermore, it is stated that learning nuggets benefit from being developed as standalone and self-contained content to support their reusability in different learning contexts, specifically the micro-nuggets (Longmire, 2000; Polsani, 2003; Berggren et al., 2005; Koper and Tattersall, 2005). Besides building a course entirely from micro-nuggets, this enables learning nuggets to extend existing learning structures that do not even relate to Microlearning (Lorenz, 2010). However, despite active research in the last decades and VR technology becoming affordable, there still exist challenges in creating and using VR experiences for educational purposes (Abdelaziz et al., 2014; Nersesian et al., 2019; Patterson and Han, 2019; Jong et al., 2020). Such challenges can inhibit the establishment of VR in different learning domains, even in technologically oriented ones such as Microlearning.
FIGURE 1. The concatenation of elementary micro-nuggets can build higher-level educational structures such as lessons, modules, and courses.
One approach that utilizes the educational authoring concept of learning nuggets is the concept of VR nuggets (Horst and Dörner, 2019a). From an educator’s perspective, a VR nugget is a concise and pattern-based VR experience. A straight-forward example is the show and tell VR nugget that annotates a 3D model with callouts (short text strings attached to the 3D model). Concise is used to emphasize a short duration of VR experiences compared to the overall learning experience that they are situated in. For example, when a VR experience with duration t is utilized as a part of a presentation with duration T, the VR experience is considered concise when the remaining presentation time minus the VR experience’s time still allows presenters to perform other presentation activities, such as presenting slides, showing videos, or including other concise VR experiences. In this case, a VR experience would not be concise when it is considered the central part of the presentation and takes a considerably high amount of the presentation’s time. We utilize the second attribute, pattern-based, within this paper to describe when a VR experience’s content is built from best-practices from a specific domain (Alexander et al., 1977) such as educational design patterns (Fincher, 1999; Eckstein and Voelter, 2000; Bergin et al., 2012). For example, a VR experience based on a hypothetical use-case where two objects shall be compared, a compare-pattern, could contrast aspects of two 3D models by drawing annotation lines between comparable parts of the models, explaining how the aspects differ or relate to each other. We might apply this pattern in different scenarios. For example, an implementation of the pattern could compare two car models by comparing different engine parts. Still, it could also be implemented comparing the composition of a fuel cell with the one of a lithium-ion battery. Besides using different 3D models, both implementations would rely on similar graphical elements, such as the connecting lines and text boxes.
From an educator’s view, VR nuggets have also no technical dependencies on other VR nuggets. To integrate such concise VR experiences in an educational structure, a similar model as illustrated in Figure 1 is used. A VR nugget replaces a slot of a learning nugget implemented with a different media or integrates as a new nugget. In the example of using a PowerPoint slide as a metaphor for one learning nugget, Figure 2 illustrates how VR nuggets could be used within a short presentation. Content of the presentation that is expected to be less suitable to be communicated with VR technology [e.g., factual information such as the agenda, some company background data (Parong and Mayer, 2018)] remains in the original media and technology. In contrast, two parts within the presentation (e.g., a novel fuel cell product and its structure) are represented as VR nuggets. This way, a VR nugget is conceptually placed in a curricular structure next to nuggets that rely on a different technology or next to other VR nuggets. Similar to the PowerPoint slide example, only one VR nugget can be active at a time. Studies indicate that VR nuggets were successfully utilized for learning purposes by learners and educators and have potential for future application for educational purposes (Horst and Dörner, 2019b; Horst and Dörner, 2019c; Dörner and Horst, 2021).
FIGURE 2. An example presentation structure to illustrate the application of VR nuggets in education.
From an author’s view, VR nuggets are readily implemented VR systems with placeholder content. For example, a show and tell VR nugget may be represented initially with a placeholder cube that is annotated by an example text. Concerning the authoring with VR nuggets, we identified three author classes (Horst et al., 2021b): System authors, who create the base system of an authoring tool for domain experts, pattern authors, who deal with identifying and representing educational design patterns as VR nuggets, and content authors, who are domain experts and utilize the authoring system to create VR experiences. The latter start their authoring with a baseline pattern implementations in the system and gradually substitute assets (e.g., 3D models, pictures, etc.) with their own material to tailor the VR nuggets for a certain use-case.
Currently, there exist two authoring systems that utilize VR nuggets—VR Forge (Horst and Dörner, 2019a) and IN Tiles (Horst et al., 2020). They differ concerning the user interfaces (UIs). VR Forge (Figure 3) is inspired by slideshow authoring tools such as Microsoft’s PowerPoint. Single VR nuggets can be aligned within a timeline and the currently active VR nugget’s content is displayed in large to conduct authoring actions. IN Tiles (Figure 4) provides content authors with an immersive authoring (Lee et al., 2004) user interface. VR nugget components are visualized as 3D tile-like affordances that can be utilized with VR hardware itself during the authoring. Besides affordances, IN Tiles’s authoring space is divided into three virtual rooms The authoring room, where content authors choose and assemble the tile structures and load assets from their hard drive, editing room, that are associated with one nugget each and where content authors perform spatially-related authoring tasks such as positioning virtual objects in 3D space, and the demo room, where content authors are placed within the exact scene of a VR nugget that learners will see upon deployment so that continuous testing can be performed.
FIGURE 3. Illustration of VR Forge’s UI using the example of a VR nugget for a wine distillation practical used within a chemicals lecture.
FIGURE 4. Illustrations of IN Tiles’s UI within the authoring room. Top: Tile-like affordances representing VR nugget components. Bottom left: Meta-authoring tools at the authors disposal (e.g., a connector tool to link VR nuggets and bring them into a timely execution order for deployment). Bottom right: Individual tile-like affordances illustrating their underlying assets.
Overall, previous work on VR nuggets has shown that VR-nugget-based authoring can be transferred to authoring tools, that it can be conducted by domain experts with the aid of the tools, and that it can produce VR experiences successfully usable for educational purposes. However, conceptual aspects concerning the stakeholders of VR nuggets, a VR nuggets’ components’ descriptions on an abstract level, their authoring processes, and an evaluation of the overall concept and its existing tools is still to be conducted.
4 Authoring With Virtual Reality Nuggets
In this section, we elaborate on the user roles and abstract components of a VR nugget and then introduce the authoring processes we identified with VR nuggets.
4.1 User Roles and Abstract Components
We refer to a VR nugget as VR software that results in a concise VR experience created based on an educative design pattern. On an abstract level, we differentiate the following user roles involved with a VR nugget.
• Author: The author role is responsible for creating VR nuggets. It can be sub-divided into roles of different author types Horst et al. (2021b).
• Educator: Educators may also be a sub-role of authors. However, concerning VR nuggets’ usage, educators are responsible for planning when to utilize a VR nugget and provide it to learners. Through an educator’s lens, a VR nugget is a sub-class of a learning nugget that is implemented with VR technology. Educators may also know that a specific VR nugget is built upon an educational design pattern and take this aspect into account when planning their lessons. In a self-learning application, where learners utilize VR nuggets independently of educators’ presence, the latter must take care that learners have access to the VR nugget (e.g., through a learning management system) and the VR hardware. In contrast, in a co-located application such as a presentation that is supplemented by VR nuggets, educators may also participate during the VR nuggets’ use by learners. However, their interface may not be implemented through VR hardware but through common desktop PC technologies.
• Learner: The learner role utilizes VR nuggets for learning purposes. The concise VR experiences are ultimately created for them by the authors and provided to them by the educators. Learners utilize VR hardware as a UI to VR nuggets. Learners may not know that a VR nugget is built based on a certain design pattern, that a VR nugget conveys concise VR learning experiences, or even that they utilize a VR nugget. Educators may hand VR nuggets to learners without a further introduction to underlying concepts. However, learners can notice certain aspects of VR nuggets during or after their usage. For example, learners could notice that some concise VR experiences utilize similar content, which may be attributed to using the same design pattern for multiple VR nuggets.
We distinguish four aspects to define a VR nugget on an abstract level—pattern, content, learner interface, and educator interface.
• Pattern: An essential difference to regular learning nuggets is that we presuppose that a VR nugget is based on an educational design pattern. A VR nugget is an instance of a design pattern that is suitable to be implemented as a concise educational VR experience. We refer to the underlying design pattern of a VR nugget as its type. The hierarchy graph in Figure 5 illustrates the relation of a VR nugget being an instance of a suitable design pattern for a VR nugget. To comply with learning nuggets being self-contained, unlike other patterns from the educational [e.g., (Goodyear, 2005)] or other domains [e.g., (Alexander et al., 1977)], a pattern suitable for VR nuggets must not rely on other patterns. A suitable pattern serves for VR nuggets similar to how an abstract class relates to object-oriented programming. It describes and specifies a VR nugget but does not implement it. A VR nugget is an actual implementation of a pattern. One pattern can be implemented by various VR nuggets, whereby implemented means that the conceptual model of a design pattern is reflected in a VR experience. Figure 6 illustrates this relationship in the example of a hypothetical show and tell design pattern in which a learner explores an annotated 3D model. This pattern can be applied in different scenarios and thus as a foundation for presenting different 3D models. For example, Figure 6 illustrates six implementations that are all based on the single mentioned design pattern.
• Content: We divide a VR nugget’s content into pattern-specific and scenario-specific content. Pattern-specific content is content that is explicitly defined by the VR nugget’s underlying pattern. To clarify this relation, we take again the example of a VR nugget that lets learners explore an annotated 3D object (Figure 6). Short text string annotations are pattern-specific content, for example, using text on a canvas connected to the components with a 3D line. Each implementation of this pattern would need such canvases and 3D lines. However, this pattern also requires the actual texts and a 3D model to be annotated. Besides specifying that these are needed, this pattern leaves open what 3D model and texts they can be. This content is referred to as scenario-specific content since it is adjusted to one particular scenario of the abstract use-case that the pattern was derived from. Pattern-specific content would reappear in different VR nuggets implemented on the same pattern and thus might make these classes of VR nuggets visually or interactively similar to learners. Based on the relation of pattern-specific and scenario-specific content, a set of parameters could be defined that is needed to implement the underlying pattern. Implementation-wise, parameters are used for describing what content authors must provide to an authoring tool for implementing a pattern. For example, parameters may specify how many 3D models must be provided if further assets or content in a broader sense are needed to implement a pattern as VR nugget with a given authoring tool. Some content may also be provided by the authoring tool itself.
• Learner interface: A VR nugget’s pattern not only describes visual but also interaction aspects concerning the learners’ UI. For concise educational VR experiences, it is crucial that learners can interact with the content. For the case of 360° video environment, this could be the possibility to look around within the scene. Besides defining parameters that must be filled during the implementation, the pattern defines a VR nugget’s interactivity for learners. In the example of the VR nugget that annotates a 3D object, the pattern may specify that learners can grab the 3D model, rotate it, and that the annotation canvas can be selected to provide more textual information on demand concerning the component that it annotates. The specification of the learners’ UI is mandatory for a VR nugget and directly encoded within its underlying design pattern. However, it may be described independently of the VR interface hardware. It remains for the actual implementation of the pattern to satisfy the interface specification with respect to available hardware within the targeted application. For example, using room-scale VR, learners might be able to walk to the 3D model and grab it by virtually touching it with their controller and pressing the hair-trigger. However, targeting a seated VR experience, the VR nugget may provide learners to teleport to specific locations and select and grab the 3D model using a laser pointer metaphor.
• Educator interface: While the learner interface is mandatory and directly encoded in a VR nugget and defined by its pattern, some applications may require or facilitate the VR nuggets’ usage when educators are provided with a UI as well. For example, in co-located Blended Learning settings, educators could steer learners through the VR when they are provided with insights into what the learner currently sees. Another example would be providing educators with the ability to let a group of learners collectively transition to or from the VR experience. Interfaces to other technologies that are related to the educator, such as sending standardized Shareable Content Object Reference Model (SCORM) data to a learning management system (LMS) for automating scores and grading, are also referred to the educator UI in this work. However, being optional, a VR nugget can be operated by learners and fulfill their purpose even without providing an educator UI. Still, such aspects may facilitate the usage of VR nuggets in various settings.
FIGURE 5. The hierarchy graph shows the connection of VR nuggets and patterns. The term VR nugget pattern is used in this graph to illustrate that there may be design patterns for educational VR that are not suitable for VR nuggets, for example, when they are not self-contained or would not result in a concise VR experience.
FIGURE 6. The relation of a pattern and a VR nugget. Several VR nuggets can implement a single pattern. For example, a VR nugget based on a pattern that is used for annotating components of a 3D model in VR may describe different parts of a fuel cell. However, we could also use the same pattern to implemented a VR nugget where components of a hard-drive, a car, or an SAE J1772 charger plug are annotated.
4.2 Authoring Processes
As pointed out in our literature review, different processes and tools that belong to a VR authoring pipeline can be distinguished, such as content creation, rendering, and hardware interfacing. We focus on content creation and sub-divide this process into two phases: 1) asset creation (e.g., 3D modeling, image/video creation): and 2) system composition. In the latter, assets are composed to form virtual worlds, for example, by specifying behaviour and interactions. Our authoring process with VR nuggets covers particularly this second phase and assumes that basic assets are either already present, created with external tools, or obtained from external sources for assets such as the Unity Asset Store (Unity Technologies, 2021a), Turbosquid (TurboSquid, 2021), or Thingiverse (MakerBot Industries, LLC, 2021). For example, basic 3D models may be already available due to the prior CAD modeling of a technical product for production purposes, that could be used in a polygon-reduced version, utilizing automated polygon reduction provided by Autodesk 3ds Max’s Batch ProOptimizer (Autodesk, Inc, 2021).
Our basic idea is to provide educators with VR nuggets that include placeholder content. These are utilized as a basis for educators to create VR content within a given VR system. During the authoring process, the content of the provided VR nuggets is adjusted to a specific scenario by replacing the placeholders successively. Regarding the system design, a distinction is made between two sub-classes of VR nuggets: 1) default VR nuggets and 2) adapted VR nuggets. A default VR nugget is a VR nugget that is initially provided to the educators. It comprises the mentioned placeholder objects (e.g., primitives as 3D models). Simple cubes can be used as placeholders for more complex 3D models, and a placeholder text can be displayed on annotations and descriptions. These default VR nuggets also include predefined interactions with the content that learners in VR can perform. In the example of show and tell, a learner is already enabled to grab and rotate the cube, without having to define this functionality first.
Providing ready-made VR applications as a starting point for domain expert authors is motivated by the findings of the literature research. During the authoring process, authors then gradually exchange the placeholder content with their own. The predefined functionalities, such as the learners’ interactions with the VR content, are preserved throughout the replacements. For example, when an initially provided cube that was defined as tangible within a default VR nugget is replaced by another 3D model, the novel 3D model is tangible as well. Providing already executable default VR nuggets as a starting point and letting authors replace the present content piece by piece ensures that a VR nugget remains in an executable state throughout the authoring process. If the content of a VR nugget has been changed from its initial state, for example, when a placeholder cube is replaced by the fuel cell 3D model, it is called an adapted VR nugget. Figure 7 illustrates the relation of default and adapted VR nuggets and their connection to patterns by means of an adjusted version of Figure 3. The original “VR nugget” node is split into a default VR nugget sub-type that instantiates and implements a suitable pattern and an adapted VR nugget that instantiates a default VR nugget.
FIGURE 7. The hierarchy graph shows the connection of VR nuggets and patterns from Figure 3 adjusted by the concepts of default and adapted VR nuggets. Here, we also differentiate between a design pattern and realizations of design patterns.
To draw further conclusions on how such interfaces should be designed, the tasks of each author role [system authors, pattern authors, and content authors Horst et al. (2021b)] are explored in more detail. The example of creating a pattern-based VR authoring system for providing educators a show and tell VR nugget is used as a representative for other patterns. It is based on the tripartite process identified in the Previous Work section (Horst et al., 2020). Each of these phases and our authoring roles are assigned tasks that are necessary to create the authoring system for educators, integrate the VR nugget, and adapt it to a specific use-case. Not all tasks can be assigned to one role exclusively. Such tasks are assigned to each applicable role so that the relation is not bijective but a surjective mapping from author roles to authoring tasks.
We call the first phase the system creation phase. This is the phase where programming experts realize the initial authoring system utilized to include VR nuggets and used by educators to create VR experiences. It includes different sub-tasks. We do not claim to present a completed task list, as sub-tasks could be sub-divided themselves again and again and would exceed the scope of this work. At the example of an authoring system created with the Unity game engine [e.g., (Horst and Dörner, 2019a)], authors that participate in creating Unity itself are already contributing to the system to a great extent. Thus they are also part of the system author role. Further tasks within the system creation phase can be carried out by authors actually using Unity as a tool to build an authoring environment with a higher abstraction of functionality. This includes creating a system design based on Unity components, implementing the system, and designing and programming UIs.
Within the second phase (pattern creation), pattern authors must first identify design patterns suitable for VR. This might require both knowledge from the application domain (e.g., education) and knowledge about VR systems. After identifying suitable patterns, they must be implemented as VR nuggets which requires expertise in Unity and programming. After their implementation, the VR nuggets must be integrated within the authoring system, which requires making changes within essential parts of the authoring system, for example, for making them available within the UI for the subsequent authoring of domain experts. This also includes providing initial 3D models, which must be either modeled professionally or can be obtained from stores such as the Unity Asset Store. Currently, the literature describes that the same type of authors performs both system creation and pattern creation (Horst and Dörner, 2019a). However, regarding the author classes (Horst et al., 2021b) and the mentioned tasks, we argue that pattern authors could be seen as power users that mediate between both the system authors’ technical perspective on the system and the content authors demands for the authoring system, its content, and UI.
The third phase is called VR nugget adaption phase. In this phase, the authoring system is ideally used by domain experts only. They would utilize the included VR nuggets as starting points to create VR content and adapt the ready-made placeholder content to the needs of their use-case. However, suitable assets are needed within this process. While text may be easily generated within the Unity-based authoring tool itself, 3D modeling, image generation (e.g., infographics), videos, or 360° videos and images are more challenging to create within one single tool. Again, professional 3D modelers, graphics designers, etc. must be included within the authoring process until the point where all basic assets are available for the domain experts to be inserted into the VR nuggets. Alternatives would be obtaining such assets, again, from asset databases like the Asset Store.
From a conceptual view, we identified three further phases in which the VR nugget adaption can be sub-divided. In the example of an educational use-case, an educator adapts a VR nugget for a specific use-case for the first time in the initialization phase. The educator would choose a suitable pattern and its associated VR nugget to start the authoring process. After this initial adaption is finished, the VR nugget can already be applied within the intended use-case. However, to further exploit and support the reusability of VR nuggets, the second phase we identified is the update phase. Single aspects might change over time. For example, an educator might change the text of an annotation, but the rest of the VR nugget’s assets are still convenient. Instead of adapting the same VR nugget again from the initialization phase, domain experts can benefit from functionalities that allow them to update already adapted VR nuggets. The last phase we identified is the adjustment phase. Similar to the update phase, already adapted VR nuggets are used as a foundation. Besides replacing the assets within VR nuggets with novel assets, it might be necessary for educators to adjust the content, for example, to a specific audience (e.g., different language, different knowledge background), a specific event location (e.g., small classroom vs. large podium), or timely constraints (e.g., restrict learners to walk/teleport to specific exhibits only). Such aspects can already be incorporated within the initialization phase. For example, an educator who knows that the educational VR experiences will be shown at two events, one with local (non-english) visitors and one with international guests, show and tell annotations can already be prepared initially. However, before using the VR nuggets, the right patterns must be selected, which is another essential authoring task for reusable pattern-based VR.
5 Evaluation
This section firstly describes our user study evaluating the VR-nugget-based authoring tool VR Forge. We use the outcome of this study to compare it with the existing study results of IN Tiles Horst et al. (2020). Thereafter, we state anecdotal results of working with and applying VR nuggets for over 3 years in different settings.
5.1 VR Forge User Study
In our user study, an implementation of VR Forge was utilized. Four aspects were considered in the user study, aligned with the IN Tiles study (Horst et al., 2020):
A1 Ease of use: Is handling the authoring tools manageable for our participants?
A2 Workflow: Can the participants use the tools from start to finish to adapt VR nuggets?
A3 Efficiency: Are participants satisfied with their results in terms of time spent creating them?
A4 Product character: How do the participants perceive the tools regarding hedonic and pragmatic qualities?
The user study involved 17 voluntary and unpaid participants [aged between 23 and 30 years with arithmetic mean (Ø) 25.6 and standard deviation (SD) 1.8, 1 female] with different backgrounds (e.g., teachers, part-time tutors, students, and lecturers). Their VR experience was captured on a 4-point scale with Ø 0.93 and SD 0.92, where 0 means they have never used VR technology, and 3 means they regularly use VR. On that basis, the participants of the study were classified as non-experts in the field of VR. One participant did not fill out demographic questions.
The user study was conducted as a moderated remote study. At the beginning of the study, each participant was given an executable prototype implementation of VR Forge adjusted to the user study tasks. After 5 mins of free exploration, the study included four tasks. The first task asked the participants to create a specific set of default VR nuggets and adapt them briefly with given content. The second task asked the participants to delete a sub-set of the adapted VR nuggets. For fulfilling the third task, we asked the participants to adapt a show and tell nugget to a given scenario (labeling automotive parts). The participants were also asked to perceive the adapted VR nugget with the VR simulator that VR Forge provides, so that desktop hardware was sufficient to conduct the remote study. At last, the participants were requested to adapt a puzzle VR nugget with given constraints and then save the adapted VR nugget on the desktop.
After the tasks were completed, the participants received a version of a questionnaire translated into German language. It included 14 custom questions Q1–Q14 with respective answer-pairs. The questions measured A1–A3.
Q1 How easy was it to select a VR nugget as a template? (difficult–easy)
Q2 How important do you find the possibility to use initial templates for VR applications? (unnecessary–helpful)
Q3 How acceptable was the time required to create the VR nuggets? (unacceptable–acceptable)
Q4 How much did the preview of the 3D content help you during the creation? (not at all–much)
Q5 How helpful were the menus for solving your tasks? (unnecessary–helpful)
Q6 How clearly did you find the division of functions in the different menus? (confusing–clearly arranged)
Q7 How easy did you find the interactions to customize the content of the VR nuggets? (difficult–easy)
Q8 How understandable was the workflow to you? (unintelligible–understandable)
Q9 How well were you able to keep track of your created “presentation” with VR nuggets? (not good–good)
Q10 Did you know at all times what you could do next? (I never knew–I always knew it)
Q11 How clearly did they find the entire application? (confusing–clearly arranged)
Q12 How easy was it to use the program after a short training period? (difficult–easy)
Q13 How well could you produce the same results with other software? (good–not good/not possible)
Q14 How much time would you have needed to create the VR nuggets with other software? (less–more)
A 7-point semantic differential scale (0–6) was utilized to capture the data from the questionnaire items stated above. The study was performed within a time frame of 45 min per participant.
Relating to the product character (Hassenzahl, 2018) (A4), we utilized the abbreviated AttrakDiff (Hassenzahl et al., 2003; User Interface Design GmbH, 2021) questionnaire to measure how experts perceived VR Forge concerning pragmatic qualities (PQs) (e.g., usability) and hedonic qualities (HQs) (Hassenzahl et al., 2000). Each questionnaire was structured into three sections. One included the questions and AttrakDiff items. Another one asked the participants about demographic information and their experience in related fields. The last one gave the participants the possibility to write down textual comments. The user studies also captured further qualitative data. Besides the written comments, this data was based on observations by the experiments, remarks and questions posed by the participants during the studies, and from thinking-aloud methodology (Lewis, 1982) we conducted during the study.
5.1.1 Analysis of the Results
Figure 8 represents the single items’ value distributions. The box and whisker plots show that the mean values of all questions lie on or above 3, which is the neutral value of the utilized 7-point scale. We utilized Microsoft’s Excel to generate the box and whisker plots in our work (Microsoft 2021) and used the excluded median function to calculate the quartiles. Dots represent outliers and stars the mean. Lines within the boxes represent the median value, which can also fall onto an edge of the boxes in case it is the same value as one of the quartiles. Items Q6, Q7, Q10, Q12, and Q13 show a large deviation ranging over five or more values. Q1, Q2, Q4, Q8, and Q12 have small variations ranging over two or fewer values. Outliers can be observed at Q1, Q3, Q8, and Q11. The outlier at Q3 has the lowest value of all outliers with 0. Only Q7 includes a value of 0 as well. Wilcoxon signed-rank tests were conducted on the individual items to analyze how VR Forge was rated by the participants compared to a hypothetical neutral rating. A test was run for each item compared to the neutral value of 3 points. With a threshold for statistical significance of 5%, ten tests did confirm significant differences. Table 1 shows all p-values together with the mean values and standard deviations for each question and each aspect.
FIGURE 8. Box and whisker plots for VR Forge’s items Q1–Q14.
TABLE 1. Mean values and output of the Wilcoxon signed-rank tests for Q1–Q14 and A1–A3. All values are rounded. 0 is the lowest, and 6 is the highest possible value.
We clustered the questions to the aspects 1–3:
A1: Q1, Q4, Q6, Q7, Q11, and Q12.
A2: Q2, Q5, Q8, Q9, and Q10.
A3: Q3, Q13, and Q14.
Figure 9 shows the value distributions of each aspect in box and whisker plots. All mean values lie above the neutral value of 3. All aspects show large deviations. Only one outlier could be detected, which lies at 0 at A1. Wilcoxon signed-rank tests were conducted to test the aspects’ outcome against a neutral rating. The results are shown in Table 1. All tests confirm significant differences.
FIGURE 9. Box and whisker plots aggregated by the three aspects A1–A3.
The free-form comments from the questionnaire, observations, and statements from the thinking-aloud protocol were used to capture additional data about the editing process and identify user errors and recoveries. They were clustered and assigned to A1–A3. Concerning A1, the ease of use, six participants mentioned the movement within the active VR nugget pane of VR Forge. Three noted it negatively with statements such as “Sometimes it moves as expected, and sometimes it moves differently.” The participant referred to the movement of 3D objects. Another one wrote that “Once you get used to the movement, it works.” The other three stated positive comments such as “All very intuitive with the movement of the mouse,” whereby the participant referred to the camera movement centered on the currently selected object of the scene. Two participants mentioned that the mini-preview of each VR nugget in the timeline was considered helpful.
Only a few comments referred to the workflow (A2) of the authoring tool. It was mentioned twice that the participants were unsure how to insert a new default VR nugget into the timeline. For both, it was observed that they just started the authoring tool. Upon starting the tool, the menu offering to insert a VR nugget was pre-opened for the participants so that nothing happened when they clicked on the button for adding a new VR nugget. After they found out, they did not mention further challenges regarding this topic.
Regarding A3, nine participants commented on the copying/saving/deleting process of VR Forge. All nine mentioned that it was inefficient to have to copy/save/delete one VR nugget after another rather than make a selection within the timeline and then perform the action to all selected VR nuggets. For example, participants commented “Can I somehow mark the nuggets all?” and “Oh, I have to select them all again.” Apart from that, it could be observed that the timeline menu could be used without further explanation and participants were able to switch between VR nuggets quickly. The previews enabled them to switch to the desired VR nugget efficiently without opening each VR nugget to see what content it contained.
The outcome of the AttrakDiff questionnaire was analyzed concerning the product character of VR Forge (A4). The results are shown in the portfolio presentation in Figure 10. It shows that the tool was assessed with higher hedonic than pragmatic quality. The visualization places VR Forge between the areas “self-oriented” and “neutral.” Its confidence rectangle intersects with both of these areas.
FIGURE 10. Portfolio-presentation of the AttrakDiff values. The small dark blue square represents the classification of the VR Forge prototype, and the light blue square represents the confidence.
The description of the word-pairs (Figure 11) shows the mean value for each item of the AttrakDiff individually. Three of four values related to the pragmatic quality are below 3 and only one item has a mean value higher than 3, which is “confusing–clearly structured.” On the contrary, three items relating to the hedonic quality have a value higher than 3, and only one was lower than 3 (“tacky–stylish”). The two items related to the tool’s overall attractiveness are placed near the 3-point mark (“ugly–attractive”) and between the values 1 and 2 (“bad–good”). “Bad–good” has also the lowest mean value of all items. The item with the highest mean value is “unimaginative–creative” with approximately 5.
FIGURE 11. Word-pair visualization of the mean values of the AttrakDiff questionnaire. Each word-pair represents a single item within the questionnaire.
5.1.2 Discussion
The user study results concerning VR Forge’s slideshow-inspired authoring show that the VR Forge prototype was generally perceived positively by the participants. This statement is supported by both the aspects and the single questionnaire items, which obtained mean values of at least 3. However, Figures 8, 9 show large deviations and indicate that not all participants agreed with each other. Still, the participants found the general idea of using pre-made applications as a start for developing their own VR helpful (Q2). The workflow of the slideshow-inspired tool was clearly recognized and understood by the participants (Q8).
The comments indicate that the participants found the slideshow-inspired UI helpful, particularly the timeline that offers previews and ordering functions. However, specific authoring tasks within VR Forge’s active VR nugget pane, such as positioning and moving through the scene with the desktop UI, were perceived as counter-intuitive and challenging. It was found acceptable after a short familiarization period for some participants, but the value distribution of Q7 indicates that these actions can be improved. This statement is also supported by Q7 having the lowest mean value of all questions and the largest deviation.
A3, the efficiency of VR Forge, was rated slightly worse than the first two aspects. Still, it shows that the participants were generally positive about the time they had to invest compared to the outcome they could create (Q3). Furthermore, the participants indicated that they would not have had a better alternative (Q14) or much of an option when they wanted to create similar VR experiences (Q13).
The outcome of the AttrakDiff shows a shift towards self-orientation and thus weaknesses of pragmatic nature that can be an indicator emphasizing that some participants had difficulties with the spatial-related authoring tasks. High hedonic qualities that were attributed to the tool indicate that the participants liked using the tool. Considering all average values of the questions, aspects, and the AttrakDiff evaluation, the study has shown that the participants could successfully use VR Forge to adapt VR nuggets.
5.2 Comparison of Virtual Reality Forge and IN Tiles
VR Forge and IN Tiles provide different UIs for authoring VR nuggets and different visualizations of the VR nugget concept. They are compared regarding A1, A2, A3, and A4. The comparison is based on the data obtained from Horst et al. (2020) and our presented study. Figure 12 shows box and whisker plots comparing the tools regarding A1–A3.
FIGURE 12. Box and whisker plots comparing the authoring tools by A1–A3.
Concerning the ease of use (A1), it illustrates similar plots for each tool. Outliers can be detected at both VR Forge and IN Tiles, at a value of 0. To draw conclusions on the differences’ significance, a Mann-Whitney U test was utilized with a threshold for statistical significance of 5%. The results are shown in Table 2, along with the mean values and the SDs. The test regarding the ease of use did not confirm any significant difference with p = 0.162 7. Still, the qualitative findings from oral comments and observations indicate that the participants could point out specific aspects of the tools they liked or did not like. For the VR Forge, criticism focused on the usability of authoring tasks within the active VR nugget pane. This pane included spatial interactions with the content, such as repositioning callouts of a show and tell. Authoring tasks of preparing the content for these spatial actions, such as inserting the 3D models and texts, and bringing the VR nuggets into spatial relations, were positively noted. This aspect was characterized to the contrary in IN Tiles. Here, such preparatory actions were described as a major point of criticism, whereas the participants commended IN Tiles’ immersive UI when spatial actions were performed within the editing room.
TABLE 2. Analysis of significant differences between the A1–A3. Mann-Whitney U tests were utilized. All values are rounded.
Regarding the workflow of the tools, Figure 12 shows greater differences for A2 than for A1. The plot for VR Forge has a larger value range than IN Tiles’s. It ranges from 1 to 6, whereas IN Tiles’s span range from 4 to 6, but outliers at lower values can be detected for the latter. Again, a Mann-Whitney U test with a threshold for statistical significance of 5% was conducted to explore the difference’s significance. The test showed with p = 0.007 0 that there exist significant differences. The post-hoc tests revealed that the differences are present as illustrated in Table 2. VR Forge has the lower mean value and significantly differs from IN Tiles. With the qualitative data, the outcome overall shows that IN Tiles incorporates the preferred authoring workflow with VR nuggets. It obtained a high mean value, a low value span, and few outliers.
The analysis of the efficiency (A3) shows the most variations of A1–A3 (Figure 12). VR Forge received a lower mean value and the higher value span. A Mann-Whitney U test with a threshold for statistical significance of 5% shows with p = 0.001 9 that the difference between the two tools is statistically significant. Qualitative data captured during the studies points out that the participants needed a short familiarization time with both tools to use them effectively.
The comparison of the two authoring tools regarding their product character (A4) is illustrated in Figure 13 and Figure 14. The portfolio-presentation shows that IN Tiles is placed within the “desired” area. In contrast, VR Forge is set apart from it, between “neutral” and “self-oriented.” The figure shows that IN Tiles obtained higher values relating to hedonic than pragmatic qualities. The word-pair visualization also supports this statement.
FIGURE 13. Portfolio-presentation of the AttrakDiff values.
FIGURE 14. Word-pair visualization of the mean values of the AttrakDiff questionnaire for the evaluation of VR Forge (blue) and IN Tiles (green).
Mann-Whitney U tests with a threshold for statistical significance of 5% were conducted on the outcomes of the AttrakDiff questionnaire. Table 3 depicts the results of the tests. Every test except for “confusing–clearly,” “cheap–premium,” and “unimaginative–creative” confirmed statistically significant differences.
TABLE 3. Mean values and output of the Wilcoxon signed-rank tests for the AttrakDiff’s outcome with regard to VR Forge and IN Tiles. All values are rounded. 0 is the lowest, and 6 is the highest possible value.
In general, the evaluation and the comparison of the tools have shown that the design of a VR-nugget-based authoring environment will benefit from using both immersive and desktop UI technologies. In such an authoring process, the authors will choose between a desktop UI and an immersive UI. Such alternating authoring can also introduce novel challenges. Existing work on VR authoring tools that incorporate immersive authoring techniques mentioned within our literature research focuses on creating a virtual environment entirely in a VR [e.g., (Dunk et al., 2010; Dunk, 2013)] or decoupling immersive and non-immersive authoring tasks to separate author roles within collaborative VR development [e.g., (Coelho et al., 2019; Nebeling et al., 2020)]. However, with respect to VR-nugget-based approaches, authors might perform such switching between immersive and desktop UIs frequently and multiple times, for example, for authoring several concise VR experiences. This aspect that we pointed out in our evaluation is not covered by existing work, yet.
The outcome of the evaluation suggests that the design of an effective tool for authoring VR nuggets incorporates a desktop UI for preparatory authoring tasks (e.g., selecting VR nuggets/patterns, inserting texts, or arranging VR nuggets temporally), and an immersive UI for the spatial modifications of the content (e.g., moving objects in 3D space, pinning callouts to certain points of a 3D model, or rotating/scaling objects). Such a design should incorporates IN Tiles’s room concept since the workflow that is accompanied by using them was well understandable for the participants and could be used efficiently. IN Tiles’ editing room should reflect the tool’s immersive part, whereas the assembly room’s functionality could be adopted from VR Forge’s desktop UI. Even though VR Forge was rated less intuitive by the participants, such a tool would include also the slideshow-based visualization concept for VR nuggets of the VR Forge. The slideshow-specific elements were perceived particularly positive (e.g., the slide-like arrangement and previews within a timeline). The temporal arrangement of VR nuggets in the timeline provided the best results. The corresponding functionalities that were provided by IN Tiles (associated with the connector tool) were particularly criticized.
5.3 Evaluation of the Virtual Reality Nuggets’ Practical Usage
During the last 3 years, we applied VR nuggets in different areas of application beyond academic research. This section gives a brief presentation on where VR nuggets were applied to complement the insights based on the quantitative data (e.g., concerning the usability) and qualitative participant feedback with anecdotal evidence. Feedback and lessons learned regarding a real-world application of VR nuggets are described.
5.3.1 Public Educational Application
VR nuggets were utilized at three different universities. One of them used a hand-made set of VR nuggets we made for them for demonstration purposes. VR nuggets enhanced a laboratory research presentation with relation to hydrogen fuel cells. The laboratory staff team appreciated the VR nuggets, could utilize them at events, and they were confident about authoring VR nuggets with programming and design experts. Particularly, they were glad about the ready-made VR nuggets that were delivered to them. An intrinsic motivation for utilizing one of the proposed authoring tools could not be identified.
The second university’s personnel that was interested in utilizing VR nuggets also had a background in chemistry and physics. They wanted to use VR nuggets as part of their preparation of students for laboratory internships. In those internships, students often have to assemble complex setups of flasks and chemicals. False arrangement of the different parts can have dangerous and costly consequences. Their idea was to let students practice the assembly task in VR before the actual laboratory internship to reduce the chance of mistakes. They were highly interested in creating VR nuggets by themselves in using VR Forge as their chosen tool. Specifically, the puzzle VR nugget was appreciated and utilized for their purposes. One item of feedback from the staff team was that they had wished for small changes in puzzle, for example, integrating small knowledge tests and applying gravity for certain objects. From these insights, we derived the Chemistry laboratory practicals preparation pattern described in (Horst et al., 2021b). However, they noted that puzzle was still sufficiently suitable for their needs. They also expressed that they would not have been able to create a similar VR experience by themselves without the tool. Furthermore, the VR Forge was utilized in students’ theses and courses of the university’s educational science faculty to explore the use for their purposes in more detail and allow students from areas outside of Applied Computer Science getting a first-hand experience in creating domain-specific VR content.
At the third institution, VR nuggets were used within three courses of its computer science department. Within one session of each course, VR Forge (two courses) or IN Tiles (one course) were handed out to the students. They were provided with assets and authoring tasks to give them an idea of VR development and technologies. Both students and lecturers did report that VR nuggets and our authoring tools did contribute in getting a rough feeling of what a VR experience is and how it is created without taking too much time and effort of the overall course. Specifically, simplistic VR nuggets such as show and tell and compare were utilized for this purpose, for being able to finish the development and letting students experiencing the results of their fellow students within a single session.
5.3.2 Commercial Application
Besides applying VR nuggets in pure public educational institutions, VR nuggets were also utilized by a company that offers commercial education in the form of continuing medical education. They wanted to use concise VR and AR experiences within their online courses for physicians. The VR nugget concept was appreciated, and they were willing to use authoring tools. They had their own IT department, who did not have experience regarding VR, but which included programming experts. Based on the initial VR nugget implementations and the Unity project of VR Forge, they started to learn and use Unity to develop their VR nuggets using the default VR nuggets as a start. Despite using Unity as an authoring environment for VR nuggets, the proposed persistence concept and the file format were retained to deploy VR nuggets to software that should “play” the VR nuggets. No particular interest in utilizing more standardized formats such as FBX or glTF was mentioned, also due to restrictions that would be associated with them such FBX being a closed and proprietary format and bound to Autodesk’s official SDKs. However, the resulting VR nuggets could not be published by the company in their courses due to certification issues of VR as a medium for continuing medical education. The company also started developing concise AR applications based on our VR nugget concepts using Unity.
6 Conclusion and Future Work
In this paper, we analyzed and summarized the current state of VR nuggets—a VR authoring concept based on educational authoring methods. We built up on the VR nugget concept and investigated in user roles and authoring processes relating to it. The results of our user study with VR Forge were compared to IN Tile’s study results to draw conclusion on how a VR-nugget-based authoring tool should be designed. Finally, we stated lessons learned from applying VR nuggets throughout the last years.
Not only should proposed refinements of the different tools be performed in the future, but also novel conceptual research can be conducted with a view to our results. Particularly, using both immersive and desktop UI technologies for layperson authoring is promising for further investigations. Benefits from both immersive and less-immersive authoring technologies could be combined in authoring tools, which let authors switch back and forth or enable them to use both simultaneously (e.g., fish tank VR). This is an aspect that may not only be applicable to VR-nugget-based authoring. However, concerning VR nuggets, technologies such as a zSpace might provide enough immersion for letting content authors perform the spatial-related authoring tasks on the one hand and may be non-intrusive enough for letting authors keep on the equipment during the preparatory tasks with the desktop UI on the other hand.
In the course of applying VR nuggets for educational purposes, we identified the switching in and out of a virtual space (or put on/take off a HMD) as a potential source of disruption of the overall learning experience. In the example of utilizing ready-made VR nuggets within a course, students need to take on and off the HMD, which might result in frustration and fatigue for the audience. Since our paper is about the authoring experience rather than the final educational experience, we did not focus on this particular challenge. We see potential to conduct further research in this direction, for example, to build up on existing work about on- and offboarding Hovhannisyan et al. (2019) of VR experiences. Particularly, we find using transitioning methodology to be promising, also because transitioning techniques have already proven useful for improving the switching into a VR [e.g., to enhance the feeling of presence Slater et al. (1994); Steinicke et al. (2009)]. Work to build up on such challenges with respect to the conciseness of VR nugget experiences Horst et al. (2021a) already proposed suitable transition techniques, however, a real-world application of such transitions with VR nuggets and the influence on concepts such as presence or the learning experience remains open.
Another aspect that can be explored based on our presented results is the use of design patterns that will result in longer VR experiences. We utilized aspects from the concept of learning nuggets on the micro-level, which do already support the reusability of learning content. However, in the educational sciences, there also exist the concepts of meso- and macro macro-nuggets Longmire, (2000), Polsani, (2003), Hug, (2005). Together with the proposed novelty of including educational design patterns, these concepts could be explored to enable domain experts to create longer VR experiences.
Generally, we have shown that the concise and pattern-based VR experiences based on VR nuggets were well accepted in educational scenarios and that VR nuggets can improve the quality of teaching in different aspects. Still, it remains open to conduct studies on the VR nuggets’ impact on the students’ learning outcomes. Aspects like task efficiency must be compared to learning nuggets that are realized with already established technologies. Research conducted by experts from the educational sciences could utilize one or more of the authoring tools to adapt VR nuggets based on the initial set of identified patterns and then evaluate their usage from a pedagogues point of view. We already stated first efforts in this area by applying VR nuggets in different institutions and a commercial context. However, besides applying VR nuggets for educational purposes, the system design and implementation concepts of VR nuggets could also be utilized by VR-experts to explore VR nuggets in different domains. This paper could be used as a guideline for researchers in the form of 1) identifying novel design patterns, 2) implement them in the form of VR nuggets, 3) provide them in the proposed authoring tools, and 4) evaluate them based on the aspects we used to assess them.
At last, suggested points for future work are the distribution and the deployment of VR nuggets. We investigated fundamental aspects of facilitating and designing software systems for these purposes. Based on these findings, such as that prospective authors might be willing to put adapted VR nuggets at the disposal of a wider community, the proposed tools should be extended by features for sharing VR nuggets with each other. Developing a VR nugget standard should be examined and established as a common format for VR content to enable cross-tool development. It can also be beneficial to include import and export options for VR nuggets in other established or even commercial authoring tools to foster VR software reuse. For example, it could be beneficial for authors familiar with Unity to be able to import and export adapted or default VR nuggets into or from a Unity editor environment. This could be realized with a separate Unity plug-in. Regarding the deployment, interfaces in domain-specific software should be investigated.
Data Availability Statement
The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.
Author Contributions
The Authors RH, RN-T-O, LR, and RD contributed to the work relative to the order of enumeration and agree to be accountable for the content of the work.
Funding
The work is supported by the Federal Ministry of Education and Research of Germany in the project Innovative Hochschule (funding number: 03IHS071).
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Acknowledgments
We acknowledge that material of this paper was subject of the dissertation by Horst (2021).
References
Abawi, D. F., and Dörner, R. (2004). “Creating Mixed Reality Content: Problems, Concepts and Solutions,” in Proceedings Computer Graphics International, 2004 (IEEE), 444–451.
Abdelaziz, M. A., Alaa El Din, M., and Senousy, M. B. (2014). Challenges and Issues in Building Virtual Reality-Based E-Learning System. Int. J. e-Education, e-Business, e-Management e-Learning. 4, 320. doi:10.7763/ijeeee.2014.v4.347
Adão, T., Pádua, L., Fonseca, M., Agrellos, L., Sousa, J. J., Magalhães, L., et al. (2018). A Rapid Prototyping Tool to Produce 360° Video-Based Immersive Experiences Enhanced with Virtual/multimedia Elements. Proced. Comput. Sci. 138, 441–453. doi:10.1016/j.procs.2018.10.062
Alexander, C., Ishikawa, S., and Silverstein, M. (1977). A Pattern Language: Towns, Buildings, Construction. New York, NY: Oxford University Press.
Armstrong, S. J., and Sadler-Smith, E. (2008). Learning on Demand, at Your Own Pace, in Rapid Bite-Sized Chunks: the Future Shape of Management Development? Amle. 7, 571–586. doi:10.5465/amle.2008.35882197
Arrighi, P.-A., and Mougenot, C. (2019). Towards User Empowerment in Product Design: a Mixed Reality Tool for Interactive Virtual Prototyping. J. Intell. Manuf. 30, 743–754. doi:10.1007/s10845-016-1276-0
Ashtari, N., Bunt, A., McGrenere, J., Nebeling, M., and Chilana, P. K. (2020). “Creating Augmented and Virtual Reality Applications: Current Practices, Challenges, and Opportunities,” in Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–13. doi:10.1145/3313831.3376722
[Dataset] Autodesk, Inc (2021). 3ds max Batch Prooptimizer Description. Available at: https://knowledge.autodesk.com/support/3ds-max/learn-explore/caas/CloudHelp/cloudhelp/2020/ENU/3DSMax-Modeling/files/GUID-97666A36-1087-41FA-9BBC-BF705C9543B3-htm.html (Accessed January 20, 2022).
Bailey, C., Zalfan, M. T., Davis, H. C., Fill, K., and Conole, G. (2006). Panning for Gold: Designing Pedagogically Inspired Learning Nuggets. Educ. Technology Soc. 9, 113–122.
Bannan-Ritland, B., Dabbagh, N., and Murphy, K. (2002). Learning Object Systems as Constructivist Learning Environments: Related Assumptions, Theories, and Applications. The instructional use Learn. objects., 61–98.
Barbosa Dias, J., Silva, A. D., Fernando Braga, L., Filgueiras Damasceno, E., Cardoso, A., and Afonso Lamounier, E. (2010). “A Software Architecture for Adapting Virtual Reality Content to mobile Devices,” in International Workshop on Future Multimedia Networking (Springer), 167–180. doi:10.1007/978-3-642-13789-1_16
Berggren, A., Burgos, D., Fontana, J., Hinkelman, D., Hung, V., Hursh, A., et al. (2005). Practical and Pedagogical Issues for Teacher Adoption of Ims Learning Design Standards in Moodle Lms. J. Interactive Media Education.1. doi:10.5334/2005-2
Bergin, J., Eckstein, J., Volter, M., Sipos, M., Wallingford, E., Marquardt, K., et al. (2012). Pedagogical Patterns: Advice for Educators (Joseph Bergin Software Tools).
Billinghurst, M., Baldis, S., Matheson, L., and Philips, M. (1997). “3d Palette: a Virtual Reality Content Creation Tool,” in Proceedings of the ACM symposium on Virtual reality software and technology, 155–156.
Coburn, J. Q., Freeman, I., and Salmon, J. L. (2017). A Review of the Capabilities of Current Low-Cost Virtual Reality Technology and its Potential to Enhance the Design Process. J. Comput. Inf. Sci. Eng. 17, 1–15. doi:10.1115/1.4036921
Coelho, H., Melo, M., Martins, J., and Bessa, M. (2019). Collaborative Immersive Authoring Tool for Real-Time Creation of Multisensory Vr Experiences. Multimed Tools Appl. 78, 19473–19493. doi:10.1007/s11042-019-7309-x
Conole, G., and Fill, K. (2005). A Toolkit for Creating Effective Learning Activities. Montreal: Ed-Media.
Conway, M., Audia, S., Burnette, T., Cosgrove, D., and Christiansen, K. (2000). “Alice: Lessons Learned from Building a 3d System for Novices,” in Proceedings of the SIGCHI conference on Human Factors in Computing Systems, 486–493.
Cruz-Neira, C., Sandin, D. J., and DeFanti, T. A. (1993). “Surround-screen Projection-Based Virtual Reality: the Design and Implementation of the Cave,” in Proceedings of the 20th annual conference on Computer graphics and interactive techniques, 135–142.
Dörner, R., Frechenhäuser, S., Gerken, K., and Berdux, J. (2014). “A 3d Metaphor for Authoring Realtime Interactive Systems Supporting Different Author Roles,” in Software Engineering and Architectures for Realtime Interactive Systems (SEARIS), 2014 IEEE 7th Workshop on (IEEE), 1–8. doi:10.1109/searis.2014.7152795
Dörner, R., and Horst, R. (2021). “Conveying Firsthand Experience: The Circuit Parcours Technique for Efficient and Engaging Teaching in Courses about Virtual Reality and Augmented Reality,” in Eurographics 2021 - Education Papers. Editors B. Sousa Santos, and G. Domik (The Eurographics Association). doi:10.2312/eged.20211002
Dörner, R., Kallmann, M., and Huang, Y. (2015). “Content Creation and Authoring Challenges for Virtual Environments: from User Interfaces to Autonomous Virtual Characters,” in Virtual Realities (Springer), 187–212. doi:10.1007/978-3-319-17043-5_11
Dunk, A., Haffegee, A., and Alexandrov, V. (2010). Selection Methods for Interactive Creation and Management of Objects in 3d Immersive Environments. Proced. Computer Sci. 1, 2609–2617. doi:10.1016/j.procs.2010.04.294
Dunk, A. (2013). Immersive Authoring for Virtual Reality. Ph.D. thesis. Reading (United Kingdom): University of Reading.
Eckstein, J., and Voelter, M. (2000). “Learning to Teach and Learning to Learn,” in Proceedings of EuroPLoP.
[Dataset] Epic Games, Inc (2021). Unreal Engine Description. Available at: https://www.unrealengine.com/(Accessed January 20, 2022).
Fayad, M., and Schmidt, D. C. (1997). Object-oriented Application Frameworks. Commun. ACM. 40, 32–38. doi:10.1145/262793.262798
Fincher, S. (1999). Analysis of Design: An Exploration of Patterns and Pattern Languages for Pedagogy. J. Comput. Mathematics Sci. Teach. 18, 331–348. doi:10.5555/336019.336029
Fransson, G., Holmberg, J., and Westelius, C. (2020). The Challenges of Using Head Mounted Virtual Reality in K-12 Schools from a Teacher Perspective. Educ. Inf. Technol. 25, 3383–3404. doi:10.1007/s10639-020-10119-1
Gasques, D., Johnson, J. G., Sharkey, T., and Weibel, N. (2019). “What You Sketch Is what You Get: Quick and Easy Augmented Reality Prototyping with Pintar,” in Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, 1–6.
Geijtenbeek, T., Steenbrink, F., Otten, B., and Even-Zohar, O. (2011). “D-flow: Immersive Virtual Reality and Real-Time Feedback for Rehabilitation,” in Proceedings of the 10th International Conference on Virtual Reality Continuum and Its Applications in Industry, 201–208.
Gerken, K., Frechenhäuser, S., Dörner, R., and Luderschmidt, J. (2013). “Authoring Support for post-wimp Applications,” in IFIP Conference on Human-Computer Interaction (Springer), 744–761. doi:10.1007/978-3-642-40477-1_51
Goodyear, P. (2005). Educational Design and Networked Learning: Patterns, Pattern Languages and Design Practice. Australas. J. Educ. Technol. 21, 82–101. doi:10.14742/ajet.1344
Green, M., and Jacob, R. (1991). SIGGRAPH '90 Workshop Report: Software Architectures and Metaphors for Non-WIMP User Interfaces. SIGGRAPH Comput. Graph. 25, 229–235. doi:10.1145/126640.126677
Grimm, P., Haller, M., Paelke, V., Reinhold, S., Reimann, C., and Zauner, R. (2002). “Amire-authoring Mixed Reality,” in The First IEEE International Workshop Agumented Reality Toolkit (IEEE), 2.
Hassenzahl, M., Burmester, M., and Koller, F. (2003). “AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität,” in Mensch & computer 2003 (Springer), 187–196. doi:10.1007/978-3-322-80058-9_19
Hassenzahl, M., Platz, A., Burmester, M., and Lehner, K. (2000). “Hedonic and Ergonomic Quality Aspects Determine a Software’s Appeal,” in Proceedings of the SIGCHI conference on Human Factors in Computing Systems, 201–208. doi:10.1145/332040.332432
Hassenzahl, M. (2018). “The Thing and I: Understanding the Relationship between User and Product,” in Funology 2 (Springer), 301–313. doi:10.1007/978-3-319-68213-6_19
Horst, R., and Dörner, R. (2019a). “Virtual Reality Forge: Pattern-Oriented Authoring of Virtual Reality Nuggets,” in 25th ACM Symposium on Virtual Reality Software and Technology (New York, NY, USA: Association for Computing Machinery), 1–12. doi:10.1145/3359996.3364261
Horst, R., and Dörner, R. (2019b). “Integration of Bite-Sized Virtual Reality Applications into Pattern-Based Knowledge Demonstration,” in Proceedings of the 16th Workshop Virtual and Augmented Reality of the GI Group VR/AR. Gesellschaft für Informatik (Shaker Verlag), 137–148. doi:10.2370/9783844068870
Horst, R., and Dörner, R. (2019c). “Mining Virtual Reality Nuggets: A Pattern-Based Approach for Creating Virtual Reality Content Based on Microlearning Methodology,” in Proceedings of the IEEE International Conference on Teaching, Assessment and Learning for Engineering (IEEE TALE), 1–8. doi:10.1109/TALE48000.2019.9225867
Horst, R., Naraghi-Taghi-Off, R., Rau, L., and Dörner, R. (2021a). Back to Reality: Transition Techniques from Short Hmd-Based Virtual Experiences to the Physical World. Multimedia Tools Appl., 1–24. doi:10.1007/s11042-021-11317-w
Horst, R., Naraghi-Taghi-Off, R., Rau, L., and Dörner, R. (2021b). “Design Patterns and Author Roles for Pattern-Based Educational Virtual Reality Experiences,” in Proceedings of the 13th Workshop Virtual and Augmented Reality of the GI Group VR/AR (Shaker Verlag).
Horst, R., Naraghi-Taghi-Off, R., Rau, L., and Dörner, R. (2020). Bite-sized Virtual Reality Learning Applications: A Pattern-Based Immersive Authoring Environment. jucs. 26, 947–971. doi:10.3897/jucs.2020.051
Horst, R. (2021). Virtual Reality Nuggets - Authoring and Usage of Concise and Pattern-Based Educational Virtual Reality Experiences. Ph.D. Thesis. Wiesbaden, Germany: RheinMain University of Applied Sciences.
Hovhannisyan, G., Henson, A., and Sood, S. (2019). “Enacting Virtual Reality: The Philosophy and Cognitive Science of Optimal Virtual Experience,” in International Conference on Human-Computer Interaction (Springer), 225–255. doi:10.1007/978-3-030-22419-6_17
Hug, T. (2005). “Micro Learning and Narration: Exploring Possibilities of Utilization of Narrations and Storytelling for the Design of “Micro Units” and Didactical Micro-learning Arrangements,” in Proceedings of Media in Transition.
Jacobson, J., and Lew, M. (2005). Game Engine Virtual Reality with Caveut. Computer. 38, 79–82. doi:10.1109/mc.2005.126
Jensen, L., and Konradsen, F. (2018). A Review of the Use of Virtual Reality Head-Mounted Displays in Education and Training. Educ. Inf. Technol. 23, 1515–1529. doi:10.1007/s10639-017-9676-0
Johnson, R. E. (1997). Frameworks = (Components + Patterns). Commun. ACM. 40, 39–42. doi:10.1145/262793.262799
Jong, M. S.-Y., Tsai, C.-C., Xie, H., and Kwan-Kit Wong, F. (2020). Integrating Interactive Learner-Immersed Video-Based Virtual Reality into Learning and Teaching of Physical Geography. Br. J. Educ. Technology. 51, 2064–2079. doi:10.1111/bjet.12947
Kemanji, K. V. (2020). “Method for Developing Virtual Reality Applications for Cognitive Intensive Training Tasks,” in Companion Proceedings of the 12th ACM SIGCHI Symposium on Engineering Interactive Computing Systems, 1–5. doi:10.1145/3393672.3398645
Koper, R., and Tattersall, C. (2005). Learning Design A Handbook on Modelling and Delivering Networked Education and Training. Berlin/Heidelberg, Germany: Springer.
Krauß, V., Boden, A., Oppermann, L., and Reiners, R. (2021). “Current Practices, Challenges, and Design Implications for Collaborative Ar/vr Application Development,” in CHI Conference on Human Factors in Computing Systems (ACM), 1–15. doi:10.1145/3411764.3445335
Kwon, C. (2019). Verification of the Possibility and Effectiveness of Experiential Learning Using Hmd-Based Immersive Vr Technologies. Virtual Reality. 23, 101–118. doi:10.1007/s10055-018-0364-1
Lee, G. A., Nelles, C., Billinghurst, M., and Kim, G. J. (2004). “Immersive Authoring of Tangible Augmented Reality Applications,” in Third IEEE and ACM International Symposium on Mixed and Augmented Reality (IEEE), 172–181.
Lewis, C. (1982). Using the” Thinking-Aloud” Method in Cognitive Interface Design. NY: IBM TJ Watson Research Center Yorktown Heights.
Longmire, W. (2000). Content and Context: Designing and Developing Learning Objects. Learn. Without Limits. 3, 21–30.
Lorenz, A. (2010). “Über kurz oder lang. ein schlichtungsversuch zur debatte über micro-und macrolearning,” in 8. Workshop on e-Learning, 79–88.
Lugrin, J.-L., Charles, F., Cavazza, M., Le Renard, M., Freeman, J., and Lessiter, J. (2012). “Caveudk: a Vr Game Engine Middleware,” in Proceedings of the 18th ACM symposium on Virtual reality software and technology, 137–144.
MacIntyre, B., Gandy, M., Dow, S., and Bolter, J. D. (2004). “Dart: a Toolkit for Rapid Design Exploration of Augmented Reality Experiences,” in Proceedings of the 17th annual ACM symposium on User interface software and technology (ACM), 197–206.
[Dataset] MakerBot Industries, LLC(2021). Thingiverse – Digital Design for Physical Objects. Available at: https://www.thingiverse.com/(Accessed January 20, 2022).
Maloney, J., Burd, L., Kafai, Y., Rusk, N., Silverman, B., and Resnick, M. (2004). “Scratch: a Sneak Preview [education],” in Proceedings. Second International Conference on Creating, Connecting and Collaborating through Computing (IEEE), 104–109.
[Dataset] Marcos, D., McCurdy, D., and Ngo, K. (2021). A-frame Description. Available at: https://aframe.io (Accessed January 20, 2022).
Mateevitsi, V., Sfakianos, M., Lepouras, G., and Vassilakis, C. (2008). “A Game-Engine Based Virtual Museum Authoring and Presentation System,” in Proceedings of the 3rd international conference on Digital Interactive Media in Entertainment and Arts, 451–457. doi:10.1145/1413634.1413714
Meyer, L., and Pfeiffer, T. (2020). “Virtual Reality-Based Digital Reusable Learning Objects in Healthcare Training (Virdipa): 360°-video Trainings,” in DELFI 2020 – Die 18. Fachtagung Bildungstechnologien der Gesellschaft für Informatik e.V. Editors R. Zender, D. Ifenthaler, T. Leonhardt, and C. Schumacher (Bonn: Gesellschaft für Informatik e.V.), 371–372.
[Dataset] Microsoft (2021). Box and Whisker Plot Documentation. Available at: https://support.microsoft.com/en-us/office/create-a-box-and-whisker-chart-62f4219f-db4b-4754-aca8-4743f6190f0d (Accessed January 20, 2022).
Nebeling, M., Lewis, K., Chang, Y.-C., Zhu, L., Chung, M., Wang, P., et al. (2020). “Xrdirector: A Role-Based Collaborative Immersive Authoring System,” in Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–12. doi:10.1145/3313831.3376637
Nebeling, M., and Speicher, M. (2018). “The Trouble with Augmented Reality/virtual Reality Authoring Tools,” in 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) (IEEE), 333–337. doi:10.1109/ismar-adjunct.2018.00098
Nersesian, E., Spryszynski, A., and Lee, M. J. (2019). “Integration of Virtual Reality in Secondary Stem Education,” in 2019 IEEE Integrated STEM Education Conference (ISEC) (IEEE), 83–90. doi:10.1109/isecon.2019.8882070
Park, N.-S., and Han, M. (2018). “Sensory Effect Authoring Tool for Virtual Reality Simulation,” in 2018 International Conference on Information and Communication Technology Convergence (ICTC) (IEEE), 1497–1499. doi:10.1109/ictc.2018.8539366
Parong, J., and Mayer, R. E. (2018). Learning Science in Immersive Virtual Reality. J. Educ. Psychol. 110, 785–797. doi:10.1037/edu0000241
Patterson, T., and Han, I. (2019). Learning to Teach with Virtual Reality: Lessons from One Elementary Teacher. TechTrends. 63, 463–469. doi:10.1007/s11528-019-00401-6
Pausch, R., Burnette, T., Capeheart, A., Conway, M., Cosgrove, D., DeLine, R., et al. (1995). Alice: Rapid Prototyping System for Virtual Reality. IEEE Computer Graphics Appl. 15, 8–11.
Roth, A. (2011). “The Arlab and Cave Libraries: On Authoring Augmented Reality and Virtual Reality Experiences Using a Graphical Programming Language,” in 2011 IEEE International Symposium on Mixed and Augmented Reality-Arts, Media, and Humanities (IEEE), 101–102. doi:10.1109/ismar-amh.2011.6093664
Slater, M., Usoh, M., and Steed, A. (1994). Depth of Presence in Virtual Environments. Presence: Teleoperators & Virtual Environments. 3, 130–144. doi:10.1162/pres.1994.3.2.130
Souza, M. I. F., and Amaral, S. F. d. (2014). Educational Microcontent for mobile Learning Virtual Environments. Ce. 05, 672–681. doi:10.4236/ce.2014.59079
Steed, A. (2008). “Some Useful Abstractions for Re-usable Virtual Environment Platforms,” in Software Engineering and Architectures for Realtime Interactive Systems-SEARIS.
Steinicke, F., Bruder, G., Hinrichs, K., Steed, A., and Gerlach, A. L. (2009). “Does a Gradual Transition to the Virtual World Increase Presence?,” in 2009 IEEE Virtual Reality Conference (IEEE), 203–210.
Takala, T. M. (2014). “Ruis: A Toolkit for Developing Virtual Reality Applications with Spatial Interaction,” in Proceedings of the 2nd ACM symposium on Spatial user interaction, 94–103.
[Dataset] TurboSquid (2021). Turbosquid 3d Model Source. Available at: https://www.turbosquid.com/(Accessed January 20, 2022).
[Dataset] Unity Technologies (2021a). Unity Asset Store. Available at: https://assetstore.unity.com/(Accessed January 20, 2022).
[Dataset] Unity Technologies (2021b). Unity Game Engine Description. Available at: https://unity.com/(Accessed January 20, 2022).
[Dataset] User Interface Design GmbH (2021). Attrakdiff Questionnaire. Available at: http://attrakdiff.de/index-en.html (Accessed January 20, 2022).
[Dataset] W3C (2021). Webxr Description. Available at: https://www.w3.org/TR/webxr/(Accessed January 20, 2022).
Wang, Y., Ijaz, K., and Calvo, R. A. (2017). “A Software Application Framework for Developing Immersive Virtual Reality Experiences in Health Domain,” in 2017 IEEE Life Sciences Conference (LSC) (IEEE), 37–30. doi:10.1109/lsc.2017.8268135
Wiley, D. A. (2000). Connecting Learning Objects to Instructional Design Theory: A Definition, a Metaphor, and a Taxonomy. instructional use Learn. objects. 2830, 1–35.
Keywords: virtual reality, authoring, virtual reality nuggets, design patterns, e-learning, system design
Citation: Horst R, Naraghi-Taghi-Off R, Rau L and Doerner R (2022) Authoring With Virtual Reality Nuggets—Lessons Learned. Front. Virtual Real. 3:840729. doi: 10.3389/frvir.2022.840729
Received: 21 December 2021; Accepted: 07 February 2022;
Published: 25 February 2022.
Edited by:
Dirk Reiners, University of Central Florida, United StatesReviewed by:
Aryabrata Basu, Emory University, United StatesChristopher Fluke, Swinburne University of Technology, Australia
Copyright © 2022 Horst, Naraghi-Taghi-Off, Rau and Doerner. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Robin Horst, robin.horst@hs-rm.de