A Review of Photogrammetry and Photorealistic 3D Models in Education From a Psychological Perspective

Nebel, Steve; Beege, Maik; Schneider, Sascha; Rey, Günter Daniel

doi:10.3389/feduc.2020.00144

REVIEW article

Front. Educ., 07 August 2020

Sec. Digital Education

Volume 5 - 2020 | https://doi.org/10.3389/feduc.2020.00144

A Review of Photogrammetry and Photorealistic 3D Models in Education From a Psychological Perspective

$\r\nSteve Nebel*$ Steve Nebel^*

Maik Beege

Sascha Schneider $Günter Daniel Rey\r\n$ Günter Daniel Rey

Psychology of Learning with Digital Media, Institute for Media Research, Faculty of Humanities, Chemnitz University of Technology, Chemnitz, Germany

Until sufficient practical experience and documented examples are available, emerging technological methods to create educational content typically require in-depth knowledge from different fields in order to apply them in the most beneficial way. The method of creating photorealistic 3D models (PR3DMs) of real-world objects with the methods of photogrammetry is an example of such emerging technologies. This paper discusses the fundamental concepts of photogrammetry to provide an overview of the necessary basic knowledge for educators and researchers interested in the presentation of realistic virtual materials. Additionally, a deeper introduction to basic psychological processes while learning with photorealistic materials is presented. This provides deeper insights into potentials and issues, as well as an overview of the most crucial research gaps in learning with PR3DMs. Finally, current and future applications, as well as the potential for learning topics apart from factual knowledge, such as emotional or behavioral outcomes, are discussed.

Introduction

Technologies used to create realistic virtual environments have made tremendous improvements over the last decades. Throughout this evolution, several milestones can be identified, where visual quality has taken a substantial leap forward, ranging from advanced methods of texture mapping (Blinn, 1978) to the widespread use of 3D engines (e.g., Hall et al., 1992). Most recently, one of these developments resulted in enthusiastic outbursts among computer game critics, as they stated that a group of developers “[…] were able to import wholesale from the real world weathered and highly detailed models that actually feel and look real. […] a world that is incredibly beautiful, immersive and tangible.” referring to the experience as “beyond awe-inspiring” (Aidan, 2014). Such reviews gained researchers’ attention, as realistic virtual environments might be useful in educational settings as well.

The technology that enabled a small developer to elicit such a positive review was photogrammetry. Despite its primary use in the context of a modern videogame design, photogrammetry is not a recent invention. Specifically, the approach of photogrammetry was already mentioned in the 19th century (e.g., Meydenbauer, 1867). Since then, the term has been used in various applications and contexts. Therefore, this paper illuminates the topic with an introduction to the field and a brief discussion about the relevant technical foundations. Using different psychological perspectives, the underlying processes during learning with the resulting digital materials are discussed. This discussion emphasizes the benefits and the challenges of photogrammetry and photorealistic 3D models (PR3DMs) in educational settings. Some examples and applications are presented next. To conclude this comprehensive review, a glimpse into the future of photogrammetry in education and research is provided.

What Is Photogrammetry?

The term photogrammetry is a variation of the word photography (Reulke and Döring, 2006; Kraus, 2012). The word photography itself is a compound of photo (from the Greek term, meaning light) and graphy (from the Greek term, meaning recording or writing). For the final term photogrammetry, the word metron (from the Greek term for a device to measure something) is added. Hence, photogrammetry can most simply be interpreted as a device or a technology that measures something using the recording of light. The American Society for Photogrammetry and Remote Sensing defines the term even more broadly as “the art, science and technology of obtaining reliable information about physical objects and the environment through processes of recording measuring and interpreting images and patterns of electromagnetic radiant energy and other phenomena” (American Society for Photogrammetry and Remote Sensing, 2019). Following this interpretation, other signals besides light waves (e.g., X-rays, radio waves) can be used in the process. This broad definition opens the door for the wide applications of photogrammetry, including archeology (Drap et al., 2007; Howland et al., 2014), architecture (Debevec, 1997; Agnello et al., 2019), topographic mapping (Fonstad et al., 2013), mining (Abbaszadeh and Rastiveis, 2017), construction (Tuttas et al., 2014), autonomous driving (Nguyen et al., 2019), detecting underground nuclear tests (Wang et al., 2018) and harvesting (Walter et al., 2018). Because of this broad definition and the widespread use, several different technologies have been subsumed under the term photogrammetry. This paper focuses on the most relevant approaches for education and educational videogame design with regard to applicability, accessibility, and required resources. Specifically, it discusses the widely available digital photography as the tool for measurement and the creation of PR3DMs as the desired outcome. This perspective excludes other technologies, such as laser scanning or satellite positioning, as well as large-scale environments, such as landscapes.

Photorealistic 3D Models

A basic definition of PR3DMs is needed to provide a common ground for the following discussion. This is especially important as computer graphics and visual representations in educational materials have evolved tremendously (e.g., Houghton and Willows(eds), 1987), and what is referred to as realistic might have changed over the last decades. Thus, a straightforward approach from computer graphics research is used. In this field, computer-generated images are referred to as realistic if they are indistinguishable from photographs (Rademacher et al., 2001). Thus, such images might also be called photorealistic. Similarly, PR3DMs can be characterized as digital models of real-world objects that appear indistinguishable from their physical counterparts, from a human observer’s perspective. In addition to the previous focus on photography, this definition includes observations from different angles and of different displays. This addition demarcates PR3DMs from other forms of digitization. For example, methods such as 360° photography are frequently used to generate photorealistic environments (e.g., Chamilothori et al., 2019). In contrast to PR3DMs, the content cannot be observed from any other angle as the viewpoint is fixed.

Technical Foundation

For the purpose of digitizing physical objects, photogrammetry has to measure and record two distinct attributes. First, the position of each surface point on the targeted object needs to be measured; second, the nature of each point (e.g., color, transparency, reflectance) should be determined. To assess the position, a series of pictures (at least two but typically dozens up to a few hundred) from different angles is needed to gather spatial information. Algorithms compare different image segments and their relative movements to each other from picture to picture (Gherardi et al., 2012; Toldo, 2013). Summarized, this process can be compared to the human perception of depth using the motion parallax, enabling humans to assess depth through the amount of movement in their visual field, with far-away objects moving at a lower rate than objects nearby (Rogers and Graham, 1979). From this, the relative position of each image, respectively, the camera parameters, are estimated (Figure 1). The figures depict one potential use-case for photogrammetry, the digitization of historical artifacts for museums and educational purposes. They are not intended as an endorsement of phrenology.

FIGURE 1

Figure 1. A screenshot showing how a PR3DM is created using a series of photographs. The blue cones represent the camera positions of photographs of a physical object (left), preliminary coordinates are calculated (right), screenshots are taken in 3DF Zephyr Lite (3Dflow, 2019), and arrows are added to indicate camera orientations.

Those parameters can be used to determine the 3D coordinates of each point of the desired object, which can be depicted as point clouds (Figure 2). These clouds are further densified, and a polygon mesh is generated (e.g., Glencross et al., 2008; for a detailed introduction, see Kraus, 2012, 2013; Toldo, 2013).

FIGURE 2

Figure 2. A sparse point cloud (top left), a dense point cloud (top right), a polygon mesh (bottom left), and a textured mesh (bottom right); screenshots taken in 3DF Zephyr Lite (3Dflow, 2019).

The resulting digital replication can be considered an exact digital duplicate of the physical object, with its accuracy depending on several variables, such as the source image quality, the number of included vertices, or the utilized algorithms. Subsequently, the object’s entire appearance is digitized, where textures are generated from the source images and mapped to the surface of the 3D mesh (Figure 3). Modern graphic engines enable advanced textures with dynamic features, such as transparencies or reflections (Figure 4). However, these effects typically need to be applied manually as they are difficult to detect automatically using the methods of photogrammetry (but see Melendez et al., 2012, for an approach of optimizing materials from photographs).

FIGURE 3

Figure 3. Mapping of the texture (left) to the 3D mesh (right); screenshot taken in Blender (Blender Foundation, 2019).

FIGURE 4

Figure 4. Rendering changes to recreate the original appearance: without changes (left), additional reflections to mimic the appearance of ceramics (middle), and a fictional metallic appearance (right); screenshot taken in Blender (Blender Foundation, 2019).

This technological approach has achieved results comparable to those of the previously established method of laser scanning (Kersten et al., 2015; Rehany et al., 2017). Additionally, the workflow is more efficient than traditional 3D modeling (Esmaeili et al., 2017), and the degree of realism can be further enhanced through post-processing (e.g., antialiasing, shadows, or depth of field; Rademacher et al., 2001; Sugano et al., 2003; Kan and Kaufmann, 2012). However, the complex process is subject to several limitations and technical considerations. For example, featureless surfaces (Nicolae et al., 2014) or moving objects might complicate the measuring process as they might provide insufficient or conflicting data. In low-light conditions that require a large aperture, some segments of the image might be blurred. Thus, a pixel cannot be used to measure a specific point on the physical object. This limits the achieved detail in the digitized duplicate. A low-light condition might also be challenging during the generation of textures since artificial amplifications of the measured light might cause artifacts that do not correspond to reality. More fundamentally, the process cannot digitize what cannot be measured with the used instrument – digital photography. This could be the case if an object is partially obstructed (e.g., by vegetation; Azzola et al., 2019), the needed perspective cannot be realized (e.g., capturing the inside of a handle; Tucci et al., 2012), or no suitable physical object is available. In such instances, researchers managed to utilize different technical solutions, such as the use of endoscopy or computer tomography (Fritsch et al., 2018). Finally, the process is resource demanding in terms of the necessary calculations. However, the capacities of current personal computers are sufficient to calculate the necessary steps within hours (depending on the complexity), and software solutions offering the required calculations have become increasingly available (e.g., 3Dflow, 2019; Capturing Reality, 2019).

Photogrammetry From the Perspective of Psychology

As the technical foundations and the produced outcomes become clear, a discussion about the learners’ perceptual processes might support the understanding of PR3DMs in educational contexts. Additionally, several subtopics in educational psychology, such as emotional or social design, outline the potential benefits of such realistic materials.

Perception and Transfer

If the perception and the transfer of spatial information about virtual objects and environments are discussed, the concept of fidelity is frequently utilized (e.g., Christel, 1994; Mania et al., 2006; Mourkoussis et al., 2010). Similar to the definition of PR3DMs, fidelity denotes the extent to which experiences in virtual environments are indistinguishable from the experiences in the real world (Waller et al., 1998). In more detail, Waller and colleagues differentiate between environmental and interface fidelity. The first describes the extent to which virtual representations resemble the real world. The latter refers to the extent to which the developed mental model is identical to the virtual environment. Naturally, deviations that might occur during the two associated transfer processes (from the real world to the virtual environment; from the virtual environment to the mental model) might induce differences between the initial object and the resulting cognitive representation. This might be especially problematic in an educational setting, as the goal might be to transfer the information as accurately as possible. To address this issue, researchers have tried to investigate different interfaces (e.g., ranging from screens to head-mounted displays). However, this research stream mostly addresses the aspect of interface fidelity (e.g., Makransky et al., 2019). In contrast, photogrammetry might be used to enhance environmental fidelity, which might be essential to ensure that the digital learning material was an adequate representation in the first place. By using photogrammetry, creators could avoid deviations that could occur during manual remodeling of physical objects. With countless measurements in the process, a much higher level of detail could also be achieved without a costly investment of work hours. The achieved geometry quality might be especially important if the objects are not yet established in mental schemata (Mourkoussis et al., 2010).

The use of PR3DMs could mitigate some negative effects that occur during the perception of virtual objects and the transfer of the associated information. For example, it is not a trivial task to infer the correct material from rendered visualizations (Vangorp et al., 2007) or to estimate the exact sizes of objects (Ahn et al., 2019) and correct distances in virtual environments (Smallman and St John, 2005; Kunz et al., 2009; El Jamiy and Marsh, 2019). Highly detailed PR3DMs could provide additional references that might facilitate these fundamental tasks. Additionally, low-texture details might be confused with the texture perspective (Cutting and Millard, 1984), resulting in the misperception that such objects are farther away than they really are (Drascic and Milgram, 1996). This potential harmful effect could be minimized with high-resolution textures that could be created during the process of photogrammetry. Furthermore, post-processing effects, such as blur or color tint, might be used to mimic the atmospheric perspective (El Jamiy and Marsh, 2019), further correcting the perception of spatiality. The associated sharpness of the virtual objects might further amplify the perceived realism of the objects (Cowan et al., 2015). However, their impacts on different information processing mechanisms need further validation to explore the effects on learning outcomes.

Cognition

The use of 3D models created by photogrammetry can significantly alter the cognitive processing involved in a learning task. To comprehend the potential benefits and drawbacks of using realistic 3D models, it is helpful to consider the mental processing of realistic imagery first. Skulmowski and Rey (2018) explain the perception of realistic learning materials with Biederman,1985,1987 geon theory. Geons are thought to be the basic perceptual units used in segmenting and processing visual stimuli (Biederman, 1987). Primitive 3D shapes, such as cylinders, cubes, and cones, have been identified as such units (Biederman, 1985). Based on the geon theory, the perception of 3D models can be considered a process where parts of the retinal image are grouped together as primitive 3D shapes to understand the spatial arrangement of the perceived object or scene (Biederman, 1985).

This perceptual process may be hindered by complex surfaces (Norman et al., 2004) and can thus be a potential source of cognitive load (i.e., information processed in the working memory; Sweller et al., 1998). Being required to mentally disassemble an object comprising hundreds of geons may induce more cognitive load than being presented with the object’s simplified version that features only a few primitive components. Based on this idea that realistic details induce cognitive load, some studies have shown that more details may interfere with learning processes (e.g., Scheiter et al., 2009). Altogether, realism can be an important aspect of cognitive processing in learning scenarios, but there may be disadvantages due to the higher perceptual complexity that it entails. Hence, future research is needed to find new ways of optimizing learning with realistic visualizations by reducing unnecessary cognitive strains. Stull et al. (2009) provide an example of such a method by demonstrating that the anatomy of a complex virtual bone model that can be freely rotated is more effectively learned if rotational axes (presented as thin cylindrical shapes next to the bone) are included as references to keep track of the object’s rotation. Another strategy to enhance learning with realistic visualizations is to use color cues that help segment complex surfaces (Skulmowski and Rey, 2018).

Learning Theories and Effects

Learning theories include cognitive frameworks as well. Both the cognitive load theory (CLT; Sweller et al., 1998; Sweller et al., 2019) and the cognitive theory of multimedia learning (CTML; Mayer, 2014) assume the working memory’s limitation in processing multimedia learning messages. These theories regard the working memory as the central bottleneck in information processing and according to the CLT, can be inundated by various types of cognitive loads. In addition to the intrinsic cognitive load (ICL), which is determined by the information complexity and the learner’s knowledge, the extraneous cognitive load (ECL) is determined by the information presentation and the learner’s requirements due to the instructional procedure (e.g., Sweller et al., 2019).

According to these two theories of cognitive psychology, realistic images should generally lead to an increase in ECL and consequently impair the learning performance. Empirically, these negative effects were demonstrated decades before the emergence of the CLT and the CTML. For example, Dwyer (1969) found that realistic images partially reduced the learning performance in comparison to less realistic representations. Even in more recent research, more realistic representations are sometimes associated with reduced learning performance (Scheiter et al., 2009), although the comparability between schematic and realistic images does not always seem to be given. Similar effects could be identified in virtual environments (Lokka et al., 2018). The influence of realism also appears to be moderated by the learners’ visual-spatial abilities. For example, Brucker et al. (2014) found that learners with lower visual-spatial abilities achieved better recognition performance in schematic visualizations, while learners with higher visual-spatial abilities achieved better recognition performance in realistic visualizations.

The use of realistic representations in the multimedia learning context can also be associated with the seductive detail effect, defined as interesting but irrelevant or unimportant additions to the actual learning topic (e.g., Harp and Mayer, 1998). Thus, the inclusion of decorative, realistic images in accordance with the seductive detail effect may lead to impaired learning performance (Rey, 2012). However, PR3DMs can also improve the learning process, among other things, given the attractiveness of the visual stimuli. The degree of realism of images can also be considered in the context of the signaling effect. The signaling effect implies that deeper understanding processes in multimedia learning occur when cues direct the learners’ attention to relevant information or highlight the organizational structure of the core content (e.g., Mayer and Fiorella, 2014; van Gog, 2014). Two recent meta-analyses (Richter et al., 2016; Schneider et al., 2018a) confirm the signaling effect. Similar to landmarks (Waller and Lippa, 2007), realistic details could serve as signaling elements. In another experiment (Skulmowski and Rey, 2020), realistic instructional visualizations served as signaling and improved learner retention in comparison to schematic visualizations. At the same time, however, realistic representations also partially increased the ECL in the study. The authors describe this result pattern as the realism paradox (Skulmowski and Rey, 2020).

The imagination effect could also help in a better estimation of the effect of realism in representations. Imagination is understood as the learning-enhancing effect through mental visualization or mental rehearsal of concepts or processes (e.g., Sweller et al., 2019). For example, two experiments conducted by Leopold and Mayer (2015) showed that students who were asked to form images corresponding to each of the nine paragraphs about the human respiratory system achieved better retention and transfer performance than students who only read the computer-based text about the topic. Additionally, schematic, less realistic may more likely help learners (further) visualize concepts or processes than more realistic images that presumably offer less scope for imaginative elaboration of the visualization.

Emotional Design

Apart from these well-established theories and effects, photogrammetry-based environments can provide new opportunities to foster learner performance and to establish constructive learning environments. The more realistic such environments are, the better people can immerse and engage in them (Walton, 1984; Dede, 2009). However, a higher degree of realism can also entail an increase in the viewers’ emotional response (Waltemate et al., 2018) or motivation (Makransky and Lilleholt, 2018). For example, watching a rather abstract map of a concentration camp is less emotionally demanding than a realistic version. Perceptions and affective feelings are inherently combined and mutually dependent (Wormwood et al., 2019). According to the integrated cognitive-affective model of multimedia learning (Plass and Kaplan, 2015) and the emotional design hypothesis (Um et al., 2012), the design of digital learning media can have a major impact on the learners’ affective states, while these states influence the selection, organization, and integration of new information into a coherent mental model stored in the long-term memory. Specifically, physically attractive stimuli in such environments, such as realistic colors, have a positive effect on the learners’ emotional response and learning performance (e.g., Dion and Walster, 1972; Tractinsky et al., 2000; Wolfson and Case, 2000). Several studies have replicated this hypothesis in terms of different colors and shapes or the inclusion of decorative elements in learning materials (e.g., Plass et al., 2014; Schneider et al., 2018b). Positive emotions, such as enjoyment of learning, elicited through realistic images, direct attention toward the learning task and allow the full use of cognitive resources to achieve the instructional objectives (Pekrun et al., 2002; Huk and Ludwigs, 2009). Moreover, elicited positive emotions may trigger an increase in cognitive processing compared to negative emotional episodes (e.g., Isen et al., 1987; Nadler et al., 2010). Eye-tracking data show that more detailed images attract more attention in both the initial learning phase and the overall learning processes than simplified illustrations (Lin et al., 2017). As suggested by Schneider et al. (2016), appealing pictures may contribute to an aesthetically pleasing design of multimedia learning materials, although learners’ cognitive load may increase (see also Plass and Kalyuga, 2019). This issue is of particular importance since an emotional overload may exceed the learners’ cognitive capacities and shift their attention away from the learning activities (Plass and Kaplan, 2015).

Consequently, the learning-diminishing effects of negative emotions elicited through highly realistic presentations might also occur. In a more realistic virtual environment tested by Slater et al. (2009), the participants reported not only a higher subjective presence but also an increased level of stress. According to the uncanny valley effect (Mori et al., 2012), the affinity to almost realistic representations of humans (e.g., a prosthetic hand) is lower than the affinity to pictures with a more simplified or perfectly realistic presentation. Instead of producing a more positive experience with increasing realism per se, strong adverse reactions are shown very frequently as well, particularly when users face highly elaborate digital entities. Specifically, digital technologies that appear to approach uniquely human qualities provoke ontological ambiguity (Moore, 2012), leading to uncertainty on how to behave toward and what to expect from them (Langer and König, 2018). This uncertainty also creates a sense of threat to physical health or human distinctiveness (Ferrari et al., 2016). These negative feelings can impair learning through aversive behaviors. In contrast, even negative emotional states can foster learning, for example, by promoting analytical-sequential, detailed, and rigid ways of processing information (Ashby et al., 1999; Pekrun et al., 2009) or by encouraging a more creative performance under specific conditions (George and Zhou, 2002). Consequently, it can be assumed that simplistic conceptions, which classify negative emotions as detrimental and positive emotions as beneficial, do not entirely apply and should therefore be avoided (Pekrun et al., 2002).

Social Learning

As one of the modern technical advantages, photogrammetry is especially suitable for creating social entities or the so-called personae (Hartmann et al., 2004) in digital environments. In an educational context, these entities can be classified as pedagogical agents (Veletsianos and Russell, 2014), which are computer-generated characters that serve instructional purposes (e.g., demonstrating, providing help, instructing; Veletsianos and Russell, 2014; Martha and Santoso, 2019). According to the media equation theory (Reeves and Nass, 1996) and the social agent theory (Moreno and Mayer, 2000), computer-based events are interpreted as social events. Social cues in instructional media environments (e.g., a photorealistic pedagogical agent) prime a social activation schema in the same way that the social processes of human-to-human communication are triggered. Thus, deep cognitive processes, such as selecting, organizing, and integrating relevant verbal and visual information into a coherent mental representation, are enhanced because learners interpret the virtual agent not only as an abstract source of information but also as a partner in social communication (Mayer, 2001). Here, the possibilities and the potential of photogrammetry become clear. The more realistic features that can be implemented, the more social cues can be displayed adequately. Empirical support can be found in recent publications in the field of virtual agents. Realistic 3D characters can be more effective for learning than simple 2D agents (Schroeder et al., 2013). According to Heidig and Clarebout (2011), the design of the agent can strengthen involvement in learning, as long as current technologies are used for a convincingly realistic creation. For example, Beege et al. (2017) point out that older participants reach higher retention scores when the information is provided by an old agent and vice versa. Photogrammetry can be used to display realistic wrinkles or smooth skin on agents’ faces and may thus play an important role in the generation of pedagogical agents. Future research could investigate whether this degree of realism reinforces the results, or simple social cues (e.g., a cartoon animation) trigger comparable effect sizes. Since the social role design of pedagogical agents enhances students’ motivation and learning behavior due to sociocognitive processes (e.g., Kim and Baylor, 2007), the agent’s clothing is relevant if the agent addresses the learner directly (Beege et al., 2019). Photogrammetry can be used to create realistic clothing in order to provide credible social role models. Additionally, photogrammetry can help designers overcome the uncanny valley effect (see section “Emotional Design”; Mori et al., 2012). In this case, a more compelling human-like value could increase involvement in learning (Schroeder and Traxler, 2017). The presence of realistic and visually credible social entities can thus unfold positive effects on learning, satisfaction, and engagement in online learning environments (Wang and Antonenko, 2017).

Applications

The comparably easy creation of PR3DMs with photogrammetry opens new avenues for various existing educational approaches. However, in the fields of education and psychology, photogrammetry is only slowly starting to gain traction. The following existing and conceivable applications might be promising starting points for applications, potentially benefiting from the previously discussed psychological perspectives.

Preservation and Museums

Apart from scientific research (e.g., computer science) and specialized fields (e.g., cartography), a wider range of potential uses can be identified in preservation and education in museums (e.g., Kersten et al., 2017; Kersten, 2018; Agnello et al., 2019). In the field of preservation or archeology, the measurement with photography provides special benefits that lead to its frequent application. For example, visual data can be gathered from almost everywhere, from underwater (Drap et al., 2007) to distant planets (Blasius, 1973). Thus, PR3DMs from almost every type of object can be created and subsequently preserved. As photography itself has been established even longer than photogrammetry, image archives can also be used as important data sources (Goesele et al., 2007). Thus, the creation of PR3DMs is feasible from objects that might not even exist anymore or are inaccessible (Kyriakaki et al., 2014; Vincent et al., 2015; Curry, 2019; The Arc/k Project, 2019). These benefits apply to museums in their tasks of preserving and presenting exhibits. Digitization also represents an important strategy to increase the exhibitions’ visibility and accessibility (e.g., museu4mpunkt0, 2019). In Germany alone, over 600 institutions already digitally share some of their materials (museum-digital, 2019). However, sharing digitized exhibits might be a challenging and resource-intensive task, especially for smaller institutions. Whereas larger museums might be capable of using advanced technologies, such as laser scanning to create digital content (e.g., Staatliche Museen zu Berlin, 2019), smaller institutions might need simpler approaches, such as photogrammetry with digital photography. The resulting PR3DMs can also be used to compare different exhibits (Dall’Asta et al., 2016), support reconstruction (Bujakiewicz et al., 2006), or produce duplicates via 3D printing (Ballarin et al., 2018).

Generative Learning in Classrooms

In addition to merely viewing 3D models or scenes generated using photogrammetry, the generative aspect of photogrammetry in the context of instruction could be a major focus of future research and applications. Numerous empirical investigations have shown that students embrace 3D-printed materials as learning resources (e.g., Li et al., 2018; Mogali et al., 2018), particularly when they are involved in creating these models (Backhouse et al., 2019). In addition, such printed models could be accompanied by digital applications, such as augmented reality (Dewitz et al., 2019). Similarly, physical props can be used as useful learning metaphors (Astrachan, 1998; Bragdon et al., 2010), supporting knowledge acquisition and skill transfer. Likewise, photogrammetry provides an opportunity for learners to engage with an object during the process of creating a digitized model as a form of generative learning (for an overview, see Fiorella and Mayer, 2016). Instead of passively viewing and learning from prefabricated materials, studies have provided evidence for the claim that letting learners create materials themselves can enhance their learning performance (Fiorella and Kuhlmann, 2019; Hoogerheide et al., 2019). However, some studies indicate that self-generating learning materials may produce unwanted cognitive load and can thus impede learning (Stull and Mayer, 2007). It is essential that photogrammetry applications become user friendly enough so that learners can generate their own 3D models with little or no supervision while being able to focus on the object and the model rather than on the demands imposed by the user interface. Additionally, pretraining could enhance learning outcomes, similar to the effect of such interventions on the learning gains using virtual reality (Meyer et al., 2019).

Research on Educational Videogames

Digital content creation can be a major challenge for videogame researchers because not every psychologist or educator is a skilled game designer and vice versa. This challenge holds true in the field of educational videogames. Researchers have to investigate learning mechanisms in a complex medium that potentially comprises a wide range of virtual environments and objects. Experimental research regularly follows a case-study approach with finished products to avoid complex content creation, as this significantly reduces the technical challenge. However, researchers state that a more detailed investigation with variations of individual facets is needed to validate different design approaches (Clark et al., 2016; Ke, 2016; Nebel, 2017). For workarounds, researchers have used games that are highly modifiable (Loh and Byun, 2009), include simplified production tools (Williams-Pierce, 2016), or focus on gameplay where the players have to create the content themselves (Nebel et al., 2016). These approaches reach their limits if the included assets are insufficient to realize the desired learning outcome or answer the research question. To fill this gap, the described process of photogrammetry could be used to create the necessary content, especially in combination with increasingly accessible game engines (e.g., Epic Games, 2018). This might empower the research on many topics that require modifications of the included virtual objects, particularly concerning the question of how realistic visualizations might influence the learning process. This topic is gaining importance because a substantial difference remains between PR3DMs and what has been considered realistic in meta-analytical investigations on educational videogames (Wouters et al., 2013; Clark et al., 2016).

Future of Photogrammetry in Education

The previous sections have highlighted the creation, processing, and applications of PR3DMs created via photogrammetry. However, to establish this approach, the research gaps need to be filled, technological skills should be acquired by instructors, and unique educational potentials ought to be discussed.

Lack of Research

A substantial lack of research suggests a major challenge for the use of photogrammetry in education. In contrast to other fields, such as cartography (Wilkening and Fabrikant, 2011), landscape visualization (Lange, 2001), virtual avatars (Latoschik et al., 2017), cognition (Lokka and Çöltekin, 2016), or the concept of presence (Cummings and Bailenson, 2016; van Gisbergen et al., 2019), research on realistic virtual objects in the field of instructional psychology is comparably scarce. Another challenge arises from the questionable transferability of the insights gained from related fields. Previous research on realism has most frequently used stimuli that noticeably differ from photorealistic materials, and these studies have often focused on fundamental processes, such as navigation or perception (e.g., Lokka and Çöltekin, 2017). Research that includes educational content is conducted less frequently. Therefore, it is not clear if research claiming positive effects on performance (Lessels and Ruddle, 2005; Slater et al., 1996) and on behavioral variables (Slater et al., 2009) or offering inconsistent insights (e.g., Hoffman et al., 2001; Zimmons and Panter, 2003; Mania et al., 2006; Yu et al., 2012; Lee et al., 2013) can be transferred to educational settings. Additionally, variables are often combined (e.g., shadows and textures; van Gisbergen et al., 2019), significantly increasing the challenge for researchers to infer specific design recommendations. Overall, this lack of dedicated empirical evidence limits the applications of PR3DMs in education because many considerations regarding the realism of virtual objects in educational settings remain speculative until empirically validated. This caution should be applied to the approaches and the considerations proposed in see section “Photogrammetry From the Perspective of Psychology,” such as emotional design or social aspects using PR3DMs.

Technical Developments and Accessability

As previously stated, the technical and mathematical foundations of photogrammetry are not entirely new, but the development in creating digitized duplicates has reached a crucial point, where even technical novices using non-professional hardware can accomplish tremendous accuracy (Figure 5). In particular, 3D creation processes, such as texture mapping, can efficiently compensate for insufficient geometric details (Rushmeier et al., 2000). The required software is also becoming more widespread (e.g., 3Dflow, 2019; Capturing Reality, 2019), and more educators can explore this approach without substantial technical or financial investments. Photogrammetry can even enable the creation of PR3DMs using a web-based image or video search (Weis, 2019).

FIGURE 5

Figure 5. A comparison of the physical object (left segment of the images) and the PR3DM (right segment of the images); no further enhancements (e.g., reflections) are used for this PR3DM rendering. Screenshots are taken in 3DF Zephyr Lite (3Dflow, 2019).

The process described in Section “Technical Foundation” indicates some subprocesses that need additional attention and are not yet fully automated in the available software. For example, the outlined phenomena that are difficult to measure with photography (e.g., reflections, transparencies) need to be applied with additional editing tools. Although available for free (e.g., Blender Foundation, 2019), such software needs considerable time to master. Similarly, the optimization of the mesh (e.g., simplifying, closing, and cleaning models) or the textures (e.g., color correction, manipulation of advanced features, such as normal mapping) can be achieved in dedicated software (e.g., Cignoni et al., 2008) that might require additional time to learn. However, it should be noted that these processes might not be necessary for creating typical PR3DMs, depending on the material and the complexity of the physical object. Altogether, further documentation of workflows (e.g., Fritsch, 2003; Orr et al., 2020) and deeper discussions about potentials, with a clear focus on educational settings, are needed.

Impact on Behavioral and Emotional Outcomes

Case studies focusing on the measuring aspect of the process might represent the majority of the existing experimental research using photogrammetry. If the learning material is used in virtual environments, factual knowledge is most commonly taught. This can be classified as basic knowledge (Bloom and Krathwohl, 1956) or as a low-level element of the information knowledge domain (Marzano and Kendall, 2007). For example, in medical studies, anatomical knowledge is tested (e.g., Nicholson et al., 2006). However, there is a lack of research on what perception of and interaction with PR3DMs might offer for different contexts. For example, learning settings relying on 3D models or immersive experiences created using photogrammetry may result in more realistic behavior (e.g., Dede, 2009) and could thus be more effective for a variety of training environments. Media psychology research has begun to investigate such effects of different types of virtual environments on emotional or behavioral variables (Zaalberg and Midden, 2010; Chirico and Gaggioli, 2019). In this vein, virtual environments are used for educational purposes that cannot be easily classified using established learning taxonomies, such as training in public speaking (Poeschl, 2017), controlling anxiety disorders (Krijn et al., 2004), treating post-traumatic stress disorder (Botella et al., 2015), managing fears and phobias (Hodges et al., 1995), or decreasing prejudice against specific groups (Stelzmann et al., 2019). It is not yet clear how PR3DMs could support such approaches. For example, would the impact of a virtual environment trying to illustrate the dangers of being a refugee be strengthened by using PR3DMs? Following the learning taxonomy of Merrill and Twitchell (1994), this could impact the emotional component as learners might react differently to related stimuli after interacting with the PR3DMs. This is in line with the discussion about naïve realism (Smallman and St John, 2005) and similar statements that realism has to serve a clear purpose (Paes and Irizarry, 2019), as even unrealistic representations can be sufficient to be correctly visually identified and processed (Mourkoussis et al., 2010). Therefore, the real potential of photogrammetry might not be observable if simple factual knowledge is the key component (see the following section for a discussion of Dwyer, 1969). Instead, in line with the presented psychological perspective, PR3DMs might be more useful if higher level learning outcomes (Bloom and Krathwohl, 1956; Marzano and Kendall, 2007) or emotional (Merrill and Twitchell, 1994) and behavioral outcomes are targeted. To shed more light on this, the creation of a taxonomy of learning gains through the use of PR3DMs might be an important task in the future, as soon as sufficient empirical studies are available.

Addressing Specific Learning Outcomes

We wish to emphasize that photogrammetry (and realism in general) is no cure-all for education. As we have summarized, there can be both positive and negative effects when using photorealistic visuals in educational settings. From older studies by Dwyer, we can learn that the benefits of realism depend on the type of knowledge in question. For instance, Dwyer (1969) found that more realistic visualizations in an anatomy learning task result in better results for learning tests that involve the visual recognition of anatomical elements, while the performance in learning tests that do not require fine-grained visual knowledge, such as vocabulary tests, can profit from the simplicity of line drawings (or even the absence of any visuals). As a result, more consideration should be given to the type of knowledge that educators wish to foster in learners. Although there are instances in which knowledge should be simplified to allow learners to grasp the essential information, there are several scenarios in which a high level of realism has been shown to be advantageous for learning as described in see section “Photogrammetry From the Perspective of Psychology.” For these scenarios, photogrammetry is a straight-forward, low-cost, and widely available technique that provides a resolution comparable to more expensive methods of digitization and thus empowers instructional designers to integrate cutting-edge visuals into their learning content. In summary, photorealism is not always desirable in learning materials, but if the exact reproduction of objects for learning is an important component of a learning task or might even enable new learning outcomes, photogrammetry currently is one of the best ways to generate PR3DMs.

Conclusion

Overall, this paper has illustrated a technological approach capable of producing photorealistic virtual objects and environments. Although not as established as other forms of learning with multimedia materials (e.g., images, Schneider et al., 2016; videos, Hoogerheide et al., 2019), the technology might reach a similar point in the near future. To foster this development, researchers should explore the potentials and the open questions discussed in this paper. Thus, the presented psychological perspective provides a crucial theoretical and empirical structure to explore the most beneficial applications of PR3DMs. This perspective could be extended by other perspectives, such as the broader field of computer graphics or skill transfer research, to gradually create a complete overview of the potential and challenges while working with PR3DMs. The outlined theoretical and practical considerations can be used to inspire more nuanced research, exploring the interaction between the effects on mental load through potentially more complex cognitive processing, potentially increased emotional or behavioral outcomes, and how such material could be connected to traditional or additional learning material (Coughenour et al., 2015). Research itself could benefit from the facilitated method of content production, and researchers addressing realistic digital materials might be capable of using PR3DMs as stimuli. Finally, this review has addressed over 50 years of research in diverse fields. The standard of realism achieved with photogrammetry was probably unimaginable in the earlier works. Thus, insights and theoretical considerations should be updated to reflect the current technical possibilities. Naturally, this presents a continuous (and stimulating) task for researchers.

Author Contributions

SN wrote major parts of the manuscript. MB, SS, and GR wrote segments of the psychological perspective and proofreading. All authors contributed to the article and approved the submitted version.

Funding

The publication of this article was funded by Chemnitz University of Technology.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We want to thank Alexander Skulmowski for his input and criticism. This was very useful to us to improve the manuscript.

References

3Dflow (2019). 3DF Zephyr. Available online at: https://www.3dflow.net/ (accessed January 28, 2020).

Google Scholar

Abbaszadeh, S., and Rastiveis, H. (2017). “A comparison of close-range photogrammetry using a non-professional camerawith field surveying for vplume estimation,” in ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-4/W4, (Heipke: International Society for Photogrammetry and Remote Sensing), 1–4. doi: 10.5194/isprs-archives-XLII-4-W4-1-2017