Immersive heritage through Gaussian Splatting: a new visual aesthetic for reality capture

This paper investigates the emerging use of 3D generative artificial intelligence (GenAI) models, particularly Gaussian Splatting (GS), for creating immersive virtual environments in architectural heritage contexts. While traditional methods like photogrammetry have long been used to replicate historical sites, advances in AI-driven tools such as Neural Radiance Fields (NeRFs) and GS have significantly broadened accessibility and creativity in 3D reality capture. Though both technologies have been explored previously for heritage conservation, their usage has always been discussed through the lens of accuracy and precision. The paper challenges these traditional notions and follows a phenomenological methodology that analyses the visual aesthetics and image-making techniques of GS from theoretical and philosophical standpoints. Through the lens of architectural and media theory, the research assesses how GS can transform historical sites into immersive experiences by combining photorealism with artistic expression. The approach is twofold: first, analyzing the visual features and characteristics of GS by documenting, representing, and experiencing a real-life heritage site (The Gatekeeper’s Cottage) through immersive mediums. The results are captured as video recordings and are examined in the results section of the paper. Secondly, the paper includes an in-depth overview of how digital artists use GS technology to evoke memory and atmosphere in virtual heritage spaces through narrative building, VFX, and post-production. The purpose of the study is to offer architects, historians, and heritage professionals a new aesthetic perspective on employing AI-driven models for heritage visualization and immersive environments, that challenge the traditional notions of accuracy and question how these techniques can reveal previously hidden or unobservable architectural features in immersive heritage environments.

1 Introduction

In media and architecture literature, there is an extensive discourse on creating immersive virtual environments that transform physical surroundings into alternate realities for historical experiences. Erik Champion, in Playing with the Past: Into the Future, explores how technologies like Virtual Reality (VR) enhance cultural learning by fostering presence and immersion in historical narratives (Champion, 2022). Marc Aurel Schnabel emphasizes the role of virtual historical environments in deepening our understanding of space and memory, while Andrew Yip introduces the concept of “immersive aesthetics” to generate visual, auditory, and biomechanical stimuli for enhanced sense-making (Schnabel and Rushton, 2022; Yip, 2020).

Amidst the intriguing discourse and practice surrounding immersive heritage environments, another emerging dimension is gaining momentum; the art and philosophy of immersive reality capture through 3D generative artificial intelligence (GenAI) algorithmic models. While reality capture through software and photogrammetry is not new, AI models have significantly democratized the process, making photorealistic 3D creation accessible on consumer hardware for mass creative production. Neural Radiance Fields (NeRFs), introduced by Mildenhall et al. (2020) marked a breakthrough in easy reality capture via mobile devices. Recently, Gaussian Splatting (GSplat or GS), introduced by Kerbl et al. (2023), has rapidly advanced, allowing researchers and artists to capture complex architectural and heritage spaces using mobile applications like LUMA.ai. Current research in architecture and heritage has increasingly adopted NeRFs and Gaussian Splatting for creating 3D virtual historical environments, showcasing their potential as successors to traditional photogrammetry (Palestini et al., 2022; Barazzetti et al., 2023; Mazzacca et al., 2023; Basso et al., 2024; Burdziakowski et al., 2024; Wang et al., 2024). These studies focus on accurate representations, photorealism, and reality replication, with only a few exploring GS as a visual method for preserving memories in immersive digital experiences (Reinhuber et al., 2024).

Simultaneously, this technology is gaining recognition for creating immersive heritage environments in digital arts, film, and gaming. Artists like Ruben Frosali are pushing the boundaries of 3D GenAI technologies, using them creatively to redefine immersion, capturing environments, moments, and memories lost in time (Rubin et al., 2022). By combining AI reality capture techniques with new media tools like gaming engines and VFX software, artists can transform historic architectural realities in innovative ways. It is evident that current practices in GS imaging and immersive environments are evolving beyond architecture and heritage, as designers, artists, and visualizers use novel AI tools to create imaginative alternate realities that push the boundaries of traditional photogrammetric “reality capture.”

This opens an opportunity for the visual examination of Gaussian Splatting through a philosophical, theoretical, and aesthetic standpoint, particularly in its potential to visually immerse audiences and evoke creative and artistic immersive heritage experiences. The advancement of 3D Gaussian Splatting (3D GS), including innovations like BiGS and GS-ID, showcases its expanding potential beyond precision, through adjustments in surface rendering, materiality, and illumination (Du et al., 2024; Liu et al., 2024). This article dissects these visual elements of Gaussian Splatting as a new approach to exploring immersive heritage environments. It challenges the traditional focus on accurate depictions in cultural heritage studies by analyzing the Gaussian Splat model as a visual artifact from practice and theory. This involves exploring the visual aesthetics of 3D GS by constructing a virtual representation of a real-life heritage site and analyzing it through architectural and media theory. This experiment involves extracting visual results by developing a GS model of the heritage structure, bringing it into Unreal Engine, and analyzing the results through immersive mediums. Additionally, the research analyses the contemporary practice of 3D GS in digital arts and media by examining the works of digital artists (Figure 1). It investigates how these creators leverage this technology to creatively convey heritage narratives and experiences through GS in an immersive and engaging way. The results of both the theoretical analysis of the experiment and the contemporary practices are discussed in the conclusion.

Figure 1

Figure 1. Article structure.

The purpose of the study is to provide architects, historians, and experts in the heritage field with a new theoretical perspective and methodology for approaching immersive heritage environments using 3D GenAI. Examining the spatial aesthetics of Gaussian Splatting for heritage virtual production from both theoretical and practical perspectives adds a novel dimension to this research. This article analyses the visual components, features, characteristics, and limitations of the GS model from both theoretical and practical viewpoints and clarifies how these aspects contribute to enhancing the immersive experience of virtual heritage spaces. How does a GS heritage setting visually differ from traditional reality capture methods? Does it reveal any hidden architectural visual features or atmospheric phenomena that were previously unobservable?

2 Methods

This research employs a theoretical framework to examine the visual impact of Gaussian Splatting in virtual architectural heritage environments (Figure 1). The approach is twofold: first, integrating GS into a gaming engine workflow to create and experience a real-life heritage site (The Gatekeeper’s Cottage) through immersive mediums. The results of the experiment are theoretically analyzed through architecture and media philosophy. The visual elements of GS are examined in this analysis through the lens of image-making and aesthetics, drawn from architect Michael Young’s book Reality Modeled After Images: Architecture and Aesthetics after the Digital Image. Young’s work, which explores the relationship between architecture and contemporary image culture, provides a theoretical framework for understanding how architectural images are crafted and their influence on perceptions of reality, history, and virtual environments. Documentation of the experiment workflow and the analysis resulted in a series of video recordings which are embedded in this paper as hyperlinks and discussed.

The second method is a comprehensive review and analysis of the contemporary practice of GS in digital arts and animation. The review focuses on the underlying motives and impacts of the digital practice of GS in media. The theoretical analysis and review incorporate phenomenological concepts related to “immersion,” including virtuality, realism, and memory, drawing insights from architects, theoreticians, and media philosophers to frame the discussion.

2.1 Framework: workflow for the Gatekeeper’s Cottage (Video 1)

The Gatekeeper’s Cottage, crafted by Joseph Reed in the 1860s stands as one of the earliest structures in Melbourne and is a fine example of early Victorian Gothic Architecture style. This research focuses on creating a 3D GS virtual construction of the Gatekeeper’s Cottage. The workflow is video recorded and is divided into three stages: Documentation, Representation, and Dissemination according to Alonzo Addison’s model of virtual heritage (Addison, 2000). This video suggests a new aesthetic paradigm for presenting the workflow as a “design in action” of trial and error rather than a structural process (Video 1).

The framework for capturing the Cottage begins with video recording using an Insta360 ONE RS 1-Inch 360 camera, divided into two stages: the interior (entrance and chambers) and the exterior. The Cottage is recorded in both sunny and overcast conditions to analyze the effect of lighting on the GS environment’s digital atmosphere. Gaussian models are created using Postshot software, which generates 3D radiance from images and videos. Multiple models are developed with varying configurations in image count, iterations, and Splat density. Based on image fidelity and lighting, one model from each category (interior and exterior) is selected for post-production and theoretical analysis. The last phase is creating an interactive and immersive dialogue between physical and virtual worlds using immersive mediums. This included importing the GS model into Unreal Engine, navigating the space as an avatar, and utilizing the Oculus Meta Quest 3 which allowed researchers to traverse the Cottage in a fully immersive way (Figure 2). The theoretical analysis of these experiences is organized into subsections, each focusing on a particular visual characteristic of GS within immersive heritage environments.

Figure 2

Figure 2. Workflow for the Gatekeeper’s Cottage– screenshots from Video 1.

3 Theoretical analysis of the Gatekeeper’s Cottage

3.1 The glitch aesthetic (Video 2)

Cultural historian Peter Krapp in his book Noise Channels profiles a digital culture that goes against the grain of efficiency and accuracy and embraces the information that resides in computational glitches and errors (Krapp, 2011). He talks about the productive use and creative potential of “unexpected” information that is generated during the conventional tightly controlled computational interface. The use of glitch aesthetics in design and media is not a new concept. Krapp gives an example of video games that are coded for potential deviations and alterations which are an essential part of the user’s “learning” for navigation. The “trial and error” approach can uncover unexpected or hidden features within the gaming interface, often resulting in innovative outcomes.

Film-maker and videographer Olli Huttunen, in his experiments with GS, recommends capturing 3D scans under overcast conditions, as fluctuating sunlight and shadows can alter the image capture results (Huttunen, 2024b). Inconsistent lighting in videos can hinder the training algorithm’s ability to accurately pinpoint locations, potentially leading to what he calls “false imagery.” Many human and environmental factors generate this error including the sun flares reflecting off the camera lens can disrupt the algorithm, causing “floating artifacts.” These artifacts are visual “errors” in 3D space formed by the AI code, creating hues and translucency that hinder human vision. These distortions in images can be claimed to be a manifestation of genuine software aesthetics (Young, 2022).

While the “floating artifacts” in splats may be problematic for those seeking precise modeling, it reveals an intriguing phenomenon of overlapping changes that become apparent only when the viewing angle is shifted (Video 2). This dynamic interplay of changing shadows occurs because the Gaussian algorithm captures lighting from overlapping images taken from different angles. For example, when navigating through the Gaussian Cottage interface, the front façade wall and window appear to change depending on the viewing angle (Figure 3). This difference is due to the change in natural lighting conditions during the time the two videos were created, which is embedded in the final rendered representation. The unique effect of the splat code defines the aesthetic characteristic of the model to be seen in multiple ways and showcases the hyper-dynamic feature of each splat. The color, texture, and translucency of each splat fluctuate in real-time, unlike the conventional point cloud approach where each point is encoded with an RGB value.

Figure 3

Figure 3. Changes in lighting, textures, and niche detail in real-time (from left to right) as the camera shifts from the base of the Cottage to the middle focusing on the front window.

The visual glitch becomes evident when it is experienced in Unreal Engine (Video 5). The entire gaming industry is based upon significant room for error – to an extent, this playful sense of potential deviations is an essential part of the gaming plot and audience immersion. Games tend to obey certain sets of rules, which are meant to be broken for users to learn and reveal new aspects (Krapp, 2011). The Gaussians tend to change, fluctuate, and reshape as the avatar navigates through the radiance. This not only redefines the “glitch in video games” paradigm but also provides a new opportunity for game developers to incorporate the fluctuating visual cues of splats in the gaming sequences.

Within the constructed ruins of glitches, new possibilities, and new meanings arise (Jones, 2022). This means that a new way of producing and imagining the world emerges from breaking and distorting the interface. Gaussian Splatting operates at the intersection of precision and abstraction, embodying a dual aesthetic that is both mathematically rigorous and visually expressive. Another characteristic that isolates them from their traditional counterparts is the creation of forms that are defined by probabilistic distributions rather than hard edges, introducing an element of softness and fluidity into the architectural image. This blurring of boundaries challenges the traditional dichotomy between the clear, precise lines of architectural drawings and gives the final model its “impressionistic” and “game-like” qualities of artistic renderings. Architect Austin Pahl describes the Gaussian Splat model as “an impressionist painting where each radiance field feels like a brush stroke and the full scene comes together when you gaze at it from far away” (Pahl, 2023). It suggests a new aesthetic paradigm where architecture is conceived not as a set of rigid structures, but as subtle dynamic fields of potentiality—forms that emerge gradually from the interplay of forces, much like the natural processes they often seek to emulate.

3.2 Virtual vs. actual

The commercial products of digital image capture are commonly associated with high resolution, faultless, “smooth and clean” imagery but the generative AI algorithm presents another, perhaps deeper, perspective on this set of technologies. The glitch moves between modes of affective and cognitive interaction—the “actual” and the “virtual”, “truth” and “wrongness” (Young, 2022). So how is Gaussian Splatting capable of depicting or visualizing the actuality or “truth” if it is constantly forming visual glitches?

This question can be examined through Timothy Barker’s essay on the Aesthetics of the Error. He initiates by discussing how certain artists rejected the traditional idea of fully controlling their creative process, instead allowing the machine or the medium to influence the outcome (Barker, 2011). Relying on the unpredictive nature of the machine algorithm to actuate a particular function. The “virtual”, concerning such interactive art, may be thought of as a field of conditions, imposed by both the internal programming and limitations of the computer as well as the processes initiated by a human user to generate unforeseen, and perhaps unwanted, information (Barker, 2011). Barker sees these fluctuating errors as potential virtual(s) that seek to be actualized. Gaussian Splatting may be criticized for the way splats continuously glitch their gradients and parameters based on the spectator’s viewing angle, rather than reflecting the actual scene. This error is inherent to the code; it is an error that the spectator causes by the act of looking at or accessing the image (Barker, 2011). The results are not definitive, but probabilistic, offering multiple interpretations and perspectives depending upon the spectator’s viewpoint (Video 2). The interpretation of the “actual” is surrounded by dynamic virtual opportunities. Every time a model is rotated, it becomes its own original; every time an error from the field of the virtual is actualized, the unforeseen emerges (Figure 4). In the context of architecture, this aesthetic suggests a move away from the deterministic, towards a more open-ended approach to design—one that acknowledges the unpredictability of the built environment.

Figure 4

Figure 4. Gaussian Splat model fluctuates between virtual and real by changing translucency and materiality in real-time with camera movement.

In the same context, areas that are not captured by the camera are hypothesized by the algorithm as extended splats. Since the algorithm is not able to justify how the area looks based on input data, it speculates. It does not leave the space empty or broken but instead, it allows the neighboring splats to stretch and fill the space with a similar color or lighting pattern. This effect results in spaces that remain fuzzy, unclear, and lack depth (Huttunen, 2024b). However, stating that this happened only because of a lack of data may be superficial since there is more information to unpack. This virtuality of the splat error is connected to the sparse point cloud that lies under the Gaussians. The underlying blanket of point clouds dictates how the splats are distributed, and which spaces can be visualized better thus reflecting what is “actual” and what the algorithm considers “virtual”.

3.3 Intangible noise (Video 3)

Splat radiance does not have geometry. While some researchers have tried to convert radiance into mesh, this representation complicates the rendering of volumetric effects and diminishes the model’s visual appeal by attempting to make something non-tactile into a physical object (Guédon and Lepetit, 2023; Waczyńska et al., 2024). The visual appeal of splats stems from their point cloud structure, which represents a three-dimensional model of millions of light points, creating a painterly depiction of space (Pahl, 2023). How does then one work with these millions of individual points? Most architectural representations store and manipulate lines to control the edges of surfaces (Young, 2022). Polygons provide a tactile quality to the designers that allow them to control and manipulate the hard edges to conform to the “reality”. Splat point cloud models, which consist of gradient color densities, are in many ways closer to the painting than to linear drawing, with sharp edges. Edges and corners in a GS model scumble and fray, holding no special importance in the field of scanned information. This complexity in the radiance point cloud makes these models challenging for architects to work with.

To design within this realm, architecture must develop techniques for working with electrical signals. This means possessing the ability to work with thresholds, with algorithms that compute local variations in pixel information. This requires adjustments in how a designer understands and makes decisions with these mediations. For instance, errant points are not necessarily errors, but simply the energy of the environment captured regardless of its meaning for humans (Young, 2022). The effects created by the dispersed repetitive patterns of the surrounding environment can be compared to the ways decoration diffuses attention, they are akin to “noise.” Krapp states that the presence of noise in a communication channel does not necessarily have to disrupt the message; instead, it can integrate itself into the message itself (Krapp, 2011). This “noise” can be considered a kind of realism that has yet to be fully engaged. The implications of this aesthetic shift become clearer when attention is directed to other artifacts of the scan. The white ghosts appearing and disappearing on the pitch of the gabled roof are the overlapped “shadows” of multiple scanning passes (Figure 4). What the mediation of the scan reveals, however, is that these gaps exist everywhere, they are created through the interference of matter and energy, a shadow floating around the roof is equivalent to a shadow behind a scanned surface.

If there are clear gaps in the point cloud and few reference points, then the area remains mostly noisy, smudged, or fuzzy and the splats have no clear information about how they should be in the 3D space. Therefore, the point cloud works like a “gauge” which is then filled with splats of different sizes (Huttunen, 2024b). So, if the “gauge” is not dense enough, the splats are not able to be filled inside properly. In the Cottage point cloud, the walls exposed to direct sunlight exhibit a complex and dense point cloud as compared to the shadowed areas (Video 3). Darker spaces tend to hide and the Postshot algorithm cannot detect them if the spaces are not bright enough. Similarly, areas not captured by the 360-camera remained largely undefined.

For Young, these empty spaces in the visualization are the “representation of depth” and correspond to the poche’ technique of manual architectural rendering (Young, 2022). Poché is understood traditionally as an abstract cut in orthographic drawings that conceals the physical labor of construction within its graphical abstractions. Poché contains hidden features of the architectural image constructed by human or non-human labor. In digital scanning or photogrammetry poche’ appears in the empty zones which the Postshot algorithm is unable to “see” or visualize (Figure 5). Akin to traditional NeRFS that form large clouds of unprocessed radiance around the objects, GS leaves the unrecognized space blacked out (Computerphile, 2024). Initially, this empty zone is recognized as the absence of data, but what is truly registered here is all the informal undesigned irregularities of reality. Such blind zones can also be described as shadows invisible to data collection, dark pockets looming on the other side of the scan (Young, 2022). Ghostly noise and empty black zones are all representations of poche’ interpreted as abstraction cuts or shadows of the artifact which are usually treated as outside the concern of the architect. However, Young argues that the ontological status of these non-sensical representations is as real as the visible veil of the surface information even if it is not illuminated by the human sensorium (Young, 2022).

Figure 5

Figure 5. Point cloud particle frame of the Cottage’s exterior and interior Gaussian Splat models. Images show the influence points of splat due to light presence while darker spaces with less splat concentration – screenshots from Video 3.

3.4 Aesthetics of simulation

Young refers to aesthetics as how qualitative information is made sensible and delivered to the senses (Young, 2022). The aesthetics of digital images are heavily influenced by software and the algorithms used in their production. His analysis of how software shapes digital aesthetics can be extended to include generative AI algorithms. The visual aesthetics of a splat model is fundamentally shaped by the algorithm as well as the software used for its simulation. “The AI aesthetics” introduces a new dimension to digital images, where the creative process is shared between human designers and AI algorithms, each contributing to cultural diversity (Manovich, 2020).

There are various approaches being explored in academia for AI inclusion in media including high-performance gaming engines. There is a huge step up in visual simulation that can be observed from the rather Grand theft auto-GTA looking photogrammetry pipeline of yesteryear to the NeRF-Unreal engine workflow of Unrecord. It is not only capturing a static place and reframing it infinitely but doing legitimate performance capture of dynamic entities in the world which can be endlessly manipulated further in post-production. Gaussian Splatting, when plugged in with gaming software, blurs the line between the real and the virtual.¹ In digital environments, images can simulate reality with a high degree of fidelity, but they can also create entirely new, imagined worlds. This raises the concern of whether the simulated environments depict the true authenticity of the heritage environment or whether is it a mere spectacle.

The idea of simulations and reproduction of the original is a well-discussed topic in both early and contemporary media philosophy. One of Walter Benjamin’s central concepts is the aura, which he describes as the unique presence and authenticity that an original work of art possesses (Benjamin, 1936). The aura is tied to the artwork’s singular existence in time and space, its history, and its connection to tradition. Benjamin argues that the original artifact loses its essence once the process of reproduction initiates — the original versus the copy. In contrast, contemporary media theorist Lev Manovich’s approach to aesthetic philosophy emphasizes the use of computer science methods to evoke and convey emotional responses towards artifacts (Manovich, 2023). He believes that the simulations that resemble something from history are more aesthetically appealing to the public because of their affiliation to the culture. This ability to simulate and manipulate reality introduces a new aesthetic dimension where the boundaries between the authentic and the artificial are fluid.

The visual output of the Gaussian image is determined by simulating parameters set within the algorithm. These parameters can control aspects such as splat shape, color, texture, and motion, leading to a wide range of possible visual outcomes. Gaussian Splatting can be seen as an aesthetic of emergence, where architectural forms are not pre-determined but arise from the interaction of these underlying data points. The model also explores the effects of bracketing the input image set to the fewest possible source images, thus reducing the amount of overlap in the depth construction. For example, the brick surfaces of the Cottage become highly realistic in certain areas and more abstract in others. Some parts appear weathered and rusticated, others seem almost like burlap or denim (Figure 6).

Figure 6

Figure 6. Gatekeeper’s Cottage Gaussian Splat model simulated in Unreal Engine with an artificial environment.

The aesthetics of simulation in architecture involve creating spaces that are hyperreal and designed to evoke a certain image, mood, or narrative. This is done by manipulating the visual parameters of the imported GS model (playing with materiality, light, and shows; by creating artificial environments) by the creator to simulate an immersive experience. Importing the Gaussian model in the Unreal Engine for VFX and environment modeling creates a version of the heritage space. It copies the original artifact, but it manipulates it to create a new reality. Even though the Cottage still retains its original value, it is reduced to a mere representation of the heritage and loses its aura. The gaming environment does not seem to do justice to the authenticity of reality. Either the environment is matched with full accuracy to preserve its aura or a creative approach is taken to shape new forms of reality within similar parameters.

3.5 Materiality, light and shadow (Video 4)

In digitally mediated environments, there is always a loss of space that manifests as gaps in scans or empty or broken geometries (Young, 2022). In the case of Gaussian Splatting, this manifests as blurred or extended splats in an attempt to hypothesize or “hide” empty zones. When simulated in a gaming engine, they appear as translucent shadows since they lack depth in color and materiality (Video 4). The technique’s emphasis on probability and distribution over fixed, solid forms introduces an aesthetic of immateriality. Architectural images created through this method appear as ephemeral, almost ghostly apparitions rather than concrete structures. This reflects broader philosophical discussions about the nature of digital heritage architecture, where the focus shifts from physical materials to data and algorithms as the primary medium of speculation. Similarly, the atmosphere in which the artifact is video recorded adds to the output radiance (Huttunen, 2024b). The splats recorded for an image that was captured in the direct sun have a different aesthetic appeal than the one captured in overcast weather. The diffused lighting of the cloudy atmosphere helps in the even distribution of light. The model from the direct sun has higher fidelity and splat concentration due to increased lighting conditions which resulted in capturing fine details of the Cottage including the “gothic window mosaic” of the Cottage (Table 1).

Table 1

Table 1. Comparative analysis of lighting effect on two Gaussian Splat models of the Gatekeeper’s Cottage.

Young states that there is a long history of architectural-historical representation that entwines lighting, perspectives, and shadow drawing (Young, 2022). Reality becomes known when the architectural representation of the historic site is simulated with artificial lighting in a digital environment. By using an HDR image, point light, and atmospheric lighting, splats can be re-lighted, resulting in dynamic visual effects. Through sub-surface scattering, the light transports through the skin of the model enhancing spatial realism (Video 4). An artifact viewed from different positions or in different lighting conditions presents very different sets of reflected energy, and thus very different patterns of pixel adjacencies (Figure 7). In the Cottage, if the researcher aims to emphasize the gothic mosaic or markings on the wall, the splats must differentiate between figure and ground to identify edges and features within specific ranges. Introducing a light source into this setup allows the splats to collectively group and be illuminated to highlight the desired area. The shape, size, and intensity with which these groups absorb and reflect light depend on the parameters of the light source (Video 4).

Figure 7

Figure 7. Experiments with point light in Unreal Engine. (A,B) Re-lighting Cottage exterior; (C,D) Re-lighting Cottage interior – screenshots from the Video 4.

The collective response of splats to the light source reflects in its materiality. The extended translucent edges of the splats allow light to pass through them, creating a dark “misty” ambiance around the edges of the scan (Video 4). Young refers to the aesthetics of such surface articulation as Mosaique which complicates the distinctions between reality and representation (Young, 2022). A method to manage depth in image and form of creative expression. Mosaïque concerns how surfaces are rendered, specifically with how attention is directed through an architectural representation (Young, 2022). Lighting, therefore, plays an important role in how the surfaces are rendered and how they impact the audience at an emotional level. By rendering the splat under diverse lighting conditions, new forms of architectural language for historical narratives can be constructed in whimsical, surreal, or other creative forms.

3.6 Immersiveness (Video 5)

Architect Erik Champion asserts that reproducing digital artifacts is not enough to convey the importance of their cultural heritage but requires a designed system of interactive and immersive media for affective communication (Champion, 2015). The immersive forms of metacommunication and gameplay through graphic interfaces see their applications in redefining how we experience cultural heritage. Games function as simulations by enabling both players and spectators to observe changes in behavior and self-revelations unfolding over time (Champion, 2017). The “play” with artifacts, wayfinding, and human-computer interactivity leads to a more successful immersion.

Interactivity becomes the defining component of virtual immersion. Human-computer interaction between the audience and their surroundings makes the virtual scenario more believable and provides the feeling of being inside the story (Stogner, 2010). Video 5 highlights the various interactions of the researcher as a virtual avatar in the gaming heritage scenario. The fragile nature of the Gaussian model becomes apparent as the avatar moves near the Gaussian noise (Video 5). The surface rendering of the splats changes in real-time based on the character’s position. When it comes to human-computer interaction, human gestures and virtual touch become the medium of communication. Unlike polygons, which can be manipulated, splats are non-reactive to virtual touch or gestures and cannot be interacted with or altered like conventional models. This lack of interactivity allows the avatar to pass through walls and other structures in the space. Huttunen notes that while the Gaussian model’s visual fidelity makes it ideal as a gaming background, environments need to be manually built to serve as collision barriers for physical interaction (Huttunen, 2024a). In the Cottage simulation, elements such as flooring and grass were added to allow the avatar to interact with the environment, with the splats’ immateriality symbolizing their unique behavior and the barriers as a revelation (Video 5). This immateriality, however, can be leveraged in gaming as “secret passages,” enhancing narrative immersion. Games often revolve around the idea of learning through failure, and incorporating such glitches in gameplay, along with discovering alternate paths, offers new forms of user embodiment within digitally mediated reality (Krapp, 2011). A brick wall might appear ordinary, but if the character can pass through it, a hidden reality may be revealed.

Visual effect artist and director Ruben Frosali in his practice of integrating Gaussian Splatting into immersive environments, emphasizes the user’s role as an “active” participant in virtual spaces (XR AI Spotlight, 2024). In his artwork, he uses Virtual Reality (VR) to enhance immersion and embodiment, focusing on both visual elements and audience interaction for a cohesive experience. VR stands out among immersive technologies for its ability to offer fully embodied and highly immersive experiences. In the realm of architectural heritage, this approach challenges historians to move beyond traditional methods and create interactive, emotionally engaging representations of history (Champion, 2023). In the case of the Gatekeeper’s Cottage, the Gaussian model is experienced through the Meta Quest 3, initiating a dialogue between virtual and physical by exploring the space at a human scale. Navigating through the surreal environment through VR reveals floating splats as unavoidable visual artifacts, creating the sensation of moving through a disintegrating, hallucinatory world. The high fidelity of certain sections, like the window mosaic, blurs the line between the enhanced gaming engine environment and the actual GS model (Video 5). Although the immersive experience does not meet Champion’s criteria for a successful virtual heritage experience due to its limited interactivity and the lack of elements needed for gestural engagement, the Gaussian model remains effective in representing the heritage environment in an immersive 3D space. The generated model can be used as a virtual backdrop for games and animations or as digital assets within Metaverse environments.² Companies specializing in immersive media, such as Meta and Epic Games, are already working to create their own Metaverse platforms, with the goal of transforming how humans interact with digital environments through virtual communication. Users worldwide can virtually explore immersive heritage environments through VR headsets. The emerging trend in virtual heritage preservation practices and research is also shifting towards enhancing user immersion and interactivity using XR-powered metaverse applications (Buragohain et al., 2024; Innocente et al., 2024). Gaussian Splatting could add an extra layer of immersive depth to these efforts, providing a simpler and more efficient AI-driven approach for integrating heritage assets into games and digital environments, enhancing human engagement with virtual spaces (Figure 8).

Figure 8

Figure 8. Modes of immersion (A) gaming avatar (B) researcher using MetaQuest 3 for virtual reality (VR) – screenshots from the Video 5.

4 Contemporary practices of Gaussian Splatting in media and heritage

In an interview with XR AI Spotlight, Frosali explained the creative practice of capturing architectural spaces in motion for digital animations, videos, and video games using GS. For Frosali, each splat point is like an “atom”, and connecting these atoms with the custom particle effects of compatible software is like playing with “reality” (XR AI Spotlight, 2024). Every physical object is composed of atoms, and Frosali developed a custom workflow that allowed him to deconstruct virtual reality into its atomic components. He could manipulate these particles at the most granular level, and once reassembled, they coalesced into a transformed reality. He uses Unity3D custom shaders and particle workflows and merges them with the “atoms” of Gaussian models to create immersive and almost psychedelic virtual artworks.

The audio-visual AfterLife 2024 event at Paris La Défense Arena by the electric duo “Tale of Us” intended to create a surprise thanks to an extraordinary audio and visual universe, punctuated by “revolutionary lighting effects.” In Dissolving Crater visuals for the event, Frosali effectively merged effective audio and visual art to create an immersive deconstructed digital reality. He made the splat particles audio-reactive using Unity3D which would simulate new visuals according to the sounds. In his Twitter, Frosali mentioned that he used the LUMA.ai application to construct the Gaussian model which begs the question that this easily accessible mass-use tool has the power to create the immersive, mystic, and almost 360 media-engulfing space when used with the right software for post-production (Frosali, 2024).

By integrating splats with third-hand software, artists can unlock the creative potential for visual production. Gaussians work on similar patterns as point clouds as they can diffuse, morph, and transform as individual points rather than polygons which have a defining shape and vertices. The particles can be exploded, diffused, or morphed to form fluid dynamic VFX in virtual environments. The simulated images offer a multiverse of imaginative effects which are a work of art for designers and VFX artists. However, this questions its effect on the virtual heritage production process and how Gaussian Splatting is acting as a new creative medium for experiencing and visualizing historic architectural narratives.

4.1 Memory capture

Crystallizing human memories in virtual space could potentially be the ultimate consumer application for 3D and 4D capture technology. To immortalize physical settings into virtual arenas of exploration. Gaussian spatial scanning is a future-proof medium we have access to today. Saving objects, spaces, and even living beings in the moments of time that one truly cares about forever in the form of digital code that could be relived, and re-embodied is a new form of photography. Apple Inc. industry has already started tapping into this direction of memory capture with their “Spatial Media” initiatives that include 3D photos and videos that are intended to capture moments and can be relived with the Apple Vision Pro headset (Apple Inc., n.d.). These are not static scenes but three-dimensional videos that can be easily captured with an iPhone – an immersive experience that “transports users back to each moment in time” (Spoonauer, 2023). While stereo magnification using multiplane images to construct 3D scenes has long been a focus of research, its relevance is growing rapidly with the advent of 4D generative AI technologies (Khakhulin et al., 2022; Wu et al., 2024). These advancements are driving renewed interest in the field to create more immersive and detailed synthetic environments from historical image data.

Memories of Tsukiji is an immersive art experience by Frosali that explores what was formerly the world’s largest fish market (Frosali, 2019). Frosali and his team recreated the fish market digitally, opting to explore the fragmented nature of photogrammetry’s dense cloud generation instead of pursuing a conventional photorealistic simulation (Rubin et al., 2022). By connecting the point cloud data with Unity’s particle shaders, Frosali manipulated the space enough to form an installation of animated memories. The photorealism is still present to some extent, but the semi-abstract spatial data powerfully conveys a sense of a memory fading over time. In a similar genre, a South Korea-based videographer and visual artist who goes by the code name Crazy Radio created an immersive Collective Memories installation by connecting the Gaussian Splat model with Touchdesigner software. The official website states that the project intended to visualize “the souls of Suho Forest and the old movie theatre… intertwined in different times and spaces associated with the old” (Crazy Radio, 2024). The project explores a new hybrid digital system by dismantling and reconstructing the time, space, and soul of architectural heritage by overlapping multiple layers of Gaussian digital scans and interactive visual effects (Figure 9). The output is more than simple photorealistic visualization but attempts to conceptualize psychedelic or transcendental moments in time as if the physical reality is fading away overtaken by immersive aesthetics. This technique of hypermapping is not new and can be seen in the works of Andrew Yip who discusses immersive aesthetics as a means of generating captivating visual, auditory, and biomechanical stimuli for memory representation (Yip, 2020). He argues that to better employ hardware and software technologies for embodied interaction, media artists need to expand the aesthetics of immersive visualization. He prophesized this aesthetic as the next emerging museological strategy of engaging the senses through an “expanded museum” and archive (Yip, 2020). In the works of Frosali and other media artists, we are seeing Yip’s prophecy coming to life where hypermapping is now employed in 3D GenAI as a strategy for memory-making of historical data.

Figure 9

Figure 9. Collective Memories [Gaussian Splat 3D capture, running in Touchdesigner, video, 3 min 13 s] © Crazy Radio KENCIDER, 2024.

4.2 Reality bending

Digital artists are constantly challenging the notion of realism in immersive heritage environments by designing new mechanisms and workflows to test the potential of AI in imaging the associated narratives. In his first GS VFX experiment Dissolving Realities, Frosali located digital graphics software as a “medium” to illustrate how AI reality capture techniques can be creatively simulated for transforming realities. He explains the video as “Awaken” through the machines; we bend the veil of reality (RubenFro, 2023). He used a splat model of a semi-abandoned medieval village in Italy to create a 1 min VR video of the heritage site, glitching to transform it into a digital hallucination of floating pixels – blending the barrier between real and virtual. This visual effect was possible because of the surrounding noise generated around the scan as discussed previously. This noise may be discarded as an error in code but within these lie the genuine software aesthetics (Krapp, 2011). Each noisy splat was converted into digital code of a pixel that played a role in the transformation of reality (Figure 10). The ability to transform a seemingly ordinary heritage space—often overlooked—into an immersive VR gaming environment with just a quick splat scan highlights the remarkable potential of this tool.

Figure 10

Figure 10. Screenshot of Avatāra – Dissolving the Veil of Reality [Gaussian Splat 3D capture, running in Unity 3D, video, 1 min 4 s] © RubdenFro, 2023.

Creative studio Bad Decisions created a Gaussian Splatting 3D virtual environment of the pyramids of Giza using archival video (Bad Decisions Studio, 2023). They have never personally visited the pyramids, nor have they recorded the video themselves. However, using archival footage, they managed to create and experience the actual heritage site. The model not only included the pyramid but also the surrounding context as floating noise. The context is a pivotal feature of these algorithms and unlike conventional photogrammetry, GS avoids isolating the artifact from its surroundings. This fosters immersion within the digital space by maintaining the integrity of the environment in which the artifact exists. However, in some experiments, the creators manipulated the surrounding noise, almost transforming it into a new narrative. For instance, they did a reality capture of the iconic Burj AlArab off a moving boat and then simulated the GS model in Unreal Engine for post-production, changing its surroundings to make the environment seem like it was outer space. Such artistic simulations preserve the visual integrity of architectural sites while enabling custom software to alter our perception by introducing new spatial features and reimagining reality. However, the increasing sophistication of these simulations may lead to a detachment from the real world, as they often prioritize creating convincing, immersive experiences over accurately representing historical spaces, potentially fostering an appreciation for their artificiality rather than their authenticity. Frosali highlights this issue and says that as much as AI helps designers with visual production, it may be too easy for artists to let AI do all the processing, and designers take the back seat (XR AI Spotlight, 2024). With AI, research about imaginative image-making can quickly pilot unrealistic depictions of history, from photorealism to abstraction. The orientation may lead to an abstract realm of endless loopholes where the viewer and the designer both are left confused and agitated about what to make out of the image (Barker, 2011). The collaboration between humans and technology needs to be mutual. Using AI for creating or testing new workflows as Frosali calls “the recipe” is a part of the creative process.

5 Limitations

While this research focuses on providing aesthetic insights into Gaussian models, it is important to acknowledge some of the study’s limitations and the ethical usage of 3D generative AI for architectural heritage visualization.

One of the limitations of this study is its focus on a single heritage site (Gatekeeper’s Cottage), which restricts the generalizability of the findings. While the use of Gaussian Splatting in this research demonstrates its potential for a new aesthetic paradigm for architectural heritage visualization, the application of this technology to a single case study inherently limits the broader applicability of the conclusions which requires further investigation. The unique characteristics of the site, including its spatial layout, architectural details, lighting, material textures, and environmental conditions, may have influenced the outcomes in ways that are not universally replicable across other heritage sites. For instance, heritage sites with more complex geometries, extensive damage, or varying scales may present additional challenges for GS that were not encountered or addressed in this study. Similarly, sites with intricate ornamentation or culturally specific elements may require tailored adjustments to the model, which were not explored within the scope of this research. The absence of quantitative analysis along with the qualitative investigation of this research may also enrich the research further. However, the aesthetic attributes and the Gaussian artifacts may retain their specific characteristics and can be seen in any 3D visualization.

Gaussian Splatting, as a newly developed visual phenomenon, has countless possibilities and usage in architecture, history, and media as explored in this research, but it also limits the user in many ways. The lack of interactivity, the dispersion of splat radiance, and the inability to edit and modify leads to several issues in the virtual production and visualization process. That said another big limitation from a philosophical standpoint is the ethical usage of AI and the authenticity in the visual production of historical data. The discourse surrounding AI ethics appears to ignore the issue of visual communication. A search for “ethics and communication and AI” in Google Scholar does not yield promising results, though researchers have occasionally raised concerns regarding the mass utilization of AI.

In his book Landscape and Memory and other writings, art historian Simon Schama critiques how historical narratives are sometimes manipulated or reduced to superficial spectacles (Schama, 1995). He warns against the danger of turning rich, complex histories into sanitized, entertainment-driven experiences that prioritize spectacle over authenticity. This aligns with the “Disneyfication” or what Benjamin called the loss of aura which prioritizes mass appeal and entertainment value. One could argue that the way artists like Frosali use Gaussian imaging of heritage spaces for entertainment purposes does violate the authenticity of the architectural and the emotional value of the physical space where the VFX overshadows the historic essence. In GenAI imaging, this leads to the production and selling of stock data – which are continuously re-produced images lacking quality, meaning, and emotional depth.

The ethical significance of generative AI is often evaluated based on its ability to achieve visual authenticity and accuracy. However, historical reconstruction, grounded in facts and evidence, is inherently an interpretative process. Historians analyze historical data and propose plausible scenarios that might have occurred based on available evidence, leaving room for informed speculation and prediction (Kalay et al., 2007). So, is there a possibility to shift this perspective from accuracy to integrity? If the emotional and sentimental integrity and memory of the artifact are not vandalized, there should be an alternate methodology for the ethics of AI. The intellectual challenges for historians today are the methodological redefinitions of the way images are processed, analyzed, and presented and this demands critical inquiry. This task is not new because digital heritage frequently had to reconstruct new methodologies for visual analysis and to stay “up to date” with the evolving technologies for the preservation of cultural value (Rodríguez-Ortega, 2020). The introduction of Gaussian splatting as a novel methodology for the visual analysis of virtual spaces will spark new critical discussions in the field of virtual heritage preservation. However, there is an essential need to reassess the ethical use of this technology to ensure it does not potentially harm the community or compromise the integrity of the physical architectural site.

6 Conclusion

Generative AI is transforming the 21st century, prompting architects and historians to reevaluate its implications. The exploration of Gaussian Splatting as a medium for immersive heritage environments shows promise for using 3D generative AI imagery to capture historical realities. Moving beyond the limitations of traditional photogrammetry, this technology fosters a richer engagement with architectural heritage. The Gatekeeper’s Cottage case study highlights how visual glitches and abstract representations create immersive experiences rather than mere replications. Artists like Ruben Frosali are pushing the boundaries of this medium, transforming architectural spaces into virtual experiences. This article analyses the components and limitations of the Gaussian model, offering insights for experts to enhance their understanding of immersive heritage visualization and explore new ways to reveal historical narratives.

As 3D generative AI continues to advance, Gaussian Splatting holds great promise for revolutionizing heritage preservation, especially in the realm of virtual living and digitally enabled universes. Humanity’s future is increasingly leaning towards virtual living, and this technology enables the capture of dynamic, multi-layered experiences, allowing individuals to relive both personal and collective memories in immersive virtual environments. 4D Gaussian Splatting research has already started pushing the limits of reality capture by enabling dynamic scene rendering, where video recordings can capture movement and respond to 4D tracking and editing. The new AI-driven imaging goes beyond simple replication, transforming events into interactive and immersive experiences. The potential of this technology is particularly significant in architectural heritage and archaeology, as it can transform static virtual tours into fully interactive, dynamic 4D explorations of historical sites. The next phase of the research will focus on the limitations of this study and will begin to examine the visual features of 4D GS applications. These applications go beyond traditional 3D Gaussian models and enable interactivity through the movement of captured moments. This investigation is ongoing, and as the technology continues to evolve, new opportunities will arise for exploring its visual aesthetics. This will enhance our understanding of what 3D Generative AI technology can offer for the future of our digital heritage.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding authors.

Ethics statement

Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author contributions

OJ: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Visualization, Writing – original draft, Writing – review & editing. AB: Conceptualization, Methodology, Project administration, Resources, Supervision, Validation, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. The research is funded by Graduate Research Scholarship by University of Melbourne, Victoria, Australia.

Acknowledgments

I would like to express my sincere gratitude to the University of Melbourne’s Campus Management team for their invaluable support and assistance in facilitating this research.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The authors declare that Gen AI was used in the creation of this manuscript. Three-dimensional Generative AI tool Jawset Postshot was used to create 3D-models of the Gatekeeper's Cottage. The produced visualizations are thoroughly discussed and analyzed in the paper.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

1. ^3D generative AI models are compatible with various gaming and interactive media software through plug-ins. Unreal Engine, Unity 3D, and Touchdesigner all have their specific plug-in for incorporating 3D generative AI models and scans such as NeRFS and Gaussian Splatting.

2. ^Metaverse is the post-reality universe, a perpetual multiuser environment that merges physical reality with digital virtuality. In a Metaverse universe, users can have multisensory interactions with virtual environments, digital objects, and other people (Mystakidis, 2022). Various companies have introduced their own Metaverse universes, i.e., Microsoft as Microsoft Mesh, Epic Games as Fortnite, etc.

References

Addison, A. C. (2000). Emerging trends in virtual heritage. IEEE Multimed. 7, 22–25. doi: 10.1109/93.848421

Crossref Full Text | Google Scholar

Apple Inc. (n.d.). Creating spatial photos and videos with spatial metadata. Apple developer. Available at: https://developer.apple.com/documentation/imageio/creating-spatial-photos-and-videos-with-spatial-metadata (accessed December 24, 2024).

Google Scholar

Bad Decisions Studio. (2023). What is 3D Gaussian splatting? [video]. YouTube. Available at: https://www.youtube.com/watch?v=Tnij_xHEnXc&t=142s (accessed March 21, 2024).

Google Scholar

Barazzetti, L., Previtali, M., Cantini, L., and Oteri, A. M. (2023). Digital recording of historical defensive structures in mountainous areas using drones: considerations and comparisons. Drones 7:512. doi: 10.3390/drones7080512

Crossref Full Text | Google Scholar

Barker, T. (2011). “Aesthetics of the error: media art, the machine, the unforeseen, and the errant” in Error: Glitch, noise, and jam in new media cultures. ed. M. Nunes (New York: Bloomsbury Collections), 42–58.

Google Scholar

Basso, A., Condorelli, F., Giordano, A., Morena, S., and Perticarini, M. (2024). Evolution of rendering based on radiance fields: the Palermo case study for a comparison between NeRF and gaussian splatting. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. XLVIII-2/W4-2024, 57–64. doi: 10.5194/isprs-Archives-XLVIII-2-W4-2024-57-2024

Crossref Full Text | Google Scholar

Benjamin, W. (1936) The work of art in the age of mechanical reproduction. ed. H. Arendt (New York: Schocken Books).

Google Scholar

Buragohain, D., Meng, Y., Deng, C., Li, Q., and Chaudhary, S. (2024). Digitalizing cultural heritage through metaverse applications: challenges, opportunities, and strategies. Herit. Sci. 12:295. doi: 10.1186/s40494-024-01403-1

Crossref Full Text | Google Scholar

Burdziakowski, P., Sužiedelytė-Visockienė, J., Bobkowska, K., and Tysiac, P. (2024). 3D Gaussian splatting for UAV night photogrammetry., in International Symposium on Applied Geoinformatics (ISAG2024), 9–10 May 2024, Wrocław, Poland: International Society for Photogrammetry and Remote Sensing (ISPRS).

Google Scholar

Champion, E. (2015). “Virtual heritage and digital culture” in Critical gaming: Interactive history and virtual heritage. eds. M. Deegan, L. Huges, P. Andrew, and H. Short (England: Ashgate Publishing Limited), 93–124.

Google Scholar

Champion, E. M. (2017). Digital humanities is text heavy, visualization light, and simulation poor. Digit. Scholarsh. Humanit. 32, fqw053–i32. doi: 10.1093/llc/fqw053

PubMed Abstract | Crossref Full Text | Google Scholar

Champion, E. (2022). Playing with the past: Into the future. Cham: Springer International Publishing.

Google Scholar

Champion, E. (2023). Rethinking virtual places. Indiana, USA: Indiana University Press.

Google Scholar

Computerphile (2024). 3D Gaussian splatting! – Computerphile [video]. YouTube. Available at: https://www.youtube.com/watch?v=VkIJbpdTujE (accessed September 20, 2024).

Google Scholar

Crazy Radio (2024). Collective memory. KENCIDER. Available at: https://kencider.xyz/cminfo/ (accessed September 13, 2024).

Google Scholar

Du, K., Liang, Z., and Wang, Z. (2024). GS-ID: Illumination decomposition on Gaussian splatting via diffusion prior and parametric light source optimization. arXiv preprint arXiv:2408.08524. doi: 10.48550/arXiv.2408.08524

Crossref Full Text | Google Scholar

Frosali, R. (2019). Memories of Tsukiji. RubenFro. Available at: https://rubenfro.com/memories-of-tsukiji (accessed November 1, 2023).

Google Scholar

Frosali, R. [@Ruben_Fro]. (2024). Uh oh, gaussian splatted something new for Afterlife Paris. [image attached]. [tweet] X. Available at: https://x.com/Ruben_Fro/status/1780051959485235273 (accessed September 16, 2024).

Google Scholar

Guédon, A., and Lepetit, V. (2023). SuGaR: surface-aligned Gaussian splatting for efficient 3D mesh reconstruction and high-quality mesh rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Available at: https://anttwo.github.io/sugar/.

Google Scholar

Huttunen, O. (2024a). My observations on Gaussian splatting and 3D scanning [video]. YouTube. Available at: https://www.youtube.com/watch?v=pfCZeJHfjVs (accessed December 23, 2024).

Google Scholar

Huttunen, O. (2024b). Unlocking the mystery of sparse point clouds in Gaussian splatting [video]. YouTube. Available at: https://www.youtube.com/watch?v=RoZg_-Npi0E (accessed December 23, 2024).

Google Scholar

Innocente, C., Nonis, F., Lo Faro, A., Ruggieri, R., Ulrich, L., and Vezzetti, E. (2024). A Metaverse platform for preserving and promoting intangible cultural heritage. Appl. Sci. 14:3426. doi: 10.3390/app14083426

PubMed Abstract | Crossref Full Text | Google Scholar

Jones, N. A. (2022) Glitch poetics. ed. J. Zylinska (London: Open Humanities Press).

Google Scholar

Kalay, Y. E., Kvan, T., and Affleck, J. (2007). “Introduction: preserving cultural heritage through digital media” in New heritage. eds. Y. Kaly, T. Kvan, and J. Affleck (London: Routledge), 1–9.

Google Scholar

Kerbl, B., Kopanas, G., Leimkuehler, T., and Drettakis, G. (2023). 3D Gaussian splatting for real-time radiance field rendering. ACM Trans. Graph. 42, 1–14. doi: 10.1145/3592433

Crossref Full Text | Google Scholar

Khakhulin, T., Korzhenkov, D., Solovev, P., Sterkin, G., Ardelean, A.-T., and Lempitsky, V. (2022). Stereo magnification with multi-layer images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Available at: https://samsunglabs.github.io/StereoLayers/.

Google Scholar

Krapp, P. (2011). Noise channels: Glitch and error in digital culture. Minneapolis: University of Minnesota Press.

Google Scholar

Liu, Z., Guo, Y., Li, X., Bickel, B., and Zhang, R. (2024). BiGS: Bidirectional Gaussian primitives for Relightable 3D Gaussian splatting. arXiv preprint arXiv:2408.13370. doi: 10.48550/arXiv.2408.13370

Crossref Full Text | Google Scholar

Manovich, L. (2020). Cultural analytics. Cambridge: MIT Press.

Google Scholar

Manovich, L. (2023). “Seven arguments about AI images and generative media” in Artificial aesthetics. eds. L. Manovich and E. Arielli. Available at: http://manovich.net/index.php/projects/artificial-aesthetics-book (Accessed September 18, 2024).

Google Scholar

Mazzacca, G., Karami, A., Rigon, S., Farella, E. M., Trybala, P., and Remondino, F. (2023). NeRF for heritage 3d reconstruction. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. XLVIII-M-2-2023, 1051–1058. doi: 10.5194/isprs-archives-xlviii-m-2-2023-1051-2023

Crossref Full Text | Google Scholar

Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., and Ng, R. (2020). NeRF: representing scenes as neural radiance fields for view synthesis. Commun. ACM 65, 99–106. doi: 10.1145/3503250

PubMed Abstract | Crossref Full Text | Google Scholar

Mystakidis, S. (2022). Metaverse. Encyclopedia 2, 486–497. doi: 10.3390/encyclopedia2010031

Crossref Full Text | Google Scholar

Pahl, A. (2023). Gaussian splatting: painting immersive scenes with reality. Available at: https://www.cablelabs.com/blog/gaussian-splatting-immersive-scenes (accessed September 23, 2024).

Google Scholar

Palestini, C., Basso, A., and Perticarini, M. (2022). Machine learning as an alternative to 3D Photomodeling employed in architectural survey and automatic design modelling. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. 48, 191–197. doi: 10.5194/isprs-archives-XLVIII-2-W1-2022-191-2022

Crossref Full Text | Google Scholar

Reinhuber, E., Seide, B., and Williams, R. A. (2024). Different layers of reality. A retrospective evaluation on diverse possibilities for digitally preserving memories of the Yunnan Garden as an immersive experience., In 29th International Symposium on Electronic Art (ISEA) Australia: ISEA, Brisbane Convention and Exhibition Centre.

Google Scholar

Rodríguez-Ortega, N. (2020). “Image processing and computer vision” in The Routledge companion to digital humanities and art history. ed. K. Brown (New York: Routledge), 338–357.

Google Scholar

RubenFro (2023). Avatāra – dissolving the veil of reality [video]. YouTube. Available at: https://www.youtube.com/watch?v=yMWYSn6lIiI (accessed December 24, 2024).

Google Scholar

Rubin, M., Cardoso, J. C. S., and Carvalho, P. M. (2022). Design explorations of interactive point cloud based virtual environments using volumetric capture and visualisation techniques., in Entertainment Computing – ICEC 2022: 21st IFIP TC 14 International Conference, ICEC 2022, Bremen, Germany Proceedings, Springer International Publishing, 256–265.

Google Scholar

Schama, S. (1995). Landscape and memory. New York: A.A. Knopf (Distributed by Random House).

Google Scholar

Schnabel, M. A., and Rushton, H. (2022). “Immersive architectural legacies: the construction of meaning in virtual realities” in Cultural computing visual heritage: digital approaches in heritage science. eds. E. Ch’ng, H. Chapman, V. Gaffney, and A. S. Wilson (Switzerland: Springer Series on Cultural Computing).

Google Scholar

Spoonauer, M. (2023). The apple vision Pro’s spatial video looks like a game changer — Here’s why. Tom’s Guide. Available at: https://www.tomsguide.com/opinion/this-apple-vision-pro-feature-could-actually-make-it-worth-dollar3500 (accessed December 24, 2024).

Google Scholar

Stogner, M. B. (2010). The immersive cultural museum experience – creating context and story with new media technology. Int. J. Incl. Mus. 3, 117–130. doi: 10.18848/1835-2014/cgp/v03i03/44339

Crossref Full Text | Google Scholar

Waczyńska, J., Borycki, P., Tadeja, S., Tabor, J., and Spurek, P. (2024). GaMeS: Mesh-based adapting and modification of Gaussian splatting. arXiv preprint arXiv:2402.01459. doi: 10.48550/arXiv.2402.01459

Crossref Full Text | Google Scholar

Wang, R., Hua, C., Shingys, T., Niu, M., Yang, Q., Gao, L., et al. (2024). Enhancement of 3D Gaussian splatting using raw mesh for photorealistic recreation of architectures. arXiv preprint arXiv:2407.15435. doi: 10.48550/arXiv.2407.15435

Crossref Full Text | Google Scholar

Wu, G., Yi, T., Fang, J., Xie, L., Zhang, X., Wei, W., et al. (2024). 4D Gaussian splatting for real-time dynamic scene rendering., in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle Convention Center.

Google Scholar

XR AI Spotlight (2024). Turning point clouds, NeRFs and Gsplats into art [video]. YouTube. Available at: https://www.youtube.com/watch?v=pO2KpUtv0r4 (accessed December 24, 2024).

Google Scholar

Yip, A. (2020). Hypermapping conflict: war, art and immersive aesthetics. Aust. N. Z. J. Art 20, 40–53. doi: 10.1080/14434318.2020.1764228

PubMed Abstract | Crossref Full Text | Google Scholar

Young, M. (2022). Reality modeled after images: Architecture and aesthetics after the digital image. New York: Routledge.

Google Scholar