Going Beyond the Classic News Narrative Convention: The Background to and Challenges of Immersion in Journalism

Domínguez, Eva

doi:10.3389/fdigh.2017.00010

REVIEW article

Front. Digit. Humanit., 17 May 2017

Sec. Human-Media Interaction

Volume 4 - 2017 | https://doi.org/10.3389/fdigh.2017.00010

This article is part of the Research Topic Immersive Journalism: Virtual Reality in the Presentation of News View all 4 articles

Going Beyond the Classic News Narrative Convention: The Background to and Challenges of Immersion in Journalism

$\r\nEva Domínguez*\r\n$ Eva Domínguez*

Department of Communication, Universitat Pompeu Fabra, Barcelona, Spain

The disappearance of the screen’s physical barrier, the fourth wall, challenges many of the narrative conventions used in the traditional journalistic account. This is an area in which we are still taking the first steps, but these moves have already raised interesting narratological challenges. Technological development allows us not only to have the sensation of being elsewhere but also to interact with the storytelling elements. In this article, the author reviews the steps that have been taken up to the present time in the use of these platforms for news. She revises as well the reflections carried out by disciplines connected with narrative construction on the challenges of this new way of factual storytelling. The implications of being inside the scene of the action and being able to interact with it means rethinking the conventions of audiovisual editing and the visual perspective of the audience, as well as the relevance of the three-dimensional nature of real sound and the freedom to act inside the story without modifying the real course of the events.

Introduction

The journalistic story is always a child of its time, and it is inserted in the narrative ways of its epoch. It is, therefore, a linguistic and narrative activity that inherits from a tradition, which it then molds to make it suitable for its circumstances (Tuchman, 1978; Eason, 1982; Roeh and Ashley, 1986; Van Dijk, 1988). From the acta diurna of ancient Rome, the first newspaper in the seventeenth century, the so-called writer’s journalism (Habermas, 1994) and so on up to contemporary journalism, the informative story has borrowed preexisting narrative resources and adapted them to its ends. The journalistic form, that is to say, the way in which news is presented, is as relevant as its content (Nerone and Barnhurst, 2003). The forms are symbolic representations (Tuchman, 1978; Huxford, 2001), which precede the content (Hartley, 1996; Ekström, 2002) and can end up asphyxiating it (Carey, 1974; Altheide and Snow, 1991). Although the formats serve to facilitate the work of the newsrooms (Gomis, 1987), the need to find new forms has also been a constant.

These days, this exploration goes hand in hand with immersive technologies. The influence of the rhetoric of the videogame (Darley, 2002; Jenkins, 2006) and technologies like virtual reality (VR) and augmented reality (AR) is obvious in the transformation of non-fictional narrative. The change goes beyond the simple adoption of some techniques or formats because it involves converting the narrative into an agency experience. This article will cover the steps that have been taken to get to the present moment and will provide elements for a reflection on the nature of narrative immersion. In the first part, we will deal with the concept of immersion in journalism as a synonym of in-depth investigation and as a digital form, as well as looking at the pioneering experiments with immersive technologies. In the second part, we briefly review the origins and characteristics of the conventional journalistic narrative, which constitute a barrier to immersion to concentrate on the analysis of the elements that have a bearing on the storytelling grammar of this new language with the aim of widening the reflection on the paths to and possibilities of immersive journalism.

On Immersion in Journalism

Immersion As a Reporting Technique

In journalism, the concept of immersion has been traditionally understood as a researching technique based on the premise that the reporter must spend long time learning about a reality before it can be narrated. Thus, journalists need to live for a while with the social reality they want to capture. Early precedents of this trend are the muckrakers at the end of nineteenth century and the beginning of the twentieth century. Although the term was unknown to its practitioners, immersion journalism is nowadays assimilated to investigative journalism. Immersion is a part of the job, as Pulitzer Prize-winning reporter Edward Humes admits: “Interviews are essential and documents can be invaluable, but there is nothing like being present for the events you’re writing about or, at the very least, becoming intimately familiar with the world and culture that your characters inhabit” (Humes, 2011). Immersion turns out to be a fundamental condition for good reporting, since the journalist needs to earn the confidence of individuals involved in the story, so they behave naturally, just as they would to a coworker or a relative (Sims, 1984).

In the majority of cases, the journalists do not hide their status as informants. The involvement and the conditions of temporary coexistence are not always the same. A paradigmatic case is that of reporter Hunter S. Thompson who often became a participant in the situations he wrote about. “Gonzo journalism” formed an important part of the surge of New Journalism, which emerged in the 1960s in the United States as an alternative to the prevailing journalistic style. Truman Capote, Tom Wolfe, Gay Talese, and Joan Didion, among others, were its most prominent figures.

The initial part of the extended cohabitation is a process of increasing the trust between the people involved in the story and the journalist. Nevertheless, in some cases, the journalist could opt to hide his identity when dealing with certain themes. For example, German journalist Günter Wallraff took on the identity of an alter ego in his personal immersion system, for which he utilized methods involving secrecy in the belief that to know the truth, the journalist cannot introduce himself as such. Wallraff had to live a hidden life and with official protection because of threats against his life.¹

Participating in a social group is a part of the immersion process. This method of achieving understanding has parallels with those used in ethnography, which bases its study on a social reality through immersion in its life, routines, and rituals. Cramer and McDevitt (2004) propose principles and techniques for practicing ethnographic journalism. The ethnographer can participate for an extended period in the life of a group in different degrees of involvement, from complete participation to complete observation.

As the well-known reporter Kapuscinski wrote, “it’s wrong to write about people without living through at least a little of what they are living through” (Kapuscinski, 1987, p. 29). Immersion is not only an investigative process but also the necessary condition to convey to the reader the feeling of being inside another reality. In other words, the journalist needs to live the situation to make the readers feel they are there. In literary journalism, while immersion is an act of imagination for the reader, it is a necessary physical experience for the reporter. Nowadays, the sensorial features of immersive technologies give immersion an extranoematic quality (Aarseth, 1997) through a feeling of presence in the represented reality.

Immersion in Digital Worlds: Early Prototyping for Journalism

Some centers experimented early with the possibilities of the new emerging technologies. Since 1966, the Integrated Media Systems Center (IMSC), based at the University of Southern California, has established itself as a reference center in multidisciplinary research on Internet and multimedia technologies. Within this frame of research, the User-Directed News project had the aim of offering information on current affairs in immersive formats, and to achieve this, they worked with panoramic video and positional audio; they used voice recognition modules to navigate through the news; they applied 360° tracking video technology; they linked the objects to contents databases and news archives and integrated the recognition and the tracking of the readers’ movements to increase the sensation of immersion (Pryor et al., 2003). The experimentation with these technologies was based on the supposition that for certain types of news, such as a natural disaster, an opening ceremony of an Olympic Games, or a presidential inauguration, the audience could be transported to the place where the event is taking place through an immersive recreation, which gives the sensation of being right there, witnessing it, and feeling it sensorially; seeing, hearing, and even touching what happens in the place where the action is unfolding, as Pryor (2000), one of the project’s participants, describes it.

Meanwhile the University of Columbia’s school of journalism, through its Center for New Media, carried out several practical experiments using omnidirectional video and AR (Hollerer et al., 1999). In 1997, they covered the traditional Saint Patrick’s Day celebrations in New York with 360° video. During the parade, the students recorded the demonstration held by gay and lesbian organizations and the subsequent arrests. Two years later, after the death of Amadou Diallo, a Bronx resident killed by 41 police bullets, they collaborated with journalists from APBNews.com on a 360° photographic report from the scene of the killing. They also used AR to discuss past events, such as the 1968 students’ revolts on the university campuses (Pavlik, 2001; Pavlik and Bridges, 2013).

More recently, experimentation with immersive technologies has received a new boost through the work led by Nonny de la Peña, initially within the Immersive Journalism² group and more recently from the Emblematic Group company. De la Peña, considered the “godmother of virtual reality,” has shown her work in festivals such as Sundance, Tribeca, SXSW, the Davos Economic Forum, and in museums (Victoria and Albert Museum in London, Moscow Museum of Modem Art). Some of her pioneering works were created on the Second Life platform, such as Cap & Trade, an investigation into the impact of the carbon markets through the personification of a digital alter ego, or avatar, and Gone Gitmo, a recreation of Guantánamo. However, it is the work created with immersive VR, which has had more impact. For example, in the Ipsress project, the subject, equipped with VR gloves and goggles, is inside the digital skin of a prisoner in a cell at Guantánamo who hears the interrogation that someone is undergoing on the other side of the cell wall. What we hear is part of the reports that were made public. In Hunger in Los Angeles, a moment is reconstructed in which a person waiting in a food line goes into a diabetic coma. The sound recordings are from the incident itself, and this, for the investigator, is one of the elements that contribute notably to augmenting the sensation of being present. In Use of Force, we see the moment in which an immigrant is gunned down by a US patrol on the border between Mexico and the United States. The reconstruction is based on witness testimony and video captured during the incident. In the Project Syria, the viewer is transported to the city of Aleppo in the moment of an attack, one in which children especially suffer the consequences. In Kiya, she reconstructs a situation of domestic violence. Once again, De la Peña bases her report on authentic recorded material to recreate the incident in detail. Out of Exile reconstructs the moment in which a gay man informs his family about his sexual tendency. After Solitary deals with the issue of isolation cells in prisons through the story of a prisoner who spent years in one of them, and for this, one of the cells of the Maine State Prison was reconstructed.

Right now, narrative experimentation with immersive formats has left the laboratories to take part in the news coverage of many companies. And even though the majority of VR goggles being produced centers around 360° video, there is no shortage of examples carried out with computer-generated images. Within this group, we find, among others, the work of the Emblematic Group and productions like Harvest of Change, from the Des Moines Register, which explores how demographic, technological, economic, and environmental changes affect the daily life of an Iowa farming family; or 6 × 9, from The Guardian, on experiencing solitary confinement. The proliferation of productions is closely connected to the reflection on and narrative challenges of a language, which is taking its first steps.

Playing and Sensing the News: In Search of an Embodied Experience

Laboratory experiments ran alongside theoretical reflection. The possibilities of VR and AR technologies, as well as the strong influence of videogames as a cultural industry, did not go unnoticed for professionals and researchers in the field of journalism, mainly in the United States, who started in the late 1990s thinking about what it could change in the way of telling factual stories (Bowman and Willis, 2001; Paul and Hansen, 2010). For example, the two consecutive seminars were organized by the University of Minnesota, through the Institute for New Media Studies in 2001 and 2002. The first one, entitled “Playing the News: Journalism, Interactive Games and Narrative,” focused on what games could bring to news reporting. The second one, “Sensing the News: What new technologies could mean for journalism,” dealt with how interactive and immersive audio, as well as video and photography in 360°, could be applied to journalism. All of them, to a greater or lesser degree, offered paths of exploration for news storytelling and how to increase the feeling of exploring a place.

The idea of what an immersive format consists of is influenced by both the notion that the user can interact somehow with the story and have a feeling of presence in a place. The expression “immersive” is flexible and can be interpreted in many ways. For some authors, this concept was associated to a story presentation that allows the Internet user to interact with story elements or data. For researcher MacGregor, the chances to delve deeper into a report are compatible with the linearity of the story. “Journalists would be able to ‘entrap’ users through an immersive multimedia story - a tale with a linear heart, with options to deepen and widen it in unprecedented ways” (MacGregor, 2003, p. 8). As we can see, the possibility of “doing something” with the story, such as making the narration interactive, could be seen even back then as a facilitator of the immersion.

Other authors (Pryor, 2000; Pavlik, 2001) early on sensed that the development of technologies that sensorially eliminated the physical border was the most fertile for terrain immersive journalism.

You’re immersed in the evening news of 2010. A ninety-second video bulletin reports an important extraterrestrial discovery on Europa, Jupiter’s largest moon. Using your head-worn display, you look around the surface simply by turning your head. You look at the left-hand portion of the three-dimensional omnivideo (your gaze acts as a mouse would today) and say ‘select’, and a second window in your immersive environment plasma display reveals a special video inset with detailed animation showing how life began under Europa’s frozen hydrogen crust (Pavlik, 2001, p. 3).

The type of scenario that Professor John V. Pavlik imagined is only possible through a sensory perception of space, such as the deep immersive journalism concept of the work lead by De la Peña et al. (2010). The subjects have the sensation that their body occupies a space created by digital technology and that they can move around the stage created by a computer. Powell (2003), who was the director of the IMSC, one of the centers that experimented with immersive technologies at the time, considered that these emerging forms obliged one to rethink journalistic conventions.

A decade later, the rapid growth of immersive technologies and the enormous production of VR content have contributed to the development of immersive characteristics associated with the sensation of presence, while the influence of video games seems to have developed more closely in alignment with the concept of a “newsgame.” Since the organization of “Playing the News” in 2001, reflection and practical experimentation in the mass media are more alive than ever. The term “newsgame” has become popularized in the journalistic world after the publication of Newsgames: Journalism at Play, by Bogost et al. (2010). Although the concept was coined originally by Frasca (2007) to define games with an editorial intention, for the authors, newsgame was understood to mean a game centered around a current affairs theme. In the game, the user had a role that affected the way the game developed. The action of the user, whether in a first-person or third-person role, is the determining factor. The influence of the rhetoric of the videogame leads authors like Bogost (2007) to talk of a procedural rhetoric to refer to a particular way of using processes as a persuasive strategy.

Nevertheless, the influence of the rhetoric of the video game goes beyond the newsgame in documentary formats and in large-scale digital reports. In the enormous spectrum of productions that fall under the umbrella of interactive documentary (Gaudenzi, 2009; Gifreu, 2014), there are examples influenced by the expressive resources of the video game. This is because the “webdoc” (from webdocumentary) is a hybrid form and a common ground in which professionals and contributions from various fields converge (Lietaert, 2011).

The ability to act through an avatar in a digital world is one prerogative of the video game that extends into other spheres. The borders are so fuzzy, and the permeability is so high that it has been debated by theoreticians from different fields with both contrasting and coinciding arguments, among them Murray (1997, 2004, 2012), Jenkins (2006), Ryan (2001a,b, 2006), and Aarseth (1997, 2004, 2012).

Both newsgames, with interactivity as its principal power, and immersive experiences, centered around the sensory perception of a situation, seek the same thing: to augment understanding and sensibility toward what is to be explained. The common denominator is the idea of situated meaning, which Bogost et al. borrow from James Paul Gee³ “[…] the idea that human learning is enhanced within an embodied context” (Bogost et al., 2010, p. 112).

Elements for a Reflection on an Immersive Language for News Storytelling

Immersion in Storyworlds: Being Transported and Performing

Immersion existed as an idea long before the advent of digital technology. Allusions to immersion in books, in writing and storytelling, are so common that they have become common expressions. The readers are immersed in a world, factual or fictional, built on their imagination with the material and guidance offered by the author. Immersion depends on the imagination of two people: the author, who triggers it with his story, and the reader, who is immersed in it. Without the cooperation of the reader, the work does not come alive. The reception theory point of view considers that the meaning of the text is built at the time of reading, in the act of cooperation that occurs in the encounter between reader and text. The idea of a reader model (Eco, 1993) or the implicit reader (Iser, 1974) implies the introduction of the important role of the reader in the interpretation of the text.

Being immersed in a story is always a trip of the imagination. As the psycholinguist Richard Gerrig put it in Experiencing Narrative Worlds, “a narrative serves to transport an experiencer away from the here and now” (Gerrig, 1993, p. 3). According to Gerrig, narrative worlds, which may or may not be fictional, are mental places where the reader, viewer, or listener arrives as a result of a series of psychological processes triggered by the story. But, as Gerrig (1993) highlights, “a text cannot force a reader to experience a narrative world” (p. 5). He explores two of the most common metaphors to describe the narrative experience: “readers are often described as being transported by a narrative by virtue of performing that narrative” (Gerrig, 1993, p. 2).

Thus, immersion is a psychological activity that, in the case of literature, can be activated by a text, and gives the reader the sense of being transported to another reality in which he or she has the mental capacity of performing the activity.

According to Gerrig, performance is carried out through inferences and participatory responses. With inference, the readers fill in the spaces in the text and in the causal structure of the narrative worlds, as they do in everyday life. The readers resort to the knowledge coming from their life experience to fill the spaces. The participative answers are expressions of the unique and particular experience that each reader has in his narrative world. As we will see, digital immersion feeds the same metaphors.

The Barrier to Immersion in the Narrative Convention

Traditional journalistic narrative has established a series of standard strategies, both written and audiovisual, in the interest of producing a sensation of facticity. Objective journalism has been based on the formal separation between facts and opinions, in the balance between different versions of the same act. This style was adopted during the Progressive Era (1890–1920) (Errico et al., 1997; Pöttker, 2003) when the journalists were better educated and better trained than in previous generations and journalism was a more developed industry. A more “scientific” journalism became the standard, in contrast to the more literary style of previous epochs, and the adjective “scientific” is associated with pure information (Schudson, 1978). The style of objective journalism becomes the standard for good journalism and establishes itself for many years in the world’s press.

This style is also characterized by the removal of the presence of the journalist and narrator. In this way, the discursive style of the traditional press is based on underlying, through different resources, factual nature of the events, constructing a solid relational structure for the facts and providing information that also has an attitudinal and emotional dimension, as stated in the linguistic analysis of Van Dijk (1988). In the audiovisual sphere, the formal news conventions are carried out in standard takes, as dissected by Gaye Tuchman. The author shows that the takes underline the visual distancing of the reporters with respect to the phenomena situated in the background and how the framing is deeply rooted in the conventions and rules shared by the profession about what is and what is not news and also how it should be shaped (Tuchman, 1972, 1973, 1978).

Therefore, the journalistic forms are constituted as symbolic representations (Tuchman, 1978; Huxford, 2001), which precede the content (Hartley, 1996; Ekström, 2002). Mediation theory (Altheide and Snow, 1991) starts from the premise that the format is the organizing principle of the news media, because anything that does not conform to existing narrative conventions and their formats cannot be narrated. The journalistic form becomes a container into which all types of story should be adapted, which can sometimes be an overrestrictive corset for carrying out one’s profession. For this reason, this standard rhetoric has had its exceptions.

In opposition to the telling of the facts or news, there coexists the “story model” (Eason, 1982; Kovach and Rosenstiel, 2001), whose natural format has been the report. In this model, the dramatic, narrative structure has been widely accepted in the academic and professional tradition. Literary journalism, exemplified by movements such as New Journalism, is an example of this search for formal freedom. Wolfe (1984) summarizes the four techniques used by this movement: scene-by-scene construction; full recording of all the dialog; the use of the characters’ point of view; and the detailed story of the characters, situations, and settings. All of these aim to immerse the reader more completely in the story. The visual correspondence of this style would capitalize on the documentary audiovisual style.

The search for journalistic narrative forms that go beyond the conventions of the moment has been a constant in the profession. At this moment, one of the ways being explored is centered on the possibilities offered by immersive and interactive audiovisual formats.

Being Inside the Image of the News Story

As mentioned earlier, narrative immersion has been considered as fundamentally a psychological quality from the point of view of narrative theory. In contrast, in the sphere of VR, which has analyzed and investigated it deeply for decades, one distinguishes between the technological quality of the experience—immersion—and the subjective experience of feeling oneself in the space being represented—presence. The elements that affect the quality of the immersion in VR have been widely analyzed, debated, and investigated empirically. Most of the efforts have concentrated on emphasizing the technological configurations, as pointed out by Cummings and Bailenson (2015), above other aspects, especially the narrative. Although there seems to be a consensus that the better the quality of the system, the better the sensation of presence, the same authors underline the fact that some characteristics are more important than others. Spatial presence resides more in the field of view, stereoscopy, and tracking level than in the image quality and sound resolution, according to these authors.

The system is very relevant in creating the illusion of being inside the image, or Place Illusion (Slater, 2009), in VR, as it is based on the correlation between the actions that the subject carries out, like turning one’s head and the view in one direction that changes position, and the reactions of the setting, or as Slater puts it, sensorimotor contingencies. Nevertheless, as the cinematic tradition has long shown, the illusion of being in a place can also be achieved in the frame of the screen. But, as Slater (2009) points out, the feeling of “being there” on desktop systems requires “additional mental recreation” since people cannot feel “located according to the rules of real-world SC’s [sensorimotor contingences]” (p, 3552).

As a narrative activity, it is fundamental to understand up to which point the construction of the story, in other words, the media content, contributes to the sensation of immersion. Immersion forms a part of the narrative strategy to awaken the empathy of the citizen for what is happening in the world. Although the technological qualities of the VR systems have proved fundamental in achieving Place Illusion, journalistic immersion cannot be based uniquely on these qualities. Furthermore, it is possible that when it comes to evaluating the weight of the narrative in the immersive journalistic experience, these are not the most relevant.

In this sense, Baños et al. (2004) tested the role of immersion and media content in the sense of presence. In their experiment, they showed that in a neutral VR setting, the sensation of presence lies fundamentally in the quality of the immersive system, but in an emotional setting, one in which the audiovisual elements convey an emotional mood, the sensation of presence depends on the content, defined as “the objects, actors and events represented by the medium” (Baños et al., 2004, p. 734). The importance of the narration in the immersion is also underlined in another study. Grassi et al. (2008) carried out an experiment to evaluate the importance of the contents in the sensation of presence on three platforms: mobile phone, desktop computer, and HMD, and they conclude that the media content influences the sense of presence.

To be inside the image is the most evident element of immersive grammar. In the case of journalism, being in the image changes to being inside the narrative scene; therefore, it is not exclusively a matter of feeling transported to a place, but rather to a place in which real stories are taking place. Apart from VR, other disciplines have reflected on the importance of this component in the storytelling.

The vicarious experience of having an itinerary through a digital world is also one of the basic expectations of the videogame player. The image has a direct effect on the satisfaction of the experience. Or, as Darley recalls, “computer game players and those who embark upon simulated rides do not expect to be told deep stories, yet alone to be intellectually taxed or challenged. They are ‘into the image’ at a more corporeal and tactile level” (Darley, 2002, p. 165). The penetration of the image in the videogames is transmitted by giving the sensation of traveling through three-dimensional space. It is a fluid image, a world image, or a stage image.

Whether without a screen or through one, this spatial reality allows the user “to experience spaces of conflict that are difficult to engage with the abstract” (Bogost et al., 2010, p. 65). The scene takes precedence: “Spatial documentary games extend cinematic mise-en-scène,” or “things on the stage” (2010, p. 66). For Bogost et al., spatial immersion is one of the three strategies that allow videogames to connect with news reporting. The other two are operational reality and procedural reality.

The production that the authors are talking about has to have the characteristics of verisimilitude that contributes to the sensation of being inside the image. In the language of the fine arts, it involves realistic detail, a synthetic hyperrealism that in its greatest expression can appear at times to be even photorealistic. The visual esthetic can favor or hinder the spatial immersion. This esthetic hyperrealism is linked to what some experts call perceptual realism (Fencott, 1999; McMahan, 2003). It is about the objects, the setting in which they find themselves in, and the events that take place there corresponding with the real ones.

At the moment, techniques like photogrammetry and videogrammetry contribute to this esthetic hyperrealism. Photogrammetry involves using images from a conventional digital camera to create three-dimensional scenes by triangulating and meshing the photos taken from different positions and angles, while videogrammetry is a method of measuring motion in 3D. As the Emblematic Group points out the potential of placing 3D moving characters “into a volumetric environment opens up huge possibilities for the level of realism achievable in walk-around VR” (Emblematic Group, 2016).

Virtual reality allows us to construct spatial narratives. The Emblematic Group defines them as “storytelling in which the physical and temporal dynamics of the scene—who moved where, and when, and their relative positions at each moment—are fundamental to fully understanding what happened” (Emblematic Group, 2016). The projects produced by Nonny de la Peña have opened up a path in journalistic spatial narrative with VR, which the mass media are now following. Spatial narrative with VR breaks with the tradition of visual cinematic representation and takes the spatial construction that is intrinsic to the digital medium to its maximum expression, as Manovich pointed out: “For the first time, space becomes a media type” (Manovich, 2001, p. 251).

This is precisely the great difference with other types of VR storytelling. These days with VR goggles you can see 360° video, also known as cinematic VR. The spherical video surrounds the viewer who has the perception of being in the center of the image but unable to enter it, of being unable to move around it. Therefore, although the image surrounds the viewer who is able to choose what he sees inside the spherical scene, he is unable to move around it. Furthermore, 360° video uses cinematic editing techniques to move between scenes.

The number of cinematic VR productions in journalism is much higher than that of spatial narrative in VR. Newspapers like the New York Times have wagered clearly on this type of format. The 360° videos can also be seen without the need for a VR device. In this case, the sensation of immersion in another place bumps into the physical border of the screen, the fourth wall. This “wall” diminishes the experience of spatial immersion. The first work in this format in a press context was published in the digital edition of MSNBC.com in 2005, after Hurricane Katrina (Domínguez, 2006, 2013). Five years after this experiment, CNN published on its web a series of five 360° videos on the devastation caused by the earthquake in Haiti.

Before 360° video arrived, the efforts toward contributing to spatial immersion in screen journalism started with spherical or 360° photography and panoramic photography. In the case of 360° photography, its application in journalistic coverage was introduced by media such as The Guardian, Washington Post, The New York Times, CNN.com, or MNSBC.com. Some of this coverage carried out using this technology includes showing the effects of Hurricane Katrina in 2005, the climbing of Tourmalet in the Tour de France in 2006, Cherie Blair’s visit to a Labour Party Congress in that same year, and the Israeli attacks on Gaza which happened between December 2008 and January of 2009 (Domínguez, 2013). In spherical photography, the viewer is in the center of an instant frozen in time. It is not possible to move around the scene in which the image itself has no movement. Some of these works increased the sensation of being in the place through the incorporation of ambient sound, as in the case of the image of the Tour de France.

In panoramic photography, in contrast, viewers are not surrounded by the image but have a certain degree of penetration into the scene through the use of a powerful zoom, allowing them to go from the full image to the tiniest detail. The panoramic image is composed of tens or hundreds of photographs, which means that the gap between one photo and another could be minutes. This type of work applied to current affairs has become popular due to a historic image, that of Barack Obama taking office as president of the United States. Taken by photographer David Bergman with 220 snapshots, it moves from the global vision to the smallest personal gesture. The panoramic picture is not, therefore, the reconstruction of a news event and not the capture of an instant, but rather the reconstruction of a scenario (Domínguez, 2009, 2013).

Finally, returning to immersive technologies, apart from VR, we can also talk of AR or mixed reality. Here, the synthetic information is added to the physical reality, blurring the boundaries between both worlds. The physical world always remains the reference point. The fluidity of movement around the scene and the ability to penetrate the image are characteristics that are inherent to this technology. Geolocalized AR allows one to create situated documentaries: “This format is a natural extension of what place-based journalism has historically done best—stories are put in a local context and act as a supplement to a citizen’s direct experience with the world” (Pavlik and Bridges, 2013, p. 21).

The Visual and Audio Point of View

In classic narrative theory and in cinematography, a distinction is established between the point of view through which the story is told (focalization), the visual point of view (ocularization), and the auditory point of view (auricularization). A shot is either fixed in the gaze of an internal instant of diegesis (internal ocularization) or it is not (zero ocularization) (Gaudreault and Jost, 1995). Internal ocularization is primary when one sees through the eyes of the character and secondary when a subjective gaze is created through the editing process, as in shot/reverse shot. In zero or external ocularization, no character sees the image. This is also called “nobody’s shot.” In this case, the shot belongs to a great imaginer, whose presence can be more evident or less (Jost, 2002).

The cinema is the great ascendant in the representation of digital spaces (Manovich, 2001), and its influence is evident in the visual techniques used, especially in the application of ocularization. The rhetoric of immersion is nowadays strongly identified with internal ocularization. The sensation of sharing the time and space of the story converts it into a potent immersive mechanism. This is what Marie-Laure Ryan calls spatiotemporal immersion. In other words, the reader feels himself in the same place as the character, in what the author calls the imaginary transportation of the virtual body of the reader into the scenario of the events (Ryan, 2001a,b).

In the case of VR platforms, the spectator can choose which part of the surrounding scene to look at in each moment. Internal ocularization is that of a visitor in the virtual space who, in the case of video chooses what to see, and in the case of spatial narrative, can also move around it. For this reason, the first rule that Nonny de la Peña recommends when constructing VR narratives is thinking of your body in the space. Thinking about how to move or from which height to see the scene is important. By default, the visualization is always in the first person, subjective, but this does not mean that one is a character in this story. Usually one is an omniscient and invisible observer of what is happening.

The visual component of spatial immersion is so evident that it may seem that the sensation of immersion depends exclusively on it, but the aural aspect has as much or even more relevance. In journalism, it is especially relevant to deal with original material captured in the time and place where the action dealt with in the report occurred.

In one of her first journalistic projects using VR, Nonny de la Peña collaborated with the expert in VR Mel Slater to reproduce an interrogation at Guantánamo. The participant, equipped with VR gloves and goggles, is situated in the digital skin of a prisoner in his interrogation cell and listens to what is happening to the prisoner on the other side of the cell wall. Although he is not directly the object of physical torture, he does feel the terror of the interrogation. The dialog that is heard is the verbatim, word for word transcription of official documents.⁴ It is a dramatization based on an interrogation.

In contrast, in another project, Hunger in Los Angeles, the participant who recreates the event that took place in a food line in Los Angeles witnesses a scene in which a person faints on the ground while waiting for food. The audio corresponds to the real moment. It is not a case of a subsequent recreation, and it becomes one of the elements that contribute powerfully to the immersion in the scene. Whenever possible, de la Peña works with real audio because its power of evocation is much higher than a dramatization in journalistic work. These journalistic examples cause us to question the assumption that in the works with VR, the auditory stimuli are less relevant than other characteristics such as the field of view, tracking level, and stereoscopy mentioned by Cummings and Bailenson (2015).

In the case of immersion in a narrative, the sound is relevant at all times in the immersive experience, even when other components of the visual and interactive grammar demand the spectator’s attention. For example, in the field of the videogame, when the game is waiting for some action from the player, a charged expectation (Galloway, 2007) is created. In other words, the world is waiting for something to happen. The way in which this expectation is made tangible is through subtle animation and sound. In journalistic work, the constant sound with an atmospheric function has the same immersive, enveloping role. The constant sound, subtle and atmospheric, contributes to creating an atmosphere.

Currently, professionals find themselves facing challenges that involve working with sound in immersive settings. The three-dimensional quality of the audio seems to have much more relevance in immersive settings than in the cinema and even more in spatial narratives than in 360° video. In the setting in which the participant can move around freely, the aural experience has to accompany this movement. The sound in an enveloping space should also be endowed with a three-dimensional quality to have auditory credibility. In short, it has to sound real. The sound should also be spatialized, as Janet H. Murray points out: “louder as you approach; surprising you from behind; cueing you to look left or right, and reflecting the materiality of the virtual world” (Murray, 2016). Therefore, the internal auricularization appears to be one of the components of the grammar of digital immersion with more power to define and develop itself on its own terms, without inheriting conventions from other languages.

In this moment of experimentation, there seems to be a clear consensus that without immersive sound spatial narrative would not work so well. Nevertheless, there’s still a long road to travel before we find a standard through which we can construct this piece of the emerging immersive grammar (Carbotte, 2015).

The Challenge of Interactive Agency

While immersion is associated with the sensory experience of presence, the great promise of the digital sphere is that the user is not passive but can be interactive. Currently, in VR environments, those which allow a greater sensation of sensorial immersion, the fundamental interaction allowed the user to choose what to see. As we have seen, in cinematic VR, the choice centers around choosing the angle of view of an enveloping image, while spatial narrative also allows one to change one’s position in the scene and therefore also to change the perspective. Choosing what to see is not a passive act. The ability to act through a persona in the digital story (VR or not VR environments) can be expressed through the movement of the “camera,” the actions of movement of the “persona” or avatar, and in the possible changes in the plot according to the options of the character (Domínguez, 2013).

Other types of actions, such as manipulating objects within a simulated space, are not necessarily linked to the projection of a persona in the digital scene. Nevertheless, they are relevant to the quality of the interactive experience and therefore to the immersion: “Immersion is created by rewarding exploration with consistent, detailed variation, and undermined by poorly defined boundaries, gaps in coverage, and confusion in navigation” (Murray, 2012, p. 102).

Therefore, the lack of perceptual realism, and how elements behave within the scene, seems to be a logical barrier to immersion. But as research in the world of videogames has demonstrated, a certain degree of incoherence in the behavior of objects and things is admissible if immersion has previously been achieved (Cheng and Cairns, 2005). Or, put differently, if there is already a certain degree of immersion (for example, immersion in the story), this contributes to making up for other less successfully achieved aspects of perceptual realism.

In the “camera” movements, the actions in choosing the perspective of vision within an image without a frame, anticipating what points of interest could be within each scene is one of the challenges of the narrative constructions in VR. This is a field where there is still a lot to experiment and to learn. These focal points (Newton and Soukup, 2016) or points of interest (Brillhart, 2016) are calls on the attention, suggestions, and so every individual is free to follow them or not and feel attracted by them or not.

Newton and Soukup (2016) show in an experiment that the excess of focuses stresses the participants because they are afraid of missing something: “Looking is doing, and it requires a lot of work for the audience.” The excess of stimuli or points of interest appear not to contribute to immersion, although they are often understood in opposition, as Janet Murray warns: “Just as interactivity is often confused with a volume of interactivity, immersion is often confused with sensory stimulation” (Murray, 2012, p. 101). The decrease of “information clutter” is one of the requisites and benefits of working in VR (Bowman and McMahan, 2007). After their experiment, Newton and Soukup point out that for the best sensory experience fewer details are remembered but the emotions and abstractions are better perceived. “Perhaps being present and retaining story details are fundamentally at odds” (Newton and Soukup, 2016).

In a scene in which the participant is the one choosing what to see, the changes between scenarios or between scenes can be experienced as a loss of control over this freedom of vision. The editing used habitually in cinematic VR is an interruption of the experience within a fluid image, world image, or scenario image. Therefore, it is a temporary limitation to the freedom of camera movements, which are the way in which a user visually perceives the changes of position in a scene. In this spatial narrative, the participant moves around a scene but cannot be inside the skin of the person represented. He is an embodied visitor and at the same time an invisible “voyeur” in the scene. To be an invisible eyewitness is a narrative experience with great potential for journalistic reporting. It would resemble a “nobody’s shot” within a 360° setting, if such a concept existed.

Interactive agency is also realized when the changes in the plot derive from the decisions or routes presented to the character. This widely used videogame strategy has also arrived in journalistic work and documentaries, especially the so-called webdocs or docugames. In these, the user has a role as a character, someone who should investigate something.⁵ The options open to the visitor are realized through itineraries through the story that affect the factual narrative. The personification is, in summary, a digital suit for passing through a world, an interactive strategy to convert the audience into participants.

These types of options within narrative space have not arrived yet to journalistic experiences in VR. Providing the capacity to act in the journalistic story puts on the table the deontological debate over the limits of the simulation, which is not new and which seems to update itself with every new technological possibility. Although narrative experience may be augmented, it presents narrative difficulties in maintaining a level of interaction that does not modify the actions represented. Mel Slater describes Psi (to distinguish it from PI or “place illusion”) as the illusion of credibility in that which is happening despite knowing that it is real. It is achieved when “events in the virtual environment over which you have no direct control refer directly to you” (Slater, 2009, p. 3553). For example, if the avatar has a front shot, the impression of eye contact communication with the visitor is direct, which means that the character looks at it directly and one connects it to what is happening. Therefore, the visitor becomes visible within the scenario. In journalism, this would mean that the visitor is not an invisible spectator, but instead becomes someone inside the plot.

The experiences of immersive journalism using VR are right now the maximum expression of the first metaphor used by Richard Gerrig, which of feeling yourself transported to the narrative world. The visitor takes this journey due to the sensation of being inside the image, traveling through it, and hearing what is going on in the scene. Gerrig sums it up using the following steps: “1. Someone (‘the traveler’) is transported 2. By some means of transportation 3. as a result of performing certain actions. 4. The traveler goes some distance from his or her world of origin 5. which makes some aspects of the world of origin inaccessible. 6. The traveler returns to the world of origin, somewhat changed by the journey” (Gerrig, 1993, p. 10–11).

The traveler returns, changed by the experience. De la Peña emphasizes the emotional impact of an embodied experience: “When you get into a thing where you are really using your body in a space the experience is stronger, the response is real. People connect to the avatars through the screen, there is no doubt about it, but when there is body connection the response is much stronger” (Domínguez, 2013. p. 12). This real answer is what Gerrig defines as performing in storyworlds. The travelers infer what is not explicit and reply with emotions, thoughts, expectations, etc. When the immersion works, they return, transformed by the experience.

Discussion and Conclusion

In journalism, immersion has traditionally been understood as a research technique based on the premise that in order to tell the reality to the public it is necessary to first learn that reality in depth. In the digital world, the meaning of immersion is also associated with engaging forms of news storytelling. The emergence of technologies such as AR, VR and the influence of the video game resignify the concept of immersion.

Immersive journalism seeks to increase the empathy of the subject toward the reality explained through a first-person experience. Currently, immersive technologies give a sensory quality to the metaphors of literary immersion: the sensation of feeling oneself transported to the narrative world and being able to perform within it. To be inside the image, move around it, hear the sounds of the scene in three-dimensional quality, and choose what to see at every moment are the most developed expressive characteristics so far, and of these, immersive VR is their maximum component.

Although the field of VR can count on decades of evaluation of the elements of the system that increases the sensation of presence, we need more experiments that throw light on which of the elements of narrative construction in VR settings favor it. Although there are studies that point out the importance of media content above media form in achieving the sensation of presence, it is necessary that these can be carried out on journalistic stories. In this way, we would be better able to evaluate which elements of the system have more importance. For example, in the case of sound, the experiments of Nonny de la Peña underline the relevance of using real audio as a way of transporting the user to the time and place of the events. The tradition in other spheres, such as the audiovisual or the videogame, reinforces the idea of its immersive value. The question is if it is above other characteristics of the system (field of view, stereoscopy and tracking level) valued as of greater importance by some VR experts. As well as sound, it would be useful to re-evaluate other elements of the journalistic content form. In any case, the spatialization of sound is currently one of the challenges in the construction of this grammar.

In the case of cinematic VR, as there is no possibility for the user to move around the scene, there is no doubt that the quality of the image and sound is fundamental. In this case, the editing of the scenes implies a challenge to immersion as it interrupts the freedom of visual exploration of the scene. It seems clear that the immersive language requires formulas that can make the leap from scenes proper to the audiovisual heritage to the transition between scenes.

The ability to choose what you see in a surrounding scene is understood to be an interactive activity. Currently, the subject is invisible within the plot. The ability to act through a persona is a case of a vehicle, a digital suit to travel though the world, since the embodied visitor is not usually representing a real character. If being nobody, but still having an embodied experience, is already part of the grammar of immersive narrative, we can ask ourselves what possibilities open up to us to occupy different points of view, to create multiform stories, using Murray’s term (Murray, 1997) in journalism.

But besides, it remains to be seen if the possibilities of making the spectator visible in the scene, through resources as simple as eye contact with an avatar, make sense and offer possibilities within immersive journalistic storytelling. This facility implies including the user as an element inside the plot. Is it possible to construct a plot that includes interactive agency without modifying the reality of the acts that it seeks to represent? There is much ground for experimentation and for debate over the capacity for agency in immersive journalistic narrative.

Is the invisible eyewitness the universal vehicle for this type of narrative experience or are there possibilities for the visitor as a visible entity to be included as part of the structure of the story? If this is so, what are the challenges and the limits? Is the recreation of real events the most suitable format for spatial narrative or could other types of experiences be constructed? Is immersion in the image enough to consolidate cinematic VR as an attractive format for the audience? If it were not so, how and where could it evolve?

That this is a language taking its first steps is obvious in the current debate, which invites us to abandon the grammar of cinematography (Murray, 2016) and at the same time to get away from some elements of the videogame (Cizek, 2016). In the current boom, all types of productions coexist and frequently the quality of the narrative experience is very different. There is a lot to explore before this language consolidates its own forms. It is important to remember that the use of technology does not guarantee narrative immersion. This occurs when the expressive resources and technologies are put to the service of the narration; when, as in literary immersion, the person feels transported to a narrative world and able to act in it. In journalism, it is necessary to consider in which cases the immersive experience is the best vehicle to achieve the aim of making people care about what happens in the world.

Author Contributions

ED meets authorship criteria and she takes public responsibility for the content, design, analysis, writing, or revision of the manuscript.

Conflict of Interest Statement

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Footnotes

^Among others, he has been a psychiatric patient, worked on a car assembly line, a tabloid press reporter, and was an undocumented Turkish immigrant in Germany for 2 years, as he reported in the book Lowest of the Low (Wallraff, 1988).
^The works carried out by Immersive Journalism can be consulted at http://www.immersivejournalism.com. The most recent works of the Emblematic Group can be found at http://emblematicgroup.com.
^From his book, What Video Games Have to Teach Us About Learning and Literacy (Gee, 2003).
^Apart from verifying the documentation, De la Peña underlines the fact that they chose to use an interrogation in which the Bush administration itself admitted having tortured the prisoner.
^Examples of this trend are the works: Journey to the End of Coal, published in 2008 (http://www.honkytonk.fr/index.php/webdoc/); L’obésité est-elle une fatalité? (http://www.samuel-bollendorff.com/fr/the-big-issue-lobesite-est-elle-une-fatalite/); and Le Challenge (http://www.canalplus.fr/c-infos-documentaires/pid3400-c-le-challenge.html), both in 2009; Jour de vote, dans la peau d’un député (http://assemblee-nationale.curiosphere.tv/), in 2012; or Fort McMoney (http://www.fortmcmoney.com/#/fortmcmoney).

References

Aarseth, E. (2004). Genre trouble: narrativism and the art of simulation. In First Person New Media as Story Performance Narrativism and Game, Edited by P. Harrigan and N. Wardrip-Fruin, 45–55. Cambridge: MIT Press.