Michael Vallance
Phillip A. Towndrow- 1Department of Media Architecture, Future University Hakodate, Hakodate, Japan
- 2Centre for Research in Pedagogy and Practice, National Institute of Education, Nanyang Technological University, Singapore, Singapore
The development and relative affordability of Virtual Reality in recent years have provided opportunities to experience representations of both concrete and abstract situations; from nuclear engineering to particle physics, art galleries to three-dimensional prehistoric paintings, person-to-person communication to artificial agent collaboration, and 360-degree journalism to animated movies. Yet, it still remains challenging for participants to create personal narratives within a virtual world beyond that structured by its original designers. Setting aside technological considerations, we attribute this limitation largely to a restricted conceptualization of time and space that is fixed to present events, emotions and experiences. Consequently, Virtual Reality scenarios, as immersive and plausible as they might be, are nonetheless prone to a thin and static view of the (virtual) world where growth and experiential learning are not always possible or privileged. In this Perspective we propose a recasting of Virtual Reality that combines novelistic storytelling in the physical world with “narrative storyliving” as a mechanism for meaning-making within and across large dialogic arenas. This involves us drawing on ideas from the Russian philosopher and theorist, Mikhail Bakhtin, relating to the literary artistic chronotope. Ultimately, we intend to advance the discourse about what Virtual Reality is at present, and where it could go as seen through a critical literary lens.
Introduction
Virtual Reality (VR) is a three-dimensional, computer-generated, veridical environment. Experiencing a narrative in VR requires immersion [“the degree which the range of sensory channel is engaged by the virtual simulation” (Kim and Biocca 2018)], presence [the engagement of one’s sense of being in the virtual world so that the experience becomes “organic, user-driven, and different for everyone” (Bailenson, 2018, p. 223)], and perceived realism “ the user’s individual judgment about the degree of realism of the VE, in terms of 1) virtual objects, sounds and scenes, 2) credibility and plausibility of the story and its characters, and 3) naturalness and ease of the interaction with the VE” (Weber et al., 2021). The experience is perceptual where the sensory system identifies events and objects, and the brain-body system automatically reacts to environmental reorganizations. However, at present, the cognitive system responds with a conclusion that VR is simply an illusion (Slater and Sanchez-Vives, 2016), and a fully immersed narrative experience has yet to be achieved.
In VR the act of narrating a story “opens possibilities for engagement in interpretative processes and relational interactions” (Astiz, 2020, p.1) and provides participants “a venue to feel, empathize, and process experiences, both their own and of others” (ibid). Opportunities for different forms of narratives are furthered by voluntarily appropriating visual-spatial intelligence so that the passive recipient can effectively become an active contributor to the story (Bucher, 2018; Damiani, 2016; Lee et al., 2021). Research of news reporting using spherical frame or volumetric video in VR, for example, found among participants that “senses of being-there, interaction, and realism mediated the relationship between storytelling medium and reader perceptions of credibility, story recall, and story-sharing intention” (Sundar et al., 2017, p.672). Subsequent embodiment and presence shift the perspective of the participants ‘into’ the story: storytelling becomes storyliving.
Storytelling and Storyliving in Virtual Reality
Storytelling in VR may not yet generate a fluid, context independent, interactive narrative, but may unfavorably appear as maze-like options to pre-defined iterative outcomes. Like traditional storytelling, the VR storyline may feel scripted: the storyteller personalizes the context ‘for’ the audience, and ‘guides’ the spectator in the immersion (Gayet, 2020). VR game narratives, for instance, allow the gamer to wander throughout a game space, but the inherent storytelling remains uneconomical (Bailenson, 2018) as rewards are either not worthy of the effort, or may require sustained endeavors. In addition, kaleidoscopic narratives, which consist of dispersed story sub-plots, entice the viewer or reader to passively engage in pre-empting surprising but formulaic connections (Murray, 2017). Popular TV dramas such as 24 and Game of Thrones exemplify this narrative form, respecting the intellectual maturity of television viewers to knit and associate characters and storylines as the overall, prescribed chronicle of events unfold. These developments of storyworlding [i.e., a mix of gameplay and linear narrative (Bailenson, 2018)] and storification [i.e., the process through which a participant engages and internalizes a story as a narrative experience (Aylett and Louchart, 2007)] champion a discovery narrative where the storyline is “embedded in the process of the group creating it so that it can be endemic and organic to the process of participants experiencing it” (Bailenson, 2018, p.225).
Storyliving in VR may be judged as participatory drama where the participants play an essential role in their lived experiences, behaviors, reactions, and outcomes. As there is intimacy and empathy (Rubin et al., 2016), the participant experiences oneself as the author of a personal story: “human brains are narrative-generating machines and selves are the protagonists of the narratives they generate” (Schechtman, 2011). Consequently, when immersed in a VR storyliving experience, the emphasis is not necessarily “on” the narrative but the psychological process through which the participant engages “with” and “within” the narrative, and as a consequence “internalizes it as a narrative experience” (Aylett and Louchart, 2007, p.117). The storytelling branching narrative “can be seen as a minimally interactive form of participation” (ibid, p.120), but the storyliving narrative consists of unfolding layers, and is active, sometimes interactive, immersive and undetermined. In storyliving the participant “becomes” the story. (Rubin et al., 2016). Storyliving has been portrayed as immersive journalism (de la Peña et al., 2010), interactive journalism (Maschio, 2017), or simply as a marketing tool (Greenwald, 2017). However, these are limited perspectives on what might be possible with future technology advancements in VR. The external authorship used to convey emotional expression or communicate vulnerabilities of the subjects being witnessed or reported in, for example, Clouds over Sidra (Milk and Gabo, 2015) and The Displaced, (Silverstein, 2015), dilutes the storyliving concept.
To engage in VR storyliving, the presence and communication of interlocutors in VR need to go beyond witnessing, and become personalized by the virtual circumstance in order to subsequently dispense a deliberate experience. Although there may be various approaches to advance such an occurrence, in this Perspective it is posited that the use of AI techniques is one way to facilitate the transformation. For instance, the dynamic interactions and personalities of artificial primary characters have previously been utilized to create narratives in Mixed Reality storytelling (Cavazza et al., 2003). Inevitably, the merging of real and virtual intelligences will advance the participant behaviorally and cognitively so that they will be able to design their own experiences; purposely re-designing prior, during, or after the VR episode. For content manipulation in VR there is already a shift from axiomatic modeling to data-driven modeling based on deep neural networks (Wang et al., 2020). Deep learning is automated predictive analytics; a type of machine leaning and AI, with knowledge gained from increasingly hierarchical complexity and abstraction. Bailenson (2018) states, “With good AI, the narratives and stories would emerge from the interactions of individuals” (p.225), confirming the notion that iterations progress from storytelling, through immersion, presence and believability, to storyliving. Consequently, time becomes developmental as events unfold due to ongoing affects or actions. The actions and interactions in the present may initiate a response in the future; a dialogue, for instance, is not chronologically time dependent but dependent upon events, actions and exigency. And when re-entering the VR space, the participant will not have the same experience. Subsequently, VR storyliving becomes a shared, social experience in a multi-layered contingency of time, place, participant and action.
Experiences in a virtual space are known to transfer over into subsequent interactions in real-world, face-to-face situations (Yee and Bailenson, 2007; Fox et al., 2013). Traversable interfaces, for instance, provide “the illusion that virtual and physical worlds are joined together and that participants can physically cross from one to the other” (Koleva et al., 2000). There is a nested relation to the participants’ subjectivity in the real world. In other words, in real-world storytelling there is a human-to-nature relation where perceptions are influenced by the natural environment impacting the participants. On the other hand, in VR storyliving the nested relation is between the participants and the VR environment. Studies show that neither the virtual nor the physical self can ever truly be liberated from the other; what we learn in one body is shared with other bodies we inhabit, whether virtual of physical (Yee and Bailenson, 2007). Therefore, within VR storyliving there is a transaction between the technology and the human participant [known as alterity (i.e., a state of being Other)], and so it may be intriguing or disturbing that virtual VR avatars can shape real-world participant behaviors in and out of virtual worlds (Bailenson, 2018).
To summarize, Aylett and Louchart (2003) position VR as a valid, unauthored, narrative form through the dimensions of contingency (i.e., how far time and space of the narrative is contingent on real time and space), presence (i.e., how far the participant or observer shares the time and space of the narrative), interactivity (i.e., how far the participant or observer interacts with the story), and narrative representation (i.e., the characteristic form of the narrative). In these narrative dimensions of VR storyliving the autobiographical memory is subsumed in an artificial environment to focus upon self and one’s actions “within” the narrative, distinct from real-life where memory is influenced by present-day, real-world stimuli. This can occur as, depending upon the 3D space design and implementation, together with the inclusion of invasive and non-invasive technologies (Clark, 2020) facilitating an “extended mind” (Clark and Chalmers, 1998), time in VR is not necessarily chronological but developmental where VR displays, and even enforces, specific time and place on the participant through immersion, presence and believability. Even so, though developmental, the participatory nature of the nested relationship of narrative storyliving in VR yields the impression, for the actor participants, of engaging in real world chronological time, even if immersed in a virtual 16th century England or 23rd century Mars.
In conclusion, it is acknowledged that the concept of VR storyliving presents a narrative paradox in which “pre-authored plot structures conflict with the freedom of action and interaction characteristic of the medium of real-time interactive graphical environments” (Aylett and Louchart, 2007, p.116). To progress, a different view of time and space which might help in directing developed narratives, and a new way of thinking about immersive stories, are needed.
The Case for Narrative Storyliving in Virtual Reality
We propose the notion and practice of narrative storyliving in VR as a way of understanding the intersecting space between storytelling and storyliving (Figure 1). Narrative storyliving is an emergent narrative where stories are created and experienced “using interaction between intelligent synthetic characters as a generative mechanism” (Aylett and Louchart, 2007, p.1). We frame narrative storyliving in VR as an exploration of the mutually-informing interface between the physical and virtual worlds leading to a consideration of what this would allow in terms of developing VR as an arena of cognitive, social and emotional learning that is personal, historical and consequential.
FIGURE 1. Narrative storyliving in VR.
The key moves in our case begin by positioning storytelling as an intentionally dynamic, socially-orientated act of meaning-making composed of interlaced discourses between stories, storytellers and storytelling. Drawing on the work of Russian literary theorist, Mikhail Bakhtin, in speech genres and architectonics (how entities in social time and space are composed), we position storytelling as essentially “chronotopic” (from the Greek chronos-time and topos-space), that is, it is always situated, axiological, and experienced in specific times, places and locations. What does this entail?
Stories abound and are heterogeneous. Further, their types (narratives, recounts and expositions) and purposes have changed and developed (albeit erratically) over time. For instance, Bakhtin (1981) traces the historical development of the European novel beginning with Greek Romances where individuals are private and their exploits are without personal, social or political significance, to the rich—characteristically grotesque—polemical landscapes of the French Renaissance humanist François Rabelais in Gargantua and Pantagruel (Rabelais and Screech, 2006). In Bakhtin’s literary artistic chronotope, seen as an instance of novelistic discourse, specific emphasis goes towards what happens in our lives and to others. Events matter as we respond to and become dialogically involved with movements in time, space, plot and history (Bakhtin, 1981).
Since its initiation and development in the 1930s, the concept of the literary artistic chronotope has influenced literature studies significantly (Bemong et al., 2010). For example, scholars have used it to understand and interpret what we might term “a sense of humanity” through the lens of temporality and the way we experience life events psychologically, morally and agentically. We see an apt example of this in Dostoyevsky’s and garnett, (2001), where the protagonist’s mind is constantly racked by his murderous conscience and guilt-ridden forebodings that lead to his inevitable undoing.
We envisage prospective applications of the literary artistic chronotope as realizations of narrative storytelling: cognitive, social and emotional learning that is personal, historical and consequential. In Narrative Storyliving there are action portals enabling the storyteller to teleport to a synthetic representation of real-world past events. Contextualized by motivations of self-reflection, these events may be autobiographic re-creations of prior experiences and memories. The deep sense of embodiment allows the participant to transfer their current emotions, behaviors and thoughts to subsequently reflect upon past actions. The action portals thus masquerade as autobiographic memories. The participant can choose to be a spectator of these captured, possibly fragmented, memories, or take action to impact these memories which, in turn, affect the present when the participant returns to the real-world. It is incumbent on Deep Learning in VR to facilitate these emergent behaviors in relation to the actions where the physical and virtual worlds will become blurred/corruptible, indistinct and ultimately inseparable; as opposed to distinct, mutually-informing, co-constructive and benign (Figure 1. Narrative Storyliving in VR).
Narrative Storyliving is thus not structured or pre-generated, but unfolds as the participant experiences the emerging, semi-authored virtual environment, providing imaginary visions of personal histories. Time is “a subjective form or foundation of all experiences” (Kern, 2003, p.11), and in VR the duration of an event may be stretched in memory; ingeniously illustrated by Salvador Dali’s overhanging clock in Persistence of Memory. The chronotope is the place where “the knots of narrative are tied and untied” (Bakhtin, 1981, p. 250). The subsequent experiences are then subjective, contingent upon the story creator, and are time-independent process-oriented (see Aylett and Louchart, 2007). Narrative Storyliving in VR becomes a form of self-representation.
There is a primacy of the self and actions within immersive narrative storyliving. The synthetic participatory environment alters as the narrator chooses through thought, emotion, behavior or action and these metrics are captured, recorded and stored, providing further knowledge to the embodied intelligent virtual agent to multimodally alter the surroundings and allow the story creator to store as present knowledge and return to the past to influence memories. For instance, (Bahktin, 1981), states that a communicative utterance has both an author and an addressee. The addressee may be an immediate participant-interlocutor, a group, or an indefinite “other”. In the construction of narrative storyliving “the notion of addressivity allows us to consider how the social world may reflexively shape and be shaped by processes of multimedia composition” (Nelson and Hull, 2008, p.127). Subsequent interactions become dialogical, social, historical, memorable, emotional and consequential.
The interactive and dynamic environment has a central role in constructing the storyliving narrative as its structure depends upon the engagement, experiences and behaviors of the participant and by co-participants such as a virtual agent. The VR environment mediates this extrasomatic communication (Gardiner, 1995) between participants (real and virtual) and also within each participant, framing objective worlds (real and virtual) within subjective perspectives. By interacting with the memory, the reflective experience becomes time-independent learning, and an understanding of self is composed.
Conclusion
Set against the backdrop of current trends in VR storytelling and storyliving, we have drawn on the concept of the literary artistic chronotope as a way of expressing and understanding what could happen within and beyond virtual worlds. From this viewpoint, time is recast from the experiential here-and-now to a larger or greater realm. This has the potential for multiple perspectives, multiple voices and event- and activity-based developments that build on each other incrementally. Importantly, our proposal for narrative storyliving opens the possibility for VR participants to more actively design their learning about themselves and others in emergent (not pre-planned or scripted) ways. This is possible when we re-experience, re-imagine and re-tell the virtual and real worlds every time we enter them.
Data Availability Statement
The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.
Author Contributions
MV and PT contributed to the ideation and development of this article.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Astiz, M. F. (2020). Storytelling in the Higher Education Classroom: Why it Matters. Coll. Teach. 68, 187–188. doi:10.1080/87567555.2020.1785382
Aylett, R., and Louchart, S. (2007). Being There: Participants and Spectators in Interactive Narrative. Lecture Notes Comp. Sci. Vol. 4871, 117–128. doi:10.1007/978-3-540-77039-8_10
Aylett, R., and Louchart, S. (2003). Towards a Narrative Theory of Virtual Reality. Virtual Reality 7, 2–9. doi:10.1007/s10055-003-0114-9
Bailenson, J. (2018). Experience on Demand: What Virtual Reality Is, How it Works, and what it Can Do. New York: W.W. Norton & Company.
Bemong, N., Borghart, P., De Dobbeleer, M., Demoen, K., De Temmerman, K., and Keunen, B. (2010). Bakhtin’s Theory of the Literary Chronotope: Reflections, Applications, Perspectives. Belgium: Academia Press.
Cavazza, M., Martin, O., Charles, F., Mead, S. J., and Marichal, X. (2003). Interacting with Virtual Agents in Mixed Reality Interactive Storytelling. Lecture Notes Comp. Sci. Vol. 2792. doi:10.1007/978-3-540-39396-2_39
Clark, A., and Chalmers, D. (1998). The Extended Mind. Analysis 58 (1), 7–19. doi:10.1093/analys/58.1.7
Damiani, J. (2016). The Great Semantic divide: Virtual Reality vs. 360-degree Video. Avaliable at: https://uploadvr.com/virtual-reality-vs-360-degree-video- semantic-divide/.
de la Peña, N., Weil, P., Llobera, J., Giannopoulos, E., Pomés, A., Spanlang, B., et al. (2010). Immersive Journalism: Immersive Virtual Reality for the First-Person Experience of News. Presence 19, 291–301. doi:10.1162/PRES_a_00005
Dostoyevsky, F., and Garnett, T. C. (2001). Crime and Punishment. London: United Kingdom: Dover Publications.
Fox, J., Bailenson, J. N., and Tricase, L. (2013). The Embodiment of Sexualized Virtual Selves: the Proteus Effect and Experiences of Self-Objectification via Avatars. Comput. Hum. Behav. 29 (3), 930–938. doi:10.1016/j.chb.2012.12.027
Gardiner, W. L. (1995). Virtual Reality Cyberspace: Challenges to Communication Studies. Ijvr 1 (1), 22–27. doi:10.20870/ijvr.1995.1.1.2597
Gayet, M. (2020). Interviews: Jeremy Pouilloux. XR Must. Avaliable at: https://www.xrmust.com/xrmagazine/interview-accused-2-walter-sisulu/.
Greenwald, M. (2017). From Storytelling to VR 'Storyliving': Future Marketing Communications. Avaliable at: https://www.forbes.com/sites/michellegreenwald/2017/07/31/from-storytelling-to-vr-storyliving-future-marketing-communications/?sh=41bde9c035e2.
Kern, S. (2003). The Culture of Time and Space 1880 – 1910. Cambridge, MA: Harvard University Press.
Kim, G., and Biocca, F. (2018). Immersion in Virtual Reality Can Increase Exercise Motivation and Physical Performance. Lecture Notes Comp. Sci. Vol. 10910, 94–102. doi:10.1007/978-3-319-91584-5_8
Koleva, B., Schnädelbach, H., Benford, S., and Greenhalgh, C. (2000). “Traversable Interfaces between Real and Virtual Worlds,” in Proceedings of the CHI 2000 Conference on Human factors in computing systems. The Hague, The Netherlands. April 1-6, 2000 (DBLB). doi:10.1145/332040.332437
Lee, M. J. W., Georgieva, M., Alexander, B., Craig, E., and Richter, J. (2021). State of XR & Immersive Learning Outlook Report 2021. Walnut, CA: Immersive Learning Research Network.
Maschio, T. (2017). Storyliving: An Ethnographic Study of How Audiences Experience VR and what that Means for Journalists. Google News Lab. Avaliable at: https://newslab.withgoogle.com/assets/docs/storyliving-a-study-of-vr-in-journalism.pdf.
Milk, C., and Gabo, A. (2015). Clouds over Sidra. Avaliable at: https://docubase.mit.edu/project/clouds-over-sidra/.
Nelson, M. E., and Hull, G. A. (2008). Self-presentation through Multimedia: A Bakhtinian Perspective on Digital Storytelling. Bristol: Peter Lang.
Rubin, M., Neria, M., and Neria, Y. (2016). Fear, Trauma, and Posttraumatic Stress Disorder: Clinical, Neurobiological, and Cultural Perspectives. Interdiscip. Handbook Trauma Cult., 303–313. doi:10.1007/978-3-319-29404-9_18
Schechtman, M. (2011). “The Narrative Self,” in The Oxford Handbook of the Self (Oxford University Press), 394–418. doi:10.1093/oxfordhb/9780199548019.003.0018
Silverstein, J. (2015). The Displaced, the New York Times Magazine. Avaliable at: https://www.nytimes.com/2015/11/08/magazine/the-displaced-introduction.html.
Slater, M., and Sanchez-Vives, M. V. (2016). Enhancing Our Lives with Immersive Virtual Reality. Front. Robot. AI 3, 3. doi:10.3389/frobt.2016.00074
Sundar, S. S., Kang, J., and Oprean, D. (2017). Being There in the Midst of the Story: How Immersive Journalism Affects Our Perceptions and Cognitions. Cyberpsychology, Behav. Soc. Networking 20 (11), 672–682. doi:10.1089/cyber.2017.0271
Wang, M., Lyu, X.-Q., Li, Y.-J., and Zhang, F.-L. (2020). VR Content Creation and Exploration with Deep Learning: A Survey. Comp. Vis. Media 6, 3–28. doi:10.1007/s41095-020-0162-z
Weber, S., Weibel, D., and Mast, F. W. (2021). How to Get There when You Are There Already? Defining Presence in Virtual Reality and the Importance of Perceived Realism. Front. Psychol. 12. 628298. doi:10.3389/fpsyg.2021.628298
Keywords: chronotope, design, storyliving, storytelling, virtual reality
Citation: Vallance M and Towndrow PA (2022) Perspective: Narrative Storyliving in Virtual Reality Design. Front. Virtual Real. 3:779148. doi: 10.3389/frvir.2022.779148
Received: 18 September 2021; Accepted: 07 March 2022;
Published: 31 March 2022.
Edited by:
Regis Kopper, University of North Carolina at Greensboro, United StatesReviewed by:
Ana Luisa Sánchez Laws, UiT The Arctic University of Norway, NorwayLei Zhang, Arkansas Tech University, United States
Copyright © 2022 Vallance and Towndrow. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Michael Vallance, michael@fun.ac.jp