Editorial: Virtual agents in virtual reality: design and implications for VR users

Editorial on the Research Topic
Virtual agents in virtual reality: design and implications for VR users

The present Research Topic has emerged in the context of the expansion of virtual communication and virtual social relations in our society. At the same time, the improvement of artificial intelligence technologies and autonomous agent designs has facilitated the populating of Virtual Environments (VE) not only with real users but also with more and more believable Virtual Human Agents (VHA) Demeure et al. (2011).

This Research Topic focuses on Virtual Reality (VR), the design of virtual agents, and their impact on VR users. Previous research has shown that the content provided to users in VR has an effect on them, mainly due to the immersion and sense of presence that VEs can induce Slater et al. (1996). The realism and believability of the environments designed for use in VR technologies are usually felt by users at least in terms of their visual perception and interaction capabilities, sometimes more. Indeed, other multisensory perceptions have been explored in the literature, such as touch Hoppe et al. (2020), sound Griol et al. (2019), or even smell Javerliat et al. (2022), which are constantly improving. This gives VR the ability to replicate real human behavior, in an ecological environment where real situations can be mimicked and elicit realistic responses from the user. These reactions can be assessed either through self-report or physiological measures during the experiment. Based on the social aspect of human life, our Research Topic aimed to collect research studies that propose VR experiments with social situations between users and virtual agents Von der Pütten et al. (2009). We wondered here which aspects of virtual agent design might influence the behavior of VR users, in what way, and how VHAs are perceived. Four papers have been published, all based on user experiments: two propose an evaluation of the effect of specific agent behaviors on VR user actions or reactions, and two design a new dataset of virtual agents and evaluate their perception by users.

Bönsch et al. propose an innovative approach to guidance to points of interest in a virtual environment through the social behavior of a virtual crowd of people. In this paper, two modalities are investigated, compared to a control condition with a virtual crowd without any guidance actions: 1) the crowd of agents actively supports the user toward the points of interest, and 2) the crowd implicitly guides the user toward the goals, as a more passive flow, such as walking in a certain direction. The results show that the active support modality is found to be more effective, but the authors note that the implicit flow of agents could be promising for certain conditions (large streets, “leader” user personality, etc.). Authors’ results support for the use of agents as guides in VR, through visual but also audio stimuli, reinforcing the interest of multisensory for the design of VR experiments with virtual agents.

Ban et al. lies in the kind of measurement the authors used. They opted for a physiological measurement of saliva to qualify the autonomic nervous system response of VR users facing stressful situations. In their experiment, VR users are sequentially exposed to virtual agents with a high-intensity stressor during customer service training. The authors are able to show similar results to previous studies, both in real life and in VR, which were previously conducted with a different user task than here - the Tester Social Stress Test. This demonstrates the ecological validity of their VR environment for eliciting physiological responses due to the amount of induced stress, and the accuracy of their assessment through the objective measures they used. Future work would be to evaluate which factors of the VHA’s behavior (gestures, voice tone) have a greater influence on the user’s stress responses.

Siehl et al. create a new publicly available dataset of animated virtual humanoid male characters by reusing and adapting a 2D face generator; they evaluate their emotional impact on users in terms of trustworthiness, valence, and arousal. Their results show that manipulations on the animations, which were expected to affect trustworthiness, accordingly, did indeed modify this emotional perception, as verified here by a first user study based on video stimuli. Furthermore, the authors conducted another study with additional audio stimuli to emphasize the increase or decrease in trustworthiness toward the users, which produced the same results, even with a stronger effect. Future improvements lie mainly in adapting the social context of such virtual character animations when presented to users, for a more contextual evaluation of this VHA dataset.

Do et al. article offers multiple contributions: 1) the creation of a new rich and diverse dataset of 210 fully rigged avatars, with high racial diversity and inclusion; 2) a precise and complete methodology for the creation of such a dataset (modeling process, recruitment strategy of VHA designers and community/race representatives, iterations, user evaluations); 3) knowledge about the perception of the dataset itself and more generally of virtual agents of different races by a very diverse panel of users. In this study, the identification of the race of VHAs by users who identified themselves as being of the same race as the VHA displayed is very accurate. Furthermore, Asian, Black, and White VHAs are recognized by all participants, whereas Hispanic and MENA VHAs are only validated as such by users who have identified themselves as belonging to these groups. Future work could also be done on diversity within groups, e.g., to represent the diversity of Hispanic profiles across Latin America, or MENA or NHPI cultural diversity, which can be directly reflected in their style, e.g., haircuts.

Through these articles, this Research Topic contributes to the dissemination of public virtual human agent datasets, to the explanation of virtual agent design methodology and evaluation approaches, and to the understanding of human-agent perception and interaction in VR. Future studies could contribute to the development of multisensory aspects in the design of virtual agent behavior, in line with the audio features explored here, and to the in-context user evaluation of social VHAs, through subjective and objective measures such as physiological data.

Author contributions

PR: Writing–original draft, Writing–review and editing. BB: Writing–review and editing. IP: Writing–review and editing. CP: Writing–review and editing.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Demeure, V., Niewiadomski, R., and Pelachaud, C. (2011). How is believability of a virtual agent related to warmth, competence, personification, and embodiment? Presence 20, 431–448. doi:10.1162/pres_a_00065

CrossRef Full Text | Google Scholar

Griol, D., Sanchis, A., Molina, J. M., and Callejas, Z. (2019). Developing enhanced conversational agents for social virtual worlds. Neurocomputing 354, 27–40. doi:10.1016/j.neucom.2018.09.099

CrossRef Full Text | Google Scholar

Hoppe, M., Rossmy, B., Neumann, D. P., Streuber, S., Schmidt, A., and Machulla, T.-K. (2020). “A human touch: social touch increases the perceived human-likeness of agents in virtual reality,” in Proceedings of the 2020 CHI conference on human factors in computing systems, 1–11. doi:10.1145/3313831.3376719

CrossRef Full Text | Google Scholar

Javerliat, C., Elst, P.-P., Saive, A.-L., Baert, P., and Lavoué, G. (2022). “Nebula: an affordable open-source and autonomous olfactory display for vr headsets,” in Proceedings of the 28th ACM symposium on virtual reality software and technology, 1–8. doi:10.1145/3562939.3565617

CrossRef Full Text | Google Scholar

Slater, M., Linakis, V., Usoh, M., and Kooper, R. (1996). “Immersion, presence and performance in virtual environments: an experiment with tri-dimensional chess,” in Proceedings of the ACM symposium on virtual reality software and technology - vrst '96., 163–172. doi:10.1145/3304181.3304216

CrossRef Full Text | Google Scholar

Von der Pütten, A., Krämer, N. C., and Gratch, J. (2009). “Whos there? can a virtual agent really elicit social presence,” in The 12th annual international workshop on presence.

Google Scholar