94% of researchers rate our articles as excellent or good
Learn more about the work of our research integrity team to safeguard the quality of each article we publish.
Find out more
ORIGINAL RESEARCH article
Front. Virtual Real.
Sec. Augmented Reality
Volume 6 - 2025 | doi: 10.3389/frvir.2025.1552321
Development and Evaluation of a Mixed Reality Music Visualization for a Live Performance based on Music Information Retrieval
Provisionally accepted- Institute of Sound and Vibration Engineering, Hochschule Duesseldorf University of Applied Science, 40476, Germany
The present study explores the development and evaluation of a mixed reality music visualization for a live music performance. Real-time audio analysis and crossmodal correspondences were used as design guidelines for creating the visualization, which was presented through a head-mounted-display. To assess the impact of the music visualization on the audience's aesthetic experience, a baseline visualization was designed, featuring the same visual elements but with random changes of color and movement. The audience's aesthetic experience of the two conditions (i.e, listening to the same song with different visualizations) was assessed using the Aesthetic Emotions Scale (AESTHEMOS) questionnaire. Additionally, participants answered questions regarding the perceived audiovisual congruence of the stimuli and questionnaires about individual musicality and aesthetic receptivity. The results show that the visualization controlled by real-time audio analysis was associated with a slightly enhanced aesthetic experience of the audiovisual composition compared to the randomized visualization, thereby supporting similar findings reported in the literature. Furthermore, the tested personal characteristics of the participants did not significantly affect aesthetic experience. Significant correlations between these characteristics and the aesthetic experience were observed only when the ratings were averaged across conditions. An open interview provided deeper insights into the participants' overall experiences of the live music performance. The results of the study offer insights into the development of real-time music visualization in mixed reality, examines how the specific audiovisual stimuli employed influence the aesthetic experience, and provides potential technical guidelines for creating new concert formats.
Keywords: music visualization1, mixed reality2, real-time audio analysis3, music information retrieval4, aesthetic experience5, concert formats6
Received: 27 Dec 2024; Accepted: 28 Feb 2025.
Copyright: © 2025 Erdmann, von Berg and Steffens. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
* Correspondence:
Matthias Erdmann, Institute of Sound and Vibration Engineering, Hochschule Duesseldorf University of Applied Science, 40476, Germany
Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.