AUTHOR=Henkel  Florian , Widmer  Gerhard TITLE=Real-Time Music Following in Score Sheet Images via Multi-Resolution Prediction JOURNAL=Frontiers in Computer Science VOLUME=3 YEAR=2021 URL=https://www.frontiersin.org/journals/computer-science/articles/10.3389/fcomp.2021.718340 DOI=10.3389/fcomp.2021.718340 ISSN=2624-9898 ABSTRACT=

The task of real-time alignment between a music performance and the corresponding score (sheet music), also known as score following, poses a challenging multi-modal machine learning problem. Training a system that can solve this task robustly with live audio and real sheet music (i.e., scans or score images) requires precise ground truth alignments between audio and note-coordinate positions in the score sheet images. However, these kinds of annotations are difficult and costly to obtain, which is why research in this area mainly utilizes synthetic audio and sheet images to train and evaluate score following systems. In this work, we propose a method that does not solely rely on note alignments but is additionally capable of leveraging data with annotations of lower granularity, such as bar or score system alignments. This allows us to use a large collection of real-world piano performance recordings coarsely aligned to scanned score sheet images and, as a consequence, improve over current state-of-the-art approaches.