Avatars for the Masses: Smartphone-Based Reconstruction of Humans for Virtual Reality

Timo Menzel^1*Erik Wolf²Stephan Wenninger¹Niklas Spinczyk¹Lena Holderrieth²Carolin Wienrich²Ulrich Schwanecke³Marc Erich Latoschik²Mario Botsch^1*

• ¹Technical University Dortmund, Dortmund, North Rhine-Westphalia, Germany
• ²Julius Maximilian University of Würzburg, Würzburg, Bavaria, Germany
• ³Hochschule RheinMain, Wiesbaden, Hesse, Germany

Realistic full-body avatars play a key role in representing users in virtual environments, where they have been shown to considerably improve important effects of immersive experiences such as body ownership and presence. Consequently, the demand for realistic virtual humans -and methods for creating them -is rapidly growing. However, despite extensive research into 3D reconstruction of avatars from real humans, an easy and affordable method for generating realistic and VR-capable avatars is still lacking: Existing methods are either limited to complex capture hardware and/or controlled lab environments, do not provide sufficient visual fidelity, or cannot be rendered at sufficient frame rates for multi-avatar VR applications. To make avatar reconstruction widely available, we developed Avatars for the Masses -a client-server-based online service for scanning real humans with an easy-to-use smartphone application that empowers even nonexpert users to capture photorealistic and VR-ready avatars. The data captured by the smartphone are transferred to a reconstruction server, where the avatar is generated in a fully automated process. Our advancements in capturing and reconstructing allow for higher-quality avatars even in less controlled in-the-wild environments. Extensive qualitative and quantitative evaluations show our method's avatars to be on par with the ones generated by expensive expert-operated systems.It also generates more accurate replicas in comparison to the current state-of-art in smartphonebased reconstruction, produces much less artifacts and provides a much higher rendering performance in VR in comparison to three representative neural methods. A comprehensive user study confirms similar perception results compared to avatars reconstructed with expensive expert-operated systems, and it underscores a sufficient usability of the overall system. To truly bring avatars to the masses, we will make our smartphone application publicly available for research purposes.