Kati Pettersson ^* Jaakko Tervonen Juuso Heininen Jani Mäntyjärvi

• VTT Technical Research Centre of Finland Ltd, Espoo, Finland

Developments in virtual reality technology have reached technical maturity to start shifting focus from the creation of devices capable of providing immersive virtual experiences to creating devices also capable of sensing and understanding the human. Engaging audiovisual stimuli, user movement scanning as well as biosensing integrated to a headset creates an unprecedented framework for studying human visual perception in controlled yet close-to-natural settings.Headsets, near eye displays, and future smart eyewear enable unobtrusive sensing of a variety of biosignals such as eye and head movements, pupil size, and blinks, heart rate, and brain activity, providing versatile information on human physiology and psychophysiology. Additionally, modern AI methods provide interpretations of multimodal data to obtain personalized estimations of the users' oculomotor behavior, visual perception, and cognitive state, and their possibilities extend to controlling, adapting, and even creating the virtual audiovisual content in real-time.This article proposes a visionary approach for visual perception and cognitive state estimation in virtual reality. We introduce ideas on using virtual content with head-area sensing to enable the estimation of the user's active vision and cognitive state as well as how embedded protocols, adaptive sampling, and artificial intelligence could improve the accuracy of the estimations. The primary focus in sensing lies on the eye and oculomotor parameters, as well as head movements, given their crucial role in visual perception. Finally, we outline the potential application domains.