Jin-Hyun Park ¹ Yu-Bin Shin ² Dooyoung Jung ³ Ji-Won Hur ⁴ Seung Pil Pack ⁵ Heon-Jeong Lee ² Hwamin Lee ^6* Chul-Hyun Cho ^2*

• ¹ Department of Biomedical Informatics, College of Medicine, Korea University, Seoul, Republic of Korea
• ² Department of Psychiatry, College of Medicine, Korea University, Seoul, Republic of Korea
• ³ Graduate School of Health Science and Technology, Department of Biomedical Engineering, Ulsan National Institute of Science and Technology, Eonyang, Ulsan, Republic of Korea
• ⁴ School of Psychiatry, Korea University, Seoul, Republic of Korea
• ⁵ Department of Biotechnology and Bioinformatics, Korea University, Sejong, Republic of Korea
• ⁶ Department of Biomedical Informatics, Korea University, Seoul, Republic of Korea

Introduction: Machine learning (ML) is an effective tool for predicting mental states and is a key technology in digital psychiatry. This study aimed to develop ML algorithms to predict the upper tertile group of various anxiety symptoms based on multimodal data from virtual reality (VR) therapy sessions for social anxiety disorder (SAD) patients and to evaluate their predictive performance across each data type.Methods: This study included 32 SAD-diagnosed individuals, and finalized a dataset of 132 samples from 25 participants. It utilized multimodal (physiological and acoustic) data from VR sessions to simulate social anxiety scenarios. This study employed extended Geneva minimalistic acoustic parameter set for acoustic feature extraction and extracted statistical attributes from time series-based physiological responses. We developed ML models that predict the upper tertile group for various anxiety symptoms in SAD using Random Forest, extreme gradient boosting (XGBoost), light gradient boosting machine (LightGBM), and categorical boosting (CatBoost) models. The best parameters were explored through grid search or random search, and the models were validated using stratified crossvalidation and leave-one-out cross-validation.The CatBoost, using multimodal features, exhibited high performance, particularly for the Social Phobia Scale with an area under the receiver operating characteristics curve (AUROC) of 0.852. It also showed strong performance in predicting cognitive symptoms, with the highest AUROC of 0.866 for the Post-Event Rumination Scale. For generalized anxiety, the LightGBM's prediction for the State-Trait Anxiety Inventory-trait led to an AUROC of 0.819. In the same analysis, models using only physiological features had AUROCs of 0.626, 0.744, and 0.671, whereas models using only acoustic features had AUROCs of 0.788, 0.823, and 0.754.This study showed that a ML algorithm using integrated multimodal data can predict upper tertile anxiety symptoms in patients with SAD with higher performance than acoustic or physiological data obtained during a VR session. The results of this study can be used as evidence for personalized VR sessions and to demonstrate the strength of the clinical use of multimodal data.