Stress can be Detected During Emotion-evoking Smartphone Use: A Pilot Study Using Machine Learning

Lydia Helene Rupp^1*Akash Kumar²Misha Sadeghi²Lena Schindler-Gmelch¹Marie Keinert¹Bjoern M Eskofier^2,3Matthias Berking¹

• ¹Lehrstuhl für Klinische Psychologie und Psychotherapie, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
• ²Machine Learning and Data Analytics Lab, Faculty of Engineering, Friedrich-Alexander-University Erlangen-Nürnberg, Erlangen, Bavaria, Germany
• ³Translational Digital Health Group, Institute of AI for Health, Helmholtz Zentrum München - German Research Center for Environmental Health, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Bavaria, Germany

The detrimental consequences of stress highlight the need for precise stress detection, as this offers a window for timely intervention. However, both objective and subjective measurements suffer from validity limitations. Contactless sensing technologies using machine learning methods present a potential alternative and could be used to estimate stress from externally visible physiological changes, such as emotional facial expressions. Although previous studies were able to classify stress from emotional expressions with accuracies of up to 88.32%, most works employed a classification approach and relied on data from contexts where stress was induced. Therefore, the primary aim of the present study was to clarify whether stress can be detected from facial expressions of six basic emotions (anxiety, anger, disgust, sadness, joy, love) and relaxation using a prediction approach. To attain this goal, we analyzed video recordings of facial emotional expressions collected from n = 69 participants in a secondary analysis of a dataset from an interventional study. We aimed to explore associations with stress (assessed by the PSS-10 and a self-constructed item one-item stress measure). Comparing two regression machine learning models (Random Forest (RF) and XGBoost), we found that facial emotional expressions were promising indicators of subjective stress scores, with model fit the best when data from all six emotional facial expressions was used to train the model (One item stress measure: MSE (XGB) = 2.31, MAE (XGB) = 1.32, MSE (RF) = 3.86, MAE (RF) = 1.69; PSS-10: MSE (XGB) = 25.65, MAE (XGB) = 4.16, MSE (RF) = 26.32, MAE (RF) = 4.14)). XGBoost showed to be more reliable for prediction, with lower error for both training and test data. The findings provide further evidence that noninvasive video recordings can complement standard objective and subjective markers of stress. subjective stress may not only be detected through standard objective and subjective markers, but also through non-invasive smartphone video recordings.