Recognizing the State of Emotion, Cognition and Action from Physiological and Behavioural Signals

48.5K
views
55
authors
11
articles
Cover image for research topic "Recognizing the State of Emotion, Cognition and Action from Physiological and Behavioural Signals"
Editors
5
Impact
Loading...
Article Cover Image
10,631 views
11 citations
5,955 views
14 citations
Original Research
14 January 2022

In future conditionally automated driving, drivers may be asked to take over control of the car while it is driving autonomously. Performing a non-driving-related task could degrade their takeover performance, which could be detected by continuous assessment of drivers' mental load. In this regard, three physiological signals from 80 subjects were collected during 1 h of conditionally automated driving in a simulator. Participants were asked to perform a non-driving cognitive task (N-back) for 90 s, 15 times during driving. The modality and difficulty of the task were experimentally manipulated. The experiment yielded a dataset of drivers' physiological indicators during the task sequences, which was used to predict drivers' workload. This was done by classifying task difficulty (three classes) and regressing participants' reported level of subjective workload after each task (on a 0–20 scale). Classification of task modality was also studied. For each task, the effect of sensor fusion and task performance were studied. The implemented pipeline consisted of a repeated cross validation approach with grid search applied to three machine learning algorithms. The results showed that three different levels of mental load could be classified with a f1-score of 0.713 using the skin conductance and respiration signals as inputs of a random forest classifier. The best regression model predicted the subjective level of workload with a mean absolute error of 3.195 using the three signals. The accuracy of the model increased with participants' task performance. However, classification of task modality (visual or auditory) was not successful. Some physiological indicators such as estimates of respiratory sinus arrhythmia, respiratory amplitude, and temporal indices of heart rate variability were found to be relevant measures of mental workload. Their use should be preferred for ongoing assessment of driver workload in automated driving.

3,687 views
10 citations
Open for submission
Frontiers Logo

Frontiers in Digital Health

Emotional Intelligence AI in Mental Health
Edited by Panagiotis Tzirakis, Zixing Zhang, Georgios Rizos, Björn Wolfgang Schuller
Deadline
31 May 2025
Submit a paper
Recommended Research Topics
Frontiers Logo

Frontiers in Psychology

Understanding Social Signals: How Do We Recognize the Intentions of Others?
Edited by Jan P De Ruiter, Sebastian Loth
123.6K
views
47
authors
13
articles
Frontiers Logo

Frontiers in Psychology

Anticipatory Systems: Humans Meet Artificial Intelligence
Edited by Mu-Yen Chen, Edwin Lughofer, Jose De Jesus Rubio, Yenchun Jim Wu
129.5K
views
48
authors
15
articles
Frontiers Logo

Frontiers in Neurorobotics

Bridging the Gap between Machine Learning and Affective Computing
Edited by Zhen Cui, Abhinav Dhall, Xiaopeng Hong, Yong Li, Wenming Zheng, Yuan Zong
79.6K
views
65
authors
13
articles
Frontiers Logo

Frontiers in Neurorobotics

Deep Learning Techniques Applied to Affective Computing
Edited by Shiqing Zhang, Xiaopeng Hong, Xiaobai Li, Zhen Cui, Wenming Zheng
57.2K
views
48
authors
11
articles
Frontiers Logo

Frontiers in Psychology

Exploring the Emotional Landscape: Cutting-Edge Technologies for Emotion Assessment and Elicitation
Edited by Federica Marcolin, Alessia Celeghin, Enrico Vezzetti, Javier Marín-Morales
24.3K
views
50
authors
11
articles