Event Abstract

Back to Event

A method for cross-task mental workload classification based on brain connectivity

Georgios N. Dimitrakopoulos1, 2, Yu Sun1, Kevin Ardian1, Nitish V. Thakor1 and Anastasios Bezerianos1, 3*
  • 1 National University of Singapore, Singapore Institute for Neurotechnology (SINAPSE), Singapore
  • 2 University of Patras, Department of Electrical and Computer Engineering, Greece
  • 3 University of Patras, School of Medicine, Greece

Introduction Mental workload estimation is a significant neuroscience problem, since continuous hard task execution can lead to mental fatigue and consequently to errors in critical tasks such as driving or piloting (Borghini, 2014). There have been several successful applications based on EEG recordings, however they are limited in a single task, being unable to generalize to multitask experiments (Baldwin, 2012; Wang 2012; Ke, 2015). Cross-task classification is of great importance, taking into account that in real life task repetitions are not exactly the same as in a controlled experimental environment. Moreover, a factor hindering classification is variability among subjects, since the same task might induce different workload to each person due to their varying capabilities. In parallel, to study effectively the complex organization of the brain, network approaches have been employed recently in a variety of problems (Sporns, 2014; Dimitriadis 2015). In this work, we present a network based method for feature selection and classification of mental workload from two independent tasks. Our method uses functional connections as features, which offer a higher level perspective of the data, and relies on paired statistics to deal with subjective data labels such as difficulty. Materials and Methods A. Data and preprocessing The experimental protocol consists of two tasks, an Arithmetic Task (AT) and N-Back Task (NBT), each one comprising of an easy and a hard level version. In the arithmetic task, first an addition of two one-digit numbers for easy level or three-digit numbers for hard and next an answer are presented to the subject, which has to indicate if the answer is correct or not. The easy level of N-Back task is Zero-Back , requiring the subject to respond if current letter shown is ‘X’, while the hard one is Two-Back, requiring a response if the letter shown was the same as in the second previous iteration. In this experiment we recorded EEG data from 25 subjects using 64 scalp electrodes. Data were pre-processed, performing filtering from 1 to 40Hz and artefact removal via ICA. Then, signal was separated into alpha, beta, delta and theta frequency bands and power spectrum was calculated. Next, source localization to 80 cortical brain regions was performed according to Automatic Anatomical Labelling atlas (Tzourio-Mazoyer, 2002). Finally, a functional brain network was constructed by Pearson Correlation Coefficient on power series of source pairs. The connectivity values were used as features for classification, resulting in 3160 features for each subject and task. B. Cross-Task Classification We developed a method for cross task workload classification, where in training part we performed a feature ranking procedure to tackle subjectivity and in the test part we defined a score based on the selected features that reflects brain workload level. First, a paired t-test was conducted upon hard and easy samples for each feature and the t-statistic was used as feature weight. Features were ranked according to absolute value of weight, since it determines the feature ability to separate difficulty levels across tasks and subjects. Additionally, the sign of the weight determines the polarity of the feature; if it is positive harder tasks assume higher numerical values than easier ones and the opposite if negative. Then, feature ranking was optimized with cross-validation within the training set in order to reduce possible outlier effects and the rankings from all validation iterations were aggregated into a global ranking. Next, we selected the top features from global ranking, where optimum number of features was found by examining all values from 1 to 100. For each feature, a reference set of points was formed, using only the pairs (easy-hard) of training set samples that were properly aligned according to feature polarity. In the test phase, we assigned a value to each point of test set reflecting its relative position in comparison with the reference set. Finally, to assess workload level of a test sample, we defined a score by summing over features the product of relative position by the weight of each feature. This score is designed to reflect the amount of mental workload required from the subject performing the task, therefore a test sample is considered as correctly classified if hard condition assumes larger score than easy condition. To evaluate accuracy, data were split into train and test sets with leave one out cross-validation. Results and Discussion We used our method on combined data of AT and NBT, to test its efficiency to distinguish mental workload independently of task. We note that both tasks required working memory resources. We focused our analysis on theta frequency band as it has been observed power increment in relation to mental workload (Langer 2013; Sammer 2007). Our method achieved a satisfactory accuracy of 82% on cross-task workload classification, which was stable using 26 to 42 features. An important reason our method can perform accurately is the use of paired statistics instead of just dividing all data in two classes, which can deal with differences in workload level across individual subjects. Moreover, data during training were not treated as fully reliable, hence some samples not complying with the model were considered as outliers. Finally, we isolated the top 30 connectivity features and examined which regions they were connected to. According to literature, theta power increment is primarily observed in frontal areas, hippocampus and superior temporal areas, which are associated with functions such as working memory, encoding and monitoring (Sammer, 2007). The majority of the loci detected lie within these areas and specifically, the right inferior frontal gyrus area (ORBinf.R) appeared in 13 connections out of 30 and it has been associated with sustained attention tasks and working memory load (de Fockert, 2001; Just, 2007). Other orbitofrontal areas detected having at least 3 connections were ORBsup.R, ORBsup.L, ORBmid.L ORBinf.L and REC.R, which are involved in decision making (Bechara, 2000). We also detected hippocampus (HIP.L) which function is related with memory encoding and retrieval processes (Sammer, 2007). Similar functions have been attributed to other areas detected within temporal gyrus (TPOsup.R, TPOmid.R, ITG.L) (Rypma, 1999; Hampson, 2006).

References

Bechara, A., Damasio, H., & Damasio, A. R. (2000). Emotion, decision making and the orbitofrontal cortex. Cerebral cortex, 10(3), 295-307.
Baldwin, C. L., & Penaranda, B. N. (2012). Adaptive training using an artificial neural network and EEG metrics for within-and cross-task workload classification. Neuroimage, 59(1), 48-56.
Borghini, G., Astolfi, L., Vecchiato, G., Mattia, D., & Babiloni, F. (2014). Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neuroscience & Biobehavioral Reviews, 44, 58-75.
de Fockert, J. W., Rees, G., Frith, C. D., & Lavie, N. (2001). The role of working memory in visual selective attention. Science, 291(5509), 1803-1806.
Dimitriadis, S. I., Sun, Y. U., Kwok, K., Laskaris, N. A., Thakor, N., & Bezerianos, A. (2015). Cognitive workload assessment based on the tensorial treatment of EEG estimates of cross-frequency phase interactions. Annals of biomedical engineering, 43(4), 977-989.
Just, M. A., & Varma, S. (2007). The organization of thinking: What functional brain imaging reveals about the neuroarchitecture of complex cognition.Cognitive, Affective, & Behavioral Neuroscience, 7(3), 153-191.
Hampson, M., Driesen, N. R., Skudlarski, P., Gore, J. C., & Constable, R. T. (2006). Brain connectivity related to working memory performance. The Journal of neuroscience, 26(51), 13338-13343.
Ke, Y., Qi, H., Zhang, L., Chen, S., Jiao, X., Zhou, P., ... & Ming, D. (2015). Towards an effective cross-task mental workload recognition model using electroencephalography based on feature selection and support vector machine regression. International Journal of Psychophysiology, 98(2), 157-166.
Langer, N., von Bastian, C. C., Wirz, H., Oberauer, K., & Jäncke, L. (2013). The effects of working memory training on functional brain network efficiency.Cortex, 49(9), 2424-2438.
Rypma, B., & D’Esposito, M. (1999). The roles of prefrontal brain regions in components of working memory: effects of memory load and individual differences. Proceedings of the National Academy of Sciences, 96(11), 6558-6563.
Sammer, G., Blecker, C., Gebhardt, H., Bischoff, M., Stark, R., Morgen, K., & Vaitl, D. (2007). Relationship between regional hemodynamic activity and simultaneously recorded EEG‐theta associated with mental arithmetic‐induced workload. Human brain mapping, 28(8), 793-803.
Sporns, O. (2014). Contributions and challenges for network models in cognitive neuroscience. Nature neuroscience, 17(5), 652-660.
Tzourio-Mazoyer, N., Landeau, B., Papathanassiou, D., Crivello, F., Etard, O., Delcroix, N., ... & Joliot, M. (2002). Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroimage, 15(1), 273-289.
Wang, Z., Hope, R. M., Wang, Z., Ji, Q., & Gray, W. D. (2012). Cross-subject workload classification with a hierarchical Bayes model. Neuroimage, 59(1), 64-69.

Keywords: Mental Workload, brain connectivity, Classification, Cross-task Classification, EEG

Conference: SAN2016 Meeting, Corfu, Greece, 6 Oct - 9 Oct, 2016.

Presentation Type: Oral Presentation in SAN 2016 Conference

Topic: Oral Presentations

Citation: Dimitrakopoulos GN, Sun Y, Ardian K, Thakor NV and Bezerianos A (2016). A method for cross-task mental workload classification based on brain connectivity. Conference Abstract: SAN2016 Meeting. doi: 10.3389/conf.fnhum.2016.220.00002

Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.

The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.

Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.

For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.

Received: 29 Jul 2016; Published Online: 30 Jul 2016.

* Correspondence: Prof. Anastasios Bezerianos, National University of Singapore, Singapore Institute for Neurotechnology (SINAPSE), Singapore, Singapore, bezer@upatras.gr