Detection and Estimation of Working Memory States and Cognitive Functions Based on Neurophysiological Measures

Editorial

30 October 2018

Editorial: Detection and Estimation of Working Memory States and Cognitive Functions Based on Neurophysiological Measures

Felix Putze

Christian Mühl

Fabien Lotte

Stephen Fairclough

and

Christian Herff

3,641 views

3 citations

Editors

Felix Putze

University of Bremen

Christian Mühl

German Aerospace Center (DLR)

Fabien Lotte

Institut National de Recherche en Informatique et en Automatique (INRIA)

Christian Herff

Maastricht University

Stephen Fairclough

Liverpool John Moores University

Impact

Average number of game deaths per minute and average score per minute per experimental condition. Error bars represent standard deviation. *p < 0.05.

Original Research

17 July 2018

Dynamic Threshold Selection for a Biocybernetic Loop in an Adaptive Video Game Context

Elise Labonte-Lemoyne

, 4 more and

Pierre-Majorique Léger

4,408 views

13 citations

Categorization of the trials based on the subjects’ source judgments (SC, source correct; SI, source incorrect; CR, correct rejection; M, miss; FA, false alarm) and subjective ratings (RS, remember source; RO, remember other; F, familiar; MN, maybe new; SN, sure new).

Original Research

10 July 2018

Single-Trial EEG Analysis Predicts Memory Retrieval and Reveals Source-Dependent Differences

Eunho Noh

, 3 more and

Virginia R. de Sa

9,487 views

20 citations

P300 topography and waveforms in ALS (Experimental Group) and Control group. Traces in the middle panel represent the grand average of the difference between target and non-target electroencephalography (EEG) amplitude as a function of time (interval between 0 = stimulus onset and 1000 ms) recorded for ALS (n = 13 patients; solid line) and Control group (n = 13 subjects; dotted line) during the brain-computer interface (BCI) session, over different electrode positions (left labels). The maps represent the scalp topographical distribution in Control (left) and ALS (right) group of the P3 and N2 centroid latency (grand average) over Fz, Cz and Pz and over PO7, PO8 and Oz respectively.

Original Research

28 May 2018

On the Relationship Between Attention Processing and P300-Based Brain Computer Interface Control in Amyotrophic Lateral Sclerosis

Angela Riccio

, 6 more and

Febo Cincotti

5,499 views

20 citations

Original Research

17 May 2018

In silico vs. Over the Clouds: On-the-Fly Mental State Estimation of Aircraft Pilots, Using a Functional Near Infrared Spectroscopy Based Passive-BCI

Thibault Gateau

, 1 more and

Frédéric Dehais

Illustration of the fNIRS based inference system. Pre-recorded ATC messages were sent to the pilot (1). The pilot's prefrontal activity was measured with a fNIRS device (2). Output measures (3) were MACD-filtered and synchronized with the temporal design of the trial (4). When all of the required data were available for the trial, a request was sent to the pilot's classifier to assess the WM load of the trial (5).

There is growing interest for implementing tools to monitor cognitive performance in naturalistic work and everyday life settings. The emerging field of research, known as neuroergonomics, promotes the use of wearable and portable brain monitoring sensors such as functional near infrared spectroscopy (fNIRS) to investigate cortical activity in a variety of human tasks out of the laboratory. The objective of this study was to implement an on-line passive fNIRS-based brain computer interface to discriminate two levels of working memory load during highly ecological aircraft piloting tasks. Twenty eight recruited pilots were equally split into two groups (flight simulator vs. real aircraft). In both cases, identical approaches and experimental stimuli were used (serial memorization task, consisting in repeating series of pre-recorded air traffic control instructions, easy vs. hard). The results show pilots in the real flight condition committed more errors and had higher anterior prefrontal cortex activation than pilots in the simulator, when completing cognitively demanding tasks. Nevertheless, evaluation of single trial working memory load classification showed high accuracy (>76%) across both experimental conditions. The contributions here are two-fold. First, we demonstrate the feasibility of passively monitoring cognitive load in a realistic and complex situation (live piloting of an aircraft). In addition, the differences in performance and brain activity between the two experimental conditions underscore the need for ecologically-valid investigations.

17,714 views

99 citations

(A) In the number discrimination task, participants pressed the assigned button after judging the size of the number as more than five or less. A monetary reward (reward condition) or reaction speed (non-reward condition) was presented as a feedback stimulus. In the control condition, the feedback was a random number. (B) The feedback value increased when the button was pressed faster than a criterion, whereas it became smaller when the button was pressed slower than the criterion. (C) Sequence of the stimulus sessions. The task comprised five sessions and each session included three blocks. Participants practiced 20 trials before starting the task. (D) Electrode positions.

Original Research

31 January 2018

Hiroyuki Takayoshi

, 1 more and

Shuhei Yamaguchi

3,172 views

6 citations

Structure of the experimental session. (A) Overall Flow of the experiment with the 8 scenarios. (B) Detailed trial for the cruise and landing phases. (C) Upper view of the plane trajectory. The starting position was pseudo randomly placed on the blue arc. The blackline delimited the cruise from the landing phase where the pilot deactivate or not the autopilot regarding the condition.

Original Research

25 January 2018

Detecting Pilot's Engagement Using fNIRS Connectivity Features in an Automated vs. Manual Landing Scenario

Kevin J. Verdière

, 1 more and

Frédéric Dehais

8,951 views

57 citations

Original Research

12 January 2018

Different Topological Properties of EEG-Derived Networks Describe Working Memory Phases as Revealed by Graph Theoretical Analysis

Jlenia Toppi

, 5 more and

Donatella Mattia

7,994 views

41 citations

Box plots showing median values for frontal theta activity, parietal alpha activity and Frontal Alpha Asymmetry (FAA). Blue boxes show frontal theta activity (A), parietal alpha activity (B) and FAA (C) for the 1-back conditions. Green boxes show frontal theta activity (A), parietal alpha activity (B) and FAA (C) for the 2-back conditions. Median values are indicated by black horizontal lines within the boxes. Top and bottom borders of the boxes represent the middle 50% of the data. Whiskers represent the smallest and largest values not classified as outliers (between 1.5 and 3 times the height of the boxes) or extreme values (more than three times the height of the boxes). Circles indicate outliers. Please note the strong variability in the data due to large inter-individual differences.

Original Research

19 December 2017

Electroencephalography Based Analysis of Working Memory Load and Affective Valence in an N-back Task with Emotional Stimuli

Sebastian Grissmann

, 3 more and

Peter Gerjets

11,349 views

55 citations

Original Research

08 August 2017

Detecting and Quantifying Mind Wandering during Simulated Driving

Carryl L. Baldwin

, 4 more and

James S. Higgins

Mind wandering is a pervasive threat to transportation safety, potentially accounting for a substantial number of crashes and fatalities. In the current study, mind wandering was induced through completion of the same task for 5 days, consisting of a 20-min monotonous freeway-driving scenario, a cognitive depletion task, and a repetition of the 20-min driving scenario driven in the reverse direction. Participants were periodically probed with auditory tones to self-report whether they were mind wandering or focused on the driving task. Self-reported mind wandering frequency was high, and did not statistically change over days of participation. For measures of driving performance, participant labeled periods of mind wandering were associated with reduced speed and reduced lane variability, in comparison to periods of on task performance. For measures of electrophysiology, periods of mind wandering were associated with increased power in the alpha band of the electroencephalogram (EEG), as well as a reduction in the magnitude of the P3a component of the event related potential (ERP) in response to the auditory probe. Results support that mind wandering has an impact on driving performance and the associated change in driver’s attentional state is detectable in underlying brain physiology. Further, results suggest that detecting the internal cognitive state of humans is possible in a continuous task such as automobile driving. Identifying periods of likely mind wandering could serve as a useful research tool for assessment of driver attention, and could potentially lead to future in-vehicle safety countermeasures.

24,531 views

172 citations

Effect of workload on fNIRS oxy-Hb. (A) Average fNIRS oxy-Hb at each workload level; (B) ANOVA F- ratio statistics and η2 effect size of workload effect; (C) Average oxy-Hb for each optode. Highlighted optodes showed a significant workload effect (optode 5, 7, 8, and 14, FDR q < 0.05). Error bars illustrated the bootstrapped 95% confidence interval. Interpretation of the effect size η2 are 0.02 = small, 0.13 = medium, and 0.26 or greater is large (Bakeman, 2005).

Original Research

27 July 2017

Multisubject “Learning” for Mental Workload Classification Using Concurrent EEG, fNIRS, and Physiological Measures

Yichuan Liu

, 1 more and

Patricia A. Shewokis

13,593 views

95 citations

Original Research

31 August 2017

Spectral Entropy Can Predict Changes of Working Memory Performance Reduced by Short-Time Training in the Delayed-Match-to-Sample Task

Yin Tian

, 5 more and

Yupan Shi

11,659 views

59 citations

Original Research

04 October 2017

Assessing the Depth of Cognitive Processing as the Basis for Potential User-State Adaptation

Irina-Emilia Nicolae

, 1 more and

Benjamin Blankertz

5,160 views

14 citations

Original Research

14 July 2017

Measuring Mental Workload with EEG+fNIRS

Haleh Aghajani

, 1 more and

Ahmet Omurtag

We studied the capability of a Hybrid functional neuroimaging technique to quantify human mental workload (MWL). We have used electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) as imaging modalities with 17 healthy subjects performing the letter n-back task, a standard experimental paradigm related to working memory (WM). The level of MWL was parametrically changed by variation of n from 0 to 3. Nineteen EEG channels were covering the whole-head and 19 fNIRS channels were located on the forehead to cover the most dominant brain region involved in WM. Grand block averaging of recorded signals revealed specific behaviors of oxygenated-hemoglobin level during changes in the level of MWL. A machine learning approach has been utilized for detection of the level of MWL. We extracted different features from EEG, fNIRS, and EEG+fNIRS signals as the biomarkers of MWL and fed them to a linear support vector machine (SVM) as train and test sets. These features were selected based on their sensitivity to the changes in the level of MWL according to the literature. We introduced a new category of features within fNIRS and EEG+fNIRS systems. In addition, the performance level of each feature category was systematically assessed. We also assessed the effect of number of features and window size in classification performance. SVM classifier used in order to discriminate between different combinations of cognitive states from binary- and multi-class states. In addition to the cross-validated performance level of the classifier other metrics such as sensitivity, specificity, and predictive values were calculated for a comprehensive assessment of the classification system. The Hybrid (EEG+fNIRS) system had an accuracy that was significantly higher than that of either EEG or fNIRS. Our results suggest that EEG+fNIRS features combined with a classifier are capable of robustly discriminating among various levels of MWL. Results suggest that EEG+fNIRS should be preferred to only EEG or fNIRS, in developing passive BCIs and other applications which need to monitor users' MWL.

29,631 views

190 citations