Frédéric Dehais
Alex Lafont
Raphaëlle Roy
Stephen Fairclough- 1ISAE-SUPAERO, Université de Toulouse, Toulouse, France
- 2School of Biomedical Engineering, Science and Health Systems, Drexel University, Philadelphia, PA, United States
- 3School of Psychology, Liverpool John Moores University, Liverpool, United Kingdom
The assessment and prediction of cognitive performance is a key issue for any discipline concerned with human operators in the context of safety-critical behavior. Most of the research has focused on the measurement of mental workload but this construct remains difficult to operationalize despite decades of research on the topic. Recent advances in Neuroergonomics have expanded our understanding of neurocognitive processes across different operational domains. We provide a framework to disentangle those neural mechanisms that underpin the relationship between task demand, arousal, mental workload and human performance. This approach advocates targeting those specific mental states that precede a reduction of performance efficacy. A number of undesirable neurocognitive states (mind wandering, effort withdrawal, perseveration, inattentional phenomena) are identified and mapped within a two-dimensional conceptual space encompassing task engagement and arousal. We argue that monitoring the prefrontal cortex and its deactivation can index a generic shift from a nominal operational state to an impaired one where performance is likely to degrade. Neurophysiological, physiological and behavioral markers that specifically account for these states are identified. We then propose a typology of neuroadaptive countermeasures to mitigate these undesirable mental states.
Introduction
A study of mental workload is fundamental to understanding the intrinsic limitations of the human information processing system. This area of research is also crucial for investigation of complex teaming relationships especially when interaction with technology necessitates multitasking or a degree of cognitive complexity.
The Growth of Mental Workload
Mental workload has a long association with human factors research into safety-critical performance (Moray, 1979; O’Donnell and Eggemeier, 1986; Hancock and Meshkati, 1988; Hancock and Desmond, 2001; Wickens and Tsang, 2014; Young et al., 2015). Forty years have passed since the publication of the seminal collection edited by Moray (1979) and the study of mental workload remains an active topic in contemporary human factors research; a keyword search based on Google Scholar listed more than 200,000 articles published on the topic since 2000, see also Table 1 in Young et al. (2015). The significance of human mental workload for those technological trends that are forecast during the second machine age (Brynjolfsson and McAfee, 2014) guarantees its importance for human factors research in future decades.
The lineage of mental workload incorporates a number of theoretical perspectives, some of which precede the formalization of the concept itself. Early work linking physiological activation to the prediction of performance (Yerkes and Dodson, 1908; Duffy, 1962) was formalized into an energetical model of attentional resources (Kahneman, 1973) that emphasized a dynamic relationship between finite information processing capacity and variable cognitive demands (Norman and Bobrow, 1975; Navon and Gopher, 1979; Wickens, 1980). The descriptive quality of the early work on attentional resources was sharpened by cognitive models of control (Broadbent, 1971; Schneider et al., 1984; Shallice and Burgess, 1993). Hybrid frameworks that place cognitive processes within a resource framework have been hugely influential in the field, such as the multiple resource model (Wickens, 1984, 2002, 2008; Wickens and Liu, 1988) whereas others introduced agentic features, such as dynamic self-regulation and adaptation, within models of human performance (Hockey et al., 1986; Hockey, 1997). For instance, Hancock and Warm (1989)’s dynamic adaptive theory (DAT) postulates that the brain seeks resource homeostasis and cognitive comfort. However, environmental stressors can progressively shift individual’s adaptive abilities from stability to instability depending on one’s cognitive and psychological resources. The DAT is an extension of the Yerkes and Dodson inverted-U law, in a sense that very low (hypostress) and very high (hyperstress) task demands can both degrade the adaptability and consequently impair performance. All these perspectives are united by a characterization of the human information processing system as a finite resource with limited capacity (Kramer and Spinks, 1991).
Mental Workload Measurement
Research into the measurement of mental workload has outstripped the development of theoretical frameworks. Measures of mental workload can be categorized as performance-based, or linked to the process of subjective self-assessment, or associated with psychophysiology or neurophysiology. Each category has specific strengths and weaknesses (O’Donnell and Eggemeier, 1986; Wierwille and Eggemeier, 1993) and the sensitivity of each measurement type can vary depending on the level of workload experienced by the operator (De Waard, 1996). The development of multidimensional measures led inevitably to an inclusive framework for mental workload. The cost of this integration is dissociation between different measures of mental workload, e.g., Yeh and Wickens (1988), and an integrated workload concept that remains poorly defined from a psychometric perspective (Matthews et al., 2015).
There are a number of reasons that explain why mental workload is easy to quantify but difficult to operationalize. The absence of a unified framework for human mental workload, its antecedents, processes and measures has generated a highly abstract concept, loosely operationalized and supported by a growing database of inconsistent findings (Van Acker et al., 2018). The absence of a general explanatory model is complicated by the fact that mental workload, like stress and fatigue (Matthews, 2002), is a transactional concept representing an interaction between the capacities of the individual and the specific demands of a particular task. Within this transactional framework, mental workload represents a confluence between inter-individual sources of trait variability (e.g., skill, IQ, personality), intra-individual variation (e.g., emotional states, motivation, fatigue), and the specific configuration of the task under investigation (see also Table 2 in Van Acker et al., 2018).
For the discipline of human factors, the study of mental workload serves two primary functions: (a) to quantify the transaction between operators and a range of task demands or technological systems or operational protocols, and (b) to predict the probability of performance impairment during operational scenarios, which may be safety-critical. One challenge facing the field is delineating a consistent relationship between mental workload measurement and performance quality on the basis of complex interactions between the person and the task. The second challenge pertains to the legacy and utility of limited capacity of resources as a framework for understanding those interactions.
In the following sections, we detail some limitations of mental resources and advocate the adoption of a neuroergonomic approach (Sarter and Sarter, 2003; Parasuraman and Rizzo, 2008; Parasuraman and Wilson, 2008; Mehta and Parasuraman, 2013; Ayaz and Dehais, 2018) for the study of mental workload and human performance. The neuroergonomic framework emphasizes a shift from limited cognitive resources to characterizing impaired human performance and associated states with respect to neurobiological mechanisms.
Toward a Limit of the Theory of Limited Resources
The concept of resources represents a foundational challenge to the development of a unified framework for mental workload and prediction of human performance. The conception of a limited capacity for information processing is an intuitive one and has been embedded within several successful models, e.g., multiple resources (Wickens, 2002). But this notion has always been problematic because resources are a general-purpose metaphor with limited explanatory powers (Navon, 1984) that incorporate both cognitive processes (e.g., attention, memory) and energetical constructs (e.g., mental effort) in ways that are difficult to delineate or operationalize. The allegorical basis of resources almost guarantees an abstract level of explanation (Van Acker et al., 2018) that is accompanied by divergent (Matthews et al., 2015), and sometimes contradictory operationalizations (Yeh and Wickens, 1988; Annett, 2002).
For example, the theory of limited cognitive resources predicts that exposure to task demands that are sustained and demanding can impair performance due to resource depletion via self-regulation mechanisms at the neuron-level (i.e., local-sleep state theory, see Van Dongen et al., 2011) or compromise access to resources mechanisms (Borragan Pedraz and Peigneux, 2016). However, this type of explanation fails to clarify why non-challenging tasks, such as passive monitoring (Matthews et al., 2002, 2010) can promote episodes of mind wandering whereby attention drifts from task-related to task-irrelevant thoughts (Smallwood et al., 2008; Durantin et al., 2015; Smallwood and Schooler, 2015). Although some propositions, such as the theory of “malleable resources” (Young and Stanton, 2002), have intuited this paradox, this theory is at a highly descriptive level and remains difficult to operationalize.
Similarly, the occurrence of stressful and unexpected operational scenarios is known to impair executive functioning and provoke perseveration, see Dehais et al. (2019) for review. Perseveration is defined as a tendency to continue an action after cessation of the original stimulation, which is no longer relevant to the goal at hand (Sandson and Albert, 1984). For example, several studies conducted on emergency evacuation situations reported irrational and perseverative behaviors even when tasks were simple and undemanding (Proulx, 2001; Kobes et al., 2010). A paradigmatic situation is the one in which people fail to escape from fire because they push the door instead of pulling it. Perseveration can also have devastating consequences during safety-critical tasks, such as aviation (O’Hare and Smitheram, 1995; Orasanu et al., 1998; Reynal et al., 2017) and in the medical domain (Bromiley, 2008). This category of performance impairment cannot be explained solely through the prism of limited mental resources. Operators who persist with an erroneous strategy, such as an aircrew who attempt to land their craft at all costs despite bad weather conditions, are generally capable of performing the required actions and tend to invest greater effort even as their task goal becomes difficult or even impossible to achieve (Dehais et al., 2010, 2012).
The concept of limited cognitive resources could explain failures of attention such as inattentional blindness (Brand-D’Abrescia and Lavie, 2008) or deafness (Raveh and Lavie, 2015). Both categories describe an inability to detect unexpected stimuli, such as alarms from the interface (Dehais et al., 2011, 2014), and represent breakdown of selective attention due to the presence of competing demands on the human information processing system. It has been demonstrated that individuals with greater information processing capacity (i.e., higher working memory span) exhibit superior ability with respect to divided and sustained attention (Colflesh and Conway, 2007; Unsworth and Engle, 2007), and therefore, should be less susceptible to the effects of inattention during the performance of demanding tasks. However, this hypothesis is contradicted by the absence of any correlation between individual differences in processing capacity and the occurrence of inattentional blindness (Bredemeier and Simons, 2012; Beanland and Chan, 2016; Kreitz et al., 2016a) or deafness (Kreitz et al., 2016b; Dehais et al., 2019).
This research suggests that the limited resource model cannot account for critical lapses of attention and executive functioning that are observed under conditions of high mental workload. Therefore, we must go beyond the limitations of the resource concept as an explanatory model of mental workload and turn our attention to the neural underpinnings of attention and behavior (Parasuraman et al., 1999).
Resources: a Neuroergonomic Perspective
The last three decades have witnessed a revolution in our understanding of neural mechanisms that are fundamental to attention and human performance. Progress in the field has been driven by the development of advanced and portable neuroimaging techniques, which permit non-invasive examination of the “brain at work.” Neuroergonomics is a multidisciplinary field born from these technical innovations that is broadly defined as the study of the human brain in relation to performance at work and in everyday settings (Parasuraman and Rizzo, 2008). The goal of this field is to integrate both theories and principles from ergonomics, neuroscience and human factors in order to provide insights into the relationship between brain function and behavioral outcomes in the context of work and everyday life (Rizzo et al., 2007; Parasuraman and Rizzo, 2008; Parasuraman and Wilson, 2008; Lees et al., 2010; Ayaz and Dehais, 2018).
The Multiple Biological Substrates of Mental Resources
The incorporation of neurophysiological measures of mental workload offers a reductive pathway to the reification of resources and those neurobiological states associated with impaired performance. At a fundamental level, the functioning of neurons within the brain is a form of limited resource (Beatty, 1986), requiring oxygen and glycose to generate cellular energy in the form of adenosine triphosphate (ATP) while having a very limited capacity to store these energy substrates (Saravini, 1999). The same logic holds for ions (e.g., potassium, calcium, sodium) that play a key role in nerve impulses. It is also reasonable to consider neural networks as resources with respect to their supporting glial cells (e.g., astrocytes), which ensure the processing of information (Mandrick et al., 2016). Understanding the interactions between neurobiological resources with reference to fundamental processes in brain physiology represents a crucial approach within neuroergonomic analysis of mental workload (Parasuraman and Rizzo, 2008; Ayaz and Dehais, 2018).
Brain and Inhibitory Mechanisms
The brain must be considered to be a “noisy” organ, whereby assembly of neurons are constantly responsive to environmental stimulations, see Pandemonium architecture as an early example, such as Selfridge (1959). Inhibitory mechanisms are implemented to cancel out cerebral noise by mitigating the activation of distracting neuronal assemblies (Polich, 2007). This process may occur at a local level via lateral inhibition, whereby groups of neurons can attenuate the activity of their neighbors in order to be “better heard” (Coultrip et al., 1992). The same mechanism can also take place via top-down regulation, known as inhibitory control, wherein high-level cortical areas (e.g., prefrontal cortex) reduce task- or stimulus-irrelevant neural activities (Munakata et al., 2011). However, these inhibitory mechanisms can also curtail the capacity of the brain to consider new or alternative information, thus leading to perseveration (Dehais et al., 2019). An appropriate metaphor is to consider a group led by an authoritarian leader who is totally engaged with one specific goal or strategy and does not listen to alternative viewpoints of other members of the group. Within this metaphor, information processing resources are present (i.e., group members) but are disregarded in the presence of an overriding directive (i.e., the leader). In other words, high mental workload leads to impaired performance, not because of limited resources per se, but because of those neurological mechanisms designed to prioritize a specific goal or directive.
The Non-linear Effects of Neuromodulation
The prefrontal cortex (PFC) is a brain structure often identified as the neurophysiological source of limited resources (Posner and Petersen, 1990; Parasuraman, 2003; Ramsey et al., 2004; Modi et al., 2017). The PFC serves a control function during routine cognitive operations, such as: action selection, retrieval/updating in working memory, monitoring and inhibition (Ramnani and Owen, 2004; Ridderinkhof et al., 2004). It is often activated during high levels of cognitive demand (Ayaz et al., 2012; Herff et al., 2014; Racz et al., 2017; Gateau et al., 2018; Fairclough et al., 2019) and dysfunction of this structure is known to degrade performance (Sandson and Albert, 1984; Dolcos and McCarthy, 2006). However, the PFC is complex and its function is subject to the quadratic influence of neuromodulation via the influence of noradrenaline and dopamine (Arnsten, 2009; Arnsten et al., 2012). Noradrenaline is associated with the mediation of arousal (Chrousos, 2009) whereas dopamine is involved in the processing of reward with regard to the ongoing tasks (Schultz, 2002). Both catecholamines exert an inverted-U relationship with the PFC neurons (Vijayraghavan et al., 2007; Robbins and Arnsten, 2009), a reduction of these neurochemicals will depress the firing rate of noradrenergic and dopaminergic PFC neurons (see Figure 1). This mechanism may explain why unstimulating and non-rewarding tasks (e.g., passive supervisory control over a sustained period) can inhibit executive functioning and induce mind wandering. Conversely, excessive levels can also have a deleterious effect by suppressing PFC neuron firing rate (Birnbaum et al., 1999). In addition to decreasing the activity of the PFC, dopamine and noradrenaline activate subcortical areas, such as basal ganglia, that trigger automated schemes and initiate automatic responses (Wickens et al., 2007). These automated behaviors have an advantage of speed compared to flexible but slower behaviors generated by the prefrontal cortex (Dolan, 2002). This neurological switch from prefrontal to subcortical areas, is presumed to derive from the early age of humanity to ensure survival (Arnsten, 2009). In modern times, it manifests itself as a process of defaulting to well-learned behaviors, which are effective for only operational situations that are simple and familiar. This is the mechanism that promotes perseveration (Dehais et al., 2019) in task scenarios that are complex and novel (Staal, 2004; Ellenbogen et al., 2006) or offer intrinsic, short-term rewards, e.g., landing at all costs after a long transatlantic flight (Causse et al., 2013). These fundamental neurological mechanisms illustrate that impaired operational performance cannot be simply explained in terms of limited resources, such as a concentration of dopamine, but must be viewed from a neuroergonomic perspective that emphasizes the complexity of interactions between brain areas that evolved over thousands of years.
Figure 1. The dopamine pathway exerts a quadratic control over the PFC. A low or a high release of this neurochemical depresses PFC activation whereas an adequate concentration ensures optimal executive functioning (Vijayraghavan et al., 2007; Robbins and Arnsten, 2009). These neurobiological considerations bring interesting highlights to understand the mechanisms underlying the Yerkes and Dodson inverted-U law and the dynamic adaptability theory (Hancock and Warm, 1989). They also provide a relevant prospect to relate motivational aspects to behavioral responses. The noradrenaline pathway mediates the PFC activity and executive functioning in a similar fashion (see Aston-Jones and Cohen, 2005).
Attentional Dynamics and Dominance Effects
The existence of information processing resources can also be conceptualized as functional attentional networks in the brain. Michael Posner was the first to pioneer a network approach to the operationalization of resources in the early days of neuroimaging (Posner and Tudela, 1997). His influential analysis (Posner and Petersen, 1990; Posner and Dehaene, 1994; Petersen and Posner, 2012; Posner, 2012) described how specific networks were dedicated to the particular functions of attentional regulation, e.g., alerting, orientation, focus. This conceptualization developed into the delineation of a dorsal fronto-parietal network (e.g., intraparietal cortex, superior frontal cortex) that supports focused attention on specific task-relevant stimuli and a corresponding ventral fronto-parietal network (e.g., temporo-parietal cortex, inferior frontal cortex) in the right hemisphere, which activates in a bottom-up fashion to reorientate attention to interruptive stimuli (Corbetta and Shulman, 2002; Corbetta et al., 2008). Under nominal conditions, interaction between the dorsal and the ventral pathways ensure optimal trade-off between those attentional strategies associated with exploitation and exploration. However, under conditions of high task demand or stress or fatigue, this mechanism may become biased toward dominance of the dorsal over the ventral network, leading to attentional phenomena associated with inflexibility (Todd et al., 2005; Durantin et al., 2017; Edworthy et al., 2018; Dehais et al., 2019a). A similar dynamic of bias and dominance is apparent in the relationship between the dorsal and ventral pathways and the default mode network (Andrews-Hanna et al., 2014), which is associated with mind-wandering, spontaneous thoughts and disengagement from task-related stimuli (Fox et al., 2015).
Performance Monitoring and Effort Withdrawal
The capacity of the brain to monitor performance quality and progress toward task goals is another important function of the PFC during operational performance. The posterior medial frontal cortex (pMFC) is a central hub in a wider network devoted to performance monitoring, action selection and adaptive behavior (Ullsperger et al., 2014; Ninomiya et al., 2018). The pMFC is sensitive to error and failure to achieve a task goal (Ullsperger et al., 2007); the detection of failure represents an important cue for compensatory strategies, such as increased investment of mental effort (Hockey, 1997). This network is particularly important when the level of task demand experienced by the operator is associated with a high rate of error and increased probability of failure. The model of motivational intensity (Richter et al., 2016) predicts that effort is withdrawn from task performance if success likelihood is appraised to be very low (Hopstaken et al., 2015); similarly, models of behavioral self-regulation (Carver and Scheier, 2000) argue that task goals can be adjusted downward (i.e., lower levels of performance are tolerated as acceptable) or even abandoned if goal attainment is perceived to be impossible. There is evidence that increased likelihood of failure is associated with deactivation of the PFC (Durantin et al., 2014; Ewing et al., 2016; Fairclough et al., 2019), for operational performance where failure can often jeopardize the safety of oneself and others, increased likelihood of failure can also provoke strong emotional responses that are associated with stress and cognitive interference (Sarason et al., 1990), which can function as distractors from task activity in their own right (Dolcos and McCarthy, 2006; Qin et al., 2009; Gärtner et al., 2014).
This neuroergonomic approach provides a biological basis upon which to develop a concept of limited human information processing, with respect to competing neurological mechanisms, the influence of neuromodulation in the prefrontal cortex and antagonist directives between different functional networks in the brain. The prominence of inhibitory control coupled with competition between these neural networks delineate a different category of performance limitations during extremes of low vs. high mental workload, i.e., simultaneous activation of functional networks with biases toward mutually exclusive stimuli (external vs. internal) or contradictory directives (focal attention vs. reorientation of attention).
Understanding Performance Related Mental States
The previous sections have highlighted the complexity of those brain dynamics and networks that can introduce inherent limitations on human information processing. On the basis of this analysis, it is reasonable to target neurophysiological states and their associated mechanisms that account for impaired human performance (see Prinzel, 2002). This review has identified a number of suboptimal neurocognitive states that are predictive of degraded performance such as: mind wandering, effort withdrawal, perseveration, inattentional blindness and deafness. These states may be conceptually mapped along orthogonal dimensions of task engagement and arousal (Figure 2). Engagement is defined as an effortful investment in the service of task/cognitive goals (Pope et al., 1995; Matthews et al., 2002; Stephens et al., 2018), whereas arousal represents a state of physiological readiness to respond to external contingencies (Pribram and McGuinness, 1975).
Figure 2. Performance, arousal and task engagement: the green zone conceptually describes the operator’s “comfort zone” where performance is optimal. The degraded mental states are mapped across a “task engagement” axis and an “arousal” axis. Interestingly, this point of view makes it possible to link the notion of engagement and degraded behavior in a simple way.
The Transactional Dimensions of Engagement and Arousal
The rationale for considering the dimension of task engagement is that performance is driven by goals and motivation (Bedny and Karwowski, 2004; Fairclough et al., 2013; Leontiev, 2014). Goal-oriented cognition theorists argue for the existence of mechanisms dedicated to maintain engagement (Atkinson and Cartwright, 1964), which are associated with an activation of an executive (Baddeley and Hitch, 1974) or task-positive network (Harrivel et al., 2013) within which the dorsolateral prefrontal cortex (DLPFC) exerts a crucial role (Goldman-Rakic, 1987; Curtis and D’Esposito, 2003). This structure plays a key role in the maintenance and updating of information that is relevant for ongoing task performance. The same structure interacts with dorsal and ventral attentional pathways to shift and focus attention to the most relevant stream of task-related information (Johnson and Zatorre, 2006). It is argued that human performance can be assessed in the context of a continuum of task engagement, ranging from disengagement (effort withdrawal, mind wandering) to high-engagement (perseveration, inattentional phenomena Lee, 2014).
Arousal makes an important contribution to the conceptual space illustrated in Figure 2 because it modulates the homeostasis of the executive (see Arnsten, 2009 for a review) and attentional networks (see Coull, 1998 and Aston-Jones and Cohen, 2005 for review) via the dopaminergic and noradrenergic pathways. For instance, both extremes of low (Harrivel et al., 2013; Durantin et al., 2015) and high arousal can disengage the DLPFC (Goldberg et al., 1998; Arnsten, 2009; Qin et al., 2009; Causse et al., 2013; Durantin et al., 2014; Fairclough et al., 2019) and impair performance (see Figure 3 for summary). Similarly, low (Dehais et al., 2018) and high levels of arousal (Hancock and Warm, 1989; Tracy et al., 2000; Pecher et al., 2011) can alter the interactions between the dorsal and ventral attentional networks and indistinctly that lead either to inattentional phenomena (Molloy et al., 2015; Todd et al., 2005) or effort withdrawal (Oei et al., 2012; Dehais et al., 2015).
Figure 3. Left part: Several types of stressors can yield to the deactivation of the DLPFC and in return drastically induce collapse of performance. Right part: An illustration with the N-Back task: the right-DLPFC deactivates when the task demands exceed mental capacity (7-Back condition) and is associated with reduced performance efficacy and effort withdrawal (from Fairclough et al., 2019).
Monitoring Performance Through Degraded Mental States
Table 1 presents a mapping between extremes of high and low engagement and arousal, their related neurocognitive states and how these states may be operationalized using neurophysiological measures in the laboratory and the field. Monitoring the activation and deactivation of the DLPFC represents a promising generic avenue to predict impaired performance across diverse states such as: mind wandering (Christoff et al., 2009; Harrivel et al., 2013), effort withdrawal (Ayaz et al., 2007; Izzetoglu et al., 2007; Durantin et al., 2014; Modi et al., 2018; Fairclough et al., 2018, 2019) and perseveration (Dehais et al., 2019). However, other neurological networks and sites should be considered as part of this analysis. Mind wandering is characterized by the concomitant activation of the default network, which includes the median prefrontal cortex (Christoff et al., 2009; Harrivel et al., 2013) and areas of the parietal cortex (Christoff et al., 2009).
Table 1. Psycho-physiological and behavioral markers of different mental states related to engagement.
Secondly, attentional states, such as inattentional deafness and blindness, result from the activation of an attentional network involving the inferior frontal gyrus, the insula and the superior medial frontal cortex (Tombu et al., 2011; Callan et al., 2018; Dehais et al., 2019). These regions represent potential candidates upon which to identify attentional failures that can be complemented by monitoring dedicated primary perceptual (see Hutchinson, 2019, for a review) and integrative cortices (Molloy et al., 2015), as well as performing connectivity analyses (Callan et al., 2018). In addition, inattentional phenomena may result from the suppression of activity in the right temporo-parietal junction (TPJ), a part of the ventral network, which also blocks reorientation of attention and the processing of unexpected stimuli (Marois et al., 2004; Todd et al., 2005).
Thirdly, measures of arousal are used to characterize high engagement and delineate distinct mental states within the category of low task engagement (Figure 2). Heart rate (HR) and heart rate variability (HRV) can be used to assess the activation or co-activation of the two branches of the autonomous nervous system (i.e., sympathetic or parasympathetic) (Fairclough, 2008; Qin et al., 2009; Kreibig, 2010). For instance, fluctuations in HR are commonly observed during high task engagement and high arousal (De Rivecourt et al., 2008; Qin et al., 2009; Dehais et al., 2011). Moreover, spectral analyses computed over the EEG signal revealed that shifts in parietal alpha [8–12] Hz and frontal theta [4–8] Hz are relevant markers of arousal (see Borghini et al., 2014, for a review, Senoussi et al., 2017).
Finally, behavioral metrics such as ocular behavior can complement the detection of low and high levels of engagement (Table 1). Hence, eye tracking metrics (e.g., fixation and dwell times, saccadic activity, blink rate) can be used to characterize mind wandering (He et al., 2011; Pepin et al., 2016), inattentional blindness (Thomas and Wickens, 2004; Wickens, 2005), perseveration (Régis et al., 2014), focal vs. diffused attention (Goldberg and Kotval, 1999; Regis et al., 2012; Dehais et al., 2015), and to characterize the level of attentional engagement in a visual task (Cowen et al., 2002; Tsai et al., 2007).
These metrics provide some relevant prospects to identify the targeted deleterious mental states for especially for field studies as long as portable devices are concerned. It is worth noting that the extraction of several features (e.g., time and frequency domains) and the use of several devices is a way for robust diagnosis. Moreover, contextual information (e.g., time of the day, time on task) should be considered as well as actions on the user interface and system parameters (e.g., flight parameters) if available so as to better quantify the user’s mental state.
Solutions to Mitigate Degraded Performance
This review has identified some undesired mental states that account for degraded performance (see section “Understanding Performance Related Mental States” and “Solutions to Mitigate Degraded Performance”). A crucial step is to design cognitive countermeasures to prevent the occurrence of these phenomena. The formal framework that we proposed (see Table 1) paves the way to design neuro-adaptive technology for augmented cognition and enhanced human-machine teaming (Peysakhovich et al., 2018; Krol et al., 2019; Stamp et al., 2019). The implementation of such neuro-adaptive technology relies on a pipeline that consists of a signal acquisition step, a preprocessing step to improve the signal-to-noise ratio, a feature extraction step, a classification step to diagnose the current mental states, and lastly an adaptation step (Zander and Kothe, 2011; Roy and Frey, 2016). This last step implies the implementation of formal decisional unit (Gateau et al., 2018) that dynamically close the loop by triggering the most appropriate cognitive countermeasures (May and Baldwin, 2009). There are currently three types of mitigating solutions to instigate a change in behaviors via: (1) adaptation of the user interface, (2) adaptation of the task and of the level automation, and the (3) “neuro-adaptation” of the end-users.
Adaptation of the User Interface
The first category of neuroadaptive countermeasure consists of triggering new types of notifications via the user interface to alert of impeding hazards. The design of these countermeasures is generally grounded on neuroergonomics basis so that these warning can reach awareness when other means have failed. Following this perspective, Dehais et al. (2010, 2012), Imbert et al. (2014) and Saint Lot et al. (2020) have demonstrated that very brief (∼200 ms) and located information removal was an efficient mean to mitigate perseveration by forcing disengagement from non-relevant tasks. Souza et al. (2016) demonstrated that digital nudging (see Weinmann et al., 2016) could be used to mitigate poor decision making and cognitive bias associated with perseveration. Imbert et al. (2014) designed attention-grabbing stimuli grounded on vision research and demonstrated that yellow chevrons pulsing at a cycle of 1 Hz can re-orientate attention and mitigate inattentional blindness. Jahanpour et al. (2018) has explored the design of pop-up videos that display the gestures to be performed by exploiting the property of mirror neurons. This visual “motor cue” approach was tested and drastically reduced reaction time to alerts during complex situations and appears to be a promising method to prevent effort withdrawal (Causse et al., 2012). In a similar fashion, Navarro et al. (2010) implemented a force-feedback steering wheel to prime the motor response from the driver. This device was found to optimize drivers’ behavior during demanding driving scenario. This latter study demonstrated how tactile notifications can alert human operators of impeding hazards (Lewis et al., 2014; Russell et al., 2016), especially when other sensory channels of information (e.g., visual stream) are saturated (Elliott et al., 2011). However, there are potential limits to the effectiveness of these types of notifications and stimulation (Murphy and Dalton, 2016; Riggs and Sarter, 2019). Other research indicates that multimodal alerts (Giraudet et al., 2015a; Gaspar et al., 2017) increase the likelihood of attentional capture. In addition, Lee et al. (2018) designed a motion seat that modifies the driver’s seat position and posture across time to diminish the potential deleterious effect of mind wandering. Similar concepts have been applied to aviation (Zaneboni and Saint-Jalmes, 2016).
Task and Automation Adaptation
The second category of neuroadaptive countermeasure is the dynamic reallocation of tasks between humans and automation to maintain the performance efficacy of the operators (Freeman et al., 1999; Parasuraman et al., 1999; Prinzel et al., 2000; Scerbo, 2008; Stephens et al., 2018). The underlying concept in this case is to optimize human-human or human(s)-system(s) cooperation according to criteria of availability and skills of human and artificial agents (Gateau et al., 2016). For instance, Prinzel et al. (2000) utilized the continuous monitoring of brain waves that could be used to drive the level of automation and optimize the user’s level of task engagement. Similarly, some authors managed to optimize air traffic controllers’ task demand by triggering different levels of assistance (Aricò et al., 2016; Di Flumeri et al., 2019). These latter studies reported better human performance when neuro-adaptive automation was switched on compared to other conditions. Gateau et al. (2016) implemented an online attentional state estimator coupled with a stochastic decision framework to dynamically adapt authority sharing between human and robots in a search and rescue scenario to prevent effort withdrawal on the part of the human. In a more extreme fashion, Callan et al. (2016) revealed that it is possible to decode user motor intention so automation can perform on behalf of the user to drastically reduce the response time in emergency situations (e.g., collision with terrain). In the future, it is assumed that aircraft designers will implement adaptive automation technology that takes over from the pilots by either inhibiting their inputs on the flight deck or performing automated evasive actions (e.g., automatic pull-up) to prevent from perseveration. A complementary approach is to modulate task difficulty to maintain the task challenging but achievable while preventing the occurrence of task withdrawal (Ewing et al., 2016) or mind wandering (Freeman et al., 2004; Ewing et al., 2016). The online modulation of the tasks does not necessarily reduce the difficulty of the task. For instance, Verwey and colleagues demonstrated that the addition of an entertaining task while driving improved the operator’s ability to maintain their level of task engagement over long period of time (Verwey and Zaidel, 1999). Similarly, it has been suggested that switching the types of tasks presented to the user can prevent the deleterious effect of fatigue and disengagement (Hockey, 2011).
Neuro-Adaptation of the End-User(s)
The third and final category aims to warn the users of their mental state and “stimulate” neurological activity in order to augment performance. One of the most promising approach relies on the implementation of Neurofeedback (see Gruzelier, 2014; Enriquez-Geppert et al., 2017 for reviews). The principle of the latter technique is to provide feedback in real-time to the users of their mental states in the form of a visual, tactile or auditory stimulus. The users can utilize these signals learn to regulate their brain activity and in return improve their executive (Enriquez-Geppert et al., 2013), mental flexibility (Enriquez-Geppert et al., 2014), and attentional abilities (Egner and Gruzelier, 2001) as well as enhance their task engagement (Egner and Gruzelier, 2004). However, the effects of this approach on mind wandering remain unclear (Gonçalves et al., 2018). Transcranial direct current stimulation (tDCS) represents a technique of neuromodulation that can be used to boost executive functioning (see Callan and Perrey, 2019; Cinel et al., 2019). This portable device can be combined with EEG and fNIRS and used in the context of real-life task performance for the purpose of on-line neuromodulation (McKendrick et al., 2015; Gateau et al., 2018). For example, a number of studies support the position that neurostimulation can: enhance mental flexibility and mitigate perseveration (Leite et al., 2011; Jeon and Han, 2012), improve visual attention (Falcone et al., 2012; Nelson et al., 2015), improve executive functioning in multitasking situations (Nelson et al., 2016) and increase alertness (McIntire et al., 2014; Nelson et al., 2014). There are other types of environmental stimulation such as vivid light exposure, especially during night flights, which can promote an optimal level of alertness (see Anund et al., 2015) without altering flight crew performance (see Caldwell et al., 2009). Promising results have also been highlighted by using light exposure in cars (Taillard et al., 2012). The use of light exposure and tDCS should be considered with caution as there is a need to investigate the very long-term efficiency and potential side effects. Alternatively, some authors proposed to use cold-air jet to decrease hypovigilance (Reyner and Horne, 1998), but with contradictory findings.
Synthesis of Neuro-Adaptive Solutions
The following illustration (see Figure 4) depicts the three families of neuro-adaptive based solutions to mitigate performance impairment.
Figure 4. The three types of Neuroadaptive countermeasures dedicated to mitigate the undesirable mental states. Inattentional deafness and Inattentional blindness mental states were merged into “Inattentional phenomena” as no neuroadaptive countermeasure were implemented to explicitly address failure of auditory attention to the exception of multimodal alerts. Moreover, no adaptive automation-based solutions were designed to prevent from inattentional states. This demonstrates the need to conduct more research in this direction.
The three types of neuroadaptive solutions offer promising prospects to mitigate the onset and likelihood of undesirable neurocognitive states. However, they should be delivered in a transparent, meaningful, and timely manner (i.e., when needed) so they are relevant and understood (Dorneich et al., 2016; Sebok et al., 2017), otherwise these types of intervention have the potential for undesirable consequences, such as performance impairment and reduced trust in technology; this point is particularly true for adaptive automation solutions that take over from humans, especially under critical scenarios (see Dorneich et al., 2016; Dehais et al., 2019). One solution is to combine different families of neuroadaptive cognitive countermeasures to maximize their efficiency. Ideally, we would argue to use a gradient of solutions such as (1) the continuous display of the users’ mental states via neurofeedback techniques to give them the opportunity to regulate their brain activity; (2) using notifications to suggest to the users to delegate some tasks to automation in case they don’t manage to modulate their mental states; (3) adapting the user interface (e.g., information removal, flashing yellow chevrons) in case of a critical situation is detected and the previous solutions were inefficient; and (4) taking over if the users do not respond to any of the previous countermeasures.
Conclusion
This paper has argued that the concept of a limited resource provides a limited explanation for the breakdown of operational performance. Our neurophysiological analysis describes a number of additional mechanisms, such as perseveration and effort withdrawal, which do not represent finite resources per se. In both cases, explanations for performance breakdown are based upon neurological processes, such as dominance of specific neural networks or the heightened activity of specific mechanisms. We propose a two-dimensional framework of engagement and arousal that captures the importance of specific degraded mental sates associated with poor performance. The rationale for including the transactional concept of engagement in this scheme is to account for the goal-oriented aspect of cognition. The benefit of including the transactional concept of arousal is to make a distinction between two categories of disengagement, one that is accompanied by high arousal (effort withdrawal) and low arousal (mind wandering) – and to link this conceptual distinction to known neurophysiological effects (see Figure 1). Nonetheless, this approach remains at the conceptual level and minimizes connections to the complexity of brain functioning. To that end, we reviewed and identified several markers at the neurophysiological, physiological and behavioral level of undesirable mental states linked to poor performance.
This neuroergonomic framework encompasses operationali- zations of these undesirable states that can be monitored continuously in an objective fashion. Such considerations eventually lead to propose a typology of neuroadaptive countermeasures and open promising perspectives to mitigate the degradation of human performance. However, to the authors’ very best knowledge, most of the neuroadaptive experimental studies have focused on human-machine dyad situations. We believe that recent research on hyperscanning (Babiloni and Astolfi, 2014), physiological synchrony (Palumbo et al., 2017) and collaborative BCIs (Cinel et al., 2019) have opened promising prospects to improve teaming such as human-human, human(s)-machine(s) interactions. Future research should involve more complex teaming scenarios and enrich the different neuroadaptive solutions. We sincerely hope that this review will encourage research efforts to identify additional degraded mental states and associated neurophysiological markers as well as to implement neuroadaptive solutions for safer and efficient human-human and human(s)-machine(s) interactions.
Author Contributions
All authors have made a substantial and intellectual contribution to this review.
Funding
This work was supported by ANITI ANR-19-PI3A-0004 (Neuroadaptive technology for mixed initiative interactions Chair).
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
Andrews-Hanna, J. R., Smallwood, J., and Spreng, R. N. (2014). The default network and self-generated thought: component processes, dynamic control, and clinical relevance. Ann. N. Y. Acad. Sci. 1316:29. doi: 10.1111/nyas.12360
Annett, J. (2002). Subjective rating scales: science or art? Ergonomics 45, 966–987. doi: 10.1080/00140130210166951
Anund, A., Fors, C., Kecklund, G., Leeuwen, W. V., and Åkerstedt, T. (2015). Countermeasures for Fatigue in Transportation: A Review of Existing Methods for Drivers on Road, Rail, Sea And In Aviation. Linköping: Statens vägoch transportforskningsinstitut.
Aricò, P., Borghini, G., Di Flumeri, G., Colosimo, A., Bonelli, S., Golfetti, A., et al. (2016). Adaptive automation triggered by EEG-based mental workload index: a passive brain-computer interface application in realistic air traffic control environment. Front. Hum. Neurosci. 10:539. doi: 10.3389/fnhum.2016.00539
Arnsten, A. F. (2009). Stress signalling pathways that impair prefrontal cortex structure and function. Nat. Rev. Neurosci. 10:410. doi: 10.1038/nrn2648
Arnsten, A. F., Wang, M. J., and Paspalas, C. D. (2012). Neuromodulation of thought: flexibilities and vulnerabilities in prefrontal cortical network synapses. Neuron 76, 223–239. doi: 10.1016/j.neuron.2012.08.038
Aston-Jones, G., and Cohen, J. D. (2005). An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu. Rev. Neurosci. 28, 403–450. doi: 10.1146/annurev.neuro.28.061604.135709
Atkinson, J. W., and Cartwright, D. (1964). Some neglected variables in contemporary conceptions of decision and performance. Psychol. Rep. 14, 575–590. doi: 10.2466/pr0.1964.14.2.575
Ayaz, H., and Dehais, F. (eds) (2018). Neuroergonomics: The Brain at Work and in Everyday Life. Cambridge, MA: Academic Press.
Ayaz, H., Izzetoglu, M., Bunce, S., Heiman-Patterson, T., and Onaral, B. (2007). “Detecting cognitive activity related hemodynamic signal for brain computer interface using functional near infrared spectroscopy,” in Proceedins of the 3rd International IEEE EMBS conference: CNE’07, Hawaii, 342–345.
Ayaz, H., Shewokis, P. A., Bunce, S., Izzetoglu, K., Willems, B., and Onaral, B. (2012). Optical brain monitoring for operator training and mental workload assessment. Neuroimage 59, 36–47. doi: 10.1016/j.neuroimage.2011.06.023
Babiloni, F., and Astolfi, L. (2014). Social neuroscience and hyperscanning techniques: past, present and future. Neurosci. Biobehav. Rev. 44, 76–93. doi: 10.1016/j.neubiorev.2012.07.006
Baddeley, A. D., and Hitch, G. (1974). “Working memory,” in The Psychology of Learning and Motivation: Advances in Research and Theory, Vol. 8, eds G. H. Bower (New York, NY: Academic Press), 47–89. doi: 10.1016/S0079-7421(08)60452-1
Beanland, V., and Chan, E. H. C. (2016). The relationship between sustained inattentional blindness and working memory capacity. Attent. Percept. Psychophys. 78, 808–817. doi: 10.3758/s13414-015-1027-x
Beatty, J. (1986). “Computation, control and energetics: a biological perspective,” in Energetics and Human Information Processing, ed. G. R. J. Hockey (Dordrecht: Springer).
Beck, D. M., Rees, G., Frith, C. D., and et Lavie, N. (2001). Neural correlates of change detection and change blindness. Nat. Neurosci. 4, 645–650. doi: 10.1038/88477
Bedny, G. Z., and Karwowski, W. (2004). Activity theory as a basis for the study of work. Ergonomics 47, 134–153. doi: 10.1080/00140130310001617921
Birnbaum, S., Gobeske, K. T., Auerbach, J., Taylor, J. R., and Arnsten, A. F. (1999). A role for norepinephrine in stress-induced cognitive deficits: α-1-adrenoceptor mediation in the prefrontal cortex. Biol. Psychiatry 46, 1266–1274. doi: 10.1016/s0006-3223(99)00138-9
Borghini, G., Astolfi, L., Vecchiato, G., Mattia, D., and Babiloni, F. (2014). Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental work- load, fatigue and drowsiness. Neurosci. Biobehav. Rev. 44, 58–75. doi: 10.1016/j.neubiorev.2012.10.003
Borragan Pedraz, G., and Peigneux, P. (2016). Behavioural Bases and Functional Dynamics of Cognitive Fatigue, Doctorat. Sciences psychologiques et de l’éducation, Louvain.
Braboszcz, C., and Delorme, A. (2011). Lost in thoughts: neural markers of low alertness during mind wandering. Neuroimage 54, 3040–3047. doi: 10.1016/j.neuroimage.2010.10.008
Brand-D’Abrescia, M., and Lavie, N. (2008). Task coordination between and within sensory modalities: effects on distraction. Percept. Psychophys. 70, 508–515. doi: 10.3758/pp.70.3.508
Bredemeier, K., and Simons, D. J. (2012). Working memory and inattentional blindness. Psychon. Bull. Rev. 19, 239–244. doi: 10.3758/s13423-011-0204-8
Brynjolfsson, E., and McAfee, A. (2014). The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies, 1st Edn. New York, NY: W. W. Norton & Company.
Caldwell, J. A., Mallis, M. M., Caldwell, J. L., Paul, M. A., Miller, J. C., and Neri, D. F. (2009). Fatigue countermeasures in aviation. Aviat. Space Environ. Med. 80, 29–59. doi: 10.3357/asem.2435.2009
Callan, D., and Perrey, S. (2019). “The use of tDCS and rTMS methods in neuroergonomics,” in Neuroergonomics, eds H. Ayaz and F. Dehais (Cambridge, MA: Academic Press), 31–33. doi: 10.1016/b978-0-12-811926-6.00005-1
Callan, D. E., Gateau, T., Durantin, G., Gonthier, N., and De-Hais, F. (2018). Disruption in neural phase synchrony is related to identification of inattentional deafness in real-world setting. Hum. Brain Mapp. 39, 2596–2608. doi: 10.1002/hbm.24026
Callan, D. E., Terzibas, C., Cassel, D. B., Sato, M. A., and Parasuraman, R. (2016). The brain is faster than the hand in split-second intentions to respond to an impending hazard: a simulation of neuroadaptive automation to speed recovery to perturbation in flight attitude. Front. Hum. Neurosci. 10:187. doi: 10.3389/fnhum.2016.00187
Carrillo-De-La-Pena, M. T., and García-Larrea, L. (2007). Right frontal event related EEG coherence (ERCoh) differentiates good from bad performers of the Wisconsin Card Sorting Test (WCST). Clin. Neurophysiol. 37, 63–75. doi: 10.1016/j.neucli.2007.02.002
Carver, C. S., and Scheier, M. F. (2000). “On the structure of behavioural self-regulation,” in Handbook of Self-Regulation, eds M. Boekaerts, P. R. Pintrich, and M. Zeidner (San Diego, CA: Academic Press), 41–84. doi: 10.1016/b978-012109890-2/50032-9
Causse, M., Imbert, J. P., Giraudet, L., Jouffrais, C., and Tremblay, S. (2016). The role of cognitive and perceptual loads in inattentional deafness. Front. Hum. Neurosci. 10:344. doi: 10.3389/fnhum.2016.00344
Causse, M., Péran, P., Dehais, F., Caravasso, C. F., Zeffiro, T., Sabatini, U., et al. (2013). Affective decision making under uncertainty during a plausible aviation task: an fMRI study. NeuroImage 71, 19–29. doi: 10.1016/j.neuroimage.2012.12.060
Causse, M., Phan, J., Ségonzac, T., and Dehais, F. (2012). Mirror neuron based alerts for control flight into terrain avoidance. Adv. Cognit. Eng. Neuroergon. 16, 157–166.
Christoff, K., Gordon, A. M., Smallwood, J., Smith, R., and Schooler, J. W. (2009). Experience sampling during fmri reveals default network and executive system contributions to mind wandering. Proc. Natl. Acad. Sci. U.S.A. 106, 8719–8724. doi: 10.1073/pnas.0900234106
Chrousos, G. P. (2009). Stress and disorders of the stress system. Nat. Rev. Endocrinol. 5:374. doi: 10.1038/nrendo.2009.106
Cinel, C., Valeriani, D., and Poli, R. (2019). Neurotechnologies for human cognitive augmentation: current state of the art and future prospects. Front. Hum. Neurosci. 13:13. doi: 10.3389/fnhum.2019.00013
Colflesh, G. J., and Conway, A. R. (2007). Individual differences in working memory capacity and divided attention in dichotic listening. Psychon. Bull. Rev. 14, 699–703. doi: 10.3758/bf03196824
Corbetta, M., Patel, G., and Shulman, G. L. (2008). The reorienting system of the human brain: from environment to theory of mind. Neuron 58, 306–324. doi: 10.1016/j.neuron.2008.04.017
Corbetta, M., and Shulman, G. L. (2002). Control of goal-directed and stimulus-driven attention in the brain. Nat. Rev. Neurosci. 3:201. doi: 10.1038/nrn755
Coull, J. T. (1998). Neural correlates of attention and arousal: insights from electrophysiology, functional neuroimaging and psychopharmacology. Prog. Neurobiol. 55, 343–361. doi: 10.1016/s0301-0082(98)00011-2
Coultrip, R., Granger, R., and Lynch, G. (1992). A cortical model of winner-take-all competition via lateral inhibition. Neural Netw. 5, 47–54. doi: 10.1016/s0893-6080(05)80006-1
Cowen, L., Ball, L. J., and Delin, J. (2002). “An eye movement analysis of web page usability,” in People and Computers Xvi-memorable Yet Invisible, eds X. Faulkner, J. Finlay, and F. Détienne (Berlin: Springer), 317–335. doi: 10.1007/978-1-4471-0105-5_19
Curtis, C. E., and D’Esposito, M. (2003). Persistent activity in the prefrontal cortex during working memory. Trends Cognit. Sci. 7, 415–423. doi: 10.1016/s1364-6613(03)00197-9
De Rivecourt, M., Kuperus, M. N., Post, W. J., and Mulder, L. J. M. (2008). Cardiovascular and eye activity measures as indices for momentary changes in mental effort during simulated flight. Ergonomics 51, 1295–1319. doi: 10.1080/00140130802120267
De Waard, D. (1996). The Measurement of Driver Mental Workload, PhD Thesis., Groningen: Rijksuniversiteit Groningen.
Dehais, F., Causse, M., and Tremblay, S. (2011). Mitigation of conflicts with automation: use of cognitive countermeasures. Hum. Fact. 53, 448–460. doi: 10.1177/0018720811418635
Dehais, F., Causse, M., Vachon, F., Régis, N., Menant, E., and Tremblay, S. (2014). Failure to detect critical auditory alerts in the cockpit: evidence for inattentional deafness. Hum. Fact. 56, 631–644. doi: 10.1177/0018720813510735
Dehais, F., Causse, M., Vachon, F., and Tremblay, S. (2012). Cognitive conflict in human–automation interactions: a psychophysiological study. Appl. Ergon. 43, 588–595. doi: 10.1016/j.apergo.2011.09.004
Dehais, F., Duprès, A., Di Flumeri, G., Verdière, K. J., Borghini, G., Babiloni, F., et al. (2018). Monitoring Pilots Cognitive Fatigue with Engagement Features in Simulated and actual Flight Conditions Using an Hybrid fNIRS-EEG Passive BCI. IEEE SMC. Availale at: https://hal.archives-ouvertes.fr/hal-01959452.
Dehais, F., Hodgetts, H. M., Causse, M., Behrend, J., Durantin, G., and Tremblay, S. (2019). Momentary lapse of control: a cognitive continuum approach to understanding and mitigating perseveration in human error. Neurosci. Biobehav. Rev. 100, 252–262. doi: 10.1016/j.neubiorev.2019.03.006
Dehais, F., Peysakhovich, V., Scannella, S., Fongue, J., and Gateau, T. (2015). “Automation surprise in aviation: real- time solutions,” in Proceedings of the 33rd annual ACM conference on human factors in computing systems, New York, NY, 2525–2534.
Dehais, F., Rida, I., Roy, R., Iversen, J., Mullen, T., and Callan, D. (2019a). “A pBCI to predict attentional error before it happens in real flight conditions,” in Proceedins of the Conference: 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), Bari.
Dehais, F., Roy, R. N., and Scannella, S. (2019b). Inattentional deafness to auditory alarms: inter-individual differences, electrophysiological signature and single trial classification. Behav. Brain Res. 360, 51–59. doi: 10.1016/j.bbr.2018.11.045
Dehais, F., Roy, R. N., Durantin, G., Gateau, T., and Callan, D. (2017). “EEG-engagement index and auditory alarm misperception: an inattentional deafness study in actual flight condition,” in Proceedins of the International Conference on Applied Human Factors and Ergonomics, Washington D.C., 227–234. doi: 10.1007/978-3-319-60642-2_21
Dehais, F., Tessier, C., Christophe, L., and Reuzeau, F. (2010). “The perseveration syndrome in the pilots’ activity: guide- lines and cognitive countermeasures,” in Human Error, Safety and Systems Development, eds P. Palanque, J. Vanderdonckt, and M. Winkler (Berlin: Springer), 68–80. doi: 10.1007/978-3-642-11750-3_6
Di Flumeri, G., De Crescenzio, F., Berberian, B., Ohneiser, O., Kramer, J., Aricò, P., et al. (2019). Brain–computer interface-based adaptive automation to prevent out-of-the-loop phenomenon in air traffic controllers dealing with highly automated systems. Front. Hum. Neurosci. 13:296. doi: 10.3389/fnhum.2019.00296
Dierolf, A. M., Fechtner, J., Böhnke, R., Wolf, O. T., and Naumann, E. (2017). Influence of acute stress on response inhibition in healthy men: an ERP study. Psychophysiology 54, 684–695. doi: 10.1111/psyp.12826
Dolcos, F., and McCarthy, G. (2006). Brain systems mediating cognitive interference by emotional distraction. J. Neurosci. 26, 2072–2079. doi: 10.1523/jneurosci.5042-05.2006
Dorneich, M. C., Rogers, W., Whitlow, S. D., and DeMers, R. (2016). Human performance risks and benefits of adaptive systems on the flight deck. Int. J. Aviat. Psychol. 26, 15–35. doi: 10.1080/10508414.2016.1226834
Durantin, G., Dehais, F., and Delorme, A. (2015). Characterization of mind wandering using fNIRS. Front. Syst. Neurosci. 9:45. doi: 10.3389/fnsys.2015.00045
Durantin, G., Dehais, F., Gonthier, N., Terzibas, C., and Callan, D. E. (2017). Neural signature of inattentional deafness. Hum. Brain Mapp. 38, 5440–5455. doi: 10.1002/hbm.23735
Durantin, G., Gagnon, J.-F., Tremblay, S., and Dehais, F. (2014). Using near infrared spectroscopy and heart rate variability to detect mental overload. Behav. Brain Res. 259, 16–23. doi: 10.1016/j.bbr.2013.10.042
Edworthy, J., Reid, S., Peel, K., Lock, S., Williams, J., Newbury, C., et al. (2018). The impact of workload on the ability to localize audible alarms. Appl. Ergon. 72, 88–93. doi: 10.1016/j.apergo.2018.05.006
Egner, T., and Gruzelier, J. H. (2001). Learned self-regulation of EEG frequency components affects attention and event-related brain potentials in humans. Neuroreport 12, 4155–4159. doi: 10.1097/00001756-200112210-00058
Egner, T., and Gruzelier, J. H. (2004). EEG biofeedback of low beta band components: frequency-specific effects on variables of attention and event-related brain potentials. Clin. Neurophysiol. 115, 131–139. doi: 10.1016/s1388-2457(03)00353-5
Ellenbogen, M. A., Schwartzman, A. E., Stewart, J., and Walker, C.-D. (2006). Automatic and effortful emotional informa- tion processing regulates different aspects of the stress response. Psychoneuroendocrinology 31, 373–387. doi: 10.1016/j.psyneuen.2005.09.001
Elliott, L. R., Schmeisser, E. T., and Redden, E. S. (2011). “Development of tactile and haptic systems for US infantry navigation and communication,” in Proceedins of the Symposium on Human Interface (Berlin: Springer), 399–407. doi: 10.1007/978-3-642-21793-7_45
Enriquez-Geppert, S., Huster, R. J., Figge, C., and Herrmann, C. S. (2014). Self-regulation of frontal-midline theta facilitates memory updating and mental set shifting. Front. Behav. Neurosci. 8:420. doi: 10.3389/fnbeh.2014.00420
Enriquez-Geppert, S., Huster, R. J., and Herrmann, C. S. (2013). Boosting brain functions: improving executive functions with behavioral training, neurostimulation, and neurofeedback. Int. J. Psychophysiol. 88, 1–16. doi: 10.1016/j.ijpsycho.2013.02.001
Enriquez-Geppert, S., Huster, R. J., and Herrmann, C. S. (2017). EEG-neurofeedback as a tool to modulate cognition and behavior: a review tutorial. Front. Hum. Neurosci. 11:51. doi: 10.3389/fnhum.2017.00051
Ewing, K. C., Fairclough, S. H., and Gilleade, K. (2016). Evaluation of an adaptive game that uses EEG measures validated during the design process as inputs to a biocybernetic loop. Front. Hum. Neurosci. 10:223. doi: 10.3389/fnhum.2016.00223
Fairclough, S., Ewing, K., Burns, C., and Kreplin, U. (2019). “Neural efficiency and mental workload: locating the red line,” in Neuroergonomics, eds A. Johnson and R. W. Proctor (Cambridge, MA: Academic Press), 73–77. doi: 10.1016/b978-0-12-811926-6.00012-9
Fairclough, S. H. (2008). Fundamentals of physiological computing. Interact. Comput. 21, 133–145. doi: 10.1016/j.intcom.2008.10.011
Fairclough, S. H., Burns, C., and Kreplin, U. (2018). FNIRS activity in the prefrontal cortex and motivational intensity: impact of working memory load, financial reward, and correlation-based signal improvement. Neurophotonics 5, 1–10.
Fairclough, S. H., and Ewing, K. (2017). The effect of task demand and incentive on neurophysiological and cardiovascular markers of effort. Int. J. Psychophysiol. 119, 58–66. doi: 10.1016/j.ijpsycho.2017.01.007
Fairclough, S. H., Gilleade, K., Ewing, K. C., and Roberts, J. (2013). Capturing user engagement via psychophysiology: measures and mechanisms for biocybernetic adaptation. Int. J. Auton. Adapt. Commun. Syst. 6, 63–79.
Falcone, B., Coffman, B. A., Clark, V. P., and Parasuraman, R. (2012). Transcranial direct current stimulation augments perceptual sensitivity and 24-hour retention in a complex threat detection task. PLoS ONE 7:e34993. doi: 10.1371/journal.pone.0034993
Fox, K. C., Spreng, R. N., Ellamil, M., Andrews-Hanna, J. R., and Christoff, K. (2015). The wandering brain: meta-analysis of functional neuroimaging studies of mind-wandering and related spontaneous thought processes. Neuroimage 111, 611–621. doi: 10.1016/j.neuroimage.2015.02.039
Freeman, F. G., Mikulka, P. J., Prinzel, L. J., and Scerbo, M. W. (1999). Evaluation of an adaptive automation system using three EEG indices with a visual tracking task. Biol. Psychol. 50, 61–76. doi: 10.1016/s0301-0511(99)00002-2
Freeman, F. G., Mikulka, P. J., Scerbo, M. W., and Scott, L. (2004). An evaluation of an adaptive automation system using a cognitive vigilance task. Biol. Psychol. 67, 283–297. doi: 10.1016/j.biopsycho.2004.01.002
Gärtner, M., Rohde-Liebenau, L., Grimm, S., and Bajbouj, M. (2014). Working memory-related frontal theta activity is decreased under acute stress. Psychoneuroendocrinology 43, 105–113. doi: 10.1016/j.psyneuen.2014.02.009
Gaspar, J. G., Brown, T. L., Schwarz, C. W., Lee, J. D., Kang, J., and Higgins, J. S. (2017). Evaluating driver drowsiness countermeasures. Traff. Inj. Prevent. 18, S58–S63.
Gateau, T., Ayaz, H., and Dehais, F. (2018). In silico versus over the clouds: on-the-fly mental state estimation of aircraft pilots, using a functional near infrared spectroscopy based passive-BCI. Front. Hum. Neurosci. 12:187. doi: 10.3389/fnhum.2018.00187
Gateau, T., Chanel, C. P. C., Le, M. H., and Dehais, F. (2016). “Considering human’s non-deterministic behavior and his availability state when designing a collaborative human-robots system,” in Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, 4391–4397.
Giraudet, L., Imbert, J. P., Bérenger, M., Tremblay, S., and Causse, M. (2015a). The Neuroergonomic evaluation of human machine interface design in air traffic control using behavioral and EEG/ERP measures. Behav. Brain Res. 294, 246–253. doi: 10.1016/j.bbr.2015.07.041
Giraudet, L., St-Louis, M.-E., Scannella, S., and Causse, M. (2015b). P300 event-related potential as an indicator of inat- tentional deafness? PLoS ONE 10:e0118556. doi: 10.1371/journal.pone.0118556
Goldberg, J. H., and Kotval, X. P. (1999). Computer interface evaluation using eye movements: methods and constructs. Int. J. Ind. Ergon. 24, 631–645. doi: 10.1016/s0169-8141(98)00068-7
Goldberg, T. E., Berman, K. F., Fleming, K., Ostrem, J., Van Horn, J. D., Esposito, G., et al. (1998). Uncoupling cognitive workload and prefrontal cortical physiology: a PER rCBF study. Neuroimage 7, 296–303. doi: 10.1006/nimg.1998.0338
Goldman-Rakic, P. (1987). Handbook of Physiology. The Nervous System. Bethesda, MD: American Physiological Society, 373417.
Gonçalves, ÓF., Carvalho, S., Mendes, A. J., Leite, J., and Boggio, P. S. (2018). Neuromodulating attention and mind-wandering processes with a single session real time EEG. Appl. Psychophysiol. Biofeedback 43, 143–151. doi: 10.1007/s10484-018-9394-4
Gouraud, J., Delorme, A., and Berberian, B. (2018). Out of the loop, in your bubble: mind wandering is independent from automation reliability, but influences task engagement. Front. Hum. Neurosci. 12:383. doi: 10.3389/fnhum.2018.00383
Grandchamp, R., Braboszcz, C., and Delorme, A. (2014). Oculometric variations during mind wandering. Front. Psychol. 5:31. doi: 10.3389/fpsyg.2014.00031
Gruzelier, J. H. (2014). EEG-neurofeedback for optimising performance. I: a review of cognitive and affective outcome in healthy participants. Neurosci. Biobehav. Rev. 44, 124–141. doi: 10.1016/j.neubiorev.2013.09.015
Hancock, P. A., and Warm, J. S. (1989). A dynamic model of stress and sustained attention. Hum. Fact. 31, 519–537. doi: 10.1177/001872088903100503
Harrivel, A. R., Weissman, D. H., Noll, D. C., and Peltier, S. J. (2013). Monitoring attentional state with fnirs. Front. Human Neurosci. 7:861. doi: 10.3389/fnhum.2013.00861
He, J., Becic, E., Lee, Y.-C., and McCarley, J. S. (2011). Mind wandering behind the wheel: performance and oculomotor correlates. Human Factors 53, 13–21. doi: 10.1177/0018720810391530
Herff, C., Heger, D., Fortmann, O., Hennrich, J., Putze, F., and Schultz, T. (2014). Mental workload during n-back task—quantified in the prefrontal cortex using fNIRS. Front. Hum. Neurosci. 7:935. doi: 10.3389/fnhum.2013.00935
Hockey, G. R. J. (1997). Compensatory control in the regulation of human performance under stress and high workload: a cognitive-energetical framework. Biol. Psychol. 45, 73–93. doi: 10.1016/s0301-0511(96)05223-4
Hockey, G. R. J. (2011). “A motivational control theory of cognitive fatigue,” in Cognitive Fatigue: Multidisciplinary Perspectives on Current Research and Future Applications, ed. P. L. Ackerman (Washington, DC: American Psychological Association).
Hockey, G. R. J., Coles, M. G., and Gaillard, A. W. (1986). “Energetical issues in research on human information processing,” in Energetics and Human Information Processing, eds G. M. Hockey, A. W. K. Gaillard, and M. Coles (Berlin: Springer), 3–21. doi: 10.1007/978-94-009-4448-0_1
Hopstaken, J. F., Van Der Linden, D., Bakker, A. B., and Kompier, M. A. (2015). A multifaceted investigation of the link between mental fatigue and task disengagement. Psychophysiology 52, 305–315. doi: 10.1111/psyp.12339
Hutchinson, B. T. (2019). Toward a theory of consciousness: a review of the neural correlates of inattentional blindness. Neurosci. Biobehav. Rev. 104, 87–99. doi: 10.1016/j.neubiorev.2019.06.003
Imbert, J. P., Hodgetts, H. M., Parise, R., Vachon, F., Dehais, F., and Tremblay, S. (2014). Attentional costs and failures in air traffic control notifications. Ergonomics 57, 1817–1832. doi: 10.1080/00140139.2014.952680
Izzetoglu, M., Bunce, S. C., Izzetoglu, K., Onaral, B., and Pour-rezaei, K. (2007). Functional brain imaging using near- infrared technology. IEEE Eng. Med. Biol. Mag. 26:38. doi: 10.1109/memb.2007.384094
Jahanpour, E., Fabre, E., Dehais, F., and Causse, M. (2018). “Giving a hand to pilots with animated alarms based on mirror system functioning,” in Proceedings of the 2nd International Neuroergonomics Conference, Philadelphia, PA.
Jeon, S. Y., and Han, S. J. (2012). Improvement of the working memory and naming by transcranial direct current stimulation. Ann. Rehabil. Med. 36, 585–595.
Johnson, J. A., and Zatorre, R. J. (2006). Neural substrates for dividing and focusing attention between simultaneous auditory and visual events. NeuroImage 31, 1673–1681. doi: 10.1016/j.neuroimage.2006.02.026
Kalia, V., Vishwanath, K., Knauft, K., Vellen, B. V. D., Luebbe, A., and Williams, A. (2018). Acute stress attenuates cognitive flexibility in males only: an fNIRS examination. Front. Psychol. 9:2084. doi: 10.3389/fpsyg.2018.02084
Kam, J. W., Dao, E., Farley, J., Fitzpatrick, K., Smallwood, J., Schooler, J. W., et al. (2011). Slow fluctuations in attentional control of sensory cortex. J. Cognit. Neurosci. 23, 460–470. doi: 10.1162/jocn.2010.21443
Kobes, M., Helsloot, I., De Vries, B., and Post, J. G. (2010). Building safety and human behaviour in fire: a literature review. Fire Saf. J. 45, 1–11. doi: 10.1016/j.firesaf.2009.08.005
Kojima, H., and Suzuki, T. (2010). Hemodynamic change in occipital lobe during visual search: visual attention allocation measured with NIRS. Neuropsychologia 48, 349–352. doi: 10.1016/j.neuropsychologia.2009.09.028
Kramer, A., and Spinks, J. (1991). “Capacity views of human information processing,” in Handbook of Cognitive Psychophysiology: Central and Nervous Systems Approaches, eds J. R. Jennings and M. G. H. Coles (New York: Wiley), 179–249.
Kreibig, S. D. (2010). Autonomic nervous system activity in emotion: a review. Biol. Psychol. 84, 394–421. doi: 10.1016/j.biopsycho.2010.03.010
Kreitz, C., Furley, P., Memmert, D., and Simons, D. J. (2016a). The influence of attention set, working memory capacity, and expectations on inattentional blindness. Perception 45, 386–399. doi: 10.1177/0301006615614465
Kreitz, C., Furley, P., Simons, D. J., and Memmert, D. (2016b). Does working memory capacity predict cross-modally induced failures of awareness? Conscious. Cognit. 39, 18–27. doi: 10.1016/j.concog.2015.11.010
Krol, L. R., Haselager, P., and Zander, T. O. (2019). Cognitive and affective probing: a tutorial and review of active learning for neuroadaptive technology. J. Neural Eng. 17:012001. doi: 10.1088/1741-2552/ab5bb5
Lee, J. D. (2014). Dynamics of driver distraction: the process of engaging and disengaging. Ann. Adv. Automot. Med. 58:24.
Lee, S., Kim, M., Choi, S., and You, H. (2018). Evaluation of a motion seat system for reduction of a driver’s passive task-related (tr) fatigue. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 62, 1843–1847. doi: 10.1177/1541931218621420
Lees, M. N., Cosman, J. D., Lee, J. D., Rizzo, M., and Fricke, N. (2010). Translating cognitive neuroscience to the driver’s operational environment: a neuroergonomics approach. Am. J. Psychol. 123:391. doi: 10.5406/amerjpsyc.124.4.0391
Leite, J., Carvalho, S., Fregni, F., and Gonçalves, O. F. (2011). Task-specific effects of tDCS-induced cortical excitability changes on cognitive and motor sequence set shifting performance. PLoS ONE 6:e24140. doi: 10.1371/journal.pone.0024140
Lewis, B. A., Eisert, J. L., and Baldwin, C. L. (2014). Effect of tactile location, pulse duration, and interpulse interval on perceived urgency. Transport. Res. Rec. 2423, 10–14. doi: 10.3141/2423-02
Lie, C. H., Specht, K., Marshall, J. C., and Fink, G. R. (2006). Using fMRI to decompose the neural processes underlying the Wisconsin Card Sorting Test. Neuroimage 30, 1038–1049. doi: 10.1016/j.neuroimage.2005.10.031
Mallat, C., Cegarra, J., Calmettes, C., and Capa, R. L. (2019). A curvilinear effect of mental workload on mental effort and behavioral adaptability: an approach with the pre-ejection period. Hum. Fact. [Epub ahead of print].
Mandrick, K., Chua, Z., Causse, M., Perrey, S., and Dehais, F. (2016). Why a comprehensive understanding of mental workload through the measurement of neurovascular coupling is a key issue for neuroergonomics? Front. Hum. Neurosci. 10:250. doi: 10.3389/fnhum.2016.00250
Marois, R., Yi, D. J., and Chun, M. M. (2004). The neural fate of consciously perceived and missed events in the attentional blink. Neuron 41, 465–472. doi: 10.1016/s0896-6273(04)00012-1
Mason, M. F., Norton, M. I., Van Horn, J. D., Wegner, D. M., Grafton, S. T., and Macrae, C. N. (2007). Wandering minds: the default network and stimulus-independent thought. Science 315, 393–395. doi: 10.1126/science.1131295
Mathewson, K. E., Gratton, G., Fabiani, M., Beck, D. M., and Ro, T. (2009). To see or not to see: prestimulus α phase predicts visual awareness. J. Neurosci. 29, 2725–2732. doi: 10.1523/jneurosci.3963-08.2009
Matthews, G. (2002). Towards a transactional ergonomics for driver stress and fatigue. Theor. Issues Ergon. Sci. 3, 195–211. doi: 10.1080/14639220210124120
Matthews, G., Campbell, S. E., Falconer, S., Joyner, L. A., Huggins, J., Gilliland, K., et al. (2002). Fundamental dimensions of subjective state in performance settings: task engagement, distress, and worry. Emotion 2, 315. doi: 10.1037/1528-3542.2.4.315
Matthews, G., Reinerman-Jones, L. E., Barber, D. J., and Abich, J. IV (2015). The psychometrics of mental work- load: multiple measures are sensitive but divergent. Hum. Fact. 57, 125–143. doi: 10.1177/0018720814539505
Matthews, G., Warm, J. S., Reinerman, L. E., Langheim, L. K., and Saxby, D. J. (2010). “Task engagement, attention, and executive control,” in Handbook of Individual Differences in Cognition, eds A. Gruszka, G. Matthews, and B. Szymura (Berlin: Springer), 205–230. doi: 10.1007/978-1-4419-1210-7_13
May, J. F., and Baldwin, C. L. (2009). Driver fatigue: the importance of identifying causal factors of fatigue when considering detection and countermeasure technologies. Transport. Res. Part F Traff. Psychol. Behav. 12, 218–224. doi: 10.1016/j.trf.2008.11.005
McIntire, L. K., McKinley, R. A., Goodyear, C., and Nelson, J. (2014). A comparison of the effects of transcranial direct current stimulation and caffeine on vigilance and cognitive performance during extended wakefulness. Brain Stimul. 7, 499–507. doi: 10.1016/j.brs.2014.04.008
McKendrick, R., Parasuraman, R., and Ayaz, H. (2015). Wearable functional near infrared spectroscopy (fNIRS) and transcranial direct current stimulation (tDCS): expanding vistas for neurocognitive augmentation. Front. Syst. Neurosci. 9:27. doi: 10.3389/fnsys.2015.00027
Mehta, R. K., and Parasuraman, R. (2013). Neuroergonomics: a review of applications to physical and cognitive work. Front. Hum. Neurosci. 7:889. doi: 10.3389/fnhum.2013.00889
Modi, H. N., Singh, H., Orihuela-Espina, F., Athanasiou, T., Fiorentino, F., Yang, G. Z., et al. (2018). Temporal stress in the operating room: brain engagement promotes “coping” and disengagement prompts “choking”. Ann. Surg. 267, 683–691. doi: 10.1097/sla.0000000000002289
Modi, H. N., Singh, H., Yang, G. Z., Darzi, A., and Leff, D. R. (2017). A decade of imaging surgeons’ brain function (part I): terminology, techniques, and clinical translation. Surgery 162, 1121–1130. doi: 10.1016/j.surg.2017.05.021
Molloy, K., Griffiths, T. D., Chait, M., and Lavie, N. (2015). Inattentional deafness: visual load leads to time-specific suppression of auditory evoked responses. J. Neurosci. 35, 16046–16054. doi: 10.1523/jneurosci.2931-15.2015
Munakata, Y., Herd, S. A., Chatham, C. H., Depue, B. E., Banich, M. T., and O’Reilly, R. C. (2011). A unified framework for inhibitory control. Trends Cognit. Sci. 15, 453–459. doi: 10.1016/j.tics.2011.07.011
Murphy, S., and Dalton, P. (2016). Out of touch? Visual load induces inattentional numbness. J. Exp. Psychol. 42, 761. doi: 10.1037/xhp0000218
Nagahama, Y., Okina, T., Suzuki, N., Nabatame, H., and Matsuda, M. (2005). The cerebral correlates of different types of perseveration in the Wisconsin Card Sorting Test. J. Neurol. Neurosurg. Psychiatry 76, 169–175. doi: 10.1136/jnnp.2004.039818
Navarro, J., Mars, F., Forzy, J. F., El-Jaafari, M., and Hoc, J. M. (2010). Objective and subjective evaluation of motor priming and warning systems applied to lateral control assistance. Accident Anal. Prevent. 42, 904–912. doi: 10.1016/j.aap.2009.07.008
Navon, D., and Gopher, D. (1979). On the economy of the human-processing system. Psychol. Rev. 86, 214–225.
Nelson, J., McKinley, R. A., Phillips, C., McIntire, L., Goodyear, C., Kreiner, A., et al. (2016). The effects of transcranial direct current stimulation (tDCS) on multitasking throughput capacity. Front. Hum. Neurosci. 10:589.
Nelson, J. M., McKinley, R. A., McIntire, L. K., Goodyear, C., and Walters, C. (2015). Augmenting visual search performance with transcranial direct current stimulation (tDCS). Milit. Psychol. 27, 335–347. doi: 10.1037/mil0000085
Nelson, J. T., McKinley, R. A., Golob, E. J., Warm, J. S., and Parasuraman, R. (2014). Enhancing vigilance in operators with prefrontal cortex transcranial direct current stimulation (tDCS). Neuroimage 85, 909–917. doi: 10.1016/j.neuroimage.2012.11.061
Ninomiya, T., Noritake, A., Ullsperger, M., and Isoda, M. (2018). Performance monitoring in the medial frontal cortex and related neural networks: from monitoring self actions to understanding others’ actions. Neurosci. Res. 137, 1–10. doi: 10.1016/j.neures.2018.04.004
Norman, D. A., and Bobrow, D. G. (1975). On data-limited and resource-limited processes. Cognit. Psychol. 7, 44–64. doi: 10.1016/0010-0285(75)90004-3
O’Connell, R. G., Dockree, P. M., Robertson, I. H., Bellgrove, M. A., Foxe, J. J., and Kelly, S. P. (2009). Uncovering the neural signature of lapsing attention: electrophysiological signals predict errors up to 20 s before they occur. J. Neurosci. 29, 8604–8611. doi: 10.1523/jneurosci.5967-08.2009
O’Donnell, R. D., and Eggemeier, F. T. (1986). “Workload assessment methodology,” in Handbook of Human Perception and Performance, Vol. 2, eds K. Boff, L. Kaufman, and J. P. Thomas (New York, NY: Wiley), 42.1–42.49.
Oei, N. Y., Veer, I. M., Wolf, O. T., Spinhoven, P., Rombouts, S. A., and Elzinga, B. M. (2012). Stress shifts brain activation towards ventral ‘affective’areas during emotional distraction. Soc. Cognit. Affect. Neurosci. 7, 403–412. doi: 10.1093/scan/nsr024
O’Hare, D., and Smitheram, T. (1995). Pressing-on into deteriorating conditions: an application of behavioral decision theory to pilot decision making. Int. J. Aviat. Psychol. 5, 351–370. doi: 10.1207/s15327108ijap0504_2
Orasanu, J., Martin, L., Davison, J., and Null, C. H. (1998). Errors in Aviation Decision Making: Bad Decisions or Bad Luck? Moffett Field, CA: NASA Ames Research Center.
Palumbo, R. V., Marraccini, M. E., Weyandt, L. L., Wilder-Smith, O., McGee, H. A., Liu, S., et al. (2017). Interpersonal autonomic physiology : a systematic review of the literature. Personal. Soc. Psychol. Rev. 21, 99–141. doi: 10.1177/1088868316628405
Parasuraman, R. (2003). Neuroergonomics: research and practice. Theor. Issues Ergon. Sci. 4, 5–20. doi: 10.1080/14639220210199753
Parasuraman, R., and Rizzo, M. (2008). Neuroergonomics: The Brain at Work, 1st Edn. New York, NY: Oxford University Press, Inc.
Parasuraman, R., and Wilson, G. F. (2008). Putting the brain to work: neuroergonomics past, present, and future. Hum. Fact. 50, 468–474. doi: 10.1518/001872008X288349
Parasuraman, R., Mouloua, M., and Hilburn, B. (1999). “Adaptive aiding and adaptive task allocation enhance human-machine interaction,” in Automation Technology and Human Performance: Current Research and Trends (Mahwah, NJ: Erlbaum), 119–123.
Peavler, W. S. (1974). Pupil size, information overload, and performance differences. Psychophysiology 11, 559–566. doi: 10.1111/j.1469-8986.1974.tb01114.x
Pecher, C., Quaireau, C., Lemercier, C., and Cellier, J.-M. (2011). The effects of inattention on selective attention: how sad- ness and ruminations alter attention functions evaluated with the attention network test. Rev. Eur. Psychol. Appl. Eur. Rev. Appl. Psychol. 61, 43–50. doi: 10.1016/j.erap.2010.10.003
Pepin, G., Malin, S., Navarro, J., Fort, A., Jallaiz, C., Moreau, F., et al. (2016). “Detection of mind-wandering in driving: contributions of cardiac measurement and eye movements,” in Proceedings of the 1st International Neuroergonomics Conference: The Brain at Work and in Everyday Life, Amsterdam: Elsevier.
Pessoa, L., and Ungerleider, L. G. (2004). Neuroimaging studies of attention and the processing of emotion-laden stimuli. Prog. Brain Res. 144, 171–182. doi: 10.1016/s0079-6123(03)14412-3
Petersen, S. E., and Posner, M. I. (2012). The attention system of the human brain: 20 years after. Ann. Rev. Neurosci. 35, 73–89. doi: 10.1146/annurev-neuro-062111-150525
Peysakhovich, V., Lefrançois, O., Dehais, F., and Causse, M. (2018). The neuroergonomics of aircraft cockpits: the four stages of eye-tracking integration to enhance flight safety. Safety 4:8. doi: 10.3390/safety4010008
Polich, J. (2007). Updating P300: an integrative theory of P3a and P3b. Clin. Neurophysiol. 118, 2128–2148. doi: 10.1016/j.clinph.2007.04.019
Pope, A. T., Bogart, E. H., and Bartolome, D. S. (1995). Biocybernetic system evaluates indices of operator engagement in automated task. Biol. Psychol. 40, 187–195. doi: 10.1016/0301-0511(95)05116-3
Posner, M. I. (2012). Imaging attention networks. Neuroimage 61, 450–456. doi: 10.1016/j.neuroimage.2011.12.040
Posner, M. I., and Petersen, S. E. (1990). The attention system of the human brain. Annu. Rev. Neurosci. 13, 25–42.
Pourtois, G., De Pretto, M., Hauert, C. A., and Vuilleumier, P. (2006). Time course of brain activity during change blindness and change awareness: performance is predicted by neural events before change onset. J. Cognit. Neurosci. 18, 2108–2129. doi: 10.1162/jocn.2006.18.12.2108
Pribram, K. H., and McGuinness, D. (1975). Arousal, activation, and effort in the control of attention. Psychol. Review 82:116. doi: 10.1037/h0076780
Prinzel, L. J. III (2002). Research on Hazardous States of Awareness and Physiological Factors in Aerospace Operations. Report No. NASA/TM-2002-211444. Washington, DC: NASA.
Prinzel, L. J., Freeman, F. G., Scerbo, M. W., Mikulka, P. J., and Pope, A. T. (2000). A closed-loop system for examining psychophysiological measures for adaptive task allocation. Int. J. Aviat. Psychol. 10, 393–410. doi: 10.1207/s15327108ijap1004_6
Proulx, G. (2001). “Occupant behaviour and evacuation,” in Proceedings of the 9th International Fire Protection Symposium (Iceland: Iceland Fire Authority), 219–232.
Puschmann, S., Sandmann, P., Ahrens, J., Thorne, J., Weerda, R., Klump, G., et al. (2013). Electrophysiological correlates of auditory change detection and change deafness in complex auditory scenes. Neuroimage 75, 155–164. doi: 10.1016/j.neuroimage.2013.02.037
Qin, S., Hermans, E. J., van Marle, H. J., Luo, J., and Fernandez, G. (2009). Acute psychological stress reduces working memory-related activity in the dorsolateral prefrontal cortex. Biol. Psychiatry 66, 25–32. doi: 10.1016/j.biopsych.2009.03.006
Racz, F. S., Mukli, P., Nagy, Z., and Eke, A. (2017). Increased prefrontal cortex connectivity during cognitive challenge assessed by fNIRS imaging. Biomed. Opt. Exp. 8, 3842–3855.
Ramnani, N., and Owen, A. M. (2004). Anterior prefrontal cortex: insights into function from anatomy and neuroimaging. Nat. Rev. Neurosci. 5:184. doi: 10.1038/nrn1343
Ramsey, N. F., Jansma, J. M., Jager, G., Van Raalten, T., and Kahn, R. S. (2004). Neurophysiological factors in human information processing capacity. Brain 127, 517–525. doi: 10.1093/brain/awh060
Raveh, D., and Lavie, N. (2015). Load-induced inattentional deafness. Attent. Percept. Psychophys. 77, 483–492. doi: 10.3758/s13414-014-0776-2
Régis, N., Dehais, F., Rachelson, E., Thooris, C., Pizziol, S., Causse, M., et al. (2014). Formal detection of atten- tional tunneling in human operator–automation interactions. IEEE Trans. Hum. Mach. Syst. 44, 326–336. doi: 10.1109/thms.2014.2307258
Regis, N., Dehais, F., Tessier, C., and Gagnon, J.-F. (2012). “Human Factors: a view from an integrative perspective,” in Proceedings HFES Europe Chapter Conference Toulouse 2012, eds D. De Waard, K. Brookhuis, F. Dehais, C. Weikert, S. Röttger, D. Manzey, et al. (Toulouse: HFES). Available online at: http://hfes-europe.org
Reynal, M., Rister, F., Scannella, S., Wickens, C., and Dehais, F. (2017). “Investigating pilots decision making when facing an unstabilized approach: an eye-tracking study,” in Proceedings of the 19th International Symposium on Aviation Psychology, Dayton, OH, 335.
Reyner, L. A., and Horne, J. A. (1998). Evaluation of ‘in-car’countermeasures to sleepiness: cold air and radio. Sleep 21, 46–51.
Richter, M., Gendolla, G. H. E., and Wright, R. A. (2016). “Three decades of research on motivational intensity theory: what we have learned about effort and what we still don’t know,” in Advances in Motivation Science, ed. A. J. Elliot (Cambridge, MA: Academic Press), 149–186.
Ridderinkhof, K. R., Van Den Wildenberg, W. P., Segalowitz, S. J., and Carter, C. S. (2004). Neurocognitive mechanisms of cognitive control: the role of prefrontal cortex in action se- lection, response inhibition, performance monitoring, and reward-based learning. Brain Cognit. 56, 129–140. doi: 10.1016/j.bandc.2004.09.016
Riggs, S. L., and Sarter, N. (2019). Tactile, visual, and crossmodal visual-tactile change blindness: the effect of transient type and task demands. Hum. Fact. 61, 5–24. doi: 10.1177/0018720818818028
Rizzo, M., Robinson, S., and Neale, V. (2007). “The brain in the wild: tracking human behavior in natural and naturalistic settings,” in Neuroergonomics: The Brain at Work, eds R. Parasuraman and M. Rizzo (New York, NY: Oxford), 113–130.
Robbins, T. W., and Arnsten, A. F. (2009). The neuropsychopharmacology of fronto-executive function: monoaminergic modulation. Annu. Rev. Neurosci. 32, 267–287. doi: 10.1146/annurev.neuro.051508.135535
Roy, R. N., and Frey, J. (2016). “Neurophysiological markers for passive brain–computer interfaces,” in Brain–Computer Interfaces 1: Foundations and Methods eds M. Clerc, L. Bougrain, and F. Lotte (Hoboken, NJ: John Wiley & Sons), 85–100. doi: 10.1002/9781119144977.ch5
Russell, D., Statz, J. K., Ramiccio, J., Henderson, M., Still, D., Temme, L., et al. (2016). Pilot Cueing Synergies for Degraded Visual Environments (No. USAARL-2016-10). Fort Rucker, AL: US Army Aeromedical Research Laboratory Fort Rucker United States.
Saint Lot, J., Imbert, J.-P., and Dehais, F. (2020). “Red Altert: a cognitive countermeasure to mitigate attentional tunneling,” in Proceedings CHI 2020, April 25–30 (Honolulu, HI). doi: 10.1145/3313831.3376709
Sandson, J., and Albert, M. L. (1984). Varieties of perseveration. Neuropsychologia 22, 715–732. doi: 10.1016/0028-3932(84)90098-8
Sarason, I. G., Sarason, B. R., and Pierce, G. R. (1990). Anxiety, cognitive interference and performance. J. Soc. Behav. Personal. 5, 1–18.
Saravini, F. (1999). “Energy and the brain: facts and fantasies,” in Mind Myths, ed. E. Della Salla (Chichester: Wiley), 43–58.
Sarter, N., and Sarter, M. (2003). Neuroergonomics: opportunities and challenges of merging cognitive neuroscience with cognitive ergonomics. Theor. Issues Ergon. Sci. 4, 142–150. doi: 10.1080/1463922021000020882
Scannella, S., Causse, M., Chauveau, N., Pastor, J., and Dehais, F. (2013). Effects of the audiovisual conflict on auditory early processes. Int. J. Psychophysiol. 89, 115–122. doi: 10.1016/j.ijpsycho.2013.06.009
Scerbo, M. W. (2008). “Adaptive automation,” in Neuroergonomics: The Brain at Work, eds R. Parasuraman and M. Rizzo (New York, NY: Oxford), 239–252. doi: 10.1093/acprof:oso/9780195177619.003.0016
Schneider, W., Dumais, S. T., and Shiffen, R. M. (1984). “Automatic and control processing and attention,” in Varieties of Attention, eds R. Parasuraman and D. R. Davies (Orlando: Academic Press), 1–27.
Scholte, H. S., Witteveen, S. C., Spekreijse, H., and Lamme, V. A. (2006). The influence of inattention on the neural correlates of scene segmentation. Brain Res. 1076, 106–115. doi: 10.1016/j.brainres.2005.10.051
Schooler, J. W., Smallwood, J., Christoff, K., Handy, T. C., Reichle, E. D., and Sayette, M. A. (2011). Meta-awareness, perceptual decoupling and the wandering mind. Trends Cognit. Sci. 15, 319–326.
Schultz, W. (2002). Getting formal with dopamine and reward. Neuron 36, 241–263. doi: 10.1016/s0896-6273(02)00967-4
Sebok, A., Wickens, C. D., Walters, B., and Fennell, K. (2017). “Alerts on the nextgen flight deck,” in Proceedings of the 19th International Symposium on Aviation Psychology, Dayton, OH, 293.
Selfridge, O. G. (1959). “Pandemonium: a paradigm for learning,” in Proceedings of the Symposium on Mechanisation of Thought Processes, eds D. V. Blake and A. M. Uttley (London: H.M. Stationery Office), 511–529.
Senoussi, M., Verdière, K. J., Bovo, A., Ponzoni Carvalho, Chanel, C., Dehais, F., et al. (2017). “Pre- stimulus antero-posterior EEG connectivity predicts performance in a UAV monitoring task,” in Proceedings of 2016 International Conference on Systems, Man, and Cybernetics (Canada: IEEE SMC, 1167–1172.
Shallice, T., and Burgess, P. (1993). “Supervisory control of action and thought selection,” in Attention: Selection, Awareness and Control, eds A. Baddeley and L. Weiskrantz (Oxford: Clarendon Press), 171–187.
Smallwood, J., Beach, E., Schooler, J. W., and Handy, T. C. (2008). Going awol in the brain: mind wandering reduces cortical analysis of external events. J. Cognit. Neurosci. 20, 458–469. doi: 10.1162/jocn.2008.20037
Smallwood, J., and Schooler, J. W. (2015). The science of mind wandering: empirically navigating the stream of consciousness. Annu. Rev. Psychol. 66, 487–518. doi: 10.1146/annurev-psych-010814-015331
Souza, P. E., Chanel, C. P. C., Dehais, F., and Givigi, S. (2016). “Towards human-robot interaction: a framing effect experiment,” in IEEE International Conference on Systems, Man, and Cybernetics, (Budapest: IEEE SMC), 001929–001934.
Staal, M. A. (2004). Stress, cognition, and human performance: a literature review and conceptual framework.
Stamp, K., Fairclough, S., Dobbins, C., and Poole, H. (2019). “A neuroadaptive approach to analgesic gaming,” in The Second Neuroadaptive Technology Conference, Liverpool, 19.
Stephens, C., Dehais, F., Roy, R. N., Harrivel, A., Last, M. C., Kennedy, K., et al. (2018). “Biocybernetic adaptation strategies: machine awareness of human engagement for improved operational performance,” in International Conference on Augmented Cognition, Copenhagen, 89–98. doi: 10.1007/978-3-319-91470-1_9
Taillard, J., Capelli, A., Sagaspe, P., Anund, A., Akerstedt, T., and Philip, P. (2012). In-car nocturnal blue light exposure improves motorway driving: a randomized controlled trial. PLoS ONE 7:e46750. doi: 10.1371/journal.pone.0046750
Thomas, L. C., and Wickens, C. D. (2004). Eye-tracking and individual differences in off-normal event detection when flying with a synthetic vision system display. Proc. Hum. Fact. Ergon. Soc. Annu. Meet. 48, 223–227. doi: 10.1177/154193120404800148
Todd, J. J., Fougnie, D., and Marois, R. (2005). Visual short-term memory load suppresses temporo-parietal junction activity and induces inattentional blindness. Psychol. Sci. 16, 965–972. doi: 10.1111/j.1467-9280.2005.01645.x
Tombu, M. N., Asplund, C. L., Dux, P. E., Godwin, D., Martin, J. W., and Marois, R. (2011). A unified attentional bottleneck in the human brain. Proc. Natl. Acad. Sci. U.S.A. 108, 13426–13431. doi: 10.1073/pnas.1103583108
Tracy, J. I., Mohamed, F., Faro, S., Tiver, R., Pinus, A., Bloomer, C., et al. (2000). The effect of autonomic arousal on attentional focus. Neuroreport 11, 4037–4042. doi: 10.1097/00001756-200012180-00027
Tsai, Y.-F., Viirre, E., Strychacz, C., Chase, B., and Jung, T.-P. (2007). Task performance and eye activity: predicting be- havior relating to cognitive workload. Aviat. Space Environ. Med. 78, B176–B185.
Ullsperger, M., Danielmeier, C., and Jocham, G. (2014). Neurophysiology of performance monitoring and adaptive behavior. Physiol. Rev. 94, 35–79. doi: 10.1152/physrev.00041.2012
Ullsperger, M., Nittono, H., and von Cramon, D. Y. (2007). When goals are missed: dealing with self-generated and externally induced failure. NeuroImage 35, 1356–1364. doi: 10.1016/j.neuroimage.2007.01.026
Unsworth, N., and Engle, R. W. (2007). The nature of individ- ual differences in working memory capacity: active main- tenance in primary memory and controlled search from secondary memory. Psychol. Rev. 114, 104. doi: 10.1037/0033-295x.114.1.104
Uzzaman, S., and Joordens, S. (2011). The eyes know what you are thinking: eye movements as an objective measure of mind wandering. Conscious. Cognit. 20, 1882–1886. doi: 10.1016/j.concog.2011.09.010
Van Acker, B. B., Parmentier, D. D., Vlerick, P., and Saldien, J. (2018). Understanding mental workload: from a clarifying concept analysis toward an implementable framework. Cognit. Technol. Work 20, 351–365. doi: 10.1007/s10111-018-0481-3
Van Dongen, H., Belenky, G., and Krueger, J. M. (2011). A local, bottom-up perspective on sleep deprivation and neurobehavioral performance. Curr. Top. Med. Chem. 11, 2414–2422. doi: 10.2174/156802611797470286
Verwey, W. B., and Zaidel, D. M. (1999). Preventing drowsiness accidents by an alertness maintenance device. Accident Anal. Prevent. 31, 199–211. doi: 10.1016/s0001-4575(98)00062-1
Vijayraghavan, S., Wang, M., Birnbaum, S. G., Williams, G. V., and Arnsten, A. F. (2007). Inverted-U dopamine D1 receptor actions on prefrontal neurons engaged in working memory. Nat. Neurosci. 10:376. doi: 10.1038/nn1846
Weinmann, M., Schneider, C., and vom Brocke, J. (2016). Digital nudging. Bus. Inform. Syst. Eng. 58, 433–436. doi: 10.1007/s12599-016-0453-1
Weissman, D. H., Roberts, K. C., Visscher, K. M., and Woldorff, M. G. (2006). The neural bases of momentary lapses in attention. Nat. Neurosci. 9:971. doi: 10.1038/nn1727
Wickens, C. D. (1984). “Processing resources in attention,” in Varieties of Attention, eds R. Parasuraman and D. R. Davies (London: Academic Press), 63–101.
Wickens, C. D. (2002). Multiple resources and performance prediction. Theor. Issues Ergon. Sci. 3, 150–177.
Wickens, C. D. (2005). Attentional tunneling and task management. Int. Symp. Aviat. Psychol. 812–817.
Wickens, C. D., and Liu, Y. (1988). Codes and modalities in multiple resources: a success and a qualification. Hum. Fact. 30, 599–616. doi: 10.1177/001872088803000505
Wickens, C. D., and Tsang, P. (2014). “Workload,” in Handbook of Human-Systems Integration, ed. F. Durso (Washington, DC: APA).
Wickens, J. R., Horvitz, J. C., Costa, R. M., and Killcross, S. (2007). Dopaminergic mechanisms in actions and habits. J. Neurosci. 27, 8181–8183. doi: 10.1523/jneurosci.1671-07.2007
Wierwille, W. W., and Eggemeier, F. T. (1993). Recommendation for mental workload measurement in a test and evaluation environment. Hum. Fact. 35, 263–281. doi: 10.1177/001872089303500205
Yeh, Y. Y., and Wickens, C. D. (1988). Dissociation of performance and subjective measures of workload. Hum. Fact. 30, 111–120. doi: 10.1177/001872088803000110
Yerkes, R. M., and Dodson, J. D. (1908). The relation of strength of stimulus to rapidity of habit formation. J. Compar. Physiol. Psychol. 18, 459–482. doi: 10.1002/cne.920180503
Young, M. S., Brookhuis, K. A., Wickens, C. D., and Hancock, P. A. (2015). State of science: mental workload in ergonomics. Ergonomics 58, 1–17. doi: 10.1080/00140139.2014.956151
Young, M. S., and Stanton, N. A. (2002). Malleable attentional resources theory: a new explanation for the effects of mental underload on performance. Hum. Fact. 44, 365–375. doi: 10.1518/0018720024497709
Zander, T. O., and Kothe, C. (2011). Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general. J. Neural Eng. 8:025005. doi: 10.1088/1741-2560/8/2/025005
Keywords: neuroergonomics, performance prediction, degraded attentional and executive mental states, task engagement, mental workload
Citation: Dehais F, Lafont A, Roy R and Fairclough S (2020) A Neuroergonomics Approach to Mental Workload, Engagement and Human Performance. Front. Neurosci. 14:268. doi: 10.3389/fnins.2020.00268
Received: 11 December 2019; Accepted: 10 March 2020;
Published: 07 April 2020.
Edited by:
Riccardo Poli, University of Essex, United KingdomReviewed by:
Ranjana K. Mehta, Texas A&M University, United StatesRyan McKendrick, Northrop Grumman Corporation, United States
Copyright © 2020 Dehais, Lafont, Roy and Fairclough. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Frédéric Dehais, ZnJlZGVyaWMuZGVoYWlzQGlzYWUtc3VhcGVyby5mcg==; ZnJlZGVyaWMuZGVoYWlzQGlzYWUuZnI=