Marcel F. Hinss
Anke M. Brock
Raphaëlle N. Roy- 1Institut Supérieur de l'Aéronautique et de l'Espace (ISAE-SUPAERO), Toulouse, France
- 2Ecole Nationale de l'Aviation Civile (ENAC), Université de Toulouse, Toulouse, France
Operators of complex systems across multiple domains (e.g., aviation, automotive, and nuclear power industry) are required to perform their tasks over prolonged and continuous periods of time. Mental fatigue as well as reduced cognitive flexibility, attention, and situational awareness all result from prolonged continuous use, putting at risk the safety and efficiency of complex operations. Mental state-based adaptive systems may be a solution to this problem. These systems infer the current mental state of an operator based on a selection of metrics ranging from operator independent measures (e.g., weather and time of day), to behavioral (e.g., reaction time and lane deviation) as well as physiological markers (e.g., electroencephalography and cardiac activity). The interaction between operator and system may then be adapted in one of many ways to mitigate any detected degraded cognitive state, thereby ensuring continued safety and efficiency. Depending on the task at hand and its specific problems, possible adaptations -usually based on machine learning estimations- e.g., include modifications of information, presentation modality or stimuli salience, as well as task scheduling. Research on adaptive systems is at the interface of several domains, including neuroergonomics, human factors, and human-computer interaction in an applied and ecological context, necessitating careful consideration of each of the aforementioned aspects. This article provides an overview of some of the key questions and aspects to be considered by researchers for the design of mental state-based adaptive systems, while also promoting their application during prolonged continuous use to pave the way toward safer and more efficient human-machine interaction.
1. Introduction
Many domains of study have over the past century seen a rapid increase in efficiency accompanied by a shift to ever more complex systems (Hollnagel, 2009). Operators now work with more complex systems, that are often larger in scale, productivity, and effectiveness, which in turn makes errors much more costly. For instance, less than a 100 years ago, between the world wars, commercial planes usually carried a lot less than 50 people (Linden and Seely, 2011), on the other hand, the worst aviation disaster claimed more than ten times that amount of lives in 1977 (McCreary et al., 1998). There are many examples that highlight the importance of studying the human-machine interaction in order to avoid catastrophic errors. The three Mile Island incident, although not caused by human error was exacerbated by faulty human-machine interaction (HMI). When radioactive hydrogen from a nuclear plant leaked into the environment, operators were inhibited in their efforts to control the damage by a multitude of alarms and faulty interface design (Cummings, 1980). The operators did not regain control immediately, resulting in further leakage. In another accident, a broken light bulb distracted the crew of an aircraft to the point that they did not realize they were flying into the ground (O'Brien and Bull Schaefer, 2020). As these examples show, neither the machine nor the operator is to be seen as an independent unit, and only if both parts perform adequately can safety be ensured (Ke et al., 2018). Other than machines, humans are prone to mental fatigue, resulting in increased risk and errors (Matthews, 2012).
What can be done if the operator cannot just be taken out of the equation by means of automation or by replacement? The effects of Time on Task (TOT), the time since the onset of a task, may be mitigated by adaptive systems. The idea of such systems is to accommodate the mental state of the operator according to their current needs and the current mission requirements (Hansberger, 2015). The ways in which systems accomplish this are vast. Adaptive automation partially or completely automates the operation in given circumstances (e.g., an auto pilot in a plane). Adaptive interfaces, on the other hand, do not automate operations, but change the interaction, by adapting their interface. Examples of this are plentiful, such as adaptive driving modes (Kesting et al., 2007), or automated night and day modes for smartphones (Langley, 1999; Lunn, 2022). These systems are all very useful, yet they are largely operator independent. However, individual differences such as experience and preferences influence how an operator's mental state may change over time (Matthews, 2012). Consequently, adaptive systems need to take into consideration the mental state of the operator. Examples of mental state-based adaptive systems often include a component of physiological computing. The idea of physiological computing is to estimate the current mental state of a human by using physiological measures such as electroencephalography (EEG), or eye-tracking (Roy et al., 2020). Adaptable systems on the other hand avoid these difficulties of estimating the mental state of the operator by leaving the human in control of when and whether to activate any change in automation or interface. For mental state-based system design, adaptive systems are most likely preferable, as they do not rely on the assumption that a human can accurately infer their own mental state (Findlater and McGrenere, 2004).
Surprisingly, relatively little research has explored the possibility of mental state-based adaptive interfaces for operator support during prolonged continuous use. It is the aim of this article to take a step toward that direction, by examining the current state of research within the field to try and answer the following questions:
• What are the effects of prolonged continuous use on cognition and behavior?
• How should an adaptive interface infer the current state of the operator?
• How can adaptive interfaces affect the interaction between humans and systems to mitigate said effects?
This article was meant to address these issues using a non systematic review process. Indeed, the study in this article reviews the effects of prolonged use on cognition as well as research on a mental state based on adaptive interfaces and different classes of metrics for mental fatigue detection. Literature authored in English was selected based on the relevance to the topic and its scientific rigor. As a result of this, the list of effects of mental fatigue on cognition in Section 2, possible and implemented adaptations in Section 3, and the set of metrics for mental fatigue detection in Section 4 is not exhaustive. The principal search engine used for the literature review was Google Scholar. In addition, the ACM (www.acm.org) and IEEE (www.ieee.org) databases were searched. Although no hard inclusion and exclusion criteria were defined, articles that were not peer reviewed were generally avoided. Furthermore, as adaptable systems and HMI are rapidly evolving fields, there was a preference for more recent literature, mostly less than 10 years old. Each of the included articles was assessed based on its scientific rigor. If articles were not deemed rigorous in all aspects they were either excluded or any potential shortcomings in the design were mentioned in the article (Kable et al., 2012).
The following section will examine how prolonged use can affect the HMI by looking at its effects on cognition and behavior.
2. Prolonged continuous use, its effects, and its relation to mental fatigue
“Prolonged continuous use” or “prolonged operation," as is defined here refers to the exercising of a task for long periods of time or beyond an expected duration. In experimental sciences, it is often referred to as TOT. Closely related to this concept and almost always studied together is the concept of mental fatigue. Mental fatigue, in contrast to physical fatigue, arises through continuous cognitive effort and may manifest itself in decreased cognitive performance, lack of motivation, or exhaustion (Boksem and Tops, 2008; Ackerman and Kanfer, 2009; Matthews, 2012). This article focuses exclusively on the cognitive aspects of mental fatigue. Mental fatigue is not the only result of prolonged continuous use. Nevertheless, both terms are related (Trejo et al., 2015). Mental fatigue is often studied by having participants perform a task for a prolonged time. Therefore, studies of mental fatigue that make use of this research design inevitably also study TOT. Such a within subjects design is used in Yu et al. (2021), Csathó et al. (2012), Guo et al. (2016), Plukaard et al. (2015), Lorist et al. (2009), Monsell et al. (2003), Krishnan et al. (2014), and Dorrian et al. (2007), while van der Linden et al. (2003a) and van der Linden et al. (2003b), use a different design. In the latter, one group performs a mentally fatiguing, but irrelevant task before performing the main task while another group does nothing before completing only the relevant task.
It is beyond the goal of this review to explore all effects of mental fatigue and TOT in detail. Rather, it will be shown that prolonged continuous use and mental fatigue have effects on critical aspects of HMI. Table 1 summarizes the effects of prolonged continuous use on cognition which are then detailed in the subsequent sections.
Table 1. Summary of prolonged continuous use effects on cognition.
2.1. Mental fatigue and attention
Attention is a critical component in many aspects of human-machine interaction (Roda, 2011). Attention is a broad concept and can encompass many sub types. A simple categorization is accomplished by dividing into exogenous “Top-Down” or endogenous “Bottom-Up” processes (Theeuwes, 1991). Top-Down attention is a voluntary allocation of attention toward some specific feature or object. For example, humans are capable of focusing on just one person speaking, even in a crowded and loud room. This is also known as the cocktail party effect (Bronkhorst, 2000). Bottom-Up attention is the exact opposite, it is involuntary attention that is strongly guided by salient objects in the environment. This is the capability to hear our own name from far away despite not paying any attention to that person (Katsuki and Constantinidis, 2014).
The general consensus seems to exist that mental fatigue has a much more pronounced effect on top-down attentional processes, as demonstrated by Holtzer et al. (2010). In this study, participants had to perform an Attention Networks Test (ANT). The ANT is based on a flanker task (Eriksen and Eriksen, 1974) and interpretation of the behavioral results allows for distinguishing between effects on bottom-up and top-down attention. After performing the task for a prolonged period, bottom-up attention showed no change due to TOT, whereas top-down attention performance significantly decreased. Similarly, Csathó et al. (2012) showed a decrease in performance (increase in reaction time (RT) and decrease in accuracy (ACC) on a visual attention task, as mental fatigue increased. Interestingly, even though participants showed initial performance enhancements, most likely due to learning effects, fatigue in the end resulted in decreased performance. Further support for the effect of fatigue on attention was seen in a study by Sun et al. (2014). Over time, attention decreased and, subsequently, reaction time increased.
Mental fatigue also impacts spacial visual sustained attention. In a study by Guo et al. (2016) participants had to perform a visual task over 63 min in which they had to track a target and respond to random stimuli. The results of this dual-task setup highlighted the participants' inability to maintain fast reaction times and accuracy as mental fatigue increased.
2.2. Cognitive flexibility, task-switching, and multitasking
The relationship between fatigue and cognitive flexibility has been less explored, but it may nevertheless be an important variable of success for prolonged use (van der Linden et al., 2003b; Lorist et al., 2005; Plukaard et al., 2015; Yu et al., 2021). Cognitive flexibility is the ability of an individual to rapidly change from one task to another task. This task-switching usually incurs a cost, resulting in increases in reaction time and decreases in accuracy (De Jong, 1995). As complex systems are often composed of multiple sub-systems (see Lim et al., 2018 as an example of the evolution of control surfaces in cockpits), between which operators have to switch, cognitive flexibility becomes of major importance to operations and prolonged continuous use (Squire and Parasuraman, 2010). The term cognitive flexibility is often used interchangeably with task-switching, while some researchers make a distinction between cognitive flexibility, seen as the ability to switch between tasks and task-switching the actual process of switching. For the remainder of this article, we will follow the latter notion (Ionescu, 2012).
Some recent evidence for decreases in task-switching performance as a result of mental fatigue comes from a 2021 study by Yu et al. (2021). In their study, 33 participants had to perform a prolonged task switching experiment. The task consisted of trials where a rule had to be followed, or a new rule had to be adapted. Behavioral results showed fatigue and task-switching effects. Over time, RT and ACC performance decreased. The surprising finding was that over time, the difference between switching trials and regular trials grew larger. Meaning, that beyond impacting general performance, mental fatigue had an even more profound effect on cognitive flexibility. Similar results were reported by Lorist et al. (2009), Petruo et al. (2018), and Plukaard et al. (2015). Petruo et al. (2018) had used a similar paradigm. However, here the task switching was initiated either by an explicit cue or it had to be based on working memory. Results mirrored those in Yu et al. (2021). Furthermore, the study showed that working memory had a particular impact on the task-switching ability when participants were fatigued.
Contrary results have also been shown. Lorist et al. (2000) studied cognitive flexibility, fatigue, and preparation time. The study showed that switch trials resulted in lower performance and that fatigue had an impact on performance. However, an interaction between fatigue and cognitive flexibility was not found. Reaction times during switch-trials were significantly lower if participants had longer inter-stimulus intervals to prepare for the switch. Even though the overall time to perform an action (interval + reaction time) increased, the relative decrease in reaction time suggests, that participants were better able to cope if given time to prepare for a switch. No effects on accuracy were seen, which may be attributed to ceiling effects, as accuracy remained very high at all times. Similar results are reported in Monsell et al. (2003).
While task-switching is strictly sequential, complex systems often require true multitasking. Task-switching and multitasking have often been investigated separately. Task-switching tasks are always tested sequentially, while multitasking (almost always dual-task) involves concurrent task engagement. However, as laid out by Koch et al. (2018), there are many similarities between task switching and multitasking. Many of the characteristics such as switch-costs also exist for multitasking. This has led to a proposal to see task-switching and multitasking not as different entities, but rather as the same process with only the degree to which the task is sequential differing along a continuum (Salvucci et al., 2009). The link between multi-tasking and mental fatigue is to the best of our knowledge not very well explored. Multitasking is more often investigated as a cause of mental fatigue, rather than examining the effect of mental fatigue on multitasking performance (Kudesia et al., 2022). Given that multitasking is argued to exist on the same continuum as task-switching, it is not unreasonable to expect that performance on multitasking may be impacted by fatigue in similar ways as task-switching (Salvucci et al., 2009).
2.3. Other effects of mental fatigue and situational awareness
Beyond the main factors that were already explored, it is worth looking at some other aspects that may play a particular role when it comes to interaction with complex systems. The article by van der Linden et al. (2003a) showed that participants' performance on a Wisconsin Card Sorting Test, got significantly worse if they were subjected to a mentally fatiguing exercise beforehand. The Wisconsin Card Sorting Test (Monchi et al., 2001) is a neurological assessment tool, in which participants are presented with cards. They are instructed to sort the cards according to unknown rules that constantly change. Based on feedback, they then have to adapt their behavior. It was shown that fatigued participants took longer to adapt to this changing of rules than non-fatigued participants, a behavior known as preservation. In the same study, participants also completed the Tower of Hanoi task (Rönnlund et al., 2001), testing their ability to plan ahead. Fatigued participants took significantly longer to make their first move, reflecting difficulties with the initial planning. Similar effects were observed following the prolonged operation of an Air-Traffic Control (ATC) task (Krishnan et al., 2014). After performing about 60 min of simulated ATC work, participants took significantly longer for their tasks.
In another study, also relying on a more ecologically valid design, prolonged continuous use was examined in the context of train driving (Dorrian et al., 2007). Professional train conductors had to perform two simulated 8 h shifts of train driving on different days. From no to moderate fatigue, the operators increased the amounts of errors of commission. For example, they used the brakes more than they had to. When extremely fatigued, they no longer made errors of the commission but made a lot more errors of omission. A likely explanation for these errors is due to decreased attention. Errors of omission may also be seen as the failure to explore the environment at hand. Van der Linden et al. investigated this possibility by studying the effect of mental fatigue on systematic exploration behavior (van der Linden et al., 2003b). Participants either had to do mentally fatiguing tasks or wait before completing an experimental computerized task. Within the complex interface, it was observed to what degree the participants used systematic exploration. Mental fatigue proved to have an impact, reducing systematic exploration significantly.
Another difficulty that may arise as a result of prolonged operations is alert fatigue. This phenomenon describes desensitization to safety alerts as a result of repeated exposure to these events (Cash, 2009). This problem has repeatedly been demonstrated in the health care sector but also in mining operations and construction (Blackmon and Gramopadhye, 1995; Bliss et al., 1995). Decreased adaptation to change, difficulties with planning, reduced exploration, errors of omission, and alert fatigue can be seen as independent entities, but may also be seen as parts of the more general concept of situational awareness (SA; Endsley, 1995). SA is a key concept that needs to be considered for any interaction with complex systems. Endsley defines SA as “the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future (Endsley, 1995). Stanton et al. explain, that SA encompasses an understanding of the past and current environment, reflections about potential future events and their meaning/relevance as well as awareness of goals and goal directed behavior (Stanton et al., 2001).
Loss of SA can have dire consequences, as perhaps best highlighted by the gruesome accident of an L1011 Lockheed Tristar of Eastern Air Lines in Florida. The pilot, co-pilot, and flight engineer of flight 401 did not realize they were losing altitude, as all three were focused on repairing a broken light bulb (O'Brien and Bull Schaefer, 2020). SA is simultaneously a vital factor during complex operations and at perpetual risk to be undermined by mental fatigue, making it an attractive target for support through adaptive interfaces, which are explored in the following section.
3. Adaptive interfaces and the question of how to adapt
3.1. General considerations for adaptive interfaces
Adaptive Interfaces aim at improving human-machine interaction by changing their behavior to accommodate the needs of the users (Hansberger, 2015). Before considering the context of prolonged continuous use, it is worth exploring some of the general considerations that are known to influence the efficacy of adaptive interfaces. Adaptations are not all or nothing phenomena, and careful calibration of several variables is required to improve the performance of operators (Taylor et al., 2013).
3.1.1. Trust
For the interaction between humans and machines trust is of vital importance. If an operator has no trust in a system they may decide not to engage with it, consequently not making any use of its assistance. On the other hand, putting too much trust in a system can have similarly detrimental effects (Lee and See, 2004). For instance, the crew of the Panamanian passenger ship Royal Majestic placed too much trust in their auto navigation systems. Complacency over 24 h resulted in the ship running around, inflicting costs of over $7 million. Fortunately, there were no fatalities (van de Merwe et al., 2016).
To avoid both over-trust and mistrust, Lee and See (2004) propose the notion of calibrated trust for automated systems. Calibrated trust occurs when the trust an operator has in a system is equal to the true trustworthiness of the system. To achieve this, three variables are identified as critical components:
1. Performance refers to how well the automation is performing and has performed.
2. Process is the degree, to which the operator understands, how the automation operates.
3. Purpose finally explains, to what extent the automation performs what it was intended to do.
Negligence in accommodating these dimensions during the development of an adaptive system can result in both mistrust and over-trust. In automation based adaptations, this is easy to imagine, as a low performance system will likely result in the operator denying the automation of tasks.
The need for trust in automation is being continuously explored, while questions regarding trust in adaptive interfaces remain, to the best of our knowledge, unanswered.
Mental state based adaptive interfaces have to make some form of inference on the current mental state of the operator (Aricò et al., 2016; Park et al., 2021). These metrics that are explored extensively in Section 4 determine when and how a system will adapt. The operator has to have trust in the ability of the system to accurately predict their own current mental state. This is further exacerbated when a system decides to trigger a system adaptation. The triggering will inevitably signal the operator about the estimation of the system on the current interaction. Any discrepancy between the operators' perception of the interaction and that of the interface will probe trust between the human and the system. If the operator does not trust in the accuracy of the interface, they might decide to disengage. This applies equally to adaptive automation. To the best of our knowledge, this area of research is not very explored, but given the rise of adaptive systems, exploring trust in adaptive systems may be critical for future advances.
In order to incorporate trust during the design of an interface, several rules should be appreciated. Again following the advice of Lee and See (2004), for adaptive automation, it is proposed to:
1. Design for appropriate trust, not greater trust.
2. Show the past performance of the automation.
3. Show the process and algorithms of the automation by revealing intermediate results in a way that is comprehensible to the operators.
4. Simplify the algorithms and operation of the automation to make it more understandable.
5. Show the purpose of the automation, design basis, and range of applications in a way that relates to the users' goals.
6. Train operators regarding its expected reliability, the mechanisms governing its behavior, and its intended use.
7. Carefully evaluate any anthropomorphization of the automation, such as using speech to create a synthetic conversational partner, to ensure appropriate trust.
3.1.2. Design considerations for effective adaptive interfaces
There are many other aspects beyond trust that need to be considered to achieve high usability for adaptive systems (Hou et al., 2007). Understanding the process, or the inner workings of adaptations, is not only critical for trust building but has been shown to play a role in the overall success of adaptions (Scott et al., 2006; Kiefer et al., 2017; Xing et al., 2021). This claim is supported in the study of Saint-Lot et al. (2020), where understanding the adaptation resulted in significantly better performance. Another important aspect is to allow the operator to be aware at all times of what the adaptation is currently doing (if it is activated or not Pritchett et al., 2014). Kiefer et al. (2017) and Scott et al. (2006) go even further and propose that the operator should have the capability to take control (deactivate the adaptation) at all times. Pritchett et al. (2014) stress the importance of handling interruptions.
The operator may get “out of the loop” either by adaptations being interrupted or by the adaptation presenting an interruption to its current tasks itself. Pritchett et al. (2014) place further emphasis on accommodating individual differences in character, preferences, and skills. This point is also raised by Hou et al. (2011). While designing adaptations secondary effects need to also be taken into account. An adaptation may prove successful on one measure while decreasing performance on another one. As in the study by Chen and Barnes (2014), where behavioral performance increased with adaptation, while Situational Awareness decreased. The problem of secondary effects opens up the more general question of how to evaluate the effectiveness of an adaptation. In simple tasks, this may be relatively easy, but as soon as the tasks become more akin to simulations of realistic complex environments, a multitude of variables need to be accounted for (Pritchett et al., 2014). The evaluation of an adaptive system is a critical aspect to consider when designing and testing a system. Furthermore, a system needs to be designed in a way, that it can function despite the human not doing what they were intended to Pritchett et al. (2014). Boundary conditions also deserve attention, as the system needs to be optimized as to when an adaptation should be activated or deactivated (Pritchett et al., 2014). Scalability and the possibility to increase complexity should also be taken into consideration. Designs that do not allow for this, lack realism and ecological validity (Hou et al., 2011). Adaptive systems, according to this doctrine, should be built in a way that they could accommodate future additions such as multiple levels of interactions, multiple modes, incorporating individual differences, adding a hierarchy of adaptations, and allowing for multiple ways to activate and deactivate the adaptations (Lajos, 1995; Höök, 2000). Finally, operator comfort is another noteworthy consideration when designing interfaces, even more so for prolonged use. Using visually comforting stimuli within an interface may reduce ocular fatigue (Ladouce et al., 2021). These considerations do not necessarily limit the design options regarding adaptive systems, but they also guide toward future, more viable systems.
3.2. Adaptive interfaces in the context of prolonged use
In their article, Feigh et al. (2012) propose a provisional taxonomy of all types of adaptations. In this taxonomy, no clear distinction between adaptive automation and adaptive interfaces is made. This difference deserves special consideration for prolonged continuous use. Adaptive interfaces in which large changes in the levels of automation occur (Parasuraman et al., 2000), may result in the change in the task being so fundamental, that it can no longer be considered as the same task and, therefore, no longer prolonged continuous use. For this reason, we propose an adapted taxonomy in Figure 1.
Figure 1. Proposed provisional taxonomy of all types of adaptations based on Feigh et al. (2012).
Central to the question of how to adapt an interface, is the question of what the goal of the adaptation is. To accommodate the effects of prolonged use and mental fatigue may mean inhibiting their onset. Alternatively, adaptive systems may set out to mitigate the effects of prolonged use, and accommodate the changes in cognition of the operator. Some research has shown the effects, of caffeine, stream-bathing, chicken essence, and physical activity as ways of reducing mental fatigue (Mizuno et al., 2009; Yamano et al., 2013; Van Cutsem et al., 2018; Jacquet et al., 2021), but these options lack in practical applicability in many critical situations. External motivation, especially in form of a reward, has on the other hand shown to effectively reduce mental fatigue (Hopstaken et al., 2015). Nonetheless, motivation may not be a viable solution due to two issues: It is unclear how long the effect of external motivation on an operator lasts and as suggested by Christie and Schrater (2015) it may only allow for short term mitigation. In addition, systems that deliver a reward tend to result in exploitation behavior by operators. The sole objective becomes maximizing direct reward (the external motivation), thereby neglecting other aspects of the system (Dubey, 1986). As direct mitigation of fatigue seems to have very little practicality in this context, efforts to contain the effects of mental fatigue seem more viable. In the following sections, we explore adaptive interfaces and ideas for adaptive interfaces related to operators' attention, cognitive flexibility, and situational awareness. Table 2 summarizes adaptations for different types of mental states, as well as their effects and the application domain.
Table 2. Summary of adaptations for various mental states.
3.2.1. Attention
Adaptive interfaces aimed at mitigating problems of attention have generally taken one of two approaches. Most adaptations, be it for unmanned vehicles, robots, etc. have used automation of some tasks to allow the operator to focus on a reduced set of tasks. Parasuraman et al. (2005) examined if partial automation of responsibilities could help operators during unmanned vehicle operations. Participants had to control both a UAS and an unmanned ground vehicle. A performance based system of adaptive automation was integrated into the task. If the performance of operators on a change detection task deteriorated, then parts of their operation were automated. A significant improvement in attention was shown, if operators were supported by the adaptive automation, as opposed to a condition without adaptive automation.
Alternatively, a set of studies have explored exploiting the benefits of bottom-up attention by adapting the salience of alarms and information, thereby decreasing the risk of misses. In order to accommodate increased mental workload in air traffic controllers, Aricò et al. (2016) proposed several adaptations, of which two focused on attention. To facilitate locating an aircraft, the system had the ability to highlight aircraft if they were calling the controller. In addition, if there was a risk of a mid-air collision, the interface would show a specific animation, highlighting the emergency. The result of this study showed, that if adaptations were activated during high mental workload, these were capable of improving performance and allowing for recovery from high mental workload states. A similar adaptive interface was used to increase the attention of UAS operators. In a preliminary study, participants had to fly a simulated UAS, while attention was monitored using a mobile EEG setup. Whenever the participants' attention decreased, they were notified of that decrease with an alert (Park et al., 2021). Effects were not significant, but this might be due to the small sample size (N = 4). Success was also seen with the air traffic control “Red Alert” system. Here, the entire screen was draped in a red hue, if there was a risk of a mid-air collision between aircraft. The increase in salience improved the behavior during the ATC task (Saint-Lot et al., 2020). The effectiveness of alarm-based systems is closely related to their accuracy. Inaccurate alarms should be avoided at all costs, as both false alarms and nuisance alarms can be detrimental to performance (alarm fatigue; Onnasch et al., 2014; Wickens and McCarley, 2019; Ruskin et al., 2021). Therefore, several adaptive interfaces have used alarm suppression to increase operational performance. These systems not only avoid alert fatigue but also allow operators to focus attention on relevant objectives, rather than low-priority alarms (Liu et al., 2001; Breznitz, 2013). This has been shown to be effective in chemical plants, where an alarm appeared approximately every 2 min, risking that the operator would disregard a true alarm (Liu et al., 2001). The concept was also applied to car phones, which would automatically direct calls to the mailbox if the mental workload of the driver was too elevated (Piechulla et al., 2003). Comparable to this are Airbus aircraft, which suppress alarms that are not critical during take-off and landing, as focusing on the task at hand may be disturbed by unnecessary alarms (Airbus, 2008). Mirroring the complexity of the concept of attention itself, the examined methods of mitigation differ widely in their approach and may even seem to contradict each other. Adaptive interactions targeting attention may best be summarized, not as an effort of adding or removing alerts and information, but rather as an effort of effective attentional management. The attentional load needs to be carefully calibrated respecting the operational demands, state of a given mission, and state of the operator (Piechulla et al., 2003).
3.2.2. Cognitive flexibility
The effects of task-switching in complex operations are well explored in the literature (Vandierendonck et al., 2010; Koch et al., 2018 for review), however relatively little research has been conducted on adaptive systems and cognitive flexibility. Some studies still point toward some possible solutions for reducing task-switching costs. Arrington and Logan (2004) investigated if voluntary switches, as opposed to scheduled switches, could reduce task-switching costs. Surprisingly, it was shown that task-switching costs still persisted in the form of higher reaction times even if switches were voluntary. Similar effects on accuracy were seen in a follow-up study (Arrington and Yates, 2009). Making switches voluntary seems to reduce the tax on cognitive flexibility, but within complex operations, it is questionable if switches can be made voluntary or have to occur at a specific time or as a reaction to critical incidents. A promising idea is to provide operators with preparation time or cues before switching. As explored in a previous section, some studies (Lorist et al., 2000; Monsell et al., 2003) showed that preparation time and cuing could reduce reaction time in trials (overall reaction time was not shorter if the added preparation time was taken into account). The effect of preparation on accuracy is not conclusively reported in the articles, as error rates were low across all conditions (possible ceiling effect) and no statistics were reported. Rubinstein et al. (2001) reported error rates in their study on cued task-switching. While not reporting any statistics regarding the significance of the data, an almost 30% reduction in errors was seen if trials were cued. These results make cuing task-switches a promising candidate for adaptive interfaces, especially if short delays are acceptable for increases in accuracy. As Petruo et al. (2018) had found, prolonged use in combination with the necessity of using working memory may be particularly detrimental to cognitive flexibility. Adaptive systems that outsource working memory-might prove viable. Interestingly, adaptive automation may have a negative effect on cognitive flexibility, as the act of automating itself may incur switching costs for the participant (Squire et al., 2006; Reinerman-Jones et al., 2011).
3.2.3. Situational awareness
Situational awareness is a critical component in complex operations (Zhang et al., 2021). Several adaptive systems have been tested to improve the situational awareness of operators in different contexts. As with attention and cognitive flexibility, automation is a lot more explored than other forms of adaptation. The results are quite striking, as SA is often negatively impacted by automation (Chen and Barnes, 2014). In one study, participants either had an agent use a path determined by the operator or an automatically proposed path, performance on the objective task measures was better with automation, but participants had a significantly decreased SA (Marquez and Cummings, 2008). In a meta-analysis of the effects of "Level of Automation" (LOA) on task performance, the authors offered a very vivid analogy, stating that LOA can be described by the lumberjack effect: "the higher they are, the further they fall" (Onnasch et al., 2013). Their analysis of 14 studies revealed, that Level of Automation had an overall negative impact on performance. Diverting from automation, Scott et al. (2006) proposed an interruption assistance interface. The idea being that an operator of multiple UASs could recover SA after a distraction. Two interfaces were compared to a baseline condition with no aid. Either a short playback of what had recently happened was displayed, or a timeline with bookmarks of events was provided. It was shown that following an interruption, both types of interfaces aided in recovering SA with the bookmark system performing marginally better. A similar tool was developed by John et al. (2005) and Smallman and John (2003). The Change History EXplicit tool was developed to automatically detect changes in a monitoring task and then notify as well as help an operator understand these changes. The implementation resulted in an increases in response speed of 80 and 150% increase in accuracy. An entirely different route was proposed by Roldán et al. (2017). In order to keep SA, they tested an adaptive interface and virtual reality (VR), for the control of multiple UASs. Adapting to the situation, the UAS that had the highest risk indicator would be highlighted and the most relevant UAS would be chosen for manual control. These conditions were either presented within a conventional interface or in an immersive VR simulation. The results proved that VR was an effective way of increasing situational awareness as well as reducing workload. The adaptive part of the system on the other hand decreased situational awareness if it was not coupled with virtual reality. A final adaptive approach to SA was proposed by Peysakhovich et al. (2018). The proposal envisions a system equipped with eye-tracking that detects fixations on objects in the cockpit and temporarily removes said objects if looked at for too long. By doing this, the pilots' attention is re-engaged allowing for recovery of SA.
Adaptations differ widely in their design, goals, and success, nevertheless, the findings may perhaps best be summarized by Onnasch et al. (2014) stating : “Automation helps when all goes well, but leaving the user out of the loop can be problematic because it leads to considerable performance impairment if the automation suddenly fails.” As shown, this claim generalizes to other forms of adaptive interfaces and also beyond SA.
4. Markers/triggers for adaptation
As outlined by Feigh et al. (2012) successful adaptive interfaces do not only depend on the type of adaptation, but also on the trigger or the metric that determines when an adaptation should be activated/deactivated.
We here propose that metrics that allow for operator mental state estimation can be categorized into being either covariate (1), behavioral (2), or physiological (3). (1) Covariate metrics, are completely independent of the operator. This can range from TOT to the type of mission, time of day, weather, or state of the system. The important aspect is that an adaptive system purely based on covariate metrics would, in the same situation, behave identically independent of any operator. (2) Behavioral metrics then encompass all metrics based on the classical system input devices such as keyboard, computer mouse, and joystick. This includes a behavioral performance that is measurable by the system, such as reaction times or accuracy, but could also extend to conscious input, such as in activating an autopilot (adaptable interfaces). (3) The major difference between behavioral and physiological measures is the need for a specific sensor or specific configuration of a sensor to capture any physiological measure hence creating a new system input device. Typical examples may include electrocardiography, electroencephalography (EEG), and eye-tracking. From these sensors' signals, metrics can be extracted, in both the temporal and the spectral domains (refer to Roy et al., 2020 for a review). Next, a decision has to be made based on the chosen metric(s). If automated, this decision can either be made using a rule-based/heuristic approach, reinforcement learning, or a machine learning approach (Sikander and Anwar, 2019; Todi et al., 2021). Machine learning in this specific context may be divided into traditional machine learning (e.g., linear discriminant analysis and support vector machines Izenman, 2008; Pisner and Schnyer, 2020) or Bayesian optimization. Table 3 lists examples of metrics modulated by prolonged continuous use that may be used in adaptive systems.
Table 3. Examples of metrics modulated by prolonged continuous use that can be used for adaptive system design.
4.1. Covariate and operator independent metrics
Covariate metrics for adaptive interfaces are commonly used. For example, time of day may trigger a smartphone to toggle to night mode (Lunn, 2022), a car may adapt its driving characteristics based on the type of road it is on (Kesting et al., 2007), or, as mentioned above, Airbus aircraft can suppress alarms in certain flight phases (Airbus, 2008). It is not surprising, that adaptive interfaces and adaptive automation for prolonged continuous use rarely rely on covariate metrics alone, as they fully neglect the role of the operator. However, some studies have made use of covariate metrics in combination with other forms of metrics. For example, TOT played a critical role in a time series clustering model that was also informed by the behavior of participants in a simulated UAS task (Rodríguez-Fernández et al., 2017). Similarly, in the automotive domain mathematical models have used time of day to produce models of driver fatigue (Sikander and Anwar, 2019). Covariate metrics being independent of their operator may be seen as a disadvantage, however, it is our opinion that operator independent systems also bring advantages. They do not require calibration on the operator and the system makes far fewer assumptions on the mental state of the operator, therefore reducing the need for trust. Finally, due to their simplicity and non obtrusiveness, they may be relatively practical.
4.2. Behavioral metrics
Behavioral metrics offer some unique advantages as they remain non obtrusive and require no additional equipment, while also allowing for increases in accuracy by taking into account individual behavior. On the other hand, compared to covariates, they generally do require a calibration phase, i.e., training a classification algorithm. As for behavioral metrics interesting for prolonged use, to monitor mental fatigue in drivers, steering (McDonald et al., 2012; Li et al., 2017) and the combination of multiple measures (Wakita et al., 2006) have proven effective. Pimenta et al. (2013) tested a multitude of metrics extracted from mouse and keyboard behavior during a task to detect mental fatigue. Six out of 15 Metrics were capable of classifying fatigue. These included keydown time, time between keys, mouse acceleration, mouse velocity, time between clicks, and errors per key. As with the covariate metrics and the study combining several driving metrics, Pimenta et al. (2013) also confirmed the superior performance of using a combination of several metrics over a single one.
4.3. Physiological metrics
Research in physiological computing -and passive brain-computer interfaces when one cerebral metric is used for estimation. Roy et al. (2020) have identified several task independent metrics that can be used with high precision. The downsides are that training data need to be acquired, as well as the intrusiveness of many systems, making them less practical. Regarding cerebral metrics, increases in EEG power of the alpha (8-12 Hz) and the theta (4–7 Hz) bands have been repeatedly associated with increased mental fatigue (Borghini et al., 2014; Roy et al., 2014; Jahanpour et al., 2020; Tran et al., 2020). EEG based connectivity measures have also been successfully used to predict performance in a simulated driving task (Sun et al., 2014). Park et al. (2021) proved that mobile setups can also function during more ecologically oriented tasks such as UAS operations. In an actual flight condition, Dehais et al. (2018) classified cognitive fatigue based on functional near infra-red spectroscopy (fNIRS; blood oxygenation measurement) and EEG connectivity measures. Ahn et al. (2016) showed that mental fatigue resulted in increases in frontal oxygenation.
Electrooculography (EOG) and the eye-tracking study also had success in detecting fatigue. Considerable discourse exists in the literature as to what metrics are most useful for detecting the effects of prolonged continuous use, as eye movements may differ substantially depending on the task. In a review of aviation and automobile studies, Borghini et al. (2014) found that TOT resulted in decreases in both blink duration and blink rate. Another metric shown to have a strong association with TOT in both naturalistic as well as highly controlled experiments is the saccadic velocity (Di Stasi et al., 2013). Roy et al. (2016a) studied the effects of prolonged continuous use during Unmanned Aerial System UAS operations and found that fixations, as well as focus on areas of interest, significantly decreased as TOT increased.
The last measure that is consistently used for TOT estimation is cardiac activity. Research indicates that TOT results in decreases in heart rate (HR) (Lal and Craig, 2002; Ahn et al., 2016) and increases in heart rate variability (HRV), the variance between heart beats (Zhao et al., 2012; Borghini et al., 2014; Ahn et al., 2016; Melo et al., 2017; Huang et al., 2018). As with the other measures, the combination of several metrics also seems to result in improved estimation (Ahn et al., 2016). Beyond these transitional measures, the researchers have also found that posture can determine TOT and mental fatigue during driving simulations (Furogori et al., 2005; Ansari et al., 2021). Finally, the merging of multiple metrics into useful measures for their specific application has also seen success. This was done by Azim et al. (2014) for drivers and by Li et al. (2020) for construction equipment operators.
5. Discussion
The goal of this article was to explore the potential of adaptive systems, and mental state-based adaptive interfaces in particular, for use during prolonged continuous operations. The literature shows that with increasing TOT, cognitive flexibility, attention, and situational awareness decrease, resulting in a higher risk of error and failure during the operation of complex tasks (Blackmon and Gramopadhye, 1995; Bliss et al., 1995; van der Linden et al., 2003a,b; Lorist et al., 2009; Holtzer et al., 2010; Csathó et al., 2012; Krishnan et al., 2014; Sun et al., 2014; Plukaard et al., 2015; Guo et al., 2016; Petruo et al., 2018; Wickens and McCarley, 2019; Yu et al., 2021). The effectiveness of adaptive interfaces, built to mitigate said effects is determined by a multitude of factors: Operators need to understand the system they are working with (Scott et al., 2006; Kiefer et al., 2017; Xing et al., 2021) and also trust the system (Lee and See, 2004). Furthermore, ecological validity needs to be accounted for in order to avoid problems with secondary effects (Chen and Barnes, 2014), boundary conditions (Pritchett et al., 2014), scalability (Hou et al., 2007), calibration (Taylor et al., 2013), and complexity (Lajos, 1995; Höök, 2000; Hou et al., 2007). The adaptations themselves may take many forms, ranging from highlighting targets (Parasuraman et al., 2009), to attentional alerts (Taylor et al., 2013; Saint-Lot et al., 2020) and information removal (Peysakhovich et al., 2018). Regarding the detection of a degraded mental state due to prolonged use, it was shown that a combination of measures, with the addition of a reliable estimation method, outperforms single measures (Wakita et al., 2006; Pimenta et al., 2013; Ahn et al., 2016; Sikander and Anwar, 2019). Furthermore, studies combining several entities such as behavioral and physiological measures reduce the disadvantages of single metrics and may also provide redundancy of measures (Azim et al., 2014; Li et al., 2020). Further details on these findings are summarized in Tables 1–3.
When looking at the body of literature reviewed for this article, it becomes evident that to this date relatively few complete mental state based adaptive interfaces have been developed and tested (Piechulla et al., 2003; Parasuraman et al., 2009; Aricò et al., 2016; Park et al., 2021). In the future, more complete systems should be developed and tested as only complete systems allow for the evaluation of certain factors such as trust (Lee and See, 2004). If the goal of these interfaces is to be incorporated into the daily complex operations that happen around the world, then researchers and designers should at all times ask themselves whether the system that they are working on could actually be applied in a real operation. But also here practical limitations have to be considered. Machine learning algorithms can still require long calibration phases depending on the metric that is being used (Aricò et al., 2017), and certification into use may be a lengthy process depending on the domain of the operations (Torens et al., 2021). The question of trust in adaptive interfaces has, to the best of our knowledge, not been explored yet, but may prove to be a further component to be considered in future research.
This article is the first step toward better understanding what is known, what currently exists, and what remains to be achieved regarding prolonged use in human-machine interaction and solutions such as adaptive interfaces. Yet, some aspects were voluntarily put aside and remain to be investigated. Among those one can list the interaction between prolonged use and sleep cycles, circadian rhythms, and shift work.
The relation between sleep and mental fatigue is complex and difficult to explore, as definitions and experimental procedures vary widely (Åkerstedt et al., 2004; Matthews, 2012; Jahanpour et al., 2020). The impact of prolonged use depending on solicited sensory modality is also to be studied, as well as the interaction with individual differences of operators, such as expertise and preferences.
It also needs to be acknowledged, that the effects of mental fatigue on cognition that are discussed in this review do not form an exhaustive list of aspects in which mental fatigue may alter cognition. Finally, a topic closely linked to prolonged use is that of learning. Indeed, learning, at least in the -possibly virtual- classroom context presupposes spending an extended amount of time on exercises and complex tasks, with potential repetitions. A few authors have started investigating adaptive solutions for e-learning and digital reading making use of users' mental state estimations with success (Walter et al., 2017; Andreessen et al., 2021). Yet, while it may benefit from adaptive interfaces, to our knowledge the interaction between learning and prolonged use of dedicated interfaces has not been assessed. A further limitation of the body of work presented here can be seen in the different TOT that was used in different studies to elicit mental fatigue Table 1. Different durations ranging from 35 to 180 min need to be taken into account when comparing the outcomes of studies. To the best of our knowledge, there is no general consensus on when mental fatigue occurs as this likely depends on the task, individual, and other factors (Matthews, 2012). However, by testing different rest periods between blocks of demanding tasks, Lim et al. (2016) present an interesting framework for deciding how long a task should be performed. A further limitation of the here presented body of work is that machine learning and related aspects such as transfer learning and practical aspects of brain-computer interfacing were left undiscussed (Brouwer et al., 2015; Zeng et al., 2021). For instance, TOT has been shown to significantly impact the distribution of classical features for mental workload estimation, as well as the accuracy of pipelines that relied on these measurements (Roy et al., 2013, 2016b).
6. Conclusion
Advances in technology as well as the trend toward ever more complex systems will likely increase the demand for more sophisticated and also more complex adaptive interfaces, in order to counteract the effects of prolonged continuous use. The incorporation of user and operator mental state estimations may in the future prove to be one of the pivoting factors for their success (Szostak, 2013). Mental state-based adaptive interface design incorporates research from cognitive science, neuroergonomics, human factors, and human-machine interaction as well as the research and particular characteristics of each domain of application (e.g., aviation and automobile). The necessity for researchers from one domain to gain proficiency in other domains due to interdisciplinarity was a driving factor for this article. To maximize the success of future mental state based adaptive interfaces, researchers and designers should take the points summarized in Figure 2 into consideration. These questions form the conclusions that can be drawn from this article and facing them before designing, implementing and testing a mental state based adaptive interface may help during the process and may increase the relevance and efficiency of a design. In the future, systems, as are reviewed and proposed here, will hopefully become more relevant, allowing for increased safety during complex operations.
Figure 2. Proposed, non exhaustive set of questions, for consideration during adaptive interface conception design and testing.
Author contributions
MH, RR, and AB: drafting of the article. RR and AB: supervision. All authors contributed to the article and approved the submitted version.
Funding
This study was funded by the French Defence procurement and technology agency (DGA) as part of the publicly funded Concorde Project.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Åkerstedt, T., Knutsson, A., Westerholm, P., Theorell, T., Alfredsson, L., and Kecklund, G. (2004). Mental fatigue, work and sleep. J. Psychosom Res. 57, 427–433. doi: 10.1016/j.jpsychores.2003.12.001
Abd-Elfattah, H. M., Abdelazeim, F. H., and Elshennawy, S. (2015). Physical and cognitive consequences of fatigue: a review. J. Adv. Res. 6, 351–358. doi: 10.1016/j.jare.2015.01.011
Ackerman, P. L., and Kanfer, R. (2009). Test length and cognitive fatigue: an empirical examination of effects on performance and test-taker reactions. J. Exp. Psychol. Appl. 15, 163–181. doi: 10.1037/a0015719
Ahn, S., Nguyen, T., Jang, H., Kim, J. G., and Jun, S. C. (2016). Exploring neuro-physiological correlates of drivers' mental fatigue caused by sleep deprivation using simultaneous EEG, ECG, and fNIRS data. Front. Hum. Neurosci. 10, 219. doi: 10.3389/fnhum.2016.00219
Andreessen, L. M., Gerjets, P., Meurers, D., and Zander, T. O. (2021). Toward neuroadaptive support technologies for improving digital reading: a passive BCI-based assessment of mental workload imposed by text difficulty and presentation speed during reading. User Model Useradapt Interact. 31, 75–104. doi: 10.1007/s11257-020-09273-5
Ansari, S., Naghdy, F., Du, H., and Pahnwar, Y. N. (2021). Driver mental fatigue detection based on head posture using new modified reLU-BiLSTM deep neural network. IEEE Trans. Intell. Transport. Syst. 23, 1–13. doi: 10.1109/TITS.2021.3098309
Aric,ò, P., Borghini, G., Di Flumeri, G., Colosimo, A., Bonelli, S., Golfetti, A., et al. (2016). Adaptive automation triggered by EEG-based mental workload index: a passive brain-computer interface application in realistic air traffic control environment. Front. Hum. Neurosci. 10, 539. doi: 10.3389/fnhum.2016.00539
Aricò, P., Borghini, G., Di Flumeri, G., Sciaraffa, N., Colosimo, A., and Babiloni, F. (2017). Passive BCI in operational environments: insights, recent advances, and future trends. IEEE Trans. Biomed. Eng. 64, 1431–1436. doi: 10.1109/TBME.2017.2694856
Arrington, C. M., and Logan, G. D. (2004). The cost of a voluntary task switch. Psychol. Sci. 15, 610–615. doi: 10.1111/j.0956-7976.2004.00728.x
Arrington, C. M., and Yates, M. M. (2009). The role of attentional networks in voluntary task switching. Psychon. Bull. Rev. 16, 660–665. doi: 10.3758/PBR.16.4.660
Azim, T., Jaffar, M. A., and Mirza, A. M. (2014). Fully automated real time fatigue detection of drivers through Fuzzy Expert Systems. Appl. Soft. Comput. 18, 25–38. doi: 10.1016/j.asoc.2014.01.020
Blackmon, R. B., and Gramopadhye, A. K. (1995). Improving construction safety by providing positive feedback on backup alarms. J. Construct. Eng. Manag. 121, 166–171. doi: 10.1061/(ASCE)0733-9364(1995)121:2(166)
Bliss, J. P., Gilson, R. D., and Deaton, J. E. (1995). Human probability matching behaviour in response to alarms of varying reliability. Ergonomics 38, 2300–2312. doi: 10.1080/00140139508925269
Boksem, M. A., and Tops, M. (2008). Mental fatigue: costs and benefits. Brain Res. Rev. 59, 125–139. doi: 10.1016/j.brainresrev.2008.07.001
Borghini, G., Astolfi, L., Vecchiato, G., Mattia, D., and Babiloni, F. (2014). Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neurosci. Biobehav. Rev. 44, 58–75. doi: 10.1016/j.neubiorev.2012.10.003
Breznitz, S. (2013). Cry Wolf: The Psychology of False Alarms. Hillsdale, NJ: Psychology Press. Available online at: https://books.google.fr/books?hl=en&lr=&id=lxwLVy16wqoC&oi=fnd&pg=PR2&ots=BJAE3PbC5S&sig=C5mK4ziXz2lh92vl54N1tHuFsE0&redir_esc=y#v=onepage&q&f=false
Bronkhorst, A. (2000). The cocktail party phenomenon: a review of research on speech intelligibility in multiple-talker conditions. Acta Acustica United Acustica 86, 117–128. Available online at: https://www.ingentaconnect.com/content/dav/aaua/2000/00000086/00000001/art00016
Brouwer, A.-M., Zander, T. O., van Erp, J. B. F., Korteling, J. E., and Bronkhorst, A. W. (2015). Using neurophysiological signals that reflect cognitive or affective state: six recommendations to avoid common pitfalls. Front. Neurosci, 9, 136. doi: 10.3389/fnins.2015.00136
Cash, J. J. (2009). Alert fatigue. Am. J. Health Syst. Pharmacy 66, 2098–2101. doi: 10.2146/ajhp090181
Chen, J. Y. C., and Barnes, M. J. (2014). Human-agent teaming for multirobot control: a review of human factors issues. IEEE Trans. Hum. Mach. Syst. 44, 13–29. doi: 10.1109/THMS.2013.2293535
Christie, S. T., and Schrater, P. (2015). Cognitive cost as dynamic allocation of energetic resources. Front. Neurosci. 9, 286. doi: 10.3389/fnins.2015.00289
Csath, ó, r., van der Linden, D., Hernádi, I., Buzás, P., and Kalmár, G. (2012). Effects of mental fatigue on the capacity limits of visual attention. J. Cogn. Psychol. 24, 511–524. doi: 10.1080/20445911.2012.658039
Cummings, G. E. (1980). Operator/instrumentation interactions during the three mile island incident. IEEE Trans. Nucl. Sci. 27, 931–934. doi: 10.1109/TNS.1980.4330953
De Jong, R. (1995). The role of preparation in overlapping-task performance. Q. J. Exp. Psychol. A 48, 2–25. doi: 10.1080/14640749508401372
Dehais, F., Dupres, A., Di Flumeri, G., Verdiere, K., Borghini, G., Babiloni, F., et al. (2018). “Monitoring pilot's cognitive fatigue with engagement features in simulated and actual flight conditions using an hybrid fNIRS-EEG passive BCI,” in 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC) (Miyazaki: IEEE), 544–549.
Di Stasi, L. L., Catena, A., Cañas, J. J., Macknik, S. L., and Martinez-Conde, S. (2013). Saccadic velocity as an arousal index in naturalistic tasks. Neurosci. Biobehav. Rev. 37, 968–975. doi: 10.1016/j.neubiorev.2013.03.011
Dorrian, J., Roach, G. D., Fletcher, A., and Dawson, D. (2007). Simulated train driving: fatigue, self-awareness and cognitive disengagement. Appl. Ergon. 38, 155–166. doi: 10.1016/j.apergo.2006.03.006
Dubey, P. (1986). Inefficiency of nash equilibria. Math. Operat. Res. 11, 1–8. doi: 10.1287/moor.11.1.1
Endsley, M. R. (1995). Toward a theory of situation awareness in dynamic systems. Hum. Factors 37, 32–64. doi: 10.1518/001872095779049543
Eriksen, B. A., and Eriksen, C. W. (1974). Effects of noise letters upon the identification of a target letter in a nonsearch task. Percept. Psychophys. 16, 143–149. doi: 10.3758/BF03203267
Feigh, K. M., Dorneich, M. C., and Hayes, C. C. (2012). Toward a characterization of adaptive systems: a framework for researchers and system designers. Hum. Factors 54, 1008–1024. doi: 10.1177/0018720812443983
Findlater, L., and McGrenere, J. (2004). “A comparison of static, adaptive, and adaptable menus,” in Proceedings of the 2004 Conference on Human Factors in Computing Systems-CHI '04 (Vienna: ACM Press), 89–96. doi: 10.1145/985692.985704
Furogori, S., Yoshizawa, N., Iname, C., and Mirua, Y. (2005). Estimation of driver fatigue by pressure distribution on seat in long term driving. RAE 26, 053–058. Available online at: https://www.safetylit.org/citations/index.php?fuseaction=citations.viewdetails&citationIds%5B%5D=citjournalarticle_185440_5
Guo, Z., Chen, R., Zhang, K., Pan, Y., and Wu, J. (2016). The impairing effect of mental fatigue on visual sustained attention under monotonous multi-object visual attention task in long durations: an event-related potential based study. PLoS ONE 11, e0163360. doi: 10.1371/journal.pone.0163360
Hansberger, J. T. (2015). Development of the Next Generation of Adaptive Interfaces. Technical Report, Army Research Lab Aberdeen Proving Ground Md Human Research And Engineering Directorate.
Hollnagel, E., (Ed.). (2009). Safer Complex Industrial Environments: A Human Factors Approach. Boca Raton, FL: CRC Press.
Holtzer, R., Shuman, M., Mahoney, J. R., Lipton, R., and Verghese, J. (2010). Cognitive fatigue defined in the context of attention networks. Aging Neuropsychol. Cogn. 18, 108–128. doi: 10.1080/13825585.2010.517826
Höök, K. (2000). Steps to take before intelligent user interfaces become real. Interact. Comput. 12, 409–426. doi: 10.1016/S0953-5438(99)00006-5
Hopstaken, J. F., van der Linden, D., Bakker, A. B., and Kompier, M. A. J. (2015). The window of my eyes: task disengagement and mental fatigue covary with pupil dynamics. Biol. Psychol. 110, 100–106. doi: 10.1016/j.biopsycho.2015.06.013
Hou, M., Kobierski, R. D., and Brown, M. (2007). Intelligent adaptive interfaces for the control of multiple UAVs. J. Cogn. Eng. Decis. Making 1, 327–362. doi: 10.1518/155534307X255654
Hou, M., Zhu, H., Zhou, M., and Arrabito, G. R. (2011). Optimizing operator-agent interaction in intelligent adaptive interface design: a conceptual framework. IEEE Trans. Syst. Man Cybern. C 41, 161–178. doi: 10.1109/TSMCC.2010.2052041
Huang, S., Li, J., Zhang, P., and Zhang, W. (2018). Detection of mental fatigue state with wearable ECG devices. Int. J. Med. Inform. 119, 39–46. doi: 10.1016/j.ijmedinf.2018.08.010
Ionescu, T. (2012). Exploring the nature of cognitive flexibility. New Ideas Psychol. 30, 190–200. doi: 10.1016/j.newideapsych.2011.11.001
Izenman, A. J. (2008). “Linear discriminant analysis,” in Modern Multivariate Statistical Techniques: Regression, Classification, and Manifold Learning, Springer Texts in Statistics, ed A. J. Izenman (New York, NY: Springer), 237–280.
Jacquet, T., Poulin-Charronnat, B., Bard, P., Perra, J., and Lepers, R. (2021). Physical activity and music to counteract mental fatigue. Neuroscience 478, 75–88. doi: 10.1016/j.neuroscience.2021.09.019
Jahanpour, E. S., Berberian, B., Imbert, J.-P., and Roy, R. N. (2020). Cognitive fatigue assessment in operational settings: a review and UAS implications. This work was supported by the defense innovation agency (AID) of the French Ministry of Defense (research project CONCORDE N°2019 65 0090004707501). IFAC-PapersOnLine 53, 330–337. doi: 10.1016/j.ifacol.2021.04.188
John, S. M., Smallman, H. S., and Manes, D. I. (2005). Recovery from interruptions to a dynamic monitoring task: the beguiling utility of instant replay. Proc. Hum. Factors Ergon. Soc. Ann. Meet. 49, 473–477. doi: 10.1177/154193120504900355
Kable, A. K., Pich, J., and Maslin-Prothero, S. E. (2012). A structured approach to documenting a search strategy for publication: a 12 step guideline for authors. Nurse Educ. Today 32, 878–886. doi: 10.1016/j.nedt.2012.02.022
Katsuki, F., and Constantinidis, C. (2014). Bottom-up and top-down attention: different processes and overlapping neural systems. Neuroscientist 20, 509–521. doi: 10.1177/1073858413514136
Ke, Q., Liu, J., Bennamoun, M., An, S., Sohel, F., and Boussaid, F. (2018). “Chapter 5-computer vision for human-machine interaction,” in Computer Vision for Assistive Healthcare, Computer Vision and Pattern Recognition, eds M. Leo and G. M. Farinella (Crawley, WA: Academic Press), 127–145. Available online at: https://research-repository.uwa.edu.au/en/publications/computer-vision-for-human-machine-interaction
Kesting, A., Treiber, M., Schönhof, M., and Helbing, D. (2007). Extending adaptive cruise control to adaptive driving strategies. Transport. Res. Rec. 2000, 16–24. doi: 10.3141/2000-03
Kiefer, P., Giannopoulos, I., Athanasios Anagnostopoulos, V., Schöning, J., and Raubal, M. (2017). Controllability matters: the user experience of adaptive maps. Geoinformatica 21, 619–641. doi: 10.1007/s10707-016-0282-x
Koch, I., Poljac, E., Müller, H., and Kiesel, A. (2018). Cognitive structure, flexibility, and plasticity in human multitasking–An integrative review of dual-task and task-switching research. Psychol. Bull. 144, 557–583. doi: 10.1037/bul0000144
Krishnan, V. K., Dasari, D., and Ding, L. (2014). EEG Correlates of Fluctuation in Cognitive Performance in an Air Traffic Control Task (No. DOT/FAA/AM-14/12). Washington, DC: Office of Aerospace Medicine. Available online at: https://rosap.ntl.bts.gov/view/dot/57166
Kudesia, R. S., Pandey, A., and Reina, C. S. (2022). Doing more with less: interactive effects of cognitive resources and mindfulness training in coping with mental fatigue from multitasking. J. Manag. 48, 410–439. doi: 10.1177/0149206320964570
Ladouce, S., Torre Tresols, J. J., Darmet, L., Ferraro, G., and Dehais, F. (2021). “Improving user experience of SSVEP-BCI through reduction of stimuli amplitude depth,” in 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC) (Melbourne, VIC: IEEE), 2936–2941.
Lajos, B. (1995). Adaptive human-computer interfaces for man-machine interaction in computer-integrated systems. Comput. Integrat. Manufact. Syst. 2, 133–142. doi: 10.1016/0951-5240(95)00007-G
Lal, S. K. L., and Craig, A. (2001). A critical review of the psychophysiology of driver fatigue. Biol. Psychol. 55, 173–194. doi: 10.1016/S0301-0511(00)00085-5
Lal, S. K. L., and Craig, A. (2002). Driver fatigue: electroencephalography and psychological assessment. Psychophysiology 39, 313–321. doi: 10.1017/S0048577201393095
Langley, P. (1999). “User modeling in adaptive interface,” in UM99 User Modeling, CISM International Centre for Mechanical Sciences, ed J. Kay (Vienna: Springer), 357–370.
Lee, J. D., and See, K. A. (2004). Trust in automation: designing for appropriate reliance. Hum. Factors 46, 50–80. doi: 10.1518/hfes.46.1.50.30392
Li, J., Li, H., Umer, W., Wang, H., Xing, X., Zhao, S., et al. (2020). Identification and classification of construction equipment operators' mental fatigue using wearable eye-tracking technology. Automat. Construct. 109, 103000. doi: 10.1016/j.autcon.2019.103000
Li, Z., Li, S. E., Li, R., Cheng, B., and Shi, J. (2017). Online detection of driver fatigue using steering wheel angles for real driving conditions. Sensors 17, 495. doi: 10.3390/s17030495
Lim, J., Teng, J., Wong, K. F., and Chee, M. W. L. (2016). Modulating rest-break length induces differential recruitment of automatic and controlled attentional processes upon task reengagement. Neuroimage 134, 64–73. doi: 10.1016/j.neuroimage.2016.03.077
Lim, Y., Gardi, A., Sabatini, R., Ramasamy, S., Kistan, T., Ezer, N., et al. (2018). Avionics human-machine interfaces and interactions for manned and unmanned aircraft. Progr. Aerospace Sci. 102, 1–46. doi: 10.1016/j.paerosci.2018.05.002
Linden, F. R. V., and Seely, V. J. (2011). The Boeing 247: The First Modern Airliner. Washington, DC: University of Washington Press.
Liu, J., Li, R., Ling, S. L., Zhou, J. R., Lim, K. W., Srinivasan, R., et al. (2001). Intelligent alarm management through suppressing nuisance alarms and providing operator advice. IFAC Proc. 34, 81–86. doi: 10.1016/S1474-6670(17)33572-3
Lorist, M. M., Bezdan, E., ten Caat, M., Span, M. M., Roerdink, J. B. T. M., and Maurits, N. M. (2009). The influence of mental fatigue and motivation on neural network dynamics; an EEG coherence study. Brain Res. 1270, 95–106. doi: 10.1016/j.brainres.2009.03.015
Lorist, M. M., Boksem, M. A., and Ridderinkhof, K. R. (2005). Impaired cognitive control and reduced cingulate activity during mental fatigue. Cogn. Brain Res. 24, 199–205. doi: 10.1016/j.cogbrainres.2005.01.018
Lorist, M. M., Klein, M., Nieuwenhuis, S., Jong, R. D., Mulder, G., and Meijman, T. F. (2000). Mental fatigue and task control: Planning and preparation. Psychophysiology 37, 614–625. doi: 10.1111/1469-8986.3750614
Lunn, E. (2022). What Is Dark Mode - And Should You Be Using It? Section: Mobile Phones. Available online at: https://www.forbes.com/uk/advisor/mobile-phones/what-is-dark-mode-and-should-you-be-using-it/
Marquez, J. J., and Cummings, M. L. (2008). Design and evaluation of path planning decision support for planetary surface exploration. J. Aerospace Comput. Inf. Commun. 5, 57–71. doi: 10.2514/1.26248
McCreary, J., Pollard, M, Stevenson, K., and Wilson, M. B. (1998). Human factors: Tenerife revisited. J. Air Transport. World Wide. 3, 23–33. Available online at: https://rosap.ntl.bts.gov/view/dot/13937
McDonald, A. D., Schwarz, C., Lee, J. D., and Brown, T. L. (2012). Real-time detection of drowsiness related lane departures using steering wheel angle. Proc. Hum. Factors Ergon. Soc. Ann. Meet. 56, 2201–2205. doi: 10.1177/1071181312561464
Melo, H. M., Nascimento, L. M., and Takase, E. (2017). Mental fatigue and heart rate variability (HRV): The time-on-task effect. Psychol. Neurosci. 10, 428–436. doi: 10.1037/pne0000110
Mizuno, K., Tanaka, M., Tajima, K., Okada, N., Rokushima, K., and Watanabe, Y. (2009). Effects of mild-stream bathing on recovery from mental fatigue. Med. Sci. Monitor 16, CR8–CR14. Available online at: https://www.researchgate.net/publication/40786240_Effects_of_mild-stream_bathing_on_recovery_from_mental_fatigue
Monchi, O., Petrides, M., Petre, V., Worsley, K., and Dagher, A. (2001). Wisconsin card sorting revisited: distinct neural circuits participating in different stages of the task identified by event-related functional magnetic resonance imaging. J. Neurosci. 21, 7733–7741. doi: 10.1523/JNEUROSCI.21-19-07733.2001
Monsell, S., Sumner, P., and Waters, H. (2003). Task-set reconfiguration with predictable and unpredictable task switches. Mem. Cogn. 31, 327–342. doi: 10.3758/BF03194391
O'Brien, J., and Bull Schaefer, R. A. (2020). Deadly distraction - Eastern Air 401: the accident that changed aviation forever. CASE J. 16, 345–368. doi: 10.1108/TCJ-10-2019-0095
Onnasch, L., Ruff, S., and Manzey, D. (2014). Operators? adaptation to imperfect automation - impact of miss-prone alarm systems on attention allocation and performance. Int. J. Hum. Comput. Stud. 72, 772–782. doi: 10.1016/j.ijhcs.2014.05.001
Onnasch, L., Wickens, C., Li, H., and Manzey, D. (2013). Human Performance consequences of stages and levels of automation an integrated meta-analysis. Hum. Factors 56, 476–488. doi: 10.1177/0018720813501549
Parasuraman, R., Cosenzo, K., and de Visser, E. (2009). Adaptive automation for human supervision of multiple uninhabited vehicles: effects on change detection, situation awareness, and mental workload. Mil. Psychol. 21, 270–297. doi: 10.1080/08995600902768800
Parasuraman, R., Galster, S., Squire, P., Furukawa, H., and Miller, C. (2005). A flexible delegation-type interface enhances system performance in human supervision of multiple robots: empirical studies with roboflag. IEEE Trans. Syst. Man Cybern. A Syst. Hum. 35, 481–493. doi: 10.1109/TSMCA.2005.850598
Parasuraman, R., Sheridan, T., and Wickens, C. (2000). A model for types and levels of human interaction with automation. IEEE Trans. Syst. Man Cybern. A Syst. Hum 30, 286–297. doi: 10.1109/3468.844354
Park, J., Park, J., Shin, D., and Choi, Y. (2021). A BCI based alerting system for attention recovery of UAV operators. Sensors 21, 2447. doi: 10.3390/s21072447
Petruo, V. A., Mückschel, M., and Beste, C. (2018). On the role of the prefrontal cortex in fatigue effects on cognitive flexibility - a system neurophysiological approach. Sci. Rep. 8, 6395. doi: 10.1038/s41598-018-24834-w
Peysakhovich, V., Lefrançois, O., Dehais, F., and Causse, M. (2018). The neuroergonomics of aircraft cockpits: the four stages of eye-tracking integration to enhance flight safety. Safety 4, 8. doi: 10.3390/safety4010008
Piechulla, W., Mayser, C., Gehrke, H., and König, W. (2003). Reducing drivers? mental workload by means of an adaptive man-machine interface. Transport. Res. F Traffic Psychol. Behav. 6, 233–248. doi: 10.1016/j.trf.2003.08.001
Pimenta, A., Carneiro, D. R., Novais, P., and Neves, J. (2013). Monitoring Mental Fatigue Through the Analysis of Keyboard and Mouse Interaction Patterns. Salamanca: Springer. doi: 10.1007/978-3-642-40846-5_23
Pisner, D. A., and Schnyer, D. M. (2020). “Chapter 6 - Support vector machine,” in Machine Learning, eds A,. Mechelli and S. Vieira (Austin, TX: Academic Press), 101–121. doi: 10.1016/B978-0-12-815739-8.00006-7
Plukaard, S., Huizinga, M., Krabbendam, L., and Jolles, J. (2015). Cognitive flexibility in healthy students is affected by fatigue: an experimental study. Learn. Individ. Differ. 38, 18–25. doi: 10.1016/j.lindif.2015.01.003
Pritchett, A. R., Kim, S. Y., and Feigh, K. M. (2014). Measuring human-automation function allocation. J. Cogn. Eng. Decis. Making 8, 52–77. doi: 10.1177/1555343413490166
Reinerman-Jones, L., Taylor, G., Sprouse, K., Barber, D., and Hudson, I. (2011). Adaptive automation as a task switching and task congruence challenge. Proc. Hum. Factors Ergon. Soc. Ann. Meet. 55, 197–201. doi: 10.1177/1071181311551041
Roda, C. (2011). “Human attention and its implications for human-computer interaction,” in Human Attention in Digital Environments (New York, NY: Cambridge University Press), 11–62.
Rodríguez-Fernández, V., Menéndez, H. D., and Camacho, D. (2017). Analysing temporal performance profiles of UAV operators using time series clustering. Expert. Syst. Appl. 70, 103–118. doi: 10.1016/j.eswa.2016.10.044
Roldán, J. J., Peña-Tapia, E., Martín-Barrio, A., Olivares-Méndez, M. A., Del Cerro, J., and Barrientos, A. (2017). Multi-robot interfaces and operator situational awareness: study of the impact of immersion and prediction. Sensors 17, 1720. doi: 10.3390/s17081720
Rönnlund, M., Lövdén, M., and Nilsson, L.-G. (2001). Adult age differences in tower of hanoi performance: influence from demographic and cognitive variables. Aging Neuropsychol. Cogn. 8, 269–283. doi: 10.1076/anec.8.4.269.5641
Roy, R. N., Bonnet, S., Charbonnier, S., and Campagne, A. (2013). “Mental fatigue and working memory load estimation: Interaction and implications for EEG-based passive BCI,” in 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (Osaka: IEEE), 6607–6610.
Roy, R. N., Bovo, A., Gateau, T., Dehais, F., and Carvalho Chanel, C. P. (2016a). Operator engagement during prolonged simulated UAV operation. IFAC-PapersOnLine 49, 171–176. doi: 10.1016/j.ifacol.2016.12.209
Roy, R. N., Charbonnier, S., and Bonnet, S. (2014). Detection of mental fatigue using an active BCI inspired signal processing chain. IFAC Proc. 47, 2963–2968. doi: 10.3182/20140824-6-ZA-1003.00897
Roy, R. N., Charbonnier, S., Campagne, A., and Bonnet, S. (2016b). Efficient mental workload estimation using task-independent EEG features. J. Neural Eng. 13, 026019. doi: 10.1088/1741-2560/13/2/026019
Roy, R. N., Drougard, N., Gateau, T., Dehais, F., and Chanel, C. P. C. (2020). How can physiological computing benefit human-robot interaction? Robotics 9, 100. doi: 10.3390/robotics9040100
Rubinstein, J. S., Meyer, D. E., and Evans, J. E. (2001). Executive control of cognitive processes in task switching. J. Exp. Psychol. Hum. Percept. Perform. 27, 763–797. doi: 10.1037/0096-1523.27.4.763
Ruskin, K. J., Corvin, C., Rice, S., Richards, G., Winter, S. R., and Clebone Ruskin, A. (2021). Alarms, alerts, and warnings in air traffic control: an analysis of reports from the aviation safety reporting system. Transport. Res. Interdisc. Perspect. 12, 100502. doi: 10.1016/j.trip.2021.100502
Saint-Lot, J., Imbert, J.-P., and Dehais, F. (2020). “Red alert: a cognitive countermeasure to mitigate attentional tunneling,” in Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI:ACM), 1–6.
Salvucci, D. D., Taatgen, N. A., and Borst, J. P. (2009). “Toward a unified theory of the multitasking continuum: from concurrent performance to task switching, interruption, and resumption,” in CHI '09: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Boston, MA: Association for Computing Machinery). doi: 10.1145/1518701.1518981
Scott, S. D., Mercier, S., Cummings, M. L., and Wang, E. (2006). Assisting interruption recovery in supervisory control of multiple uavs. Proc. Hum. Factors Ergon. Soc. Ann. Meet. 50, 699–703. doi: 10.1177/154193120605000518
Sikander, G., and Anwar, S. (2019). Driver fatigue detection systems: a review. IEEE Trans. Intell. Transport. Syst. 20, 2339–2352. doi: 10.1109/TITS.2018.2868499
Smallman, H. S., and John, M. S. (2003). Chex (change history explicit): new HCI concepts for change awareness. Proc. Hum. Factors Ergon. Soc. Ann. Meet. 47, 528–532. doi: 10.1177/154193120304700358
Squire, P., Trafton, G., and Parasuraman, R. (2006). “Human control of multiple unmanned vehicles: effects of interface type on execution and task switching times,” in Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction (New York, NY: Association of Applied Linguistics), 26–32. doi: 10.1145/1121241.1121248
Squire, P. N., and Parasuraman, R. (2010). Effects of automation and task load on task switching during human supervision of multiple semi-autonomous robots in a dynamic environment. Ergonomics 53, 951–961. doi: 10.1080/00140139.2010.489969
Stanton, N., Chambers, P., and Piggott, J. (2001). Situational awareness and safety. Saf. Sci. 39, 189–204. doi: 10.1016/S0925-7535(01)00010-8
Sun, Y., Lim, J., Kwok, K., and Bezerianos, A. (2014). Functional cortical connectivity analysis of mental fatigue unmasks hemispheric asymmetry and changes in small-world networks. Brain Cogn. 85, 220–230. doi: 10.1016/j.bandc.2013.12.011
Szostak, R. (2013). The State of the Field: Interdisciplinary Research 31, 22–43. doi: 10.4135/9781483349541.n1
Taylor, G. S., Reinerman-Jones, L. E., Szalma, J. L., Mouloua, M., and Hancock, P. A. (2013). What to automate: addressing the multidimensionality of cognitive resources through system design. J. Cogn. Eng. Decis. Making 7, 311–329. doi: 10.1177/1555343413495396
Theeuwes, J. (1991). Exogenous and endogenous control of attention: the effect of visual onsets and offsets. Percept. Psychophys. 49, 83–90. doi: 10.3758/BF03211619
Todi, K., Bailly, G., Leiva, L., and Oulasvirta, A. (2021). “Adapting user interfaces with model-based reinforcement learning,” in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama: ACM), 1–13.
Torens, C., Durak, U., and Dauer, J. C. (2021). “Guidelines and regulatory framework for machine learning in aviation,” in AIAA SCITECH 2022 Forum (American Institute of Aeronautics and Astronautics). Available online at: https://arc.aiaa.org/doi/pdf/10.2514/6.2022-1132. doi: 10.2514/6.2022-1132
Tran, Y., Craig, A., Craig, R., Chai, R., and Nguyen, H. (2020). The influence of mental fatigue on brain activity: evidence from a systematic review with meta-analyses. Psychophysiology 57, e13554. doi: 10.1111/psyp.13554
Trejo, L. J., Kubitz, K., Rosipal, R., Kochavi, R. L., and Montgomery, L. D. (2015). EEG-based estimation and classification of mental fatigue. Psychology 06, 572. doi: 10.4236/psych.2015.65055
Van Cutsem, J., De Pauw, K., Marcora, S., Meeusen, R., and Roelands, B. (2018). A caffeine-maltodextrin mouth rinse counters mental fatigue. Psychopharmacology 235, 947–958. doi: 10.1007/s00213-017-4809-0
van de Merwe, F., Kähler, N., and Securius, P. (2016). Crew-centred design of ships - The CyClaDes project. Transport. Res. Procedia 14, 1611–1620. doi: 10.1016/j.trpro.2016.05.126
van der Linden, D., Frese, M., and Meijman, T. F. (2003a). Mental fatigue and the control of cognitive processes: effects on perseveration and planning. Acta Psychol. 113, 45–65. doi: 10.1016/S0001-6918(02)00150-6
van der Linden, D., Frese, M., and Sonnentag, S. (2003b). The impact of mental fatigue on exploration in a complex computer task: rigidity and loss of systematic strategies. Hum. Factors 45, 483–494. doi: 10.1518/hfes.45.3.483.27256
Vandierendonck, A., Liefooghe, B., and Verbruggen, F. (2010). Task switching: Interplay of reconfiguration and interference control. Psychol. Bull. 136, 601–626. doi: 10.1037/a0019791
Wakita, T., Ozawa, K., Miyajima, C., Igarashi, K., Itou, K., Takeda, K., et al. (2006). Driver Identification Using Driving Behavior Signals. IEICE Trans. Inf. Syst. E89-D, 1188–1194. doi: 10.1093/ietisy/e89-d.3.1188
Walter, C., Rosenstiel, W., Bogdan, M., Gerjets, P., and Spüler, M. (2017). Online EEG-based workload adaptation of an arithmetic learning environment. Front. Hum. Neurosci. 11, 286. doi: 10.3389/fnhum.2017.00286
Xing, Y., Lv, C., Cao, D., and Hang, P. (2021). Toward human-vehicle collaboration: review and perspectives on human-centered collaborative automated driving. Transport. Res. C Emerg. Technol. 128, 103199. doi: 10.1016/j.trc.2021.103199
Yamano, E., Tanaka, M., Ishii, A., Tsuruoka, N., Abe, K., and Watanabe, Y. (2013). Effects of chicken essence on recovery from mental fatigue in healthy males. Med. Sci. Monit. 19, 540–547. doi: 10.12659/MSM.883971
Yu, S., Mückschel, M., and Beste, C. (2021). Event-related synchronization/desynchronization and functional neuroanatomical regions associated with fatigue effects on cognitive flexibility. J. Neurophysiol. 126, 383–397. doi: 10.1152/jn.00228.2021
Zeng, H., Li, X., Borghini, G., Zhao, Y., Aricò, P., Di Flumeri, G., et al. (2021). An EEG-based transfer learning method for cross-subject fatigue mental state prediction. Sensors 21, 2369. doi: 10.3390/s21072369
Zhang, X., Sun, Y., Zhang, Y., and Su, S. (2021). Multi-agent modelling and situational awareness analysis of human-computer interaction in the aircraft cockpit: a case study. Simulat. Model. Pract. Theory 111, 102355. doi: 10.1016/j.simpat.2021.102355
Keywords: adaptive interfaces, mental fatigue, physiological computing, estimation, EEG, eye-tracking, ECG
Citation: Hinss MF, Brock AM and Roy RN (2022) Cognitive effects of prolonged continuous human-machine interaction: The case for mental state-based adaptive interfaces. Front. Neuroergon. 3:935092. doi: 10.3389/fnrgo.2022.935092
Received: 03 May 2022; Accepted: 28 July 2022;
Published: 26 August 2022.
Edited by:
Carryl L. Baldwin, Wichita State University, United StatesReviewed by:
Alpo Juhani Vuorio, University of Helsinki, FinlandMaria Karagianni, Aristotle University of Thessaloniki, Greece
Gianluca Borghini, Sapienza University of Rome, Italy
Copyright © 2022 Hinss, Brock and Roy. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Marcel F. Hinss, marcel.hinss@isae-supaero.fr