Developing an EEG-based on-line closed-loop lapse detection and mitigation system

In America, 60% of adults reported that they have driven a motor vehicle while feeling drowsy, and at least 15–20% of fatal car accidents are fatigue-related. This study translates previous laboratory-oriented neurophysiological research to design, develop, and test an On-line Closed-loop Lapse Detection and Mitigation (OCLDM) System featuring a mobile wireless dry-sensor EEG headgear and a cell-phone based real-time EEG processing platform. Eleven subjects participated in an event-related lane-keeping task, in which they were instructed to manipulate a randomly deviated, fixed-speed cruising car on a 4-lane highway. This was simulated in a 1st person view with an 8-screen and 8-projector immersive virtual-reality environment. When the subjects experienced lapses or failed to respond to events during the experiment, auditory warning was delivered to rectify the performance decrements. However, the arousing auditory signals were not always effective. The EEG spectra exhibited statistically significant differences between effective and ineffective arousing signals, suggesting that EEG spectra could be used as a countermeasure of the efficacy of arousing signals. In this on-line pilot study, the proposed OCLDM System was able to continuously detect EEG signatures of fatigue, deliver arousing warning to subjects suffering momentary cognitive lapses, and assess the efficacy of the warning in near real-time to rectify cognitive lapses. The on-line testing results of the OCLDM System validated the efficacy of the arousing signals in improving subjects' response times to the subsequent lane-departure events. This study may lead to a practical on-line lapse detection and mitigation system in real-world environments.

Introduction

Fatigue-related performance decrements such as lapses in attention and slowed reaction time could lead to catastrophic incidents in occupations ranging from ship navigators to airplane pilots, railroad engineers, truck and auto drivers, and nuclear plant monitors. Fatigue (or drowsiness) “concerns the inability or disinclination to continue an activity, generally because the activity has been going on for too long,” defined by European Transport Safety Council (Croo et al., 2001). Sixty percent of American adults reported that they have been driving a motor vehicle when feeling drowsy (National Sleep Foundation, 2005). Furthermore, studies have concluded that at least 15–20% of fatal car accidents are fatigue-related (The Royal Society for the Prevention of Accidents, 2001; Connor et al., 2002; National Highway Traffic Safety Administration, 2003). Therefore, an earlier detection of driving fatigue is a crucial issue for preventing catastrophic incidents.

In order to detect the driving fatigue, several approaches have been proposed in scientific literature. (1) Computer vision-based systems (Bergasa et al., 2006; D'Orazio et al., 2007; Golz et al., 2010). Bergasa et al. (2006) used a real-time image-acquisition system to monitor drivers' visual behaviors that revealed a drivers' alertness level. Six parameters: percentage of eye closure, eye closure duration, blink frequency, nodding frequency, face position, and fixed gaze were included in a fuzzy classifier for identifying a driver's vigilance level. D'Orazio et al. (2007) proposed a neural classifier to recognize the eye activities from images without being constrained to head rotation or partially occluded eyes. (2) Driving behavior counter-measurements (Lin et al., 2009, 2010a, 2013). Lin et al. (2009) performed an event-related, lane-keeping driving task in an immersive virtual-reality environment. Subjects were asked to steer the stimulated car back to the middle of the cruising lane once they perceived the randomized lane-departure events. The results showed that the reaction time (RT), defined as the time interval between the onset of the simulated car deviation and the user response, could be improved by providing arousing auditory warning to the subjects combating with fatigue.

A Brain Computer Interface (BCI) translates neural activities into control signals to provide a direct communication pathway between the human brain and an external device (Wolpaw et al., 2002). Broadly speaking, BCIs can be grouped into three categories: active, passive and reactive BCIs (Zander and Kothe, 2011). Electroencephalogram (EEG)-based passive BCIs measure brain electrical activities from the scalp and enrich a human–machine interaction with implicit information on the actual user state without conscious effort from the user (Lehne et al., 2009; Zander et al., 2009, 2010; Zander and Jatzev, 2012). Given appropriate signal-processing algorithms in the passive BCIs, meaningful information can be directly extracted from the EEGs. For instance, time-domain analysis such as averaging across different channels, moving average with a specific window length, standard deviation, linear correlation and so on are useful approaches to extract information from EEGs (Dong et al., 2011). In a frequency-domain analysis, the short-time Fourier transform (STFT) is often applied to the EEG data to estimate the power spectral density in distinct frequency bands, including delta (1–3 Hz), theta (4–7 Hz), alpha (8–13 Hz), beta (14–30 Hz), and gamma (31–50 Hz). Many studies have shown that the brain dynamics linked to fatigue and behavioral lapses can be assessed by EEG power spectra (Kecklund and Akerstedt, 1993; Makeig and Inlow, 1993; Jung and Makeig, 1995; Makeig and Jung, 1996; Jung et al., 1997; Lal and Craig, 2002; Campagne et al., 2004; Horne and Baulk, 2004; Debener et al., 2005; Peiris et al., 2006; Davidson et al., 2007; Eichele et al., 2010, 2008; Golz et al., 2010; Lin et al., 2010a), combinations of EEG band power (Jap et al., 2011), alpha spindle parameters (Simon et al., 2011) and autoregressive features (Rosipal et al., 2007). These studies provided solid evidence for the neurophysiological correlates of fatigue and behavioral lapses. In short, while the physical- and behavioral symptom-based methods indirectly measure drivers' cognitive states, the neurophysiology-based methods offer a more direct path to assess the brain dynamic linked to fatigue and behavioral lapses with a high temporal resolution.

Efforts have also been made to assist individuals in combating fatigue and/or preventing lapses in concentration. For instance, Dingus et al. (1997) and Spence and Driver (1998) proposed using warning signals to maintain drivers' attention. The types of warning signals could be auditory (Spence and Driver, 1998; Lin et al., 2009), visual (Liu, 2001), tactile (Ho et al., 2005) or mixed (Liu, 2001). Empirical results showed that auditory warning could reduce the number of lapses in sustained-attention tasks (Spence and Driver, 1998), and could help subjects to maintain driving performance (Lin et al., 2009). More recent studies demonstrated that arousing auditory signals presented to individuals experiencing momentarily behavioral lapses could not only agitate their behavioral responses but also change their EEG theta and alpha power in a sustained-attention driving task (Jung et al., 2010; Lin et al., 2010a, 2013). However, the studies also showed that sometimes subjects did not respond to the arousing signals, and more importantly the EEG activity of these non-responsive episodes showed little or no changes following the ineffective warning (Jung et al., 2010; Lin et al., 2010a, 2013). Lin et al. (2013) later demonstrated the feasibility of using the post-warning EEG power spectra to predict the (in)efficacy of the arousing warning. A caveat of their studies was that the arousing warning was delivered to subjects after they behaviorally failed to respond to lane-departure events. In reality, the delivery of arousing warning could have been too late because the behavioral lapse might have led to catastrophic consequences. A truly EEG-based lapse monitoring system needs to continuously and non-invasively observe EEG dynamics to predict fatigue-related lapses, deliver arousing signal to arouse the user, and assess the efficacy of the arousing signal to trigger a repeated or secondary warning signal if necessary. Furthermore, all of the aforementioned studies were conducted with traditional bulky and tethered EEG systems and were performed in well-controlled laboratories. However, it is argued that there might be fundamentally dynamic differences between laboratory-based and naturalistic human behavior in the brain (McDowell et al., 2013). It thus remains unclear how well the current laboratory-oriented knowledge of EEG correlates of cognitive-state changes can be translated into the highly dynamic real world.

This study aims to extend previous studies to design, develop and test a truly On-line Closed-loop Lapse Detection and Mitigation (OCLDM) System that can continuously monitor EEG dynamics, predict fatigue-related lapses based on EEG signals, arouse the fatigued users by delivering arousing signals, and assess the efficacy of the arousing signal based on EEG spectra. This study hypothesized that (1) EEG spectral values would differ under different arousal states; (2) it is feasible to predict lapses based on the spectral changes in the spontaneous EEG; (3) arousing warning delivered to cognitively challenged subjects would mitigate cognitive lapse, and (4) the rectified performance would be accompanied by the changes in EEG power spectra. This study conducted an off-line experiment to explore the neurophysiological correlates of lapses, which tested the abovementioned hypotheses and guided the development of a truly OCLDM system. The system was then validated by an on-line driving experiment. Furthermore, to be practical for routine use in a car or workplace by freely moving individuals, the EEG-based lapse monitoring system must be non-invasive, non-intrusive, lightweight, battery-powered, and easily to put on and take off (Lin et al., 2008a,b). This study thus also investigates the feasibility of using a practical, low-density, lightweight dry EEG headgear and a smartphone-based EEG-processing platform (Wang et al., 2012) to build a truly mobile and wireless OCLDM System for real-life applications.

Materials and Methods

Subjects

Eleven healthy and naive subjects (10 males and one female) with normal hearing and aged 20–28 years old participated in this study. All of them were free of neurological and psychological disorders. They were introduced how to manipulate the car and practiced ~10 min to get acquainted before the experiment started. None of them worked night shifts or traveled across multiple time zones in the previous 2 months. All participants were asked to read and sign the informed consent form before participating in the studies. After the experiments, subjects were asked to complete the questionnaire for assessing their cognitive states during the experiments.

Experimental Equipment

Experiments of this study were conducted in an 8-screen and 8-projector immersive virtual-reality (VR) environment that simulates the 1st person view scene of highway driving. This study adapted an event-related lane-departure driving paradigm originally proposed by Huang et al. (2006, 2007a,b, 2009) that allowed objective and quantitative measures of momentary event-related brain dynamics following lane-departure events and driving-performance fluctuations over longer periods. The VR scenes simulated driving at a constant speed (at ~100 km/h) on a highway with the simulated car randomly drifting away from the center of the cruising lane to simulate driving on non-ideal road surfaces or with poor alignment (Huang et al., 2007a,b, 2009; Lin et al., 2008b, 2009). The scene was updated according to the land-departure events and the subject's manipulation. The vehicle trajectory, user's input, and lane-departure events could be accurately logged and time-synchronized to the EEG recordings (Huang et al., 2007a,b, 2009; Lin et al., 2008b). There were no traffic or distractive objects other than 4-lane roads and dark sky appeared in the VR while the simulated car was cruising on the highway.

Thirty-two channel EEG data were collected from participants by the NuAmp system (32-channels Quick-Cap, Compumedics Ltd., VIC, Australia). The electrodes were placed according to a modified international 10–20 system with a unipolar reference at the right earlobe. The EEG activities were recorded with 500 Hz sampling rate and 16-bit quantization level.

Experimental Paradigm

Figure 1A shows the experimental paradigm of this study. The simulated car starts cruising at a fixed speed (~100 km/h) on the 3rd lane and drifting to either right or left with equal probability within 8–10 s. Subjects were instructed to steer the simulated car back to the 3rd lane as soon as they noticed the lane drift. The simulated car keeps cruising on the right (or left) most lane if the subjects failed to respond to lane drift. The baseline period of each lane-departure epoch is defined as the 3 s before the onset of a lane-drifting event. The empty circle in Figure 1A represents the unexpected lane-departure events marked as the “deviation onset.” After the deviation onset, subjects were instructed to steer the simulated car back to the center of the cruising lane immediately (double circle), and the time when the subjects started steering was marked as the “response onset.” The moment that the simulated car reached the center of the cruising lane (circle with cross) was marked as the “response offset.” A subject's response time (RT) was defined as the time between the deviation onset and the response onset. At the first 5 min of the experiment, subjects were asked to be fully alert, verified by the vehicle trajectory and the video from a surveillance camera, to obtain an averaged alert RT (aRT) for each subject (1.51~2.54 s), which is a threshold for the entire experiment. The entire experiment consisted of 5 min training and 85 min driving periods.

FIGURE 1

Figure 1. Experiment paradigm. (A) Event-related lane-departure driving tasks. The solid arrows represent the driving trajectory. The empty circle represents the deviation onset. The double circle represents the response onset. The circle with the cross represents the response offset. The baseline is defined as the 3 s period prior to deviation onset. The response time (RT) of a driver is the interval from the deviation (empty circle) to the response onset (the double circle). A trial starts at deviation onset and ends at response offset (circle with a cross). The next deviation begins 8–12 s after response offset. (B) Criterion for delivering auditory warning during driving tasks. The height of an arrow represents the response time in a single trial. The warning was delivered to the subject when the RT in the trial exceeded three times the mean RT of trials in the first 5 min of the task, when the subject was presumably alert and fully attended to lane-departure events. In this figure is adapted with permission from Figure 1 of Lin et al. (2010a).

Figure 1B shows the criterion of delivering auditory warnings in the experiment. When a subject failed to respond within three times the aRT, the system treated the trials as a behavioral lapse and triggered a 1750 Hz tone-burst to arouse the subject from fatigue-related lapse in half (50%) of these drowsy trials (marked as the “current trial (CT)” in Figure 1A). The very next trial is defined as CT + 1, and so on. The lapse trials that were randomly selected to receive arousing warning were referred to as CT with warning, whereas the remaining half of trials that did not receive auditory warning were referred to as CT without warning. Note that our previous studies showed that in some trials subjects remained non-responsive following the arousing warning, which was analogous to sleeping through an alarm clock (Jung et al., 2010; Lin et al., 2013). If the RT of the following trial (CT + 1) was shorter than the double of the averaged aRT, the warning signal delivered in the CT trial was defined as an “effective warning.” On the other hand, if the RT of the CT + 1 trial was longer than triple of the averaged aRT, the warning was defined as an “ineffective warning.” This study did not include the trials with RTs between 2 and 3 aRT to define the alert vs. fatigue spectral thresholds because the cognitive states of the subjects during those trials were unclear. Note that, subjects didn't know about the warning before the experiments.

Data Analysis

The 32-channel EEG data were first down-sampled to 250 Hz, and a low-pass filter of 50 Hz and a high-pass filter of 0.5 Hz were applied. Channels or trials with severe artifacts (such as body movements or muscle activities) were manually removed (less than three channels and 20% trials per subject in general). The remaining EEG data were segmented into several 115 s trials, each of them consisting of 15 s before and 100 s after the lane-deviation onsets. Independent Component Analysis (ICA, Bell and Sejnowski, 1995; Makeig et al., 1997) implemented in EEGLAB (Delorme and Makeig, 2004) was then applied to decompose the ~32-channel EEG into ~32 independent components (ICs), based on the assumption that the collected EEG data from the scalp were a weighted linear mixture of electrical potentials projected instantaneously from temporally ICs accounting for distinct brain sources. The comparable ICs across subjects were grouped into component clusters based on their scalp maps, equivalent dipole locations and baseline power spectra of component activations (Jung et al., 2001; Delorme and Makeig, 2004). Across 11 subjects, there were 155 trials with warning (30 trials were ineffective and 125 trials were effective) and 192 trials without warning.

Since the RT and EEG power were not normally distributed, non-parametric statistic tests were performed for the data analysis (Delorme and Makeig, 2004). The Wilcoxon rank-sum test (Matlab statistical toolbox, Mathworks) was used to assess the effects of warning on RTs. Bootstrapping (EEGLAB toolbox, University of California, San Diego) was used to test the statistical significance of EEG power changes at specific frequency bins from 2 to 30 Hz with a 0.25 Hz resolution. To test group statistics, the intrinsic inter-subject RT differences were reduced by dividing RTs by the mean RT. The EEG spectra were normalized by dividing the spectral power by the standard deviation of the spectral distribution.

Results: Neurophysiological Correlates of Behavioral Lapses

Efficacy of Arousing Auditory Signals for Rectifying Lapses

This study first explored the efficacy of the delivery of arousing auditory signals by measuring the change in subjects' reaction time. Figure 2A shows the boxplots of RTs of three trial groups: Alertness, CT, and CT + 1 (left to right). The averaged aRT of trials within the Alertness group across 11 subjects was ~676 ms. The RTs of the CT group with arousing warning (red and light blue) were statistically significantly shorter than those of trials without receiving arousing warning (dark blue). The RTs of the CT + 1 group with effective vs. ineffective warning differed while the RTs of the preceding group (CT) were comparable. Even though the subjects responded to the arousing warning by immediately steering the simulated car back to the cruising position, they could well be totally non-responsive to the very next lane-departure event (~10 s later). In other words, the arousing signals reliably rectified human behavioral lapses, but did not guarantee that subjects were fully awake, alert, or attentive. This suggests an analogous regime of snooze after an alarm is turned off.

FIGURE 2

Figure 2. Experiment results. (A) The boxplot for the RT distribution of trials with effective warning, ineffective warning, and without warning among CTs and CTs + 1. Note that middle horizontal line is the median of the distribution, and the top and bottom of the rectangle are the third and first quartile, and the dash line ends are the maximum and minimum after outlier removal. (B) The component spectra of the alert CTs (black curve), with an effective warning (red curve), with an ineffective warning (light blue curve) and without warning (dark blue curve). The red, light blue and blue horizontal lines mark the spectral differences between the alert trials and trials with an effective warning, with an ineffective warning, and without warning, respectively. All the spectral plots were calculated from the activity of the bilateral occipital components separated by ICA.

EEG Dynamics Preceding Behavioral Lapses

Figure 2B shows the mean scalp map of the bilateral occipital cluster (upper-right corner) and its component baseline power of drowsy trials without auditory warning (dark blue), with either effective (red) or ineffective warning (light blue). First, among the resultant ICA clusters, bilateral occipital components exhibited statistically significant spectral differences between trials with and without auditory warning. Second, the component power spectra exhibited tonic increases in theta (4–7 Hz), alpha (8–12 Hz), and beta (13–30 Hz) bands in drowsy trials (red, dark blue, and light blue), compared to the alert trials (black). Horizontal lines mark the frequency bins under which the spectral differences between alert trials and drowsy trials with either (in)effective warning, or without warning were statistically significant (alpha = 0.05, Bonferroni adjusted p-value of 0.05/(112 frequency bins) = 0.0004 for multiple comparisons). Note that the spectra shown here were calculated from the component activities prior to the lane-deviation onset. The nearly identical pre-lapse spectra of these three groups of non-responsive trials demonstrate the robustness of the broadband spectral augmentation preceding the behavioral lapses, suggesting the feasibility of using theta and alpha power from the lateral occipital areas to predict behavioral lapses in this sustained-attention driving task.

Effects of Arousing Auditory Signals on the EEG

Next, this study explored temporal spectral dynamics preceding, during and following fatigue-related behavioral lapses and following arousing warning. Figure 3 shows time courses of spectral changes in the bilateral occipital area following ineffective warning (light blue trace), effective warning (red trace), and without warning (dark blue trace), compared to those of the alert trials (black trace). Figure 3 shows that both theta- an alpha-band power steadily increased prior to the lane-departure onset (at time 0 s). Again, the trends of steady increasing theta- and alpha-band power leading to behavioral lapses in the three groups of drowsy trials were nearly identical, indicating the robustness of the theta and alpha augmentation preceding the behavioral lapses.

FIGURE 3

Figure 3. Average component EEG power changes in alpha (top panel) and theta (bottom panel) bands from the bilateral occipital components (lower right corner). All the trials are aligned to the lane-deviation onsets at time 0 s (vertical solid black line). The red, light blue, dark blue, and black traces are the averaged spectra of trials with effective feedback, with ineffective feedback, without feedback, and in alertness, respectively. The green horizontal line indicates the statistically significant differences (p < 0.01) between trials with effective feedback and without feedback. The brown indicates the statistically significant differences (p < 0.01) between trials with effective feedback and ineffective feedback.

Figure 3 also shows that after the lane-departure onset (at time 0 s), the alpha (top panel), and theta (bottom panel) power abruptly decreased by over 10 and 5 dB to nearly the alert (black trace) baseline, respectively. More importantly, following the subjects' responses, the spectra of trials with ineffective warning (light blue trace) and without warning (dark blue trace) rapidly rose from the alert baseline to the drowsy level in 5–15 s. The theta and alpha power of trials with effective warning, however, remained low for ~40 s. The green horizontal lines mark the time points when the difference between the spectra of trials with effective warning and without warning were statistically significant (p < 0.01). The spectral difference between the trials with effective warning and without warning was significant from 7 to 18 s in alpha band and from 7 to 21 s in the theta band (p < 0.01). Furthermore, the spectral difference between the trials with effective and ineffective warning was significant from 7 to 16 s in both alpha and theta bands (brown horizontal lines).

In sum, these results provided invaluable insights into the optimal electrode locations (lateral occipital region) and EEG features (theta- and alpha-band power) for a practical OCLDM system detailed below. The EEG and behavioral data collected from this experiment were used to assess the EEG correlates of fatigue-related lapses and build a lapse prediction model for the second experiment.

Developing a OCLDM System

Our previous study (Wang et al., 2012) proposed a cell-phone based drowsiness monitoring and management system to continuously and wirelessly monitor brain dynamics using a lightweight, portable, and low-density EEG acquisition headgear. The system was designed to assess brain activities over the forehead, detect drowsiness, and deliver arousing warning to users experiencing momentary cognitive lapses, and assess the efficacy of the warning in near real-time. However, the system was not fully implemented nor experimentally validated in humans. Furthermore, according to the neurophysiological results in section Results: Neurophysiological Correlates of Behavioral Lapses, EEG signals collected over the lateral occipital regions were more informative for lapse detection. This study extends the previous work to design, develop, and test an OCLDM System.

System Architecture

Figure 4A shows the system diagram of the proposed OCLDM System. The system consists of two major components: (1) a mobile platform featuring the OCLDM algorithm, and (2) a mobile and wireless 4-channel headgear measuring EEG signals over the hair-bearing occipital regions with dry EEG sensors (Liao et al., 2011). The OCLDM System was implemented as an App on an Android-based platform (e.g., Samsung Galaxy S3). The smartphone has a Bluetooth module, 16 GB RAM, an ARM Cortex-A9 processor, Android (Ice Cream Sandwich) OS, and other components. When the App is launched, it can automatically search and connect to a nearby EEG headgear to receive data from the EEG acquisition headgear. In the mean time, the App opened an USB port to receive the events from a four-lane highway scene to synchronize the EEG data and scene events. The build-in speaker (or plug-in a ear set) of the smartphone delivers auditory warning signal once the OCLDM System detects that the subject is experiencing a cognitive lapse. Both the EEG data and scene-generated events could be logged onto either smartphone's build-in memory or an external microSD card for further analysis.

FIGURE 4

Figure 4. The system diagram of the proposed OCLDM System. (A) The EEG headgear collected 4-channel brain activities from the lateral occipital area while a subject was performing the lane-drifting experiment. The mobile signal-processing platform received the acquired EEG raw data through Bluetooth, and the event markers generated from the lane-departure scene through an USB interface. Finally, the auditory feedback was delivered to the subject when the averaged EEG power across four channels was 3 dB over the alert baseline. (B) A photo of a subject performing the on-line driving experiment while wearing a 4-channel EEG headgear (the white small box attached on a flexible band) over the lateral occipital area.

The mobile and wireless EEG acquisition headgear features a 4-channel lightweight portable bio-signal acquisition device powered by a 3.7 v Li-ion battery (Lin et al., 2010b). It consists of a TI MSP430 microprocessor, a pre-amplifier, a battery-charging circuit, a 24 bit ADC, a Bluetooth module, and dry spring-loaded EEG sensors (Liao et al., 2011). The spring-loaded probes of the sensor can penetrate the hair to provide good electrical conductivity with the scalp. The microprocessor controls all the components including the amplifiers, digitizers, and transmits the digitalized EEG data to the Bluetooth module. The 4-channel EEG data are then transmitted to the authorized receiver of the OCLDM System. Depending on the applications, the system's sampling rate can be programmed at 128, 256, or 512 Hz. An experienced subject can easily put on this EEG acquisition device within 1–3 min without any help from a technician. Figure 4B shows a photo of a subject wearing a 4-channel EEG headgear and performing the simulated driving experiment.

System Software Design

Figure 5 shows the program's state diagram of the proposed OCLDM System. Three major states, including Baseline Collecting (BC), Driving Performance Monitoring (DPM), and Warning Efficacy Assessment (WEA), were implemented in the program. When the program is launched, one can modify the parameters in the SETTING page, shown as a square box in the figure. For instance, the parameters can be the duration of baseline data collection, or the threshold of auditory warning delivering for the other two states. Depending on the applications, the lapse threshold in the DPM state can be calculated accordingly. For example, one can use a combination of power of alpha, beta, theta, and delta bands to detect cognitive lapse. The program then enters the next (DPM) state after the Baseline (calibration) data collection has completed. The DPM module continuously monitors the driver's neurophysiological data. The program stays in the DPM state until the lapse threshold is met, which depends on the neurophysiological results as shown in section Results: Neurophysiological Correlates of Behavioral Lapses. For instance, when the subject's power spectrum in alpha band is 3 dB higher than the threshold (alert baseline collected in the BC state), the program delivers an auditory warning to arouse the subject and enters the FEA state. The current value is stored as the lapse reference in the FEA module. The system repeatedly delivers auditory warning until the EEG power decreases to another threshold.

FIGURE 5

Figure 5. The software state diagram of the OCLDM System. The program first goes through SETTING and Baseline collecting state. Then, the system continuously detects and monitors subjects' driving performance until the EEG spectra indicate cognitive lapse. Note that, Lapse¹ represents the averaged EEG power across four channels is 3 dB over the alert baseline. Lapse² represents that the averaged EEG power across four channels has not yet dropped 3 dB from the lapse power.

On-Line Experimental Paradigm

Three new male subjects (who did not participated in the first experiment) with normal hearing and aged 25–30 years old participated in the on-line closed-loop lane-departure driving experiments to evaluate the OCLDM System in a more naturalistic setting (in a regular office without any electromagnetic shielding). All of them were asked to read and sign the informed consent form before participating in the studies.

The entire experiment consisted of a 1 min training and a ~60 min driving periods. During the training session, subjects were asked to stay fully alert. The averaged alpha power collected in the BC session was used as an alert baseline to determine whether a subject is experiencing cognitive lapses in the driving task. The subject performed the lane-departure driving experiments following the protocol below:

(1) Subjects seated in an armchair and the driving scene was displayed on a 27″ monitor, placed at ~60 cm in front of the subject.

(2) Subjects used a keyboard to control a vehicle cruising on a high way, i.e., a left key turns the simulated car to the left while a right key turns to the right.

(3) Four electrodes were placed over the lateral occipital area to collect EEG data non-invasively. The data were transmitted to a smartphone for processing via Bluetooth.

(4) When the averaged power spectra in alpha band met a certain criterion, arousing auditory warning (~65 dB 1750 Hz tone-burst) would be delivered in half of these lapse episodes through an ear set to the subjects. Note that, the subjects didn't know the warning before the experiments.

(5) The arousing tone-burst would be continuously delivered to the subjects until the averaged power spectra in the alpha band has dropped 3 dB from the lapse power.

The cognitive lapses were detected when the subject's alpha-band power (Jung et al., 2010; Lin et al., 2010a, 2013), calculated by a moving-averaged STFT with a 256-point sliding window advanced at 1 s step running on the smartphone, was 3 dB over the alert baseline power (Lin et al., 2010a, 2013 and Results in section Results: Neurophysiological Correlates of Behavioral Lapses). This study used the alpha power fluctuations to monitor cognitive lapses because (1) a recent study showed that the alpha augmentation was sensitive to the transition from full alertness to mediate drowsiness, while the theta augmentation was more sensitive to the transition from mediate to deep drowsiness (Chuang et al., 2012); (2) the empirical results of this study showed that the augmentation of alpha-band power changes was greater than that of the theta-band power (Figure 2). The system would repeatedly deliver auditory warning until the alpha-band power amplitude has dropped to 3 dB below the power level when the cognitive lapse was identified.

Results from the OCLDM System

The numbers of detected cognitive lapses varied across subjects. Table 1 lists the numbers of trials with effective, ineffective warning and without warning, respectively. Here, the way we defined the effective trials was based on the RT in response to the lane-departure event immediately following the arousing signal (CT + 1 whose RT was shorter than two times aRT); while the ineffective trials had RT longer than three times aRT.

TABLE 1

Table 1. Number of trials collected from the on-line experiment.

Figure 6 shows the boxplot of behavioral performance (RTs) of trials with effective trials (red), ineffective trials (light blue), and without warning (dark blue), compared to the averaged aRT (black) during the on-line experiments. The effective trials had RTs comparable to the averaged aRT (less than 1 s) in both CT and CT + 1. Note that the RTs of CT + 1 with effective vs. ineffective warning differed largely because that was how the effective and ineffective trials were defined. However, the RTs of CT trials (red and light blue) of these two groups of trials were very comparable. That is, even though the subjects responded to the arousing warning by steering the simulated car immediately back to the cruising position, they could well be totally non-responsive to the very next lane-departure event. This finding is consistent to our off-line study reported in section Results: Neurophysiological Correlates of Behavioral Lapses in which the arousing warning was delivered to the subjects who just had a behavioral lapse.

FIGURE 6

Figure 6. The behavioral performance comparison for the trials of with effective feedback (red), with ineffective feedback (light blue), and without feedback (blue), compared to alert trials (black) after removing the outliers.

Figure 7 showed the averaged alpha-band spectral time courses across subjects and trials with effective warning (red trace), with ineffective warning (light blue trace), and without warning (dark blue trace), compared to averaged aRT (black trace). All spectral time courses were aligned to the user response onset (thick vertical black line at time 0 s), and the auditory warning for effective- and ineffective-trials were delivered ~5 s before the user response. In the trials following effective auditory warning, the alpha power decreased steadily and reached the averaged aRT in ~7 s. The power spectra remained as low as that of the alert baseline from 7 to 20 s after response onset. In the trials with ineffective auditory warning, the spectral time series fluctuated fierously due to the small number of trials. In the trials without warning, the alpha power fluctuated before response onset and steadily dereased until ~7 s. Thereafter, the alpha power increased again from ~7 to 13 s, suggesting the subjects might be partially arouse by the lane-departure event and their own bebavioral reposense temportally but returned to the fatigue state rapidly thereafter.

FIGURE 7

Figure 7. The averaged alpha power time course plotting time-locked to subject response onset (vertical solid line at time 0 s). Averaged alpha power of trials with effective feedback (red trace), with ineffective feedback (light blue), and without feedback (blue trace), compared to trial with aRT (black). The time course of power was estimated by short time Fourier transform with 256 points of time window and 224 points overlapping.

Discussions

Many studies have shown that the brain dynamics correlated with behavioral lapses can be assessed from EEG data. Recent studies have also shown auditory signals can arouse drowsy subjects and affect EEG activities (Lin et al., 2010a, 2013). However, in these studies, the arousing warning was delivered to subjects after they displayed behavioral lapses, which in reality may be too late because the behavioral lapse might have already had catastrophic consequences. Therefore, a system that features real-time lapse detection and delivers warnings to the drowsy subjects is desirable in preventing catastrophic incidents while driving.

The first experiment of this study showed that EEG power changes in either alpha or theta band can be used as an indicator for assessing the subjects' fatigue (cf. Figure 3), and auditory warning temporarily reduces the alpha and theta band power and mitigates the behavioral lapses (cf. Figure 2A). In addition, EEG changes after delivery of auditory warning are a good indicator of the efficacy of arousing warning. More importantly and interestingly, empirical results of the first study showed that arousing auditory signals could always reliably mitigate human behavioral lapses, but these immediate behavioral responses could not guarantee the subjects were fully awake, alert, or attentive, similar to snooze after an alarm is turned off. This finding may open a new research direction of how to accurately confirm a subject's cognitive level for some sustained-attention tasks, such as an aircraft navigator or a long-haul truck driver. In other words, further studies to explore the brain changes in this sleep inertia period may provide valuable insights of brain dynamics during a transitional state of lowered arousal occurring immediately after awakening from sleep. Based on previous studies (Lin et al., 2010a, 2013) and the results of the first experiment, this study further developed a truly OCLDM system to detect/predict cognitive lapse based on the EEG spectra, deliver arousing warning on the occurrence of cognitive lapse, and assess the efficacy of the arousing warning, again, based on the EEG spectra. Most importantly, the EEG spectra changes within ~10 s after delivering arousing warning were closely monitored, such that any false-awake situations could be decreased. This study then documented the design, development, and on-line evaluation of the proposed OCLDM System that featured a lightweight wireless EEG acquisition headgear and a smartphone-based signal-processing platform. Experimental results showed that subjects' EEG power could almost remain at the alert state without bouncing back to the drowsy level (cf. Figure 7). These results suggest that the proposed system could prevent potential behavioral lapses based solely on the EEG signals, and this demonstration could lead to a real-life application of the dry and wireless EEG technology and smartphone-based signal-processing platform. An interesting question is if the neural correlates of fatigue could be generalized across different sustained-attention tasks and different recording conditions. In the past few years, we have conducted several sustained-attention tasks, including auditory target detection tasks (Makeig and Jung, 1995, 1996; Jung et al., 1997), visual compensatory tracking tasks (Huang et al., 2008), and simulated driving tasks (Lin et al., 2005, 2006, 2008b) and found that performance-related EEG dynamics were comparable across tasks (Huang et al., 2007b). Results of these studies also showed the fatigue-related brain dynamics were quite consistent across different recording environments (within a well-controlled EEG laboratory vs. a 6-degree-of-freedom motion platform) and responding methods (using a button press or a steering wheel). Therefore, it is reasonable to believe the methods developed under this study could be translated from laboratory settings to real-world environments.

In sum, this study demonstrated the feasibility of translating a laboratory-based passive BCI system to a neuroergonomic device that is capable of continuously monitoring and mitigating operator neurocognitive fatigue using a pervasive smartphone in real-world environments. The passive BCI technologies might also be applicable to other real-world cognitive-state monitoring, such as attention, distraction, comprehension, confusion, and emotion. We thus believe more real-world passive BCI implementations will emerge in the foreseeable future.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgment

This work was supported in part by US Office of Naval Research, Army Research Office (W911NF-09-1-0510), Army Research Laboratory (W911NF-10-2-0022), and DARPA (DARPA/USDI D11PC20183). This work was also supported in part by the UST-UCSD International Center of Excellence in Advanced Bio-engineering sponsored by the Taiwan National Science Council I-RiCE Program under Grant Number: NSC-102-2911-I-009-101, and the Aiming for the Top University Plan of National Chiao Tung University sponsored by the Ministry of Education of Taiwan, under Grant Number 103W963. The authors also appreciate Melody Jung's editorial assistance.

References

Bell, A. J., and Sejnowski, T. J. (1995). An information-maximization approach to blind separation and blind deconvolution. Neural Comput. 7, 1129–1159. doi: 10.1162/neco.1995.7.6.1129

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Bergasa, L. M., Nuevo, J., Sotelo, M. A., Barea, R., and Lopez, M. E. (2006). Real-time system for monitoring driver vigilance. IEEE Trans. Intell. Transport. Syst. 7, 63–67. doi: 10.1109/TITS.2006.869598

CrossRef Full Text | Google Scholar

Campagne, A., Pebayle, T., and Muzet, A. (2004). Correlation between driving errors and vigilance level: influence of the driver's age. Physiol. Behav. 80, 515–524. doi: 10.1016/j.physbeh.2003.10.004

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Chuang, C.-H., Huang, C.-S., Lin, C.-T., Ko, L.-W., Chang, J.-Y., and Yang, J.-M. (2012). “Mapping information flow of independent source to predict conscious level: a granger causality based brain-computer interface,” in International Symposium on Computer, Consumer and Control (IS3C) (Taichung: IEEE), 813–816. doi: 10.1109/IS3C.2012.209

CrossRef Full Text

Connor, J., Norton, R., Ameratunga, S., Robinson, E., Civil, I., and Dunn, R. (2002). Driver sleepiness and risk of serious injury to car occupants: population based case control study. BMJ 324:1125. doi: 10.1136/bmj.324.7346.1125

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Croo, H. D., Bandmann, M., Mackay, G. M., Rumar, K., and Vollenhoven, P. V. (2001). The Role of Driver Fatigue in Commercial Road Transport Crashes. Etterbeek: European Transport Safety Council.

Davidson, P. R., Jones, R. D., Peiris, M., and Davidson, T. R. (2007). EEG-based lapse detection with high temporal resolution. IEEE Trans. Biomed. Eng. 54, 832–839. doi: 10.1109/TBME.2007.893452

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Debener, S., Ullsperger, M., Siegel, M., Fiehler, K., Von Cramon, D. Y., and Engel, A. K. (2005). Trial-by-trial coupling of concurrent EEG and fMRI identifies the dynamics of performance monitoring. J. Neurosci. 25, 11730–11737. doi: 10.1523/JNEUROSCI.3286-05.2005

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Delorme, A., and Makeig, S. (2004). EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 134, 9–21. doi: 10.1016/j.jneumeth.2003.10.009

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Dingus, T. A., McGehee, D. V., Manakkal, N., Jahns, S. K., Carney, C., and Hankey, J. M. (1997). Human factors field evaluation of automotive headway maintenance/collision warning devices. Hum. Factors 39, 216–229. doi: 10.1518/001872097778543930

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Dong, Y., Hu, Z., Uchimura, K., and Murayama, N. (2011). “Driver inattention monitoring system for intelligent vehicles: a review,” in 2009 IEEE Intelligent Vehicles Symposium (Xi'an), 875–880.

D'Orazio, T., Leo, M., Guaragnella, C., and Distante, A. (2007). A visual approach for driver inattention detection. J. Pattern Recogn. 40, 2341–2355. doi: 10.1016/j.patcog.2007.01.018

CrossRef Full Text | Google Scholar

Eichele, H., Juvodden, H. T., Ullsperger, M., and Eichele, T. (2010). Mal-adaptation of event-related EEG responses preceding performance errors. Front. Hum. Neurosci. 4:65. doi: 10.3389/fnhum.2010.00065

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Eichele, T., Debener, S., Calhoun, V. D., Specht, K., Engel, A. K., Hugdahl, K., et al. (2008). Prediction of human errors by maladaptive changes in event-related brain networks. Proc. Natl. Acad. Sci. U.S.A. 105, 6173–6178. doi: 10.1073/pnas.0708965105

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Golz, M., Sommer, D., Trutschel, U., Sirois, B., and Edwards, D. (2010). Evaluation of fatigue monitoring technologies. Somnologie Schlafforschung und Schlafmedizin 14, 187–199. doi: 10.1007/s11818-010-0482-9

CrossRef Full Text | Google Scholar

Ho, C., Tang, H. Z., and Spence, C. (2005). Using spatial vibrotactile cues to direct visual attention in driving scenes. Transp. Res. F Traffic Psychol. Behav. 8, 397–412. doi: 10.1016/j.trf.2005.05.002

CrossRef Full Text | Google Scholar

Horne, J. A., and Baulk, S. D. (2004). Awareness of sleepiness when driving. Psychophysiology 41, 161–165. doi: 10.1046/j.1469-8986.2003.00130.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Huang, R.-S., Jung, T.-P., Delorme, A., and Makeig, S. (2008). Tonic and phasic electroencephalographic dynamics during continuous compensatory tracking. Neuroimage 39, 1896–1909. doi: 10.1016/j.neuroimage.2007.10.036

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Huang, R.-S., Jung, T.-P., Delorme, A., and Makeig, S. (2009). “Tonic and phasic brain dynamics during responses to simulated driving challenges,” in 15th Annual Meeting, Organization of Human Brain Mapping, NeuroImage 47, Supplement 1, S103 (San Francisco, CA).

Huang, R.-S., Jung, T.-P., Duann, J.-R., and Makeig, S. (2006). Imaging Brain Dynamics during Continuous Driving using Independent Component Analysis, in Human Brain Mapping. Florence: Wiley Periodicals Inc.

Huang, R.-S., Jung, T.-P., and Makeig, S. (2007a). “Multi-scale EEG brain dynamics during sustained attention tasks,” in Acoustics, Speech and Signal Processing, 2007, ICASSP 2007, IEEE International Conference (Honolulu, HI), 1173–1176.

Huang, R.-S., Jung, T.-P., and Makeig, S. (2007b). “Event-related brain dynamics in continuous sustained-attention tasks,” in Foundations of Augmented Cognition, eds D. Schmorrow and L. Reeves (Berlin; Heidelberg: Springer), 65–74.

Jap, B. T., Lal, S., and Fischer, P. (2011). Comparing combinations of EEG activity in train drivers during monotonous driving. Expert Syst. Appl. 38, 996–1003. doi: 10.1016/j.eswa.2010.07.109

CrossRef Full Text | Google Scholar

Jung, T.-P., Huang, K.-C., Chuang, C.-H., Chen, J.-A., Ko, L.-W., Chiu, T.-W., et al. (2010). “Arousing feedback rectifies lapse in performance and corresponding EEG power spectrum,” in Engineering in Medicine and Biology Society (EMBC), 2010 Annual International Conference, IEEE) (Buenos Aires), 1792–1795.

Pubmed Abstract | Pubmed Full Text

Jung, T.-P., and Makeig, S. (1995). “Prediction failures in auditory detection from changes in the EEG spectrum,” in Proceeding of the 17th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (Montreal, QC), 927–928.

Jung, T.-P., Makeig, S., Stensmo, M., and Sejnowski, T. J. (1997). Estimating alertness from the EEG power spectrum. IEEE Trans. Biomed. Eng. 44, 60–69. doi: 10.1109/10.553713

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Jung, T.-P., Makeig, S., Westerfield, M., Townsend, J., Courchesne, E., and Sejnowski, T. J. (2001). Analysis and visualization of single-trial event-related potentials. Hum. Brain Mapp. 14, 166–185. doi: 10.1002/hbm.1050

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Kecklund, G., and Akerstedt, T. (1993). Sleepiness in long-distance truck driving: an ambulatory EEG study of night driving. Ergonomics 36, 1007–1017. doi: 10.1080/00140139308967973

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Lal, S. K., and Craig, A. (2002). Driver fatigue: electroencephalography and psychological assessment. Psychophysiology 39, 313–321. doi: 10.1017/S0048577201393095

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Lehne, M., Ihme, K., Brouwer, A. M., van Erp, J., and Zander, T. O. (2009). “Error-related EEG patterns during tactile human-machine interaction,” in Affective Computing and Intelligent Interaction and Workshops, 2009. ACII2009. 3rd International Conference (Los Alamitos, CA), 1–9.

Liao, L.-D., Wang, I.-J., Chen, S.-F., Chang, J.-Y., and Lin, C.-T. (2011). Design, fabrication and experimental validation of a novel dry-contact sensor for measuring electroencephalography signals without skin preparation. Sensors 11, 5819–5834. doi: 10.3390/s110605819

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Lin, C.-T., Chen, Y.-C., Huang, T.-Y., Chiu, T.-T., Ko, L.-W., Liang, S.-F., et al. (2008a). Development of wireless brain computer interface with embedded multitask scheduling and its application on real-time driver's drowsiness detection and warning. IEEE Trans. Biomed. Eng. 55, 1582–1591. doi: 10.1109/TBME.2008.918566

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Lin, C.-T., Huang, K.-C., Chao, C.-F., Chen, J.-A., Chiu, T.-W., Ko, L.-W., et al. (2010a). Tonic and phasic EEG and behavioral changes induced by arousing feedback. Neuroimage 52, 633–642. doi: 10.1016/j.neuroimage.2010.04.250

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Lin, C.-T., Huang, K.-C., Chao, C.-F., Chuang, C.-H., Ko, L.-W., and Jung, T.-P. (2013). Can Arousing feedback rectify lapses in driving? Prediction from EEG power spectra. J. Neural Eng. 10:056024. doi: 10.1088/1741-2560/10/5/056024

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Lin, C.-T., Huang, T.-Y., Liang, W.-C., Chiu, T.-T., Chao, C.-P., Hsu, S.-H., et al. (2009). Assessing effectiveness of various auditory warning signals in maintaining drivers' attention in virtual reality-based driving environments. Percept. Mot. Skills 108, 825–835. doi: 10.2466/pms.108.3.825-835

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Lin, C.-T., Ko, L.-W., Chang, M.-H., Duann, J.-R., Chen, J.-Y., Su, T.-P., et al. (2010b). Review of wireless and wearable electroencephalogram systems and brain–computer interfaces—a mini-review. Gerontology 56, 112–119. doi: 10.1159/000230807

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Lin, C.-T., Ko, L.-W., Chiou, J.-C., Duann, J.-R., Chiu, T.-W., Huang, R.-S., et al. (2008b). Noninvasive neural prosthesis using mobile and wireless EEG. Proc. IEEE 96, 1167–1183. doi: 10.1109/JPROC.2008.922561

CrossRef Full Text | Google Scholar

Lin, C.-T., Ko, L.-W., Chung, I.-F., Huang, T.-Y., Chen, Y.-C., Jung, T.-P., et al. (2006). Adaptive EEG-based alertness estimation system by Using ICA-based fuzzy neural networks. IEEE Trans. Circuits Syst. I 53, 2469–2476. doi: 10.1109/TCSI.2006.884408

CrossRef Full Text | Google Scholar

Lin, C.-T., Wu, R.-C., Liang, S.-F., Huang, T.-Y., Chao, W.-H., Chen, Y.-J., et al. (2005). EEG-based drowsiness estimation for safety driving using independent component analysis. IEEE Trans. Circuits Syst. 52, 2726–2738. doi: 10.1109/TCSI.2005.857555

CrossRef Full Text | Google Scholar

Liu, Y.-C. (2001). Comparative study of the effects of auditory, visual and multimodality displays on drivers' performance in advanced traveller information systems. Ergonomics 44, 425–442. doi: 10.1080/00140130010011369

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Makeig, S., and Inlow, M. (1993). Lapses in alertness: coherence of fluctuations in performance and EEG spectrum. Electroencephalogr. Clin. Neurophysiol. 86, 23–35. doi: 10.1016/0013-4694(93)90064-3

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Makeig, S., and Jung, T.-P. (1995). Changes in alertness is principal component of variance in the EEG spectrum. Neuroreport 7, 213–216, doi: 10.1097/00001756-199512000-00051

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Makeig, S., and Jung, T.-P. (1996). Tonic, phasic, and transient EEG correlates of auditory awareness in drowsiness. Brain Res. Cogn. Brain Res. 4, 15–25. doi: 10.1016/0926-6410(95)00042-9

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Makeig, S., Jung, T.-P., Bell, A. J., Ghahremani, D., and Sejnowski, T. J. (1997). Blind separation ofauditory event-related brain responses into independent components. Proc. Natl. Acad. Sci. U.S.A. 94, 10979–10984.

Pubmed Abstract | Pubmed Full Text | Google Scholar

McDowell, K., Lin, C.-T., Oie, K. S., Jung, T.-P., Gordon, S., Whitaker, K. W., et al. (2013). Real-world neuroimaging technologies. IEEE Access 1, 131–149. doi: 10.1109/ACCESS.2013.2260791

CrossRef Full Text | Google Scholar

National Highway Traffic Safety Administration. (2003). Drowsy Driving and Automobile Crashes. New Jersey, NJ: N.H.T.S. Administration.

National Sleep Foundation. (2005). Summary Findings of the 2005 Sleep in America Poll. Arlington, VA. Available online at: http://sleepfoundation.org/sites/default/files/2005_summary_of_findings.pdf

Peiris, M. T. R., Jones, R. D., Davidson, P. R., Carroll, G. J., and Bones, P. J. (2006). Frequent lapses of responsiveness during an extended visuomotor tracking task in non-sleep-deprived subjects. J. Sleep Res. 15, 291–300. doi: 10.1111/j.1365-2869.2006.00545.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Rosipal, R., Peters, B., Kecklund, G., Akerstedt, T., Gruber, G., Woertz, M., et al. (2007). “EEG-based drivers'drowsiness monitoring using a hierarchical Gaussian mixture model,” in Foundations of Augmented Cognition, eds D. Schmorrow and L. Reeves (Berlin; Heidelberg: Springer), 294–303.

Simon, M., Schmidt, E. A., Kincses, W. E., Fritzsche, M., Bruns, A., Aufmuth, C., et al. (2011). EEG alpha spindle measures as indicators of driver fatigue under real traffic conditions. Clin. Neurophysiol. 122, 1168–1178. doi: 10.1016/j.clinph.2010.10.044

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Spence, C., and Driver, J. (1998). Inhibition of return following an auditory cue. The role of central reorienting events. Exp. Brain Res. 118, 352–360. doi: 10.1007/s002210050289

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

The Royal Society for the Prevention of Accidents. (2001). Driver Fatigue and Road Accidents. A Literature Review and Position Paper. Birmingham. Available online at: http://www.rospa.com/roadsafety/info/fatigue.pdf

Wang, Y.-T., Cheng, C.-K., Huang, K.-C., Lin, C.-T., Wang, Y., and Jung, T.-P. (2012). “Cell-phone based drowsiness monitoring and management system,” in 2012 IEEE on Biomedical Circuits and Systems Conference (BioCAS) (Hsinchu), 200–203.

Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfutscheller, G., and Vaughan, T. M. (2002). Brain-computer interfaces for communication and control. Clin. Neurophysiol. 113, 767–791. doi: 10.1016/S1388-2457(02)00057-3

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Zander, T. O., and Jatzev, S. (2012). Context-aware brain–computer interfaces: exploring the information space of user, technical system and environment. J. Neural Eng. 9:016003. doi: 10.1088/1741-2560/9/1/016003

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Zander, T. O., and Kothe, C. (2011). Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general. J. Neural Eng. 8:025005. doi: 10.1088/1741-2560/8/2/025005

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Zander, T. O., Kothe, C., Jatzev, S., and Gaertner, M. (2010). “Enhancing human-computer interaction with input from active and passive brain-computer interfaces,” in Brain-Computer Interfaces, eds D. S. Tan and A. Nijholt (London: Springer), 181–199.

Google Scholar

Zander, T. O., Kothe, C., Welke, S., and Roetting, M. (2009). “Utilizing secondary input from passive brain-computer interfaces for enhancing human-machine interaction,” in Foundations of Augmented Cognition. Neuroergonomics and Operational Neuroscience, eds D. Schomorrow, I. Estabrooke, and M. Grootjen (Berlin; Heidelberg: Springer), 759–771.