Chama Belkhiria
Vsevolod Peysakhovich- ISAE-SUPAERO, Université de Toulouse, Toulouse, France
Electro-encephalography (EEG) and electro-oculography (EOG) are methods of electrophysiological monitoring that have potentially fruitful applications in neuroscience, clinical exploration, the aeronautical industry, and other sectors. These methods are often the most straightforward way of evaluating brain oscillations and eye movements, as they use standard laboratory or mobile techniques. This review describes the potential of EEG and EOG systems and the application of these methods in aeronautics. For example, EEG and EOG signals can be used to design brain-computer interfaces (BCI) and to interpret brain activity, such as monitoring the mental state of a pilot in determining their workload. The main objectives of this review are to, (i) offer an in-depth review of literature on the basics of EEG and EOG and their application in aeronautics; (ii) to explore the methodology and trends of research in combined EEG-EOG studies over the last decade; and (iii) to provide methodological guidelines for beginners and experts when applying these methods in environments outside the laboratory, with a particular focus on human factors and aeronautics. The study used databases from scientific, clinical, and neural engineering fields. The review first introduces the characteristics and the application of both EEG and EOG in aeronautics, undertaking a large review of relevant literature, from early to more recent studies. We then built a novel taxonomy model that includes 150 combined EEG-EOG papers published in peer-reviewed scientific journals and conferences from January 2010 to March 2020. Several data elements were reviewed for each study (e.g., pre-processing, extracted features and performance metrics), which were then examined to uncover trends in aeronautics and summarize interesting methods from this important body of literature. Finally, the review considers the advantages and limitations of these methods as well as future challenges.
Introduction
Electro-encephalography (EEG) and electro-oculography (EOG) are methods of electrophysiological monitoring in neuroscience and clinical exploration. EEG and EOG signals can be used in the design of brain-computer interfaces (BCI) that interpret brain activity. Due to the fact that they are straightforward approaches to evaluating brain oscillations and eye movements, and because they use standard laboratory and/or mobile techniques, EEG and EOG have in recent years been applied to the aeronautical industry.
This review describes the potential of these systems when applied in aeronautics. The main objectives are, (i) to offer an in-depth review of literature on the basics of EEG and EOG and their application in aeronautics; (ii) to explore the methodology and trends of research in combined EEG-EOG studies over the last decade; and (iii) to provide methodological guidelines for beginners and experts when applying these methods in environments outside the laboratory, with a particular focus on human factors and aeronautics.
The review is structured as follows: section Introduction first describe EEG and EOG techniques, the main approaches to acquiring signals, and the use of them in aeronautics, before concluding with discussions of the motivations for applying them to aeronautics, and a discussion of contributions to this field. Section Methodology of the Review describes the methodology used to construct the review and taxonomy table, respectively. Section Results then presents the results of the review, including the trend analyses. Finally, section Discussion discusses research over the last decade with a focus on combined approaches to EEG-EOG and the relevance of this approach to aeronautics.
Electroencephalography (EEG)
Origin of EEG Response
EEG is one of the most important methods of evaluating brain disorders and monitoring the electrical behavior of the brain. The EEG also has the major advantage of excellent temporal resolution, which enables it to study neural activity at a millisecond scale, and best approximates the neural timing. EEG allows for the analysis of the various rhythms generated by different cortical regions. The current produced by the electrical activity of neurons reaches the surface of the scalp. EEG offers a non-invasive method of recording the difference between the potentials that are generated by neural sources and annoying non-neural artifacts. As the signal induces important temporal and spatial variations, the electrode positions are determined using multiple channels settled by the international 10–20 standard. Recommendations for the use of EEG equipment in assessments are provided by the International Federation of Clinical Neurophysiology (Babiloni et al., 2020). The electrical signal is diffused from electrodes placed on the scalp to an external amplifier that intensifies the potentials. EEG signals are commonly detected between 20 and 150 μV in the 0.5–60 Hz band (Binnie et al., 1982). The signal is constantly sampled to provide a relevant temporal resolution to explore event-related potentials (ERP) and the EEG power spectrum. In research focused on frequency-based analyses (such as prefrontal lateralization of alpha or beta bands), a sampling rate of 128 Hz can be sufficient. When the objective is high time precision measurements (such as language-related high gamma activity), the EEG should ideally collect data at a high sampling rate (>500 Hz).
EEG analysis could be conducted in the time domain, frequency domain, or time-frequency domain. Extracting temporal features (e.g., amplitude, power, average periodicity, and synchronization) provides useful qualitative information for the classification. However, all of these temporal characteristics do not describe the signal in its integrity. During a seizure, for instance, the signal is not stable, and it is necessary to separate the frequency components to classify the seizure. Therefore, in addition to time-domain features, frequency domain exploration is also needed to detect and classify all types of seizures. A time-domain analysis provides better spatial information, thus poor frequency content information is required for EEG classification. The frequency-domain can provide time information when the function is windowed. The choice of window size is the biggest challenge in frequency analysis. Time-frequency analysis solves these two problems. Some EEG investigations consider that wavelet analysis is the best method for time-frequency analysis. Generally, authors apply a series of transformations e.g., Fourier transform (Radha et al., 2014), Short Time Fourier Transform (Görür et al., 2003), Wavelet transform (Fraiwan et al., 2012), Hilbert-Hung transform (Li et al., 2009), and Empirical Mode Decomposition (Hassan and Bhuiyan, 2016). Spectral analyses, based on Fourier Transform, are then commonly used to convert the time function into different frequencies and to calculate the amplitude in each frequency band. The frequency bands are universally classified as the following: slow and sleep wave delta (2–4 Hz), arousal wave theta (4–8 Hz), relaxation wave alpha (8–12 Hz), and active wave beta (13–32 Hz). Sensorimotor rhythm frequency bands (13–15 Hz) are related to the sensorimotor rhythm and entitled as low beta. Delta waves are commonly frontally located in adults and posteriorly in children. Theta waves are mainly recorded in frontal areas during low brain activities, sleep, or drowsiness and cognitive processing. Alpha waves are among the first rhythmic waves documented and are recorded during relaxed conditions at decreased attention levels and in a wakeful state. The alpha waves are located in the occipital area and can be induced by closing eyes. Beta waves are often recorded in frontal or central areas when the eyes are open and are related to consciousness, alertness, arousal, and motor behaviors (Barry and De Blasio, 2017). Cognitive processes such as attention, learning, and diverse types of memory occur during gamma frequencies (over 33 Hz). Unconventional classifications have also been analyzed in some studies (Caldwell et al., 2002; Gevins et al., 2003; Dahlstrom and Nahlinder, 2009; Holm et al., 2009; De Vico Fallani et al., 2012; Zhang et al., 2019a). It is worth mentioning that the frequency limits of specific waves are conventional, as there is no proper way of determining their exact values.
Many studies on oscillations in brain dynamics have indicated that during fatigue accumulation and sustained attention, increased EEG power is detected in theta frequency bands in the frontal, parietal, and central regions. Theta power increase has been detected during working memory load situations (Klimesch et al., 2007), visual tasks (Yamada, 1998), flight simulations (Smith-Jentsch et al., 2001; Dussault et al., 2004; Borghini et al., 2014), and air-traffic control simulations (Postma et al., 2005). A decrease in the alpha is known to occur during complex and cognitively demanding tasks. It has also been shown (Postma et al., 2005) that alpha and beta bands are different between the beginning and the end of a mental fatigue task. Increases in theta and decreases in alpha oscillations have also been associated with an increase in the accuracy of task performance (Klimesch et al., 1999). Interestingly, such an increase in EEG power in the theta band can be used to characterize a single task from a multi-task activity performed by pilots (Borghini et al., 2014). Varied EEG approach montages are given in complex higher-order cognitive operations.
EEG Approaches
Although great importance has been placed on classic unmovable EEG montages, they have disadvantages in that they are much bulkier and more time and effort consuming than the new generation of wearable EEG. The concept of wearable EEG improves upon the bulky and limited mobility of classic montages, using small devices that can record EEG outside of laboratory conditions. These miniaturized devices have the advantage of being able to detect EEG signals for short or long periods, for example when sleeping, which significantly improves the brain-computer interface (BCI) monitoring method. The importance of using the wireless EEG in BCI monitoring is to maximize wearability, enabling unconstrained mobility, usability, and reliability in operational environments.
As well as involving long preparation time and bulky design, classical scalp-mounted EEG are not suited to situations that involve environmental artifacts (e.g., aviation and space operation, or patients with a cochlear implant or hearing aid; Nogueira et al., 2019). To address these issues, novel EEG technologies use tiny electrodes that are placed externally around the ear (Debener et al., 2015; Bleichner and Debener, 2017) or involve in-ear electrodes (Looney et al., 2011). The cEEGrid electrode array (Debener et al., 2015) is a promising device with 10 electrodes printed onto a C-shaped flexible board, enabling it to fit around and measure EEG data behind the ear. It also uses a small amount of electrolyte electrode gel, which ensures low-impedance contact between the cEEGrid electrodes and the skin. The capacity is stable because the gel does not dry-up. The cEEGrid electrode array is then connected to a micro wireless amplifier and uses signal recordings from a cellphone. This accessible design means that it has potential applications in clinical settings, aeronautics, and other research areas.
With a reduced number of electrodes, the difference between cEEGrid performance and scalp EEG performance can be explained by the position of the reference electrode. The cEEGrid uses a local reference, with ground and recording electrodes allocated around the ear, giving out small signal amplitudes. However, for a scalp EEG with a smaller number of electrodes around the ear, the tip of the nose is used as a reference, and its position is farther from the recording electrode, and therefore gives higher amplitudes. In line with conventional EEG results, Debener et al. (2015) identified ERPs and alpha frequencies during an auditory oddball task with open eyes using the cEEGrid. Advanced explorations proved that the cEEGrid can detect neural signals to select the voice of a speaker with high precision, and can be even used as a BCI monitor for hearing aids (Mirkovic et al., 2016). Recently, the cEEGrid was successfully used in decoding selective attention in normal hearing and cochlear implant patients (Nogueira et al., 2019). Looney et al. (2011) presented the in-ear EEG device. It records to the same standards as the conventional scalp electrodes. However, the device relies on custom-made hearing aid earplugs, which take a wax impression of the ears (outer ear and external ear canal). The important benefits of in-ear EEG includes easy set-up; time of installation; accessibility for people with hair that is incompatible with EEG recordings; and durability because they are fixed in the ear canal. They are comfortable to wear, discreet, resembling earphones, earbuds, and earplugs, and facilitating everyday use. Contrary to scalp EEG, the in-ear EEG devices are easy to place without the presence of experienced assistants. They are held firmly in place and thus diminish motion artifacts. Since the electrodes are fixed on the earpiece surface, they offer a precise spatial positioning which decreases the inter-experiment variability. The feasibility of in-ear EEG for cognitive assessment has been studied in a few exploratory papers (Stochholm et al., 2016; Von Rosenberg et al., 2016; Zibrandtsen et al., 2016), which claim that in-ear EEG is a promising candidate for forthcoming explorations based on human monitoring technology (BCI, aviation, and space).
Despite advances in mobile EEG systems, around and in-ear EEG have some drawbacks. Compared to conventional scalp EEGs, these alternative systems have fewer electrodes and cover much-reduced regions. Therefore, it is recognized that its brain source analysis is less accurate than that of conventional scalp EEG. Note that physiological artifacts caused by the electrical activity of the skin are unavoidable, but may be relatively easy to deal with by temporal filtering and other post-processing procedures (Reis et al., 2014). The quality of the scalp EEG signal depends on the connection between the amplifier input and the skin surface. Wet electrodes based on conductive gel guarantee low impedance levels (<10 kΩ). Given that dry electrodes are placed on the skin without any gel application, the dry EEG system typically results in larger impedance than wet systems (Brown et al., 2010; Chen et al., 2014). To date, few studies have directly compared the data quality between these two systems. A recent study (Hinrichs et al., 2020) found that the resting state EEG power and ERP were comparable between the two systems. Di Flumeri et al. (2019) evaluated three different dry electrode types when compared with wet electrodes in terms of signal spectral features, mental state classification, and usability. The dry electrodes included a gold-coated single pin, multiple pins, and solid-gel electrodes. The results confirmed the high quality achieved by dry electrodes. They offered the same levels as wet electrodes with significantly reduced times of montage and increased the comfort of users. Although the signal quality is inevitably reduced, the dry electrodes are a reliable system for non-clinical and goal-oriented investigations, such as a comparison between two different mental states during real flight conditions.
EEG in Aeronautics
Since the late 1950s, many studies have investigated changes in EEG rhythms during flight conditions (Carl et al., 1959) despite the noisy environment (e.g., vibration, wind, acoustic noise, physiological artifacts, and important pilot physical movements). The recorded EEG signals show specific features and changes in the power spectrum of the various frequency bands associated with flight performance (Carl et al., 1959). One EEG experiment (Callan et al., 2015) conducted under real flight conditions, showed that the neural signature of inattentional deafness (e.g., inadvertently missing an auditory stimulus) was revealed by a reduction in phase resetting in the alpha and theta band frequencies. Sauvet et al. (2014) have used a single EEG channel during real long-haul flights to detect low states of vigilance. Another EEG study involved a critical scenario in a flight simulator (Dehais et al., 2016), which detected the existence of an early and unconscious gating mechanism based on the association between the auditory alarm and the N100 and P300 amplitude. Di Stasi et al. (2016) analyzed the in-flight EEG activity of military helicopter pilots during real flights. They found that highly demanding procedures related to takeoff and landing were associated with higher power EEG frequency bands, whereas less demanding procedures were associated with lower EEG power over the same frequency bands. Several EEG investigations have focused on the mental workload of the operator during a flight task execution. For example, EEG revealed variations in alertness and could predict lower performance caused by increased mental workload during flight operations (Borghini et al., 2014). Theta changes were observed over the frontal brain areas when comparing the training improvements of novice pilots in flight simulation tasks (Borghini et al., 2014).
Workload perception was shown to be dependent on the level of experience, the abilities, or just the individual differences between pilots. The EEG was sensitive to workload level variance between novice and expert aircraft pilots during the execution of the identical task (Doppelmayr et al., 2008; Parasuraman and Jiang, 2012).
Recent studies exploring Air Traffic Controllers (Bernhardt et al., 2019) have computed an EEG-based workload index that could differentiate between task workload requirements exploring front-parietal brain function. Yet, EEG exploration has achieved traction in aviation and space operations, current studies face challenges related to the intrusive and bulky nature of the equipment (Caldwell et al., 2002), the discomfort of long preparation time, and dependence on gel electrodes (e.g., wet electrodes). In a recent study, Dehais et al. (2019) tested a dry EEG system on pilots during the low and high load traffic patterns associated with the passive auditory oddball. Analysis of ERPs and frequency characteristics has confirmed that dry EEG can be used to study cognition during highly ecological and noisy conditions. Recent developments in dry electrodes (Liao et al., 2012) allow them to reduce preparation time by eliminating the conductive gel or saline patch and preparing the skin to reduce the contact impedance. Some dry electrodes use a ring model with pins to touch the scalp (Hairston et al., 2014), while other dry electrodes use foam-based supplies covered in conductive textile materials. Other promising non-contact electrode techniques enable weak biopotentials to be amplified using a contact-free electrode-skin.
Electro-Oculograghy (EOG)
Origin of EOG Response
The EOG method recognizes the differences in potential changes induced by eye movements between two electrodes placed either horizontally or vertically around the eyes. Early eye movement studies revealed that the human eye is an electric dipole, comprising a positively charged cornea and a negatively charged retina (Anderson, 1937). When the eyeball moves in the direction of the electrode, the electrical potential increases and it decreases when the eye moves in the other direction. The voltage variation between the poles is known as corneoretinal potential and varies between 0.4 and 1 mV. The acquired potential varies based on the viewing angle, up to an angle of 30 degrees (Anderson, 1937). When the eyes do not move and are fixed, the potential does not vary. When the eye moves in the direction of the sensor, the potential greatly increases. Importantly, for people who are blind, or when people have their eyes are closed, the electrical changes remain. There are many configurations for electrode placement. Commonly, the horizontal electrodes are placed at the external borders of the eyes while the vertical ones are positioned above and below the eye (Singh and Singh, 2012). The horizontal EOG signal is the difference of the voltage between horizontal electrodes and the vertical EOG signal is the subtraction of the voltage between the vertical electrodes. Usually, the reference electrode is placed on the forehead (Barea et al., 2002) in the middle of the eyes (Yamagishi et al., 2006) or on the mastoids (Pettersson et al., 2013). Other atypical electrode placements have been extensively tested such as six electrodes positioned above and below both the eyes (Pettersson et al., 2013). Some typical and atypical electrode placements are presented in Figure 1.
Figure 1. Six different electrode positions, used to measure EOG. The blue circles indicate the locations of horizontal and vertical EOG electrodes. The yellow circle (A) Standard placement with 4 electrodes, (B) Frontal placement, (C) One-side two-electrode placement, (D) Two-side symmetrical 4-electrode placement, (E) Nasal two-electrode placement, (F) Near-ear two-electrode placement indicates the reference electrode.
Other studies have used a headband to place electrodes close to the skin and minimize noise, but without measuring the vertical eye movements (Chang, 2019). Adopting this technique, Yan et al. (2013) positioned five electrodes, one horizontal and one vertical around each eye in addition to the reference. Kanoh et al. (2015) placed electrodes near nasion and on both sides of the rhinion, which correspond to the bridge and nose pads of eyewear. Another remarkable placement was proposed by Manabe et al. (2013), who pioneered in-ear EOG signal detection by testing different material- electrode combinations that are suitable for daily use. They highlighted the relation between the in-ear electrical signal and eye movement but did not estimate the accuracy of the eye position. Favre-Felix et al. (2017) proposes a novel fitted ear EOG device that uses a single model taken from the participant's ear canals. Interestingly, they found a strong correlation between conventional EOG and ear EOG signals. Hence, ear EOG can be used to accurately estimate eye gaze in real-time situations, which is particularly relevant for aeronautics and the hearing aid industry. For example, a visually directed hearing aid could be faster and easier to manipulate than a hearing aid piloted by other conventional tools like a pointer or remote control (Hart et al., 2009).
In addition to these techniques, numerous tests have explored the placement options for mobile electrodes that measure changes in EOG voltage. JinsMeme (JINS Inc., Japan) implanted an EOG amplifier into eyeglasses. BioPac (CA, USA) connected a mobile EOG device to a stationary system (MP160) with wireless communication. Some other EOG systems have been connected to a wheelchair (Rajesh, 2014) or fixed into goggles that can be used in everyday environments (Bulling et al., 2009). One study (Acuna et al., 2014) has shown that a low-cost EOG system (<50 euros) can give good results for eye tracking. The performance achieved by this system is different depending on whether one considers vertical or horizontal movements; the monitoring of the latter being much more precise. Thus, an EOG system gives an accuracy of <3° on the horizontal plane (with movements ranging from −50° to +50°), and an accuracy of <4° on the vertical plane (movements ranging from −10° to +10°). For larger vertical movements (from −30° to +30°), the imprecision increases to 11°. Other systems could be used as an alternative way of measuring combined EEG and EOG including Muse (Toronto, Canada), Melomind (Paris, France), Open BCI (Brooklyn, NY, USA), and Emotiv system (San Francisco, USA). They present several advantages over conventional wired, bulky EEG devices in that they are economical, portable, and easy to use. In summary, the main reason for such diversity in electrode placement is due to the specificity of each experimental paradigm and whether other equipment is associated with EOG. Largely, these systems recognize eye movements with accuracy and further investigation of electrode placement and the quality of the EOG signal will be the subject of future research.
EOG Approaches
The main eye movements captured with EOG are fixations and saccades (Singh and Singh, 2012). Fixational movement is an attempt to capture the steady image on the retina. Depending on the quality of the processed information and the current cognitive load, fixation can last between 100 and 1,000 ms, with a mean range of 200–500 ms. The saccade movement changes the eye direction around the field of view and brings the object of interest into the foveal region. Saccades are characterized by a simultaneous and rapid change of the eye position between two fixation points. The duration of saccades is determined by the angular distance the eye travels during the movement. The saccades involve distinct patterns in the EOG signal. They are relatively easy to identify as the deflected amplitudes are above the common high-frequency noise level, and they are short (in duration). When the target is slowly moving, smooth pursuit movement allows us to maintain focus. Smooth pursuit eye movements are typically initiated with a delay of 100–180 ms relative to the onset of an unpredictable motion (Lisberger and Westbrook, 1985). It is difficult to distinguish these movements on EOG recordings because it can be confounded with linear signal drift. Other movements (such as vestibulo-ocular movements and optokinetic nystagmus) allow for the gaze to stabilize during head and body movement. Nystagmus is a rhythmic, often rapid, involuntary eye movement that takes place when the head moves rapidly and the eyes move in response to the moving fluid in the vestibular system (Kang and Shaikh, 2017). Physiological nystagmus occurs during the motions of the head (vestibular nystagmus), or in the presence of patterns in the visual field (optokinetic nystagmus). It is often accompanied by a feeling of disorientation or vertigo and can be a reliable indicator of vestibular pathology. Vergence movements rotate the eyes inwards and outwards, with slow 10°/s disconjugate movements. They allow the visual system to incorporate deep targets, permitting the perception of the world across three dimensions (Alvarez et al., 2005). Even if the eyeballs participate in the contraction of medial and lateral rectal muscles until the paired images are projected onto the fovea, they will activate shared and independent nerve regions. While saccadic and vergence eye movements rely on different muscles to restore the globes, they will activate both shared and distinct neural areas (Semmlow et al., 1998).
Numerous algorithms exist for detecting and modeling oculomotor parameters including eye blink, saccade direction, and fixation. Nolan et al. (2010) developed a fully automated statistical thresholding method that detects and classifies a portion of the signal as an eye blink if the potential of the portion exceeds the threshold. Niemenlehto (2009) analyzed EOG signals by exploring a detection method of a constant false alarm rate that detects saccades. Pettersson et al. (2013) detected eye blinks and saccades by identifying the threshold of the temporal derivative of the EOG signal. Behrens et al. (2010) explored the deviation of the eye-movement acceleration of the EOG signals. Toivanen et al. (2015) computed a real-time algorithm EOG signal that automatically detects blinks, saccades, and fixations and analyzes the temporal features of these reflexes. Although EOG signals have a good signal-to-noise ratio, due to their large amplitude and the relative ease with which they can detect saccades and blinks (Skotte et al., 2007), they are continuously contaminated by physiological artifacts such as electromyography (EMG). EMG artifacts are detected when the participant moves their facial muscles or body during EOG recording, such as jaw clenching, raising an eyebrow, or smiling. The forms and amplitudes of these artifacts depend on the types of movement, the position of the electrode, and sampling rates. High-frequency noise and smooth waveforms are beyond the potential range of eye movement-related signals and are usually differentiated. Median and low-pass filters are the most commonly used techniques to preserve these noises while retaining edge. The filters are based on split windows with varied cut-off frequencies and can be used for either online and/or offline analysis.
Besides EOG, other eye-tracking techniques include photo- and video-oculography, or scleral research lenses (Duchowski, 2017). These techniques allow the capture of eye movements with greater precision than EOG but involve complex set-ups (flight deck video-camera integration, for instance) and involve processing pipelines that demand a large amount of power (computer vision algorithms). Alternatively, EOG is a technique associated with simple electrodes that can be embedded in a headset, providing good insights into the mental state and brain activity of a participant with lower consumption of energy and simpler processing pipelines, which are useful in aeronautics.
EOG in Aeronautics
The assessment of eye movements is particularly relevant for aeronautic (Peysakhovich et al., 2018) and neuro-ergonomic applications (Peysakhovich et al., 2019). Given the EOG's capacity to detect eye movements, it is an excellent candidate to be embedded into aeronautical systems. A number of important studies have related eye movement analysis to fatigue, mental workload, and cognitive performance in pilots. One of the most important causes of aviation accidents is provoked by human errors e.g., drowsiness or fatigue (Velazquez, 2018). Pilot fatigue and loss of control are considered by the Federal Aviation Administration as one of four common aviation hazards (Federal Aviation Administration, “Risk Management Handbook: U.S. Department of Transportation,” 2016). For example, a study described how two pilots from a commercial aircraft missed the target airport because both were sleeping (Borghini et al., 2014). Thus, EOG is considered to be an effective and predictive tool in detecting drowsiness markers such as a reduction in performance, and changes in the frequency of eye blinking (Oken et al., 2006). Interestingly, eye movement analysis can serve as a reference for the mental workload and state of pilots (Di Nocera et al., 2007). It has been shown that the workload reflected by eye fixation distribution varies according to the phases of the flight. The highest workload was noted during takeoff and landing, and the lowest during the cruise phase (Di Nocera et al., 2007). Brams et al. (2018) discuss how the gaze behavior of expert pilots differs from that of novices, suggesting relevant information about basic processes that explain the successful performance of expert pilots in flight. The authors explain that the expert pilots have expanded visual range that analyzes the global scene using the field of view next to the fovea, and that they then shift their attention to the pertinent area (Gegenfurtner et al., 2011). Experts have larger saccades that cover more areas and spend less time focusing on task-related regions. In addition to the important capacity to process information from multiple sources, this ability is also related to advanced cognitive performances. The gaze behavior of fighter pilots during flight conditions also varies according to altitude and speed (Svensson et al., 1997). An augmented workload can induce a lower percentage of eye fixation distribution outside and a higher percentage of fixation on the tactical display.
In addition to investigating the eye movement patterns underlying cognitive functions, alternative EOG (around the ear) may be a promising tool in real aeronautic environments. In numerous activities such as air traffic control or piloting an aircraft, the operators are equipped with peripherals (typically headsets). As the EOG requires only a few electrodes, it does not obstruct the visual field nor does it unnecessarily illuminate the eyes with infrared light. Therefore, this technique is convenient for head-mounted peripherals such as the audio headset used by pilots. Moreover, exploring EOG integration in control and communication peripherals may enhance human-system interaction and make psycho-physiological monitoring possible (e.g., based on blink rate or saccades). Such an approach could have numerous applications in aeronautics (fighters, helicopters, and unmanned aerial vehicle operation), naval systems, and control-command centers.
Motivations and Contributions
Although a large number of surveys are published for EEG and EOG in several fields, to the best of our knowledge, we have not identified any exhaustive review that summarizes a taxonomy of combined EEG-EOG studies. To bridge this gap, this work analyzes and categorizes developments from the last decade, examining published literature on combined EEG-EOG in different applications related to aeronautics.
This non-exhaustive review analyzes literature from the oldest to the most recent relevant studies. Section Introduction presents EEG and EOG signals, their methodological approaches, and a summary of their application in aeronautics. This section could be particularly beneficial for beginners in the field of electrophysiology.
The review presents a novel taxonomy model for classifying different approaches to EEG and EOG by emphasizing the extracted features, the applied pre-processing treatment, and the performance metrics. The studies belonging to each main category are sub-categorized as per the corresponding domain of application. Sections Methodology of the Review and Results are particularly beneficial for expert researchers in the field. Sections Discussion and Limitations and Conclusions Outline Insights and research directions as a means of providing guidelines for beginner and expert researchers who are interested in combined EEG-EOG studies in the future.
This review is useful for both beginners and experts in this field. It is intended to be a time- and resource- saving guide for those searching for exploration of neuro-physiological correlates in aeronautics. On the one hand, this review is beneficial for beginners who may easily explore specific literature on EEG and EOG from basic approaches to aeronautics applications in a single document and investigate it by spending less effort. On the other hand, it is useful for expert researchers who may explore the literature to discover trends and methodologies for exploring brain and eye signals. Experts may explore these approaches as valuable tools and could be useful in building and analyzing experimental paradigms.
Methodology of the Review
Peer-reviewed journal articles in the English language and conference papers published on PubMed for the decade from 2010 to 2020 (January 2010-March 2020) were identified as targets for our review. The database was last queried on March 23, 2020. Search items related only to studies combining EEG and EOG methods in experiments. This filtered selection resulted in 249 studies that were then included or excluded according to subsequent criteria. The inclusion criteria were: (i) combined EEG-EOG studies; (ii) original research papers; and, (iii) that the experiments involved human subjects. The exclusion criteria were: (i) review, methodology, and proof-of-concept papers; (ii) dataset publications; (iii) animal model experiments; and, (iv) studies focusing solely on EEG or EOG.
Article titles were examined to evaluate the relevance of a selected article. If the title did not noticeably specify whether it corresponds to the inclusion and exclusion criteria, the abstract was then considered. Lastly, during the full paper examination and data gathering process, an article that did not meet the criteria was excluded. Non-peer reviewed papers, such as arXiv or BiorXiv electronic preprints were considered as a possible source of bias. Thus, preprints that had not been peer-reviewed were not selected. Finally, 150 studies were included in our final database. We constructed a data extraction table containing several data items related to our investigation question, according to previous comments with a similar scope and the author's past work in the field.
A taxonomy table was used to classify and describe the included items for each selected study. The first section is the justification category that emphasizes the domains of application of the included studies (e.g., sleep, BCI, signal processing, cognition, and driving). This valuable information allows an understanding of the scope of the research and also enabled us to determine trends in the analysis. The second section shows the bibliographic reference attributed to each study. The third section shows the name of the first author, giving a specific identification for each paper in the database. The fourth section outlines the year of publication, which is relevant for our selection criteria and statistical analysis. The fifth section regroups the number of subjects in each study to give a quick overview of the coherence of the relative main findings. The sixth section covers the publication category of the article, such as whether it was a journal article or a conference publication. These first six sections indicate the types of included papers and the main selected items. The seventh section includes all relevant information about the EEG and EOG data. This comprises the category of EEG and EOG equipment used, in addition to the different extracted features.
Standard EEG and standard EOG sections refer to classic wet electrode gels that are not mobile, while non-standard refers to dry wearable electrodes and any other equipment that is different from the standard category. Features Classification (Table 1) refers to the properties of the analyzed EEG and EOG signal. The content of this dataset includes the support of the statistical analysis relevant to the critical component of our discussion. The EEG features an extraction section and covers the signal processing methodology that was analyzed in each study, including the waveband frequency Delta (2–4 Hz), Theta (4–8 Hz), Alpha (9–13 Hz), Beta (14–32 Hz), Gamma (over 33 Hz), the ERP, or the basic raw signal. The EOG feature extractions consist of the explored signal processing for eye movements including blinks, saccades, fixations, raw signal, or whether the EOG was used as an artifact handling methodology. Polysomnography is a multi-parameter test based on several different types of physiological signals called polysomnograms used in sleep diagnosis. Here, polysomnography was included in the taxonomy as an extracted feature from sleeping studies that analyzed both EEG and EOG signals.
Table 1. Taxonomy table.
Results
A total of 150 papers were selected for inclusion in this review. Our search methodology returned 121 journal papers and 29 conference papers that met our criteria. We noted that the article papers were published in different journals; however, all the conference papers were only published in numerous years of the International Conferences of the Institute of Electrical and Electronics Engineers (IEEE). The included papers have combined EEG-EOG in various domains of application (see Figure 2). Most studies focused notably on sleep, signal processing, and BCI categories, respectively 34, 33, and 21% of the total selected studies. Sleep category concerns EEG and EOG data for sleep classification in healthy and clinic patients as well as using deep learning for classification of sleep stages and disorders. The signal processing category regroups the development of tools, such as analyzing, modifying, and synthesizing signals, handling artifacts, learning features, and training models. BCI category groups the manipulation of hybrid EEG-EOG systems as a valuable communication tool for monitoring computers, wheelchairs, or a robotic exoskeleton. Seven percent of the selected studies belong to the cognition category. They were particularly related to analyzing the performance of attention and cognition. The remaining papers (5%) explored different analyses of driving with conditions such as sleep deprivation or fatigue.
Figure 2. Focus of the studies, gathering papers according to their domain of application. N, Number of studies that fit in a category according to the central focus.
Figure 3 indicates the evolution of EEG-EOG investigations since 2010 in each domain of application. We did not observe any clear tendency apart from a growing concern for BCI. The first 3 months of 2020 associated with the years 2019 and 2018 alone account for 42% of the total selected publications. Nevertheless, given the relatively small number of publications to date, it is too early to draw conclusions about trends.
Figure 3. The number of publications for each domain of application per year.
The number of subjects included in each study varies expressively across the different domains of application (see Figure 4). Seventy-five percent of the included datasets contained fewer than 30 participants. Some studies have datasets with a higher number of participants with at least 600 subjects (Singh et al., 2014; Olsen et al., 2017; Korkalainen et al., 2019), while others included studies used datasets with <5 subjects, particularly in BCI and signal processing categories.
Figure 4. The number of studies and subjects per domain of application. Each bar represents the number of studies and the number of subjects for each domain in database of this review.
When reviewing our included studies, we regrouped some of the common pre-processing steps employed (Figure 5A). The pre-processing methodology presented some routine steps, such as bandpass filtering, downsampling, windowing, interpolating the bad channels, computing the average reference, or removing line noise. Eighty-seven percent of the selected studies used band-pass filters and notch filters that allow the extraction of characteristic signals located in the stimulus frequency and that remove noise and artifacts. Forty-seven percent of studies describe artifact handling, which consisted of eliminating certain types of noise, such as eyes and muscle artifacts. Seventeen percent of the studies downsampled the signal acquired at a higher sampling rate to 256 Hz or less. Even when deleting the noise might be essential to achieving relevant EEG decoding efficiency, 9% of the included studies did not explicitly mention pre-processing steps. The distribution of the selected pre-processing steps according to each domain of application is shown in Figure 6A. In particular, the filtering process was employed by studies in BCI (n = 26), sleep (n = 43), and signal processing (n = 45). Artifact handling was mostly used in signal processing (n = 34) and sleep (n = 18) domains. Downsampling was used in BCI, sleep (n = 7), and signal processing (n = 7) categories.
Figure 5. Percentage of the methodology employed in the selected EEG-EOG studies. (A) Pre-processing procedures, (B) identified performance metrics, (C) EEG extracted features, and (D) EOG extracted features.
Figure 6. Methodology applied choices in each domain of application. (A) Pre-processing methodology, (B) performance metrics, (C) EEG extracted features, (D) EOG extracted features.
When assessing the performance of signal processing in the selected studies (Figure 5B), we were not surprised to find that most of them applied common metrics that are derived from confusion matrices. We classified the performance metrics into the component that emerged from the included studies: accuracy estimation (54%), sensitivity and specificity values (23%), false positive and/or false negative detection (22%), classification score (F1-score, Kappa score or Support Vector Machine score) (18%), an approximation of error correction (15%), and other non-common metrics (31%).
The distribution of the selected performance metrics according to each domain of application is shown in Figure 6B. As expected, we noted an important evaluation of accuracy in the BCI (n = 23), signal processing (n = 21), and sleep (n = 33) domains.
Figure 5C details the performed analysis on the registered EEG signals from the included studies. The evaluation revealed the following extracted features: raw signal (40%), alpha frequency (27%), polysomnography (20%), beta frequency (18%), theta frequency (17%), ERPs (17%), gamma frequency (10%), and delta frequency (6%).
The distribution of the EEG extracted features according to each domain of application is shown in Figure 6C. Particularly, alpha, beta, and theta frequencies were investigated in sleep and signal processing domains. ERPs were analyzed in BCI (n = 10), cognition (n = 5), and signal processing domains (n = 11). The raw signal was used in the majority of the domains BCI (n = 16), signal processing (n = 29), and sleep (n = 12). As expected, polysomnography was exanimated principally in sleep studies (n = 29).
The classification of the extracted features from the registered EOG signals showed the following results: blinks (33%), raw signal (30%), artifact removal (27%), polysomnography (20%), saccades (7%) and fixation (3%) (Figure 5D). The analysis of blinks movements was particularly present in BCI (n = 17) and algorithms (n = 21) studies. The artifact removal was majority used in algorithms (n = 24) and equally used in BCI and sleep (n = 6). The raw signal was analyzed in BCI, (n = 11), algorithms (n = 16) and sleep (n = 13).
Discussion
The objective of this work is to present a comprehensive summarized review of combined EEG-EOG studies and a taxonomy model of the last decade. Our taxonomy model presents an overview of classification by domains of application: sleep, BCI, signal processing, cognition, and driving. It is worth mentioning that these identified categories are significantly relevant to research in the exploration of human factors in the aeronautical industry and neural engineering. Figures 2–5 present the distribution of combined EEG-EOG papers of the taxonomy according to domains of application, while Figure 6 illustrates the different methodological approaches. The motivation behind this is to derive trends about how each feature type and domain of application have been investigated. This may assist researchers in exploring human factors and the identification of feature types in future combined EEG-EOG work. Here, we discuss the most relevant outcomes from our results section and review the involvement of the different findings highlighted above in aeronautics. We also provide recommendations for combined EEG-EOG studies to facilitate extension in the field. Finally, we present some limitations in our work and future perspectives.
Sleep and fatigue prediction are the main challenges explored in investigations on human factors. In this context, one of the outcomes of this review was the identification of EEG and EOG features that may help to detect unintentional sleep in airline pilots. Our results highlight relevant features that help to study aspects related to the sleep of pilots, aiming to improve safety conditions. For example, typical markers of sleep are the disappearance of the alpha rhythm and the appearance of roving eye movements. Fatigue, irregular, and long working hours, and time zone crossings can change sleep-wake cycles, alert levels, and affect pilot decisions during a flight. These issues cause excessive sleepiness, unintentional sleep, and increase the risk of accidents. Some investigations with airline pilots observed that this working environment typically exposes several sleep and health issues, particularly extreme sleepiness, unintentional naps, and fatigue (Petrie and Dawson, 1997; Wright and McGown, 2001). However, such studies are still incipient, and knowing the characteristics of sleep prediction is necessary in supporting safety policies and working practices. The tasks performed by pilots are complex and involve several functions, including cognitive, technical, and relationship skills. They involve concentration, the ability to work under pressure, adaptation to operational modifications, teamwork, the prediction of the consequences, the interpretation of signals, and quick decisions (Itani, 2009). When excessive sleepiness occurs, such skills may be impaired, thereby affecting flight safety. Ingre et al. (2014) have observed that unintentional sleep in pilots may compromise the safety of flights. Therefore, the ability to observe and predict fatigue provides a significant benefit in avoiding incidents and accidents.
Fatigue and sleepiness decrease task-related activity in the frontal and parietal regions and also reduce activity in and connectivity with, the extrastriate visual cortex during tasks that require visuospatial attention (Chee, 2010). These neural changes affect the behavior of pilots, for example, they might miss specific visual or auditory stimulus. This disturbance alternates the top-down allocation of attentional resources, such as the attentional orientation of a target. Sleep disorders during flights impair sustained attention because of the decreased activity of the dorsolateral prefrontal cortex and parietal sulcus (Kong and Soon, 2012). According to these neural alterations, our review suggests that combined EEG-EOG analysis may provide useful information in evaluating sleep and sleep-related disorders in pilots during flights. Future research could focus on improving EOG models and conduct a comprehensive evaluation of the content of various EOG results, studying the proportion of slow eye movement, rapid eye movements, and no eye movements, the time distribution, and mixtures of slow/fast/no eye movements.
Figure 3 revealed the distribution of combined EEG-EOG works each year since 2010. Although the absolute number of combined EEG-EOG studies is relatively smaller compared to other single EEG (Bonanni et al., 2016) or eye movement applications (Hodgson et al., 2019), there is an important amount of interest in combined EEG-EOG studies. Unexpectedly, we were unable to draw conclusions about trends in the number of studies. We only observed that the first 3 months of 2020 associated with the years 2019 and 2018 alone account for 42% of total BCI publications. Due to the relatively small number of publications to date, it is too early to make assumptions about trends, but a possible explanation for this could be that BCI technology is rapidly gaining attention from scientists, engineers, clinicians, and the general public. The initial applications of BCI aimed to rehabilitate patients with neuromuscular diseases such as amyotrophic lateral sclerosis and injuries to the spinal cord. Here, we observed various patterns in BCI techniques, including EEG based spelling system (Lee et al., 2018), control of cursors (He et al., 2020), robotic arms (Zhang et al., 2019b), prostheses (Soekadar et al., 2015), wheelchairs (Huang et al., 2019), and other complex devices.
In aeronautics, passive BCIs have been successfully used to detect and characterize several operator mental states such as workload and fatigue (Zander et al., 2010; Khan and Hong, 2015; Roy and Frey, 2016). Functional Near-InfraRed Spectroscopy (fNIRS) connectivity based on passive BCI metrics has been explored to detect a pilot's engagement when undertaking automated and manual landing scenarios (Verdière et al., 2018). Analysis confirmed that these two situations contrast, as manual landing led to significantly higher subjective NASA-TLX scores than automated landing. Designing a system capable of measuring continuous monitoring based on eyes and brain signal features or detecting an operator's degraded states would enhance both safety and performance. The achievements of BCI tend to be that it improves aeronautic performance in time-critical situations by decoding an operator's neural activity associated with the act. As brain activity precedes motor performance (Belkhiria et al., 2019), the decoded output in real time could ameliorate the operator's action. Our review showed that some EEG-EOG studies used online BCI to detect error-related potentials and reduce error-rate, improving overall performance. While these methods are promising, they have not been tested in aeronautics. An interesting study (Callan et al., 2016) used magnetoencephalography (MEG) and BCI to explore neuroadaptive automation to reduce a pilot's time response to a hazardous event by decoding their perceptual-motor intentions. The BCI system succeeded in decoding motor intention faster than manual control in response to a change in attitude while ignoring ongoing motor and visual induced activity related to piloting the airplane. While this system used 400 channels, the authors expressed the possibility of making a dedicated real-time system working with mobile EEG. Such signal processing handling methods could separate artifacts from brain-related activity in flight even in an open cockpit biplane (Callan et al., 2016). In the future, research in combined EEG-EOG systems could use a BCI-decoder to distinguish between brain activity that responds to changes in the visual field and motor intention in a flight simulator or even in real aircraft.
Our analysis shows that the number of subjects included in each study varies across the different domains of application. It should also be highlighted that database availability is different from one domain to another. We noted that the most important datasets come from clinical investigations of conditions such as epilepsy, Parkinson's Disease, and sleep disorders. However, in other fields with more exploratory goals, studies rely on data registered in laboratory settings with a limited number of participants (e.g., 30 participants). Some studies explained that the reduced number of subjects is due to experimental conditions, time, and device limitations (Al-Hudhud et al., 2019). Further studies should use more strict inclusion and exclusion criteria for their datasets. This may generate a more robust statistical result.
Figures 5, 6 show methodologies of data pre-processing, various extracted features from EEG and EOG signals, and performance metrics. The literature shows that performance metrics, including accuracies classification, determine computer-based analysis in various applications (Acharya et al., 2011). The objective is to construct algorithms that overcome recognized methods (Faust et al., 2010). Here, most of the selected studies used an offline or an online system. Commonly, the system involves three consecutive processing steps: (i) pre-processing (Kalayci and Özdamar, 1995; Rao and Derakhshani, 2005); (ii) feature extraction (Tyagi and Nehra, 2018); and (iii) classification (Wang et al., 2014b). All of the selected studies explored at least one parameter from the processing pipeline steps (Table 1). Recent advances in signal processing analysis provide a powerful tool for modeling complex probability distributions by automatically discovering intermediate abstractions from a huge amount of basic features. Deep machine learning and artificial intelligence have shown great promise in helping make sense of EEG signals due to their capacity to learn good feature representations from raw data (Dehais et al., 2018). In that sense, the number of aeronautics publications applying these techniques to EEG signal processing has seen an exponential increase in recent years. The increasing interest in methodologies for processing EEG-EOG signal data (e.g., deep learning, machine learning, and artificial intelligence) in relation to human factors noticeably reflects an emergent interest in these kinds of approaches. Research in aeronautic sensors and signal processing systems (e.g., EEG and EOG) is exploring revolutionary improvements, potentially enhancing civil and military applications in fighter planes, helicopters, and the remote operation of drones. Yet another trend in sensor and signal processing involves blending artificial intelligence and machine learning into system designs and the incorporation of electrophysiological sensors and associated signal processing methods into equipment, such as aviation headsets, may enhance operational safety (Wilson et al., 2020).
Limitations and Conclusions
The review presented state-of-the-art research on the characteristics and applications of both EEG and EOG signals in aeronautics. Our taxonomy and statistical analysis described a number of operational mechanisms, such as extracted features, pre-processing treatment, and performance metrics. We discussed how these methodologies could be adopted by researchers examining human factors and aeronautics. The effectiveness of combining EEG and EOG as a psycho-physiological tool is unequivocal. However, numerous challenges still need to be resolved. An exciting and unprecedented approach would be the assessment of both eye movement and brain activity during novel paradigms using dry electrodes that are integrated into existing control and communication peripherals. An equipped headset measuring real-time EEG and EOG may pose a great challenge in terms of applicability and generalizability to both commercial and scientific research into mobile EEG devices. This method would improve human-system interaction by making it possible to follow up on data about eye movements and use it to determine the psycho-physiological state of a person. Studies combining both EEG and EOG technologies and a review of the application of these in several fields, including in laboratory and real-world situations, are of particular value, as these technologies could be of interest in areas using both visual and auditory information simultaneously through headphones and gaze inputs, including in aeronautics, helicopters, teleoperation drones, naval systems, and control-command centers.
Despite the clear value of data collected and presented here, this review has some limitations. First, despite the use of a well-founded methodology to identify pertinent studies on the topic; the review did not cover all existing papers. It is also important to stress the error risk that could occur as a result of the inclusion methodology. Even though we examined 150 studies among a wider pool of 255 from Pubmed research, this is an exhaustive list nor should we undervalue the relevance of the studies that were not included. Second, in managing the length of the review, we limited our analysis to the main domains of applications. However, some studies could involve two or more application domains. Some topics could overlap, such as BCI and signal processing (Daly et al., 2015; Ivorra et al., 2018). We did consider categorizing another field of application related to clinical applications. However, this would have moved the focus away from aeronautics. Finally, as with any literature review, new articles are being published and new trends are being established and future studies should eventually be added to the analysis.
Author Contributions
CB: conceptualization, methodology, data curation, and writing-original draft preparation. VP: conceptualization, methodology, writing, reviewing, and editing. Both authors contributed to the article and approved the submitted version.
Funding
This work was funded by the ANR ASTRID program (Grant No. ANR-18-ASTR-0026—ELOCANS project).
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
Åkerstedt, T., Hallvig, D., Anund, A., Fors, C., Schwarz, J., and Kecklund, G. (2013). Having to stop driving at night because of dangerous sleepiness - awareness, physiology and behaviour. J. Sleep Res. 22, 380–388. doi: 10.1111/jsr.12042
Acharya, U. R., Sree, S. V., Chattopadhyay, S., Yu, W., and Ang, P. C. A. (2011). Application of recurrence quantification analysis for the automated identification of epileptic EEG signals. Int. J. Neural Syst. 21, 199–211. doi: 10.1142/S0129065711002808
Acuna, O. V., Aqueveque, P., and Pino, E. J. (2014). “Eye-tracking capabilities of low-cost EOG system,” in 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (Chicago, IL: IEEE), 610–613. doi: 10.1109/EMBC.2014.6943665
Ahn, S., Nguyen, T., Jang, H., Kim, J. G., and Jun, S. C. (2016). Exploring neuro-physiological correlates of drivers' mental fatigue caused by sleep deprivation using simultaneous EEG, ECG, and fNIRS data. Front. Hum. Neurosci. 10:219. doi: 10.3389/fnhum.2016.00219
Al-Hudhud, G., Alqahtani, L., Albaity, H., Alsaeed, D., and Al-Turaiki, I. (2019). Analyzing passive BCI signals to control adaptive automation devices. Sensors 19:3042. doi: 10.3390/s19143042
Alvarez, T. L., Semmlow, J. L., and Pedrono, C. (2005). Divergence eye movements are dependent on initial stimulus position. Vision Res. 45, 1847–1855. doi: 10.1016/j.visres.2005.01.017
Anderson, I. H. (1937). Studies in the eye movements of good and poor readers. Psychol. Monogr. 48, 1–35. doi: 10.1037/h0093391
Andreotti, F., Phan, H., Cooray, N., Lo, C., Hu, M. T. M., and De Vos, M. (2018). “Multichannel sleep stage classification and transfer learning using convolutional neural networks,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (Honolulu, HI: Institute of Electrical and Electronics Engineers Inc.), 171–174. doi: 10.1109/EMBC.2018.8512214
Arnin, J., Anopas, D., Horapong, M., Triponyuwasi, P., Yamsa-Ard, T., Iampetch, S., et al. (2013). “Wireless-based portable EEG-EOG monitoring for real time drowsiness detection,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (Osaka: Conf Proc IEEE Eng Med Biol Soc), 4977–4980. doi: 10.1109/EMBC.2013.6610665
Arnin, J., Kahani, D., Lakany, H., and Conway, B. A. (2018). “Evaluation of different signal processing methods in time and frequency domain for brain-computer interface applications,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (Honolulu, HI: Institute of Electrical and Electronics Engineers Inc.), 235–238. doi: 10.1109/EMBC.2018.8512193
Arslan Tuncer, S., and Kaya, T. (2018). True random number generation from bioelectrical and physical signals. Comput. Math. Methods Med. 2018:3579275. doi: 10.1155/2018/3579275
Assi, E. B., Rihana, S., and Sawan, M. (2014). “Kmeans-ICA based automatic method for ocular artifacts removal in a motorimagery classification,” in 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2014 (Chicago, IL: Institute of Electrical and Electronics Engineers Inc.), 6655–6658.
Aziz, F., Arof, H., Mokhtar, N., and Mubin, M. (2014). HMM based automated wheelchair navigation using EOG traces in EEG. J. Neural Eng. 11:056018. doi: 10.1088/1741-2560/11/5/056018
Babiloni, C., Barry, R. J., Başar, E., Blinowska, K. J., Cichocki, A., Drinkenburg, W. H. I. M., et al. (2020). International federation of clinical neurophysiology (IFCN) – EEG research workgroup: recommendations on frequency and topographic analysis of resting state EEG rhythms. Part 1: applications in clinical research studies. Clin. Neurophysiol. 131, 285–307. doi: 10.1016/j.clinph.2019.06.234
Babiloni, C., Vecchio, F., Infarinato, F., Buffo, P., Marzano, N., Spada, D., et al. (2011). Simultaneous recording of electroencephalographic data in musicians playing in ensemble. Cortex 47, 1082–1090. doi: 10.1016/j.cortex.2011.05.006
Badesa, F. J., Diez, J. A., Catalan, J. M., Trigili, E., Cordella, F., Nann, M., et al. (2019). Physiological responses during hybrid BNCI control of an upper-limb exoskeleton. Sensors 19:4931. doi: 10.3390/s19224931
Bai, Y., Wan, X., Zeng, K., Ni, Y., Qiu, L., and Li, X. (2016). Reduction hybrid artifacts of EMG-EOG in electroencephalography evoked by prefrontal transcranial magnetic stimulation. J. Neural Eng. 13:066016. doi: 10.1088/1741-2560/13/6/066016
Barea, R., Boquete, L., Mazo, M., and López, E. (2002). Wheelchair guidance strategies using EOG. J. Intell. Robot. Syst. Theory Appl. 34, 279–299. doi: 10.1023/A:1016359503796
Barry, R. J., and De Blasio, F. M. (2017). EEG differences between eyes-closed and eyes-open resting remain in healthy ageing. Biol. Psychol. 129, 293–304. doi: 10.1016/j.biopsycho.2017.09.010
Barry, R. J., De Blasio, F. M., Bernat, E. M., and Steiner, G. Z. (2015). Event-related EEG time-frequency PCA and the orienting reflex to auditory stimuli. Psychophysiology 52, 555–561. doi: 10.1111/psyp.12376
Barthélemy, Q., Mayaud, L., Renard, Y., Kim, D., Kang, S. W., Gunkelman, J., et al. (2017). Online denoising of eye-blinks in electroencephalography. Neurophysiol. Clin. 47, 371–391. doi: 10.1016/j.neucli.2017.10.059
Behrens, F., MacKeben, M., and Schröder-Preikschat, W. (2010). An improved algorithm for automatic detection of saccades in eye movement data and for calculating saccade parameters. Behav. Res. Methods 42, 701–708. doi: 10.3758/BRM.42.3.701
Belkhiria, C., Mssedi, E., Habas, C., Driss, T., and de Marco, G. (2019). Collaboration of cerebello-rubral and cerebello-striatal loops in a motor preparation task. Cerebellum 18, 203–211. doi: 10.1007/s12311-018-0980-z
Bernhardt, K. A., Poltavski, D., Petros, T., Ferraro, F. R., Jorgenson, T., Carlson, C., et al. (2019). The effects of dynamic workload and experience on commercially available EEG cognitive state metrics in a high-fidelity air traffic control environment. Appl. Ergon. 77, 83–91. doi: 10.1016/j.apergo.2019.01.008
Binnie, C. D., Dekker, E., Smit, A., and Van Der Linden, G. (1982). Practical considerations in the positioning of EEG electrodes. Electroencephalogr. Clin. Neurophysiol. 53, 453–458. doi: 10.1016/0013-4694(82)90010-4
Bizopoulos, P. A., Al-Ani, T., Tsalikakis, D. G., Tzallas, A. T., Koutsouris, D. D., and Fotiadis, D. I. (2013). “An automatic electroencephalography blinking artefact detection and removal method based on template matching and ensemble empirical mode decomposition,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS) (Osaka), 5853–5856. doi: 10.1109/EMBC.2013.6610883
Bleichner, M. G., and Debener, S. (2017). Concealed, unobtrusive ear-centered EEG acquisition: cEEGrids for transparent EEG. Front. Hum. Neurosci. 11:163. doi: 10.3389/fnhum.2017.00163
Bonanni, L., Franciotti, R., Nobili, F., Kramberger, M. G., Taylor, J. P., Garcia-Ptacek, S., et al. (2016). EEG markers of dementia with lewy bodies: a multicenter cohort study. J. Alzheimer's Dis. 54, 1649–1657. doi: 10.3233/JAD-160435
Borghini, G., Aricò, P., Flumeri, G., Cartocci, G., Colosimo, A., Bonelli, S., et al. (2017). EEG-based cognitive control behaviour assessment: an ecological study with professional air traffic controllers. Sci. Rep. 7:547. doi: 10.1038/s41598-017-00633-7
Borghini, G., Astolfi, L., Vecchiato, G., Mattia, D., and Babiloni, F. (2014). Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neurosci. Biobehav. Rev. 44, 58–75. doi: 10.1016/j.neubiorev.2012.10.003
Brams, S., Hooge, I. T. C., Ziv, G., Dauwe, S., Evens, K., De Wolf, T., et al. (2018). Does effective gaze behavior lead to enhanced performance in a complex error-detection cockpit task? PLoS ONE 13:e0207439. doi: 10.1371/journal.pone.0207439
Breitwieser, C., Neuper, C., and Müller-Putz, G. R. (2011). “A concept to standardize raw biosignal transmission for brain-computer interfaces,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (Boston, MA: Conf Proc IEEE Eng Med Biol Soc), 6377–6380. doi: 10.1109/IEMBS.2011.6091574
Brown, L., Van De Molengraft, J., Yazicioglu, R. F., Torfs, T., Penders, J., and Van Hoof, C. (2010). “A low-power, wireless, 8-channel EEG monitoring headset,” in 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC'10 (Buenos Aires), 4197–4200. doi: 10.1109/IEMBS.2010.5627393
Bulling, A., Roggen, D., and Tröster, G. (2009). Wearable EOG goggles: seamless sensing and context-awareness in everyday environments. J. Ambient Intell. Smart Environ. 1, 157–171. doi: 10.3233/AIS-2009-0020
Caldwell, J. A., Hall, K. K., and Erickson, B. S. (2002). EEG data collected from helicopter pilots in flight are sufficiently sensitive to detect increased fatigue from sleep deprivation. Int. J. Aviat. Psychol. 12, 19–32. doi: 10.1207/S15327108IJAP1201_3
Callan, D. E., Durantin, G., and Terzibas, C. (2015). Classification of single-trial auditory events using dry-wireless EEG during real and motion simulated flight. Front. Syst. Neurosci. 9:11. doi: 10.3389/fnsys.2015.00011
Callan, D. E., Terzibas, C., Cassel, D. B., Sato, M. A., and Parasuraman, R. (2016). The brain is faster than the hand in split-second intentions to respond to an impending hazard: a simulation of neuroadaptive automation to speed recovery to perturbation in flight attitude. Front. Hum. Neurosci. 10:187. doi: 10.3389/fnhum.2016.00187
Camfferman, D., Kennedy, J. D., Gold, M., Simpson, C., and Lushington, K. (2013). Sleep and neurocognitive functioning in children with eczema. Int. J. Psychophysiol. 89, 265–272. doi: 10.1016/j.ijpsycho.2013.01.006
Cannon, J., Krokhmal, P. A., Chen, Y., and Murphey, R. (2012). Detection of temporal changes in psychophysiological data using statistical process control methods. Comput. Methods Programs Biomed. 107, 367–381. doi: 10.1016/j.cmpb.2011.01.003
Carl, C., Açik, A., König, P., Engel, A. K., and Hipp, J. F. (2012). The saccadic spike artifact in MEG. Neuroimage 59, 1657–1667. doi: 10.1016/j.neuroimage.2011.09.020
Carl, W., Sem-Jacobsen, M. D., Nilseng, O., Patten, C., and Eriksen, O. (1959). Electroencephalographic recording in simulated combat flight in a jet fighter plane. The pilot's level of consciousness. Electroencephalogr. Clin. Neurophysiol. 11, 154–155. doi: 10.1016/0013-4694(59)90018-5
Casson, A. J., and Rodriguez-Villegas, E. (2012). “Signal agnostic compressive sensing for body area networks: comparison of signal reconstructions,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (San Diego, CA: Conf Proc IEEE Eng Med Biol Soc), 4497–4500. doi: 10.1109/EMBC.2012.6346966
Chan, H. L., Tsai, Y. T., Meng, L. F., and Wu, T. (2010). The removal of ocular artifacts from EEG signals using adaptive filters based on ocular source components. Ann. Biomed. Eng. 38, 3489–3499. doi: 10.1007/s10439-010-0087-2
Chang, W.-D. (2019). Electrooculograms for human–computer interaction: a review. Sensors 19:2690. doi: 10.3390/s19122690
Chang, W. D., Cha, H. S., and Im, C. H. (2016). Removing the interdependency between horizontal and vertical eye-movement components in electrooculograms. Sensors 16:227. doi: 10.3390/s16020227
Charbonnier, S., Zoubek, L., Lesecq, S., and Chapotot, F. (2011). Self-evaluated automatic classifier as a decision-support tool for sleep/wake staging. Comput. Biol. Med. 41, 380–389. doi: 10.1016/j.compbiomed.2011.04.001
Chee, M. T. J. (2010). Lapsing when sleep deprived: neural activation characteristics of resistant and vulnerable individuals. Neuroimage 51, 835–843. doi: 10.1016/j.neuroimage.2010.02.031
Chen, Y.-H., de Beeck, M., Vanderheyden, L., Carrette, E., Mihajlović, V., Vanstreels, K., et al. (2014). Soft, comfortable polymer dry electrodes for high quality ECG and EEG recording. Sensors 14, 23758–23780. doi: 10.3390/s141223758
Cho, H., Kwon, M., Jang, H., Lee, J. B., Yoon, K. C., and Jun, S. C. (2016). Herbal extracts that reduce ocular oxidative stress may enhance attentive performance in humans. Comput. Intell. Neurosci. 2016:4292145. doi: 10.1155/2016/4292145
Christensen, J. A. E., Frandsen, R., Kempfner, J., Arvastson, L., Christensen, S. R., Jennum, P., et al. (2012). “Separation of Parkinson's patients in early and mature stages from control subjects using one EOG channel,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (San Diego, CA: Conf Proc IEEE Eng Med Biol Soc), 2941–2944. doi: 10.1109/EMBC.2012.6346580
Christensen, J. A. E., Zoetmulder, M., Koch, H., Frandsen, R., Arvastson, L., Christensen, S. R., et al. (2014). Data-driven modeling of sleep EEG and EOG reveals characteristics indicative of pre-parkinson's and parkinson's disease. J. Neurosci. Methods 235, 262–276. doi: 10.1016/j.jneumeth.2014.07.014
Cooray, N., Andreotti, F., Lo, C., Symmonds, M., Hu, M. T. M., and De Vos, M. (2019). Detection of REM sleep behaviour disorder by automated polysomnography analysis. Clin. Neurophysiol. 130, 505–514. doi: 10.1016/j.clinph.2019.01.011
Crea, S., Nann, M., Trigili, E., Cordella, F., Baldoni, A., Badesa, F. J., et al. (2018). Feasibility and safety of shared EEG/EOG and vision-guided autonomous whole-arm exoskeleton control to perform activities of daily living. Sci. Rep. 8:10823. doi: 10.1038/s41598-018-29091-5
Dahlstrom, N., and Nahlinder, S. (2009). Mental workload in aircraft and simulator during basic civil aviation training. Int. J. Aviat. Psychol. 19, 309–325. doi: 10.1080/10508410903187547
Daly, I., Scherer, R., Billinger, M., and Müller-Putz, G. (2015). FORCe: fully online and automated artifact removal for brain-computer interfacing. IEEE Trans. Neural Syst. Rehabil. Eng. 23, 725–736. doi: 10.1109/TNSRE.2014.2346621
De Vico Fallani, F., Toppi, J., Di Lanzo, C., Vecchiato, G., Astolfi, L., Borghini, G., et al. (2012). “Redundancy in functional brain connectivity from eeg recordings,” in International Journal of Bifurcation and Chaos (World Scientific Publishing Co Pte Ltd). doi: 10.1142/S0218127412501581
Debener, S., Emkes, R., De Vos, M., and Bleichner, M. (2015). Unobtrusive ambulatory EEG using a smartphone and flexible printed electrodes around the ear. Sci. Rep. 5:16743. doi: 10.1038/srep16743
Dehais, F., Duprès, A., Blum, S., Drougard, N., Scannella, S., Roy, R. N., et al. (2019). Monitoring pilot's mental workload using erps and spectral power with a six-dry-electrode EEG system in real flight conditions. Sensors 19:1324. doi: 10.3390/s19061324
Dehais, F., Duprès, A., Di Flumeri, G., Verdière, K. J., Borghini, G., Babiloni, F., et al. (2018). “Monitoring pilot's cognitive fatigue with engagement features in simulated and actual flight conditions using an hybrid fNIRS-EEG passive BCI,” in 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC) (Miyazaki: IEEE). doi: 10.1109/SMC.2018.00102
Dehais, F., Roy, R. N., Gateau, T., and Scannella, S. (2016). “Auditory alarm misperception in the cockpit: an EEG study of inattentional deafness,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Toronto, ON: Springer Verlag), 177–187. doi: 10.1007/978-3-319-39955-3_17
Delisle-Rodriguez, D., Villa-Parra, A. C., Bastos-Filho, T., López-Delis, A., Frizera-Neto, A., Krishnan, S., et al. (2017). Adaptive spatial filter based on similarity indices to preserve the neural information on EEG signals during on-line processing. Sensors 17:2725. doi: 10.3390/s17122725
Di Flumeri, G., Arico, P., Borghini, G., Colosimo, A., and Babiloni, F. (2016). “A new regression-based method for the eye blinks artifacts correction in the EEG signal, without using any EOG channel,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society (Orlando, FL: EMBS), 3187–3190. doi: 10.1109/EMBC.2016.7591406
Di Flumeri, G., Aricò, P., Borghini, G., Sciaraffa, N., Di Florio, A., and Babiloni, F. (2019). The dry revolution: evaluation of three different EEG dry electrode types in terms of signal spectral features, mental states classification and usability. Sensors 19:1365. doi: 10.3390/s19061365
Di Nocera, F., Camilli, M., and Terenzi, M. (2007). A random glance at the flight deck: pilots' scanning strategies and the real-time assessment of mental workload. J. Cogn. Eng. Decis. Mak. 1, 271–285. doi: 10.1518/155534307X255627
Di Stasi, L. L., McCamy, M. B., Martinez-Conde, S., Gayles, E., Hoare, C., Foster, M., et al. (2016). Effects of long and short simulated flights on the saccadic eye movement velocity of aviators. Physiol. Behav. 153, 91–96. doi: 10.1016/j.physbeh.2015.10.024
Dimitriadis, S. I., Salis, C., and Linden, D. (2018). A novel, fast and efficient single-sensor automatic sleep-stage classification based on complementary cross-frequency coupling estimates. Clin. Neurophysiol. 129, 815–828. doi: 10.1016/j.clinph.2017.12.039
Doppelmayr, M., Finkenzeller, T., and Sauseng, P. (2008). Frontal midline theta in the pre-shot phase of rifle shooting: differences between experts and novices. Neuropsychologia 46, 1463–1467. doi: 10.1016/j.neuropsychologia.2007.12.026
Duchowski, A. T. (2017). Eye Tracking Methodology: Theory and Practice. 3rd Edn. Springer International Publishing. doi: 10.1007/978-3-319-57883-5
Dussault, C., Jouanin, J.-C., and Guezennec, C.-Y. (2004). EEG and ECG changes during selected flight sequences. Aviat. Space. Environ. Med. 75, 889–897.
Faust, O., Acharya, U. R., Min, L. C., and Sputh, B. H. C. (2010). Automatic identification of epileptic and background eeg signals using frequency domain parameters. Int. J. Neural Syst. 20, 159–176. doi: 10.1142/S0129065710002334
Favre-Felix, A., Graversen, C., Dau, T., and Lunner, T. (2017). “Real-time estimation of eye gaze by in-ear electrodes,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (Seogwipo: Institute of Electrical and Electronics Engineers Inc.), 4086–4089. doi: 10.1109/EMBC.2017.8037754
Fietze, I., Penzel, T., Partinen, M., Sauter, J., Küchler, G., Suvoro, A., et al. (2015). Actigraphy combined with EEG compared to polysomnography in sleep apnea patients. Physiol. Meas. 36, 385–96. doi: 10.1088/0967-3334/36/3/385
Fraiwan, L., Lweesy, K., Khasawneh, N., Wenz, H., and Dickhaus, H. (2012). Automated sleep stage identification system based on time-frequency analysis of a single EEG channel and random forest classifier. Comput. Methods Programs Biomed. 108, 10–19. doi: 10.1016/j.cmpb.2011.11.005
Gao, J. F., Yang, Y., Lin, P., Wang, P., and Zheng, C. X. (2010). Automatic removal of eye-movement and blink artifacts from eeg signals. Brain Topogr. 23, 105–114. doi: 10.1007/s10548-009-0131-4
Gegenfurtner, A., Lehtinen, E., and Säljö, R. (2011). Expertise differences in the comprehension of visualizations: a meta-analysis of eye-tracking research in professional domains. Educ. Psychol. Rev. 23, 523–552. doi: 10.1007/s10648-011-9174-7
Gevins, A., Smith, M. E., and McEvoy, L. K. (2003). “EEG and ERP imaging of brain function,” in Detection of Change (Boston, MA: Springer US), 133–148. doi: 10.1007/978-1-4615-0294-4_8
Gharagozlou, F., Nasl Saraji, G., Mazloumi, A., Nahvi, A., Motie Nasrabadi, A., Rahimi Foroushani, A., et al. (2015). Detecting Driver Mental Fatigue Based on EEG Alpha Power Changes during Simulated Driving. Iran. J. Public Health 44, 1693–1700.
Gharbali, A. A., Najdi, S., and Fonseca, J. M. (2018). Investigating the contribution of distance-based features to automatic sleep stage classification. Comput. Biol. Med. 96, 8–23. doi: 10.1016/j.compbiomed.2018.03.001
Glos, M., Fietze, I., Blau, A., Baumann, G., and Penzel, T. (2014). Cardiac autonomic modulation and sleepiness: physiological consequences of sleep deprivation due to 40h of prolonged wakefulness. Physiol. Behav. 125, 45–53. doi: 10.1016/j.physbeh.2013.11.011
Gordon, S. M., Lawhern, V., Passaro, A. D., and McDowell, K. (2015). Informed decomposition of electroencephalographic data. J. Neurosci. Methods 256, 41–55. doi: 10.1016/j.jneumeth.2015.08.019
Görür, D., Halici, U., Aydin, H., Ongun, G., Özgen, F., and Leblebicioglu, K. (2003). “Sleep spindles detection using autoregressive modeling,” in Proceedings of the International Joint Conference on Neural Networks (Portland, OR).
Guénolé, F., Chevrier, É., Stip, E., and Godbout, R. (2014). A microstructural study of sleep instability in drug-naive patients with schizophrenia and healthy controls: sleep spindles, rapid eye movements, and muscle atonia. Schizophr. Res. 155, 31–38. doi: 10.1016/j.schres.2014.03.013
Gunnarsdottir, K. M., Gamaldo, C., Salas, R. M., Ewen, J. B., Allen, R. P., Hu, K., et al. (2020). A novel sleep stage scoring system: combining expert-based features with the generalized linear model. J. Sleep Res. 29:e12991. doi: 10.1111/jsr.12991
Gunnarsdottir, K. M., Gamaldo, C. E., Salas, R. M. E., Ewen, J. B., Allen, R. P., and Sarma, S. V. (2018). “A novel sleep stage scoring system: combining expert-based rules with a decision tree classi fier,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (Institute of Electrical and Electronics Engineers Inc.), 3240–3243. doi: 10.1109/EMBC.2018.8513039
Guragain, B., Rad, A. B., Wang, C., Verma, A. K., Archer, L., Wilson, N., et al. (2019). “EEG-based classification of microsleep by means of feature selection: an application in aviation,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (Honolulu, HI: Institute of Electrical and Electronics Engineers Inc.), 4060–4063. doi: 10.1109/EMBC.2019.8856429
Haavisto, M. L., Porkka-Heiskanen, T., Hublin, C., Härmä, M., Mutanen, P., Müller, K., et al. (2010). Sleep restriction for the duration of a work week impairs multitasking performance. J. Sleep Res. 19, 444–454. doi: 10.1111/j.1365-2869.2010.00823.x
Hairston, W. D., Whitaker, K. W., Ries, A. J., Vettel, J. M., Bradford, J. C., Kerick, S. E., et al. (2014). Usability of four commercially-oriented EEG systems. J. Neural Eng. 11:046018. doi: 10.1088/1741-2560/11/4/046018
Hallvig, D., Anund, A., Fors, C., Kecklund, G., and Åkerstedt, T. (2014). Real driving at night - predicting lane departures from physiological and subjective sleepiness. Biol. Psychol. 101, 18–23. doi: 10.1016/j.biopsycho.2014.07.001
Han, C. H., Kim, E., and Im, C. H. (2020). Development of a brain–computer interface toggle switch with low false-positive rate using respiration-modulated photoplethysmography. Sensors 20:348. doi: 10.3390/s20020348
Hart, J., Onceanu, D., Sohn, C., Wightman, D., and Vertegaal, R. (2009). “The attentive hearing aid: eye selection of auditory sources for hearing impaired users,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Uppsala; Berlin; Heidelberg: Springer), 19–35. doi: 10.1007/978-3-642-03655-2_4
Hassan, A. R., and Bhuiyan, M. I. H. (2016). Computer-aided sleep staging using complete ensemble empirical mode decomposition with adaptive noise and bootstrap aggregating. Biomed. Signal Process. Control 24, 1–10. doi: 10.1016/j.bspc.2015.09.002
He, S., Tan, H., Li, Y., Zhou, Y., Yu, T., Zhang, R., et al. (2020). EEG- and EOG-based asynchronous hybrid BCI: a system integrating a speller, a web browser, an e-mail client, and a file explorer. IEEE Trans. Neural Syst. Rehabil. Eng. 28, 519–530. doi: 10.1109/TNSRE.2019.2961309
He, S., Yu, T., Gu, Z., and Li, Y. (2017). “A hybrid BCI web browser based on EEG and EOG signals,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (Seogwipo: Institute of Electrical and Electronics Engineers Inc.), 1006–1009. doi: 10.1109/EMBC.2017.8036996
He, Y., Luu, T. P., Nathan, K., Nakagome, S., and Contreras-Vidal, J. L. (2018). Data descriptor: a mobile brainbody imaging dataset recorded during treadmill walking with a brain-computer interface. Sci. Data 5, 1–10. doi: 10.1038/sdata.2018.74
Hinrichs, H., Scholz, M., Baum, A. K., Kam, J. W. Y., Knight, R. T., and Heinze, H. J. (2020). Comparison between a wireless dry electrode EEG system with a conventional wired wet electrode EEG system for clinical applications. Sci. Rep. 10:5218. doi: 10.1038/s41598-020-62154-0
Hodgson, T. L., Ezard, G., and Hermens, F. (2019). “Eye movements in neuropsychological tasks,” in Current Topics in Behavioral Neurosciences (Springer), 393–418. doi: 10.1007/7854_2019_98
Holm, A., Lukander, K., Korpela, J., Sallinen, M., and Müller, K. M. I. (2009). Estimating brain load from the EEG. Sci. World J. 9, 639–651. doi: 10.1100/tsw.2009.83
Hsu, W. Y. (2011). Continuous EEG signal analysis for asynchronous BCI application. Int. J. Neural Syst. 21, 335–350. doi: 10.1142/S0129065711002870
Hsu, W. Y. (2013a). Application of quantum-behaved particle swarm optimization to motor imagery EEG classification. Int. J. Neural Syst. 23:13500263. doi: 10.1142/S0129065713500263
Hsu, W. Y. (2013b). Independent component analysis and multiresolution asymmetry ratio for brain-computer interface. Clin. EEG Neurosci. 44, 105–111. doi: 10.1177/1550059412463660
Hsu, W. Y. (2015). Assembling a multi-feature eeg classifier for left-right motor imagery data using wavelet-based fuzzy approximate entropy for improved accuracy. Int. J. Neural Syst. 25:1550037. doi: 10.1142/S0129065715500379
Huang, Q., Chen, Y., and Zhang, Z. (2019). An EOG-based wheelchair robotic arm system for assisting patients with severe spinal cord injuries. J. Nerual Eng. 16:026021. doi: 10.1088/1741-2552/aafc88
Ingre, M., Van Leeuwen, W., Klemets, T., Ullvetter, C., Hough, S., Kecklund, G., et al. (2014). Validating and extending the three process model of alertness in airline operations. PLoS ONE 9:e0108679. doi: 10.1371/journal.pone.0108679
Issa, M. F., and Juhasz, Z. (2019). Improved EOG artifact removal using wavelet enhanced independent component analysis. Brain Sci. 9:355. doi: 10.3390/brainsci9120355
Itani, A. (2009). Health and management in aviation. The experience of air flight pilots and air trafic controllers. Psicol. e Soc. 21, 203–212. doi: 10.1590/S0102-71822009000200007
Ivorra, E., Ortega, M., Catalán, J. M., Ezquerro, S., Lledó, L. D., Garcia-Aracil, N., et al. (2018). Intelligent multimodal framework for human assistive robotics based on computer vision algorithms. Sensors 18:2408. doi: 10.3390/s18082408
Jaleel, A., Ahmed, B., Tafreshi, R., Boivin, D. B., Streletz, L., and Haddad, N. (2014). Improved spindle detection through intuitive pre-processing of electroencephalogram. J. Neurosci. Methods 233, 1–12. doi: 10.1016/j.jneumeth.2014.05.009
Javaid, M. A., Weeden, J., Flom, P., Avitable, M., Glazman, S., and Bodis-Wollner, I. (2010). Perisaccadic gamma modulation in Parkinson disease patients and healthy subjects. Clin. EEG Neurosci. 41, 94–101. doi: 10.1177/155005941004100209
Javed, E., Faye, I., Malik, A. S., and Abdullah, J. M. (2017). Removal of BCG artefact from concurrent fMRI-EEG recordings based on EMD and PCA. J. Neurosci. Methods 291, 150–165. doi: 10.1016/j.jneumeth.2017.08.020
Jia, Y., and Tyler, C. W. (2019). Measurement of saccadic eye movements by electrooculography for simultaneous EEG recording. Behav. Res. Methods 51, 2139–2151. doi: 10.3758/s13428-019-01280-8
Jiang, D., Lu, Y., nan, Ma, Y., and Wang, Y. (2019). Robust sleep stage classification with single-channel EEG signals using multimodal decomposition and HMM-based refinement. Expert Syst. Appl. 121, 188–203. doi: 10.1016/j.eswa.2018.12.023
Jiang, J., Zhou, Z., Yin, E., Yu, Y., and Hu, D. (2014). “Hybrid brain-computer interface (BCI) based on the EEG and EOG signals,” in Bio-Medical Materials and Engineering (Beijing: IOS Press), 2919–2925. doi: 10.3233/BME-141111
Justen, C., Herbert, C., Werner, K., and Raab, M. (2014). Self vs. other: neural correlates underlying agent identification based on unimodal auditory information as revealed by electrotomography (sLORETA). Neuroscience 259, 25–34. doi: 10.1016/j.neuroscience.2013.11.042
Kalayci, T., and Özdamar, O. (1995). Wavelet preprocessing for automated neural network detection of EEG spikes. IEEE Eng. Med. Biol. Mag. 14, 160–166. doi: 10.1109/51.376754
Kang, S., and Shaikh, A. G. (2017). Acquired pendular nystagmus HHS public access. J. Neurol Sci. 375, 8–17. doi: 10.1016/j.jns.2017.01.033
Kanoh, S., Ichi-Nohe, S., Shioya, S., Inoue, K., and Kawashima, R. (2015). “Development of an eyewear to measure eye and body movements,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society (Milan: EMBS), 2267–2270. doi: 10.1109/EMBC.2015.7318844
Kaur, M., Lagopoulos, J., Lee, R. S. C., Ward, P. B., Naismith, S. L., Hickie, I. B., et al. (2013). Longitudinal associations between mismatch negativity and disability in early schizophrenia- and affective-spectrum disorders. Prog. Neuro Psychopharmacol. Biol. Psychiatry 46, 161–169. doi: 10.1016/j.pnpbp.2013.07.002
Kempfner, J., Jennum, P., Nikolic, M., Christensen, J. A. E., and Sorensen, H. B. D. (2012). “Automatic detection of REM sleep in subjects without atonia,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (San Diego, CA), 4242–4245. doi: 10.1109/EMBC.2012.6346903
Kempfner, J., Jennum, P., Sorensen, H. B. D., Christensen, J. A. E., and Nikolic, M. (2013). “Automatic SLEEP staging: from young aduslts to elderly patients using multi-class support vector machine,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (Osaka: Conf Proc IEEE Eng Med Biol Soc), 5777–5780. doi: 10.1109/EMBC.2013.6610864
Kempfner, J., Sorensen, G. L., Sorensen, H. B. D., and Jennum, P. (2011). “Automatic REM sleep detection associated with idiopathic REM sleep behavior disorder,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (Conf Proc IEEE Eng Med Biol Soc), 6063–6066. doi: 10.1109/IEMBS.2011.6091498
Keshavarz, B., and Berti, S. (2014). Integration of sensory information precedes the sensation of vection: a combined behavioral and event-related brain potential (ERP) study. Behav. Brain Res. 259, 131–136. doi: 10.1016/j.bbr.2013.10.045
Khalighi, S., Sousa, T., and Nunes, U. (2012). “Adaptive automatic sleep stage classification under covariate shift,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (San Diego, CA: Conf Proc IEEE Eng Med Biol Soc), 2259–2262. doi: 10.1109/EMBC.2012.6346412
Khalighi, S., Sousa, T., Oliveira, D., Pires, G., and Nunes, U. (2011). “Efficient feature selection for sleep staging based on maximal overlap discrete wavelet transform and SVM,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (Boston, MA), 3306–3309. doi: 10.1109/IEMBS.2011.6090897
Khan, M. J., and Hong, K.-S. (2015). Passive BCI based on drowsiness detection: an fNIRS study. Biomed. Opt. Express 6:4063. doi: 10.1364/BOE.6.004063
Khushaba, R. N., Kodagoda, S., Lal, S., and Dissanayake, G. (2011). Driver drowsiness classification using fuzzy wavelet-packet-based feature-extraction algorithm. IEEE Trans. Biomed. Eng. 58, 121–131. doi: 10.1109/TBME.2010.2077291
Killane, I., Browett, G., and Reilly, R. B. (2013). “Measurement of attention during movement: acquisition of ambulatory EEG and cognitive performance from healthy young adults,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (Osaka: Conf Proc IEEE Eng Med Biol Soc), 6397–6400. doi: 10.1109/EMBC.2013.6611018
Kim, D. J., Bolbecker, A. R., Howell, J., Rass, O., Sporns, O., Hetrick, W. P., et al. (2013). Disturbed resting state EEG synchronization in bipolar disorder: a graph-theoretic analysis. Neuroimage Clin. 2, 414–423. doi: 10.1016/j.nicl.2013.03.007
Kirenskaya, A. V., Myamlin, V. V., Novototsky-Vlasov, V. Y., Pletnikov, M. V., and Kozlovskaya, I. B. (2011). The contingent negative variation laterality and dynamics in antisaccade task in normal and unmedicated schizophrenic subjects. Span. J. Psychol. 14, 869–883. doi: 10.5209/rev_SJOP.2011.v14.n2.34
Kleifges, K., Bigdely-Shamlo, N., Kerick, S. E., and Robbins, K. A. (2017). BLINKER: automated extraction of ocular indices from EEG enabling large-scale analysis. Front. Neurosci. 11:12. doi: 10.3389/fnins.2017.00012
Klein, A., and Skrandies, W. (2013). A reliable statistical method to detect eyeblink-artefacts from electroencephalogram data only. Brain Topogr. 26, 558–568. doi: 10.1007/s10548-013-0281-2
Klimesch, W., Sauseng, P., and Hanslmayr, S. (2007). EEG alpha oscillations: the inhibition-timing hypothesis. Brain Res. Rev. 53, 63–88. doi: 10.1016/j.brainresrev.2006.06.003
Klimesch, W., Vogt, F., and Doppelmayr, M. (1999). Interindividual differences in alpha and theta power reflect memory performance. Intelligence 27, 347–362. doi: 10.1016/S0160-2896(99)00027-6
Klok, A. B., Edin, J., Cesari, M., Olesen, A. N., Jennum, P., and Sorensen, H. B. D. (2018). “A new fully automated random-forest algorithm for sleep staging,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (Honolulu, HI: Institute of Electrical and Electronics Engineers Inc.), 4920–4923. doi: 10.1109/EMBC.2018.8513413
Koch, H., Christensen, J. A. E., Frandsen, R., Zoetmulder, M., Arvastson, L., Christensen, S. R., et al. (2014). Automatic sleep classification using a data-driven topic model reveals latent sleep states. J. Neurosci. Methods 235, 130–137. doi: 10.1016/j.jneumeth.2014.07.002
Koch, H., Jennum, P., and Christensen, J. A. E. (2019). Automatic sleep classification using adaptive segmentation reveals an increased number of rapid eye movement sleep transitions. J. Sleep Res. 28:e12780. doi: 10.1111/jsr.12780
Kong, D., and Soon, C. (2012). Functional imaging correlates of impaired distractor suppression following sleep deprivation. Neuroimage 61, 50–55. doi: 10.1016/j.neuroimage.2012.02.081
Kong, W., Zhou, Z., Hu, S., Zhang, J., Babiloni, F., and Dai, G. (2013). Automatic and direct identification of blink components from scalp EEG. Sensors 13, 10783–10801. doi: 10.3390/s130810783
Korkalainen, H., Leppanen, T., Aakko, J., Nikkonen, S., Kainulainen, S., Leino, A., et al. (2019). Accurate deep learning-based sleep staging in a clinical population with suspected obstructive sleep apnea. IEEE J. Biomed. Heal. Inform. 24, 2073–2081. doi: 10.1109/JBHI.2019.2951346
Krakovská, A., and Mezeiová, K. (2011). Automatic sleep scoring: a search for an optimal combination of measures. Artif. Intell. Med. 53, 25–33. doi: 10.1016/j.artmed.2011.06.004
Krauss, P., Schilling, A., Bauer, J., Tziridis, K., Metzner, C., Schulze, H., et al. (2018). Analysis of multichannel EEG patterns during human sleep: a novel approach. Front. Hum. Neurosci. 12:121. doi: 10.3389/fnhum.2018.00121
Kumar, D., Das, A., Lahiri, U., and Dutta, A. (2016). A human-machine-interface integrating low-cost sensors with a neuromuscular electrical stimulation system for post-stroke balance rehabilitation. J. Vis. Exp. 2016:52394. doi: 10.3791/52394
Kuo, T. B. J., Chen, C. Y., Hsu, Y. C., and Yang, C. C. H. (2016). EEG beta power and heart rate variability describe the association between cortical and autonomic arousals across sleep. Auton. Neurosci. Basic Clin. 194, 32–37. doi: 10.1016/j.autneu.2015.12.001
Laszlo, S., Ruiz-Blondet, M., Khalifian, N., Chu, F., and Jin, Z. (2014). A direct comparison of active and passive amplification electrodes in the same amplifier system. J. Neurosci. Methods 235, 298–307. doi: 10.1016/j.jneumeth.2014.05.012
Lee, M. H., Williamson, J., Won, D. O., Fazli, S., and Lee, S. W. (2018). A high performance spelling system based on EEG-EOG signals with visual feedback. IEEE Trans. Neural Syst. Rehabil. Eng. 26, 1443–1459. doi: 10.1109/TNSRE.2018.2839116
Lee, S. H., Sung, K., Lee, K. S., Moon, E., and Kim, C. G. (2014). Mismatch negativity is a stronger indicator of functional outcomes than neurocognition or theory of mind in patients with schizophrenia. Prog. Neuro Psychopharmacol. Biol. Psychiatry 48, 213–219. doi: 10.1016/j.pnpbp.2013.10.010
Li, T., Zhang, J., Xue, T., and Wang, B. (2017). Development of a novel motor imagery control technique and application in a gaming environment. Comput. Intell. Neurosci. 2017:5863512. doi: 10.1155/2017/5863512
Li, Y., Yingle, F., Gu, L., and Qinye, T. (2009). “Sleep stage classification based on EEG hilbert-huang transform,” in 2009 4th IEEE Conference on Industrial Electronics and Applications, ICIEA (Xi'an) 3676–3681. doi: 10.1109/ICIEA.2009.5138842
Liang, S. F., Kuo, C. E., Hu, Y. H., and Cheng, Y. S. (2011). “A rule-based automatic sleep staging method,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (Boston, MA: Conf Proc IEEE Eng Med Biol Soc), 6067–6070.
Liao, L., De Chen, C. Y., Wang, I. J., Chen, S. F., Li, S. Y., Chen, B. W., et al. (2012). Gaming control using a wearable and wireless EEG-based brain-computer interface device with novel dry foam-based sensors. J. Neuroeng. Rehabil. 9, 1–12. doi: 10.1186/1743-0003-9-5
Lisberger, S. G., and Westbrook, L. E. (1985). Properties of visual inputs that initiate horizontal smooth pursuit eye movements in monkeys. J. Neurosci. 5, 1662–1673. doi: 10.1523/JNEUROSCI.05-06-01662.1985
Liu, R., Zhang, Z., Duan, F., Zhou, X., and Meng, Z. (2017). Identification of anisomerous motor imagery EEG signals based on complex algorithms. Comput. Intell. Neurosci. 2017:2727856. doi: 10.1155/2017/2727856
Liu, Y., Zhou, Z., and Hu, D. (2011). Gaze independent brain-computer speller with covert visual search tasks. Clin. Neurophysiol. 122, 1127–1136. doi: 10.1016/j.clinph.2010.10.049
Looney, D., Park, C., Kidmose, P., Rank, M. L., Ungstrup, M., Rosenkranz, K., et al. (2011). “An in-the-ear platform for recording electroencephalogram,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (Boston, MA), 6882–6885. doi: 10.1109/IEMBS.2011.6091733
Ma, J., Bayram, S., Tao, P., and Svetnik, V. (2011). High-throughput ocular artifact reduction in multichannel electroencephalography (EEG) using component subspace projection. J. Neurosci. Methods 196, 131–140. doi: 10.1016/j.jneumeth.2011.01.007
Ma, J., Zhang, Y., Cichocki, A., and Matsuno, F. (2015). A novel EOG/EEG hybrid human-machine interface adopting eye movements and ERPs: application to robot control. IEEE Trans. Biomed. Eng. 62, 876–889. doi: 10.1109/TBME.2014.2369483
Ma, J. X., Shi, L. C., and Lu, B. L. (2010). “Vigilance estimation by using electrooculographic features,” in 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC'10 (Buenos Aires), 6591–6594.
MacDonald, B., and Barry, R. J. (2014). Trial effects in single-trial ERP components and autonomic responses at very long ISIs. Int. J. Psychophysiol. 92, 99–112. doi: 10.1016/j.ijpsycho.2014.03.007
Manabe, H., Fukumoto, M., and Yagi, T. (2013). “Automatic drift calibration for EOG-based gaze input interface,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (Osaka), 53–56. doi: 10.1109/EMBC.2013.6609435
Meyberg, S., Sommer, W., and Dimigen, O. (2017). How microsaccades relate to lateralized ERP components of spatial attention: a co-registration study. Neuropsychologia 99, 64–80. doi: 10.1016/j.neuropsychologia.2017.02.023
Min, B. K., Dähne, S., Ahn, M. H., Noh, Y. K., and Müller, K. R. (2016). Decoding of top-down cognitive processing for SSVEP-controlled BMI. Sci. Rep. 6:36567. doi: 10.1038/srep36267
Mirkovic, B., Bleichner, M. G., De Vos, M., and Debener, S. (2016). Target speaker detection with concealed EEG around the ear. Front. Neurosci. 10:349. doi: 10.3389/fnins.2016.00349
Müller, C., Nicoletti, C., Omlin, S., Brink, M., and Läubli, T. (2015). Relationship between sleep stages and nocturnal trapezius muscle activity. J. Electromyogr. Kinesiol. 25, 457–462. doi: 10.1016/j.jelekin.2015.01.010
Nguyen, T., Ahn, S., Jang, H., Jun, S. C., and Kim, J. G. (2017). Utilization of a combined EEG/NIRS system to predict driver drowsiness. Sci. Rep. 7:43933. doi: 10.1038/srep43933
Nguyen, T., Babawale, O., Kim, T., Jo, H. J., Liu, H., and Kim, J. G. (2018). Exploring brain functional connectivity in rest and sleep states: a fNIRS study. Sci. Rep. 8:16144. doi: 10.1038/s41598-018-33439-2
Niemenlehto, P. H. (2009). Constant false alarm rate detection of saccadic eye movements in electro-oculography. Comput. Methods Programs Biomed. 96, 158–171. doi: 10.1016/j.cmpb.2009.04.011
Nogueira, W., Dolhopiatenko, H., Schierholz, I., Büchner, A., Mirkovic, B., Bleichner, M. G., et al. (2019). Decoding selective attention in normal hearing listeners and bilateral cochlear implant users with concealed ear EEG. Front. Neurosci. 13:720. doi: 10.3389/fnins.2019.00720
Nolan, H., Whelan, R., and Reilly, R. B. (2010). FASTER: fully automated statistical thresholding for EEG artifact rejection. J. Neurosci. Methods 192, 152–162. doi: 10.1016/j.jneumeth.2010.07.015
Noureddin, B., Lawrence, P. D., and Birch, G. E. (2012). Online removal of eye movement and blink EEG artifacts using a high-speed eye tracker. IEEE Trans. Biomed. Eng. 59, 2103–2110. doi: 10.1109/TBME.2011.2108295
Oken, B. S., Salinsky, M. C., and Elsas, S. M. (2006). Vigilance, alertness, or sustained attention: physiological basis and measurement. Clin. Neurophysiol. 117, 1885–1901. doi: 10.1016/j.clinph.2006.01.017
Olsen, A. V., Stephansen, J., Leary, E., Peppard, P. E., Sheungshul, H., Jenum, P., et al. (2017). Diagnostic value of sleep stage dissociation as visualized on a 2-dimensional sleep state space in human narcolepsy. J. Neurosci. Methods 282, 9–19. doi: 10.1016/j.jneumeth.2017.02.004
Pan, S. T., Kuo, C. E., Zeng, J. H., and Liang, S. F. (2012). A transition-constrained discrete hidden Markov model for automatic sleep staging. Biomed. Eng. Online. 11:52. doi: 10.1186/1475-925X-11-52
Parasuraman, R., and Jiang, Y. (2012). Individual differences in cognition, affect, and performance: Behavioral, neuroimaging, and molecular genetic approaches. Neuroimage 59, 70–82. doi: 10.1016/j.neuroimage.2011.04.040
Petrie, K. J., and Dawson, A. G. (1997). Symptoms of fatigue and coping strategies in international pilots. Int. J. Aviat. Psychol. 7, 251–258. doi: 10.1207/s15327108ijap0703_5
Pettersson, K., Jagadeesan, S., Lukander, K., Henelius, A., Hæggström, E., and Müller, K. (2013). Algorithm for automatic analysis of electro-oculographic data. Biomed. Eng. Online 12:110. doi: 10.1186/1475-925X-12-110
Peysakhovich, V., Dehais, F., and Duchowski, A. T. (2019). “Why is eye tracking an essential part of neuroergonomics?” in Neuroergonomics: The Brain at Work and in Everyday Life (Elsevier), 27–30. doi: 10.1016/B978-0-12-811926-6.00004-X
Peysakhovich, V., Lefrançois, O., Dehais, F., and Causse, M. (2018). The neuroergonomics of aircraft cockpits: the four stages of eye-tracking integration to enhance flight safety. Safety 4:8. doi: 10.3390/safety4010008
Pham, T. T. H., Croft, R. J., and Cadusch, P. J. (2011a). Temporal stability of regression-based electrooculographic correction coefficients. Psychophysiology 48, 96–101. doi: 10.1111/j.1469-8986.2010.01036.x
Pham, T. T. H., Croft, R. J., Cadusch, P. J., and Barry, R. J. (2011b). A test of four EOG correction methods using an improved validation technique. Int. J. Psychophysiol. 79, 203–210. doi: 10.1016/j.ijpsycho.2010.10.008
Postma, M. A., Schellekens, J. M., Hanson, E. K., and Hoogeboom, P. J. (2005). Fz Theta Divided by Pz Alpha as an Index of Task Load During a PC-Based Air Traffic Control Simulation. Maastricht: Shaker Publishing, 465–469.
Punsawad, Y., Wongsawat, Y., and Parnichkun, M. (2010). “Hybrid EEG-EOG brain-computer interface system for practical machine control,” in 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC'10 (Buenos Aires), 1360–1363. doi: 10.1109/IEMBS.2010.5626745
Radha, M., Garcia-Molina, G., Poel, M., and Tononi, G. (2014). “Comparison of feature and classifier algorithms for online automatic sleep staging based on a single EEG signal,” in 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC (Chicago, IL), 1876–1880. doi: 10.1109/EMBC.2014.6943976
Rajesh, N. (2014). EOG controlled motorized wheelchair for disabled persons. Int. J. Med. Heal. Biomed. Bioeng. Pharm. Eng. 8, 302–305. doi: 10.5281/zenodo.1337385
Rao, R., and Derakhshani, R. (2005). “A comparison of EEG preprocessing methods using time delay neural networks,” in 2nd International IEEE EMBS Conference on Neural Engineering (Arlington, VA), 262–264.
Reed, C. M., Birch, K. G., Kamiński, J., Sullivan, S., Chung, J. M., Mamelak, A. N., et al. (2017). Automatic detection of periods of slow wave sleep based on intracranial depth electrode recordings. J. Neurosci. Methods 282, 1–8. doi: 10.1016/j.jneumeth.2017.02.009
Reis, P. M. R., Hebenstreit, F., Gabsteiger, F., von Tscharner, V., and Lochmann, M. (2014). Methodological aspects of EEG and body dynamics measurements during motion. Front. Hum. Neurosci. 8:156. doi: 10.3389/fnhum.2014.00156
Rezaei, M., Mohammadi, H., and Khazaie, H. (2017). EEG/EOG/EMG data from a cross sectional study on psychophysiological insomnia and normal sleep subjects. Data Br. 15, 314–319. doi: 10.1016/j.dib.2017.09.033
Rosales-Lagarde, A., Rodriguez-Torres, E. E., Itzá-Ortiz, B. A., Miramontes, P., Vázquez-Tagle, G., Enciso-Alva, J. C., et al. (2018). The color of noise and weak stationarity at the NREM to REM sleep transition in mild cognitive impaired subjects. Front. Psychol. 9:1205. doi: 10.3389/fpsyg.2018.01205
Roy, R. N., and Frey, J. (2016). “Neurophysiological Markers for Passive Brain-Computer Interfaces,” in Brain-Computer Interfaces 1: Foundations and Methods (Wiley Online Library), 85–100. doi: 10.1002/9781119144977.ch5
Sameni, R., and Gouy-Pailler, C. (2014). An iterative subspace denoising algorithm for removing electroencephalogram ocular artifacts. J. Neurosci. Methods 225, 97–105. doi: 10.1016/j.jneumeth.2014.01.024
Sauvet, F., Bougard, C., Coroenne, M., Lely, L., Van Beers, P., Elbaz, M., et al. (2014). In-flight automatic detection of vigilance states using a single EEG channel. IEEE Trans. Biomed. Eng. 61, 2840–2847. doi: 10.1109/TBME.2014.2331189
Scarlatelli-Lima, A. V., Sukys-Claudino, L., Watanabe, N., Guarnieri, R., Walz, R., and Lin, K. (2016). How do people with drug-resistant mesial temporal lobe epilepsy sleep? A clinical and video-EEG with EOG and submental EMG for sleep staging study. eNeurologicalSci 4, 34–41. doi: 10.1016/j.ensci.2016.06.002
Semmlow, J. L., Yuan, W., and Alvarez, T. L. (1998). Evidence for separate control of slow version and vergence eye movements: support for hering's law. Vision Res. 38, 1145–1152. doi: 10.1016/S0042-6989(97)00251-4
Shustak, S., Inzelberg, L., Steinberg, S., Rand, D., David Pur, M., Hillel, I., et al. (2018). Home monitoring of sleep with a temporary-tattoo EEG, EOG and EMG electrode array: a feasibility study. J. Neural Eng. 16:026024. doi: 10.1088/1741-2552/aafa05
Singh, H., and Singh, J. (2012). Human eye tracking and related issues: a review. Int. J. Sci. Res. Publ. 2.
Singh, S., Shukla, G., Goyal, V., Srivastava, A. K., Singh, M. B., Vibha, D., et al. (2014). Impact of sleep on the localizing value of video EEG in patients with refractory focal seizures - a prospective video-EEG with EOG and submental EMG study. Clin. Neurophysiol. 125, 2337–2343. doi: 10.1016/j.clinph.2014.03.021
Skotte, J. H., Nøjgaard, J. K., Jørgensen, L. V., Christensen, K. B., and Sjøgaard, G. (2007). Eye blink frequency during different computer tasks quantified by electrooculography. Eur. J. Appl. Physiol. 99, 113–119. doi: 10.1007/s00421-006-0322-6
Smith-Jentsch, K. A., Brannick, M. T., and Salas, E. (2001). To transfer or not to transfer? Investigating the combined effects of trainee characteristics, team leader support, and team climate. J. Appl. Psychol. 86, 279–292. doi: 10.1037/0021-9010.86.2.279
Soekadar, S. R., Witkowski, M., Vitiello, N., and Birbaumer, N. (2015). An EEG/EOG-based hybrid brain-neural computer interaction (BNCI) system to control an exoskeleton for the paralyzed hand. Biomed. Tech. 60, 199–205. doi: 10.1515/bmt-2014-0126
Sokolovsky, M., Guerrero, F., Paisarnsrisomsuk, S., Ruiz, C., and Alvarez, S. A. (2019). Deep learning for automated feature discovery and classification of sleep stages. IEEE/ACM Trans. Comput. Biol. Bioinforma. 1545–5963. doi: 10.1109/TCBB.2019.2912955
Sommer, D., and Golz, M. (2010). “Evaluation of PERCLOS based current fatigue monitoring technologies,” in 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC'10, (Buenos Aires) 4456–4459. doi: 10.1109/IEMBS.2010.5625960
Stochholm, A., Mikkelsen, K., and Kidmose, P. (2016). “Automatic sleep stage classification using ear-EEG,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (Orlando, FL: Institute of Electrical and Electronics Engineers Inc.), 4751–4754. doi: 10.1109/EMBC.2016.7591789
Sun, C., Chen, C., and Fan, J., Li, W., Zhang, Y., and Chen, W. (2019). A hierarchical sequential neural network with feature fusion for sleep staging based on EOG and RR signals - IOPscience. J. Neural Eng. 16:066020. doi: 10.1088/1741-2552/ab39ca
Supratak, A., Dong, H., Wu, C., and Guo, Y. (2017). DeepSleepNet: a model for automatic sleep stage scoring based on raw single-channel EEG. IEEE Trans. Neural Syst. Rehabil. Eng. 25, 1998–2008. doi: 10.1109/TNSRE.2017.2721116
Svensson, E., Angelborg-Thanderez, M., Sjöberg, L., and Olsson, S. (1997). Information complexity-mental workload and performance in combat aircraft. Ergonomics 40, 362–380. doi: 10.1080/001401397188206
Tagluk, M. E., Sezgin, N., and Akin, M. (2010). Estimation of sleep stages by an artificial neural network employing EEG, EMG and EOG. J. Med. Syst. 34, 717–725. doi: 10.1007/s10916-009-9286-5
Tan, T., Hakenberg, J. P., and Guan, C. (2013). Estimation of glance from EEG for cursor control. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. 2013, 2919–2923. doi: 10.1109/EMBC.2013.6610151
Tang, Y., Zhang, X., Simmonite, M., Li, H., Zhang, T., Guo, Q., et al. (2013). Hyperactivity within an extensive cortical distribution associated with excessive sensitivity in error processing in unmedicated depression: a combined event-related potential and sLORETA study. Int. J. Psychophysiol. 90, 282–289. doi: 10.1016/j.ijpsycho.2013.09.001
Toivanen, M., Pettersson, K., and Lukander, K. (2015). A probabilistic real-time algorithm for detecting blinks, saccades, and fixations from EOG data. J. Eye Mov. Res. 8, 1–14. doi: 10.16910/jemr.8.2.1
Torres-Valencia, C. A., Alvarez, M. A., and Orozco-Gutierrez, A. A. (2014). “Multiple-output support vector machine regression with feature selection for arousal/valence space emotion assessment,” in 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society EMBC (Chicago, IL: IEEE), 970–973. doi: 10.1109/EMBC.2014.6943754
Tyagi, A., and Nehra, V. (2018). A Comparison of Feature Extraction and Dimensionality Reduction Techniques for EEG-Based BCI System. IUP J. Comput. Sci. 11, 51–66. Available online at: https://ssrn.com/abstract=3159745
Usakli, A., Gurkan, S., Aloise, F., Vecchiato, G., and Babiloni, F. (2010). On the use of electrooculogram for efficient human computer interfaces. Comput. Intell. Neurosci. 2010:135629. doi: 10.1155/2010/135629
Velazquez, J. (2018). The presence of behavioral traps in U.S. airline accidents: a qualitative analysis. Safety 4:2. doi: 10.3390/safety4010002
Verdière, K. J., Roy, R. N., and Dehais, F. (2018). Detecting pilot's engagement using fNIRS connectivity features in an automated vs. manual landing scenario. Front. Hum. Neurosci. 12:6. doi: 10.3389/fnhum.2018.00006
Verma, G. K., and Tiwary, U. S. (2014). Multimodal fusion framework: a multiresolution approach for emotion classification and recognition from physiological signals. Neuroimage 102, 162–172. doi: 10.1016/j.neuroimage.2013.11.007
Von Rosenberg, W., Chanwimalueang, T., Goverdovsky, V., Looney, D., Sharp, D., and Mandic, D. P. (2016). Smart helmet:wearable multichannel ECG and EEG. IEEE J. Transl. Eng. Heal. Med. 4:2700111. doi: 10.1109/JTEHM.2016.2609927
Wang, D., Miao, D., and Blohm, G. (2012). Multi-class motor imagery EEG decoding for brain-computer interfaces. Front. Neurosci. 6:151. doi: 10.3389/fnins.2012.00151
Wang, F., Xu, Q., and Fu, R. (2019). Study on the effect of man-machine response mode to relieve driving fatigue based on EEG and EOG. Sensors 19:4883. doi: 10.3390/s19224883
Wang, G., Teng, C., Li, K., Zhang, Z., and Yan, X. (2016). The removal of EOG artifacts from EEG signals using independent component analysis and multivariate empirical mode decomposition. IEEE J. Biomed. Heal. Informatics 20, 1301–1308. doi: 10.1109/JBHI.2015.2450196
Wang, H., Li, Y., Long, J., Yu, T., and Gu, Z. (2014a). An asynchronous wheelchair control by hybrid EEG–EOG brain–computer interface. Cogn. Neurodyn. 8, 399–409. doi: 10.1007/s11571-014-9296-y
Wang, X. W., Nie, D., and Lu, B. L. (2014b). Emotional state classification from EEG data using machine learning approach. Neurocomputing 129, 94–106. doi: 10.1016/j.neucom.2013.06.046
Whitehead, K., Laudiano-Dray, M. P., Meek, J., and Lorenzo, F. (2018). Emergence of mature cortical activity in wakefulness and sleep in healthy preterm and full-term infants. Sleep 41:zsy096. doi: 10.1093/sleep/zsy096
Wilson, N., Guragain, B., Verma, A., Archer, L., and Tavakolian, K. (2020). Blending human and machine: feasibility of measuring fatigue through the aviation headset. Hum. Factors. 62, 553–564. doi: 10.1177/0018720819849783
Winkler, I., Debener, S., Muller, K. R., and Tangermann, M. (2015). On the influence of high-pass filtering on ICA-based artifact reduction in EEG-ERP. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. 2015, 4101–4105. doi: 10.1109/EMBC.2015.7319296
Witkowski, M., Cortese, M., Cempini, M., Mellinger, J., Vitiello, N., and Soekadar, S. R. (2014). Enhancing brain-machine interface (BMI) control of a hand exoskeleton using electrooculography (EOG). J. Neuroeng. Rehabil. 11:165. doi: 10.1186/1743-0003-11-165
Wright, N., and McGown, A. (2001). Vigilance on the civil flight deck: Incidence of sleepiness and sleep during long-haul flights and associated changes in physiological parameters. Ergonomics 44, 82–106. doi: 10.1080/00140130150203893
Wu, J., Zhang, J., Ding, X., Li, R., and Zhou, C. (2013). The effects of music on brain functional networks: a network analysis. Neuroscience 250, 49–59. doi: 10.1016/j.neuroscience.2013.06.021
Yaghouby, F., and Sunderam, S. (2015). Quasi-supervised scoring of human sleep in polysomnograms using augmented input variables. Comput. Biol. Med. 59, 54–63. doi: 10.1016/j.compbiomed.2015.01.012
Yamada, F. (1998). Frontal midline theta rhythm and eyeblinking activity during a VDT task and a video game: useful tools for psychophysiology in ergonomics. Ergonomics 41, 678–688. doi: 10.1080/001401398186847
Yamagishi, K., Hori, J., and Miyakawa, M. (2006). Development of EOG-based communication system controlled by eight-directional eye movements. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2006, 2574–2577. doi: 10.1109/IEMBS.2006.259914
Yamaguchi, K., Hoshiyama, M., and Takano, M. (2011). Biological observation during the daytime of elderly patients with advanced dementia cared for with and without artificial nutrition by percutaneous endoscopic gastrostomy. Geriatr. Gerontol. Int. 11, 221–228. doi: 10.1111/j.1447-0594.2010.00657.x
Yan, M., Tamura, H., and Tanno, K. (2013). “Gaze estimation using electrooculogram signals and its mathematical modeling,” in Proceedings of The International Symposium on Multiple-Valued Logic (Toyama), 18–22.
Yildirim, O., Baloglu, U. B., and Acharya, U. R. (2019). A deep learning model for automated sleep stages classification using PSG signals. Int. J. Environ. Res. Public Health 16:599. doi: 10.3390/ijerph16040599
Yu, Y., Liu, Y., Yin, E., Jiang, J., Zhou, Z., and Hu, D. (2019). An asynchronous hybrid spelling approach based on EEG-EOG signals for chinese character input. IEEE Trans. Neural Syst. Rehabil. Eng. 27, 1292–1302. doi: 10.1109/TNSRE.2019.2914916
Zander, T. O., Kothe, C., Jatzev, S., and Gaertner, M. (2010). “Enhancing Human-Computer Interaction with Input from Active and Passive Brain-Computer Interfaces,” in Brain-Computer Interfaces. Human-Computer Interaction Series, eds D. Tan and A. Nijholt (London: Springer), 181–199. doi: 10.1007/978-1-84996-272-8_11
Zeng, H., and Song, A. (2014). Removal of EOG artifacts from EEG recordings using stationary subspace analysis. Sci. World J. 2014:259121. doi: 10.1155/2014/259121
Zennifa, F., Ageno, S., Hatano, S., and Iramina, K. (2018). Hybrid system for engagement recognition during cognitive tasks using a CFS + KNN algorithm. Sensors 18:3691. doi: 10.3390/s18113691
Zhang, C., Yu, X. L., and Rao, N. N. (2019a). Linear descriptor parameter analysis of mental fatigue's EEG in multi-task. J. Univ. Electron. Sci. Technol. China 48, 613–618. doi: 10.3969/j.issn.1001-0548.2019.04.020
Zhang, J., Wang, B., Zhang, C., Xiao, Y., and Wang, M. Y. (2019b). An EEG/EMG/EOG-based multimodal human-machine interface to real-time control of a soft robot hand. Front. Neurorobot. 13:7. doi: 10.3389/fnbot.2019.00007
Zhang, L., Chi, Y. M., Edelstein, E., Schulze, J., Gramann, K., Velasquez, A., et al. (2010a). “Wireless physiological monitoring and ocular tracking: 3D calibration in a fully-immersive virtual health care environment,” in 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology Society EMBC'10 (Buenos Aires: IEEE Computer Society), 4464–4467. doi: 10.1109/IEMBS.2010.5625969
Zhang, L., He, W., He, C., and Wang, P. (2010b). Improving mental task classification by adding high frequency band information. J. Med. Syst. 34, 51–60. doi: 10.1007/s10916-008-9215-z
Zhang, Y., Zhang, X., Liu, W., Luo, Y., Yu, E., Zou, K., et al. (2014). Automatic sleep staging using multi-dimensional feature extraction and multi-kernel fuzzy support vector machine. J. Healthc. Eng. 5, 505–520. doi: 10.1260/2040-2295.5.4.505
Zheng, W.-L., and Lu, B.-L. (2016). A multimodal approach to estimating vigilance using EEG and forehead EOG - IOPscience. J. Neural Eng. 14:026017. doi: 10.1088/1741-2552/aa5a98
Zhimin, Z., Shoushui, W., Guohun, Z., Feifei, L., Yuwen, L., Xiaotong, D., et al. (2018). Efficient sleep classification based on entropy features and a support vector machine classifier - IOPscience. Physiol. Meas. 39:1–6. doi: 10.1088/1361-6579/aae943
Zhou, Y., He, S., Huang, Q., and Li, Y. (2020). A hybrid asynchronous brain-computer interface combining SSVEP and EOG Signals. IEEE Trans. Biomed. Eng. 67, 2881–2892. doi: 10.1109/TBME.2020.2972747
Keywords: human factors, aeronautics, cognition, brain computer interface, signal processing, mental workload, fatigue
Citation: Belkhiria C and Peysakhovich V (2020) Electro-Encephalography and Electro-Oculography in Aeronautics: A Review Over the Last Decade (2010–2020). Front. Neuroergon. 1:606719. doi: 10.3389/fnrgo.2020.606719
Received: 15 September 2020; Accepted: 17 November 2020;
Published: 21 December 2020.
Edited by:
Manousos A. Klados, International Faculty of the University of Sheffield, GreeceReviewed by:
Ioannis Yiannis Kompatsiaris, Information Technologies Institute, GreecePietro Aricò, Sapienza University of Rome, Italy
Copyright © 2020 Belkhiria and Peysakhovich. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Chama Belkhiria, chama.belkhiria@isae-supaero.fr
†These authors have contributed equally to this work