Anti-Heartbeat-Evoked Potentials Performance in Event-Related Potentials-Based Mental Workload Assessment

The aim of this study was to determine the effect of heartbeat-evoked potentials (HEPs) on the performance of an event-related potential (ERP)-based classification of mental workload (MWL). We produced low- and high-MWLs using a mental arithmetic task and measured the ERP response of 14 participants. ERP trials were divided into three conditions based on the effect of HEPs on ERPs: ERP_HEP, containing the heartbeat in a period of 280–700ms in ERP epochs after the target; ERP_A-HEP, not including the heartbeat within the same period; and ERP_T, all trials including ERP_A-HEP and ERP_HEP. We then compared MWL classification performance using the amplitude and latency of the P600 ERP among the three conditions. The ERP_A-HEP condition achieved an accuracy of 100% using a radial basis function-support vector machine (with 10-fold cross-validation), showing an increase of 14.3 and 28.6% in accuracy compared to ERP_T (85.7%) and ERP_HEP (71.4%), respectively. The results suggest that evoked potentials caused by heartbeat overlapped or interfered with the ERPs and weakened the ERP response to stimuli. This study reveals the effect of the evoked potentials induced by heartbeats on the performance of the MWL classification based on ERPs.

Introduction

Event-related potentials (ERPs) provide a powerful method for interpreting the relationship between the human mind and the brain. ERPs measure brain responses which are a direct result of a specific input in the form of sensory, cognitive, memory, or motor events (Luck, 2014). Because brain activity in response to a single event or stimulus is not usually visible in electroencephalogram (EEG) signals, the ERP technique is required to measure the response to a stimulus in many trials (Coles and Rugg, 1995; Boudewyn et al., 2018). The brain activity of a single event or stimulus trial would be averaged out, and the relevant or dominant potentials would remain (Coles and Rugg, 1995).

Electroencephalogram signals are often contaminated by various artifacts, such as eyeblinks, ocular movements, and muscular and cardiac activity, which are typically not of interest (Uriguen and Garcia-Zapirain, 2015). These artifacts can affect EEG signals and interfere with relevant or dominant potentials in ERPs. Thus, many previous studies have sought to remove artifacts, such as muscular activity (Chen et al., 2019; Zou et al., 2020), cardiac activity (Hamaneh et al., 2014; Dai et al., 2019), eyeblinks, and ocular movements (Dimigen, 2020; Egambaram et al., 2020). The effect of noise on ERP analysis has been minimized thanks to the development of methods to remove noise in EEG signals. However, we hypothesized that other factors, such as changes in mental state (i.e., stress, emotion, and cognitive load) and evoked potentials [i.e., heartbeat-evoked potential (HEP)], in addition to noise, could affect the ERP signals, leading to a decrease in performance (Zhang et al., 2020; Zheng et al., 2020). Some previous studies have sought to improve the classification performance by considering the changes in mental state (Ko et al., 2020; Zhang et al., 2020), but no study using HEP has been reported. Evoked potentials are difficult to remove or recover because, unlike noise, they do not cause a change in the dominant pattern of the EEG signal but are instead contained in the EEG signal itself (Schandry and Montoya, 1996; McCraty et al., 2009).

Heartbeat-evoked potentials are characteristic changes in brain waves caused by evoked potentials that can occur due to changes in cardiac activity, such as heart rhythms and heart rate variability (Schandry and Montoya, 1996; McCraty et al., 2009; Park et al., 2015). The vagus nerve transmits cardiac output information via the visceral-afferent pathway (medulla, amygdalae, hypothalamic and thalamic nuclei, and nucleus tractus solitarius) from the heart to the brain (Montoya et al., 1993; Janig, 1996; Nieuwenhuys et al., 2007). HEPs reflect a synchronization in the communication between the brain and heart based on efferent and afferent pathways, leading to evoked potentials involving changes in alpha activation in EEG signals (McCraty et al., 2009; Park et al., 2015; Park and Whang, 2018). HEP is divided into two components. The first HEP component (50–250ms post-R-wave) is defined as the interval required for afferent information from the heart to reach the brain. An increase in afferent processing is indicated by the synchronization of the alpha wave. The second HEP component (250–600ms post-R-wave) is defined as the time interval needed for blood pressure to reach the brain area from the heart. When the blood pressure wave synchronizes with brain activity, alpha synchronization occurs, which is associated with the higher cognitive centers’ processing of the sensory input (Wölk et al., 1989; McCraty et al., 2009; Park et al., 2015).

As previously mentioned, HEP causes evoked potentials in EEG signals due to the synchronization between the brain and heart. HEP is similar to ERP in terms of how it causes evoked potentials in EEG signals based on a trigger point. We believe that HEP can be contained in the ERP when there is little difference in the trigger time between the heartbeat and the event. In ERP analyses, the detection of a significant ERP pattern may be impeded by evoked potentials overlapping a heartbeat. Thus, this study sought to determine the effect of evoked potentials by HEPs on changes in significant patterns in ERP signals during a cognitive task, as well as the ERP-based classification performance of mental workload, by separating ERP trials that are affected by HEP from those that are not.

Materials and Methods

Participants

Fourteen undergraduate students (seven male and seven female) aged between 22 and 29years (mean 25.2±3.4) participated in the experiment. Each subject participated voluntarily and was paid 100,000 KRW. All participants were right-handed and had no family or medical history of cardiovascular, autonomic, or central nervous system disorders. All participants were required to abstain from alcohol, cigarettes, and caffeine for 24h prior to the experiment and to sleep normally. Informed consent was acquired from all participants who were notified of the restrictions and requirements. All experimental protocols were approved by the Sangmyung University Institutional Bioethics Review Board in Seoul, South Korea (BE2019-46).

Experimental Stimuli and Apparatus

ERP Task

A stimulator was designed to measure the ERP response to mental workload based on our previous studies (Mun et al., 2012; Park et al., 2014, 2015, 2019). The stimulator was located on the left and right sides of the screen, and participants were required to focus their attention on the instructed side of the screen according to an arrow (to ignore the left side and attend to the right one, or vice versa). The stimulator consisted of the presentation of 12 alphanumeric characters involving non-targets (“A” to “K”) and a target (“5”). The alphanumeric characters were randomly updated at the rate of 6Hz. One trial consisted of five sequences involving 60 alphanumeric characters lasting 10s, with an inter-trial interval of 2s (total 60s). One block consisted of five trials, and the entire task consisted of 15 blocks. The target was presented with a probability of 5% within each trial, and the interval between targets lasted less than 1s to avoid overlapping ERPs during analysis, as shown in Figure 1.

FIGURE 1

Figure 1. Experimental stimuli to measure ERP responses (left) and cause mental workload (right).

Mental Workload: Mental Arithmetic Task

The mental arithmetic task was designed to cause mental workload (MWL) based on previous studies (So et al., 2017; Jost et al., 2019) and was divided into two task levels: low- and high-MWL. The low-MWL task consisted of easy questions involving single-digit addition and subtraction (i.e., 3+2, 4–1, with numbers ranging from 1 to 9). The high-MWL task consisted of difficult questions involving mixed arithmetic operations (i.e., 36×7–24, 43+72/9, number range 1–99). The mental arithmetic task questions were randomly presented within a defined range, and included a true answer result and two false confusion results. The two false results were automatically calculated by randomly adding or subtracting a number in the range of 1–5 from the correct answer. Participants were required to select the correct answer using the arrow and spacebar keys of the keyboard, as shown in Figure 1. ERP and mental arithmetic tasks were developed using LabVIEW2016 (National Instruments Inc., Austin, TX, United States).

Experimental Procedure

Participants were required to report their MWL state as a subjective rating both before and after the experiment. The subjective mental effort questionnaire (SMEQ) (Sauro and Dumas, 2009), a questionnaire with a 0–150 scale for rating the MWL was used, as shown in Figure 2. Participants performed the pre-ERP task for 15min. Over the course of this session, all participants were required to fixate on a red cross at the center of the screen, 60cm from the display and press a spacebar key when presented with the target “5.” The performance and response times were measured for the target. Following the pre-ERP task, they performed the mental arithmetic task for 15min. All participants were asked to select the correct answer to the mental arithmetic question, from three options, using the arrow and spacebar keys of the keyboard. For each correct answer, the participant was awarded 10 points, whereas 10 points were deducted for an incorrect answer. In order to increase the subjects’ motivation and engagement, those who achieved the top 15% score were paid 150% of the test fee. Participants were divided into low-MWL and high-MWL task groups. On the first day, they performed either the low-MWL or High-MWL task and on the next day, they performed the other MWL task at the same time (e.g., first day low-MWL task and second day high-MWL task; the order randomized across subjects). Participants then performed the post-ERP task, which was the same as the pre-ERP task. The experimental environment and procedure are illustrated in Figure 3.

FIGURE 2

Figure 2. Subjective mental effort questionnaire (SMEQ).

FIGURE 3

Figure 3. Experimental environment (left) and procedure (right).

Signal Processing and Data Acquisition

Electroencephalogram signals were recorded at a sampling rate of 2,048Hz from 64 channels mounted on an EEG electrode cap (Active-two, Biosemi S.V., Amsterdam, Netherlands) based on the international 10–20 montage with separate reference and ground electrodes for each system (common mode sense, CMS and driven right leg, DRL, respectively). Impedance from all electrodes was kept below 5kΩ (below 10kΩ for the two eye channels). The measured EEG signals were down-sampled to 512Hz and re-referenced using a common average referencing (CAR) procedure (Perrin et al., 1989). The CAR was calculated by subtracting the average potential over all the channels from each channel. This re-referencing maintains the activity from local sources while removing the global background activity (Lew et al., 2012). To avoid contaminating meaningful ERP patterns, we minimized preprocessing by specifying a threshold for each trial. In the trials, where the amplitude exceeded ±100μV at any electrode, we conducted an independent component analysis to remove oculomotor and muscle artifacts by visual inspection. Additionally, a few trials with anomalous patterns were excluded due to the difficulty of reconstruction (Rampone et al., 2019; Katus et al., 2020; Li et al., 2020).

The ERP analysis procedure was as follows. (1) The EEG signals were divided into epochs of 1,000ms based on the event (200ms before target onset to 800ms after target onset) and averaged. (2) Average ERPs were constructed from trials containing 1,000ms epochs of raw data, of which the pre-stimulus period (first 200ms) was used to correct the baseline averaged ERP epochs lasting 800ms (Mitchell et al., 2016). (3) Next, the P600 latency and amplitude were detected from the averaged ERP epochs. The P600 latency and amplitude were determined from its highest point and mean amplitude, respectively, within a time window between 530 and 750ms following stimulus presentation (Causse et al., 2016) at the F3, F4, C3, C4, P3, P4, O1, and O2 electrodes (Mun et al., 2014; Park et al., 2015, 2019). Each electrode site corresponds to a brain region identified by a letter: frontal (F), central (C), parietal (P), and occipital (O). These brain areas are associated with the following functions: (1) the frontal area is associated with reasoning, motor skills, higher level cognition, and expressive language; (2) the central area is associated with motor and sensory information; (3) the parietal area is associated with processing of tactile sensations; and (4) the occipital area is associated with interpreting visual stimuli and information (Dimond and Beaumont, 1974). All signal processing and data analyses were performed using EEGlab, a MATLAB toolbox (2020b, Mathworks Inc., Natick, MA, United States).

Participants were required to perform all the 375 trials for the ERP task. In order to assess the effect of the evoked potentials caused by heartbeat (i.e., the alpha activation of HEPs) on the latency and amplitude of the P600 component in ERPs, and the classification performance in distinguishing between low- and high-MWL states, the entire ERP epochs were categorized into two conditions based on the heartbeat within each ERP epoch: including the heartbeat within the ERP epoch (denoted ERP_HEP) or not including the heartbeat within the ERP epoch (denoted ERP_A-HEP, for anti-HEP). ERP_HEP and ERP_A-HEP accounted for 171.79±10.31 and 203.21±10.31 trials, respectively, on average, for each subject. More specifically, ERP_HEP was defined as containing the heartbeat during the period from 280 to 700ms in the ERP epochs after the target was presented. This period was determined by considering the interval in which the evoked potentials overlapped the heartbeat (50–250ms after the heartbeat) and events related to P600 (530–750ms after target). ERP_A-HEP was defined as not including the heartbeat within the same period. Finally, we defined the ERP_T condition as including all the ERP_A-HEP and ERP_HEP trials. We divided the ERP_A-HEP, ERP_HEP, and ERP_T trials from entire ERP epochs, and compared their classification performance in separating low- and high-MWL states and the pattern of latency and amplitude of the P600 component, as shown in Figure 4.

FIGURE 4

Figure 4. Examples of signal processing compared across ERPA-HEP, ERPHEP, and ERPT.

Statistical Analysis and Classification

This study followed a within-subject design with respect to low- and high-MWL. A paired t-test was used based on the results of the Shapiro–Wilk normality test. Differences between pre- and post-features were calculated to take into account the state before the task. The confidence level in the statistical tests was controlled by the number of individual hypotheses (i.e., α=05∕n) based on the Bonferroni correction. Bonferroni correction was performed to assess statistical significance while correcting for multiple comparisons (Dunnett, 1955; Johnson et al., 2019). Thus, the statistically significant levels of performance and ERP measures were set to 0.025 (accuracy and response time, α=0.05/2=0.025) and 0.0031 (ERP latency and amplitude at eight electrodes, α=0.05/16=0.0031). The effect size based on Hedges’ g was calculated to assess not only the statistical significance, but also the practical significance (Hedges and Olkin, 2014). All effect sizes were corrected for small sample sizes according to Hedges’ g. We also calculated the expected effect size for the paired t-test (Hedges’ g) using G*power software (Faul et al., 2007). The expected effect size in this study was 0.781 (paired t-test).

In EEG research, the radial basis function kernel-based support vector machine (RBF-SVM) is considered one of the optimized classifiers (Alomari et al., 2013). We used RBF-SVM to conduct a binary classification for a total of 28 samples × number of features after feature selection and standardization. The optimized kernel scales for each condition were as follows: ERP_T, 117.8; ERP_HEP, 2.7; and ERP_A-HEP, 21.5. We conducted 10-fold cross-validation and represented the performance of the classification using accuracy, sensitivity, specificity, and area under the curve (AUC) for the receiver operating characteristics curve. Statistical analysis and classification were performed using the Statistics and Machine Learning Toolbox in MATLAB (2020b, Mathworks Inc., Natick, MA, United States).

Results

Subjective Ratings: SMEQ

Figure 5 represents the comparison of the SMEQ scores between low- and high-MWL conditions. In the paired t-test, the SMEQ score of the high-MWL condition was significantly higher than that of the low-MWL condition [t(13)=−9.238, Hedges’ g=3.796, 95% CI 2.556–5.036, p<0.001]. The mean (M) and standard deviation (SD) for each condition were as follows: low-MWL, M=13.79 and SD=6.37; high-MWL, M=71.64 and SD=20.59. Hedges’ g satisfied the expected effect size for the paired t-test in this study.

FIGURE 5

Figure 5. Comparison of accuracy (A), response time (B) for target in ERP task, and SMEQ score (C) between the low- and high-MWL conditions (paired t-test; ^**p<0.0031 and ^***p<0.001).

ERP Task Performance: Accuracy and Response Time

A paired t-test of target accuracy in the ERP task showed a significant difference between the low- (M=−1.27, SD=3.29) and high-MWL (M=−10.13, SD=8.38) conditions [t(13)=3.762, Hedges’ g=−1.392, 95% CI –2.217 to −0.566, p<0.01]. The response time of the high-MWL (M=14.30, SD=19.54) condition was significantly delayed compared to the low-MWL (M=103.67, SD=57.76) condition [t(13)=−4.643, Hedges’ g=2.073, 95% CI 1.154 to −2.991, p<0.001], as shown in Figure 5. Hedges’ g for the performance of the ERP task satisfied the expected effect size for the paired t-test in this study.

ERP Waveform: P600 Amplitude and Latency

P600 Amplitude

Figure 6 represents the results of the statistical analysis of the P600 amplitudes, comparing the low- and high-MWL conditions, for the ERP_T, ERP_A-HEP, and ERP_HEP epochs. In the cases of ERP_T and ERP_HEP, no significant differences between low- and high-MWL conditions were found at any electrode site (F3, F4, C3, C4, P3, P4, O1, and O2) after Bonferroni correction (p>0.0031). However, in the case of ERP_A-HEP, a paired t-test revealed a significant difference between the P600 amplitudes of the low- and high-MWL conditions at F3 [t(13)=5.505, Hedges’ g=−1.988, 95% CI –2.893 to −1.082, p<0.001], F4 [t(13)=4.787, Hedges’ g=−1.265, 95% CI –2.076 to −0.453, p<0.001], P4 [t(13)=4.383, Hedges’ g=−1.559, 95% CI –2.281 to −0.616, p<0.001), and O1 [t(13)=3.818, Hedges’ g=−1.391, 95% CI –2.217 to −0.566, p<0.0031]. No significant differences were found at the other electrode sites (C3, C4, P3, and O2). Detailed results are presented in Table 1. Hedges’ g for the P600 amplitude satisfied the expected effect size for the paired t-test.

FIGURE 6

Figure 6. Comparison of the P600 amplitude in ERP between the low- and high-MWL conditions. (A) ERPT condition. (B) ERPHEP condition. (C) ERPA-HEP condition (paired t-test; ^*p<0.05, ^**p<0.0031, and ^***p<0.001).

TABLE 1

Table 1. Comparison by paired t-test of the P600 amplitude between low- and high-MWL conditions.

P600 Latency

Figure 7 represents the results of the statistical analysis comparing the P600 latency between the low- and high-MWL conditions for the ERP_T, ERP_A-HEP, and ERP_HEP epochs. In the case of ERP_T, a paired t-test revealed a significant difference between low- and high-MWL conditions at the O1 site only [t(13)=−3.935, Hedges’ g=1.462, 95% CI 0.628 to 2.296, p<0.0031], while no significant differences were found at the other electrode sites. In the case of ERP_HEP, no significant differences were observed between the P600 latencies of low- and high-MWL conditions at any electrode site after Bonferroni correction (p>0.0031). For the ERP_A-HEP, P600 latency was significantly prolonged in the high-MWL condition compared with the low-MWL condition at F3 [t(13)=−4.348, Hedges’ g=1.823, 95% CI 0.942 to 2.704, p<0.001], F4 [t(13)=−3.833, Hedges’ g=1.533, 95% CI 0.690 to 2.375, p<0.001], P4 [t(13)=−4.283, Hedges’ g=1.662, 95% CI 0.803 to 2.521, p<0.001], O1 [t(13)=−5.115, Hedges’ g=1.714, 95% CI 0.848 to 2.581, p<0.001], and O2 [t(13)=−5.526, Hedges’ g=1.871, 95% CI 0.983 to 2.760, p<0.001] sites. No significant differences were found at the other electrode sites (C3, C4, and P3). Detailed results are presented in Table 2. Hedges’ g for the P600 latency satisfied the expected effect size for the paired t-test.

FIGURE 7

Figure 7. Comparison of the P600 latency in ERP between the low- and high-MWL conditions. (A) ERPT condition. (B) ERPHEP condition. (C) ERPA-HEP condition (paired t-test; ^*p<0.05, ^**p<0.0031, and ^***p<0.001).

TABLE 2

Table 2. Comparison by paired t-test of the P600 latency the low- and high-MWL conditions.

To visually confirm the pattern of ERP features between low- and high-MWL, we produced scatterplots with amplitude and latency as the two axes for the ERP_T, ERP_HEP, and ERP_A-HEP conditions. As shown in Figure 8, only the ERP_A-HEP condition revealed a clear pattern distinguishing the two MWL states. A paired t-test of the heart rate showed no significant difference between the low-MWL (0.81±0.05) and high-MWL (0.79±0.05) conditions [t(13)=1.479, p =0.163].

FIGURE 8

Figure 8. Scatterplots of P600 amplitude (X axis) and latency (Y axis) between low- and high-MWL for the ERPT, ERPHEP, and ERPA-HEP conditions.

Classification and Correlation Results

Classification

We compared the classification performance of ERP_T, ERP_HEP, and ERP_A-HEP in distinguishing between low- and high-MWL in order to assess the effect of the HEP on the predictive power of ERPs (10-fold cross-validation). The RBF-SVM was selected as the classification method since it is widely used in EEG-related studies. In the three conditions (ERP_T, ERP_HEP, and ERP_A-HEP), for all ERP amplitude and latency, the following values of the performance metrics were achieved as: accuracy: 85.7,% 71.4, and 100%, respectively; sensitivity, 85.7, 64.3, and 100%; specificity, 85.7, 78.6, and 100%; and AUC: 0.93, 0.76, and 1, as shown in Table 3 and Figure 9.

TABLE 3

Table 3. Results of the classification by RBF-SVM (10-fold cross-validation) with ERP_T, ERP_HEP, and ERP_A-HEP epochs (n=24).

FIGURE 9

Figure 9. Receiver operating characteristics curves for ERPT, ERPHEP, and ERPA-HEP according to the RBF-SVM classifier.

Correlation

We conducted a correlation analysis among ERP features and SMEQ scores and compared the correlation coefficients for the ERP_T, ERP_HEP, and ERP_A-HEP conditions. Specifically, we assessed the partial correlation between post-ERP features and post-SMEQ scores while controlling for two covariates (pre-ERP features and pre-SMEQ scores) to take into account the state before the experiment (Liu, 1988). In the ERP_T, ERP_HEP, and ERP_A-HEP conditions, we found significant correlations between SMEQ score and seven, four, and ten ERP features, respectively. The correlation coefficients in the ERP_A-HEP condition (amplitude: 0.410 to 0.885; latency: −0.401 to −0.586) were higher than those of the ERP_T (amplitude: 0.414 to 0.497; latency: −0.432 to −0.483) and ERP_HEP (amplitude: 0.440; latency: −0.419 to −0.493) conditions. Detailed correlation results are presented in Table 4.

TABLE 4

Table 4. Partial correlation analysis between ERP features and SMEQ scores for each condition (ERP_T, ERP_HEP, and ERP_A-HEP).

Discussion and Conclusion

This study sought to determine the effect of HEP on the amplitude and latency of the P600 component of ERPs and to compare the classification accuracy for the MWL task among the ERP_T, ERP_HEP, and ERP_A-HEP conditions. SMEQ score and MWL performance (accuracy and response time for target) were significantly different between low- and high-MWL tasks. These results confirmed that a difference in the MWL state of participants resulted from low- and high-mental arithmetic tasks. The P600 ERP amplitude and latency were significantly lower and higher, respectively, in the high-MWL than in the low-MWL tasks. This result is consistent with that of previous studies (Baetens et al., 2011; Mun et al., 2012, 2017; Park et al., 2015, 2019). Late positive potentials (LPPs), such as the P600 and P700 components reflect a high-level MWL required to process difficult tasks (Li et al., 2008; Pastor et al., 2008; Mun et al., 2012, 2014). LPPs are seen as a delayed P300 component and are related to high-level MWL (Friederici et al., 1993; Mun et al., 2012). Reduced LPP amplitudes reflect a decrease in cognitive and neural resources caused by MWL, impairing the attentional allocation mechanisms (Kato et al., 2009; Mun et al., 2012, 2014).

The analysis revealed significant differences in the P600 amplitude and latency among the ERP_T, ERP_HEP, and ERP_A-HEP conditions. The number of features (i.e., P600 amplitude and latency in each brain region) with significant differences (p <0.0031) between low- and high-MWL was greater in the ERP_A-HEP (10 significant features) than in the other conditions (ERP_T, one significant feature; ERP_HEP, no significant feature). The P600 amplitude and latency in the ERP_A-HEP condition revealed a stronger correlation with SMEQ scores than the other two conditions. These results suggest that the pattern of P600 amplitude and latency in the ERP_A-HEP condition, not affected by the heartbeat, revealed a clearer response to ERP stimuli, without interference from other evoked potentials, compared with other conditions that were affected by the heartbeat.

The vagus nervous system in the heart continuously communicates with the brain via efferent and afferent pathways. Neuronal connectivity causes an evoked potential (i.e., alpha rhythms) in brain waves, known as the HEP (Schandry and Montoya, 1996; McCraty et al., 2009; Park et al., 2015). Many previous studies have used the HEP phenomenon in various research fields, such as attention (Petzschner et al., 2019), anxiety (Judah et al., 2018; Pang et al., 2019), mental workload (Park et al., 2015), emotion (Couto et al., 2015), sleep (Lechinger et al., 2015), and medicine (Leopold and Schandry, 2001; Perogamvros et al., 2019; Flasbeck et al., 2020).

We divided the ERP trials into three conditions according to whether the heartbeat had an effect on the evoked potential on EEG oscillations. The condition related to the heartbeat (ERP_HEP) resulted in a change in the P600 amplitude for the target due to the overlap with the evoked potentials induced by the heartbeat, which also led to variations in latency. The phenomenon whereby the evoked potentials caused by the heartbeat affect EEG oscillations has been reported in many previous studies (Wölk et al., 1989; Schandry and Montoya, 1996; McCraty et al., 2009; Park et al., 2015, 2019; Villena-Gonzalez et al., 2017; Perogamvros et al., 2019). Our argument regarding the effect of cardiac activity on the P600 in ERP is supported by the high statistical significance and correlation coefficient we observed in the ERP_A-HEP condition compared with the ERP_HEP one.

This study also assessed the classification performance using P600 features between low- and high-MWL for the three conditions (ERP_T, ERP_HEP, and ERP_A-HEP), finding that the accuracy in the ERP_A-HEP condition increased by 14.3 and 28.6% compared to the ERP_T and ERP_HEP conditions, respectively. No previous studies have tried to detect the pure ERP components and improve the MWL classification performance based on ERP while considering the effect of the evoked potentials (i.e., HEP) caused by the cardiac activity (i.e., heartbeat). However, some studies have attempted to improve the accuracy of brain activity classification using approaches similar to that used in our studies. Brain oscillations can be affected by various mental states, such as fatigue, stress, emotion, and mental workload. Changes in mental state can affect EEG oscillations and interfere with the targeting response. This leads to a decrease in classification performance (Lorenz et al., 2014; Myrden and Chau, 2017; Zhang et al., 2020; Zheng et al., 2020). A previous study compared brain-computer interface (BCI) performance under low- and high-stress conditions and reported that the use of steady-state visually evoked potential-based BCI under stress leads to a decrease in accuracy and an increase in the required concentration and the resulting fatigue (Zhang et al., 2020). Another study revealed an increase in BCI performance considering the mental focus and lost-in-thought states of participants (Ko et al., 2020). In addition, many studies have sought to improve performance by considering various mental states (Myrden and Chau, 2017; Acı et al., 2019; Zhang et al., 2019; Foong et al., 2020), which is highly relevant to our research approach. Therefore, we believe that the evoked potential caused by the heartbeat must be considered to improve the detection of the evoked potential components in ERPs and that our approach can contribute to the measurement of accurate ERP responses and improve the performance in classifying MWL.

However, this study has several limitations. (1) Only the effects of the HEP on the P600 element of ERP were confirmed, and other components, such as P300, N400, and P200, were not considered. Since our approach selected trials suitable for each ERP element based on the heartbeat, other ERP components may behave similarly to the P600 component, although this needs to be confirmed through further research based on various task environments. (2) Our study sought to confirm a significant difference in MWL classification performance by ERP trials according to whether or not the HEP was affected. In future research, we will conduct a paradigm study that controls the timing of stimuli in ERP tasks based on heartbeats, or a method of training by classifying ERP trials based on whether they are affected by HEP.

In conclusion, this study confirmed that the performance in the classification of MWL states in the ERP_A-HEP condition was significantly superior to that of the ERP_T and the ERP_HEP conditions. We interpret these results as showing that the pattern of ERPs in the ERP_T and the ERP_HEP conditions, which were affected by the heartbeat, resulted from the overlap of the HEP and the ERPs. On the other hand, the pattern of ERPs in the ERP_A-HEP condition, which was not affected by the heartbeat, showed a clear or pure response to ERP stimuli without the effect of other evoked potentials. Therefore, in ERP studies, the effect of HEPs on ERP patterns (i.e., amplitude and latency in ERP components) needs to be considered in order to obtain a clear and pure ERP response. This study used the P600 component to improve ERP-based MWL classification performance, but the same approach can be used in the application of ERPs to various fields, such as brain-computer interface, emotion recognition, language processing, working memory, and neurotherapy.

Data Availability Statement

The data generated and analyzed in this study are available from the corresponding author upon reasonable request.

Ethics Statement

All experimental protocols were approved by the Sangmyung University Institutional Bioethics Review Board (SMUIBRB) in Seoul, South Korea (BE2019-46). The patients/participants provided their written informed consent to participate in this study.

Author Contributions

SP: conceptualization, methodology, data analysis, experiment, and writing – original draft. JH: investigation, visualization, data analysis, and experiment. LK: conceptualization, writing – review and editing, and supervision. All authors contributed to the article and approved the submitted version.

Funding

This work was partly supported by the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korean Government (MSIT) (No. 2017-0-00432), for the development of a non-invasive integrated BCI SW platform to control home appliances and external devices by user’s thoughts via AR/VR interface.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Abbreviations

HEPs, heartbeat-evoked potentials; ERPs, event-related potentials; MWL, mental workload; ERP_HEP, containing the heartbeat in ERP epochs; ERP_A-HEP, not containing the heartbeat in ERP epochs; ERP_T, ERP_A-HEP and ERP_HEP together; RBF-SVM, radial basis function-support vector machine; EEG, electroencephalogram; SMEQ, subjective mental effort questionnaire; CAR, common average referencing; ICA, independent component analysis; F, frontal region; C, central region; P, parietal region; O, occipital region; AUC, area under curve; ROC, receiver operating characteristics; LPPs, late positive potentials; BCI, brain-computer interface; SVEP, steady-state visually evoked potential.

References

Acı, É. İ., Kaya, M., and Mishchenko, Y. (2019). Distinguishing mental attention states of humans via an EEG-based passive BCI using machine learning methods. Expert Syst. Appl. 134, 153–166. doi: 10.1016/j.eswa.2019.05.057

CrossRef Full Text | Google Scholar

Alomari, H. M., Samaha, A., and Alkamha, K. (2013). Automated classification of L/R hand movement EEG signals using advanced feature extraction and machine learning. Int. J. Adv. Comput. Sci. Appl. 4.

Google Scholar

Baetens, K., Der Cruyssen, L. V., Achtziger, A., Vandekerckhove, M., and Van Overwalle, F. (2011). N400 and LPP in spontaneous trait inferences. Brain Res. 1418, 83–92. doi: 10.1016/j.brainres.2011.08.067

PubMed Abstract | CrossRef Full Text | Google Scholar

Boudewyn, M. A., Luck, S. J., Farrens, J. L., and Kappenman, E. S. (2018). How many trials does it take to get a significant ERP effect? It depends. Psychophysiology 55:e13049. doi: 10.1111/psyp.13049

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, X., Xu, X. Y., Liu, A. P., Lee, S., Chen, X., Zhang, X., et al. (2019). Removal of muscle artifacts from the EEG: A review and recommendations. IEEE Sensors J. 19, 5353–5368. doi: 10.1109/JSEN.2019.2906572

CrossRef Full Text | Google Scholar

Coles, M. G., and Rugg, M. D. (1995). Event-Related Brain Potentials: An Introduction. Oxford: Oxford University Press.

Google Scholar

Couto, B., Adolfi, F., Velasquez, M., Mesow, M., Feinstein, J., Canales-Johnson, A., et al. (2015). Heart evoked potential triggers brain responses to natural affective scenes: A preliminary study. Auton. Neurosci. 193, 132–137. doi: 10.1016/j.autneu.2015.06.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Dai, C. X., Wang, J. J., Xie, J. L., Li, W. M., Gong, Y. S., and Li, Y. Q. (2019). Removal of ECG artifacts From EEG using an effective recursive least square notch filter. IEEE Access 7, 158872–158880. doi: 10.1109/ACCESS.2019.2949842

CrossRef Full Text | Google Scholar

Dimigen, O. (2020). Optimizing the ICA-based removal of ocular EEG artifacts from free viewing experiments. NeuroImage 207:116117. doi: 10.1016/j.neuroimage.2019.116117

PubMed Abstract | CrossRef Full Text | Google Scholar

Dimond, S. J., and Beaumont, J. (1974). Hemisphere Function in the Human Brain John Wiley & Sons.

Google Scholar

Dunnett, C. W. (1955). A multiple comparison procedure for comparing several treatments with a control. J. Am. Stat. Assoc. 50, 1096–1121. doi: 10.1080/01621459.1955.10501294

CrossRef Full Text | Google Scholar

Egambaram, A., Badruddin, N., Asirvadam, V. S., Begum, T., Fauvet, E., and Stolz, C. (2020). FastEMD-CCA algorithm for unsupervised and fast removal of eyeblink artifacts from electroencephalogram. Biomed. Signal Process. Control 57:101692. doi: 10.1016/j.bspc.2019.101692

CrossRef Full Text | Google Scholar

Faul, F., Erdfelder, E., Lang, A. G., and Buchner, A. (2007). G*power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav. Res. Methods 39, 175–191. doi: 10.3758/BF03193146

PubMed Abstract | CrossRef Full Text | Google Scholar

Flasbeck, V., Popkirov, S., Ebert, A., and Brune, M. (2020). Altered interoception in patients with borderline personality disorder: a study using heartbeat-evoked potentials. Borderline Personality Disord. Emotion Dysregulation 7, 1–13.

Google Scholar

Foong, R., Ang, K. K., Quek, C., Guan, C., Phua, K. S., Kuah, C. W. K., et al. (2020). Assessment of the efficacy of EEG-based MI-BCI With visual feedback and EEG correlates of mental fatigue for upper-limb stroke rehabilitation. I.E.E.E. Trans. Biomed. Eng. 67, 786–795. doi: 10.1109/TBME.2019.2921198

PubMed Abstract | CrossRef Full Text | Google Scholar

Friederici, A. D., Pfeifer, E., and Hahne, A. (1993). Event-related brain potentials during natural speech processing: effects of semantic, morphological and syntactic violations. Brain Res. Cogn. Brain Res. 1, 183–192. doi: 10.1016/0926-6410(93)90026-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Hamaneh, M. B., Chitravas, N., Kaiboriboon, K., Lhatoo, S. D., and Loparo, K. A. (2014). Automated removal of EKG artifact from EEG data using independent component analysis and continuous wavelet transformation. I.E.E.E. Trans. Biomed. Eng. 61, 1634–1641. doi: 10.1109/TBME.2013.2295173

PubMed Abstract | CrossRef Full Text | Google Scholar

Hedges, L. V., and Olkin, I. (2014). Statistical Methods for Meta-Analysis Academic Press.

Google Scholar

Janig, W. (1996). Neurobiology of visceral afferent neurons: neuroanatomy, functions, organ regulations and sensations. Biol. Psychol. 42, 29–51. doi: 10.1016/0301-0511(95)05145-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Johnson, J. L., Slentz, C. A., Ross, L. M., Huffman, K. M., and Kraus, W. E. (2019). Ten-year legacy effects of three eight-month exercise training programs on Cardiometabolic health parameters. Front. Physiol. 10:452. doi: 10.3389/fphys.2019.00452

CrossRef Full Text | Google Scholar

Jost, P., Cobb, S., and Hämmerle, I. (2019). Reality-based interaction affecting mental workload in virtual reality mental arithmetic training. Behav. Inform. Technol. 39, 1062–1078. doi: 10.1080/0144929X.2019.1641228

CrossRef Full Text | Google Scholar

Judah, M. R., Shurkova, E. Y., Hager, N. M., White, E. J., Taylor, D. L., and Grant, D. M. (2018). The relationship between social anxiety and heartbeat evoked potential amplitude. Biol. Psychol. 139, 1–7. doi: 10.1016/j.biopsycho.2018.09.013

PubMed Abstract | CrossRef Full Text | Google Scholar

Kato, Y., Endo, H., and Kizuka, T. (2009). Mental fatigue and impaired response processes: event-related brain potentials in a Go/NoGo task. Int. J. Psychophysiol. 72, 204–211. doi: 10.1016/j.ijpsycho.2008.12.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Katus, L., Mason, L., Milosavljevic, B., Mccann, S., Rozhko, M., Moore, S. E., et al. (2020). ERP markers are associated with neurodevelopmental outcomes in 1-5 month old infants in rural Africa and the UK. NeuroImage 210:116591. doi: 10.1016/j.neuroimage.2020.116591

PubMed Abstract | CrossRef Full Text | Google Scholar

Ko, L. W., Chikara, R. K., Lee, Y. C., and Lin, W. C. (2020). Exploration of user’s mental state changes during performing brain-computer Interface. Sensors 20:3169. doi: 10.3390/s20113169

PubMed Abstract | CrossRef Full Text | Google Scholar

Lechinger, J., Heib, D. P., Gruber, W., Schabus, M., and Klimesch, W. (2015). Heartbeat-related EEG amplitude and phase modulations from wakefulness to deep sleep: interactions with sleep spindles and slow oscillations. Psychophysiology 52, 1441–1450. doi: 10.1111/psyp.12508

PubMed Abstract | CrossRef Full Text | Google Scholar

Leopold, C., and Schandry, R. (2001). The heartbeat-evoked brain potential in patients suffering from diabetic neuropathy and in healthy control persons. Clin. Neurophysiol. 112, 674–682. doi: 10.1016/S1388-2457(01)00480-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Lew, E., Chavarriaga, R., Silvoni, S., and Millan Jdel, R. (2012). Detection of self-paced reaching movement intention from EEG signals. Front. Neuroeng. 5:13. doi: 10.3389/fneng.2012.00013

CrossRef Full Text | Google Scholar

Li, H.-C. O., Seo, J., Kham, K., and Lee, S. (2008). “Measurement of 3D visual fatigue using event-related potential (ERP): 3D oddball paradigm”, in 2008 3DTV Conference: The True Vision-Capture, Transmission and Display of 3D Video: IEEE), 213–216.

Google Scholar

Li, F. L., Tao, Q., Peng, W. J., Zhang, T., Si, Y. J., Zhang, Y. S., et al. (2020). Inter-subject P300 variability relates to the efficiency of brain networks reconfigured from resting- to task-state: evidence from a simultaneous event-related EEG-fMRI study. NeuroImage 205:116285. doi: 10.1016/j.neuroimage.2019.116285

PubMed Abstract | CrossRef Full Text | Google Scholar

Liu, K. (1988). Measurement error and its impact on partial correlation and multiple linear regression analyses. Am. J. Epidemiol. 127, 864–874. doi: 10.1093/oxfordjournals.aje.a114870

PubMed Abstract | CrossRef Full Text | Google Scholar

Lorenz, R., Pascual, J., Blankertz, B., and Vidaurre, C. (2014). Towards a holistic assessment of the user experience with hybrid BCIs. J. Neural Eng. 11:035007. doi: 10.1088/1741-2560/11/3/035007

PubMed Abstract | CrossRef Full Text | Google Scholar

Luck, S. J. (2014). An Introduction to the Event-Related Potential Technique MIT Press.

Google Scholar

Mccraty, R., Atkinson, M., Tomasino, D., and Bradley, R. T. (2009). The coherent heart heart-brain interactions, psychophysiological coherence, and the emergence of system-wide order. Integral Rev. 5.

Google Scholar

Montoya, P., Schandry, R., and Müller, A. (1993). Heartbeat evoked potentials (HEP): topography and influence of cardiac awareness and focus of attention. Electroencephalography Clin. Neurophysiol./Evoked Potentials Sect. 88, 163–172. doi: 10.1016/0168-5597(93)90001-6

CrossRef Full Text | Google Scholar

Mun, S., Kim, E. S., and Park, M. C. (2014). Effect of mental fatigue caused by mobile 3D viewing on selective attention: an ERP study. Int. J. Psychophysiol. 94, 373–381. doi: 10.1016/j.ijpsycho.2014.08.1389

PubMed Abstract | CrossRef Full Text | Google Scholar

Mun, S., Park, M. C., Park, S., and Whang, M. (2012). SSVEP and ERP measurement of cognitive fatigue caused by stereoscopic 3D. Neurosci. Lett. 525, 89–94. doi: 10.1016/j.neulet.2012.07.049

PubMed Abstract | CrossRef Full Text | Google Scholar

Mun, S., Whang, M., Park, S., and Park, M. C. (2017). Effects of mental workload on involuntary attention: A somatosensory ERP study. Neuropsychologia 106, 7–20. doi: 10.1016/j.neuropsychologia.2017.08.021

PubMed Abstract | CrossRef Full Text | Google Scholar

Myrden, A., and Chau, T. (2017). A passive EEG-BCI for single-trial detection of changes in mental state. IEEE Trans. Neural Syst. Rehabil. Eng. 25, 345–356. doi: 10.1109/TNSRE.2016.2641956

PubMed Abstract | CrossRef Full Text | Google Scholar

Nieuwenhuys, R., Voogd, J., and Van Huijzen, C. (2007). The Human Central Nervous System: A Synopsis and Atlas Springer Science & Business Media.

Google Scholar

Pang, J., Tang, X., Li, H., Hu, Q., Cui, H., Zhang, L., et al. (2019). Altered interoceptive processing in generalized anxiety disorder-A heartbeat-evoked potential research. Front. Psych. 10:616. doi: 10.3389/fpsyt.2019.00616

CrossRef Full Text | Google Scholar

Park, S., Mun, S., Lee, D. W., and Whang, M. (2019). IR-camera-based measurements of 2D/3D cognitive fatigue in 2D/3D display system using task-evoked pupillary response. Appl. Opt. 58, 3467–3480. doi: 10.1364/AO.58.003467

PubMed Abstract | CrossRef Full Text | Google Scholar

Park, S., and Whang, M. (2018). Infrared camera-based non-contact measurement of brain activity From pupillary rhythms. Front. Physiol. 9:1400. doi: 10.3389/fphys.2018.01400

CrossRef Full Text | Google Scholar

Park, S., Won, M. J., Lee, E. C., Mun, S., Park, M. C., and Whang, M. (2015). Evaluation of 3D cognitive fatigue using heart-brain synchronization. Int. J. Psychophysiol. 97, 120–130. doi: 10.1016/j.ijpsycho.2015.04.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Park, S., Won, M. J., Mun, S., Lee, E. C., and Whang, M. (2014). Does visual fatigue from 3D displays affect autonomic regulation and heart rhythm? Int. J. Psychophysiol. 92, 42–48. doi: 10.1016/j.ijpsycho.2014.02.003

CrossRef Full Text | Google Scholar

Pastor, M. C., Bradley, M. M., Low, A., Versace, F., Molto, J., and Lang, P. J. (2008). Affective picture perception: emotion, context, and the late positive potential. Brain Res. 1189, 145–151. doi: 10.1016/j.brainres.2007.10.072

PubMed Abstract | CrossRef Full Text | Google Scholar

Perogamvros, L., Park, H. D., Bayer, L., Perrault, A. A., Blanke, O., and Schwartz, S. (2019). Increased heartbeat-evoked potential during REM sleep in nightmare disorder. Neuroimage Clin. 22:101701. doi: 10.1016/j.nicl.2019.101701

PubMed Abstract | CrossRef Full Text | Google Scholar

Perrin, F., Pernier, J., Bertrand, O., and Echallier, J. F. (1989). Spherical splines for scalp potential and current density mapping. Electroencephalogr. Clin. Neurophysiol. 72, 184–187. doi: 10.1016/0013-4694(89)90180-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Petzschner, F. H., Weber, L. A., Wellstein, K. V., Paolini, G., Do, C. T., and Stephan, K. E. (2019). Focus of attention modulates the heartbeat evoked potential. NeuroImage 186, 595–606. doi: 10.1016/j.neuroimage.2018.11.037

PubMed Abstract | CrossRef Full Text | Google Scholar

Rampone, G., Makin, A. D. J., Tatlidil, S., and Bertamini, M. (2019). Representation of symmetry in the extrastriate visual cortex from temporal integration of parts: An EEG/ERP study. NeuroImage 193, 214–230. doi: 10.1016/j.neuroimage.2019.03.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Sauro, J., and Dumas, J. S. (2009). “Comparison of three one-question, post-task usability questionnaires”, in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems), 1599–1608.

Google Scholar

Schandry, R., and Montoya, P. (1996). Event-related brain potentials and the processing of cardiac activity. Biol. Psychol. 42, 75–85. doi: 10.1016/0301-0511(95)05147-3

PubMed Abstract | CrossRef Full Text | Google Scholar

So, W. K. Y., Wong, S. W. H., Mak, J. N., and Chan, R. H. M. (2017). An evaluation of mental workload with frontal EEG. PLoS One 12:e0174949. doi: 10.1371/journal.pone.0174949

PubMed Abstract | CrossRef Full Text | Google Scholar

Uriguen, J. A., and Garcia-Zapirain, B. (2015). EEG artifact removal-state-of-the-art and guidelines. J. Neural Eng. 12:031001. doi: 10.1088/1741-2560/12/3/031001

PubMed Abstract | CrossRef Full Text | Google Scholar

Villena-Gonzalez, M., Moenne-Loccoz, C., Lagos, R. A., Alliende, L. M., Billeke, P., Aboitiz, F., et al. (2017). Attending to the heart is associated with posterior alpha band increase and a reduction in sensitivity to concurrent visual stimuli. Psychophysiology 54, 1483–1497. doi: 10.1111/psyp.12894

PubMed Abstract | CrossRef Full Text | Google Scholar

Wölk, C., Velden, M., Zimmermann, U., and Krug, S. (1989). The interrelation between phasic blood pressure and heart rate changes in the context of the “baroreceptor hypothesis”. J. Psychophysiol. 3, 397–402.

Google Scholar

Zhang, H. Y., Stevenson, C. E., Jung, T. P., and Ko, L. W. (2020). Stress-induced effects in resting EEG spectra predict the performance of SSVEP-based BCI. IEEE Trans. Neural Syst. Rehabil. Eng. 28, 1771–1780. doi: 10.1109/TNSRE.2020.3005771

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhang, T., Zhang, X., Zhang, Y., Lu, Z., and Li, H. (2019). “Effects of user fatigue mental state on the facial-expression paradigm of BCI”, in 2019 WRC Symposium on Advanced Robotics and Automation (WRC SARA): IEEE), 394–400.

Google Scholar

Zheng, X., Xu, G., Zhang, Y., Liang, R., Zhang, K., Du, Y., et al. (2020). Anti-fatigue performance in SSVEP-based visual acuity assessment: A comparison of six stimulus paradigms. Front. Hum. Neurosci. 14:301. doi: 10.3389/fnhum.2020.00301

CrossRef Full Text | Google Scholar

Zou, L., Chen, X., Dang, G., Guo, Y., and Wang, Z. J. (2020). Removing muscle artifacts From EEG data via underdetermined joint blind source separation: A simulation study. IEEE Trans. Circuits Syst. II Express Briefs 67, 187–191. doi: 10.1109/TCSII.2019.2903648

CrossRef Full Text | Google Scholar