Patient-specific approach using data fusion and adversarial training for epileptic seizure prediction

Epilepsy is the second common neurological disorder after headache, accurate and reliable prediction of seizures is of great clinical value. Most epileptic seizure prediction methods consider only the EEG signal or extract and classify the features of EEG and ECG signals separately, the improvement of prediction performance from multimodal data is not fully considered. In addition, epilepsy data are time-varying, with differences between each episode in a patient, making it difficult for traditional curve-fitting models to achieve high accuracy and reliability. In order to improve the accuracy and reliability of the prediction system, we propose a novel personalized approach based on data fusion and domain adversarial training to predict epileptic seizures using leave-one-out cross-validation, which achieves an average accuracy, sensitivity and specificity of 99.70, 99.76, and 99.61%, respectively, with an average error alarm rate (FAR) of 0.001. Finally, the advantage of this approach is demonstrated by comparison with recent relevant literature. This method will be incorporated into clinical practice to provide personalized reference information for epileptic seizure prediction.

1. Introduction

Epilepsy is a chronic disorder in which abnormal neuronal discharges occur suddenly, causing temporary brain dysfunction. According to the World Health Organization (WHO), there are approximately 70 million people with epilepsy worldwide, and epilepsy has become the second common neurological disorder after headache (Rasheed et al., 2021). In addition, before an epileptic seizure occurs, there are abnormalities in the EEG (electroencephalogram) and ECG (electrocardiogram) that make it possible to predict epileptic seizures (Rakhade and Jensen, 2009). If the seizure can be predicted in advance, there is enough time for the medical staff to intervene with the patient’s medication or to apply electrical stimulation treatment to protect the patient from side effects.

Electroencephalogram is a method of recording brain activity using electrophysiological indicators (Zhang et al., 2020) and is the overall reflection of the electrophysiological activity of brain nerve cells in the cerebral cortex. In recent years, the use of EEG for seizure detection and prediction has received widespread attention from the academic community. Based on the dynamic characteristics of the EEG signal, brain activity in epileptic situations can be classified into four phases (Jana and Mukherjee, 2021a,b): preictal, ictal, postictal, and interictal. Studies have shown that the transition from interictal to ictal state in patients with epilepsy is not instantaneous, but rather goes through a transitional period, so accurate and efficient identification of the preictal state becomes the key to epileptic seizure prediction (Yuan et al., 2018). According to previous research, ECG signal analysis can help predict epileptic seizures. Some studies found significant changes along with the preictal parts of ECG signals by analyzing the parameters of heart rate variability and comparing them statistically in the preictal and interictal parts. These changes indicate high sympathetic activity and occur especially 5 min before the onset of seizures. Therefore, ECG signal analysis can be a beneficial tool for predicting epileptic seizures (Seifi et al., 2022).

The pathogenesis of epilepsy is complex and the types of epileptic seizures are diverse. In addition, since the patient’s status during each seizure cycle is different, such as physical condition, pathogenesis, seizure intensity, seizure type, environmental influences, mood, etc., these multiple factors lead to the preictal and interictal periods before each ictal having different characteristics. Therefore, in the same patient, the signal characteristics of the preictal period and the interictal period preceding each ictal may be the same or different (Babb et al., 1987; Fisher et al., 2017).

Most of the existing epileptic seizure prediction methods focus on the patient-specific scenario (Detti et al., 2018) which refers to predicting a patient’s epileptic seizure by learning from his own historical records, this method is easy to implement and has high prediction accuracy, and is favored by a wide range of researchers, but the accuracy is unstable, that is, large differences in results between patients, so the traditional methods need further improvement.

To eliminate the negative effect of data distribution shift, DANN (domain-adversarial training of neural networks) (Ganin et al., 2016; Yu et al., 2019; Matsuura and Harada, 2020) used marginal distribution alignment to eliminate data differences between subjects, and MADA (multi-adversarial domain adaptation) (Pei et al., 2018) used conditional distribution alignment to achieve better invariant feature learning. However, these proposed methods were only for eliminating differences between patients and did not address the negative effect of multiple seizures in a patient to predict epilepsy, resulting in unstable prediction results on the patient-specific scenario.

To further improve the prediction accuracy and reliability, data fusion is an effective way; in actual EEG acquisition, simultaneous acquisition of ECG data can meet the requirement of data fusion; in addition, there is increasing evidence that the nervous system plays an important role in regulating cardiac function, and seizure-onset arrhythmias have been shown to be the result of autonomic imbalance induced by seizure activity. Studies have shown that seizures can be predicted using the ECG (Moridani and Farhadi, 2017; Ufongene et al., 2020; Costagliola et al., 2021), so combining the EEG with the ECG signal not only improves the accuracy of seizure prediction, but also allows better detection and understanding of brain-heart interactions for monitoring and treatment of potential arrhythmias.

Deep learning (DL), a branch of machine learning (ML), has become increasingly popular in recent years. DL has been shown to outperform traditional ML in many areas, and through the efforts of researchers, the DL method has been extensively applied to the seizure prediction task and has achieved promising results (Jana and Mukherjee, 2021a,b; Usman et al., 2021; Zhang et al., 2021). However, most of the current methods use only EEG signals for seizure prediction and do not consider the effect of multimodal data on prediction performance.

In studies using simultaneous EEG and ECG signals for epileptic seizure prediction, Phomsiricharoenphant et al. (2014) used the Hilbert Huang Transform (HHT) to extract the instantaneous mean frequency from EEG and the R-R interval from ECG simultaneously for epileptic seizure prediction independently, the possibility of using EEG and ECG for seizure prediction was first explored; Hoyos-Osorio et al. (2016) used EEG features extracted by discrete wavelet transform and ECG features extracted by calculating heart rate changes to predict epileptic seizures independently; Billeci et al. (2019) used a combination of EEG and ECG to extract synchronous mode and time-frequency features from EEG, and extracted inter-beat (RR) information from ECG using recurrence quantification analysis to predict epileptic seizures independently, and used SVM classifier to classify preictal and interictal phases by combining features extracted from both signals. Most of these methods analyze the EEG and ECG data independently and derive the final prediction results independently, without sufficiently considering the combined EEG and ECG data for seizure prediction, and the prediction performance could be further improved.

Therefore, in order to improve prediction accuracy and reliability, eliminate the negative impact of data differences of each seizure in a patient, it is necessary to develop a personalized epileptic prediction approach with data fusion and adversarial training method for specific patient.

The rest of this paper is organized as follows. In section “2. Scalp EEG dataset and methods,” the Scalp EEG dataset and the epileptic seizure prediction method based on data fusion and adversarial training is described in details. In section “3. Results and discussion,” results and discussion on benchmark dataset are provided. In the end, some conclusions are given in section “4. Conclusion.”

2. Scalp EEG dataset and methods

2.1. Scalp EEG data

The proposed approach is evaluated on a benchmark dataset published by the University of Siena, Italy. This dataset consists of 9 males (36–71), 5 females (20–58), collected from scalp electrodes using the international 10–20 system at 512 Hz sampling rate, most of the data include EEG and ECG signals simultaneously. Personal information and epileptic seizure information are shown in Table 1.

TABLE 1

Table 1. Siena scalp EEG dataset information.

For each patient, the minimum length of the preictal and interictal periods is set at 5 min, the preictal and interictal periods before each seizure as a unit and recorded as an episode; we evaluated patients who had at least three seizures in the EEG recordings. This is because less than three preictal and interictal periods cause an overfitting problem in training. Taking all these restrictions into account, eight patients are available, such as P00, P05, P06, P09, P10, P12, P13, and P14.

2.2. Epileptic seizure prediction method based on data fusion and adversarial training

The proposed epileptic seizure prediction method explores the data fusion strategy of EEG and ECG signal and eliminates the negative impact of data difference of each seizure in a patient. It consists of three sections (Figure 1): (1) EEG feature extraction and classification module; (2) ECG feature extraction and classification module; and (3) decision level fusion module. The EEG feature extraction and classification module is composed of a long short-term memory network (LSTM) and four layers of 1-D CNN, and the ECG feature extraction and classification module is composed of five layers of 2-D CNN, the decision-level fusion module combines the output results of the EEG and ECG classification modules according to dynamic weight, so that the EEG classification module and the ECG classification module achieve prediction performance complementarity. In addition, to eliminate the time-varying of each episode in some epilepsy patients, the adversarial training is used, which increases the ability of extracting the invariant features.

FIGURE 1

Figure 1. Flowchart of the epileptic seizure prediction model. A block diagram showing the components of the prediction model, in which the electroencephalogram (EEG) and electrocardiogram (ECG) signals are feature extracted and classified, respectively, and then the classification results are post-processed by a decision-level fusion strategy to obtain the final prediction results.

2.2.1. Data pre-processing

Data pre-processing is an important step before the feature extraction stage, and perfect data pre-processing will guarantee the high accuracy in the later stage; in pre-processing, firstly, useless electrodes must be removed, followed by filtering, because the frequency range from 0.1 to 70 Hz contains most of the epilepsy-related information (Das et al., 2020), we set the bandpass filter range as follows: low-frequency filtering is set to 0.1 Hz, high frequency filtering is set to 70 Hz, and a notch filter is set from 48 to 52 Hz to eliminate power frequency interference; in addition, to improve the accuracy of the algorithm, we use min-max regularization technology (Sola and Sevilla, 1997) to regularize the data, the data is between 0 and 1 after regularization. The min-max regularization method is shown below:

${\mathit{X}}_{\text{normal}}=\frac{{\mathit{X}}_{\text{i}}-{\mathit{X}}_{\min }}{{\mathit{X}}_{\max }-{\mathit{X}}_{\min }}(1)$

Where,X_i is the original data, X_normal is the data after regularization, X_min and X_max is the minimum and maximum values of the original data. After regularizing the EEG data, all channel data are selected (30 channels are included in the Siena scalp EEG dataset after removal of useless electrodes, of which 29 EEG channels, 1 ECG channel), to facilitate later feature extraction and classification, the EEG and ECG data are segmented separately; the data are segmented using moving window analysis, with a window length of 1 s and 0% overlapping; For labeling the data segment, when the data segments are in the interictal state, the segment data is labeled as 0, and when the data segments are in the preictal state, the segmented data is labeled as 1; For labeling each episode in a patient, the episodes are labeled from 0 to N-1 (N is the number of episodes in a patient). Finally, the EEG dataset and the ECG dataset are formed.

In order to extract the spectral characteristics of ECG data, we adopt a continuous wavelet transform, the wavelet scale is 32 and the mother wavelet is “mexh,” so as to obtain the effective characterization signal of the original signal in the frequency domain (Shukla et al., 2022), and the continuous wavelet transform is shown below:

$WTf(a,b)=\frac{1}{\sqrt{a}}{\int }_{-\mathrm{\infty }}^{+\mathrm{\infty }}f(t){\Psi }^{*}(\frac{t-b}{a})dt(2)$

Where, WT represents the wavelet transform, _f(t) is the original signal; _a (_a>0), _b are scale parameters and translation parameters, respectively; _{Ψ((t–b)/a)} is sub-wave, it is the scaling and translation of the mother wavelet.

After pre-processing, the pre-processed data is fed into the feature extraction and classification module for feature extraction and classification.

2.2.2. Feature extraction

2.2.2.1. Bi-LSTM+1D CNN network

We use the Bi-LSTM network (Gers et al., 1999; Tsiouris et al., 2018; Cheng et al., 2021) to extract features from the EEG data. Bi-LSTM network is a kind of RNN, where each block is composed of two LSTM blocks, respectively, processing forward and backward EEG signal, the forward module processes the time feature vector along the positive sequence of the time series, while the backward transmission block processes the same time feature in the reverse sequence of the time series. The output of each block is a combination of processing results in two opposite directions, the Bi-LSTM network not only processes the current feature extraction, but also extracts future features, so it can effectively improve the recognition accuracy. Bi-LSTM+1D CNN network is mainly to extract features of EEG data, then the extracted features are classified by the two full connection layer networks.

2.2.2.2. 2D CNN network

Since the ECG signal does not change significantly in the time domain, there are subtle feature changes in the frequency domain, therefore the feature extraction and classification of the ECG data adopts 2D image format, due to the convolution, pooling and other operations contained in the CNN network (Khan et al., 2018; Liu et al., 2020), it is very applicable to process 2D image, so the feature extraction of the ECG signal adopts the 2D CNN network. Finally, the two full connection layers are used to classify the extracted ECG features.

2.3. Data fusion strategy

Data fusion is the combination of data and information from multi-sensor information sources, the purpose is to timely mine hidden information from big data, so how to reasonably and effectively integrate and intelligently process big data is a problem of data fusion. According to different stages of data fusion, data fusion is divided into three categories: data-level fusion, feature-level fusion and decision-level fusion (Balasubramaniam and Ananthi, 2014; Javed et al., 2014; Versaci et al., 2022).

For data-level fusion, the pre-processed raw data from several different sources are combined according to certain rules, e.g., the EEG signal and the ECG signal are combined; note that the combined data must keep the sampling rate consistent and the data time aligned.

For feature level fusion, to alleviate the problem of inconsistencies between the original data in each mode, the representation of the features can be taken separately from each mode, and then the extracted feature is combined to form a new feature, which is provided for later data classification.

In decision level fusion, the DL model trains the data from different modes and then fuses the output results for later data classification; decision level fusion mainly uses rules such as the mean method, voting method and ensemble learning method.

In our work, to further improve the performance of the DL model, we proposed a novel network consisting of two branches (EEG feature extraction and classification branch, ECG feature extraction and classification branch), each branch is an independent network, and the output result of the network is the weighted average of each branch (Saranya et al., 2021):

$y=\sum _{i=1}^L{\mathit{w}}_i{\mathit{d}}_i(3)$

Where, _wi is the weight, _{w1 + w2 + wL=1}, L is the number of branch network (L = 2), _di is the prediction results of the ith branch network. For weighted averages _wi, can be fixed to 0.5, however, after such a fixed setting, the classification error rate of the branch network cannot affect the weights of the branch, resulting in no improvement in the classification accuracy, so we use the classification error rate of the branch to dynamically adjust the weight of the branch, the weights are calculated as follows:

$w_1=1+arctan(\frac{1-err1}{err1})(4)$

$w_2=1+arctan(\frac{1-err2}{err2})(5)$

$W_1=\frac{w_1}{w_1+w_2}(6)$

$W_2=\frac{w_2}{w_1+w_2}(7)$

Where, _err1 and _err2 are the classification error rate of the two branch networks, _w1 and_w2 are the weights of each branch network, _W1 and _W2 are the normalized value of _w1 and_{w2, W1 + W2=1}. By dynamically weighting the output of each branch, a network model with better performance, stronger generalization ability, and higher stability is constructed.

2.4. Adversarial training

For a particular patient, the EEG and ECG data are time-varying, resulting in each episode being different, in order to eliminate the negative impact of data distribution shift between episodes, we adopt adversarial training (Ganin et al., 2016), which aims to improve the generalization ability and prediction accuracy of the model. For the EEG data, the features are extracted by feature extraction networks _Gf1, then, the features are fed into the episode discriminator 1; For the ECG data, the features are extracted by feature extraction networks _Gf2, then, the features are fed into the episode discriminator 2; the losses of episode discriminator 1 and 2 are as follows:

$L_{d\mathrm{\_}eeg}=\frac{1}{N}\sum _{x_{eeg\mathrm{\_}i}\in D_{eeg}}L(G_{d\mathrm{\_}eeg}(G_{f1}(x_{eeg\mathrm{\_}i})),d_i)(8)$

$L_{d\mathrm{\_}ecg}=\frac{1}{N}\sum _{x_{ecg\mathrm{\_}i}\in D_{ec\text{g}}}L(G_{d\mathrm{\_}ecg}(G_{f2}(x_{ecg\mathrm{\_}i})),d_i)(9)$

Where, _L is the cross entropy loss function, G_f1 is the EEG feature extraction network, G_f2 is the ECG feature extraction network, G_{d_eeg} is the EEG episode discriminator, G_{d_ecg} is the ECG episode discriminator, d_i is the episode label, x_{eeg_i} and x_{ecg_i} are the EEG and ECG data sample, D_eeg and D_ecg are the dataset of EEG data and ECG data in a patient, _N is the number of samples.

2.5. Training details

The proposed approach adopts data fusion strategy, the loss function of classifier is:

$L_c=\frac{1}{N}\sum _{x_{eeg-i}\in D_{eeg;ecg-i}\in D_{ecg}}L(M(G_{c1}(G_{f1}(x_{eeg-i})),(10)$

$G_{c2}(G_{f2}(x_{ecg-i}))),y_i)$

Where, _L is the cross entropy loss function, G_c1 is the EEG category classifier, G_c2 is the ECG category classifier, y_i is the category label, and _Mis data fusion strategy:

$M=W_1*G_{c1}(G_{f1}(x_{eeg\mathrm{\_}i})+W_2*G_{c2}(G_{f2}(x_{ecg\mathrm{\_}i})(11)$

Where, _W1 and _W2 are weights of each branch which can be learnt by the training.

We propose an adversarial training strategy to jointly train all the loss functions:

$L_{sum}=L_c-\mathrm{\lambda }\times (L_{d\mathrm{\_}eeg}+L_{d\mathrm{\_}ecg})(12)$

Where, _λ = 0.1; During optimization, episode discriminator 1 and episode discriminator 2 are trained by a special layer called Gradient Reversal Layer (GRL), which connects feature extraction network and episode discriminator, this GRL is omitted during forward propagation, the gradient is reversed in backpropagation. Finally, we search the optimal parameters $\stackrel{\wedge }{{\text{θ}}_{f1}}\stackrel{\wedge }{{\text{θ}}_{f2}}\stackrel{\wedge }{{\text{θ}}_{c1}}\stackrel{\wedge }{{\text{θ}}_{c2}}\underset{\mathit{d\_eeg}}{\overset{\wedge }{\text{θ}}}\underset{\mathit{d\_ecg}}{\overset{\wedge }{\text{θ}}}\underset{M}{\overset{\wedge }{\text{θ}}}$ to meet the following requirements:

$(\underset{f1}{\overset{\mathrm{\wedge }}{\theta }},\underset{f2}{\overset{\mathrm{\wedge }}{\theta }},\underset{c1}{\overset{\mathrm{\wedge }}{\theta }},\underset{c2}{\overset{\mathrm{\wedge }}{\theta }},\underset{M}{\overset{\mathrm{\wedge }}{\theta }})=\arg \underset{{\mathrm{\theta }}_{f1},{\mathrm{\theta }}_{f2},{\mathrm{\theta }}_{c1},{\mathrm{\theta }}_{c2},{\mathrm{\theta }}_M}{\min }{L}_{\text{sum}}({\mathrm{\theta }}_{f1},{\mathrm{\theta }}_{f2},{\mathrm{\theta }}_{c1},{\mathrm{\theta }}_{c2},{\mathrm{\theta }}_{d\mathrm{\_}eeg},{\mathrm{\theta }}_{d\mathrm{\_}ecg})(13)$

$(\underset{d\mathrm{\_}eeg}{\overset{\mathrm{\wedge }}{\theta }},\underset{d\mathrm{\_}ecg}{\overset{\mathrm{\wedge }}{\theta }})=\arg \underset{{\mathrm{\theta }}_{d\mathrm{\_}eeg},{\mathrm{\theta }}_{d\mathrm{\_}ecg}}{\max }{L}_{\text{sum}}({\mathrm{\theta }}_{f1},{\mathrm{\theta }}_{f2},{\mathrm{\theta }}_{c1},{\mathrm{\theta }}_{c2},{\mathrm{\theta }}_{d\mathrm{\_}eeg},{\mathrm{\theta }}_{d\mathrm{\_}ecg})(14)$

2.6. Evaluation

Leave-one-out cross-validation is used in the training and test phases, where the number of training sessions for a given patient is equal to the number of episodes. During each training session, except for one episode which is used in the test, all other episodes participated in the training process, and the process is repeated by changing the patient’s episode.

2.6.1. Experimental environment

The experimental environment is: Windows 10 operation system, the program language is Python 3.7.4, and the DL framework is Pytorch (its version is 11.1). The graphics card is: GeForce RTX 3060.

2.6.2. Experimental parameters

This experiment uses two independent training branches, including LSTM+CNN network for EEG data and 2-D CNN network for ECG data, training epoch is set to 100 times and batch size is set to 128. Cross entropy loss function and Center loss function were combined as the loss function, the model uses the Adam optimizer and learning rate is set to 0.001, center loss function is optimized using the SGD optimizer, and learning rate is set to 0.05. For the setting of hyperparameters in the experiment, we use grid search method.

2.6.3. Evaluate metrics

The experiment used accuracy (ACC), sensitivity (SN), and specificity (SP) to quantify the performance of the algorithm (Prasanna et al., 2021).

$ACC=\frac{TP+TN}{TP+TN+FP+FN}(15)$

$SN=\frac{TP}{TP+FN}(16)$

$SP=\frac{TN}{TP+TN}(17)$

Where, TP (True Positive): the sample that is positive is judged to be positive, TN (True Negative): the sample that is negative is judged to be negative, FP (False Positive): the negative sample is considered positive, FN (False Negative): the sample that is positive is judged to be negative.

To evaluate the prediction accuracy of the forecasting system, the false alarm rate is measured as follows (Cheng et al., 2021):

$FAR=\frac{N_{alarm}}{N_{wo}}(18)$

Where, _Nalarm represents the number of time windows for alarms in the interictal, _Nwo represents the number of time windows in interictal.

3. Results and discussion

3.1. Results of the proposed approach

To evaluate the advantages and disadvantages of the proposed approach, we selected eight patients with a number of epileptic seizures greater than or equal to three from the Siena scalp EEG dataset, the results of each epilepsy patient are shown in Table 2.

TABLE 2

Table 2. Test results on Siena scalp dataset.

Since there is very little literature on seizure prediction using the Siena scalp EEG dataset, we set the preictal period length to T = 300 s (5 min), the same preictal period length setting proposed in the literature (Detti et al., 2020), and compared with the literature (Phomsiricharoenphant et al., 2014; Hoyos-Osorio et al., 2016; Billeci et al., 2019; Detti et al., 2020). The comparison results of the related methods are shown in Table 3. The results show that the proposed approach has an advantage in all test metrics.

TABLE 3

Table 3. Comparison results of related methods.

In order to compare performance with other approaches, we used t-test in the case of normal distributions to calculate the p-values of the accuracy between literatures (Phomsiricharoenphant et al., 2014; Hoyos-Osorio et al., 2016; Billeci et al., 2019; Detti et al., 2020) and the proposed approach, the results are 1.34E-09, 2.22E-08, 5.94E-08, 0.029, respectively, all less than 0.05, therefore, the approach in this paper significantly outperforms the contrastive literatures.

3.2. Comparison of the methods of data fusion

To further demonstrate the superiority of the decision-level fusion method proposed in this paper, we compare the performance of the data-level fusion and feature-level fusion approaches. The two approaches are shown in Figures 2, 3.

FIGURE 2

Figure 2. Flowchart of the data-level fusion epileptic seizure prediction model. This method first combines EEG and ECG signals, and then performs feature extraction and classification.

FIGURE 3

Figure 3. Flowchart of the feature-level fusion epileptic seizure prediction model. This method performs pre-processing and feature extraction from EEG and ECG signals independently, and then fuses and classifies the features of the EEG and ECG signals.

The comparison results of the three data fusion approaches are shown in Table 4.

TABLE 4

Table 4. Comparison results of three data fusion methods.

In order to compare performance with other data fusion approaches, we used t-test in the case of normal distributions to calculate the p-values of the accuracy between data-level/feature-level fusion approach and the proposed approach, the results are 8.29E-08, 1.92E-09, respectively, all less than 0.05, therefore, the approach in this paper significantly outperforms the contrastive data fusion approaches.

Through comparison, it can be seen that the effect of the decision-level fusion approach is optimal, followed by the data-level fusion approach, the feature-level fusion approach is the worst, analyze its reasons, there are three reasons, first, EEG and ECG signals can be used as complementary to each other in epileptic seizure prediction, through independent feature extraction of EEG and ECG signals, the shortcomings of the extraction features of a single network can be effectively avoided; second, for the feature-level fusion approach, the two independent branch networks are used for EEG and ECG signal feature extraction, but the separately extracted feature dimensions are limited by artificial settings, the specific setting is still unknown how much is the best state, so the result is the worst; third, for the data level fusion approach, the EEG and ECG data are combined in the same mode, then the feature extraction is performed by a unified network, which ignores the diversity of the data, resulting in the information from the multimodal data is not extracted and processed, so the classification effect of the data level fusion approach is inferior to that of the decision level fusion approach.

3.3. Comparison of the fixed weight setting and the dynamic weight setting

The proposed approach adopts the dynamic setting of the output weights of each branch network, which is based on the error rate of each branch. To verify the superiority of the dynamic weight method, the output weights of the two branches are fixed at 0.5, and the average accuracy, average sensitivity, average specificity, and average error alarm rate are 93.35%, 94.46%, 92.98%, and 0.026, respectively. When compared with the proposed approach, the approach of dynamically setting the output weight of each branch is better than the approach of fixedly setting the output weight of the branch network. The comparison result of the two approaches is shown in Figure 4.

FIGURE 4

Figure 4. Comparison results with and without dynamic weight setting.

3.4. Comparison of results of adversarial training and non-adversarial training

In order to eliminate the negative impact of data difference of each episode in a patient, we adopt adversarial training to improve the generalization ability of the network, as a comparison method, the GRL and domain adversarial network are removed, the average accuracy, average sensitivity, average specificity and average FAR of eight epilepsy patients were 95.68%, 93.44%, 97.21% and 0.015, respectively, and by comparing with the proposed approach, the result shows that using adversarial training has better prediction performance. The comparison result of the two approaches is shown in Figure 5.

FIGURE 5

Figure 5. Comparison results with and without adversarial training.

3.5. Advantages and limitations

This approach has the following advantages:

1. The proposed approach adopts the decision level data fusion strategy, the results of EEG and ECG data classification are combined, in the combining process, the classification error rate of each branch is used to calculate the weight, the branch with low error rate is set by a high weight, the branch with high error rate is set by a low weight, so that the network branch with low classification error rate plays a leading role in prediction, and the result of the network branch with high prediction error rate is used as a supplement, thus improving the prediction accuracy.

2. Since epilepsy data are time-varying, with differences between each seizure in a patient, this approach adopts domain adversarial training to extract the invariant features, thereby improving the generalization ability of the model.

This approach has limitations:

The proposed approach requires a dataset with both EEG and ECG signals, but there are few public datasets that meet this condition. The performance of the approach on multiple datasets cannot be demonstrated, so it is necessary to collect epilepsy data with both EEG and ECG signals from the local hospital for the demonstration.

4. Conclusion

In this work, a novel patient-specific approach to seizure prediction based on data fusion and adversarial training is proposed. To improve the prediction accuracy and robustness of the network, the data fusion strategy is adopted. To eliminate the instability of the network caused by the negative impact of data differences of each episode in a patient, domain adversarial training is adopted to extract the invariant features of a specific patient. We use the Siena scalp EEG dataset and leave-one-out cross-validation strategy, the average accuracy, average sensitivity, average specific and average FAR on the 8 epilepsy patients are calculated, our results outperform the competitive state-of-the-art baselines. In addition, the advantages of this approach are verified by comparing it with different data fusion methods and the use of domain adversarial training methods.

For future work, it is necessary to use techniques such as domain generalization to reduce the differences between patients for the clinical application of epileptic seizure prediction, and to develop a portable system that includes a wearable electrode cap and a smart device for epileptic seizure prediction.

Data availability statement

Publicly available datasets were analyzed in this study. This data can be found here: https://physionet.org/content/siena-scalp-eeg/1.0.0.

Ethics statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent from the participants’ legal guardian/next of kin was not required to participate in this study in accordance with the national legislation and the institutional requirements.

Author contributions

YY and FL conceptualized the study. YY and XQ performed the methodology. YY and HW accounted the software. YY and XL wrote and prepared the original draft. XQ reviewed and edited the written draft. All authors read and agreed to the published version of the manuscript.

Funding

This research was supported by Sichuan Science and Technology Plan of China Grants (2019ZDZX0005, 2019ZDZX0006, and 2020YFQ0056).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Babb, T. L., Wilson, C. L., and Isokawa-Akesson, M. (1987). Firing patterns of human limbic neurons during stereoencephalography (SEEG) and clinical temporal lobe seizures. Electroencephalogr. Clin. Neurophysiol. 66, 467–482. doi: 10.1016/0013-4694(87)90093-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Balasubramaniam, P., and Ananthi, V. P. (2014). Image fusion using intuitionistic fuzzy sets. Inf. Fusion 20, 21–30. doi: 10.1016/j.inffus.2013.10.011

CrossRef Full Text | Google Scholar

Billeci, L., Tonacci, A., Varanini, M., Detti, P., De Lara, G. Z. M., and Vatti, G. (2019). “Epileptic seizures prediction based on the combination of EEG and ECG for the application in a wearable device,” in 2019 IEEE 23rd International symposium on consumer technologies (ISCT), (Italy: IEEE), 28–33. doi: 10.1109/ISCE.2019.8900998

CrossRef Full Text | Google Scholar

Cheng, C. C., You, B., Liu, Y., and Dai, Y. K. (2021). Patient-specific method of sleep electroencephalography using wavelet packet transform and Bi-LSTM for epileptic seizure prediction. Biomed. Signal Process. Control 70:102963. doi: 10.1016/j.bspc.2021.102963

CrossRef Full Text | Google Scholar

Costagliola, G., Orsini, A., Coll, M., Brugada, R., Parisi, P., and Striano, P. (2021). The brain-heart interaction in epilepsy: Implications for diagnosis, therapy, and SUDEP prevention. Ann. Clin. Transl. Neurol. 8, 1557–1568. doi: 10.1002/acn3.51382

PubMed Abstract | CrossRef Full Text | Google Scholar

Das, K., Daschakladar, D., Roy, P. P., Chatterjee, A. I. M., and Saha, S. P. (2020). Epileptic seizure prediction by the detection of seizure waveform from the pre-ictal phase of EEG signal. Biomed. Signal Process. Control. 57:101720. doi: 10.1016/j.bspc.2019.101720

CrossRef Full Text | Google Scholar

Detti, P., De Lara, G. Z. M., Bruni, R., Pranzo, M., Sarnari, F., and Vatti, G. (2018). A patient-specific approach for short-term epileptic seizures prediction through the analysis of EEG synchronization. IEEE Trans. Biomed. Eng. 66, 1494–1504. doi: 10.1109/TBME.2018.2874716

PubMed Abstract | CrossRef Full Text | Google Scholar

Detti, P., Vatti, G., and Lara, G. Z. M. (2020). EEG synchronization analysis for seizure prediction: A study on data of noninvasive recordings. Processes 8:846. doi: 10.3390/pr8070846

CrossRef Full Text | Google Scholar

Fisher, R. S., Cross, J. H., French, J. A., Higurashi, N., Hirsch, E., Jansen, F. E., et al. (2017). Operational classification of seizure types by the international league against epilepsy: Position paper of the ILAE commission for classification and terminology. Epilepsia 58, 522–530. doi: 10.1111/epi.13670

PubMed Abstract | CrossRef Full Text | Google Scholar

Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., et al. (2016). Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17, 2096–2130. doi: 10.48550/arXiv.1505.07818

PubMed Abstract | CrossRef Full Text | Google Scholar

Gers, F., Schmidhuber, J., and Cummins, F. (1999). “Learning to forget: Continual prediction with LSTM,” in 1999 Ninth international conference on artificial neural networks ICANN 99, Vol. 2, (Edinburgh: IET), 850–855. doi: 10.1049/cp:19991218

CrossRef Full Text | Google Scholar

Hoyos-Osorio, K., Castañeda-Gonzaiez, J., and Daza-Santacoloma, G. (2016). “Automatic epileptic seizure prediction based on scalp EEG and ECG signals,” in 2016 XXI symposium on signal processing, images and artificial vision (STSIVA), (Colombia: IEEE), 1–7. doi: 10.1109/STSIVA.2016.7743357

CrossRef Full Text | Google Scholar

Jana, R., and Mukherjee, I. (2021a). Deep learning based efficient epileptic seizure prediction with EEG channel optimization. Biomed. Signal Process. Control 68:102767. doi: 10.1016/j.bspc.2021.102767

CrossRef Full Text | Google Scholar

Jana, R., and Mukherjee, I. (2021b). Epileptic seizure prediction from raw EEG signal using convolutional neural network. Mach. Vis. Augment.Intell. Theory Appl. 796, 235–244. doi: 10.1007/978-981-16-5078-9_20

CrossRef Full Text | Google Scholar

Javed, U., Riaz, M. M., Ghafoor, A., Ali, S. S., and Cheema, T. A. (2014). MRI and PET image fusion using fuzzy logic and image local features. Sci. World J. 2014:708075. doi: 10.1155/2014/708075

PubMed Abstract | CrossRef Full Text | Google Scholar

Khan, H., Marcuse, L., Fields, M., Swann, K., and Yener, B. (2018). Focal onset seizure prediction using convolutional networks. IEEE Trans. Biomed. Eng. 65, 2109–2118. doi: 10.1109/TBME.2017.2785401

PubMed Abstract | CrossRef Full Text | Google Scholar

Liu, Y., Sivathamboo, S., Goodin, P., Bonnington, P., Kwan, P., Kuhlmann, L., et al. (2020). “Epileptic seizure detection using convolutional neural network: A multi-biosignal study,” in Proceedings of the Australasian computer science week multiconference 2020, ACSW 2020, ed. A. Forkan (New York, NY: ACM), 1–8. doi: 10.1145/3373017.3373055

CrossRef Full Text | Google Scholar

Matsuura, T., and Harada, T. (2020). Domain generalization using a mixture of multiple latent domains. Proc. AAAI Conf. Artif. Intell. 34, 11749–11756. doi: 10.1609/AAAI.V34I07.6846

CrossRef Full Text | Google Scholar

Moridani, M. K., and Farhadi, H. (2017). Heart rate variability as a biomarker for epilepsy seizure prediction. Bratisl Lek Listy 118, 3–8. doi: 10.4149/BLL_2017_001

PubMed Abstract | CrossRef Full Text | Google Scholar

Pei, Z. Y., Cao, Z. J., Long, M. S., and Wang, J. M. (2018). “Multi-adversarial domain adaptation,” in AAAI.Proceedings of AAAI conference on artificial intelligence, (Palo Alto, CA: AAAI Press), doi: 10.48550/arXiv.1809.02176

PubMed Abstract | CrossRef Full Text | Google Scholar

Phomsiricharoenphant, W., Ongwattanakul, S., and Wongsawat, Y. (2014). “The preliminary study of EEG and ECG for epileptic seizure prediction based on Hilbert Huang Transform,” in The 7th 2014 biomedical engineering international conference, (Fukuoka: IEEE), 1–4. doi: 10.1109/BMEiCON.2014.7017433

CrossRef Full Text | Google Scholar

Prasanna, J., Subathra, M., Mohammed, M., Damaševièius, R., Sairamya, N., and George, S. (2021). Automated epileptic seizure detection in pediatric subjects of CHB-MIT EEG database-a survey. J. Pers. Med. 11:1028. doi: 10.3390/jpm11101028

PubMed Abstract | CrossRef Full Text | Google Scholar

Rakhade, S. N., and Jensen, F. E. (2009). Epileptogenesis in the immature brain: Emerging mechanisms. Nat. Rev. Neurol. J. 5, 380–391. doi: 10.1038/nrneurol.2009.80

PubMed Abstract | CrossRef Full Text | Google Scholar

Rasheed, K., Qayyum, A., Qadir, J., Sivathamboo, S., Kwan, P., and Kuhlmann, L. (2021). Machine learning for predicting epileptic seizures using EEG signals: A review. IEEE Rev. Biomed. Eng. 14, 139–155. doi: 10.1109/RBME.2020.3008792

PubMed Abstract | CrossRef Full Text | Google Scholar

Saranya, N., Karthika Renuka, D., and Geetha Rajakumari, R. (2021). “Ensemble classification for epileptic seizure prediction,” in 2021 2nd international conference on smart electronics and communication (ICOSEC), (India: IEEE), 1–5. doi: 10.1109/ICOSEC51865.2021.9591882

CrossRef Full Text | Google Scholar

Seifi, B., Barfi, M., and Esmaeilpour, M. (2022). “ECG-based prediction of epileptic seizures using machine learning methods,” in 2022 9th Iranian joint congress on fuzzy and intelligent systems (CFIS), (Iran: IEEE), 1–7. doi: 10.1109/CFIS54774.2022.9756421

CrossRef Full Text | Google Scholar

Shukla, R., Kumar, B., Gaurav, G., Singh, G., and Sahani, A. K. (2022). Epileptic seizure detection using continuous wavelet transform and deep neural networks. Lect. Notes Electr. Eng. 886, 291–300. doi: 10.1007/978-3-030-98886-9_23

CrossRef Full Text | Google Scholar

Sola, J., and Sevilla, J. (1997). Importance of input data normalization for the application of neural networks to complex industrial problem. IEEE Trans. Nucl. Sci. 44, 1464–1468. doi: 10.1109/23.589532

CrossRef Full Text | Google Scholar

Tsiouris, KM, Pezoulas, V. C., Zervakis, M., Konitsiotis, S., Koutsouris, D. D., and Fotiadis, D. I (2018). A long short-term memory deep learning network for the prediction of epileptic seizures using EEG signals. Comput. Biol. Med. 99, 24–37. doi: 10.1016/j.compbiomed.2018.05.019

PubMed Abstract | CrossRef Full Text | Google Scholar

Ufongene, C., Atrache, R. E., Loddenkemper, T., and Meisel, C. (2020). Electrocardiographic changes associated with epilepsy beyond heart rate and their utilization in future seizure detection and forecasting methods. Clin. Neurophysiol. 131, 866–879. doi: 10.1016/j.clinph.2020.01.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Usman, S. M., Khalid, S., and Bashir, Z. (2021). Epileptic seizure prediction using scalp electroencephalogram signals. Biocybern. Biomed. Eng. 41, 211–220. doi: 10.1016/j.bbe.2021.01.001

CrossRef Full Text | Google Scholar

Versaci, M., Angiulli, G., Crucitti, P., Carlo, D. D., Laganà, F., Pellicanò, D., et al. (2022). A fuzzy similarity-based approach to classify numerically simulated and experimentally detected carbon fiber-reinforced polymer plate defects. Sensors 22:4232. doi: 10.3390/s22114232

PubMed Abstract | CrossRef Full Text | Google Scholar

Yu, C. H., Wang, J. D., Chen, Y. Q., and Huang, M. Y. (2019). “Transfer learning with dynamic adversarial adaptation network,” in IEEE international conference on data mining (ICDM), (Beijing: IEEE), 778–786. doi: 10.1109/ICDM.2019.00088

CrossRef Full Text | Google Scholar

Yuan, S., Zhou, W., and Chen, L. (2018). Epileptic seizure prediction using diffusion distance and Bayesian linear discriminate analysis on intracranial EEG. Int. J. Neural Syst. 28:1750043. doi: 10.1142/S0129065717500435

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhang, Q., Ding, J., Kong, W., Liu, Y., Wang, Q., and Jiang, T. (2021). Epilepsy prediction through optimized multidimensional sample entropy and Bi-LSTM. Biomed. Signal Process. Control 64:102293. doi: 10.1016/j.bspc.2020.102293

CrossRef Full Text | Google Scholar

Zhang, Q., Hu, Y., Potter, T., Li, R., Quach, M., and Zhang, Y. (2020). Establishing functional brain networks using nonlinear partial directed coherence method to predict epileptic seizures. J. Neurosci. Methods 329:108447.

Google Scholar