Clustered event related spectral perturbation (ERSP) feature in right hand motor imagery classification

A technology that allows humans to interact with machines more directly and efficiently would be desirable. Research on brain-computer interfaces (BCIs) provides the possibility for computers to understand human thoughts in a straightforward manner thereby facilitating communication. As a branch of BCI research, motor imagery (MI) techniques analyze the brain signals and help people in many aspects such as rehabilitation, clinical applications, entertainment, and system controlling. In this study, an imagery experiment consisting of four kinds of right-hand movements (gripping, opening, pronation, and supination) was designed. Then a novel feature, namely, clustered feature was proposed based on the event-related spectral perturbation (ERSP) calculated from the EEG signal. Based on the selected features, two classical classifiers (support vector machine and linear discriminant classifier) were trained, achieving an acceptable accurate result (80%, on average).

1. Introduction

In recent decades, the development of the brain-computer interface (BCI) has provided novel and intuitive interactions that make our lives more convenient. In medicine, BCI helps stroke patients with rehabilitation, the disabled to control prosthetic limbs, and amyotrophic lateral sclerosis (ALS) patients to communicate with others. In addition, various BCI studies have been conducted on remote robot control, emotion analysis, and entertainment equipment.

The implementation of these applications is inseparable from the brain activities signal measurement. There are different measurement techniques, such as functional magnetic resonance imaging (fMRI), magnetoencephalography (MEG), electrocorticography (ECoG), and electroencephalogram (EEG) for recording the signals representing brain activity. ECOG is an invasive method that records high-quality signals reflecting delicate brain activities, such as finger movement (Yoshimura et al., 2017). However, there is a risk associated with ECoG because the electrodes or probes are set on the exposed surface of the cortex. Therefore, research on signals from non-invasive methods, such as fMRI, MEG, and EEG signals has become more popular. MEG and fMRI provide detailed brain activity analysis with high spatial resolution (Ramadan and Vasilakos, 2017) although the temporal resolution is low because of the measurement technique. Meanwhile, the devices used to acquire fMRI and MEG are bulky, making it inconvenient to move or carry. In contrast, EEG applications have characteristics opposite to those of fMRI and MEG. The advantages of high temporal resolution, portability, and safety make EEG applicable to various scenarios, regardless of indoor or outdoor environment.

EEG signal-based research, such as sentiment analysis, motor control, attention detection, and motor imagery have been applied in many different fields to help unravel the complexity of brain activity. Analyzing the EEG signal provides insight into the activities of the brain and control devices. Motor imagery (MI) research plays a significant role in bridging this gap for people and devices, such as prosthetic limbs, robotic assistance systems, and rehabilitation equipment (Miladinović et al., 2020). As a result, there are a number of EEG signal-based motor imagery studies, such as the classification of the right vs. left hand or hand vs. feet movement (Pfurtscheller et al., 2006). However, the analysis of the same limb motor imagery is still challenging because of the complexity of the brain activity and low discrimination of the EEG signal (Srinivasan, 1999; Corley and Huang, 2018). Thus, the main contributions of this study are as follows:

(1) A motor imagery experiment was designed to record the EEG signal of the right-hand imagery movement (grip, opening, pronation, and supination). Through a visual stimulus, the subjects could obtain a detailed hint on how to perform the motor imagery task.

(2) A novel feature calculation method was proposed to enhance brain oscillatory activities in the frequency domain. To improve the classifier performance, clustered event relative spectral perturbation feature was proposed as input.

(3) A comparative study on the feature performance of different baseline. Comparing the baseline before observation period and imagery period, we obtain an acceptable accuracy from each subject on a small size dataset. This result provides a possibility that the strategy can be changed as an online real-time classification after the quick configuration of collecting the subject's signal.

2. Previous and relative research

In contrast to motor movement, in which the EEG signal is recorded while the subject is performing the actual action, motor imagery is a mental process that rehearses or simulates a given action while the subject does not move his/her body or limbs (Decety and Ingvar, 1990). Therefore, motor imagery bridges the gap in communication between people with disabilities, such as those wearing prostheses and devices, and computers (Kappes and Morewedge, 2016). Moreover, healthy people can also enjoy the benefits of MI research.

Physiological arguments indicate that motor imagery is strongly related to brain activity in the μ (8–12 Hz) and β (16–24 Hz) bands (da Silva, 1991). By analyzing the signals in these two bands, various MI applications have been developed as novel interfaces between humans and computers, such as prostheses (Elstob and Secco, 2016), exoskeletons (Rodríguez-Ugarte et al., 2018), drones (Kos' Myna et al., 2014), and robotic arms (Meng et al., 2016).

Researchers have attempted to improve the analysis accuracy of high sensitivity EEG signals in various aspects through preprocessing, feature extraction, and classifier design. Preprocessing methods consisting of spatial filters, such as the common average reference (CAR; McFarland et al., 1997), classical Laplacian method (Hjorth, 1975), spherical spline Laplacian (Perrin et al., 1989), and adaptive methods (Togha et al., 2019) have been used to improve the signal-to-noise ratio (SNR) of the EEG signal and suppress artifact interference.

In addition to the research on signal preprocessing, the development of algorithms for feature extraction, based on event-related desynchronization (ERD; Pfurtscheller and Aranibar, 1977), event-related synchronization (ERS; Pfurtscheller, 1992), event-related potential (ERP; Luck, 2014) have also promoted MI research (Bashashati et al., 2007; Ramadan and Vasilakos, 2017). Feature extraction methods, such as wavelet transformation (WT; Adeli et al., 2003; Hsu and Sun, 2009) and Fourier transformation (Akin, 2002; Polat and Güneş, 2007) are effective in MI classification. Besides, common spatial pattern (CSP; Wang et al., 2006) method, which extremizes the variance of two-class signals, is widely applied in MI research. Methods based on CSP, such as filter bank common spatial pattern (FBCSP; Abbas and Khan, 2018; Das et al., 2020) also perform very well in MI classification.

Designing an efficient classifier is an important part of MI research. Classic classifiers including linear discriminant analysis (LDA; Tariq et al., 2019), support vector machine (SVM; Suwannarat et al., 2018), Naïve Bayesian classifier (NBC; Ang et al., 2012), classification and regression tree (CART; Bentlemsan et al., 2014), and k-nearest neighbor (kNN; Lotte et al., 2007), are widespread in various motor imagery tasks. With the development of the computing power, the neural network such as convolutional neural network (Lawhern et al., 2018) and deep learning methods have attracted increasing attention in recent years (Schirrmeister et al., 2017; Ma et al., 2019). In this study, the performance of SVM and LDA are compared in the right hand motor imagery classification.

3. Experiment and methodology

3.1. Experiment design

Nine healthy subjects (seven males and two females), aged 23–31 years (mean, 26.1; SD, 2.7) participated in the motor imagery experiment. All of them were right-handed. None of the subjects had any physical or psychological diseases. Studies involving human participants were reviewed and approved by the ethics committee of the Tokyo Institute of Technology (ethics number: 2019001). The patients provided written informed consent to participate in the study.

The EEG signal was recorded in a soundproof room. The subjects placed their forearm on the armrest of an armchair in a comfortable sitting posture. In front of the subjects, there was a 27” monitor displaying pictures to guide the subject through experimental instructions. One experimental trial was divided into three parts, as shown in Figure 1A.

1. Preparation period: In this period lasting 3 s, the subjects could adjust their sitting posture and feel relaxed to prepare for the instructions.

2. Observation period: In this period lasting 3 s, the subjects learned the right hand motor imagery from the figure. As visual stimulus, the figure helps the subject imagine a more detailed right hand movement (Neuper et al., 2005).

3. Imagination period: In this period lasting 4 s, the subjects imagined the right hand movement (opening, gripping, pronation, and supination) as shown on the monitor while trying to avoid eye and body movement.

FIGURE 1

Figure 1. The experimental design of right hand motor imagery. (A) Illustrates the grip trial of right-hand motor imagery. (B) Shows that the subject adjusted his position and relaxed during the rest period.

The experiment was repeated 80 times for each participant. A 64-channel-scalp sensor (BIOSEMI Active Two, sampling at 2,048 Hz) was used to record the EEG signal, as shown in Figure 1.

3.2. Preprocessing

The first stage in preprocessing is to improve the quality of the EEG signal. A Hamming-windowed FIR filter is utilized to acquire the 6–30 Hz EEG data, including the μ (8–12 Hz) and β (16–24 Hz) bands. The common average reference, namely CAR, is a popular and efficient method for enhancing SNR, and is expressed as follows:

$\begin{array}{ll}{v}_i^{\text{CAR}}=v_i-\frac{1}{N}{\displaystyle \sum _{j=1}^N}{v}_j& (1)\end{array}$

where v_i is the value of the ith electrode and N is the number of electrodes. By subtracting the average value from all electrode values in large areas of the scalp, CAR emphasizes local activity in the EEG data, such as motor imagery activity in the μ and β bands. As shown in Figure 2A, CAR removes artifacts, such as eye movement and electromyography (EMG), and decreases the correlation between EEG channels (McFarland et al., 1997). Based on the result from previous research and clinical report (Pfurtscheller and Neuper, 1997), human motor movement and motor imagery originate from the motor cortex of the brain. Therefore, 21 channels which cover the motor cortex are selected for the further analysis.

FIGURE 2

Figure 2. The preprocessing of EEG signal. (A) Illustrates the EEG imagery epochs extracted from preprocessed (band pass filter and common average reference) EEG signal. (B) Shows the standard 10–20 electrode layout of the 64-channel EEG sensor. Twenty-one channels in the red rectangular area are selected for further processing.

3.3. Feature extraction

3.3.1. Event related spectral perturbation (ERSP)

Event related desynchronization and synchronization (ERD/S), which are brain oscillatory activities in specific frequency bands extracted from the EEG signal, are widely utilized in motor imagery research (Pfurtscheller and Aranibar, 1977; Pfurtscheller, 1992). To quantify and generalize ERD/S, the concept of event-related spectral perturbation (ERSP) (Grandchamp and Delorme, 2011) is introduced. ERSP describes the relative changes in the EEG amplitude spectrum of similar events in the experimental trials, and is computed using a divisive baseline and discrete sliding window. The event related spectrum for a single trial can be estimated as

$\begin{array}{ll}{P}_i=\frac{|F_i(f,t){|}^2}{{\mu }_B(f,i)}& (2)\end{array}$

where F_i(f, t) is the spectral estimate at frequency f and time point t of the ith trial. We apply Fast Fourier Transform (FFT) by using a Hanning window of 512 sample length and 75% overlapping of the window length to assume the spectral estimate. μ_B(f, i) is the mean baseline spectral estimate for trial k at frequency f and defined as

$\begin{array}{ll}{\mu }_B(f,i)=\frac{1}{m}{\displaystyle \sum _{t^{\prime }\in B}}|F_i(f,t^{\prime }){|}^2& (3)\end{array}$

where B is the ensemble of time points in the baseline period and m is the total number of time points in the baseline period. The ERSP estimate of one single trial can calculate as

$\begin{array}{ll}ERS{P}_{single}=10lo{g}_{10}(P_i)& (4)\end{array}$

Considering the EEG signal is a non-stationary signal with a high degree of variability, even if the signal is from the same subject, it is necessary to reduce the signal's randomness. By averaging the values of ERSP in subjects, we can not only obtain a stationary signal feature, but also can observe the activities of ERS and ERD directly from the visualization as shown in Figure 3. The mean event related spectrum can be estimated as

$\begin{array}{ll}S(f,t)=\frac{\frac{1}{n}{\displaystyle {\sum }_{i=1}^n}|F_i(f,t){|}^2}{{\mu }_B^{\prime }(f)}& (5)\end{array}$

FIGURE 3

Figure 3. The visualization of ERSP calculation in C3 channel. (A) Is the result calculated by one trial and (B) is calculated by 10 trials picked randomly from all trials. Blue areas show the ERD while the red areas show the ERS.

where ${\mu }_B^{\prime }(f)$ is the mean spectral estimate for all baseline points at frequency f and defined as

$\begin{array}{ll}{\mu }_B^{\prime }(f)=\frac{1}{mn}{\displaystyle \sum _{i=1}^n}{\displaystyle \sum _{t^{\prime }\in B}}|F_i(f,t^{\prime }){|}^2& (6)\end{array}$

where n is the number of trials to average the mean calculation. The mean ERSP can be calculated as

$\begin{array}{ll}ERS{P}_{mean}=10lo{g}_{10}(S(f,t))& (7)\end{array}$

3.3.2. Clustered feature

In this study, we suppose that each single trial ERSP is distributed in a high dimensional space. There must be a ground truth value which represents a specific motor imagery existing in this space. The assumption is that all the single trial ERSP is distributed near the ground truth value within a threshold distance. We apply the bootstrap method and randomly pick k trials as the center cluster to estimate the ground truth value. The Euclidean distance from single trial to estimated ERSP can be calculated as

$\begin{array}{ll}{d}_i=\Vert ERS{P}_{single}-\widetilde{ERS{P}_i}{\Vert }_2& (8)\end{array}$

where the ERSP_single is acquired from Equation (4), $\widetilde{ERS{P}_i}$ is the estimated ERSP of a specific motor imagery, i ∈ {Griping, Opening, Pronation, Supination}, $d_i\in {ℝ}^{fb\times 1}$ and fb is frequency band. For binary classification such like griping vs. opening or pronation vs. supination, the ERSP distance feature of a channel can be denoted as

$\begin{array}{ll}{D}_j=\left[d_{griping},d_{opening}\right]or{D}_j=\left[d_{pronation},d_{supination}\right]& (9)\end{array}$

where $D_j\in {ℝ}^{2fb\times 1}$ is the distance feature of jth channel. Therefore, the multi-channel feature can be denoted as

$\begin{array}{ll}F=\left[D_1,D_2,...,D_j\right]& (10)\end{array}$

where F ∈ ℝ^2fb×nc and nc is the number of channels.

3.4. Classification

In this section, we introduce two classical machine learning classifiers such as support vector machine (SVM) and linear discriminate analysis (LDA).

3.4.1. Discriminant analysis models (LDA)

There are various discriminant analysis models such as regularized linear, quadratic etc. Discriminant analysis attempts to maximize the variance between the classes and minimize the variance within the classes by mapping the data to a low dimensional space and assuming the classes' covariance. The prediction function is described as (Malki et al., 2015):

$\begin{array}{ll}\widehat{\omega }=\underset{\omega =1,2,...K}{argmin}{\displaystyle \sum _{k=1}^K}\widehat{P}(k|x)C(\omega |k)& (11)\end{array}$

where $\widehat{\omega }$ is the predicted classification, K is the number of classes, $\widehat{P}(k|x)$ is the posterior probability of class k for observation x and C(ω|x) is cost of classifying an observation as ω when its true class is k. Regularized linear discriminant analysis (LDA) in which all classes have the same covariance matrix is employed for further classification. Meanwhile, we utilize MATLAB 2022 to optimize the hyperparameter automatically and apply “expected-improvement-plus” as optimization options.

3.4.2. Support vector machine (SVM)

In binary classification problems, support vector machine(SVM) shows an effective role. SVM algorithm is to find a hyperplane which can separate two classes with the largest margin by projecting the input vector into a high dimensional space. The hyperplane we expected can separate two classes with the largest margin where margin means the maximum width of the parallel to the hyperplane that has no interior data points. For training data d dimension x_j with their categories y_i, the expected hyperplane can be denoted as:

$\begin{array}{ll}\widehat{f}(x)=x^{\prime }\widehat{\beta }+\widehat{b}=0& (12)\end{array}$

where $x_j\in {ℝ}^d$, y_i = ±1, $\widehat{\beta }$ and $\widehat{b}$ meet the inequality $y_i\widehat{f}(x_j)\ge 1$. Therefore, the prediction function can be denoted as:

$\begin{array}{ll}class(z)=sign(z^{\prime }\widehat{\beta }+\widehat{b})=sign\widehat{f}(z)& (13)\end{array}$

where z is the predicted data. We employ a linear kernel to set SVM and utilize the same hyperparameter optimization option as LDA in MATLAB 2022.

4. Result

4.1. Accuracy based on single trial ERSP estimate

The baseline time is set at when the visual stimulus emerges. A 5,000 ms lasting motor imagery epoch which is from −1,000 ms before baseline to 4,000 ms after baseline is extracted in the ERSP calculation. Ten-fold cross validation is applied on single trial ERSP to train the SVM and LDA. The multi channels which cover the motor cortex contain 21 channels (FC5, FC3, FC1, FCz, FC2, FC4, FC6, C5, C3, C1, Cz, C2, C4, C6, CP5, CP3, CP1, CPz, CP2, CP4, CP6) as Figure 2B. Table 1 shows the average accuracy and corresponding standard deviation of 9 subjects in griping vs. opening classification. Based on single channel ERSP, the best accuracy of the SVM model is 55.0% from subject 9 and the average accuracy is 47.95%. The best accuracy of the LDA model is 59.59% from subject 8, and the average accuracy is 51.28%. The average loss is 0.51 of SVM and 0.48 of LDA. Based on the multi-channel ERSP from motor cortex area, the best accuracy of the SVM model is 55.94% from subject 9 and the average accuracy is 51.15%. The best accuracy of the LDA model is 59.06% from subject 8, and the average accuracy is 52.50%. The average loss is 0.49 of the SVM and 0.48 of the LDA.

TABLE 1

Table 1. Griping vs. opening classification based on ERSP estimation.

Similarly, Table 2 shows the performance of SVM and LDA in the pronation vs. supination classification. Based on single channel ERSP, subject 3 obtains the best accuracy in both SVM (62.50%) and LDA (59.06%) models. The average accuracy is 52.19% of SVM and 52.32% of LDA. The average loss is 0.47 of the SVM and 0.48 of the LDA. Based on motor cortex area channels, the best accuracy of the SVM model is 56.25% from subject 6 and the best accuracy of the LDA model is 58.44% from subject 1. The average accuracy is 50.66% of the SVM and 48.13% of the LDA. The average loss is 0.49 of the SVM and 0.52 of the LDA.

TABLE 2

Table 2. Pronation vs. supination classification based on ERSP estimation.

4.2. Accuracy based on clustered feature

In this section, 20 trials are utilized to estimate the mean ERSP value as the center cluster reference. The binary classification strategy is implemented as Figure 4 shown. Table 3 shows the average accuracy and corresponding standard deviation of 9 subjects in griping vs. opening classification. Through applying clustered features on a single channel, the best accuracy is 91.25% of SVM model and 90.94% of LDA model from subject 6. The average accuracy is 76.15% of the SVM and 71.81% of the LDA. The average loss is 0.28 and 0.24, respectively. Through applying multi-channel clustered features, subject 1 obtains the best accuracy (95.94%) of SVM and subject 7 obtains the best accuracy (95.63%) of LDA. The average accuracy is 85.87% of the SVM and 84.03% of the LDA. The average loss is 0.14 and 0.16, respectively.

FIGURE 4

Figure 4. The architecture of the binary right hand motor imagery classification.

TABLE 3

Table 3. Griping vs. opening classification based on clustered features.

Table 4 shows the average accuracy and corresponding standard deviation of 9 subjects in pronation vs. supination classification. Subject 7 obtains the best accuracy in both single (94.06% of the SVM and 90.94% of the LDA) and multi-channels clustered features (98.75% of SVM and 97.50% of LDA). The average accuracy based on single channel clustered features is 74.89% of the SVM and 72.88% of the LDA. The average loss is 0.27 and 0.25, respectively. Besides, the average accuracy based on multi-channel clustered features is 84.55% of the SVM and 83.81% of the LDA. The average loss is 0.15 and 0.16, respectively.

TABLE 4

Table 4. Pronation vs. supination classification based on clustered features.

4.3. Parameter tuning

4.3.1. Trial number for clustered feature calculation

In this part, the accuracy of calculating the multi-channel clustered feature by different trial number is enumerated. Table 5 shows the accuracy of SVM model performance when applying clustered feature on multi-channel in griping and opening classification. All the subjects obtain the highest accuracy when utilizing 10 trials to compute the multi-channel clustered feature. The accuracy drops a lot from 90.77 to 84.03% when comparing the result of applying 10 and 20 trials. The accuracy of all subjects drops gradually and slightly when the trial number of calculating multi-channel clustered features raises from 20 to 80. The mean average varies within 5% between adjacent two groups.

TABLE 5

Table 5. The accuracy (%) for applying different trial numbers to calculate the multi-channels clustered feature in griping vs. opening classification.

4.3.2. Clustered ERSP estimation of different baseline

The key point of event related spectral perturbation is the baseline setting. According to the previous experimental report and the assumption we proposed, an observation period is added to enhance the motor imagery activities in our experiment design. As a result, we utilize a different baseline to calculate the cluster mean ERSP estimation and compare the performance in both classifiers based on the multi-channel clustered feature in this part. Based on the SVM model, onlyăsubject 3 and 8 in the observation period obtain a better accuracy than others in the imagery period. Meanwhile, subject 2 and 8 in the observation period obtained better accuracy than others in the imagery period based on the LDA model. In general, the clustered feature estimated by imagery period baseline yields a better result than observation period baseline. (In SVM, observation vs. imagery, p = 0.02, In LDA, observation vs. imagery, p = 0.08).

4.3.3. Channel selection

It is necessary to take the channel selection into account. As the description above, 21 channels are employed to calculate the clustered features because these channels cover the brain motor cortex area and record the EEG signals for motor imagery tasks. Considering both contralateral motor cortex and the ipsilateral motor cortex has its own physiological effect on brain activities, we test the clustered features performance by applying different channel selection combinations in this part. As Table 7 shown, channel FC5, FC3, FC1, C5, C3, C1, CP5, CP3, CP1 represent the left contralateral motor cortex while channel FC6, FC4, FC2, C6, C4, C2, CP6, CP4, CP2 represent the right ipsilateral motor cortex. In the LDA model, full motor cortex channel selection obtains the best accuracy except for subject 6 by single C3 channel, subject 8 by single C4 channel and subject 7,9 by left contralateral motor cortex. In addition, the average accuracy is 72.71% based on the C4 channel, 78.47% based on the left motor cortex, and 77.99% based on the right motor cortex. In addition, only the p-value (full motor cortex channels vs. single channel C3 or C4) is <0.05.

4.3.4. Frequency band selection

As the most informative frequency band in motor imagery tasks, μ band (8–12 Hz) and β band (16–24 Hz) play a vital role in the EEG signal analysis. Considering their different physiological meanings, the clustered feature performance of LDA model in each frequency is illustrated as Table 8 shown. The average accuracy of μ band is 75.31% while the average accuracy of β band is 79.20%. In addition, the wide band (8–24 Hz) which contains the μ band and β band obtains the average accuracy at 84.03%. The p-value of wide band vs. μ band is <0.05 while the p-value of wide band vs. μ band and β band vs. μ band is >0.05.

5. Discussion

5.1. Performance of classification

The classification result and significance level is compared as Figure 5 shown. In griping vs. opening classification (Figures 5A,B), the average accuracy (47.95, 51.15, 51.28, and 52.50%) is around chance level (50%) through single channel and multi-channel ERSP input. The result shows the low performance of SVM and LDA which means the classifiers cannot work properly to distinguish these two classes. Through applying clustered feature on single channel and multi-channel, the accuracy raises from 47.95 to 76.15%, 51.15 to 85.87% in SVM model and 51.28 to 71.81%, 52.50 to 84.03% in LDA.

FIGURE 5

Figure 5. Comparisons of classification performances using single channel ERSP, multi-channel ERSP, single channel clustered feature, and multi-channel feature. *p < 0.05, **p < 0.001. Dashed line denotes the chance level and CF represents clustered feature. (A,B) Illustrate the ANOVA test result of SVM and LDA in griping vs. opening classification. (C,D) illustrate the ANOVA test result of SVM and LDA in pronation vs. supination classification.

Similarly, in pronation vs. supination classification (Figures 5C,D), both classifiers, whose accuracy are 52.19, 50.66, 52.32, and 48.13%, cannot distinguish the motor imagery through single and multi-channel ERSP input as well. By applying clustered feature on single channel and multi-channel, the accuracy raises from 52.19 to 74.89%, 50.66 to 85.87% in SVM model and 52.32 to 72.88%, 48.13 to 84.03% in LDA.

From statistical analysis as Figure 5 shown, there is a significant improvement by applying clustered feature (p < 0.001) in all classifiers and motor imagery classifications. In addition, there also is a significant difference between single channel and multi-channel clustered feature input in both classifiers(SVM and LDA).

5.2. Parameter setting

The feature extraction method we proposed in this paper is utilizing a mean ERSP estimation as a cluster to calculate the Euclidean distance. The main factor which highly affects the performance is how many trials are suitable for the cluster ERSP estimation. The result from Table 5 shows that the average accuracy drops gradually with the increasement of trials. However, from the statistical analysis, the p < 0.05 between each group except trial number equaling to 10. The result shows that there's no significant difference between utilizing 20 or up to 80 trials to calculate the clustered feature. In the cluster methods, the centroid value may deviate from the true value with the reduction of trial number for estimation. This causes the overfitting during the classifier's training and tends to obtain a very high accuracy. Therefore, there's a trade off in the cluster ERSP estimation. In this paper, we employ 20 trials to estimate the cluster ERSP considering the size of the dataset.

As for baseline setting, the observation period which is supposed to enhance the motor imagery activities causes the baseline changing of the ERSP estimation in our experiment design. The result from Table 6 shows that imagery baseline estimation yields a better accuracy than observation period baseline. However, the p-value of two classifiers in both conditions shows that there is no significant difference between two baselines (p = 0.02 in the SVM and p = 0.08 in the LDA). As a result, we set 1,000 ms before imagery epoch as baseline considering better accuracy and signal continuity.

TABLE 6

Table 6. The multi-channel clustered feature calculation based on different baseline time in griping vs. opening classification.

In addition, we select different channels and frequency bands to explore the best combination in the motor imagery task. The accuracy from Table 7 illustrates that the channels covering the full motor cortex area obtains the best result. Based on ANOVA test, single contralateral channel C3 shows no significant difference with the single ipsilateral channel C4. Similarly, there is no significant difference between multi contralateral channels (i.e., left motor cortex), multi ipsilateral channels (i.e., right motor cortex), and full motor cortex channels (i.e., 21 channels). However, clustered features based on the full motor cortex area have significant differences with single channel C3 and C4. Therefore, we employ 21 channels to compute the clustered feature considering its better accuracy. The accuracy from Table 8 compares the LDA performance based on multi- channel clustered feature different frequency bands. The wide frequency band obtains the best average accuracy though subject 8 obtains the best accuracy in μ band and subject 6 obtains the best accuracy in β band. The ANOVA test shows that the wide frequency band feature has the significant difference with the μ band feature while no significant difference with the β band feature. As a result, the wide frequency band is extracted to compute the clustered feature in our classification strategy.

TABLE 7

Table 7. The LDA model performance of clustered feature based on different channel selection in griping vs. opening classification.

TABLE 8

Table 8. The LDA model performance of clustered feature based on different frequency band in griping vs. opening classification.

5.3. Comparative result

In motor imagery research based on EEG signals, feature extraction algorithms such as common spatial pattern (CSP) and filter bank common spatial pattern (FBCSP) are widely employed. Suwannarat et al. (2018) utilized these methods on same hand motor imagery classification. Meanwhile, Chu et al. (2020) applied tangent space (TS) features and partial least square (PLS) algorithms to improve the performance of discriminant analysis classifiers. We compare our proposed feature with their results of the same classifier(LDA) and show the accuracy as Table 9 follows.

TABLE 9

Table 9. Comparative accuracy (%) with other features in linear discriminant analysis.

Clustered feature yields a better accuracy than applying CSP, FBCSP and tangent feature while has the similar accuracy to TS-PLS (80.02% in griping vs. opening and 78.55% in pronation vs. supination). However, our proposed feature also has limitations that it needs prior information to estimate the cluster.

6. Conclusion

In this paper, we estimate the mean ERSP from EEG signal. Then we employ the mean ERSP as cluster reference to calculate the feature by Euclidean distance and utilize the clustered feature to train two classical machine learning classifiers. The accuracy obtained by both classifiers in all motor imagery tasks is over 80%. After fast configuration on the subject to estimate the clustered feature, we propose a possibility to apply on online analysis.

In future work, we will consider the optimization method on cluster estimation, develop an adaptive method on searching the best channels and employ other feature extraction algorithms. On the basis of this paper, we will develop a real-time brain computer interface such as robotic grasping system, prosthesis controlling, etc.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by Ethics Committees of the Tokyo Institute of Technology (Ethics Number: 2019001). The patients/participants provided their written informed consent to participate in this study.

Author contributions

ZZ and YK contributed to conception and design of the study. ZZ organized the database, performed the statistical analysis, wrote the first draft, and sections of the manuscript. Both authors contributed to manuscript revision, read, and approved the submitted version.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Abbas, W., and Khan, N. A. (2018). “FBCSP-based multi-class motor imagery classification using BP and TDP features,” in 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (Honolulu, HI), 215–218. doi: 10.1109/EMBC.2018.8512238

PubMed Abstract | CrossRef Full Text | Google Scholar

Adeli, H., Zhou, Z., and Dadmehr, N. (2003). Analysis of EEG records in an epileptic patient using wavelet transform. J. Neurosci. Methods 123, 69–87. doi: 10.1016/S0165-0270(02)00340-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Akin, M. (2002). Comparison of wavelet transform and FFT methods in the analysis of EEG signals. J. Med. Syst. 26, 241–247. doi: 10.1023/A:1015075101937

PubMed Abstract | CrossRef Full Text | Google Scholar

Ang, K. K., Chin, Z. Y., Wang, C., Guan, C., and Zhang, H. (2012). Filter bank common spatial pattern algorithm on BCI competition IV datasets 2a and 2b. Front. Neurosci. 6, 39. doi: 10.3389/fnins.2012.00039

PubMed Abstract | CrossRef Full Text | Google Scholar

Bashashati, A., Fatourechi, M., Ward, R. K., and Birch, G. E. (2007). A survey of signal processing algorithms in brain-computer interfaces based on electrical brain signals. J. Neural Eng. 4, R32. doi: 10.1088/1741-2560/4/2/R03

PubMed Abstract | CrossRef Full Text | Google Scholar

Bentlemsan, M., Zemouri, E.-T., Bouchaffra, D., Yahya-Zoubir, B., and Ferroudji, K. (2014). “Random forest and filter bank common spatial patterns for EEG-based motor imagery classification,” in 2014 5th International Conference on Intelligent Systems, Modelling and Simulation (Langkawi), 235–238. doi: 10.1109/ISMS.2014.46

CrossRef Full Text | Google Scholar

Chu, Y., Zhao, X., Zou, Y., Xu, W., Song, G., Han, J., et al. (2020). Decoding multiclass motor imagery EEG from the same upper limb by combining Riemannian geometry features and partial least squares regression. J. Neural Eng. 17, 046029. doi: 10.1088/1741-2552/aba7cd

PubMed Abstract | CrossRef Full Text | Google Scholar

Corley, I. A., and Huang, Y. (2018). “Deep EEG super-resolution: upsampling EEG spatial resolution with generative adversarial networks,” in 2018 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI) (Las Vegas, NV), 100–103. doi: 10.1109/BHI.2018.8333379

CrossRef Full Text | Google Scholar

da Silva, F. L. (1991). Neural mechanisms underlying brain waves: from neural membranes to networks. Electroencephalogr. Clin. Neurophysiol. 79, 81–93. doi: 10.1016/0013-4694(91)90044-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Das, R., Lopez, P. S., Khan, M. A., Iversen, H. K., and Puthusserypady, S. (2020). “FBCSP and adaptive boosting for multiclass motor imagery BCI data classification: a machine learning approach,” in 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC) (Toronto, ON), 1275–1279.

Google Scholar

Decety, J., and Ingvar, D. H. (1990). Brain structures participating in mental simulation of motor behavior: a neuropsychological interpretation. Acta Psychol. 73, 13–34. doi: 10.1016/0001-6918(90)90056-L

PubMed Abstract | CrossRef Full Text | Google Scholar

Elstob, D., and Secco, E. L. (2016). A low cost EEG based BCI prosthetic using motor imagery. arXiv preprint arXiv:1603.02869. doi: 10.5121/ijitcs.2016.6103

CrossRef Full Text | Google Scholar

Grandchamp, R., and Delorme, A. (2011). Single-trial normalization for event-related spectral decomposition reduces sensitivity to noisy trials. Front. Psychol. 2, 236. doi: 10.3389/fpsyg.2011.00236

PubMed Abstract | CrossRef Full Text | Google Scholar

Hjorth, B. (1975). An on-line transformation of EEG scalp potentials into orthogonal source derivations. Electroencephalogr. Clin. Neurophysiol. 39, 526–530. doi: 10.1016/0013-4694(75)90056-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Hsu, W.-Y., and Sun, Y.-N. (2009). EEG-based motor imagery analysis using weighted wavelet transform features. J. Neurosci. Methods 176, 310–318. doi: 10.1016/j.jneumeth.2008.09.014

PubMed Abstract | CrossRef Full Text | Google Scholar

Kappes, H. B., and Morewedge, C. K. (2016). Mental simulation as substitute for experience. Soc. Pers. Psychol. Compass 10, 405–420. doi: 10.1111/spc3.12257

CrossRef Full Text | Google Scholar

Kos' Myna, N., Tarpin-Bernard, F., and Rivet, B. (2014). “Bidirectional feedback in motor imagery BCIs: learn to control a drone within 5 minutes,” in CHI'14 Extended Abstracts on Human Factors in Computing Systems (New York, NY), 479–482. doi: 10.1145/2559206.2574820

CrossRef Full Text | Google Scholar

Lawhern, V. J., Solon, A. J., Waytowich, N. R., Gordon, S. M., Hung, C. P., and Lance, B. J. (2018). EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces. J. Neural Eng. 15, 056013. doi: 10.1088/1741-2552/aace8c

PubMed Abstract | CrossRef Full Text | Google Scholar

Lotte, F., Congedo, M., Lécuyer, A., Lamarche, F., and Arnaldi, B. (2007). A review of classification algorithms for EEG-based brain-computer interfaces. J. Neural Eng. 4, R1–R13. doi: 10.1088/1741-2560/4/2/R01

PubMed Abstract | CrossRef Full Text | Google Scholar

Luck, S. J. (2014). An Introduction to the Event-Related Potential Technique. MIT Press.

Google Scholar

Ma, X., Qiu, S., Wei, W., Wang, S., and He, H. (2019). Deep channel-correlation network for motor imagery decoding from the same limb. IEEE Trans. Neural Syst. Rehabil. Eng. 28, 297–306. doi: 10.1109/TNSRE.2019.2953121

PubMed Abstract | CrossRef Full Text | Google Scholar

Malki, A., Yang, C., Wang, N., and Li, Z. (2015). “Mind guided motion control of robot manipulator using EEG signals,” in 2015 5th International Conference on Information Science and Technology (ICIST) (Changsha), 553–558. doi: 10.1109/ICIST.2015.7289033

CrossRef Full Text | Google Scholar

McFarland, D. J., McCane, L. M., David, S. V., and Wolpaw, J. R. (1997). Spatial filter selection for EEG-based communication. Electroencephalogr. Clin. Neurophysiol. 103, 386–394. doi: 10.1016/S0013-4694(97)00022-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Meng, J., Zhang, S., Bekyo, A., Olsoe, J., Baxter, B., and He, B. (2016). Noninvasive electroencephalogram based control of a robotic arm for reach and grasp tasks. Sci. Rep. 6, 1–15. doi: 10.1038/srep38565

PubMed Abstract | CrossRef Full Text | Google Scholar

Miladinović, A., Ajčević, M., Jarmolowska, J., Marusic, U., Silveri, G., Battaglini, P. P., et al. (2020). Performance of EEG motor-imagery based spatial filtering methods: a BCI study on stroke patients. Proc. Comput. Sci. 176, 2840–2848. doi: 10.1016/j.procs.2020.09.270

CrossRef Full Text | Google Scholar

Neuper, C., Scherer, R., Reiner, M., and Pfurtscheller, G. (2005). Imagery of motor actions: differential effects of kinesthetic and visual-motor mode of imagery in single-trial EEG. Cogn. Brain Res. 25, 668–677. doi: 10.1016/j.cogbrainres.2005.08.014

PubMed Abstract | CrossRef Full Text | Google Scholar

Perrin, F., Pernier, J., Bertrand, O., and Echallier, J. F. (1989). Spherical splines for scalp potential and current density mapping. Electroencephalogr. Clin. Neurophysiol. 72, 184–187. doi: 10.1016/0013-4694(89)90180-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Pfurtscheller, G. (1992). Event-related synchronization (ERS): an electrophysiological correlate of cortical areas at rest. Electroencephalogr. Clin. Neurophysiol. 83, 62–69. doi: 10.1016/0013-4694(92)90133-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Pfurtscheller, G., and Aranibar, A. (1977). Event-related cortical desynchronization detected by power measurements of scalp EEG. Electroencephalogr. Clin. Neurophysiol. 42, 817–826. doi: 10.1016/0013-4694(77)90235-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Pfurtscheller, G., Brunner, C., Schlögl, A., and Da Silva, F. L. (2006). Mu rhythm (de) synchronization and EEG single-trial classification of different motor imagery tasks. NeuroImage 31, 153–159. doi: 10.1016/j.neuroimage.2005.12.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Pfurtscheller, G., and Neuper, C. (1997). Motor imagery activates primary sensorimotor area in humans. Neurosci. Lett. 239, 65–68. doi: 10.1016/S0304-3940(97)00889-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Polat, K., and Güneş, S. (2007). Classification of epileptiform EEG using a hybrid system based on decision tree classifier and fast Fourier transform. Appl. Math. Comput. 187, 1017–1026. doi: 10.1016/j.amc.2006.09.022

CrossRef Full Text | Google Scholar

Ramadan, R. A., and Vasilakos, A. V. (2017). Brain computer interface: control signals review. Neurocomputing 223, 26–44. doi: 10.1016/j.neucom.2016.10.024

CrossRef Full Text | Google Scholar

Rodríguez-Ugarte, M., Iáñez, E., Ortiz, M., and Azorín, J. M. (2018). Improving real-time lower limb motor imagery detection using TDCS and an exoskeleton. Front. Neurosci. 12, 757. doi: 10.3389/fnins.2018.00757

PubMed Abstract | CrossRef Full Text | Google Scholar

Schirrmeister, R. T., Springenberg, J. T., Fiederer, L. D. J., Glasstetter, M., Eggensperger, K., Tangermann, M., et al. (2017). Deep learning with convolutional neural networks for EEG decoding and visualization. Hum. Brain Mapp. 38, 5391–5420. doi: 10.1002/hbm.23730

PubMed Abstract | CrossRef Full Text | Google Scholar

Srinivasan, R. (1999). Methods to improve the spatial resolution of EEG. Int. J. Bioelectromagn. 1, 102–111.

Google Scholar

Suwannarat, A., Pan-Ngum, S., and Israsena, P. (2018). Comparison of EEG measurement of upper limb movement in motor imagery training system. Biomed. Eng. Online 17, 1–22. doi: 10.1186/s12938-018-0534-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Tariq, M., Trivailo, P. M., and Simic, M. (2019). Classification of left and right knee extension motor imagery using common spatial pattern for BCI applications. Proc. Comput. Sci. 159, 2598–2606. doi: 10.1016/j.procs.2019.09.256

CrossRef Full Text | Google Scholar

Togha, M. M., Salehi, M. R., and Abiri, E. (2019). Improving the performance of the motor imagery-based brain-computer interfaces using local activities estimation. Biomed. Signal Process. Control 50, 52–61. doi: 10.1016/j.bspc.2019.01.008

CrossRef Full Text | Google Scholar

Wang, Y., Gao, S., and Gao, X. (2006). “Common spatial pattern method for channel selection in motor imagery based brain-computer interface,” in 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference (Shanghai), 5392–5395. doi: 10.1109/IEMBS.2005.1615701

PubMed Abstract | CrossRef Full Text | Google Scholar

Yoshimura, N., Tsuda, H., Kawase, T., Kambara, H., and Koike, Y. (2017). Decoding finger movement in humans using synergy of EEG cortical current signals. Sci. Rep. 7, 1–11. doi: 10.1038/s41598-017-09770-5

PubMed Abstract | CrossRef Full Text | Google Scholar