- College of Intelligence Science and Technology, National University of Defense Technology, Changsha, China
An electroencephalogram (EEG) functional connectivity (FC) network is individualized and plays a significant role in EEG-based person identification. Traditional FC networks are constructed by statistical dependence and correlation between EEG channels, without considering the spatial relationships between the channels. The individual identification algorithm based on traditional FC networks is sensitive to the integrity of channels and crucially relies on signal preprocessing; therefore, finding a new presentation for FC networks may help increase the performance of the identification algorithms. EEG signals are smooth across space owing to the volume conduction effect. Considering such spatial relationships among channels can provide a more accurate representation of FC networks. In this study, we propose an EEG FC network with virtual nodes that combines the spatial relationships and functional connectivity of channels. The comparison results for individual identification show that the novel EEG network is more individualized and achieves an accuracy of 98.64% for data without preprocessing. Furthermore, our algorithm is more robust in reducing the number of channels and can perform well even when a large area of channels is removed.
1. Introduction
Electroencephalogram (EEG) records the electrical activity in the brain, usually along the scalp surface (Jalaly Bidgoly et al., 2020). Individual differences in EEG, which result from specific patterns of individual brain activity, are mainly determined by the genetic and developmental environment (van Beijsterveldt and Boomsma, 1994). Taking advantage of the significant individual differences, EEG has been used as a more appropriate biometric than traditional methods such as fingerprints or faces (Gui et al., 2019).
In previous studies, univariate features such as the coefficients of auto-regression (AR) (Campisi and Rocca, 2014; Rocca et al., 2014; Keshishzadeh et al., 2016; Bhateja et al., 2019), power spectral density (PSD) (Palaniappan and Mandic, 2007; Ashby et al., 2011; Pham et al., 2015; Di et al., 2019), and wavelet transform (WT) (Kumari and Vaish, 2015; Kaur et al., 2017; Alyasseri et al., 2018; Yang et al., 2018) have been widely used to represent individual differences in EEG. As they are obtained by calculating signals from each electrode, univariate features are sensitive to changes in EEG amplitudes (Liu et al., 2023), which may amplify the intra-person variation. Bivariate features such as the connectivity between channels can effectively be less sensitive to amplitude interference owing to inevitably physiological or psychological factors (Zhang et al., 2021, 2022).
Functional connectivity (FC) is the most commonly used bivariate feature that captures statistical dependence or correlations between EEG channels (Bastos and Schoffelen, 2016). Ashenaei et al. found that using time-frequency FC metrics as a brain connectivity matrix can extract more discriminative features (Ashenaei et al., 2022). Chang et al. (2020) proposed a new feature extraction method based on a directed FC network and found it to be efficient for EEG-based person identification. Wang et al. (2019a). represented EEG signals as graphs based on within-frequency and cross-frequency FC estimates. The results showed that this was a more robust biometric trait than directly using the univariate features.
Traditional EEG FC networks largely rely on the functional connectivity between two channels, and recent studies have reported the limitations of such FC networks (Hassan and Wendling, 2018). Fusing other information such as the spatial relationship between channels may provide a more accurate description of FC networks. According to the volume conduction effect (Brunner et al., 2016), the signals of the two adjacent electrodes were more similar. Therefore, introducing such spatial relationship between channels could provide more information for EEG FC network.
In this study, an EEG FC network with virtual nodes that fuse spatial structures was proposed. A person identification algorithm was put forwarded using the EEG network with virtual nodes as the input of a graph convolutional neural network (VN-GCN) for EEG identification. For the data without preprocessing, our method achieved 98.64% accuracy, which is a significant improvement over existing-methods. This implies that more individualized information can be represented by integrating the spatial structure of the channels. In channel reduction experiments, VN-GCN showed less sensitivity to the number of channels and can work well even when removing a large area of channels.
The remainder of this paper is organized as follows. In Section 2, we briefly present the details of the method and the proposed VN-GCN model. In Section 3, we introduce and discuss the experimental results on person identification. Section 4 summarizes the findings of this study.
2. Methods
2.1. Functional connectivity calculation
To construct the EEG FC network, a variety of functional connectivity indices can be employed to estimate the relationship and weight between nodes, such as coherence, correlation coefficient, phase-lag index, and phase-locking value (PLV). Among them, PLV is a popular method used in EEG identification tasks. PLV (Niso et al., 2013) represents the instantaneous phase difference between two channel signals and is defined as
2.2. EEG functional connectivity network with spatial structure
Contrary to most traditional EEG FC networks, the influence of the spatial structure of channels on signal conduction patterns was considered. By employing virtual nodes, the channel information was spatially integrated. The structure of the proposed EEG network is shown in Figure 1.
Figure 1. Schematic of the EEG functional connectivity network with virtual nodes. (A) Channel distribution of the 10–10 international system. The 64 electrodes are evenly divided into eight groups which are represented by different color. (B) For each group, one local virtual node was set to connect channels in the group and a global virtual node was set to connect all local virtual nodes.
The 10–10 international system is an internationally recognized method to describe and apply the location of scalp electrodes. The channel distribution of the 10–10 international system is shown in Figure 1A. It defines the spatial relationship of channels. Our FC network consists of 64 real nodes, eight local virtual nodes and one global virtual node. The 64 real nodes correspond to 64 electrodes and were divided into eight groups of equal numbers (Figure 1A). The real nodes connect to each other with weights determined by the functional connectivity. Each virtual node corresponds to one group and connects all real nodes of the corresponding group. The global-level virtual node connects all local virtual nodes. Taking the 10–10 international system as an example, the size of the new adjacency matrix is and the size of the new node strength matrix is (N is the dimension of the node feature).
2.3. Graph convolutional neural network
A graph consisting of n nodes can be defined as , where V denotes the set of vertices and E denotes the set of edges , where . The connection relationship between vertices can be represented by an adjacency matrix , which indicates the connection weight between i and j. In this work, the adjacency matrix A is defined as the sparse FC matrix with virtual nodes mentioned above. Matrix can represent the feature of all nodes. As we mainly concentrated on the connection patterns of channels, the feature of all nodes can be simply put as an all-1 matrix.
Let L denotes the Laplacian matrix of graph G, defined as:
The normalized Laplacian matrix (Kipf and Welling, 2017) is usually used for graph convolution. Based on the renormalization technique, can be obtained by transforming the adjacency matrix A, which is defined as
The GCN graph convolution is realized by aggregating the features of the nodes and their adjacent nodes. The single convolutional layer k of GCN can be defined as
2.4. VN-GCN framework for EEG identification
The design of the VN-GCN model, which uses the EEG FC network with virtual nodes as the input of the GCN, is shown in Figure 2. The EEG signals and distribution of channels can be obtained from the acquisition process. The original EEG network was constructed by computing the functional connectivity between the EEG signals. By combining the spatial information of the channels, an EEG FC network with virtual nodes was constructed, as shown in Section 2.2, and was set as the input of the two graph convolution layers. Subsequently, a flattened layer expanded the output matrix into one dimension, and a dense layer was used to realize mapping from the feature to label spaces. Finally, a softMax layer was used to classify the output layer. The network was trained by iterating categorical cross entropy loss.
3. Results and discussion
In order to evaluate the individualization of EEG FC network after adding virtual nodes, person identification experiments based on VN-GCN were designed. The raw EEG data (n channels × t seconds) from different subjects was feed to the input terminal of VN-GCN. And the task of VN-GCN is to predict the identity tags of the subjects at the output terminal (see Figure 2). The accuracy of the prediction is calculated to evaluate the individualization of FC network with virtual nodes, and compared with person identification methods which based on traditional FC network.
3.1. Dataset
An online dataset, the PhysioNet EEG Motor Movement/Imagery Dataset (Goldberger et al., 2000; Schalk et al., 2004), was used to evaluate the performance of the VN-GCN and other methods. This dataset consists of EEG recordings from 109 volunteers, each of which performed 14 experimental runs: two one-minute baseline runs [one with eyes open (EO) and the other with eyes closed (EC)], and three two-minute runs of each of the following four tasks (opening or closing the left or right fist, imaging opening or closing the left or right fist, opening or closing both fists or both feet, and imaging opening or closing both fists or feet). EEGs were recorded from 64 electrodes (n = 64) as the 10–10 international system.
To evaluate the generalizability of VN-GCN, another popular public EEG dataset, the UCI KDD EEG dataset (Zhang et al., 1995) was also used to make further experiments. The records of this dataset are obtained by 64 electrodes (n = 64) at a sampling rate of 256 Hz. The number of subjects was 122 including 45 normal and 77 alcoholic males.
A moving window of 1 s with a 50% overlap was used to generate the training and testing samples. Most existing studies tend to add preprocessing steps such as ICA (Winkler et al., 2014) to reduce the noise of the original EEG signals. However, in practice, different noise reduction methods should be designed based on the EEG acquisition equipment. This indicates that the model requires more computation and time. In the subsequent experiments, EEG data without preprocessing were used.
3.2. Model accuracy and comparison
The identification accuracy of our VN-GCN was compared with that of two of the most popular EEG identification models developed in recent years. These are Tina’s GCN model, which was proposed in 2021 (Behrouzi and Hatzinakos, 2021), and an EEG identification model based on CNN and functional connectivity, which was proposed by Wang in 2019 (Wang et al., 2019b). The structures of these three models are listed in Table 1. To evaluate the computational complexity in the training of VN-GCN and the other two model, the parameter amounts of each layer are listed in Table 2. In creating a personalized FC network, the major computational step is calculating the PLV between each pair of channels, so the computational complexity of constructing a FC network is , where T is the number of time points, n is the number of channels.
The comparison results on the PhysioNet EEG Motor Movement/Imagery Dataset are presented in Table 3, and the results on the UCI KDD EEG dataset are given at Table 4. Any preprocessing was not conducted. A five-fold cross-validation was performed.
Clearly from the results in Table 3, the accuracy achieved by the VN-GCN model was 8.66 and 2.93% higher than that of the CNN-FC and Tina’s GCN models, respectively, when using EEG data in the resting-state EO. For the EC, the values were 6.63 and 0.77%, respectively. This demonstrates that VN-GCN can achieve higher accuracy with un-preprocessed data and fuse the spatial structure of channels and functional connectivity through virtual nodes. Comparing the accuracy in the resting-state EO and EC, it can be found that the accuracy at EC was much lower than that in EO. One possible reason for this is that people tend to have better control over other brain activities during EO, whereas the brain is more active and uncontrollable during EC. The result on the UCI KDD EEG dataset also shown that our VN-GCN model could achieve the best results.
3.3. Effects of channel reduction
In the aforementioned experiment, 64-channel EEGs were used. In theory, more channels of EEG signals indicate that the characterization of brain connections is more detailed, and the EEG FC network is more individualized. However, higher costs are incurred in both the data acquisition and analysis stages when there are more channels. Channel-reduction experiments were conducted to provide support for the application of the technology in the case of fewer channels. The original distribution of the 64 channels was uniformly reduced to 8, 16, 24, 32, 40, and 48 channels, as shown in Figure 3.
Figure 3. Layout of channels after uniformly reduction. (A–F): 56, 48, 40, 32, 24, and 16 channels were, respectively, retained.
The experiment was conducted when the three models had the same structure described in Section 3.2, and the same functional connectivity (PLV) was used. The results are summarized in Table 5 and compared in Figure 4.
Figure 4. Comparison of classification accuracy of three models after uniformly reducing the channels. (A) In resting state EO and (B) in resting state EC.
The results indicate that the VN-GCN model was more robust than Tina’s GCN and CNN-FC models. When using EEG data in resting-state EO, the accuracy of VN-GCN changed slightly and was maintained at more than 90% when the number of the removed channels was less than half. When reduced to only 16 channels, the classification accuracy of the VN-GCN model was less than 40%, which is similar to that of Tina’s GCN and CNN-FC models.
Therefore, when using EEG data with fewer channels, the number of channels should be maintained between 64 and 32 to ensure the accuracy of the model.
3.4. Sensitive to brain region removal
The arrangement of EEG channels on the brain surface corresponds to specific brain regions. To explore the contribution of brain regions to individual differences, an experiment on brain region removal was conducted. Figure 5 shows the relationship between the channels and brain regions based on 10–10 international system.
Figure 5. Schematic of the brain region removal. All the channels are divided into five regions which are overlapping according to their signs in the 10–10 international system.
According to the division of brain regions given above, we successively removed the corresponding channels and compared the accuracy of VN-GCN with that of Tina’s GCN and CNN-FC models. The results are presented in Table 6 and compared in Figure 6.
Figure 6. Comparison of classification accuracy of three models after brain region removal. (A) In resting state EO and (B) in resting state EC.
First, the VN-GCN had the lowest classification accuracy drop compared to the other two models when the brain regions were removed. It can also be seen that when channels are removed according to the brain regions, the change rule of the accuracy of the three models is the same. The classification accuracy decreased the most when the channels of the frontal lobe region were removed and least when the channels of the occipital and temporal lobe regions were removed. This is because the frontal lobe region has the largest number of channels. The number of channels in the parietal and central lobe regions was not significantly different from that in the frontal lobe region; however, the accuracy drop in the former was much smaller. Second, this rule seems to be amplified in resting-state EC, which means that the models are more sensitive in this state.
4. Conclusion
In this study, an EEG FC network that fuses the spatial structures of channels was proposed and used for EEG identification. By introducing virtual nodes that communicate with local channels, the structural information of the channels can be integrated into the FC network. The basic principle of integrating the spatial structure of channels is based on the volume conduction effect, which reflects the EEG conduction pattern. Taking an FC network with virtual nodes as the input to the graph convolutional network, the VN-GCN model was proposed for person identification tasks. The results show that, compared with traditional methods that construct an EEG network with only functional connectivity, VN-GCN achieves a higher identification accuracy. In the channel-reduction experiments, the results demonstrated that VN-GCN is more robust against channel reduction, and the result still holds when the channels of an entire brain region are removed. However, adding virtual nodes increases the scale of the FC network, which leads to larger computational complexity of the model, and this may limit the application in some cases. Besides, the local virtual nodes can only integrate the electrodes in the same group and cannot integrate electrodes in different groups, even if they are spatially adjacent. Finally, although our method can achieve the accuracy of more than 98% without preprocessing, the performance of the method in the presence of noise or artifacts needs further study.
In conclusion, by integrating the spatial structure with virtual nodes, the EEG FC network was more discriminated and robust among different people. Moreover, this is an effective method to achieve more accessible EEG-based person identification for use in real-world scenarios.
Data availability statement
The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.
Author contributions
ML: Writing – original draft, Writing – review & editing. JY: Writing – original draft, Writing – review & editing. WT: Writing – review & editing. XJ: Writing – review & editing.
Funding
The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This work was supported by the Natural Science Foundation of China, grant number 62076248.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Alyasseri, Z. A. A., Khader, A. T., Al-Betar, M. A., Papa, J. P., and Alomari, O. A. (2018). “EEG-based person authentication using multi-objective flower pollination algorithm” in 2018 IEEE congress on evolutionary computation (CEC). Presented at the 2018 IEEE congress on evolutionary computation (CEC) (Rio de Janeiro: IEEE), 1–8.
Ashby, C., Bhatia, A., Tenore, F., and Vogelstein, J. (2011). “Low-cost electroencephalogram (EEG) based authentication” in 2011 5th international IEEE/EMBS conference on neural engineering. Presented at the 5th international IEEE/EMBS conference on neural engineering (NER 2011) (Cancun: IEEE), 442–445.
Ashenaei, R., Asghar Beheshti, A., and Yousefi Rezaii, T. (2022). Stable EEG-based biometric system using functional connectivity based on time-frequency features with optimal channels. Biomed. Sig Proc. Control 77:103790. doi: 10.1016/j.bspc.2022.103790
Bastos, A. M., and Schoffelen, J.-M. (2016). A tutorial review of functional connectivity analysis methods and their interpretational pitfalls. Front. Syst. Neurosci. 9:175. doi: 10.3389/fnsys.2015.00175
Behrouzi, T., and Hatzinakos, D. (2021). Understanding power of graph convolutional neural network on discriminating human EEG signal 2021 IEEE Canadian conference on electrical and computer engineering (CCECE). pp. 1–7
Behrouzi, T., and Hatzinakos, D. (2022). Graph variational auto-encoder for deriving EEG-based graph embedding. Pattern Recogn. 121:108202. doi: 10.1016/j.patcog.2021.108202
Bhateja, V., Gupta, A., Mishra, A., and Mishra, A. (2019). “Artificial neural networks based fusion and classification of EEG/EOG signals” in Information systems design and intelligent applications, advances in intelligent systems and computing. eds. S. C. Satapathy, V. Bhateja, R. Somanah, X.-S. Yang, and R. Senkerik (Singapore: Springer), 141–148.
Brunner, C., Billinger, M., Seeber, M., Mullen, T. R., and Makeig, S. (2016). Volume conduction influences scalp-based connectivity estimates. Front. Comput. Neurosci. 10:121. doi: 10.3389/fncom.2016.00121
Campisi, P., and Rocca, D.L. (2014). Brain waves for automatic biometric-based user recognition IEEE transactions on information forensics and security 9 26 February 2014.
Chang, W., Wang, H., Yan, G., and Liu, C. (2020). An EEG based familiar and unfamiliar person identification and classification system using feature extraction and directed functional brain network. Expert Syst. Appl. 158:113448. doi: 10.1016/j.eswa.2020.113448
Di, Y., An, X., He, F., Liu, S., Ke, Y., and Ming, D. (2019). Robustness analysis of identification using resting-state EEG signals. IEEE Access 7, 42113–42122. doi: 10.1109/ACCESS.2019.2907644
Goldberger, A. L., Amaral, L. A., Glass, L., Hausdorff, J. M., Ivanov, P. C., Mark, R. G., et al. (2000). PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. Circulation 101, E215–E220. doi: 10.1161/01.cir.101.23.e215
Gui, Q., Ruiz-Blondet, M. V., Laszlo, S., and Jin, Z. (2019). A survey on brain biometrics. ACM Comput. Surv. 51, 1–38. doi: 10.1145/3230632
Hassan, M., and Wendling, F. (2018). Electroencephalography source connectivity: aiming for high resolution of brain networks in time and space. IEEE Signal Process. Mag. 35, 81–96. doi: 10.1109/MSP.2017.2777518
Jalaly Bidgoly, A., Jalaly Bidgoly, H., and Arezoumand, Z. (2020). A survey on methods and challenges in EEG based authentication. Comput. Secur. 93:101788. doi: 10.1016/j.cose.2020.101788
Kaur, B., Singh, D., and Roy, P. P. (2017). A novel framework of EEG-based user identification by analyzing music-listening behavior. Multimed. Tools Appl. 76, 25581–25602. doi: 10.1007/s11042-016-4232-2
Keshishzadeh, S., Fallah, A., and Rashidi, S. (2016). “Improved EEG based human authentication system on large dataset” in 2016 24th Iranian conference on electrical engineering (ICEE). Presented at the 2016 24th Iranian conference on electrical engineering (ICEE) (Shiraz, Iran: IEEE), 1165–1169.
Kipf, T.N., and Welling, M. (2017). “Semi-supervised classification with graph convolutional networks,” in International Conference on Learning Representations (ICLR 2017). Toulon: OpenReview.net
Kumari, P., and Vaish, A. (2015). Brainwave based user identification system: a pilot study in robotics environment. Robot. Auton. Syst. 65, 15–23. doi: 10.1016/j.robot.2014.11.015
Liu, Y., Yu, Y., Ye, Z., Li, M., Zhang, Y., Zhou, Z., et al. (2023). Fusion of spatial, temporal, and spectral EEG signatures improves multilevel cognitive load prediction. IEEE Transact. Hum. Mach. Syst. 53, 357–366. doi: 10.1109/THMS.2023.3235003
Niso, G., Bruña, R., Pereda, E., Gutiérrez, R., Bajo, R., Maestú, F., et al. (2013). HERMES: towards an integrated toolbox to characterize functional and effective brain connectivity. Neuroinformatics 11, 405–434. doi: 10.1007/s12021-013-9186-1
Palaniappan, R., and Mandic, D. P. (2007). Biometrics from brain electrical activity: a machine learning approach. IEEE Trans. Pattern Anal. Mach. Intell. 29, 738–742. doi: 10.1109/TPAMI.2007.1013
Pham, T., Ma, W., Tran, D., Tran, D.S., and Phung, D. (2015). A study on the stability of EEG signals for user authentication 2015 7th international IEEE/EMBS conference on neural engineering (NER). 22–24 April 2015 122–125.
Rocca, D. L., Campisi, P., Vegso, B., Cserti, P., Kozmann, G., Babiloni, F., et al. (2014). Human brain distinctiveness based on EEG spectral coherence connectivity. IEEE Trans. Biomed. Eng. 61, 2406–2412. doi: 10.1109/TBME.2014.2317881
Schalk, G., McFarland, D. J., Hinterberger, T., Birbaumer, N., and Wolpaw, J. R. (2004). BCI2000: a general-purpose brain-computer interface (BCI) system. IEEE Trans. Biomed. Eng. 51, 1034–1043. doi: 10.1109/TBME.2004.827072
van Beijsterveldt, C. E., and Boomsma, D. I. (1994). Genetics of the human electroencephalogram (EEG) and event-related brain potentials (ERPs): a review. Hum. Genet. 94, 319–330. doi: 10.1007/BF00201587
Wang, M., El-Fiqi, H., Hu, J., and Abbass, H. A. (2019a). Convolutional neural networks using dynamic functional connectivity for EEG-based person identification in diverse human states. IEEE Trans. Inform. Forensic Secur. 14, 3259–3272. doi: 10.1109/TIFS.2019.2916403
Wang, M., Hu, J., and Abbass, H. A. (2019b). Stable EEG biometrics using convolutional neural networks and functional connectivity. Aust. J. Intell. Inf. Process. Syst. 15, 19–26.
Winkler, I., Brandl, S., Horn, F., Waldburger, E., Allefeld, C., and Tangermann, M. (2014). Robust artifactual independent component classification for BCI practitioners. J. Neural Eng. 11:035013. doi: 10.1088/1741-2560/11/3/035013
Yang, S., Deravi, F., and Hoque, S. (2018). Task sensitivity in EEG biometric recognition. Pattern. Anal. Applic. 21, 105–117. doi: 10.1007/s10044-016-0569-4
Zhang, X. L., Begleiter, H., Porjesz, B., Wang, W., and Litke, A. (1995). Event related potentials during object recognition tasks. Brain Res. Bull. 38, 531–538. doi: 10.1016/0361-9230(95)02023-5
Zhang, Y., Li, M., Shen, H., and Hu, D. (2021). On the specificity and permanence of electroencephalography functional connectivity. Brain Sci. 11:1266. doi: 10.3390/brainsci11101266
Keywords: virtual node, EEG, individual difference, graph convolution neural network, functional connectivity
Citation: Li M, Yang J, Tian W and Ju X (2023) Fusing the spatial structure of electroencephalogram channels can increase the individualization of the functional connectivity network. Front. Comput. Neurosci. 17:1263710. doi: 10.3389/fncom.2023.1263710
Edited by:
Wei Zhang, University of California, San Francisco, United StatesReviewed by:
Ninghao Liu, University of Georgia, United StatesSen He, University of Arizona, United States
Copyright © 2023 Li, Yang, Tian and Ju. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Ming Li, liming78@nudt.edu.cn
†These authors have contributed equally to this work and share first authorship