Skip to main content

ORIGINAL RESEARCH article

Front. Neurosci.
Sec. Brain Imaging Methods
Volume 18 - 2024 | doi: 10.3389/fnins.2024.1519970

STAFNet: An adaptive multi-feature learning network via spatiotemporal fusion for EEG-based emotion recognition

Provisionally accepted
Fo Hu Fo Hu 1Kailun He Kailun He 1Mengyuan Qian Mengyuan Qian 1Xiaofeng Liu Xiaofeng Liu 1Zukang Qiao Zukang Qiao 2Lekai Zhang Lekai Zhang 3*Junlong Xiong Junlong Xiong 2*
  • 1 College of Information Engineering, Zhejiang University of Technology, Hangzhou, Jiangsu Province, China
  • 2 First Affiliated Hospital, Zhejiang Chinese Medical University, Hangzhou, China
  • 3 Zhejiang University of Technology, Hangzhou, Zhejiang Province, China

The final, formatted version of the article will be published soon.

    Emotion recognition based on electroencephalography (EEG) is a pivotal component of braincomputer interface research. To attain precision in emotion recognition from EEG signals, it's crucial to adeptly extract and integrate both their spatial and temporal attributes. Nevertheless, prevailing research often concentrates on one aspect, neglecting the integration of spatial and temporal dynamics fully. To address this limitation, we propose a novel Spatiotemporal Adaptive Fusion Network (STAFNet) that enhances both the accuracy and robustness of EEGbased emotion recognition. The model incorporates an adaptive graph convolutional module, which leverages an adaptive adjacency matrix to thoroughly capture brain connectivity patterns through spatial dynamic evolution. In addition, we introduce a multi-structured transformer fusion module that fully exploits the latent correlations between spatial and temporal features, yielding a fused representation for emotion classification. Extensive experiments were conducted on two public datasets, SEED and SEED-IV, to evaluate the model's performance. The results demonstrate that STAFNet achieved accuracies of 97.89% and 93.64% on SEED and SEED-IV, respectively, outperforming current state-of-the-art methods. Our findings highlight the critical role of spatiotemporal feature extraction in emotion recognition and introduce a novel framework for feature fusion.

    Keywords: EEG, emotion recognition, deep learning, spatiotemporal fusion, adaptive adjacency matrix

    Received: 30 Oct 2024; Accepted: 18 Nov 2024.

    Copyright: © 2024 Hu, He, Qian, Liu, Qiao, Zhang and Xiong. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

    * Correspondence:
    Lekai Zhang, Zhejiang University of Technology, Hangzhou, 310014, Zhejiang Province, China
    Junlong Xiong, First Affiliated Hospital, Zhejiang Chinese Medical University, Hangzhou, China

    Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.