Group-member selection for RSVP-based collaborative brain-computer interfaces

Objective: The brain-computer interface (BCI) systems based on rapid serial visual presentation (RSVP) have been widely utilized for the detection of target and non-target images. Collaborative brain-computer interface (cBCI) effectively fuses electroencephalogram (EEG) data from multiple users to overcome the limitations of low single-user performance in single-trial event-related potential (ERP) detection in RSVP-based BCI systems. In a multi-user cBCI system, a superior group mode may lead to better collaborative performance and lower system cost. However, the key factors that enhance the collaboration capabilities of multiple users and how to further use these factors to optimize group mode remain unclear.

Approach: This study proposed a group-member selection strategy to optimize the group mode and improve the system performance for RSVP-based cBCI. In contrast to the conventional grouping of collaborators at random, the group-member selection strategy enabled pairing each user with a better collaborator and allowed tasks to be done with fewer collaborators. Initially, we introduced the maximum individual capability and maximum collaborative capability (MIMC) to select optimal pairs, improving the system classification performance. The sequential forward floating selection (SFFS) combined with MIMC then selected a sub-group, aiming to reduce the hardware and labor expenses in the cBCI system. Moreover, the hierarchical discriminant component analysis (HDCA) was used as a classifier for within-session conditions, and the Euclidean space data alignment (EA) was used to overcome the problem of inter-trial variability for cross-session analysis.

Main results: In this paper, we verified the effectiveness of the proposed group-member selection strategy on a public RSVP-based cBCI dataset. For the two-user matching task, the proposed MIMC had a significantly higher AUC and TPR and lower FPR than the common random grouping mode and the potential group-member selection method. Moreover, the SFFS with MIMC enabled a trade-off between maintaining performance and reducing the number of system users.

Significance: The results showed that our proposed MIMC effectively optimized the group mode, enhanced the classification performance in the two-user matching task, and could reduce the redundant information by selecting the sub-group in the RSVP-based multi-user cBCI systems.

1 Introduction

Brain-computer interfaces (BCIs) are human-machine interaction systems that forge a direct pathway between the user's brain and the external world, bypassing conventional peripheral pathways (Vidal, 1973; Martins et al., 2019; Moioli et al., 2021). Traditional BCIs are designed to provide communication and control solutions for people with severe neuromuscular disorders (McFarland and Wolpaw, 2011). Therefore, the typical BCI applications include brain-controlled spellers (Farwell and Donchin, 1988; Volosyak et al., 2009), brain-controlled wheelchair (Long et al., 2012), and brain-controlled cursor (Li et al., 2008), etc. Recently, some kinds of electroencephalogram (EEG)-based BCIs have been developed for able-bodied users, aiming to enhance human capabilities (Värbu et al., 2022). The Rapid Sequence Visual Presentation (RSVP) based BCI is one of them (Lees et al., 2018).

RSVP focuses on enhancing users' visual search capabilities by utilizing split-second perceptual judgments (Huang et al., 2011; Matran-Fernandez and Poli, 2017). Visual search is a perceptual process that involves scanning the environment to find an item of interest. RSVP-based BCI systems can be employed in designing spellers (Acqualagna and Blankertz, 2013) and for detecting targets, including both static images (Bigdely-Shamlo et al., 2008; Poolman et al., 2008) and videos (Weiden et al., 2012; Rosenthal et al., 2014). The RSVP-based BCIs for target image detection can be applied in counterintelligence and policing for detecting potential threats (Marathe et al., 2015), in medical diagnostics for screening mammograms (Hope et al., 2013), and in geoscientific research for analyzing complex images (Sivarajah et al., 2014). As an example, in counterintelligence and policing work, teams composed of multiple police officers screen large volumes of images daily to identify suspicious individuals or items.

As shown in Figure 1, in RSVP-based target image detection, sequences of image stimuli are rapidly presented at a consistent spatial position (Lees et al., 2018). The stream of images comprises frequent non-target samples and infrequent target samples. Compared to non-target samples, target samples are rare, thus a P300 event-related potential (ERP) is elicited when users observe a target sample (Polich and Donchin, 1988). By recording EEG signals and detecting the single-trial ERPs, the RSVP-based BCI systems can distinguish target and non-target images.

Figure 1

Figure 1. The overview of RSVP stimulation. The target sample is highlighted with a red frame, while non-target samples are marked with a green frame. In a target sample on the left, the human figure is further highlighted with a yellow frame. The images were from Zheng et al. (2020).

Due to the low signal-to-noise ratio (SNR) of EEG signals, the performance of single-trial ERP detection remains limited in RSVP-based BCI systems (Henry, 2006; McFarland and Wolpaw, 2011). Many feature extraction and classification algorithms have been developed to enhance the performance of RSVP-based BCIs (Lees et al., 2018; Lotte et al., 2018; Wu and Wu, 2022; Wang et al., 2023). Typical feature extraction algorithms include xDAWN (Rivet et al., 2009) and SIM (Wu and Gao, 2011). Major classification algorithms include hierarchical discriminant component analysis (HDCA; Sajda et al., 2010), discriminative canonical pattern matching (DCPM; Xiao et al., 2020) and discriminant analysis and classification for interval ERPs (DACIE; Li et al., 2021). With the development of deep learning, the network models such as EEGNet (Lawhern et al., 2018) and its enhanced variants (Zhang et al., 2022) have shown superior classification performance in RSVP-based BCIs.

Furthermore, with the widespread application of BCI technology, the socialization of BCI has emerged as a trend (Hu et al., 2024). In this context, collaborative brain-computer interfaces (cBCIs), which fuse EEG signals from multiple users, have become another approach to enhancing the SNR of EEG signals. In the cBCI paradigm, multiple subjects participate in identical tasks simultaneously (Wang and Jung, 2011; Zheng et al., 2020). The EEG data from these subjects are concurrently recorded and integrated to derive the final classification result. Numerous studies have demonstrated that, when compared to traditional single-brain BCI systems, cBCI systems exhibit superior performance, particularly in terms of speed and accuracy (Wang and Jung, 2011; Stoica et al., 2013; Zhang et al., 2021).

In recent years, the exploration and development of multi-user cBCI systems have garnered significant attention in the scientific community. Wang and Jung (2011) categorized cBCI paradigms into centralized cBCI and distributed cBCI as shown in Figure 2, and they proposed three distinct approaches for fusing EEG signals from multiple users. Cecotti and Rivet (2014) further refined the typology of BCI systems, building upon the hybrid BCI (Pfurtscheller et al., 2010) and the cBCI. They suggested a more nuanced categorization of BCIs, based on the diversity of BCI-based tasks and the number of participating subjects. To improve group decisions in cBCIs, Valeriani et al. (2015, 2017a,b) introduced the confidence-weighted voting method specifically designed for cBCI systems. They assessed the confidence level of each group member based on their response times, using this as a measure to evaluate individual capabilities. Subsequently, these confidence levels were used to assign weights to the decisions of each group member, thereby enhancing the overall performance of group decision-making. Furthermore, Salvatore et al. (2022) introduced optimization methods for EEG confidence decoders that take into account both individual capabilities and the overall composition of the group. They also used a hyperparameter to fine-tune the balance between the confidence weights of group members, aiming to strike an optimal balance between accuracy and fairness within the group. However, there is still a gap in research concerning the development of group formation strategies to enhance the performance of cBCI systems.

Figure 2

Figure 2. Different types of cBCI. (A) The centralized cBCI. (B) The distributed cBCI.

Two kinds of group mode optimization tasks are shown in Figure 3. The two-user matching task aims to pair a specific user with the most suitable collaborator to enhance collaborative performance. In conventional two-user cBCI systems, the prevailing approach of randomly matching collaborators is less effective (Matran-Fernandez and Poli, 2014; Zhao et al., 2024). Matran-Fernandez and Poli (2014) proposed a two-user matching method for forming cBCI groups by assessing the similarity of individual performances. They utilized a trained SVM classifier to match users in a two-user cBCI system, focusing on pairing those with minimal dissimilarity in their AUC scores. However, the performance of this method was suboptimal and heavily dependent on the manually set threshold. Upon analyzing various cBCI group modes, Zhao et al. (2024) suggested that the classification AUC might be a criterion for identifying the ideal matched subject in an RSVP-based cBCI system. Yet, their findings indicated that the highest AUC did not always align with the ideal match, leading them to hypothesize that feature distribution similarity might play a role. However, despite this insight, Zhao et al. did not incorporate feature distribution similarity into their selection method, suggesting there may be further opportunities to refine the two-user matching strategy. Moreover, the sub-group selection task aims to select a subset from all collaborators to complete the tasks that are initially assigned to the all-member group. Some studies (Wang and Jung, 2011; Zheng et al., 2020; Zhao et al., 2024) have shown that as the number of subjects increases, there is a significant enhancement in classification accuracy and a substantial reduction in standard deviation. However, with the involvement of more subjects in the system, there is a corresponding increase in hardware and labor costs, as well as computational complexity. Furthermore, the experimental results in Du et al. (2023) showed that the performance of two-user cBCI might outperform that of three-user or four-user cBCI, indicating there might be redundant information between the EEG of the collaborators. Therefore, how to reduce the number of members while still preserving the original critical information is one of the significant challenges in cBCI research.

Figure 3

Figure 3. Two kinds of group-member selection tasks. The green boxes with the circle mean a specific user who needs a collaborator. The blue boxes mean users which can be selected and the green boxes with the cross mean the selected collaborator. (A) Two-user matching task. (B) Sub-group selection task.

To address the above drawbacks, this paper proposed a group-member selection strategy for the two-user matching task, which was then further applied to tackle the issue of sub-group selection in RSVP-based cBCI systems. Firstly, in the two-user matching task, we proposed a novel performance score for candidates, taking into account both the individual performance of the candidates and the correlation between a predefined user and the candidates. Drawing inspiration from xDAWN (Rivet et al., 2009, 2011; Xiao et al., 2021), we estimated the signal to signal-plus-noise ratio (SSNR) of each candidate's EEG signal to quantify their individual capabilities. Subsequently, inspired by Liu et al. (2020), we used the Pearson correlation coefficient to estimate the similarity of the ERP waveform from the two users as their collaborative capability. Then the parameter μ was utilized to assign weights to individual capability and collaborative capability, and the user with maximum individual capability and maximum collaborative capability (MIMC) was considered the most suitable collaborator. Secondly, we combined the MIMC strategy with sequential forward floating selection (SFFS) (Pudil et al., 1994) to select the optimal sub-group to reduce the hardware costs, the labor costs, and the computational complexity in multi-user cBCI systems. Thirdly, HDCA (Sajda et al., 2010) was applied for single-trial ERP signal classification. The Euclidean space data alignment (EA) (He and Wu, 2020) was further employed to address the issue of inter-trial variability in cross-session conditions. To the best of our knowledge, it is the first attempt to optimize the group mode by using both the individual capabilities and the collaborative capabilities to improve the system performance in the multi-user RSVP-based cBCI system.

The remainder of this paper is organized as follows. Section 2 introduces the experimental procedure, the dataset, and the proposed methods. Then, the Section 3 presents the classification performance of the proposed method on both the two-user matching task and the sub-group selection task. Finally, the advantages and the limitations of the proposed method and the future research directions are drawn in the Section 4.

2 Material and methods

2.1 Data description

We used a cross-session RSVP-based cBCI dataset (Zheng et al., 2020) to verify the effectiveness of the proposed method. In this dataset, all 14 subjects were divided into seven fixed groups, each comprising two subjects. These groups respectively participated in two separate sessions of experiments on different days, and each session consisted of three blocks. Each block comprised 14 stimulus sequences, and within each stimulus sequence, 100 street scene images were presented with a presentation rate of 10 Hz in the center of the screen. As shown in Figure 1, these 100 images consisted of four target images which contained humans, and the target images were interspersed within each stimulus sequence, with a minimum time gap of 500 ms between consecutive target images. Thus, there are 1,400 image presentations (56 target and 1,344 non-target image presentations) in one block. Both subjects in a group pressed keys to start a stimulus sequence, and their 62-channel EEG signals were simultaneously recorded. The subjects pressed a button as soon as they detected a target. Regardless of whether the subjects successfully responded, all 56 target trials EEG signals were used for subsequent verification analysis. The experiment was conducted at a sample rate of 1, 000 Hz and a notch filter at 50 Hz was used to remove the power-line noise.

2.2 Data preprocessing

The data pre-processing stage includes down-sampling, data segmentation, band-bass filtering, and re-referencing. The EEG data from each subject in the group was down-sampled to 250 Hz. For each block, the down-sampled data was segmented into trials based on event triggers. Each trial included 1, 200 ms of EEG data, starting from 200 ms before the event trigger and extending to 1, 000 ms after the event trigger. Then, the EEG data were band-pass filtered at 2–30 Hz, and the average of all electrodes was used for re-reference. It should be noted that, for each trial EEG signal, we used the 0–500 ms data after the event trigger for group-member selection and classification.

2.3 MIMC for two-user matching

In this subsection, we introduced the proposed two-user matching strategy, MIMC. Suppose that S_all = {s₁, s₂, ⋯ , s_NM} was the set of all members in a multi-people collaborative group, where N_M was the number of all members. The two-user matching task aimed to pair a specific user, such as s_α∈S_all, with the most suitable collaborator. The set of the candidate collaborator was denoted as S_candi = S_all−{s_α}, and MIMC was a performance score for the candidate collaborator in S_candi.

For a candidate collaborator s_β, $X_{k,\beta }\in {ℝ}^{N_C\times N_S}$ was the kth trial preprocessed EEG signal, where N_C was the number of channels and N_S was the number of sampling points. The template signal induced by the non-target and target image was respectively denoted by $P_{\beta }^{(0)}\in {ℝ}^{N_C\times N_S}$ and $P_{\beta }^{(1)}\in {ℝ}^{N_C\times N_S}$:

$\begin{array}{ll}{P}_{\beta }^{(0)}=\frac{1}{N_T^{(0)}}{\displaystyle \sum _{i=1}^{N_T^{(0)}}}{X}_{i,\beta }^{(0)},P_{\beta }^{(1)}=\frac{1}{N_T^{(1)}}{\displaystyle \sum _{i=1}^{N_T^{(1)}}}{X}_{i,\beta }^{(1)},& (1)\end{array}$

where $N_T^{(j)}$ and $X_i^{(j)}$ respectively represented the number of trials of pattern j and the ith trial data of pattern j ( with j = 0, 1).

2.3.1 Individual capability estimation

xDAWN (Rivet et al., 2009, 2011; Xiao et al., 2021) was proposed to maximize the SSNR of P300 evoked EEG signals by estimating spatial filters. For a candidate collaborator s_β, the estimated spatial filter was denoted as

$\begin{array}{ll}{U}_{\beta }=\underset{U_{\beta }}{\arg \max }\rho (U_{\beta }),& (2)\end{array}$

and the estimated SSNR ρ(U_β) was given by

$\begin{array}{ll}\rho (U_{\beta })=\frac{U_{\beta }^T{\widehat{\Sigma }}_{\beta }^{(1)}{U}_{\beta }}{U_{\beta }^T{\widehat{\Sigma }}_{\beta }{U}_{\beta }},& (3)\end{array}$

where

$\begin{array}{ll}{\widehat{\Sigma }}_{\beta }^{(1)}={(P_{\beta }^{(1)})}^T{P}_{\beta }^{(1)}& (4)\end{array}$

was the covariance matrix of $P_{\beta }^{(1)}$, and

$\begin{array}{ll}{\widehat{\Sigma }}_{\beta }=X_{r,\beta }^T{X}_{r,\beta }& (5)\end{array}$

was the covariance matrix of the reshaped preprocessed EEG signals $X_{r,\beta }\in {ℝ}^{N_C\times (N_T\times N_S)}$.

The spatial filter U_β could be estimated by the generalized eigenvalue decomposition of pair (${\widehat{\Sigma }}_{\beta }^{(1)}$, ${\widehat{\Sigma }}_{\beta }$) such that:

$\begin{array}{ll}{\widehat{\Sigma }}_{\beta }^{(1)}{U}_{1,\beta }={\lambda }_{1,\beta }{\widehat{\Sigma }}_{\beta }{U}_{1,\beta },& (6)\end{array}$

where λ_{1, β} was the largest generalized eigenvalue and U_{1, β} was the associated eigenvector. The estimated spatial filter was U_β = U_{1, β}, and the estimated SSNR ρ_β = λ_{1, β} was the estimated individual capability score for candidate collaborator s_β.

We estimated the ρ of each candidate in S_candi and the individual capability matrix was

$\begin{array}{ll}{M}_{in,\alpha }=\left[{\rho }_1,{\rho }_2,\cdots ,{\rho }_{N_{candi}}\right],& (7)\end{array}$

where N_candi = N_M−1 was the number of candidates for the predefined user s_α.

2.3.2 Collaborative capability estimation

Liu et al. (2020) proposed selecting a subset of source domain subjects to form a new source domain based on the correlation between source and target domain subjects to enhance cross-subject classification performance in RSVP-based BCI systems. Inspired by their work, we utilized the ERP waveform similarity between the predefined user and the candidate as an indicator of their collaborative capability.

Suppose that $X_{k,\alpha ,flat}^{(1)}\in {ℝ}^{1\times (N_C\times N_S)}$ and $X_{k,\beta ,flat}^{(1)}\in {ℝ}^{1\times (N_C\times N_S)}$ respectively represented the kth trial flattened processed EEG data of the predefined user s_α and candidate collaborator s_β induced by the target image. The collaborative capability between the predefined user s_α and candidate collaborator s_β was denoted as

$\begin{array}{ll}{\sigma }_{\alpha \beta }=\frac{1}{N_T^{(1)}}{\displaystyle \sum _{i=1}^{N_T^{(1)}}}corr(X_{i,\alpha ,flat}^{(1)},X_{i,\beta ,flat}^{(1)}),& (8)\end{array}$

where the corr(·) was the Pearson correlation. We estimated the collaborative capability between each candidate in S_candi and user s_α, then the collaborative capability matrix was denoted as

$\begin{array}{ll}{M}_{co,\alpha }=\left[{\sigma }_{\alpha 1},{\sigma }_{\alpha 2},\cdots ,{\sigma }_{\alpha N_{candi}}\right],& (9)\end{array}$

2.3.3 Two-user matching

Considering the different significance of individual capability and collaborative capability for optimizing group mode, we set the parameter μ to balance the weights assigned to each capability. Z-score normalization was conducted to unify the two capabilities, and $M_{in,\alpha }^{*}$ and $M_{co,\alpha }^{*}$ were the unified scores, respectively. The performance score matrix of the candidate for the special user s_α was

$\begin{array}{ll}{M}_{\alpha }=\mu M_{in,\alpha }^{*}+(1-\mu )M_{co,\alpha }^{*}.& (10)\end{array}$

To optimize μ, we employed five-fold cross-validation for each session. Taking session 1 as an example, block 1 of session 1 was used to optimize μ and to execute the two-user matching strategy. There are 1,400 trials of EEG data in block 1, and every 100 trials consists of four target and 96 non-target trials. Given the imbalance of RSVP-based EEG data, we partitioned block 1 into five folds by trial. The first four folds each contained 300 trials, while the last fold contained 200 trials. For each fold, we defined the range of μ as [0, 1], incrementing by a step size of 0.01. Then, for each μ, we used it to construct M_si for s_i∈S_all, i = 1…N_M, and selected the candidate collaborator corresponding to the maximum value in M_si as the matched collaborator for s_i. We applied the average ERP strategy in Section 2.5 and HDCA in Section 2.6 to validate the collaborative capability of these groups selected using this μ. The average AUC of these groups was considered the score for this μ in this fold. We recorded the score for each μ across all folds. The μ with the highest average score in the five folds was considered the optimal result.

2.4 SFFS with MIMC for sub-group selection

We implemented the SFFS with MIMC to select the sub-group and reduce the redundant information in the training stage. The pseudo-code was shown in Algorithm 1.

Algorithm 1

Algorithm 1. SFFS with MIMC for sub-group selection.

Suppose that S_all = {s₁, s₂, ⋯ , s_NM} was the set of all members in a multi-people collaborative group, where N_M was the number of all members. We determined the parameter μ in the two-user matching task, and then the μ was used as the input. The output S_sub = {s₁, s₂, ⋯ , s_Nsub} was the set of the sub-group selected, where N_sub was the number of members in the selected sub-group.

In the initialization phase, we estimated the individual capabilities of all members according to Equation 6. The user with the best individual capability was used as the initial sub-group S₁ = {s_best}. Algorithm 1 selected group members through the following steps until the termination condition was met. In the ith iteration, we first treated the inclusion process as a two-user matching problem. The fusion(·) referred to the average ERP strategy in Section 2.5. The S_{sub, i} was considered as a whole, with the members' EEG signals fused using the average ERP strategy. We included a member by maximizing the objective function J(S_{sub, i}∪{s}). Then, we also treated the conditional exclusion step as a special two-user matching task. When a member s was excluded from S_{sub, i+1}, we regarded S_{sub, i+1}−{s} as a whole and then s and S_{sub, i+1}−{s} were viewed as a pair of collaborators. The fused data of S_{sub, i+1}−{s} could be treated as a specific member and s could be seen as the candidate. The objective function J(S_{sub, i+1}−{s}) was minimized to remove a member. We used the average ERP strategy in Section 2.5 and HDCA in Section 2.6 to calculate the AUC of S_{sub, i} and S_{sub, i+1} in the termination step.

2.5 Fusion strategies

In this subsection, we introduced several common fusion strategies for the cBCI paradigm. For the centralized cBCI, the EEG signals or features from multiple users were fused before being input into the classification model. ERP averaging and data combination were two popular centralized fusion strategies. The average ERP strategy averaged the pre-processed EEG signals or extracted features from the collaborators. The combination could be divided into parallel combination and serial combination. Parallel combination concatenated information across the spatial domain, while serial combination concatenated information across the temporal domain. Moreover, the distributed cBCI was based on multiple sub-classifiers and a voting system. In a distributed cBCI system, each user's EEG signals were processed by their respective sub-classifier, which made individual predictions based on those signals. These predictions were then fused using a voting system to form the final decision. Weighted voting was a classic distributed method, in which individual decisions were weighted by the training performance or other confidence weight.

2.6 Classification

HDCA (Sajda et al., 2010) was commonly used to classify the ERP component in the RSVP-based BCI system. Suppose that $X^{{}^{\prime }}\in {ℝ}^{N_T\times N_C^{{}^{\prime }}\times N_S^{{}^{\prime }}}$ was the fused EEG signal and N_T, $N_C^{{}^{\prime }}$, $N_S^{{}^{\prime }}$ were the number of trials, channels and sample points, respectively. The core idea of HDCA was to train spatial projection matrix $u\in {ℝ}^{N_C^{{}^{\prime }}\times W}$ and temporal projection vectors v∈ℝ^W×1 for single-trial classification, where W was the number of non-overlapping time windows. Suppose that $X_w^{{}^{\prime }}\in {ℝ}^{N_T\times N_C^{{}^{\prime }}\times N_w}$ was the fused data in the wth time window. In this study, we set $N_w=\frac{1}{5}{N}_S^{{}^{\prime }}$. First, at each time window, Fisher Linear Discriminant (FLD) was used to calculate spatial projection vectors $u_w\in {ℝ}^{N_C^{{}^{\prime }}\times 1}$,

$\begin{array}{ll}{u}_w=FLD(X_w^{{}^{\prime }}),w=1,\cdots ,W,& (11)\end{array}$

In the wth time window, the fused data induced by the non-target and target image was respectively denoted by $X_w^{{(0)}^{\prime }}\in {ℝ}^{N_T^{(0)}\times N_C^{{}^{\prime }}\times N_w}$ and $X_w^{{(1)}^{\prime }}\in {ℝ}^{N_T^{(1)}\times N_C^{{}^{\prime }}\times N_w}$. Suppose that $X_{w,r}^{{}^{\prime }}\in {ℝ}^{(N_T\times N_w)\times N_C^{{}^{\prime }}}$ was the reshaped $X_w^{{}^{\prime }}$, then $X_{w,r}^{{(0)}^{\prime }}\in {ℝ}^{(N_T^{(0)}\times N_w)\times N_C^{{}^{\prime }}}$ and $X_{w,r}^{{(1)}^{\prime }}\in {ℝ}^{(N_T^{(1)}\times N_w)\times N_C^{{}^{\prime }}}$ were the reshaped $X_w^{{(0)}^{\prime }}$ and $X_w^{{(1)}^{\prime }}$ respectively. The template signal of the reshaped fused data in the wth time window induced by the non-target and target image was respectively denoted as $P_{w,r}^{(0)}\in {ℝ}^{1\times N_C^{{}^{\prime }}}$ and $P_{w,r}^{(1)}\in {ℝ}^{1\times N_C^{{}^{\prime }}}$:

$\begin{array}{ll}{P}_{w,r}^{(0)}=\frac{1}{N_T^{(0)}}{\displaystyle \sum _{i=1}^{N_T^{(0)}}}{X}_{i,w,r}^{(0)},P_{w,r}^{(1)}=\frac{1}{N_T^{(1)}}{\displaystyle \sum _{i=1}^{N_T^{(1)}}}{X}_{i,w,r}^{(1)}.& (12)\end{array}$

The total within-class scatter matrix of the fused data in the wth time window was

$\begin{array}{l}{S}_{W,w,r}={\displaystyle {\sum }_{i=1}^{N_T^{(0)}}(X_{i,w,r}^{(0)}-P_{w,r}^{(0)}){(X_{i,w,r}^{(0)}-P_{w,r}^{(0)})}^T}\\ +{\displaystyle {\sum }_{i=1}^{N_T^{(1)}}(X_{i,w,r}^{(1)}-P_{w,r}^{(1)}){(X_{i,w,r}^{(1)}-P_{w,r}^{(1)})}^T}.& (13)\end{array}$

The spatial projection vectors were calculated by

$\begin{array}{ll}{u}_w={({(S_{W,w,r})}^{-1}{(P_{w,r}^{(0)}-P_{w,r}^{(1)})}^T)}^T,& (14)\end{array}$

$\begin{array}{ll}u=\left[u_1,u_2,\cdots ,u_W\right].& (15)\end{array}$

For wth time window, $Y_w\in {ℝ}^{N_T\times 1}$ represents the data after spatial projection:

$\begin{array}{ll}{Y}_w=u_w^T{(X_{w,r}^{{}^{\prime }})}^T,& (16)\end{array}$

$\begin{array}{ll}Y=\left[Y_1,Y_2,\cdots ,Y_W\right],& (17)\end{array}$

where $Y\in {ℝ}^{N_T\times W}$.

Then Fisher Linear Discriminant (FLD) was used to calculate temporal projection vector v∈ℝ^W×1 for the signals after spatial projection,

$\begin{array}{ll}v=FLD(Y).& (18)\end{array}$

Suppose that P′(0)∈ℝ^{1 × W} and P′(1)∈ℝ^{1 × W} were the non-target and target templates of the signals after spatial projection.

$\begin{array}{ll}{P}^{{}^{\prime }(0)}=\frac{1}{N_T^{(0)}}{\displaystyle \sum _{i=1}^{N_T^{(0)}}}{Y}^{(0)},P^{{}^{\prime }(1)}=\frac{1}{N_T^{(1)}}{\displaystyle \sum _{i=1}^{N_T^{(1)}}}{Y}^{(1)}.& (19)\end{array}$

The total within-class scatter matrix of the signals after spatial projection was

$\begin{array}{l}{S}_W={\displaystyle {\sum }_{i=1}^{N_T^{(0)}}(Y^{(0)}-P^{\prime }(0)){(Y^{(0)}-P^{\prime }(0))}^T}\\ +{\displaystyle {\sum }_{i=1}^{N_T^{(1)}}(Y^{(1)}-P^{\prime }(1)){(Y^{(1)}-P^{\prime }(1))}^T}.& (20)\end{array}$

The temporal projection vector was calculated by

$\begin{array}{ll}v={({(S_W)}^{-1}{(P^{{}^{\prime }(0)}-P^{{}^{\prime }(1)})}^T)}^T,& (21)\end{array}$

Here, Z_th represented the threshold value.

$\begin{array}{ll}{Z}_{th}=\frac{1}{2}(P^{{}^{\prime }(0)}+P^{{}^{\prime }(1)})v& (22)\end{array}$

For a single-trial fused signal $X_k^{{}^{\prime }}\in {ℝ}^{N_C^{{}^{\prime }}\times N_S^{{}^{\prime }}}$, the data of wth window was denoted as $X_{k,w}^{{}^{\prime }}\in {ℝ}^{N_C^{{}^{\prime }}\times N_w}$. When the value of Z_k was greater than Z_th, the classification result would equal 1.

$\begin{array}{ll}{Z}_k=\left[u_1^T{X}_{k,1}^{{}^{\prime }},u_2^T{X}_{k,2}^{{}^{\prime }},\cdots ,u_w^T{X}_{k,w}^{{}^{\prime }}\right]v& (23)\end{array}$

We used HDCA for the within-session classification and the EA-HDCA for the cross-session classification. EA (He and Wu, 2020) was proposed for transfer learning in BCI system. The main idea of EA was to make the data distribution from different domains more similar to improve the transfer performance of the classifier on a new domain. Suppose that the reference matrix R was the mean covariance matrix of all n trials fused EEG signals of a group:

$\begin{array}{ll}R=\frac{1}{n}{\displaystyle \sum _{i=1}^n}{X}_i^{{}^{\prime }}{X}_i^{{}^{\prime }T}& (24)\end{array}$

To make the data distributions from different sessions more similar, the mean covariance matrices of all sessions should be equal to the identity matrix I after alignment. Suppose ${\tilde{X}}_k^{{}^{\prime }}$ was the kth trial after alignment:

$\begin{array}{ll}{\tilde{X}}_k^{{}^{\prime }}=R^{-\frac{1}{2}}{X}_k^{{}^{\prime }}& (25)\end{array}$

$\begin{array}{l}\frac{1}{n}{\displaystyle {\sum }_{i=1}^n{{\tilde{X}}^{\prime }}_i}{{\tilde{X}}^{\prime }}^T{}_i=\frac{1}{n}{\displaystyle {\sum }_{i=1}^n{R}^{-\frac{1}{2}}}{{\tilde{X}}^{\prime }}_i{{\tilde{X}}^{\prime }}^T{}_i{R}^{-\frac{1}{2}}{\tilde{X}}^{\prime }\\ =R^{-\frac{1}{2}}\left(\frac{1}{n}{\displaystyle {\sum }_{i=1}^n{{\tilde{X}}^{\prime }}_i}{{\tilde{X}}^{\prime }}^T{}_i\right)R^{-\frac{1}{2}}=I& (26)\end{array}$

For EA-HDCA, the aligned training data were utilized to train the HDCA model, and the aligned test data were used for analysis.

2.7 Framework overview

For an example of the sub-group selection task in a four-user centralized cBCI, our proposed framework is shown in Figure 4. For the training stage, the raw EEG signals from all subjects were preprocessed first. Then, the SFFS with MIMC was used to select a sub-group. The EEG data of the selected members were fused in the fusion stage and HDCA was trained to project the single-trial EEG signals to the decision. For the testing stage, the raw EEG signals from the selected members in the training stage were fused and the trained classifier was used to detect the ERP component.

Figure 4

Figure 4. The framework of the cBCI system with group-member selection.

3 Results

This section presented the performance of the proposed method in both the two-user matching task and the sub-group selection task. The area under the receiver operating characteristic curve (AUC), true positive rate (TPR), and false positive rate (FPR) were used to analyze the effectiveness of the proposed algorithm. The significant difference was analyzed by one-way ANOVA and paired t-test. The statistical significance was defined as p-values < 0.05, and the Post Hoc tests were the Least Significant Difference (LSD) corrected in the one-way ANOVA. LSD is a statistical method used in multiple comparisons. It helps to determine which specific group averages are significantly different from one another.

3.1 AUC comparison of different fusion strategies

In this subsection, we compared the AUC of four different fusion strategies for the seven fixed groups in the dataset to find the best fusion strategy. The AUC of average ERP (AE), parallel combination (PC), serial combination (SC), and AUC-weighted vote (WV) in session 1 (S1) and session 2 (S2) were shown in Figure 5. Taking S1 as an example, block 1 of S1 was used to train the classifier, and block 2 and block 3 of session 1 were used to test (Zheng et al., 2020). For all two sessions, the experiments were denoted as S1–S1 and S2–S2 in within-session conditions, respectively. The one-way ANOVA results showed that there was no significantly better one among the four fusion strategies. Therefore, the average ERP with the highest averaged AUC value was taken as the best fusion strategy for HDCA. For the later analysis, HDCA with average ERP was taken as a model to evaluate the classification performance.

Figure 5

Figure 5. Comparison of the AUC with four different fusion strategies.

3.2 Two-user matching performance

In this subsection, to verify the effectiveness of the proposed method in the two-user matching task, we first compared the MIMC with other conventional group modes in both within-session and cross-session conditions. Then, to analyze the key factors affecting the collaboration capabilities of two users, we compared the MIMC with other potential group-member selection strategies.

3.2.1 Performance comparison between MIMC and conventional group modes

Table 1 showed the AUC, FPR, and TPR of single-user mode, several conventional group modes in cBCI and proposed MIMC in S1–S1 and S2–S2. The fixed grouping was the seven fixed group in the dataset. The random grouping was the group that was randomly selected from all users. The matched group was the group that had the best test performance for each subject (Zhao et al., 2024). One-way ANOVA was conducted for the group modes, which showed significant differences in AUC, TPR, and FPR among these group modes in both S1–S1 and S2–S2 (p-values < 0.001). As shown in Table 1, in the within-session conditions, compared with other group modes except for the matched grouping, the MIMC had higher AUC and TPR and lower FPR. Moreover, the one-way ANOVA result showed that there was no significant difference between the MIMC and the matched grouping. It was indicated that the MIMC significantly improved the collaborative performance in two-user matching tasks.

Table 1

Table 1. Within-session performance comparison between MIMC and conventional group mode.

The group-member selection process in the cBCI system was labor-intensive and time-consuming for the whole group due to the need for substantial computational resources to collect and analyze EEG signals from all potential collaborators. Cross-session analysis was necessary to validate the robustness of the selected groups, ensuring that a group selected on one day could be effectively used on other days. This avoids the need for repeated labor-intensive and time-consuming selection processes before each session. The cross-session condition was denoted as S1–S2 and S2–S1, in which block 1 of one session was used to train, and block 2 and block 3 of another session were used to test. To ensure that the experimental results are not influenced by the classifier's own cross-session capabilities, we conducted cross-session experiments and used EA-HDCA as the classifier. This was crucial because a stable and reliable classifier allows us to accurately assess the robustness of the selected collaborative groups across different sessions, eliminating variability introduced by the classifier itself and ensuring that any observed performance differences are due to the collaborative groups rather than the classifier's instability. Figure 6 compared the cross-session classification performance of the HDCA and EA-HDCA under the single subject mode. In Figure 6, the red dashed line represented the chance level, and the TPR of HDCA was lower than the chance level in S2–S1. It was suggested that the HDCA was not effective in overcoming the ERP variability among different sessions. Moreover, the paired t-test results showed that the EA-HDCA had higher AUC values and lower FPR values than HDCA. Therefore, for the later analysis of cross-session conditions, EA-HDCA with ERP averaging was used to evaluate the classification performance.

Figure 6

Figure 6. Comparison of the performance with HDCA and EA-HDCA in cross-session situations. The “^*” indicated the significant difference between HDCA and EA-HDCA. “^**” and “^***” meant p-values < 0.01 and p-values < 0.001, respectively. Without “^*” meant p-values >0.05.

As shown in Table 2, similar to the within-session experiment results, the one-way ANOVA results showed that there were significant differences in AUC, TPR, and FPR among these group modes in both S1–S2 and S2–S1 (p-values < 0.001) and the proposed MIMC had better performance compared with other group modes. There was no significant difference between the MIMC and the matched grouping in cross-session conditions. It was indicated that the member combinations selected using MIMC on one day could be used on the other days. It could contribute to time reduction and process simplification of the cBCI system calibration. This might be attributed to the fact that, although different subjects exhibit varied neural responses to the same stimulus, these individual differences may demonstrate temporal consistency. For example, the group that had superior performance in one session was likely to exhibit superior performance in another session.

Table 2

Table 2. Cross-session performance comparison between MIMC and conventional group mode.

3.2.2 Performance comparison between MIMC and other group-member selection methods

We also compared the proposed MIMC with other group-member selection methods. The Individual Performance Dissimilarity (IPD) (Matran-Fernandez and Poli, 2014) method used the AUC dissimilarity between two users as the performance score to pair users. Drawing inspiration from Zhao et al. (2024), we implemented the Best Individual Performance (BIP) approach, wherein the user exhibiting the highest AUC value was identified as the optimal collaborator. We also considered that the group with the best collaborative AUC on the training stage would constitute an effective group mode, and implemented the Best Collaborative Performance (BCP) method. Note that, we used the HDCA with average ERP to implement these AUC-based methods.

Table 3 showed the comparison results. The one-way ANOVA results illustrated that there were significant differences among the four group-member selection methods in S1–S1 (p-values < 0.001) and S2–S2 (p-values < 0.05). In S1–S1, according to the Post Hoc test with LSD adjustment results, the proposed MIMC provided higher AUC and TPR than IPD and BCP and lower FPR than IPD. MIMC and BIP selected the same collaborators for each user. The IPD provided the worst classification performance among the four methods (p-values < 0.05). In S2–S2, the MIMC provided higher AUC and TPR than the other three methods and provided lower FPR than IPD and BCP. There were no significant differences between IPD, BIP, and BCP in S2–S2. Therefore, the proposed MIMC selected better collaborators for each subject and improved the classification performances.

Table 3

Table 3. Performance comparison between MIMC and other group-member selection methods.

Additionally, we compared IPD, BIP, and BCP with other group modes. The one-way ANOVA with LSD adjustment results showed that there was no significant difference between IPD and random grouping. It might be because although the two subjects had similar AUC, it did not necessarily mean they had more similar neural responses. BIP and BCP also did not demonstrate a significant difference when compared to random grouping in session 2. One possible reason could be the AUC-based methods suffered from over-fitting. For BIP, as shown in Table 1, the decrease in individual performance variability among subjects in session 2 could be another reason.

3.2.3 Performance comparison between different values of μ

In MIMC, the parameter μ played a crucial role in balancing the contribution of individual capability and collaborative capability. As shown in Table 4, the one-way ANOVA results indicated there were significant differences in AUC among different values of μ in S1–S1 (p-values < 0.001), S1–S2 (p-values < 0.05), and S2–S1 (p-values < 0.05). The Post Hoc tests with LSD revealed that, in S1–S1 and S1–S2, the AUC for the method focusing solely on collaborative capabilities (μ = 0) was surpassed by both other approaches. Similarly, in S2–S1, the approach centered exclusively on individual capabilities (μ = 1) demonstrated lower AUC values compared to its counterparts. The sub-optimal performance of (μ = 0) and (μ = 1) could likely be attributed to the participants' varying familiarity with the RSVP paradigm across sessions. In session 1, subjects' unfamiliarity with the RSVP paradigm led to considerable differences in their individual capabilities. Under these conditions, methods emphasizing individual capabilities could potentially achieve higher AUC values. However, in session 2, as participants became more familiar with the RSVP paradigm, the emphasis shifted toward the importance of collaborative capability. The optimal μ value adeptly combined both of them, enhancing system performance. Our experimental results determined optimal μ values to be 0.86 for session 1 and 0.64 for session 2, suggesting that individual capability played a more important role than collaborative capability.

Table 4

Table 4. AUC (%) of different values of μ.

3.3 Sub-group selection performance

In this subsection, we extended the proposed MIMC from the two-user cBCI system to the multi-user cBCI system. The all-member mode was the group with all 14 subjects in the dataset. The sub-group mode was the sub-group selected by the proposed SFFS with MIMC from the 14 subjects. The AUC, TPR, FPR, and the group size of the all-member mode and sub-group mode in both within-session conditions and cross-session conditions were shown in Table 5. In general, the all-member mode provided slightly higher AUC and TPR with lower FPR compared to the sub-group mode. However, the group size of the two modes indicated that the group member number of the sub-group drastically reduced. Specifically, the selected sub-group consisted of merely one-seventh of the total subjects in session 1 and two-seventh in session 2, respectively. Therefore, the proposed method could sacrifice a slight amount of system performance to substantially reduce the number of members in a multi-user cBCI system. Notably, in the S2–S1, the sub-group has a higher AUC and TPR than the all-member mode, indicating that if a better sub-group was selected, the performance might be improved in the multi-user cBCI system.

Table 5

Table 5. Classification performance with all-member and sub-group selected by SFFS with MIMC.

As shown in Figure 7, we discussed the effects of the number of group members on AUC under the random grouping mode and the MIMC mode in the multi-user cBCI system. For random grouping mode in multi-user cBCI, with a group size of n, we formed a group by randomly choosing n participants from all 14 subjects. Considering the computational complexity, for a group size of n, the maximum number of random combinations generated from choosing n out of 14 was capped at 100 to manage the variety of possible combinations. As the number of collaborators within the group increased, the AUC demonstrated an upward trend under the random grouping mode. For the group-member selection mode, we selected a sub-group from all 14 subjects using SFFS with MIMC. To analyze the effectiveness of the proposed method, Figure 7 showed the AUC for the selected sub-group terminating at the predefined number of subjects first. Under this termination policy, SFFS with MIMC significantly outperformed random grouping with a smaller predefined number, such as 2–4 members. This trend highlighted the MIMC strategy's superior initial selection of team members, where the benefits of the selection strategy were maximized with a limited number of collaborators. This aligned with our original intention of striking a balance between the size of the group and the collaborative performance. The blue stars represented the AUC and group size of the selected sub-group using SFFS with MIMC, indicating that SFFS with MIMC effectively balanced the number of selected members and the collaborative AUC.

Figure 7

Figure 7. Performance of the random grouping and SFFS with MIMC in the sub-group selection task. The blue stars represented the group size and AUC of the selected sub-group using SFFS with MIMC.

4 Discussion

In this section, we discussed and concluded the effectiveness and the limitations of the proposed method, and introduced the possible future research directions.

4.1 Effectiveness of proposed method

In this study, we proposed the MIMC strategy to pair collaborative groups for the two-user matching task. To verify the effectiveness of the MIMC strategy in the two-user matching task, we first compared MIMC with other conventional group modes under both within-session and cross-session conditions. In both conditions, MIMC achieved higher AUC and TPR, lower FPR compared to other group modes, and showed no significant difference from the matched grouping. These results demonstrated that the MIMC strategy significantly improves collaborative performance. Secondly, we compared MIMC with other existing and potential group-member selection strategies, including IPD (Matran-Fernandez and Poli, 2014), BIP (Zhao et al., 2024), and BCP. The IPD strategy (Matran-Fernandez and Poli, 2014) selected the candidate whose AUC is most similar to the specific user. The BIP strategy (Zhao et al., 2024) selected the candidate with the highest AUC. In contrast, we estimated the SSNR of each candidate's EEG signal to quantify their individual capabilities and used the Pearson correlation coefficient to estimate the similarity of the ERP waveform from the two users as their collaborative capability. The proposed MIMC strategy selected the collaborator with the maximum individual capability and maximum collaborative capability. The comparison results showed that the MIMC outperformed other group-member selection strategies. Thirdly, we compared the classification performance with different values of μ. The results showed that the optimized μ could balance the contribution of individual capability and collaborative capability. Furthermore, we combined the MIMC strategy with SFFS to address the issue of sub-group selection in RSVP-based cBCI systems. The results showed that the proposed method could sacrifice a slight amount of system performance to substantially reduce the number of members in a multi-user cBCI system.

4.2 Limitations and future research

Although the proposed method can effectively optimize the group mode in the RSVP-based cBCI system, there are three major limitations to this method. First, this study is based on public datasets and requires further validation through online experimentation. Another limitation is that only a group with fourteen collaborators was included in our experiment. In follow-up studies, the EEG data of more groups and the EEG data of groups with more members should be recorded, and the proposed method should be tested on these groups. Finally, it should be noted that the pseudo-multi-user collaboration experiments might be not identical to the actual multi-user collaboration experiments. It is necessary to verify our method in a real-world multi-user cBCI system.

Additionally, the development of cBCI technology is expected to support the innovation of the next generation of human-computer interaction systems (Jiang et al., 2019; Gu et al., 2021). We believe that the Internet of Brains (IoB) is an essential development direction for multi-user human-computer interaction systems (Martins et al., 2019; Moioli et al., 2021; Hu et al., 2024). The IoB can be considered as the next step in the Internet of Things (IoT) (Silva et al., 2017). In the IoB, multiple brains are seamlessly connected to the wireless network as nodes of the communication grid. Compared with the IoT, the IoB with “brain in the loop” would allow more direct interactions between users and networks. The future cBCI-based IoB should be expanded in the following aspects: (1) Group-member selection strategies: The random grouping mode results in poor collaborative performance. Therefore, cBCI systems should optimize the group mode based on both individual capabilities and relationships between the users to improve the group's performance. (2) Task allocation strategies for single BCI paradigm (Gu et al., 2021): The users performed the same task together in the existing cBCI system, which did not fully consider the rationality of task allocation. The division-of-work strategies that reduced the number of users recognizing the same instructions could reduce individual workload and improve overall system performance. (3) Task allocation strategies for hybrid BCI (Cecotti and Rivet, 2014): Cecotti and Rivet proposed the cooperative-hybrid BCI, which involved multi-user and multi-paradigm. We assume that the collaborative has varying adaptability to different BCI paradigms. Based on this hypothesis, task allocation for hybrid BCI should consider user adaptability to different paradigms, such as the P300 speller and RSVP, to further optimize the division-of-work strategies.

5 Conclusion

In this work, we introduced a novel group-member selection strategy that considered both individual capability and collaborative capability within the RSVP-based cBCI system. The effectiveness of the proposed MIMC was demonstrated through its application to both the two-user matching task and the sub-group selection task. For the two-user matching task, the classification results showed that the specially optimized group mode surpasses traditional random grouping and other group-member selection methods based on AUC. Furthermore, for the sub-group selection task, the implementation of SFFS with MIMC successfully achieved the trade-off between maintaining performance and enhancing system efficiency. Consequently, our research contributes to the practical advancement of RSVP-based cBCI systems for real-world applications.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Author contributions

YS: Data curation, Formal analysis, Methodology, Project administration, Writing – original draft. ZhW: Conceptualization, Formal analysis, Methodology, Writing – review & editing. GX: Validation, Writing – review & editing, Formal analysis. ZiW: Validation, Writing – review & editing, Formal analysis. TX: Writing – review & editing, Validation. TZ: Validation, Writing – review & editing. HH: Conceptualization, Supervision, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This work was supported by the Shanghai Industrial Collaborative Innovation Project under Grant XTCX-KJ-2023-05.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Acqualagna, L., and Blankertz, B. (2013). Gaze-independent BCI-spelling using rapid serial visual presentation (RSVP). Clin. Neurophysiol. 124, 901–908. doi: 10.1016/j.clinph.2012.12.050

PubMed Abstract | Crossref Full Text | Google Scholar

Bigdely-Shamlo, N., Vankov, A., Ramirez, R. R., and Makeig, S. (2008). Brain activity-based image classification from rapid serial visual presentation. IEEE Trans. Neural Syst. Rehabil. Eng. 16, 432–441. doi: 10.1109/TNSRE.2008.2003381

PubMed Abstract | Crossref Full Text | Google Scholar

Cecotti, H., and Rivet, B. (2014). Subject combination and electrode selection in cooperative brain-computer interface based on event related potentials. Brain Sci. 4, 335–355. doi: 10.3390/brainsci4020335

PubMed Abstract | Crossref Full Text | Google Scholar

Du, P., Li, P., Cheng, L., Li, X., and Su, J. (2023). Single-trial p300 classification algorithm based on centralized multi-person data fusion CNN. Front. Neurosci. 17:1132290. doi: 10.3389/fnins.2023.1132290

PubMed Abstract | Crossref Full Text | Google Scholar

Farwell, L. A., and Donchin, E. (1988). Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr. Clin. Neurophysiol. 70, 510–523. doi: 10.1016/0013-4694(88)90149-6

PubMed Abstract | Crossref Full Text | Google Scholar

Gu, B., Xu, M., Xu, L., Chen, L., Ke, Y., Wang, K., et al. (2021). Optimization of task allocation for collaborative brain–computer interface based on motor imagery. Front. Neurosci. 15:683784. doi: 10.3389/fnins.2021.683784

PubMed Abstract | Crossref Full Text | Google Scholar

He, H., and Wu, D. (2020). Transfer learning for brain–computer interfaces: a euclidean space data alignment approach. IEEE Trans. Biomed. Eng. 67, 399–410. doi: 10.1109/TBME.2019.2913914

PubMed Abstract | Crossref Full Text | Google Scholar

Henry, J. C. (2006). Electroencephalography: basic principles, clinical applications, and related fields, fifth edition. Neurology 67, 2092–2092-a. doi: 10.1212/01.wnl.0000243257.85592.9a

Crossref Full Text | Google Scholar

Hope, C., Sterr, A., Elangovan, P., Geades, N., Windridge, D., Young, K., et al. (2013). “High throughput screening for mammography using a human-computer interface with rapid serial visual presentation (RSVP),” in Medical Imaging 2013: Image Perception, Observer Performance, and Technology Assessment, Vol. 8673, eds. C. K. Abbey, and C. R. Mello-Thoms (Bellingham, WA: SPIE), 9–16. doi: 10.1117/12.2007557

Crossref Full Text | Google Scholar

Hu, H., Wang, Z., Zhao, X., Li, R., Li, A., Si, Y., et al. (2024). A survey on brain-computer interface-inspired communications: opportunities and challenges. IEEE Commun. Surv. Tutor. 1, 1–1. doi: 10.1109/COMST.2024.3396847

Crossref Full Text | Google Scholar

Huang, Y., Erdogmus, D., Pavel, M., Mathan, S., and Hild, K. E. (2011). A framework for rapid visual image search using single-trial brain evoked responses. Neurocomputing, 74, 2041–2051. doi: 10.1016/j.neucom.2010.12.025

PubMed Abstract | Crossref Full Text | Google Scholar

Jiang, L., Stocco, A., Losey, D. M., Abernethy, J. A., Prat, C. S., Rao, R. P. N., et al. (2019). Brainnet: a multi-person brain-to-brain interface for direct collaboration between brains. Sci. Rep. 9:6115. doi: 10.1038/s41598-019-41895-7

PubMed Abstract | Crossref Full Text | Google Scholar

Lawhern, V. J., Solon, A. J., Waytowich, N. R., Gordon, S. M., Hung, C. P., Lance, B. J., et al. (2018). EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces. J. Neural Eng. 15:056013. doi: 10.1088/1741-2552/aace8c

PubMed Abstract | Crossref Full Text | Google Scholar

Lees, S., Dayan, N., Cecotti, H., McCullagh, P., Maguire, L., Lotte, F., et al. (2018). A review of rapid serial visual presentation-based brain–computer interfaces. J. Neural Eng. 15:021001. doi: 10.1088/1741-2552/aa9817

PubMed Abstract | Crossref Full Text | Google Scholar

Li, B., Lin, Y., Gao, X., and Liu, Z. (2021). Enhancing the EEG classification in RSVP task by combining interval model of ERPs with spatial and temporal regions of interest. J. Neural Eng. 18:016008. doi: 10.1088/1741-2552/abc8d5

PubMed Abstract | Crossref Full Text | Google Scholar

Li, Y., Wang, C., Zhang, H., and Guan, C. (2008). “An EEG-based BCI system for 2D cursor control,” in 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence) (Hong Kong), 2214–2219.

Google Scholar

Liu, S., Wang, W., Sheng, Y., Zhang, L., Xu, M., Ming, D., et al. (2020). Improving the cross-subject performance of the ERP-based brain–computer interface using rapid serial visual presentation and correlation analysis rank. Front. Hum. Neurosci. 14:296. doi: 10.3389/fnhum.2020.00296

PubMed Abstract | Crossref Full Text | Google Scholar

Long, J., Li, Y., Wang, H., Yu, T., Pan, J., Li, F., et al. (2012). A hybrid brain computer interface to control the direction and speed of a simulated or real wheelchair. IEEE Trans. Neural Syst. Rehabil. Eng. 20, 720–729. doi: 10.1109/TNSRE.2012.2197221

PubMed Abstract | Crossref Full Text | Google Scholar

Lotte, F., Bougrain, L., Cichocki, A., Clerc, M., Congedo, M., Rakotomamonjy, A., et al. (2018). A review of classification algorithms for eeg-based brain–computer interfaces: a 10 year update. J. Neural Eng. 15:031005. doi: 10.1088/1741-2552/aab2f2

PubMed Abstract | Crossref Full Text | Google Scholar

Marathe, A. R., Ries, A. J., Lawhern, V. J., Lance, B. J., Touryan, J., McDowell, K., et al. (2015). The effect of target and non-target similarity on neural classification performance: a boost from confidence. Front. Neurosci. 8:270. doi: 10.3389/fnins.2015.00270

PubMed Abstract | Crossref Full Text | Google Scholar

Martins, N. R. B., Angelica, A., Chakravarthy, K., Svidinenko, Y., Boehm, F. J., Opris, I., et al. (2019). Human brain/cloud interface. Front. Neurosci. 13:112. doi: 10.3389/fnins.2019.00112

PubMed Abstract | Crossref Full Text | Google Scholar

Matran-Fernandez, A., and Poli, R. (2014). “Collaborative brain-computer interfaces for target localisation in rapid serial visual presentation,” in 2014 6th Computer Science and Electronic Engineering Conference (CEEC) (Colchester), 127–132. doi: 10.1109/CEEC.2014.6958567

Crossref Full Text | Google Scholar

Matran-Fernandez, A., and Poli, R. (2017). Brain–computer interfaces for detection and localization of targets in aerial images. IEEE Trans. Biomed. Eng. 64, 959–969. doi: 10.1109/TBME.2016.2583200

PubMed Abstract | Crossref Full Text | Google Scholar

McFarland, D. J., and Wolpaw, J. R. (2011). Brain-computer interfaces for communication and control. Commun. ACM 54, 60–66. doi: 10.1145/1941487.1941506

PubMed Abstract | Crossref Full Text | Google Scholar

Moioli, R. C., Nardelli, P. H. J., Barros, M. T., Saad, W., Hekmatmanesh, A., Silva, P. E. G., et al. (2021). Neurosciences and wireless networks: the potential of brain-type communications and their applications. IEEE Commun. Surv. Tutor. 23, 1599–1621. doi: 10.1109/COMST.2021.3090778

Crossref Full Text | Google Scholar

Pfurtscheller, G., Allison, B., Bauernfeind, G., Brunner, C., Solis Escalante, T., Scherer, R., et al. (2010). The hybrid bci. Front. Neurosci. 4:3. doi: 10.3389/fnpro.2010.00003

PubMed Abstract | Crossref Full Text | Google Scholar

Polich, J., and Donchin, E. (1988). P300 and the word frequency effect. Electroencephalogr. Clin. Neurophysiol. 70, 33–45. doi: 10.1016/0013-4694(88)90192-7

PubMed Abstract | Crossref Full Text | Google Scholar

Poolman, P., Frank, R. M., Luu, P., Pederson, S. M., and Tucker, D. M. (2008). A single-trial analytic framework for EEG analysis and its application to target detection and classification. Neuroimage 42, 787–798. doi: 10.1016/j.neuroimage.2008.03.031

PubMed Abstract | Crossref Full Text | Google Scholar

Pudil, P., Novovičová, J., and Kittler, J. (1994). Floating search methods in feature selection. Pattern Recognit. Lett. 15, 1119–1125. doi: 10.1016/0167-8655(94)90127-9

Crossref Full Text | Google Scholar

Rivet, B., Cecotti, H., Souloumiac, A., Maby, E., and Mattout, J. (2011). “Theoretical analysis of xdawn algorithm: application to an efficient sensor selection in a p300 BCI,” in 2011 19th European Signal Processing Conference (Barcelona), 1382–1386.

Google Scholar

Rivet, B., Souloumiac, A., Attina, V., and Gibert, G. (2009). xdawn algorithm to enhance evoked potentials: application to brain–computer interface. IEEE Trans. Biomed. Eng. 56, 2035–2043. doi: 10.1109/TBME.2009.2012869

PubMed Abstract | Crossref Full Text | Google Scholar

Rosenthal, D., DeGuzman, P., Parra, L. C., and Sajda, P. (2014). Evoked neural responses to events in video. IEEE J. Sel. Top. Signal Process. 8, 358–365. doi: 10.1109/JSTSP.2014.2313022

Crossref Full Text | Google Scholar

Sajda, P., Pohlmeyer, E., Wang, J., Parra, L. C., Christoforou, C., Dmochowski, J., et al. (2010). In a blink of an eye and a switch of a transistor: cortically coupled computer vision. Proc. IEEE 98, 462–478. doi: 10.1109/JPROC.2009.2038406

Crossref Full Text | Google Scholar

Salvatore, C., Valeriani, D., Piccialli, V., and Bianchi, L. (2022). Optimized collaborative brain-computer interfaces for enhancing face recognition. IEEE Trans. Neural Syst. Rehabil. Eng. 30, 1223–1232. doi: 10.1109/TNSRE.2022.3173079

PubMed Abstract | Crossref Full Text | Google Scholar

Silva, J. S., Zhang, P., Pering, T., Boavida, F., Hara, T., Liebau, N. C., et al. (2017). People-centric internet of things. IEEE Commun. Mag. 55, 18–19. doi: 10.1109/MCOM.2017.7841465

PubMed Abstract | Crossref Full Text | Google Scholar

Sivarajah, Y., Holden, E.-J., Togneri, R., Price, G., and Tan, T. (2014). Quantifying target spotting performances with complex geoscientific imagery using ERP p300 responses. Int. J. Hum. Comput. Stud. 72, 275–283. doi: 10.1016/j.ijhcs.2013.10.007

Crossref Full Text | Google Scholar

Stoica, A., Matran-Fernandez, A., Andreou, D., Poli, R., Cinel, C., Iwashita, Y., et al. (2013). (2013). “Multi-brain fusion and applications to intelligence analysis,” in Multisensor, Multisource Information Fusion: Architectures, Algorithms, and Applications, Vol. 8756, ed. B. V. Dasarathy (Bellingham, WA: SPIE), 205–212. doi: 10.1117/12.2016456

Crossref Full Text | Google Scholar

Valeriani, D., Cinel, C., and Poli, R. (2017b). Group augmentation in realistic visual-search decisions via a hybrid brain-computer interface. Sci. Rep. 7:7772. doi: 10.1038/s41598-017-08265-7

PubMed Abstract | Crossref Full Text | Google Scholar

Valeriani, D., Poli, R., and Cinel, C. (2015). “A collaborative brain-computer interface to improve human performance in a visual search task,” in 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER) (Montpellier), 218–223. doi: 10.1109/NER.2015.7146599

Crossref Full Text | Google Scholar

Valeriani, D., Poli, R., and Cinel, C. (2017a). Enhancement of group perception via a collaborative brain–computer interface. IEEE Trans. Biomed. Eng. 64, 1238–1248. doi: 10.1109/TBME.2016.2598875

PubMed Abstract | Crossref Full Text | Google Scholar

Värbu, K., Muhammad, N., and Muhammad, Y. (2022). Past, present, and future of EEG-based BCI applications. Sensors 22:3331. doi: 10.3390/s22093331

PubMed Abstract | Crossref Full Text | Google Scholar

Vidal, J. J. (1973). Toward direct brain-computer communication. Annu. Rev. Biophys. Bioeng., 2, 157–180. doi: 10.1146/annurev.bb.02.060173.001105

PubMed Abstract | Crossref Full Text | Google Scholar

Volosyak, I., Cecotti, H., Valbuena, D., and Graser, A. (2009). “Evaluation of the bremen SSVEP based BCI in real world conditions,” in 2009 IEEE International Conference on Rehabilitation Robotics (Kyoto), 322–331. doi: 10.1109/ICORR.2009.5209543

Crossref Full Text | Google Scholar

Wang, Y., and Jung, T.-P. (2011). A collaborative brain-computer interface for improving human performance. PLoS ONE 6:e20422. doi: 10.1371/journal.pone.0020422

PubMed Abstract | Crossref Full Text | Google Scholar

Wang, Z., Zhang, H., Ji, Z., Yang, Y., and Wang, H. (2023). A review of deep learning methods for cross-subject rapid serial visual presentation detection in world robot contest 2022. Brain Sci. Adv. 9, 195–209. doi: 10.26599/BSA.2023.9050013

Crossref Full Text | Google Scholar

Weiden, M., Khosla, D., and Keegan, M. (2012). “Electroencephalographic detection of visual saliency of motion towards a practical brain-computer interface for video analysis,” in Proceedings of the 14th ACM international conference on Multimodal interaction (New York, NY: ACM), 601–606. doi: 10.1145/2388676.2388800

Crossref Full Text | Google Scholar

Wu, H., and Wu, D. (2022). Review of training-free event-related potential classification approaches in the world robot contest 2021. Brain Sci. Adv. 8, 82–98. doi: 10.26599/BSA.2022.9050001

Crossref Full Text | Google Scholar

Wu, W., and Gao, S. (2011). “Learning event-related potentials (ERPs) from multichannel EEG recordings: a spatio-temporal modeling framework with a fast estimation algorithm,” in 2011 33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (Boston, MA), 6959–6962. doi: 10.1109/IEMBS.2011.6091759

PubMed Abstract | Crossref Full Text | Google Scholar

Xiao, X., Xu, M., Han, J., Yin, E., Liu, S., Zhang, X., et al. (2021). Enhancement for p300-speller classification using multi-window discriminative canonical pattern matching. J. Neural Eng. 18:046079. doi: 10.1088/1741-2552/ac028b

PubMed Abstract | Crossref Full Text | Google Scholar

Xiao, X., Xu, M., Jin, J., Wang, Y., Jung, T.-P., Ming, D., et al. (2020). Discriminative canonical pattern matching for single-trial classification of ERP components. IEEE Trans. Biomed. Eng. 67, 2266–2275. doi: 10.1109/TBME.2019.2958641

PubMed Abstract | Crossref Full Text | Google Scholar

Zhang, H., Wang, Z., Yu, Y., Yin, H., Chen, C., Wang, H., et al. (2022). An improved EEGNet for single-trial EEG classification in rapid serial visual presentation task. Brain Sci. Adv. 8, 111–126. doi: 10.26599/BSA.2022.9050007

Crossref Full Text | Google Scholar

Zhang, H., Zhu, L., Xu, S., Cao, J., and Kong, W. (2021). Two brains, one target: design of a multi-level information fusion model based on dual-subject rsvp. J. Neurosci. Methods 363:109346. doi: 10.1016/j.jneumeth.2021.109346

PubMed Abstract | Crossref Full Text | Google Scholar

Zhao, Z., Lin, Y., Wang, Y., and Gao, X. (2024). Single-trial eeg classification using spatio-temporal weighting and correlation analysis for rsvp-based collaborative brain computer interface. IEEE Trans. Biomed. Eng. 71, 553–562. doi: 10.1109/TBME.2023.3309255

PubMed Abstract | Crossref Full Text | Google Scholar

Zheng, L., Sun, S., Zhao, H., Pei, W., Chen, H., Gao, X., et al. (2020). A cross-session dataset for collaborative brain-computer interfaces based on rapid serial visual presentation. Front. Neurosci. 14:579469. doi: 10.3389/fnins.2020.579469

PubMed Abstract | Crossref Full Text | Google Scholar