Advances in P300 brain–computer interface spellers: toward paradigm design and performance evaluation

A brain-computer interface (BCI) is a non-muscular communication technology that provides an information exchange channel for our brains and external devices. During the decades, BCI has made noticeable progress and has been applied in many fields. One of the most traditional BCI applications is the BCI speller. This article primarily discusses the progress of research into P300 BCI spellers and reviews four types of P300 spellers: single-modal P300 spellers, P300 spellers based on multiple brain patterns, P300 spellers with multisensory stimuli, and P300 spellers with multiple intelligent techniques. For each type of P300 speller, we further review several representative P300 spellers, including their design principles, paradigms, algorithms, experimental performance, and corresponding advantages. We particularly emphasized the paradigm design ideas, including the overall layout, individual symbol shapes and stimulus forms. Furthermore, several important issues and research guidance for the P300 speller were identified. We hope that this review can assist researchers in learning the new ideas of these novel P300 spellers and enhance their practical application capability.

Introduction

A brain-computer interface (BCI) is a technology that directly controls external devices by analyzing the electrical signals sent by the nerves of the brain (Wolpaw et al., 2000; Wolpaw, 2007). It was developed to offer patients with motor neuron disease (MND; Hanagasi et al., 2002), including amyotrophic lateral sclerosis (ALS) and locked-in syndrome, a way to communicate with the outside world and regain their social life to a relatively high extent. There are many types of BCI systems related to the detection method of brain activities, including those based on electroencephalography (EEG), near-infrared spectroscopy, magnetoencephalography, and functional magnetic resonance (He and Liu, 2008; Millán and Carmena, 2010), and the most common are EEG-based BCI systems due to their features of noninvasiveness, portability and high signal responses (Wolpaw et al., 1991; De Vos et al., 2014). EEG signals included P300 (Sellers et al., 2006; Kleih and Kübler, 2013; Xu M. et al., 2013), steady-state visual evoked potential (SSVEP; Müller-Putz and Pfurtscheller, 2008; Wu et al., 2008; Jalilpour et al., 2020), and motor imagery (MI; Devlaminck et al., 2011; Park et al., 2012; Kevric and Subasi, 2017). P300 is a positive event related potential (ERP) component that can be generated during an oddball paradigm. In an oddball paradigm, users were presented with a sequence of events that could be divided into two categories (target and non-target), one of which was rarely presented. The P300 is detected 300 ms after the occurrence of a small probability event stimulus (Sutton et al., 1965). SSVEP is a signal induced by presenting a visual stimulus with a specific flickering frequency. P300 and SSVEPs are commonly used EEG signals that do not require training on subjects, and the information transfer rate (ITR) is relatively high. MI is an EEG signal that the subject can emit without external stimulation (Rao, 2013).

At present, BCI is used in many fields, including communication (Kennedy and Bakay, 1998; Hochberg et al., 2006), medical rehabilitation (Rohani and Puthusserypady, 2015), home automation (Corralejo et al., 2014; Park et al., 2015), and game entertainment (Leeb et al., 2013). The BCI speller is one of the most common applications and has been developed for decades. Different types of spellers based on various EEG signals have been studied. The P300-based BCI speller is the most common BCI speller. The earliest BCI speller is a P300-based speller proposed by Farwell and Donchin, which is the basis of many researchers conducted later. Another relatively common type of BCI speller is an SSVEP-based BCI speller. In addition, there are BCI spellers based on MI, and now there are some spellers that combine two EEG signals, such as P300+SSVEP and P300+MI.

Reviews on P300 spelling systems have been conducted to better understand the progress of this field. In Cecotti (2011), Cecotti gave an overview of BCI spellers, including those based on P300, SSVEP, and MI, and the limitations and challenges were pointed out and proposed to help future research tasks. In Philip and George (2020) and Xu et al. (2021), algorithms applied in the P300 system were introduced. Rezeika et al. (2018) gave a systematic introduction of BCI spellers and mainly focused on some successful spellers proposed since 2010. In recent years, to our knowledge, no review has described P300 spellers about their paradigm design and performance. In this article, we will focus on introducing P300 spellers, especially those proposed in recent years, and mention the paradigm design ideas, including the overall layout, individual symbol shapes, and stimulus forms.

This review was conducted according to the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) guidelines (Moher et al., 2009), as shown in Figure 1. We searched the online database Web of Science and title-abstract keywords (“brain-computer interface” or “BCI”) and (“speller”). First the search was conducted without year constraints and Web of Science showed a total of 936 articles. Later we excluded 91 articles published before 2010 and the remaining 845 were published from 2010 to October 2022. Then 189 articles not related to P300 were excluded and the number of remaining studies was 656 which indicated that nearly 80% of research on BCI spellers was based on P300. Finally we selected 54 of them related to non-invasive BCI and the development of a new graphical user interface.

FIGURE 1

Figure 1. Prisma flow chart.

Methods

Many efforts have been devoted to the development of the P300 spelling paradigm. On the one hand various parameters of this paradigm have been studied and optimized such as matrix size (Allison and Pineda, 2003), stimulus interval (Sellers et al., 2006), stimulus color (Takano et al., 2009; Zhang et al., 2021), and so on (Xue et al., 2019). On the other hand, researches on developing novel P300 spellers have evolved considerably in the past two decades. In this section, we divide the existing research on P300 spellers into four categories, each of which will highlight some of the representatives, along with their design tenets, paradigms, algorithms, experimental performance, and corresponding advantages.

Single-modal P300 speller

P300 can be induced in different ways, such as by visual stimuli, auditory stimuli, tactile stimuli, etc. Among them, visual stimuli are the most typically used to evoke the P300, which are also extensively utilized in P300 BCI spellers. The majority of visual spellers are designed based on them. In this subsection, we first reviewed the classic row-column P300 speller and then summarized different structure improvements based on it.

The first P300 speller was proposed by Farwell and Donchin (1988). The paradigm was a matrix with six rows and columns (row-column paradigm, RCP), as shown in Figure 2. Each row and column were intensified in a random order as the “stimulus event” to induce the P300 potential. At least 12 flashed were needed to cover all items in the matrix. Subjects were instructed to focus on the target letter and count the flashing times of the row and column containing the target to help focus. When the row and column containing the target item flashed, the P300 wave was generated and processed. The system recognized the row and column that elicited the largest P300s. The intersection letter placed at the exact row and column was identified as the target.

FIGURE 2

Figure 2. This is an example of the row-column paradigm for the traditional P300 BCI speller. Entire rows (A) and columns (B) of symbols on a 6×6 grid are flashed randomly.

The research showed that the typing speed and the maximum accuracy in this study are 12 bits/min and 95%, respectively. This speed was relatively slow compared to the traditional typing methods and seemed to make no sense for healthy individuals, but it was extremely important for those who lacked alternative communication methods. In the development of BCI, the matrix speller played a crucial role since it was the first time that the P300 potential was used to help communication and opened the floodgates to new research of P300.

Guan et al. (2004) first proposed the single display (SD) speller. This speller had a 6×6 matrix similar to that of the RCP, and each character was intensified individually. The paradigm is presented in Figure 3. Thirty-six flashes were required to cover all symbols in the matrix, which was three times more than the RCP. In the SD paradigm, the stimulus event that flashed a single character was scarcer than in the RCP, thus eliciting a higher P300 potential. This will also take the subjects more time to catch the target letter in the SD speller. The online experiment showed that the SD speller significantly improved the accuracy. In addition, according to their comparative experiment with RCP, which was conducted based on six subjects, the error rate of the SD paradigm was reduced by up to 80% when 10 trials were used for ensemble average. The SD paradigm was more flexible in that it allowed more kinds of user interface designs, such that the items could spread in different regions rather than gathering in the same square matrix. In addition, the SD paradigm also causes less fatigue, but the typing speed would be lower than that of the RCP.

FIGURE 3

Figure 3. The single display paradigm for the P300 BCI speller. Modified from Pan et al. (2013).

Fazel-Rezai (2007) showed that in the RCP, subjects tended to have a perceptual error called the “adjacency problem”. Flashes of the rows and columns adjacent to targets seemed to distract the subjects, causing a great number of incorrect detections. To avoid this problem, a region-based speller was proposed by Fazel-Rezai and Abhari (2009). The experimental results in their research indicated that the adjacency problem was reduced by using the region-based paradigm compared with the RCP. Shi et al. (2012) proposed a submatrix-based paradigm, which divided the 6×6 matrix of RCP into several submatrices. Characters in submatrices intensified randomly and individually. The results showed that the submatrix-based paradigm effectively reduced the error caused by the adjacency problem.

Except for the adjacency problem, the “double flash” issue in the RCP was mentioned, which was said that the successive flashes of target might cause wrong detection of P300 (Woldorff, 1993; Martens et al., 2009). To avoid these two problems, the checkerboard paradigm was proposed in Townsend et al. (2010), which was composed of an 8×9 matrix superimposed by an invisible checkerboard. The 36 items placed in the white checkerboard and 36 items in the black checkerboard were reorganized into white and black 6×6 matrices, respectively. The rows in the two matrices were intensified from top to bottom, one by one, followed by their columns from left to right. During the experiment, subjects only saw a standard unchanged matrix with symbols flashing randomly but could not realize the checkerboard. It was proved that the checkerboard paradigm could effectively address the adjacency problem and “double flash” issue.

In subsequent research, many of them tried to modify the paradigm design based on the RCP to improve the performance. In Kaufmann et al. (2011), Kaufmann et al. used the RCP and improved classic character flashing (CF) and novel face flashing (FF) to pixelated face flashing (PFF). CF is the traditional flashing method, while FF is the face flashing method. PFF combined CF and FF to use the intensified rows and columns covered with a familiar face. The experimental results showed that under the PFF mode, subjects’ ERPs increased significantly, which indicated that they reacted faster to the familiar face paradigm and reached higher accuracy than the traditional RCP. Later in Lu et al. (2020), Lu et al. replaced the familiar face with self-faces of subjects and obtained better performance of the spelling system. In Höhne et al. (2011), Höhne et al. proposed an audio speller system with a T9 (text on nine keys) paradigm similar to the nine-key input keyboard used in early cellphones. In Akram et al. (2015), Akram et al. modified the paradigm in Höhne et al. (2011), proposing a visual T9 P300 speller with a dictionary to give suggestions of words to improve typing speed. The experimental results showed that the modified T9 speller significantly reduced the typing time compared to the traditional RCP. Aygün and Kavsaoğlu (2022) expanded the matrix to 7 rows and 7 columns, adding 20 shortcut symbols to select 20 prediction words on the right sight of the screen. Although the 7×7 matrix needed more time to detect a single character, the shortcut elements effectively reduced the time to spell a complete word. According to their comparative experiment, it took only 1.31 min to spell an 8-letter world but 4.53 min for the conventional 6×6 RCP.

The spelling task could also be done in auditory and tactile modality. In van der Waal et al. (2012), van der Waal et al. developed a speller based on tactile stimulation. They used a tactile stimulator and applied it to six of the 10 fingers, each of which represented a character. Subjects selected their desired character by counting the number of tactile stimuli that occurred in the corresponding finger. Chang et al. (2014) presented an auditory speller in Japanese with a two-step input procedure to offer a simple and user-friendly interface. Table 1 gives a summary of the studies mentioned in this subsection.

TABLE 1

Table 1. Summary of single-modal spellers discussed in this review.

There are few P300 spellers based on auditory stimuli or tactile stimuli due to their quite low ITR and accuracy. P300 spellers based on visual stimuli were the most successful. Visual stimuli have an inherent advantage in that they induce higher P300 amplitudes and lower latency and are thus much easier to classify (Aloise et al., 2007). However, some patients who suffer from serious visual impairment have no ability to use visual P300 spellers. In this situation, the non-visual P300 spellers may be an alternative way to communicate with them. Therefore, more attention should be given to spellers based on auditory and tactile stimuli regardless of their relatively poor performance.

P300 speller based on multiple brain patterns

Many studies have attempted to enhance the performance of the P300 speller system by combining other signals (e.g., P300 and SSVEPs or P300 and MI) with the P300 spellers to improve their performance. For example, the brain patterns induced by the P300 and SSVEP paradigms are different. The utilization of both P300 and SSVEP features from different domains (time domain and frequency domain) is beneficial to construct more efficient BCI systems.

In Yin et al. (2013), Yin et al. first presented hybrid BCI spellers based on the SSVEP and P300, which showed that incorporating other EEG signals into the P300 paradigm was an effective way to improve the performance of speller systems. Later, in Yin et al. (2014), they proposed another hybrid one using two presentation modes: subarea/location and RC mode. In the subarea/location mode, they divided the 6×6 matrix into six groups, and each subgroup flashed at a specific frequency, each of which corresponded to an SSVEP feature. Meanwhile, the symbols in the subgroups flashed with orange crosses, which were used to induce the P300 potential. With the detected SSVEP and P300 signals, the target character can be located. For the RC mode, each column of the matrix flashed at a specific frequency to elicit SSVEPs, and the characters of each row flashed to induce P300. The comparison experiment in the research indicated that the RC mode reached a higher ITR of 53.06 bits/min than the subarea/location mode.

In Santamaría-Vázquez et al. (2019), Santamaría-Vázquez et al. proposed an asynchronous BCI speller based on the P300 and SSVEP. The researchers hold the hypothesis that the peripheral stimuli (i.e., non-target stimuli) of the RCP trigger a weak SSVEP in the user’s EEG. In this article, they put forward two kinds of analysis. The first was to determine how the characteristics of SSVEPs change as a function of stimulus frequency, and the second was to determine the reason for triggering SSVEPs and evaluate their independence from transient ERPs. The conclusion was drawn from the controlled and non-controlled operations. In the control state, the SSVEPs of the EEG signal can be detected. Conversely, SSVEPs should not be found in the noncontrol state. The spelling paradigm was similar to the RCP. The author proposed a novel algorithm called the oddball steady response detection method, which provided a binary output. If the detection was a control state, the system used stepwise linear discriminant analysis (SWLDA) to select a command. In contrast, if the detection was not controlled, no further action was taken. Finally, 60 real controlled observations and 60 synthetic non-control observations of the training set are used to evaluate the performance of oddball steady response detection in offline sessions and online sessions. The results showed that the average accuracy of this speller can reach 95.5%.

Reference (Jalilpour et al., 2020) introduced a hybrid BCI system using rapid serial visual presentation (RSVP) in conjunction with SSVEP. The RSVP (eliciting a P300 response) speller is gaze-independent and space-independent, so it can be used by persons with visual impairment. The proposed paradigm in Jalilpour et al. (2020) divided 27 characters into nine groups, and each group included three characters. The character groups were presented in triple RSVP, and simultaneously, a white square was placed in the center of the screen surrounded by a group of three characters flickering at a fixed frequency. When the subjects looked at each of three different characters, the spatial maps related to the SSVEP potential were different, which was used to identify the target character of the group. Since the P300 and SSVEPs played different roles in this paradigm, researchers used different ways to extract and classify the two signals. In the feature extraction, the authors used discrete wavelet transform and canonical correlation analysis for the P300 and SSVEP, respectively. In addition, they used regularized linear discriminant analysis to classify the P300 potential, and as the SSVEP had a multiclass feature, the researchers used two classifiers, radial basis function (RBF) and support vector machine (SVM), called RBF-SVM classification, to distinguish the correct directions in the SSVEP mode. For the performer of this speller, research showed that it could achieve an accuracy of 93.06% and an ITR of 23.41 bits/min averaged from the data of six subjects. Compared with the single RSVP and triple RSVP spellers in Acqualagna and Blankertz (2013) and Lin et al. (2018), respectively, this speller can reach an accuracy almost as high as the single RSVP and an ITR higher than both. It can be seen that RSVP in conjunction with SSVEP is a recommended way to raise the ITR of the only simple RSVP speller.

In Hwang et al. (2012), a QWERTY (nickname of the traditional keyboard) keyboard layout was first used as the SSVEP speller paradigm by. The QWERTY keyboard layout is that the characters and numbers are arranged in the sequence as the standard keyboard. In Katyal and Singla (2021), Katyal and Singla added the P300 signal into the QWERTY SSVEP speller and achieved a relatively high ITR. In their hybrid speller paradigm, the 36 symbols in the traditional QWERTY SSVEP speller were separated into two sets, one of which included four groups and the other of which included five groups of symbols. Each group had four symbols. It used only five flickering frequencies to induce SSVEPs, while the traditional QWERTY SSVEP speller had 36 different frequencies, which were so close to each other that they reduced the accuracy of the system. The P300 was used to identify the target set, separate 36 symbols and reduce the number of flickering frequencies, so the gap between frequencies could be widened and classification accuracy was improved significantly. The average accuracy and ITR of this hybrid QWERTY speller reached 96.42% and 131 bits/min, respectively. Except for SSVEP, Yu et al. achieved an asynchronous control speller system by combining a MI signal as a switch to turn on/off the speller system (Yu et al., 2016). Table 2 is a summary of the studies mentioned in this subsection.

TABLE 2

Table 2. Summary of BCI spellers based on SSVEP and P300.

Ideally, the P300 speller combined with other signals can reach a relatively high ITR compared with the single-modal P300 speller. However, in actual situations, individuals react differently to different kinds of EEG signals. For example, some subjects could produce an intense P300 potential but a weak SSVEP and vice versa. Because the spellers depended on multimodal brain signals, if any one of them could not be well detected, the whole spelling system would not work properly. In addition, a speller should not only allow the users to do spelling but also spell at their own pace, so achieving asynchronous control became necessary. One method was to merge MI into speller systems, but MI required long training sessions in subjects, which increased the usage cost of spellers. Future work can attempt to combine other signals (EEG or others) to achieve asynchronous control.

P300 speller with multisensory stimuli

Researchers have developed P300 BCI based on various kinds of stimuli, including visual P300 BCI (Kaufmann et al., 2011; Li et al., 2015), auditory P300 BCI (Guo et al., 2010; Halder et al., 2010; Xu H. et al., 2013), and tactile P300 BCI (Brouwer and van Erp, 2010), with the intention to be used more universally. The most common P300 BCI is the visual-based BCI since it performs much better than the other types of stimuli, as is the P300 speller. It is difficult to find a P300 speller based on audio-only or tactile-only stimuli. Recently, some researchers have proposed P300 spellers based on multiple senses. In reference Sellers and Donchin (2006), Sellers and Donchin designed a simple paradigm providing three presentation modes, auditory, visual, and audiovisual, and texted it with ALS patients. The result found that it is feasible to use audiovisual stimuli with ALS patients. In reference Belitski et al. (2011), Belitski et al. introduced a matrix audiovisual P300 speller, which numbered every row/column, and each of them intensified with a spoken number. The results showed that the performance of this multisensory speller slightly outperformed both the uni-visual and uni-audio P300 spellers. This paradigm laid the groundwork for later research.

Lu et al. (2019a) proposed a two-level audiovisual paradigm on the basis of the traditional regional flashing paradigm. As shown in Figure 4, the paradigm included two levels: level 1 was made up of six group areas, each of which included six characters (Figure 4A); level 2 consisted of six single characters spreading from one group area of level 1 (Figure 4B). In level 1, six areas were arranged in three rows and two columns, numbered from top to bottom and from left to right. Each area was intensified randomly and was selected with a broadcast of its serial number from one sight of the earphone (Figure 4A). Similarly, in level 2, six characters were arranged into a 3×2 matrix, randomly intensified, and each was targeted with a broadcasting of its pronunciation (Figure 4B). It should be noted that the sound coming from the side (left/right) of the earphone corresponded to the side of the column (left/right), ensuring spatial and semantic congruence, which can make some areas of the brain more activating, thus helping more accurate classification (Plank et al., 2012).

FIGURE 4

Figure 4. This is an example of a regional audiovisual paradigm for a P300 BCI speller. (A) The layout of level 1. (B) The layout of level 2, which corresponded to the region placed in the upper left corner of level 1. Modified from Pan et al. (2013).

The feature extraction of the speller is based on temporal and spatial features of the EEG data. For the temporal feature, the researchers chose the time window with differences between the target and non-target stimuli and with evident amplitudes evoked by target stimuli. The r² values were used to choose the channels. The calculation method is shown in formula (Wolpaw, 2007). In the process of classification, researchers used Bayesian linear discriminant analysis, an extension of Fisher’s linear discriminant analysis, to classify the data. In the offline experiment, fivefold cross-validation was adopted to calculate the individual accuracy.

$r^2={\left(\frac{\sqrt{N_1{N}_2}\left(mean\left(x_1\right)-mean\left(x_2\right)\right)}{\left(N_1+N_2\right)std\left(x_1\cup x_2\right)}\right)}^2(1)$

N₁ is the sample size of the target; N₂ is the sample size of the non-target; x₁ represents the feature vector of the target; x₂ and represents the feature vector of the target.

Researchers compared the paradigm with the unimodal visual speller and found that in some areas, the latency of P3a was significantly shorter and the amplitudes of P3b were higher in the audiovisual paradigm. However, the length of the time to ensure the pronunciation for each character and transition between the two levels might reduce the speed of character spelling.

In Oralhan (2019), Oralhan generated a two-stage audiovisual region-based P300 speller. In the first stage, the author divided the 30 characters into five groups and numbered each of them. The group characters intensified with the speakers playing its serial number. At the second stage, six characters in the detected region at the first stage are displayed separately, flashing with their pronunciation played by the speakers. In the process of signal classification, the researcher used SWLDA, which is a type of Fisher linear discriminant analysis (Krusienski et al., 2006; Hoffmann et al., 2008) that is widely used in P300 spellers due to its higher signal classification accuracy in P300-based BCI applications (Farwell and Donchin, 1988; Krusienski et al., 2006, 2008). In this research, the author compared the proposed speller’s performance with the audio-only and visual-only paradigms and found that the proposed one obtained the highest average accuracy of 90.31% and 78.06% and 54.08% of the visual-only and audio-only paradigms, respectively. In addition, the audiovisual speller reached an average ITR of 6.12 bits/second, which was higher than that of the other two spellers used in the experiments.

The audio stimuli of hybrid BCI speller systems cannot be limited to use as semantic clues. In Lu et al. (2019b), Lu et al. proposed an audiovisual region-based speller, which used smiling faces and audible chuckles corresponding to the smiling faces as visual and audio stimuli, respectively. This cross-modal system proved that the audio stimuli could compensate for the visual stimuli when users got tired of staring at the screen for long periods of time. In Jiang et al. (2019), Jiang et al. proposed an auditory-tactile P300 speller. Although the performance of the visual independent BCI system was worse than those depending on visualization, it was significant for patients with severe visual dysfunction. Table 3 lists the representative P300 spellers based on multisensory stimuli.

TABLE 3

Table 3. Summary of P300 spellers based on multisensory stimuli.

Multisensory stimuli can help support the loss of a single sensory modality and help BCI users concentrate (Belitski et al., 2011). In addition, the subjects of the comparative experiments reported that in multisensory stimulation, the spelling tasks were easier to comprehend than systems based on single stimuli (Oralhan, 2019). One problem was that sometimes multisensory stimuli had a longer duration than the unimodal stimuli. For example, the duration of an audiovisual stimuli should ensure the pronunciation integrity of each character or other voice prompts, which might increase the stimulus onset asynchrony, resulting in loss in ITR. Researchers should consider setting a proper length of stimulus onset asynchrony and weight between classification accuracy and the spelling speed in the design of P300 speller with multisensory.

P300 speller with multiple intelligent techniques

Recent studies have used 3-D interfaces and tested the functionality of the novel paradigms in P300-based BCI spellers. In Huang et al. (2019), Huang et al. replaced the 2-D interface with a 3-D picture and proved that the virtual reality system was more effective and suitable for usage. In Qu et al. (2018), Qu et al. used 3-D cubes as visual stimuli to evoke P300. Each cube flashed individually with a 3-D motion. The comparison experiment conducted in this research found that the proposed 3-D paradigm induced a higher P300 potential than the traditional 2-D P300 speller. In addition, subjects reported that they felt more comfortable for the use in 3-D paradigm, which could be explained by the attractive and realistic feature of the 3-D visual scene.

Noorzadeh et al. proposed the 3-D paradigm (Noorzadeh et al., 2020), as shown in Figure 5. The 3-D devices needed to consider how to display 3-D data in a separate image of each eye, so a virtual keyboard was specially set to protect the visual system. It intensified the rows, columns, and depths to cover the symbols, while the classical 2-D paradigm only flashed the rows and columns. The intensified letters would become bigger and the color turned out to be green in the 3-D paradigm to hint the subjects the intensifying row/column/depth. This study compared the classification accuracy and capacity between the 2-D page and the 3-D extended page of the classic P300 speller and verified them through marginal probability, which was based on the experiment of 16 volunteers as the dataset. However, experimental data showed that the classification accuracy of the 3-D interface was lower than that of the classical 2-D paradigm due to the low detection accuracy of the depth planes. This might be because when focusing on the target symbol, the subjects might be distracted by the non-target flashing depth layers. The 3-D paradigm also achieved a lower ITR than the classical 2-D paradigm.

FIGURE 5

Figure 5. A 3-D virtual paradigm proposed by Noorzadeh et al. (2020). (A) Flash on rows. (B) Flash on columns. The three depicted target flashes indicate character “F”. Modified from Noorzadeh et al. (2020).

In addition, the authors proposed another flashing approach, which considered each 3-D layer as separate 2-D planes, and all 2-D planes functioned in parallel, called parallel 2-D. The flashing approach of each plane followed the classical row/column paradigm. From the conclusion of the research, the parallel 2-D paradigm reached a similar classification accuracy and ITR as the classical one. The accuracies of the classical 2-D, 3-D, and parallel 2-D methods are 75.16%, 73.25%, and 74.65%, respectively, and the ITRs are 62.4, 58.35, and 62.25 bits/Flash, respectively. In addition, subjects considered the 3-D paradigm to be more comfortable than the classical 2-D paradigm. Seventy-five percent of the subjects preferred the 3-D paradigm for its comfort and ergonomic design in the survey carried out in the experiment.

Inspired by the ideas of the 3-D single flashing paradigm in Qu et al. (2018) and column-only flashing in Ramirez-Quintana et al. (2021), Korkmaz et al. (2022) proposed a 3-D column-only P300 speller in. The columns of the paradigm flashed in a pseudorandom order, and the rows were transposed and intensified also by column. The proposed paradigm not only performed better than the traditional 2-D paradigm but also reduced the workload of the users. In addition, the system used few EEG electrodes, which made it more suitable for practical implementations.

In Du et al. (2019), Du et al. designed three 3-D paradigms in virtual reality (VR) and tried to explore whether higher or similar responses of P300 potentials were induced compared with the 2-D paradigm. They found that stimuli presented with the 3-D effect in VR had better class discriminant than the 2-D paradigm.

Yao et al. (2018) presented an SSVEP speller and achieved good performance. To our knowledge, no complete P300 speller system in VR has been proposed until now, while VR has great potential to evoke higher ERP amplitudes in stimuli presentation. Table 4 lists the representative P300 spellers with multiple intelligent techniques. In future work, research on the P300 speller can focus on presenting the 3-D interface in a VR environment. In addition, researchers can try to address the issue of asynchronous control of these novel P300 spellers in their future work.

TABLE 4

Table 4. Summary of the P300 speller with multiple intelligent techniques.

Discussion

P300-based BCI spellers are one of the most traditional BCI applications. This article primarily discusses the progress of research into P300 BCI spellers. Several representative P300 spellers were reviewed, including their design principles, paradigms, algorithms, experimental performance, and corresponding advantages. Ever since its concept was first conceived 34 years ago, the P300 has evolved from what was the single-modal speller model designed for only one type of sensory organ into that which is predicated on multiple brain patterns and that featuring multi-sensory stimuli. At this point, researchers are focusing on the angle of multimodal integration to make strides toward optimizing the P300’s performance and user experience. In fact, the most recent iteration highlighting multiple intelligent techniques is the product of relevant optimization that emphasizes the 3D design paradigm and ergonomics.

Compared with other EEG signals such as SSVEP and MI, P300 has its own unique advantage. Many people can produce P300 signals through oddball stimuli, which have been proven to be the most suitable ERP signal. In addition, P300 is well suited for solving problems with many options due to its oddball inducing mechanism and is thus often used in spelling systems. The SSVEP-based paradigm tends to cause visual fatigue in healthy subjects and can even induce epilepsy-like EEG activity in some patient groups, although it seems to be superior to the P300-based paradigm in terms of accuracy and ITR (Allison et al., 2010). MI-based BCI systems need more training time, and many users find it difficult to finish MI tasks without training.

P300 spellers are quite popular due to the relatively stable ERPs and few training requirements among BCI spellers, but inevitably, there are some disadvantages of P300 spellers. First, the ITR of the P300 spellers is relatively low. This is mainly because the P300 spellers were originally designed for patients with severe paralysis. In this regard, the accuracy and reliability of the system is of paramount importance. Additionally, the ITR and accuracy are always trade-offs in the study of BCI applications. Second, the comfort of the P300 spellers needs to be further improved. Most of the existing P300 spellers are designed from a researcher’s point of view and thus lack humane thinking. Third, while using the P300 speller system, subjects are more likely to become visually fatigued with the extension of the usage time, which lowers the detection efficiency of the P300. Then, stimuli of the P300 spellers always require a combination of visual stimuli, and those studies on auditory and tactile stimuli are quite uncommon. As a result, patients with severe paralysis are unable to use the P300 spellers to communicate with others. To improve the above disadvantages, researchers have made many efforts around the system paradigm and algorithm design. For example, Jalilpour et al. (2020) proposed a hybrid paradigm of RSVP spellers and achieved higher classification accuracy and ITR in comparison with previous single-mode spellers. Oralhan (2019) developed a two-stage audiovisual P300 speller based on multisensory stimuli. However, there is still a long way from P300-based BCI to practical application. In addition to the abovementioned directions, researchers should pay more attention to application background and develop more practical systems.

The BCI speller is designed for patients losing their ability to communicate in a traditional way. A visual P300-based speller can be utilized for disables with visual ability. For patients with visual impairment, the multisensory speller may enable them to reclaim their social lives. Audio-visual spellers are suitable for those with deteriorating vision over time (Lu et al., 2019a). Although BCI speller systems have a lower typing rate than traditional typing methods, researchers hope that in the future, BCI spellers will become comparable to traditional tools and use in our daily life. It should be noted that most of the subjects participating in the research on P300 spellers are in good health, which may make the data obtained from the experimental results biased in the use of patients. Specifically, Huang et al. (2021) implemented an asynchronous P300 BCI (variants of the P300 speller) as a communication channel and three patients with disorder of consciousness achieve accuracies significantly higher than chance level. Therefore, to stand on the position of patients, we should invite more end users (e.g., MND patients) to participate in the experiment and consider more from the aspects of appearance, paradigm, algorithm, and application in the research on P300 speller.

Researchers are trying to look for methods to improve the performance of P300 spellers. One strategy is to optimize the design of the interface, which is the part that the users interact directly. This article mainly focuses on recent developments in the paradigm design of P300-based BCI spellers, which aim to provide a better user experience. At the same time, researchers are trying his best to improve the classification algorithms of the spellers. In Aloise et al. (2013), researchers combined a gaze-independent P300 algorithm with an asynchronous algorithm, but the results were not significantly different. Then, in Schettini et al. (2014), the team successfully added a module that can calibrate the parameters of the classifier automatically to the system so it could reach the optimal accuracy. In Bianchi et al. (2019), researchers proposed a method that can dynamically stop the stimulation process to reduce the number of iterations and improve the accuracy of information transmission. Furthermore, one recent study (Gao et al., 2021) provided a subject-independent P300 BCI using an invariant pattern learning method based on a convolutional neural networks and big EEG data that eliminated the calibration procedure and greatly increased the viability of BCIs.

Many studies on the classification algorithm have been carried out to improve the P300 speller performance in term of information transfer rate and accuracy. However, what users directly interact with is the paradigm design of the systems. The evolution of the system from the users is directly impacted by how comfortable and user-friendly it is. Thus, the friendliness, usability, and flexibility are important factors to assess whether people enjoy using the P300 spellers. In fact, questionnaires and other means have been employed to explore subjective factors in many studies (Pan et al., 2013; Li et al., 2019). Furthermore, it is also recommended that researchers pay more attention to the paradigm design of the P300 speller. For example, utilizing other sensory stimuli helps increase the variety of users. Although there are studies utilizing audio-only and tactile-only stimuli (Höhne et al., 2011; van der Waal et al., 2012; Chang et al., 2014), they have poor results with a quietly low ITR. Recent studies have demonstrated that combining visual stimuli with other types of stimuli can result in good performance (Belitski et al., 2011; Lu et al., 2019a, b). However, there are still some patients with vision issues or late-stage ALS; thus, it still makes sense to conduct research on BCI spellers based on audio and tactile stimuli. In addition, we noticed that different paradigms calculate the ITR and accuracy differently, which means that the comparison between them becomes meaningless. In future research, more scientific comparison experiments should be conducted to attain a more trustworthy and objective result.

Conclusion

Considerable progress has been made in improving the paradigm design of P300-based BCI spellers. The main aim of this article is to gather and describe information on some relatively successful P300-based BCI spellers, especially those published during this decade. Scientists have worked on BCI spellers to achieve higher accuracy and faster typing speed, although there are still many things to do to make spellers more usable and satisfactory. In addition, researchers should pay more attention to the paradigm design since it is the first factor when end-users judge spelling systems. BCI is a commutation system for patients losing their abilities to communicate in a traditional way. In the future, research should prioritize practicality and comfort and propose a more practical and user-friendly P300 speller system. The different systems discussed below may inspire further studies and improvements.

Author Contributions

JP: conceptualization, supervision, project administration, and funding acquisition. JP, XC, and HH: methodology and validation. XC, NB, and JH: software. XC, HH, and JC: formal analysis and visualization. JP, XC, and NB: investigation and writing—original draft preparation. JP, XC, NB, and HH: writing —review and editing. All authors contributed to the article and approved the submitted version.

Funding

This study was supported by the Key Realm R&D Program of Guangzhou under grant 202007030005, the National Natural Science Foundation of China under grant 62076103, and the Special Innovation Projects of General Universities in Guangdong Province under grant 2022KTSCX035.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Acqualagna, L., and Blankertz, B. (2013). Gaze-independent BCI-spelling using rapid serial visual presentation (RSVP). Clin. Neurophysiol. 124, 901–908. doi: 10.1016/j.clinph.2012.12.050

PubMed Abstract | CrossRef Full Text | Google Scholar

Akram, F., Han, S. M., and Kim, T.-S. (2015). An efficient word typing P300-BCI system using a modified T9 interface and random forest classifier. Comput. Biol. Med. 56, 30–36. doi: 10.1016/j.compbiomed.2014.10.021

PubMed Abstract | CrossRef Full Text | Google Scholar

Allison, B. Z., and Pineda, J. A. (2003). ERPs evoked by different matrix sizes: implications for a brain computer interface (BCI) system. IEEE Trans. Neural Syst. Rehabil. Eng. 11, 110–113. doi: 10.1109/TNSRE.2003.814448

PubMed Abstract | CrossRef Full Text | Google Scholar

Allison, B., Luth, T., Valbuena, D., Teymourian, A., Volosyak, I., and Graser, A. (2010). BCI demographics: how many (and what kinds of) people can use an SSVEP BCI. IEEE Trans. Neural Syst. Rehabil. Eng. 18, 107–116. doi: 10.1109/TNSRE.2009.2039495

PubMed Abstract | CrossRef Full Text | Google Scholar

Aloise, F., Aricò, P., Schettini, F., Salinari, S., Mattia, D., and Cincotti, F. (2013). Asynchronous gaze-independent event-related potential-based brain-computer interface. Artif. Intell. Med. 59, 61–69. doi: 10.1016/j.artmed.2013.07.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Aloise, F., Lasorsa, I., Schettini, F., Brouwer, A., Mattila, D., Babiloni, F., et al. (2007). Multimodal stimulation for a P300-based BCI. Int. J. Bioelectromagnetics 9, 128–130.

Google Scholar

Aygün, A. B., and Kavsaoğlu, A. R. (2022). An innovative P300 speller brain-computer interface design: easy screen. Biomed. Signal Process. Control 75:103593. doi: 10.1016/j.bspc.2022.103593

CrossRef Full Text | Google Scholar

Belitski, A., Farquhar, J., and Desain, P. (2011). P300 audio-visual speller. J. Neural Eng. 8:025022. doi: 10.1088/1741-2560/8/2/025022

PubMed Abstract | CrossRef Full Text | Google Scholar

Bianchi, L., Liti, C., and Piccialli, V. (2019). A new early stopping method for P300 spellers. IEEE Trans. Neural Syst. Rehabil. Eng. 27, 1635–1643. doi: 10.1109/TNSRE.2019.2924080

PubMed Abstract | CrossRef Full Text | Google Scholar

Brouwer, A.-M., and van Erp, J. B. F. (2010). A tactile P300 brain-computer interface. Front. Neurosci. 4:19. doi: 10.3389/fnins.2010.00019

PubMed Abstract | CrossRef Full Text | Google Scholar

Cecotti, H. (2011). Spelling with non-invasive brain-computer interfaces-current and future trends. J. Physiol. Paris 105, 106–114. doi: 10.1016/j.jphysparis.2011.08.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Chang, M., Mori, K., Makino, S., and Rutkowski, T. M. (2014). Spatial auditory two-step input Japanese syllabary brain-computer interface speller. Procedia Technol. 18, 25–31. doi: 10.1016/j.protcy.2014.11.007

CrossRef Full Text | Google Scholar

Corralejo, R., Nicolás-Alonso, L. F., Álvarez, D., and Hornero, R. (2014). A P300-based brain-computer interface aimed at operating electronic devices at home for severely disabled people. Med. Biol. Eng. Comput. 52, 861–872. doi: 10.1007/s11517-014-1191-5

PubMed Abstract | CrossRef Full Text | Google Scholar

De Vos, M., Kroesen, M., Emkes, R., and Debener, S. (2014). P300 speller BCI with a mobile EEG system: comparison to a traditional amplifier. J. Neural Eng. 11:036008. doi: 10.1088/1741-2560/11/3/036008

PubMed Abstract | CrossRef Full Text | Google Scholar

Devlaminck, D., Wyns, B., Grosse-Wentrup, M., Otte, G., and Santens, P. (2011). Multisubject learning for common spatial patterns in motor-imagery BCI. Comput. Intell. Neurosci. 2011:217987. doi: 10.1155/2011/217987

PubMed Abstract | CrossRef Full Text | Google Scholar

Du, J., Ke, Y., Kong, L., Wang, T., He, F., and Ming, D. (2019). “3D stimulus presentation of ERP-speller in virtual reality,” in 2019 9th International IEEE/EMBS Conference on Neural Engineering (NER) (San Francisco, CA, USA: IEEE), 167–170. doi: 10.1109/NER.2019.8717164

CrossRef Full Text | Google Scholar

Farwell, L. A., and Donchin, E. (1988). Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr. Clin. Neurophysiol. 70, 510–523. doi: 10.1016/0013-4694(88)90149-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Fazel-Rezai, R. (2007). “Human error in P300 speller paradigm for brain-computer interface,” in 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (Lyon, France: IEEE), 2516–2519. doi: 10.1109/IEMBS.2007.4352840

CrossRef Full Text | Google Scholar

Fazel-Rezai, R., and Abhari, K. (2009). A region-based P300 speller for brain-computer interface. Can. J. Electrical Comput. Eng. 34, 81–85. doi: 10.1109/CJECE.2009.5443854

CrossRef Full Text | Google Scholar

Gao, W., Yu, T., Yu, J.-G., Gu, Z., Li, K., Huang, Y., et al. (2021). Learning invariant patterns based on a convolutional neural network and big electroencephalography data for subject-independent P300 brain-computer interfaces. IEEE Trans. Neural Syst. Rehabil. Eng. 29, 1047–1057. doi: 10.1109/TNSRE.2021.3083548

PubMed Abstract | CrossRef Full Text | Google Scholar

Guan, C., Thulasidas, M., and Wu, J. (2004). “High performance P300 speller for brain-computer interface,” in IEEE International Workshop on Biomedical Circuits and Systems, 2004 (Singapore: IEEE), S3/5/INV–S3/13. doi: 10.1109/BIOCAS.2004.1454155

CrossRef Full Text | Google Scholar

Guo, J., Gao, S., and Hong, B. (2010). An auditory brain-computer interface using active mental response. IEEE Trans. Neural Syst. Rehabil. Eng. 18, 230–235. doi: 10.1109/TNSRE.2010.2047604

PubMed Abstract | CrossRef Full Text | Google Scholar

Höhne, J., Schreuder, M., Blankertz, B., and Tangermann, M. (2011). A novel 9-class auditory ERP paradigm driving a predictive text entry system. Front. Neurosci. 5:99. doi: 10.3389/fnins.2011.00099

PubMed Abstract | CrossRef Full Text | Google Scholar

Halder, S., Rea, M., Andreoni, R., Nijboer, F., Hammer, E. M., Kleih, S. C., et al. (2010). An auditory oddball brain-computer interface for binary choices. Clin. Neurophysiol. 121, 516–523. doi: 10.1016/j.clinph.2009.11.087

PubMed Abstract | CrossRef Full Text | Google Scholar

Hanagasi, H. A., Gurvit, I. H., Ermutlu, N., Kaptanoglu, G., Karamursel, S., Idrisoglu, H. A., et al. (2002). Cognitive impairment in amyotrophic lateral sclerosis: evidence from neuropsychological investigation and event-related potentials. Brain Res. Cogn. Brain Res. 14, 234–244. doi: 10.1016/s0926-6410(02)00110-6

PubMed Abstract | CrossRef Full Text | Google Scholar

He, B., and Liu, Z. (2008). Multimodal functional neuroimaging: integrating functional MRI and EEG/MEG. IEEE Rev. Biomed. Eng. 1, 23–40. doi: 10.1109/RBME.2008.2008233

PubMed Abstract | CrossRef Full Text | Google Scholar

Hochberg, L. R., Serruya, M. D., Friehs, G. M., Mukand, J. A., Saleh, M., Caplan, A. H., et al. (2006). Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 442, 164–171. doi: 10.1038/nature04970

PubMed Abstract | CrossRef Full Text | Google Scholar

Hoffmann, U., Vesin, J.-M., Ebrahimi, T., and Diserens, K. (2008). An efficient P300-based brain-computer interface for disabled subjects. J. Neurosci. Methods 167, 115–125. doi: 10.1016/j.jneumeth.2007.03.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Huang, J., Qiu, L., Lin, Q., Xiao, J., Huang, Y., Huang, H., et al. (2021). Hybrid asynchronous brain-computer interface for yes/no communication in patients with disorders of consciousness. J. Neural Eng. 18:056001. doi: 10.1088/1741-2552/abf00c

PubMed Abstract | CrossRef Full Text | Google Scholar

Huang, Y., Qu, J., Xiao, J., and Li, K. (2019). “A novel P300-based interactive method for virtual reality system,” in 2019 WRC Symposium on Advanced Robotics and Automation (WRC SARA) (Beijing, China: IEEE), 2019, 309–314. doi: 10.1109/WRC-SARA.2019.8931926

CrossRef Full Text | Google Scholar

Hwang, H.-J., Lim, J.-H., Jung, Y.-J., Choi, H., Lee, S. W., and Im, C.-H. (2012). Development of an SSVEP-based BCI spelling system adopting a QWERTY-style LED keyboard. J. Neurosci. Methods 208, 59–65. doi: 10.1016/j.jneumeth.2012.04.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Jalilpour, S., Sardouie, S. H., and Mijani, A. (2020). A novel hybrid BCI speller based on RSVP and SSVEP paradigm. Comput. Methods Programs Biomed. 187:105326. doi: 10.1016/j.cmpb.2020.105326

PubMed Abstract | CrossRef Full Text | Google Scholar

Jiang, J., Zhang, B., Yin, E., Wang, C., and Deng, B. (2019). “A novel auditory-tactile P300-based BCI paradigm,” in 2019 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications (CIVEMSA) (Tianjin, China: IEEE), 2019, 1–6. doi: 10.1109/CIVEMSA45640.2019.9071600

CrossRef Full Text | Google Scholar

Katyal, E., and Singla, R. (2021). EEG-based hybrid QWERTY mental speller with high information transfer rate. Med. Biol. Eng. Comput. 59, 633–661. doi: 10.1007/s11517-020-02310-w

PubMed Abstract | CrossRef Full Text | Google Scholar

Kaufmann, T., Schulz, S. M., Grünzinger, C., and Kübler, A. (2011). Flashing characters with famous faces improves ERP-based brain-computer interface performance. J. Neural Eng. 8:056016. doi: 10.1088/1741-2560/8/5/056016

PubMed Abstract | CrossRef Full Text | Google Scholar

Kennedy, P. R., and Bakay, R. A. (1998). Restoration of neural output from a paralyzed patient by a direct brain connection. Neuroreport 9, 1707–1711. doi: 10.1097/00001756-199806010-00007

PubMed Abstract | CrossRef Full Text | Google Scholar

Kevric, J., and Subasi, A. (2017). Comparison of signal decomposition methods in classification of EEG signals for motor-imagery BCI system. Biomed. Signal Process. Control 31, 398–406. doi: 10.1016/j.bspc.2016.09.007

CrossRef Full Text | Google Scholar

Kleih, S. C., and Kübler, A. (2013). Empathy, motivation and P300 BCI performance. Front. Hum. Neurosci. 7:642. doi: 10.3389/fnhum.2013.00642

PubMed Abstract | CrossRef Full Text | Google Scholar

Korkmaz, O. E., Aydemir, O., Oral, E. A., and Ozbek, I. Y. (2022). An efficient 3D column-only P300 speller paradigm utilizing few numbers of electrodes and flashings for practical BCI implementation. PLoS One 17:e0265904. doi: 10.1371/journal.pone.0265904

PubMed Abstract | CrossRef Full Text | Google Scholar

Krusienski, D. J., Sellers, E. W., Cabestaing, F., Bayoudh, S., McFarland, D. J., Vaughan, T. M., et al. (2006). A comparison of classification techniques for the P300 speller. J. Neural Eng. 3, 299–305. doi: 10.1088/1741-2560/3/4/007

PubMed Abstract | CrossRef Full Text | Google Scholar

Krusienski, D. J., Sellers, E. W., McFarland, D. J., Vaughan, T. M., and Wolpaw, J. R. (2008). Toward enhanced P300 speller performance. J. Neurosci. Methods 167, 15–21. doi: 10.1016/j.jneumeth.2007.07.017

PubMed Abstract | CrossRef Full Text | Google Scholar

Leeb, R., Lancelle, M., Kaiser, V., Fellner, D. W., and Pfurtscheller, G. (2013). Thinking penguin: multimodal brain-computer interface control of a VR game. IEEE Trans. Comput. Intell. AI Games 5, 117–128. doi: 10.1109/TCIAIG.2013.2242072

CrossRef Full Text | Google Scholar

Li, Q., Liu, S., Li, J., and Bai, O. (2015). Use of a green familiar faces paradigm improves P300-speller brain-computer interface performance. PLoS One 10:e0130325. doi: 10.1371/journal.pone.0130325

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, Q., Lu, Z., Gao, N., and Yang, J. (2019). Optimizing the performance of the visual P300-speller through active mental tasks based on color distinction and modulation of task difficulty. Front. Hum. Neurosci. 13:130. doi: 10.3389/fnhum.2019.00130

PubMed Abstract | CrossRef Full Text | Google Scholar

Lin, Z., Zhang, C., Zeng, Y., Tong, L., and Yan, B. (2018). A novel P300 BCI speller based on the Triple RSVP paradigm. Sci. Rep. 8:3350. doi: 10.1038/s41598-018-21717-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Lu, Z., Li, Q., Gao, N., and Yang, J. (2020). The self-face paradigm improves the performance of the P300-speller system. Front. Comput. Neurosci. 13:93. doi: 10.3389/fncom.2019.00093

PubMed Abstract | CrossRef Full Text | Google Scholar

Lu, Z., Li, Q., Gao, N., Yang, J., and Bai, O. (2019a). A novel audiovisual P300-speller paradigm based on cross-modal spatial and semantic congruence. Front. Neurosci. 13:1040. doi: 10.3389/fnins.2019.01040

PubMed Abstract | CrossRef Full Text | Google Scholar

Lu, Z., Li, Q., Gao, N., Yang, J., and Bai, O. (2019b). Happy emotion cognition of bimodal audiovisual stimuli optimizes the performance of the P300 speller. Brain Behav. 9:e01479. doi: 10.1002/brb3.1479

PubMed Abstract | CrossRef Full Text | Google Scholar

Martens, S., Hill, N., Farquhar, J., and Schölkopf, B. (2009). Overlap and refractory effects in a brain-computer interface speller based on the visual P300 event-related potential. J. Neural Eng. 6:026003. doi: 10.1088/1741-2560/6/2/026003

PubMed Abstract | CrossRef Full Text | Google Scholar

Millán, J. d. R., and Carmena, J. (2010). Invasive or noninvasive: understanding brain-machine interface technology. IEEE Eng. Med. Biol. Mag. 29, 16–22. doi: 10.2217/cer-2022-0146

PubMed Abstract | CrossRef Full Text | Google Scholar

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., and Group, P. (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 6:e1000097. doi: 10.1371/journal.pmed.1000097

PubMed Abstract | CrossRef Full Text | Google Scholar

Müller-Putz, G. R., and Pfurtscheller, G. (2008). Control of an electrical prosthesis with an SSVEP-based BCI. IEEE Trans. Biomed. Eng. 55, 361–364. doi: 10.1109/TBME.2007.897815

PubMed Abstract | CrossRef Full Text | Google Scholar

Noorzadeh, S., Rivet, B., and Jutten, C. (2020). 3-D Interface for the P300 speller BCI. IEEE Trans. Hum. Mach. Systems 50, 604–612. doi: 10.1109/THMS.2020.3016079

CrossRef Full Text | Google Scholar

Oralhan, Z. (2019). A new paradigm for region-based P300 speller in brain computer interface. IEEE Access 7, 106618–106627. doi: 10.1109/ACCESS.2019.2933049

CrossRef Full Text | Google Scholar

Pan, J., Li, Y., Gu, Z., and Yu, Z. (2013). A comparison study of two P300 speller paradigms for brain-computer interface. Cogn. Neurodyn. 7, 523–529. doi: 10.1007/s11571-013-9253-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Park, C., Looney, D., ur Rehman, N., Ahrabian, A., and Mandic, D. P. (2012). Classification of motor imagery BCI using multivariate empirical mode decomposition. IEEE Trans. Neural Syst. Rehabil. Eng. 21, 10–22. doi: 10.1109/TNSRE.2012.2229296

PubMed Abstract | CrossRef Full Text | Google Scholar

Park, H.-J., Kim, K.-T., and Lee, S.-W. (2015). “Towards a smart TV control system based on steady-state visual evoked potential,” in The 3rd International Winter Conference on Brain-Computer Interface (Gangwon, South Korea: IEEE), 1–2. doi: 10.1109/IWW-BCI.2015.7073039

CrossRef Full Text | Google Scholar

Philip, J. T., and George, S. T. (2020). Visual P300 mind-speller brain-computer interfaces: a walk through the recent developments with special focus on classification algorithms. Clin. EEG Neurosci. 51, 19–33. doi: 10.1177/1550059419842753

PubMed Abstract | CrossRef Full Text | Google Scholar

Plank, T., Rosengarth, K., Song, W., Ellermeier, W., and Greenlee, M. W. (2012). Neural correlates of audio-visual object recognition: effects of implicit spatial congruency. Hum. Brain Mapp. 33, 797–811. doi: 10.1002/hbm.21254

PubMed Abstract | CrossRef Full Text | Google Scholar

Qu, J., Wang, F., Xia, Z., Yu, T., Xiao, J., Yu, Z., et al. (2018). A novel three-dimensional P300 speller based on stereo visual stimuli. IEEE Trans. Hum. Mach. Syst. 48, 392–399. doi: 10.1109/THMS.2018.2799525

CrossRef Full Text | Google Scholar

Ramirez-Quintana, J. A., Madrid-Herrera, L., Chacon-Murguia, M. I., and Corral-Martinez, L. F. (2021). Brain-computer interface system based on p300 processing with convolutional neural network, novel speller and low number of electrodes. Cogn. Comput. 13, 108–124. doi: 10.1007/s12559-020-09744-2

CrossRef Full Text | Google Scholar

Rao, R. P. N. (2013). Brain-Computer Interfacing: an Introduction. New York, NY, USA: Cambridge University Press.

Google Scholar

Rezeika, A., Benda, M., Stawicki, P., Gembler, F., Saboor, A., and Volosyak, I. (2018). Brain-computer interface spellers: a review. Brain Sci. 8:57. doi: 10.3390/brainsci8040057

PubMed Abstract | CrossRef Full Text | Google Scholar

Rohani, D. A., and Puthusserypady, S. (2015). BCI inside a virtual reality classroom: a potential training tool for attention. EPJ Nonlin. Biomed. Physics 3:12. doi: 10.1140/epjnbp/s40366-015-0027-z

CrossRef Full Text | Google Scholar

Santamaría-Vázquez, E., Martínez-Cagigal, V., Gomez-Pilar, J., and Hornero, R. (2019). Asynchronous control of ERP-based BCI spellers using steady-state visual evoked potentials elicited by peripheral stimuli. IEEE Trans. Neural Syst. Rehabil. Eng. 27, 1883–1892. doi: 10.1109/TNSRE.2019.2934645

PubMed Abstract | CrossRef Full Text | Google Scholar

Schettini, F., Aloise, F., Aricò, P., Salinari, S., Mattia, D., and Cincotti, F. (2014). Self-calibration algorithm in an asynchronous P300-based brain-computer interface. J. Neural Eng. 11:035004. doi: 10.1088/1741-2560/11/3/035004

PubMed Abstract | CrossRef Full Text | Google Scholar

Sellers, E. W., and Donchin, E. (2006). A P300-based brain-computer interface: initial tests by ALS patients. Clin. Neurophysiol. 117, 538–548. doi: 10.1016/j.clinph.2005.06.027

PubMed Abstract | CrossRef Full Text | Google Scholar

Sellers, E. W., Krusienski, D. J., McFarland, D. J., Vaughan, T. M., and Wolpaw, J. R. (2006). A P300 event-related potential brain-computer interface (BCI): the effects of matrix size and inter stimulus interval on performance. Biol. Psychol. 73, 242–252. doi: 10.1016/j.biopsycho.2006.04.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Shi, J.-h., Shen, J.-z., Ji, Y., and Du, F.-l. (2012). A submatrix-based P300 brain-computer interface stimulus presentation paradigm. J. Zhejiang Univ. Sci. C 13, 452–459. doi: 10.1631/jzus.C1100328

CrossRef Full Text | Google Scholar

Sutton, S., Braren, M., Zubin, J., and John, E. (1965). Evoked-potential correlates of stimulus uncertainty. Science 150, 1187–1188. doi: 10.1126/science.150.3700.1187

PubMed Abstract | CrossRef Full Text | Google Scholar

Takano, K., Komatsu, T., Hata, N., Nakajima, Y., and Kansaku, K. (2009). Visual stimuli for the P300 brain-computer interface: a comparison of white/gray and green/blue flicker matrices. Clin. Neurophysiol. 120, 1562–1566. doi: 10.1016/j.clinph.2009.06.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Townsend, G., LaPallo, B. K., Boulay, C. B., Krusienski, D. J., Frye, G., Hauser, C., et al. (2010). A novel P300-based brain-computer interface stimulus presentation paradigm: moving beyond rows and columns. Clin. Neurophysiol. 121, 1109–1120. doi: 10.1016/j.clinph.2010.01.030

PubMed Abstract | CrossRef Full Text | Google Scholar

van der Waal, M., Severens, M., Geuze, J., and Desain, P. (2012). Introducing the tactile speller: an ERP-based brain-computer interface for communication. J. Neural Eng. 9:045002. doi: 10.1088/1741-2560/9/4/045002

PubMed Abstract | CrossRef Full Text | Google Scholar

Woldorff, M. G. (1993). Distortion of ERP averages due to overlap from temporally adjacent ERPs: analysis and correction. Psychophysiology 30, 98–119. doi: 10.1111/j.1469-8986.1993.tb03209.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Wolpaw, J. R. (2007). “Brain-computer interfaces (BCIs) for communication and control,” in Assets ’07: Proceedings of the 9th International ACM SIGACCESS Conference on Computers and Accessibility (New York, NY: Association for Computing Machinery), 1–2. doi: 10.1145/1296843.1296845

CrossRef Full Text | Google Scholar

Wolpaw, J. R., McFarland, D. J., Neat, G. W., and Forneris, C. A. (1991). An EEG-based brain-computer interface for cursor control. Electroencephalogr. Clin. Neurophysiol. 78, 252–259. doi: 10.1016/0013-4694(91)90040-b

PubMed Abstract | CrossRef Full Text | Google Scholar

Wolpaw, J., Birbaumer, N., Heetderks, W., McFarland, D., Peckham, P., Schalk, G., et al. (2000). Brain-computer interface technology: a review of the first international meeting. IEEE Trans. Rehabil. Eng. 8, 164–173. doi: 10.1109/tre.2000.847807

PubMed Abstract | CrossRef Full Text | Google Scholar

Wu, Z., Lai, Y., Xia, Y., Wu, D., and Yao, D. (2008). Stimulator selection in SSVEP-based BCI. Med. Eng. Physics 30, 1079–1088. doi: 10.1016/j.medengphy.2008.01.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Xu, L., Xu, M., Jung, T.-P., and Ming, D. (2021). Review of brain encoding and decoding mechanisms for EEG-based brain-computer interface. Cogn. Neurodyn. 15, 569–584. doi: 10.1007/s11571-021-09676-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Xu, H., Zhang, D., Ouyang, M., and Hong, B. (2013). Employing an active mental task to enhance the performance of auditory attention-based brain-computer interfaces. Clin. Neurophysiol. 124, 83–90. doi: 10.1016/j.clinph.2012.06.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Xu, M., Qi, H., Wan, B., Yin, T., Liu, Z., and Ming, D. (2013). A hybrid BCI speller paradigm combining P300 potential and the SSVEP blocking feature. J. Neural Eng. 10:026001. doi: 10.1088/1741-2560/10/2/026001

PubMed Abstract | CrossRef Full Text | Google Scholar

Xue, Y., Tang, J., He, F., Xu, M., and Qi, H. (2019). Improve P300 speller performance by changing stimulus onset asynchrony (SOA) without retraining the subject-independent model. IEEE Access 7, 134137–134144. doi: 10.1109/ACCESS.2019.2940593

CrossRef Full Text | Google Scholar

Yao, Z., Ma, X., Wang, Y., Zhang, X., Liu, M., Pei, W., et al. (2018). High-speed spelling in virtual reality with sequential hybrid BCIs. IEICE Trans. Info. Syst. 101, 2859–2862. doi: 10.1587/transinf.2018EDL8122

CrossRef Full Text | Google Scholar

Yin, E., Zhou, Z., Jiang, J., Chen, F., Liu, Y., and Hu, D. (2013). A novel hybrid BCI speller based on the incorporation of SSVEP into the P300 paradigm. J. Neural Eng. 10:026012. doi: 10.1088/1741-2560/10/2/026012

PubMed Abstract | CrossRef Full Text | Google Scholar

Yin, E., Zhou, Z., Jiang, J., Chen, F., Liu, Y., and Hu, D. (2014). A speedy hybrid BCI spelling approach combining P300 and SSVEP. IEEE Trans. Biomed. Eng. 61, 473–483. doi: 10.1109/TBME.2013.2281976

PubMed Abstract | CrossRef Full Text | Google Scholar

Yu, Y., Jiang, J., Zhou, Z., Yin, E., Liu, Y., Wang, J., et al. (2016). “A self-paced brain-computer interface speller by combining motor imagery and P300 potential,” in 2016 8th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC) (Hangzhou, China: IEEE), 160–163. doi: 10.1109/IHMSC.2016.80

CrossRef Full Text | Google Scholar

Zhang, X., Jin, J., Li, S., Wang, X., and Cichocki, A. (2021). Evaluation of color modulation in visual P300-speller using new stimulus patterns. Cogn. Neurodyn. 15, 873–886. doi: 10.1007/s11571-021-09669-y

PubMed Abstract | CrossRef Full Text | Google Scholar