Individual finger movement decoding using a novel ultra-high-density electroencephalography-based brain-computer interface system

Brain-Computer Interface (BCI) technology enables users to operate external devices without physical movement. Electroencephalography (EEG) based BCI systems are being actively studied due to their high temporal resolution, convenient usage, and portability. However, fewer studies have been conducted to investigate the impact of high spatial resolution of EEG on decoding precise body motions, such as finger movements, which are essential in activities of daily living. Low spatial sensor resolution, as found in common EEG systems, can be improved by omitting the conventional standard of EEG electrode distribution (the international 10–20 system) and ordinary mounting structures (e.g., flexible caps). In this study, we used newly proposed flexible electrode grids attached directly to the scalp, which provided ultra-high-density EEG (uHD EEG). We explored the performance of the novel system by decoding individual finger movements using a total of 256 channels distributed over the contralateral sensorimotor cortex. Dense distribution and small-sized electrodes result in an inter-electrode distance of 8.6 mm (uHD EEG), while that of conventional EEG is 60 to 65 mm on average. Five healthy subjects participated in the experiment, performed single finger extensions according to a visual cue, and received avatar feedback. This study exploits mu (8–12 Hz) and beta (13–25 Hz) band power features for classification and topography plots. 3D ERD/S activation plots for each frequency band were generated using the MNI-152 template head. A linear support vector machine (SVM) was used for pairwise finger classification. The topography plots showed regular and focal post-cue activation, especially in subjects with optimal signal quality. The average classification accuracy over subjects was 64.8 (6.3)%, with the middle versus ring finger resulting in the highest average accuracy of 70.6 (9.4)%. Further studies are required using the uHD EEG system with real-time feedback and motor imagery tasks to enhance classification performance and establish the basis for BCI finger movement control of external devices.

1 Introduction

Brain-Computer Interface (BCI) technology employs brain activity and the underlying neural information to control external equipment such as computers, wheelchairs, and prostheses without needing to physically move body parts (Belkacem et al., 2020). This technology can help people suffering from severe motor disabilities to communicate with the outside world and indirectly restore motor function (Wolpaw et al., 2002). The applications for complete or partially paralyzed patients, such as stroke patients (e.g., Hemiplegia), have been widely investigated for decades (Harkema et al., 2011; Bundy et al., 2017; Guger et al., 2017; Bai et al., 2020; Sebastián-Romagosa et al., 2020; Bhatia et al., 2021).

Rehabilitating motor functions of the hand, especially fingers, is essential for improving activities of daily living (ADLs) for people who experience upper limb motor impairment. Finger movements are required to manipulate tools, grab and move objects, and make signs, which are functions repeatedly used and indispensable (Landsmeer, 1962; Jones and Lederman, 2006; Mawase et al., 2020). Therefore, to better understand and rehabilitate the finger-related motor cortex, a considerable amount of research has been conducted (Kubánek et al., 2009; Acharya et al., 2010; Yanagisawa et al., 2011; Flamary and Rakotomamonjy, 2012; Liang and Bougrain, 2012; Hotson et al., 2016). For example, Acharya et al. (2010) and Kubánek et al. (2009) observed a focused brain activation region during hand grasp and thumb flexion, respectively. Hotson et al. (2016) and Gruenwald et al. (2019) classified individual finger tapping with accuracies above 95% using high-density electrocorticography (ECoG) grids.

These results are outstanding but cannot be generalized because most studies used invasive methods such as ECoG to acquire brain activity. Since ECoG acquires electroencephalographic signals directly from the cerebral cortex, it provides high spatio-temporal resolution and signal-to-noise ratio (SNR), producing excellent results in detecting single finger movements. However, invasive methods require clinical surgery to place the subdural electrode grids directly on the surface of the cortex.

This limitation demands using non-invasive methods, such as electroencephalography (EEG). EEG acquires electrophysiological signals from the scalp. Since EEG does not require clinical surgery and offers a high temporal resolution with low price and high portability, it is the most widely used among various brain signal acquisition methods (Mason et al., 2007). Moreover, these characteristics empower EEG-based BCI systems to be used as real-time closed-loop applications, the final goal of BCI application systems.

However, EEG systems have several limitations. It has low spatial resolution caused by common EEG electrodes’ large layout and size and its susceptibility to artifacts that often originate from movement and electromyographic (EMG) activity (Al-Fahoum and Al-Fraihat, 2014). Due to its limitations, the current state-of-the-art EEG systems only allow the distinction of gross movements. For example, research was done to discriminate hand grasp and release (Lange et al., 2016), right and left-hand movements (Pfurtscheller et al., 1997), upper-limb movements (Ofner et al., 2017), and lower-limb movements (Mejia Tobar et al., 2018).

Decoding more precise movements, such as individual finger movements, is highly challenging for conventional EEG systems. Quandt et al. (2012), Stankevich et al. (2016), and Bera et al. (2019) decoded individual finger movements using frequency bands under 40 Hz. They resulted in an average classification accuracy of 60, 50, and 43%, where the chance level was 33, 25, and 25%, respectively. Liao et al. (2014) classified finger pairs and reached 59 and 77% accuracy using the mu frequency band (8–12 Hz) and broadband features, respectively.

We assume a higher spatial resolution can improve the performance of EEG on precise tasks. Since denser electrodes can increase the spatial resolution, the amount and density of EEG electrodes have historically been steadily increasing. The first standard placement of EEG electrodes was proposed by Jasper (1958) with the definition of the 10–20 electrode system using 21 electrode locations over the whole scalp (see Figure 1A, dark gray circles). In 1985, the extended version of the 10–20 system, the 10–10 system, was proposed using 74 electrode locations (see Figure 1A, light gray circles), which is currently accepted as the standard by the American Electroencephalographic Society (Chatrian et al., 1985; Sharbrough, 1991). Based on the 10–10 system, another extended version, the extended 10–10 system, was proposed using 128 electrodes [(Oostenveld and Praamstra, 2001); see Figure 1A, gray non-filled circles]. Several high-density EEG systems are available commercially, primarily using from 160 to 256 channels. However, some devices contain electrodes placed on the cheeks and neck that record less motor-related brain activity. Therefore, more electrodes do not necessarily result in high motor classification performance.

FIGURE 1

Figure 1. (A) Comparison of the 10–20, 10–10, extended 10–10, and ultra-high-density electroencephalography (uHD EEG) systems–black circles with labels illustrate the 10–20 system; gray filled circles with electrode names show the positions of the original 10–10 system; gray non-filled circles indicate the extended positions of the 10–10 system; the entirety of the small black circles show the 1,024 uHD EEG system. The green-colored area indicates the electrode positions used in this study (256 electrodes out of 1,024 electrodes). (B) 3D model of the human scalp with the marked electrode positions. Cz is pointed out with an arrow. Each sphere corresponds to a single electrode, where black-colored electrodes indicate the region of interest for the classification.

Can’t the number of electrodes further increase? Yes, but only by breaking the convention that the electrode position should follow the international 10–20 system. Several studies have demonstrated that narrowing down the inter-electrode distance under 2 cm–commonly and widely used inter-electrode distance–is beneficial since it provides additional neural information (Freeman et al., 2003; Odabaee et al., 2013; Petrov et al., 2014; Robinson et al., 2017). For this reason, some recent studies explored denser electrode layouts in visual and auditory contexts (Robinson et al., 2017; Chamanzar et al., 2021; Narayanan et al., 2021). Here, we define a novel array formatted dense electrode layout (see small black circles in Figure 1A) as ultra-high-density EEG (uHD EEG). Theoretically, 1,024 electrodes can be placed over the whole scalp following the electrode size and position of this layout.

In order to achieve high SNR, active and wet electrodes are preferred over passive and dry electrodes. Active and wet electrodes use pre-amplifiers to amplify the acquired signal and conductive gel to bridge the scalp and electrodes (Mathewson et al., 2017). However, if the uHD EEG system consists of wet electrodes and is placed with a cap or a similar stretchable structure, as is the case for conventional EEG, the conductive gel of adjacent electrodes may touch due to its proximity (Rashid et al., 2020). This problem can be solved by changing the attachment method from a cap type to a medical adhesive. The conductive gel can be separated into holes using medical adhesives that have resections at each electrode site, consequently providing consistent low impedance and electrode separation.

In this study, we used a newly proposed uHD EEG, g.Pangolin (g.tec Medical Engineering GmbH, Schiedlberg, UA, Austria), and explored the performance by decoding individual finger movements of one hand. To the best of our knowledge, no experimental study using such uHD EEG in motor tasks has been reported (while visual and auditory tasks have been explored). The electrode grids were directly attached to the scalp using medical adhesives. Five subjects participated in the experiment and extended individual fingers according to a visual cue. Collected data were preprocessed, and features were extracted for the mu (8–12 Hz) and beta (13–25 Hz) frequency bands. In order to assess the performance of the system, topography plots were generated for the band power features, and a linear support vector machine (SVM) classifier was used for pairwise finger classification.

2 Materials and methods

2.1 Subjects

Five healthy subjects [S1–S5, one female and four males, age: 28 (7) years in mean (SD)] participated in the experiment. Subjects S2–S5 were right-handed, whereas subject S1 was left-handed. The participants did not receive any prior training regarding this experiment. The study was approved by the Institutional Review Board of Korea Advanced Institute of Science and Technology (KH2018-127). All participants gave written consent forms about recruitment prior to data collection.

2.2 Experimental protocol

Prior to the experiments, the subjects’ heads were shaved and cleaned with medical alcohol to attach uHD EEG grids directly to the scalp. As in Figures 2A,B, subjects sat on a comfortable armchair with their dominant hand placed on the table and their palm facing down. Figure 2C shows the experimental protocol, with the whole experiment consisting of 10 sequential runs. Each run lasted roughly 5 min, and the subject could take a break between runs. A single run started with a 30 s baseline period, followed by alternating rest and extension periods. During the rest period, the left picture of Figure 2D was shown on the monitor, and the subject could relax, swallow, or blink their eyes. The rest period lasted randomly between 3 and 4 s to prevent the subjects from adapting to the protocol. During the 5 s extension (i.e., task) period, the subject extended the finger according to the instruction cue and maintained it without moving any other part of their body. The instruction cue reflected a hand for which a single finger was colored and displayed in the monitor’s top-center (see Figure 2D right picture, middle finger extension). All fingers were pseudo-randomly cued five times in each run, resulting in 25 finger extensions per run. The hand avatar provided ideal (i.e., non-online feedback) visual feedback by imitating the finger rest and extension movements (see Figure 2D). Thus, the visual feedback can be thought of as an instruction that subjects should follow.

FIGURE 2

Figure 2. (A,B) Illustration and picture of overall system setup. The ultra-high-density electroencephalography (EEG) system is placed over the contralateral sensorimotor cortex of the subject’s scalp and connected via pre-amplifiers and HDMI cables to an interface box. Further, the 256-channel amplifier g.HIamp (g.tec Medical Engineering GmbH, Schiedlberg, UA, Austria) is connected through USB to the acquisition computer. A camera recorded the whole experiment. (C) The subject extended one of the five fingers and rested according to the experimental paradigm. A single run consisted of 25 trials with five trials per finger. 10 runs were conducted for each subject. (D) According to the paradigm, the monitor showed instruction cues and the hands of an avatar. The instruction was provided at the top center of the monitor with the colored finger the subject should extend. The right picture shows an example of when the middle finger should be extended. During the rest period, no finger was colored, and the hand was grayed out as shown in the left picture. As ideal visual feedback, the hand avatar simultaneously performed the finger extension as the subject should do.

2.3 Ultra-high-density electroencephalography system

Brain activity was measured using a novel uHD EEG system. The g.Pangolin system can be considered uHD due to its spatial resolution. A single electrode grid consists of 16 electrode points with a diameter of 5.9 mm and a center-to-center spacing of 8.6 mm. In this study, 16 electrode grids (256 electrodes) were attached over the contralateral sensorimotor cortex of the subject’s scalp (see Figure 1A, the green-colored area). Instead of using the 1,024 electrodes, only using electrodes over the contralateral sensorimotor cortex would improve the performance of motor control when channel selection algorithms or feature weights are used together (Arvaneh et al., 2011; López-Larraz et al., 2014). Figure 1B shows the electrode position on the subject’s scalp. The black-colored electrodes (158 electrodes) reflect the region of interest (ROI), while gray-colored electrodes were excluded from the classification step for this motor movement task. We removed the most anterior, posterior, central, and temporal electrodes to reduce the number of features during classification while retaining electrodes over the subjects’ sensorimotor hand area.

The electrode grids were attached sequentially in the downward direction laterally of the head, starting with the first electrode grid at the Cz position. Since the electrode grid is diamond-shaped and attached to share each other’s side, the sensor locations were pre-defined only with the starting point and the direction. The ground electrode from the uHD EEG system was placed on the left mastoid for all experiments.

Figures 2A,B show the system setup and data acquisition procedure for the conducted experiments. The electrode grid was attached to the skin using medical adhesives that insulated the electrode sites. The resection in the adhesive was filled with conductive paste, Elefix (Nihon Kohden Corporation, Japan), to ensure optimal skin contact and low impedance at the electrode-skin junction. For better signal quality and higher SNR, a pre-amplifier is used. Through this stage, the raw EEG recorded from the scalp is amplified with a gain of 10. An interface box connects the high-resolution electrode grids with the pre-amplifier and the biosignal amplifier. The pre-amplifier is connected to the interface box using a micro-HDMI to HDMI cable. The connection to the g.HIamp is established with a 65-pin interconnect cable with LEMO connectors to transmit 64 channels and Ground (GND). The interface box allows 256 channels to be recorded with the g.HIamp biosignal amplifier using 4 LEMO cables.

Data acquisition and paradigm presentation were performed with g.HIsys Professional (g.tec Medical Engineering GmbH, Schiedlberg, UA, Austria) running under MATLAB/Simulink (The MathWorks, Inc., Natick, MA, USA). Both the visual cue and avatar were presented using Unity (Unity Technologies, San Francisco, CA, USA). The signals were recorded with a sampling frequency of 600 Hz, and the cue information was synchronously recorded with the EEG data in real-time.

2.4 Data processing

The EEG processing pipeline consisted of Preprocessing, Feature Extraction, Epoching, Topography plots, and Classification of the data and was implemented in MATLAB (see Figure 3). MATLAB commands are provided to improve reproducibility and are formatted in italic. The EEG processing pipeline is quasi causal (except for the bad channel identification and moving average), deterministic and computation cost was kept low as the pipeline is intended to be used in real-time and online BCI experiments. Note that both bad channel identification and moving average could however be implemented in a causal manner. The recorded EEG is unlikely to be affected by ocular (e.g., eye movements and eye blink) and movement artifacts as the electrodes did not cover the frontal lobe, and subjects only moved their fingers during the tasks.

FIGURE 3

Figure 3. Electroencephalography (EEG) processing pipeline consisting of preprocessing, feature extraction, epoching, topography plots, and classification. “Identification of Bad Channels and Runs” returns only the information which channel(s) and run(s) are bad (dashed arrow) while leaving the EEG data (solid arrows) unchanged. The data were visualized after each processing step for an example dataset with the dashed line indicating the start of the first trial. n_ch, n_f, and n_t are the number of channels, fingers, and trials, respectively.

2.4.1 Preprocessing

The raw EEG recordings were first down-sampled from 600 to 200 Hz (resample) and notch-filtered at 60 Hz and its harmonics using a notch-filter cascade. 4th-order Butterworth filters were utilized, using butter to calculate filter coefficients and apply the filter coefficients to the signal. Next, bad channels and runs were identified based on EEG data during the task period. Note that this step only provides the identified channels and runs but leaves the previously notch-filtered EEG data unchanged (see Figure 3). First, data were common average referenced and filtered using a band-pass filter from 8 to 25 Hz. Second, for each channel, the mean power was computed by subtracting their respective mean, calculating the power by the squaring of each time sample, and calculating the mean power over time. Third, the mean channel powers were log-transformed to improve Gaussianity and z-transformed. Finally, channels with a z-score greater than 6 were considered bad, and runs in which more than 10% of channels are bad were removed. Furthermore, a disjunction was applied to bad channels over the remaining runs (i.e., if a channel is considered bad in one run, it is considered bad in all runs). Bad channels found using this procedure were removed from the notch-filtered EEG data, which were finally common average referenced.

2.4.2 Feature extraction and epoching

The features extracted are band power features for the mu and beta bands, as these frequency bands are associated with motor functions (Miller et al., 2010; Cheyne, 2013). The EEG data were band-pass filtered for the respective frequency band, and the power was calculated by squaring each time sample. Then the power was estimated in non-overlapping 0.25 s segments by averaging the power samples and applying a centered moving average with a 0.75 s window length. A centered moving average was used instead of a causal moving average in order to make the interpretation of results easier, as the latter would introduce time delays. Again, band power features were log-transformed to improve Gaussianity because non-log-transformed band power features are Chi-squared distributed. Finally, a power shift compensation was applied, subtracting the mean band power over the last 25 s. This is important as the band power of any channel can have slow drifts over time which will bias the classifier (see Supplementary material–Supplementary Figure 2 and also investigated by Benwell et al. (2019). A window length of 25 s was chosen as it comprises multiple trials while still staying responsive. Specifically, choosing the window length too short would lead to removing task-related information. Finally, the band power features were epoched using 0.5 s pre-and 7 s post-cue.

High-gamma activity (70–170 Hz) was also investigated (see Supplementary material–Supplementary Figure 1 and Supplementary Tables 1, 2) as these features showed positive results in classifying individual finger movements (Liao et al., 2014). However, we did not find high-gamma activity to have any added value for visualization of activity patterns or classification.

2.4.3 Topography plots

A 3D model of the MNI-152 template brain (Montreal Neurological Institute, Canada) was employed for visualization of the activation plots generalized over all subjects. 16 uHD EEG electrode grids were placed on the MNI-152 head using a custom montage creator software (g.tec Medical Engineering GmbH, Schiedlberg, UA, Austria). The resulting montage contained 256 electrodes located on the left hemisphere.

All subjects except subject S1 are right-handed. Therefore, the left hemisphere was selected for plotting the topographies over all subjects. In the case of subject S1, the sensor locations are redistributed by multiplying the x-axis coordinates by −1, which results in the same electrode layout mirrored on the sagittal plane having consistent plots.

The relative change in band power, commonly known as event-related desynchronization/synchronization [ERD/S; Pfurtscheller and Lopes da Silva (1999)], was calculated as:

$\text{ERDS}\left[\text{n}\right]=\log \left(p\left[n\right]\right)-\frac{1}{\left|{\mathrm{𝒮}}_{REF}\right|}\sum _{k\in {\mathrm{𝒮}}_{REF}}\log \left(p\left[k\right]\right)$

where p are the band power features and 𝒮_REF contains the time-discrete band power features within the reference period ranging from 0.5 to 0 s before the cue. While this may seem like a short reference period, note that a centered moving average was used in the feature extraction (see Figure 3). Therefore, each band power sample is related to the previous two-band power samples. Finally, mu and beta ERD/S during the average trial were illustrated in the topographies and reflect changes in dB.

Heatmaps based on ERD/S activation were generated according to Kubanek and Schalk (2015). For each electrode, the surface vertices closer than the Euclidean distance to the adjacent electrode were determined. An activation value was assigned to each vertex by convolving the activation of each electrode with a linear decay kernel. The kernel was chosen to reach zero at the Euclidean distance to the closest electrode. Finally, the activation values assigned to each vertex were summed up, and the vertices were colored accordingly. Bad channels identified during the preprocessing (see sections “2.4.1” and “2.4.2”) are excluded from the heatmap interpolation. Depending on the number of bad channels, the process of channel removal can result in irregular heatmaps with vacant sensor locations.

Spatial ERD/S dynamics were explored to investigate the spatial capabilities of the uHD EEG system. Focal activation plots from each finger extensions were created using the electrodes with the greatest ERD. Electrodes with the greatest ERD were selected by thresholding the mean ERD/S during seconds 1–2 post cue of the average trial. The threshold was set at the 2.5th percentile of all electrodes for the respective finger extension. In other words, electrodes which show greater ERD than 97.5% of all electrodes were selected. This selection results in ∼6 electrodes per finger extension.

Time-frequency maps were generated to investigate the temporal ERD/S dynamics. First, the power was estimated using the short-time Fourier transform (spectrogram) from 8 to 30 Hz at 1 Hz steps employing a 1 s Hamming window with 0.95 s overlap, resulting in a sampling frequency of 20 Hz. Second, the power samples were log-transformed and epoched using 1 s pre-and 7 s post-cue. ERD/S was computed with reference period 1 to 0 s before the cue. Finally, the ERD/S of the average trial was interpolated by a factor of 10 in time and frequency (ndgrid, griddedInterpolant) to improve image quality.

2.4.4 Classification

Classification models were employed to investigate if the extracted band power features have predictive power. Specifically, band power features during the task period (i.e., 0 to 5 s) for electrodes contained in the ROI were classified at each time step. Each subject’s finger movements were pairwise classified using a linear SVM, resulting in 5(5–1)/2 = 10 two-class classification problems. The reasons for choosing a simple linear SVM (fitcsvm) are two-fold. It was chosen instead of more complex models as it is suitable for online BCI experiments due to its relatively short training times (Lotte et al., 2007, 2018; Rashid et al., 2020). Secondly, it is not vulnerable to numerical instabilities when the number of features exceeds the number of training data points (i.e., instances, trials, observations), compared to a linear discriminant analysis (LDA) which would need strong regularization. Similarly, common spatial patterns (CSP) and filter bank CSP (FBCSP) were not used as the optimal size and position of the CSP window would need to be optimized for using a cross-validation (CV) procedure. Furthermore, due to the high number of electrodes regularized CSP (rCSP) is likely necessary, adding additional optimization parameters (Lu et al., 2009; Lotte and Guan, 2011). Nonetheless, we are not implying that the approach chosen in this study is necessarily superior to CSP based approaches.

For the SVM, the kernel scale of a linear kernel was automatically selected by MATLAB using a heuristic approach, features were standardized and a box constraint of 1 was used. This box constraint is used for regularization and helps to prevent overfitting. Notably, the results reported here are based on SVM models without hyperparameter optimization to keep training times to a minimum. While not reported here, results obtained using hyperparameter optimization of kernel scale and box constraint were similar, which is in line with findings by Probst et al. (2019).

An n-times of k-fold CV framework was used to estimate the predictive power of the band power features, with n and k both being 10. In other words, all trials of all runs (i.e., recordings) were used in a 10-fold CV, which was performed 10 times leadings to different CV assignments in each of the 10 iterations and thus a more reliable estimation of the performance. The random seed (rng) was set to the respective iteration (i.e., 1 to 10) to make cross-folds, and thus results reproducible. 10-fold CV was used instead of a leave-one-out CV, because the latter provides unstable and biased model performance estimates (Varoquaux et al., 2017). Ten-times of 10-fold CV were used as 10 iterations already provide reliable estimates of model performance (Bouckaert and Frank, 2004). The model performance was defined as the mean classification accuracy over these 10 iterations, at the time point with highest mean classification accuracy. Note that one may also perform a leave-one-run-out CV in which one trains the model using all but one run and then uses the remaining run as the test data. The model performances obtained using this framework are slightly better than the ones obtained using 10-times of 10-fold CV and can be found in the Supplementary material (see Supplementary Table 3).

Empirical P-values were calculated to quantify the chance of the observed model performance being obtained by chance. These P-values were obtained using a permutation testing approach in which the dependence between the features and the target variable (i.e., class) is broken (i.e., null model). In these null models, the target variable is permuted (i.e., shuffled), and then the 10-times of 10-fold CV is carried out to estimate the null model performance. This was done 100 times, resulting in 100 model performances obtained under the null hypothesis of independence of features and the target variable. As these null model performances are normally distributed, a normal distribution was fitted to these data, and the P-value for obtaining a performance measure as extreme or more extreme than the observed one was computed. This approach results in similar P-values as the approach used by Ojala and Garriga (2009). They calculated the fraction of null models with equal or greater model performance than the observed one, except that P-values smaller than 0.0099 can be obtained.

Empirical P-values were calculated for the time point which showed the greatest classification accuracy in the observed model and for each of the 10 classification problems and the respective subject. Model performance was considered to be better than the chance for P < 0.05. The P-values computed and reported were corrected for multiplicity (i.e., multiple hypothesis testing) per subject using the Benjamini-Hochberg [i.e., false discovery rate (FDR)] procedure (Benjamini and Hochberg, 1995).

In order to investigate if the additional electrodes introduced by the uHD EEG system increase classification accuracy, electrode subsets reflecting the 10–10 and extended 10–10 system were selected. The electrode subsets were generated by selecting individual uHD electrodes which best fit the respective 10–10 and extended 10–10 system positions (see Figure 1A and Supplementary Figure 3). For these two electrode subsets the band power features were computed (see sections “2.4.1” and “2.4.2”) and their predictive power was estimated. Finally, the 50 model performances (5 subjects with their 10 pairwise classifications) for the uHD, 10–10 and extended 10–10 system were compared using a two-tailed Wilcoxon signed-rank test.

2.4.5 Correlation between neighboring electrodes

A correlation analysis was performed to investigate to which extent neighboring electrodes provide identical information in the case of uHD EEG system, as well as the electrode subsets reflecting the 10–10 and extended 10–10 system. For all, but electrodes at the edge of the montage, neighboring electrodes were identified as the four-nearest neighbors according to their center-to-center Euclidean distance (see Figures 1A, 2A). Edge electrodes only served as neighbors as they are not enclosed by four-nearest neighbors. Note that only unique electrode pairs were allowed (e.g., electrode e_i is electrode $e_j^{\prime }s$ nearest neighbor and vice versa, resulting in one unique electrode pair). EEG data provided by the preprocessing step (see Figure 3) were band pass filtered between 1 and 30 Hz to contain the frequency components typically used in EEG experiments. Finally, the center-to-center Euclidean distance, Pearson’s linear correlation coefficient and coefficient of determination (R²) were calculated for the neighboring electrodes. A frequency band specific analysis can be found in Supplementary Tables 6, 7 in the Supplementary material.

3 Results

Figure 4 shows the remaining channels and runs after the preprocessing step. Data quality was good, with just five channels removed for subjects S1, S2, and S5. For subjects S3 and S4, 23 and 24 channels were removed. For subject S3, two runs were rejected, whereas none were rejected for the other subjects (S1, S2, S4, and S5).

FIGURE 4

Figure 4. Number of channels and runs used for each subject. Other channels and runs were excluded because of artifacts in the data. Black dots indicate good channels. Red dots indicate channels that were rejected.

3.1 Topography plots

Event-related desynchronization/synchronization (ERD/S) activation is shown in Figures 5, 6. Every single electrode of the total 256 electrode locations was used to plot the corresponding activation value and interpolated on the scalp of the 3D head model to generate the heatmaps. Bad channels (see Figure 4) were set to 0 for heatmap interpolation.

FIGURE 5

Figure 5. Topography event-related desynchronization/synchronization (ERD/S) plots calculated using the beta frequency band (13–25 Hz) for individual finger movements (rows) from the respective subjects (columns). The selected time point for the calculation is noted in brackets adjoining the subject name. The range of ERD/S was individually set per subject, which is indicated by the range of the color bar on the bottom of each subject’s column and reflects dB.

FIGURE 6

Figure 6. Topography event-related desynchronization/synchronization (ERD/S) plots and time courses of accuracy from subject S2. The subject received the instruction to extend a single finger at cue (0 s), performed the extension, and maintained it until 5 s post-cue. The topography plots were generated using the mu (8–12 Hz) and beta (13–25 Hz) frequency bands with all finger trials averaged. Values reflect dB. The accuracy time courses were obtained using linear support vector machine (SVM) for mu and beta band power features in mean and SD. The classification accuracy reaches a maximum of about 80% at 1.5 s. Accuracy_mean is the classification accuracy, whereas Accuracy₁ and Accuracy₂ are the accuracies of the respective classes. The most significant accuracy graph was shown here and compared with topography plots.

For Figure 5, the time point selected to plot the ERD/S activation was chosen individually for each subject with either 1.0 s (S1 and S4) or 1.5 s (S2, S3, and S5) post-cue. Finally, the average activity from all trials of the selected time point was plotted for each finger and subject, respectively. For subject S1, we found a clear spot around C3 for the thumb, index, and middle finger extension covering ∼30 electrodes. The activation was smaller for the ring and little finger movements. Subject S2 showed a focal activation spot for all fingers, whereas sparse activation was observed for subjects S3 and S4. Subject S5 had a clear spot for the thumb, index, middle, and ring finger, whereas the little finger showed less activation (see Figure 5).

Figure 6 shows the topography plots obtained from the ERD/S calculation using the mu and beta frequency band features from subject S2 in the time window from 0.5 s pre-cue to 7 s post-cue. A total of 16 subplots are generated with 0.5 s time steps for each frequency band. The mu ERD occurs from 0.5 to 2.5 s and is followed by an ERS from 3.5 to 5 s. The beta ERD occurs from 0.5 to 5.0 s and terminates with an ERS from 6 to 7 s.

Figure 7 shows spatiotemporal ERD/S dynamics of subject S2. The bubble plot shows the ERD/S for all finger extensions (see Figure 7A). As the ∼30 selected electrodes showed overlap across fingers a one vs. all approach was employed, resulting in 18 electrodes being color-coded according to the finger with the greatest ERD. The radius of the bubbles reflects the ERD magnitude. In order to superimpose the overlapping fingers, the bubbles were replaced by hand schematics representing whether the electrode exceeds the threshold (see Figure 7B). Figure 7C shows the temporal ERD/S dynamics for all finger extensions. The time-frequency maps reflect the ERD/S for the electrode which showed the greatest ERD (i.e., within the top 2.5% greatest ERD across all electrodes) from all five finger extensions.

FIGURE 7

Figure 7. (A) Event-related desynchronization/synchronization (ERD/S) bubble plot for all finger movements of subject S2 based on beta (13–25 Hz) frequency band features. Each finger represents a color: thumb (red), index (orange), middle (green), ring (turquoise), and little (blue). The bubbles are scaled according to the mean ERD/S calculated from 1 to 2 s post cue for the average trial in dB. For each finger the ∼6 channels with the greatest ERD were selected (see section “2.4.3” for details). For each electrode position, bubbles were color-coded according to the finger with the greatest ERD. The radius of the bubbles reflects the ERD magnitudes. (B) Bubbles from panel (A) were replaced with hand schematics. Fingers of each schematic were colored if the electrode was part of the subset which showed the greatest ERD of the respective finger. For electrode No. 125 (see arrow) this was the case for all fingers. (C) Time-frequency maps showing the average trial for electrode No. 125 and all finger extensions from subject S2.

3.2 Classification

Table 1 shows the maximum classification accuracies obtained using the linear SVM model and mu and beta band power features. All classification pairs for subject S1 performed above chance (P < 0.05). For subjects S2, S3, and S4, 80% of the classification pairs performed above chance level. Finally, for subject S5, 70% of the classification pairs performed above chance level. The highest classification accuracy for subject S1 was 85.4% for middle vs. ring finger. Subject S2 reached 79.2% for index vs. ring finger. Subject S3 achieved 65.0% for thumb vs. middle. Subject S4 reached 71.5% for middle vs. ring, and subject S5 had 66.2% for thumb vs. ring. Finger pairs which include the ring finger generally showed greater classification accuracies than the others. Tasks classifying middle vs. ring finger performed the best overall at 70.6 (9.4)% in mean (SD).

TABLE 1

Table 1. Model performance (classification accuracy) obtained using a linear support vector machine (SVM) in a 10-times of 10-fold CV framework for mu (8–12 Hz) and beta (13–25 Hz) band power features.

The classification accuracy decreased significantly when using the 10–10 and extended 10–10 system electrode subsets compared to the uHD EEG system (P < 0.0001, respectively). Detailed classification results are provided in the Supplementary material, where Supplementary Table 4 shows the results using the 10–10 system subset, and Supplementary Table 5 shows the model performance using the extended 10–10 system. Specifically, using the 10–10 and extended 10–10 system electrode subsets resulted in a classification accuracy decrease of 6.9 [4.6; 10.0]% and 5.9 [0.6; 9.7]% in median [IQR], respectively over all subjects and pairwise classifications.

3.3 Correlation between neighboring electrodes

Table 2 shows the Euclidean distance, Pearson’s linear correlation coefficient and R² value for neighboring electrodes in median [IQR] for the 10–10, extended 10–10 and uHD EEG system, respectively. The median correlation between neighboring electrodes increases with their median Euclidean distance decreasing.

TABLE 2

Table 2. Euclidean distance, Pearson’s linear correlation coefficient (r) and coefficient of determination (R²) for neighboring electrodes for the 10–10, extended 10–10 and ultra-high-density electroencephalography (uHD EEG) system.

4 Discussion

4.1 Topography plots

We could clearly show that the better spatial resolution of the uHD EEG system compared to standard EEG systems allows a more detailed understanding of temporal and spatial dynamics. The superimposed plot for all five fingers allows a detailed investigation of the 18 sensor locations with highest activation for all finger movements in subject S2 (see Figure 7). This activation area was approx. five times six columns wide, which resulted in 45 mm times 54 mm coverage surface. In standard EEG recordings, only two to four electrodes would overlay this area.

Electrocorticography (ECoG) electrode grids typically have a center-to-center distance of 10 mm, which is even larger than the uHD EEG system with 8.6 mm. However, the big advantage of ECoG remains–the possibility to capture high gamma or even ultra-high gamma frequencies, which is difficult with uHD EEG due to skin, bones, and fluids attenuating the signal captured on the scalp.

The electrodes seen in Figure 7A represent a focal spot overlaying the sensorimotor cortex (Ejaz et al., 2015). Across the five fingers, overlap in electrodes which showed the greatest beta ERD can be observed. This is in line with fMRI studies which reported overlapping areas throughout the primary motor cortex (M1) hand region (Sanes et al., 1995; Schieber, 1999; Ejaz et al., 2015) which are likely due to co-existence of somatotopic- and action maps (Graziano and Aflalo, 2007; Huber et al., 2020). Note that this is not the case for primary somatosensory cortex (S1) which has more distinct somatotopic organization (Schweisfurth et al., 2014; Huber et al., 2020; O’Neill et al., 2020). However, as the subjects’ exact central sulcus location is unknown in this work we could not determine which electrodes are located over M1 or S1. Furthermore the current work focused on the low-frequency bands (mu and beta), whereas for cortical mapping high frequency components are used [e.g., Kapeller et al. (2018)].

For both the time-frequency maps and the heatmap topographies (see Figures 6, 7) one can observe desynchronization with movement on- and offset. In fact, this phenomenon was discovered for all subjects and is consistent with findings in literature (Cassim et al., 2000; Sun et al., 2016). The observed ERD increases at the end of sustained movements, which may be associated with the cortical preparation before movement termination. ECoG studies also report similar increases in high-gamma activity for movement onset and termination (Flint et al., 2016). The decrease in ERD around 3–4 s post-cue matches the findings of Cassim et al. (2000) that mu and beta power return to baseline values in sustained movements, which may be associated with requiring less muscular activity and concentration (Bernard et al., 2002; Cramer et al., 2002). Indeed, this could explain why ERD is much more sustained (around 3–4 s post-cue) for the ring finger, as a controlled ringer finger extension requires more effort and concentration. Finally, post-movement beta rebounds (PMBR) can be observed for both heatmap topographies and time-frequency maps after movement termination at roughly 6.5 s. While PMBR is a well-known phenomenon, its function(s) is still not fully understood [see Hervault et al. (2021) for a recent investigation].

This first study was done with real finger movements without real-time feedback. Such non-feedback experiments prove the system’s feasibility to be used for the intended use, finger movement detection in this case. Studies from Guger, Pfurtscheller, and many others have shown that feedback change brain patterns and make them much more stable (Guger et al., 2000; Pfurtscheller, 2001; Miller et al., 2010). Therefore, our next studies will include feedback. The disadvantage of real movement studies is that brain activation terminates quickly as soon as no movement is performed. On the other hand, if motor imagery is used, subjects are able to activate the brain much longer (Gruenwald et al., 2017).

In some subjects, more bad channels had to be excluded from further processing. This happened because the adhesive did not fixate the grids strongly enough. In future experiments, a stronger adhesive will be used to ensure that more channels show good signals. This will also ensure that fewer trials will show artifacts.

4.2 Classification

We could show that single fingers can be discriminated with the uHD EEG system. Peak accuracy ranged from 85.4(S1), 79.2(S2), 69.4(S3), 71.5(S4) to 66.2(S5). Mean accuracies of 70.7(S1), 65.3(S2), 63.7(S3), 63.7(S4), and 60.6(S5) are in the range of other motor movement studies without feedback (Quandt et al., 2012; Bera et al., 2019). The difference is that now we are classifying single fingers, not left/right motor imagery. As stated in section “1,” the possibility of distinguishing between finger movements was sufficiently shown by ECoG studies (Kubánek et al., 2009; Hotson et al., 2016; Gruenwald et al., 2019) but few for EEG with limited performance (Liao et al., 2014; Stankevich et al., 2016; Bera et al., 2019). Essential to push the classification accuracy much further is to include feedback which will be done in the subsequent study.

The observation that finger pairs that include the ring finger have greater classification accuracy than others could be due to the ring finger extension requiring more effort than the other finger extensions. Reasonably, one may expect thumb vs. little finger classification accuracy to be the best as the somatotopic distance between the two fingers is large according to fMRI (e.g., Berlot et al., 2019) and ECoG studies (e.g., Miller et al., 2012). In contrast, we observed lower performance for this finger pair with a classification accuracy of 61.8 (3.4)% across subjects. However, this phenomenon may be explained by the fact that blood oxygenation level-dependent (BOLD) imaging or high-gamma activity is typically used for somatotopic investigations. Importantly, BOLD and high-gamma activity are related (Logothetis et al., 2001; Mukamel et al., 2005; Niessing et al., 2005; Engell et al., 2012) and have better somatotopic characteristics than low-frequency (i.e., mu and beta) band power (Miller et al., 2007, 2012). This may explain that somatotopic distance between fingers did not positively impact the observed classification accuracies.

The obtained classification accuracies [64.8 (6.3)% on average] can only be indirectly compared to Bera et al. (2019) and Liao et al. (2014). While as they also performed pairwise classification of finger movements, they used different features and classification models. Specifically, Bera et al. (2019) conducted finger extension tasks including thumb, index, and middle fingers but used CSPs, LDA and Extra Trees Classifiers for classification, achieving an overall average accuracy of 60.3% with two preceding training days. Liao et al. (2014) conducted two repetitive finger flexions and extensions tasks during 2 s. They used a SVM with a radial basis kernel function and achieved an average classification accuracy of 58.6 and 57.7% using mu and beta band power features, respectively. Note that since the exact experimental protocol (e.g., finger movements, number of trials) and analysis procedures are different, the results should not be directly compared.

Classification models based on the uHD EEG system outperformed models based on electrode subsets reflecting the 10–10 and extended 10–10 system by roughly 6 and 7% classification accuracy on median over all subjects and pairwise classifications.

4.3 Correlation between neighboring electrodes

Unsurprisingly, the correlation between neighboring electrodes increases with decreasing Euclidean distance (see Table 2). However, the observed correlations for the uHD EEG system are still far from perfect correlations. Furthermore, the relationship between Euclidean distance and correlation is not linear, which can be shown using the R² value. In the current context, R² quantifies the percentage of variance explained for one electrode given its neighboring electrode and is thus a measure of similarity. Specifically, when comparing the extended 10–10 and uHD EEG system the Euclidean distance roughly decreases by a factor of 2.7, while the amount of similarity between neighboring electrodes increases by 12% on median. Importantly, this increase in similarity is less than going from 10–10 to extended 10–10 system for which a median increase of 14% can be observed and in this case Euclidean distance only decreases by a factor of 1.5. Notably, the amount of unexplained variance (1-R²) is still 56% on median for neighboring electrodes for the uHD EEG system.

4.4 Limitations and outlook

In this study pairwise finger classification (i.e., binary classification) was performed instead of multiclass classification, where all five fingers are classified at once. The latter approach would allow for online (i.e., real-time) feedback provided by the hand avatar, which likely increases subjects’ engagement and classification accuracy. However, for a multiclass classifier to work reasonably well, pairwise classifications would need to reach high classification accuracies (e.g., > 85%). Achieving higher classification accuracies is likely possible as different features, feature/electrode selection methods and machine learning models can be employed. Indeed, methods such as Riemannian spatial patterns (Larzabal et al., 2021) may also be applied to gain insights in which electrodes are most important for decoding individual finger movements.

Topography ERD/S plots benefit from the uHD EEG technology, as seen in this study. Michel and Brunet (2019) stated that source reconstruction profits from high-density EEG technologies. The larger number of electrode points and the higher spatial density provide the necessary information. Therefore, the domain of source localization with procedures such as Low-Resolution Brain Electromagnetic Tomography Analysis (LORETA) could take advantage of the higher spatial resolution and electrode numbers. However, source reconstruction methods were not explored in the current work but should be explored in future work.

Adopting spatial mapping of the central sulcus using fMRI scans could provide important neuroanatomical information for the exact allocation of the electrode positions. Thus, more detailed assignment of acquired signals to the real spatial positions of the electrodes themselves can be done.

Data availability statement

The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found below: https://osf.io/4dwjt/?view_only=d23acfd50655427fbaae381a17cbfbcc.

Ethics statement

The studies involving human participants were reviewed and approved by Institutional Review Board of Korea Advanced Institute of Science and Technology (KH2018-127). The patients/participants provided their written informed consent to participate in this study.

Author contributions

HL, S-HJ, CG, and H-SP conceived the study. HP provided scientific input. HL and S-HJ designed and conducted experiments. MJ and LS created topography plots. SS performed data analyses. LS, S-HJ, and SS interpreted the result. HL, LS, S-HJ, and SS wrote and edited the manuscript. All authors contributed to the article and approved the submitted version.

Funding

This manuscript was based on research that has been conducted as part of the KAIST-funded Global Singularity Research Program for 2022. This work was partially funded via the European Commission project RHUMBO-H2020-MSCAITN-2018-813234 and g.tec Medical Engineering GmbH.

Acknowledgments

The authors would like to thank Johannes Grünwald and Christoph Kapeller for providing scientific input and advice on data analysis.

Conflict of interest

LS, SS, and MJ were employed at g.tec Medical Engineering GmbH. CG was the CEO of g.tec Medical Engineering GmbH.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnins.2022.1009878/full#supplementary-material

References

Acharya, S., Fifer, M. S., Benz, H. L., Crone, N. E., and Thakor, N. V. (2010). Electrocorticographic amplitude predicts finger positions during slow grasping motions of the hand. J. Neural Eng. 7:046002. doi: 10.1088/1741-2560/7/4/046002

CrossRef Full Text | Google Scholar

Al-Fahoum, A. S., and Al-Fraihat, A. A. (2014). Methods of EEG signal features extraction using linear analysis in frequency and time-frequency domains. ISRN Neurosci. 2014:730218. doi: 10.1155/2014/730218

PubMed Abstract | CrossRef Full Text | Google Scholar

Arvaneh, M., Guan, C., Ang, K. K., and Quek, C. (2011). Optimizing the channel selection and classification accuracy in EEG-based BCI. IEEE Trans. Biomed. Eng. 58, 1865–1873. doi: 10.1109/TBME.2011.2131142

PubMed Abstract | CrossRef Full Text | Google Scholar

Bai, Z., Fong, K. N. K., Zhang, J. J., Chan, J., and Ting, K. H. (2020). Immediate and long-term effects of BCI-based rehabilitation of the upper extremity after stroke: A systematic review and meta-analysis. J. Neuroeng. Rehabil. 17:57. doi: 10.1186/s12984-020-00686-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Belkacem, A. N., Jamil, N., Palmer, J. A., Ouhbi, S., and Chen, C. (2020). Brain computer interfaces for improving the quality of life of older adults and elderly patients. Front. Neurosci. 14:692. doi: 10.3389/fnins.2020.00692

PubMed Abstract | CrossRef Full Text | Google Scholar

Benjamini, Y., and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. R. Stat. Soc. Ser. B 57, 289–300. doi: 10.1111/j.2517-6161.1995.tb02031.x

CrossRef Full Text | Google Scholar

Benwell, C. S. Y., London, R. E., Tagliabue, C. F., Veniero, D., Gross, J., Keitel, C., et al. (2019). Frequency and power of human alpha oscillations drift systematically with time-on-task. Neuroimage 192, 101–114. doi: 10.1016/j.neuroimage.2019.02.067

PubMed Abstract | CrossRef Full Text | Google Scholar

Bera, S., Roy, R., Sikdar, D., and Mahadevappa, M. (2019). An ensemble learning based classification of individual finger movement from EEG. arXiv [Preprint] doi: 10.48550/arXiv.1903.10154

PubMed Abstract | CrossRef Full Text | Google Scholar

Berlot, E., Prichard, G., O’Reilly, J., Ejaz, N., and Diedrichsen, J. (2019). Ipsilateral finger representations in the sensorimotor cortex are driven by active movement processes, not passive sensory input. J. Neurophysiol. 121, 418–426. doi: 10.1152/jn.00439.2018

PubMed Abstract | CrossRef Full Text | Google Scholar

Bernard, R. A., Goran, D. A., Sakai, S. T., Carr, T. H., McFarlane, D., Nordell, B., et al. (2002). Cortical activation during rhythmic hand movements performed under three types of control: An fMRI study. Cogn. Affect. Behav. Neurosci. 2, 271–281. doi: 10.3758/cabn.2.3.271

PubMed Abstract | CrossRef Full Text | Google Scholar

Bhatia, D., Jo, S. H., Ryu, Y., Kim, Y., Kim, D. H., and Park, H.-S. (2021). Wearable triboelectric nanogenerator based exercise system for upper limb rehabilitation post neurological injuries. Nano Energy 80:105508. doi: 10.1016/j.nanoen.2020.105508

CrossRef Full Text | Google Scholar

Bouckaert, R. R., and Frank, E. (2004). “Evaluating the replicability of significance tests for comparing learning algorithms,” in Advances in knowledge discovery and data mining lecture notes in computer science, eds H. Dai, R. Srikant, and C. Zhang (Berlin: Springer), 3–12. doi: 10.1007/978-3-540-24775-3_3

CrossRef Full Text | Google Scholar

Bundy, D. T., Souders, L., Baranyai, K., Leonard, L., Schalk, G., Coker, R., et al. (2017). Contralesional brain–computer interface control of a powered exoskeleton for motor recovery in chronic stroke survivors. Stroke 48, 1908–1915. doi: 10.1161/STROKEAHA.116.016304

PubMed Abstract | CrossRef Full Text | Google Scholar

Cassim, F., Szurhaj, W., Sediri, H., Devos, D., Bourriez, J.-L., Poirot, I., et al. (2000). Brief and sustained movements: Differences in event-related (de)synchronization (ERD/ERS) patterns. Clin. Neurophysiol. 111, 2032–2039. doi: 10.1016/S1388-2457(00)00455-7

CrossRef Full Text | Google Scholar

Chamanzar, A., Haigh, S. M., Grover, P., and Behrmann, M. (2021). Abnormalities in cortical pattern of coherence in migraine detected using ultra high-density EEG. Brain Commun. 3:fcab061. doi: 10.1093/braincomms/fcab061

PubMed Abstract | CrossRef Full Text | Google Scholar

Chatrian, G. E., Lettich, E., and Nelson, P. L. (1985). Ten percent electrode system for topographic studies of spontaneous and evoked EEG activities. Am. J. EEG Technol. 25, 83–92. doi: 10.1080/00029238.1985.11080163

CrossRef Full Text | Google Scholar

Cheyne, D. O. (2013). MEG studies of sensorimotor rhythms: A review. Exp. Neurol. 245, 27–39. doi: 10.1016/j.expneurol.2012.08.030

PubMed Abstract | CrossRef Full Text | Google Scholar

Cramer, S. C., Weisskoff, R. M., Schaechter, J. D., Nelles, G., Foley, M., Finklestein, S. P., et al. (2002). Motor cortex activation is related to force of squeezing. Hum. Brain Mapp. 16, 197–205. doi: 10.1002/hbm.10040

PubMed Abstract | CrossRef Full Text | Google Scholar

Ejaz, N., Hamada, M., and Diedrichsen, J. (2015). Hand use predicts the structure of representations in sensorimotor cortex. Nat. Neurosci. 18, 1034–1040. doi: 10.1038/nn.4038

PubMed Abstract | CrossRef Full Text | Google Scholar

Engell, A. D., Huettel, S., and McCarthy, G. (2012). The fMRI BOLD signal tracks electrophysiological spectral perturbations, not event-related potentials. Neuroimage 59, 2600–2606. doi: 10.1016/j.neuroimage.2011.08.079

PubMed Abstract | CrossRef Full Text | Google Scholar

Flamary, R., and Rakotomamonjy, A. (2012). Decoding finger movements from ECoG signals using switching linear models. Front. Neurosci. 6:29. doi: 10.3389/fnins.2012.00029

PubMed Abstract | CrossRef Full Text | Google Scholar

Flint, R. D., Rosenow, J. M., Tate, M. C., and Slutzky, M. W. (2016). Continuous decoding of human grasp kinematics using epidural and subdural signals. J. Neural Eng. 14:016005. doi: 10.1088/1741-2560/14/1/016005

CrossRef Full Text | Google Scholar

Freeman, W. J., Holmes, M. D., Burke, B. C., and Vanhatalo, S. (2003). Spatial spectra of scalp EEG and EMG from awake humans. Clin. Neurophysiol. 114, 1053–1068. doi: 10.1016/S1388-2457(03)00045-2

CrossRef Full Text | Google Scholar

Graziano, M. S. A., and Aflalo, T. N. (2007). Rethinking cortical organization: Moving away from discrete areas arranged in hierarchies. Neuroscientist 13, 138–147. doi: 10.1177/1073858406295918

PubMed Abstract | CrossRef Full Text | Google Scholar

Gruenwald, J., Kapeller, C., Guger, C., Ogawa, H., Kamada, K., and Scharinger, J. (2017). “Comparison of Alpha/Beta and high-gamma band for motor-imagery based BCI control: A qualitative study,” in Proceedings of the 2017 IEEE international conference on systems, man, and cybernetics (SMC), Banff, 2308–2311. doi: 10.1109/SMC.2017.8122965

CrossRef Full Text | Google Scholar

Gruenwald, J., Znobishchev, A., Kapeller, C., Kamada, K., Scharinger, J., and Guger, C. (2019). Time-variant linear discriminant analysis improves hand gesture and finger movement decoding for invasive brain-computer interfaces. Front. Neurosci. 13:901. doi: 10.3389/fnins.2019.00901

PubMed Abstract | CrossRef Full Text | Google Scholar

Guger, C., Ramoser, H., and Pfurtscheller, G. (2000). Real-time EEG analysis with subject-specific spatial patterns for a brain-computer interface (BCI). IEEE Trans. Rehabil. Eng. 8, 447–456. doi: 10.1109/86.895947

CrossRef Full Text | Google Scholar

Guger, C., Spataro, R., Allison, B. Z., Heilinger, A., Ortner, R., Cho, W., et al. (2017). Complete locked-in and locked-in patients: Command following assessment and communication with vibro-tactile P300 and motor imagery brain-computer interface tools. Front. Neurosci. 11:251. doi: 10.3389/fnins.2017.00251

PubMed Abstract | CrossRef Full Text | Google Scholar

Harkema, S., Gerasimenko, Y., Hodes, J., Burdick, J., Angeli, C., Chen, Y., et al. (2011). Effect of epidural stimulation of the lumbosacral spinal cord on voluntary movement, standing, and assisted stepping after motor complete paraplegia: A case study. Lancet 377, 1938–1947. doi: 10.1016/S0140-6736(11)60547-3

CrossRef Full Text | Google Scholar

Hervault, M., Zanone, P.-G., Buisson, J.-C., and Huys, R. (2021). Cortical sensorimotor activity in the execution and suppression of discrete and rhythmic movements. Sci. Rep. 11:22364. doi: 10.1038/s41598-021-01368-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Hotson, G., McMullen, D. P., Fifer, M. S., Johannes, M. S., Katyal, K. D., Para, M. P., et al. (2016). Individual finger control of a modular prosthetic limb using high-density electrocorticography in a human subject. J. Neural Eng. 13:026017. doi: 10.1088/1741-2560/13/2/026017

CrossRef Full Text | Google Scholar

Huber, L., Finn, E. S., Handwerker, D. A., Bönstrup, M., Glen, D. R., Kashyap, S., et al. (2020). Sub-millimeter fMRI reveals multiple topographical digit representations that form action maps in human motor cortex. Neuroimage 208:116463. doi: 10.1016/j.neuroimage.2019.116463

PubMed Abstract | CrossRef Full Text | Google Scholar

Jasper, H. H. (1958). The ten-twenty electrode system of the international federation. Electroencephalogr. Clin. Neurophysiol. 10, 370–375.

Google Scholar

Jones, L. A., and Lederman, S. J. (2006). Human hand function. New York, NY: Oxford University Press.

Google Scholar

Kapeller, C., Ogawa, H., Schalk, G., Kunii, N., Coon, W. G., Scharinger, J., et al. (2018). Real-time detection and discrimination of visual perception using electrocorticographic signals. J. Neural Eng. 15:036001. doi: 10.1088/1741-2552/aaa9f6

PubMed Abstract | CrossRef Full Text | Google Scholar

Kubánek, J., Miller, K. J., Ojemann, J. G., Wolpaw, J. R., and Schalk, G. (2009). Decoding flexion of individual fingers using electrocorticographic signals in humans. J. Neural Eng. 6:066001. doi: 10.1088/1741-2560/6/6/066001

CrossRef Full Text | Google Scholar

Kubanek, J., and Schalk, G. (2015). NeuralAct: A tool to visualize electrocortical (ECoG) activity on a three-dimensional model of the cortex. Neuroinformatics 13, 167–174. doi: 10.1007/s12021-014-9252-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Landsmeer, J. M. F. (1962). Power grip and precision handling. Ann. Rheum. Dis. 21, 164–170. doi: 10.1136/ard.21.2.164

PubMed Abstract | CrossRef Full Text | Google Scholar

Lange, G., Low, C. Y., Johar, K., Hanapiah, F. A., and Kamaruzaman, F. (2016). Classification of electroencephalogram data from hand grasp and release movements for BCI controlled prosthesis. Proc. Technol. 26, 374–381. doi: 10.1016/j.protcy.2016.08.048

CrossRef Full Text | Google Scholar

Larzabal, C., Auboiroux, V., Karakas, S., Charvet, G., Benabid, A.-L., Chabardes, S., et al. (2021). The Riemannian spatial pattern method: Mapping and clustering movement imagery using Riemannian geometry. J. Neural Eng. 18:056014. doi: 10.1088/1741-2552/abf291

PubMed Abstract | CrossRef Full Text | Google Scholar

Liang, N., and Bougrain, L. (2012). Decoding finger flexion from band-specific ECoG signals in humans. Front. Neurosci. 6:91. doi: 10.3389/fnins.2012.00091

PubMed Abstract | CrossRef Full Text | Google Scholar

Liao, K., Xiao, R., Gonzalez, J., and Ding, L. (2014). Decoding individual finger movements from one hand using human EEG signals. PLoS One 9:e85192. doi: 10.1371/journal.pone.0085192

PubMed Abstract | CrossRef Full Text | Google Scholar

Logothetis, N. K., Pauls, J., Augath, M., Trinath, T., and Oeltermann, A. (2001). Neurophysiological investigation of the basis of the fMRI signal. Nature 412, 150–157. doi: 10.1038/35084005

PubMed Abstract | CrossRef Full Text | Google Scholar

López-Larraz, E., Montesano, L., Gil-Agudo, Á., and Minguez, J. (2014). Continuous decoding of movement intention of upper limb self-initiated analytic movements from pre-movement EEG correlates. J. Neuroeng. Rehabil. 11:153. doi: 10.1186/1743-0003-11-153

PubMed Abstract | CrossRef Full Text | Google Scholar

Lotte, F., Bougrain, L., Cichocki, A., Clerc, M., Congedo, M., Rakotomamonjy, A., et al. (2018). A review of classification algorithms for EEG-based brain–computer interfaces: A 10 year update. J. Neural Eng. 15:031005. doi: 10.1088/1741-2552/aab2f2

PubMed Abstract | CrossRef Full Text | Google Scholar

Lotte, F., Congedo, M., Lécuyer, A., Lamarche, F., and Arnaldi, B. (2007). A review of classification algorithms for EEG-based brain–computer interfaces. J. Neural Eng. 4, R1–R13. doi: 10.1088/1741-2560/4/2/R01

CrossRef Full Text | Google Scholar

Lotte, F., and Guan, C. (2011). Regularizing common spatial patterns to improve BCI designs: Unified theory and new algorithms. IEEE Trans. Biomed. Eng. 58, 355–362. doi: 10.1109/TBME.2010.2082539

PubMed Abstract | CrossRef Full Text | Google Scholar

Lu, H., Plataniotis, K. N., and Venetsanopoulos, A. N. (2009). “Regularized common spatial patterns with generic learning for EEG signal classification,” in Proceedings 2009 annual international conference of the IEEE engineering in medicine and biology society, Minneapolis, MN, 6599–6602. doi: 10.1109/IEMBS.2009.5332554

PubMed Abstract | CrossRef Full Text | Google Scholar

Mason, S. G., Bashashati, A., Fatourechi, M., Navarro, K. F., and Birch, G. E. (2007). A comprehensive survey of brain interface technology designs. Ann. Biomed. Eng. 35, 137–169. doi: 10.1007/s10439-006-9170-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Mathewson, K. E., Harrison, T. J. L., and Kizuk, S. A. D. (2017). High and dry? Comparing active dry EEG electrodes to active and passive wet electrodes. Psychophysiology 54, 74–82. doi: 10.1111/psyp.12536

PubMed Abstract | CrossRef Full Text | Google Scholar

Mawase, F., Cherry-Allen, K., Xu, J., Anaya, M., Uehara, S., and Celnik, P. (2020). Pushing the rehabilitation boundaries: Hand motor impairment can be reduced in chronic stroke. Neurorehabil. Neural Repair. 34, 733–745. doi: 10.1177/1545968320939563

PubMed Abstract | CrossRef Full Text | Google Scholar

Mejia Tobar, A., Hyoudou, R., Kita, K., Nakamura, T., Kambara, H., Ogata, Y., et al. (2018). Decoding of ankle flexion and extension from cortical current sources estimated from non-invasive brain activity recording methods. Front. Neurosci. 11:733. doi: 10.3389/fnins.2017.00733

PubMed Abstract | CrossRef Full Text | Google Scholar

Michel, C. M., and Brunet, D. (2019). EEG source imaging: A practical review of the analysis steps. Front. Neurol. 10:325. doi: 10.3389/fneur.2019.00325

PubMed Abstract | CrossRef Full Text | Google Scholar

Miller, K. J., Hermes, D., Honey, C. J., Hebb, A. O., Ramsey, N. F., Knight, R. T., et al. (2012). Human motor cortical activity is selectively phase-entrained on underlying rhythms. PLoS Comput. Biol. 8:e1002655. doi: 10.1371/journal.pcbi.1002655

PubMed Abstract | CrossRef Full Text | Google Scholar

Miller, K. J., Leuthardt, E. C., Schalk, G., Rao, R. P. N., Anderson, N. R., Moran, D. W., et al. (2007). Spectral changes in cortical surface potentials during motor movement. J. Neurosci. 27, 2424–2432. doi: 10.1523/JNEUROSCI.3886-06.2007

PubMed Abstract | CrossRef Full Text | Google Scholar

Miller, K. J., Schalk, G., Fetz, E. E., den Nijs, M., Ojemann, J. G., and Rao, R. P. N. (2010). Cortical activity during motor execution, motor imagery, and imagery-based online feedback. Proc. Natl. Acad. Sci. 107, 4430–4435. doi: 10.1073/pnas.0913697107

PubMed Abstract | CrossRef Full Text | Google Scholar

Mukamel, R., Gelbard, H., Arieli, A., Hasson, U., Fried, I., and Malach, R. (2005). Coupling between neuronal firing, field potentials, and fMRI in human auditory cortex. Science 309, 951–954. doi: 10.1126/science.1110913

PubMed Abstract | CrossRef Full Text | Google Scholar

Narayanan, A. M., Zink, R., and Bertrand, A. (2021). EEG miniaturization limits for stimulus decoding with EEG sensor networks. J. Neural Eng. 18:056042. doi: 10.1088/1741-2552/ac2629

PubMed Abstract | CrossRef Full Text | Google Scholar

Niessing, J., Ebisch, B., Schmidt, K. E., Niessing, M., Singer, W., and Galuske, R. A. W. (2005). Hemodynamic signals correlate tightly with synchronized gamma oscillations. Science 309, 948–951. doi: 10.1126/science.1110948

PubMed Abstract | CrossRef Full Text | Google Scholar

Odabaee, M., Freeman, W. J., Colditz, P. B., Ramon, C., and Vanhatalo, S. (2013). Spatial patterning of the neonatal EEG suggests a need for a high number of electrodes. Neuroimage 68, 229–235. doi: 10.1016/j.neuroimage.2012.11.062

PubMed Abstract | CrossRef Full Text | Google Scholar

Ofner, P., Schwarz, A., Pereira, J., and Müller-Putz, G. R. (2017). Upper limb movements can be decoded from the time-domain of low-frequency EEG. PLoS One 12:e0182578. doi: 10.1371/journal.pone.0182578

PubMed Abstract | CrossRef Full Text | Google Scholar

Ojala, M., and Garriga, G. C. (2009). “Permutation tests for studying classifier performance,” in Proceedings of the 2009 ninth IEEE international conference on data mining, Miami Beach, FL: IEEE, 908–913. doi: 10.1109/ICDM.2009.108

CrossRef Full Text | Google Scholar

O’Neill, G. C., Sengupta, A., Asghar, M., Barratt, E. L., Besle, J., Schluppeck, D., et al. (2020). A probabilistic atlas of finger dominance in the primary somatosensory cortex. Neuroimage 217:116880. doi: 10.1016/j.neuroimage.2020.116880

PubMed Abstract | CrossRef Full Text | Google Scholar

Oostenveld, R., and Praamstra, P. (2001). The five percent electrode system for high-resolution EEG and ERP measurements. Clin. Neurophysiol. 112, 713–719. doi: 10.1016/S1388-2457(00)00527-7

CrossRef Full Text | Google Scholar

Petrov, Y., Nador, J., Hughes, C., Tran, S., Yavuzcetin, O., and Sridhar, S. (2014). Ultra-dense EEG sampling results in two-fold increase of functional brain information. Neuroimage 90, 140–145. doi: 10.1016/j.neuroimage.2013.12.041

PubMed Abstract | CrossRef Full Text | Google Scholar

Pfurtscheller, G. (2001). Functional brain imaging based on ERD/ERS. Vis. Res. 41, 1257–1260. doi: 10.1016/S0042-6989(00)00235-2

CrossRef Full Text | Google Scholar

Pfurtscheller, G., and Lopes da Silva, F. H. (1999). Event-related EEG/MEG synchronization and desynchronization: Basic principles. Clin. Neurophysiol. 110, 1842–1857. doi: 10.1016/S1388-2457(99)00141-8

CrossRef Full Text | Google Scholar

Pfurtscheller, G., Neuper, C., Flotzinger, D., and Pregenzer, M. (1997). EEG-based discrimination between imagination of right and left hand movement. Electroencephalogr. Clin. Neurophysiol. 103, 642–651. doi: 10.1016/S0013-4694(97)00080-1

CrossRef Full Text | Google Scholar

Probst, P., Boulesteix, A.-L., and Bischl, B. (2019). Tunability: Importance of hyperparameters of machine learning algorithms. J. Mach. Learn. Res. 20, 1–32.

Google Scholar

Quandt, F., Reichert, C., Hinrichs, H., Heinze, H. J., Knight, R. T., and Rieger, J. W. (2012). Single trial discrimination of individual finger movements on one hand: A combined MEG and EEG study. Neuroimage 59, 3316–3324. doi: 10.1016/j.neuroimage.2011.11.053

PubMed Abstract | CrossRef Full Text | Google Scholar

Rashid, M., Sulaiman, N., Majeed, A. P. P., Musa, R. M., Nasir, A. F. A., Bari, B. S., et al. (2020). Current status, challenges, and possible solutions of EEG-based brain-computer interface: A comprehensive review. Front. Neurorobot. 14:25. doi: 10.3389/fnbot.2020.00025

PubMed Abstract | CrossRef Full Text | Google Scholar

Robinson, A. K., Venkatesh, P., Boring, M. J., Tarr, M. J., Grover, P., and Behrmann, M. (2017). Very high density EEG elucidates spatiotemporal aspects of early visual processing. Sci. Rep. 7:16248. doi: 10.1038/s41598-017-16377-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Sanes, J. N., Donoghue, J. P., Thangaraj, V., Edelman, R. R., and Warach, S. (1995). Shared neural substrates controlling hand movements in human motor cortex. Science 268, 1775–1777. doi: 10.1126/science.7792606

PubMed Abstract | CrossRef Full Text | Google Scholar

Schieber, M. H. (1999). Somatotopic gradients in the distributed organization of the human primary motor cortex hand area: Evidence from small infarcts. Exp. Brain Res. 128, 139–148. doi: 10.1007/s002210050829

PubMed Abstract | CrossRef Full Text | Google Scholar

Schweisfurth, M. A., Frahm, J., and Schweizer, R. (2014). Individual fMRI maps of all phalanges and digit bases of all fingers in human primary somatosensory cortex. Front. Hum. Neurosci. 8:658. doi: 10.3389/fnhum.2014.00658

PubMed Abstract | CrossRef Full Text | Google Scholar

Sebastián-Romagosa, M., Cho, W., Ortner, R., Murovec, N., Von Oertzen, T., Kamada, K., et al. (2020). Brain computer interface treatment for motor rehabilitation of upper extremity of stroke patients—A feasibility study. Front. Neurosci. 14:591435. doi: 10.3389/fnins.2020.591435

PubMed Abstract | CrossRef Full Text | Google Scholar

Sharbrough, F. (1991). American electroencephalographic society guidelines for standard electrode position nomenclature. J. Clin. Neurophysiol. 8, 200–202.

Google Scholar

Stankevich, L. A., Sonkin, K. M., Shemyakina, N. V., Nagornova, Z. V., Khomenko, J. G., Perets, D. S., et al. (2016). EEG pattern decoding of rhythmic individual finger imaginary movements of one hand. Hum. Physiol. 42, 32–42. doi: 10.1134/S0362119716010175

CrossRef Full Text | Google Scholar

Sun, S., Cao, X., and Wang, Q. (2016). “Continuous decoding of movement onset and offset of sustained movements from cortical activities,” in Proceedings of the 2016 IEEE/SICE international symposium on system integration (SII), Sapporo, 809–814. doi: 10.1109/SII.2016.7844099

CrossRef Full Text | Google Scholar

Varoquaux, G., Raamana, P. R., Engemann, D. A., Hoyos-Idrobo, A., Schwartz, Y., and Thirion, B. (2017). Assessing and tuning brain decoders: Cross-validation, caveats, and guidelines. Neuroimage 145, 166–179. doi: 10.1016/j.neuroimage.2016.10.038

PubMed Abstract | CrossRef Full Text | Google Scholar

Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., and Vaughan, T. M. (2002). Brain–computer interfaces for communication and control. Clin. Neurophysiol. 113, 767–791. doi: 10.1016/S1388-2457(02)00057-3

CrossRef Full Text | Google Scholar

Yanagisawa, T., Hirata, M., Saitoh, Y., Goto, T., Kishima, H., Fukuma, R., et al. (2011). Real-time control of a prosthetic hand using human electrocorticography signals: Technical note. J. Neurosurg. 114, 1715–1722. doi: 10.3171/2011.1.JNS101421

PubMed Abstract | CrossRef Full Text | Google Scholar