Classification of Schizophrenia by Combination of Brain Effective and Functional Connectivity

At present, lots of studies have tried to apply machine learning to different electroencephalography (EEG) measures for diagnosing schizophrenia (SZ) patients. However, most EEG measures previously used are either a univariate measure or a single type of brain connectivity, which may not fully capture the abnormal brain changes of SZ patients. In this paper, event-related potentials were collected from 45 SZ patients and 30 healthy controls (HCs) during a learning task, and then a combination of partial directed coherence (PDC) effective and phase lag index (PLI) functional connectivity were used as features to train a support vector machine classifier with leave-one-out cross-validation for classification of SZ from HCs. Our results indicated that an excellent classification performance (accuracy = 95.16%, specificity = 94.44%, and sensitivity = 96.15%) was obtained when the combination of functional and effective connectivity features was used, and the corresponding optimal feature number was 15, which included 12 PDC and three PLI connectivity features. The selected effective connectivity features were mainly located between the frontal/temporal/central and visual/parietal lobes, and the selected functional connectivity features were mainly located between the frontal/temporal and visual cortexes of the right hemisphere. In addition, most of the selected effective connectivity abnormally enhanced in SZ patients compared with HCs, whereas all the selected functional connectivity features decreased in SZ patients. The above results showed that our proposed method has great potential to become a tool for the auxiliary diagnosis of SZ.

Introduction

Schizophrenia (SZ) is one of the most severe and common mental disorders, and the World Health Organization (WHO) stated that more than 21 million people worldwide are affected by SZ. SZ is often accompanied by severe clinical symptoms including interrupted thinking and speech, hallucinations, and cognitive impairments (Van and Kapur, 2009; Haro et al., 2014), which usually lead to a significant reduction in the patients’ quality of life and bring huge economic burden to the society and family. Therefore, accurate diagnosis has important implications for subsequent treatment of SZ. Traditional clinical diagnosis of SZ mainly relies on interviews with patients by experienced psychiatrists, but this method is sometimes inaccurate and subjective (Andreas et al., 2009). Therefore, lots of researchers tried to use different techniques to identify objective and quantitative biomarkers of SZ, and these techniques were involved in immunology (Nurjono et al., 2012), genetics (Consortium, 2014), structural or morphological imaging (James et al., 2016; Pergola et al., 2016; Kubera et al., 2017; Woodward et al., 2018), functional imaging (Chyzhyk et al., 2015; Moghimi et al., 2018; Ji et al., 2019), and electroencephalography (EEG; Akar et al., 2016; Gomez-Pilar et al., 2017; Hunt et al., 2017; Devia et al., 2019). Compared with other methods, EEG still remains one of the most widespread techniques so far due to its high time resolution and low cost.

In recent years, researchers have reported the following hallmarks in SZ patients with the aid of EEG technique: abnormal event-related potential (ERP) amplitude/latency (Wang et al., 2003; Morris et al., 2008; Hamilton et al., 2017), altered spectral power of different frequency bands (Kam et al., 2013; BaAr et al., 2015; Hirano et al., 2015; Phillips and Uhlhaas, 2015; Hunt et al., 2017), and changes in complexity of EEG signal (Akar et al., 2015, 2016; Yu et al., 2016). However, most of the above EEG hallmarks were identified by using group-level statistical methods that do not provide a mechanism for classifying SZ at the individual level. Thus, some researchers have tried to use machine learning (ML) methods with these EEG biomarkers to differentiate SZ patients from healthy controls (HCs). Devia et al. (2019) used the ERP component evoked in a free-viewing paradigm as a feature and linear discriminant analysis (LDA) as a classifier to distinguish between SZ patients and HCs, and they reported an overall accuracy of 71%. Shim et al. (2016) used a combination of sensor-level (P300 amplitudes/latencies) and source-level (cortical current density values) EEG features during an auditory oddball task to train a support vector machine (SVM) classifier and finally got a high classification accuracy of 88.24%. Kim et al. (2015) tried to use delta band power to classify SZ patients but got a relatively poor classification accuracy of 62.2%. Bose et al. (2016) reported that when alpha band power during hyperventilation and post-hyperventilation was used as an input of a SVM classifier, a high classification accuracy of 83.33% was yielded. Chu et al. (2017) used EEG entropy during visual evocation of emotion as a feature and a SVM classifier to yield a relatively high classification accuracy of 81.5%. However, these studies seemed to have a common limitation: they usually used univariate indexes, i.e., spectral power/ERP/nonlinear measures of local electrodes or brain regions, as an input of a ML classifier.

It has been recently suggested that SZ could affect distributed brain neuronal network and induce a functional brain disconnection syndrome (Andreou et al., 2015a). A large number of EEG studies have confirmed dysfunctional connectivity in SZ patients at the group level (Andreou et al., 2015b; Di et al., 2015; Zhao et al., 2018). Thus, alterations in the brain connectivity patterns seem to serve as additional biomarkers of SZ. The so-called “brain functional connectivity” refers to the statistical dependence between two or more brain regions’ electrophysiological or other signals. Among the usually applied methods to study functional connectivity, such as spectral coherence and correlation coefficient, phase lag index (PLI), which allows for quantification of phase synchronization, has been widely applied in many studies because its most significant advantage is insensitivity to volume conduction effect as compared with other methods (Stam et al., 2007; Doesburg et al., 2013; Olejarczyk and Jernajczyk, 2017; Wang et al., 2017). The so-called “brain effective connectivity” that is another important dimension of functional connectivity can measure the directional flows of information among different brain areas. One of the most commonly used methods to construct effective connectivity is called partial directed coherence (PDC), which is a full multivariate spectral measure to determine the directed influences of Granger causality between any given signals in a multivariate set and has been successfully applied in measuring the multichannel directed cortical interactions (Silfverhuth et al., 2012; Sun et al., 2014a,b; Huang et al., 2016; Khandoker et al., 2019). Although altered brain connectivity patterns can provide useful information in discriminating SZ patients from HCs, it seems that only a few studies used EEG-based brain connectivity features as features to differentiate SZ. For example, Khandoker et al. (2019) used PDC effective connectivity as an input of a deep convolutional neural network (CNN) classifier to achieve a remarkable accuracy of 93.06%. However, brain functional connectivity and effective connectivity are two distinctly different methods and can reveal abnormal brain connectivity patterns of SZ from different aspects. Therefore, we hypothesize that the combination of brain functional connectivity and effective connectivity could provide a good discrimination between SZ patients and HCs.

In order to test the above hypothesis, ERP data were first recorded from SZ patients and HCs during a reinforcement learning task. Then, PLI and PDC methods were used to construct EEG-based brain functional and effective connectivity, respectively. Based on the constructed functional and effective connectivity, three strategies were tried to select appropriate brain connectivity features as an input of a supervised SVM classifier to classify SZ patients and HCs: SVM with only functional connectivity as features, SVM with only effective connectivity as features, and SVM with a combination of functional and effective connectivity. To the best of our knowledge, this is the first study using a combination of PDC effective connectivity and PLI functional connectivity as features for ML-based classification of SZ.

Materials and Methods

Electroencephalography Data

The EEG data used in our study were provided by Albrecht et al. (2016) and can be downloaded from an open access database called Zenodo (doi://10.5281/zenodo.29601 and doi://10.5281/zenodo.29064). The subjects, task, EEG recording, and preprocessing have been described in detail in Albrecht et al. (2016) and our recently published paper (Zhao et al., 2020). Therefore, we followed the methods of Zhao et al. (2020) in the following part. In brief, 45 SZ patients were recruited from the Maryland Psychiatric Research Center or other nearby clinics according to the Structured Clinical Interview for the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV), and a group of 30 healthy subjects matched for sex and age was also recruited. For the experimental task, four image stimuli were presented to subjects (48 times/each image), and subjects were asked to respond by pressing a button (“Go”) or withholding a response (“NoGo”) to gain rewards (“Win,” monetary gain) or avoid punishments (“Avoid,” monetary loss). Therefore, four kinds of stimuli would appear throughout the experiment, i.e., “Go-to-Win,” “Go-to-Avoid,” “NoGo-to-Win,” and “NoGo-to-Avoid”; and subjects were asked to make the best response to win as much money as possible. For “Go-to-Win” and “NoGo-to-Win” stimuli, the subject was rewarded at a probability of 80% and got nothing (no reward and punishment) at a probability of 20% for a correct response; the subject got nothing at a probability of 80% and was rewarded at a probability of 20% for a wrong response. For “Go-to-Avoid” and “NoGo-to-Avoid” stimuli, the subject got nothing at a probability of 80% and punishment at a probability of 20% for a correct response; the subject got nothing at a probability of 20% and punishment at a probability of 80% for a wrong response. Monetary gain or loss was set at $0.05 each trial. The stimulus presentation sequence was as follows: a cross was presented for 400–600 ms, and then one of the four image stimuli was presented for 1,000 ms; next, a blank period was shown for 250–2,000 ms, and then a response period presented for 2,500 ms is indicated with an “O” for 1,500 ms; next a cross was presented for 1,000 ms; and finally, feedback image was shown for 2,000 ms. In this task, there were three types of feedback, i.e., positive feedback (thumb up image reflecting monetary gain), negative feedback (thumb down image reflecting monetary loss), and neutral feedback (thumb to the side image reflecting no monetary gain or loss). Here, we only used negative feedback-evoked ERPs. In their previously published paper, Albrecht et al. (2016) have described the procedure of EEG recording and preprocessing in detail, so we do not want to repeat them here. In a word, 32 channel EEG data were recorded during the above task, re-referenced to linked mastoids and down-sampled to 256 Hz. After standard EEG preprocessing process, the cleaned negative feedback-locked ERPs [(-1,500, 1,500) ms around negative feedback stimulus, baseline corrected to (-1,000) ms] were obtained for further analysis. In addition, 20 EEG channels (FP1, FP2, Fz, F3, F4, F7, T7, T8, C3, C4, Cz, Pz, P4, P3, F8, P8, P7, Oz, O2, and O1) were selected in our study.

Time-Frequency Power Calculation

Time-frequency power calculation of a single trial data was performed with Morlet wavelet transform where peak frequency ranging from 3.9 to 40 Hz and cycle number varying from 3 cycles at 3.9 Hz to 11.4 cycles at 40 Hz were linearly divided into 50 points, respectively. Finally, for every trial data, we obtained a 2D matrix of 50 (frequency points) × 768 (time points).

Phase Lag Index Functional Connectivity Feature

Phase lag index, a phase synchronization index, was used for quantification of functional connectivity between each EEG channel pair in the present study (Stam et al., 2007):

$$PL{I}_{(f,t)}=\left|\frac{1}{M}\sum _{m=1}^{M}sgn(\mathrm{△}{\mathrm{\phi }}_{a,b}^m(f,t))\right|(1)$$

where $\mathrm{△}{\mathrm{\phi }}_{a,b}^m(f,t)$ stands for phase difference between channel a and b at frequency f and time t of trial m, M stands for the number of trials, and sgn indicates the sign (−1 for negative values, +1 for positive values, and 0 for zero values).

With the use of the above equation, a 20 × 20 PLI functional connectivity matrix could be obtained at each time-frequency point. In the present study, PLI values were averaged over theta band (4–7 Hz) and a time period from 0.1 to 0.6 s after stimulus onset to obtain a weighted 20 × 20 functional connectivity matrix used as candidate feature for each subject.

Partial Directed Coherence Effective Connectivity Feature

In this study, PDC was used to determine the directed influences among these 20 channel EEG signals. Detailed descriptions and calculation methods can be found in some previously published literatures (Sun et al., 2014a,b). In brief, let X(n) = [x₁(n), x₂(n), x₃(n), …, x_N(n)]^T stand for EEG signal of N channels (N = 20 here), and then a multivariate autoregressive (MVAR) model can be used to describe X(n):

$$X\left(n\right)=\sum _{r=1}^p{A}_{r}X(n-r)+W\left(n\right)(2)$$

where W(n) is a multivariate uncorrelated noise vector, A_r is the coefficient matrix, and p is the MVAR model order, which can be measured by using the Bayesian information criterion (BIC). In the current study, an optimal order of 9 was used. The A_r can be computed by using Yule–Walker equations:

$$\sum _{r=1}^p{A}_{r}R\left(-k+r\right)=0(3)$$

where R(m) = < X(n) X^T(n + m)> are the covariance matrices of all X(n) with lag m, k = 1, 2, …, p. The Levinson–Wiggins–Robinson (LWR) algorithm is one of the most used methods to determine A_r(Morf et al., 1978). For ERP data, many trials of the same process are available, and a modified adaptive procedure for estimating the MVAR model can be applied (Ding et al., 2000). Briefly, the covariance matrices R_n(m) (n = 1, 2, …, T) for T trials were computed to obtain the averaged covariance matrix ($\overline{R}\left(m\right)=\sum _{n=1}^T{R}_n(m)/T$) for each subject, then the R(m) in Eq (3) was replaced with $\overline{R}\left(m\right)$, and finally, the A_r was determined.

After the MVAR model is estimated, the PDC value from channel j to channel i at frequency f can be computed as follows:

$$\text{PDC}(i,j,f)=\frac{\left|A_{ij}\left(f\right)\right|}{\sqrt{{\sum }_k{A}_{kj}^{*}\left(f\right)A_{kj}\left(f\right)}}(4)$$

where A(f) stands for the difference between the N-dimensional identity matrix and the Fourier transform of A_r, * stands for matrix transposition and complex conjugation, and A_ij(f) are the elements of the matrix A(f).

A nonparametric method based on surrogate data was applied here to test the significance of PDC value (Astolfi et al., 2005). In brief, the phase of the Fourier coefficients of original EEG signals from each epoch was randomly shuffled, and the inverse Fourier transform was performed to obtain a surrogate data. Then, the obtained surrogate data were used to re-compute PDC. After the above-mentioned shuffling procedure were repeated 5,000 times, an empirical distribution of PDC values could be obtained. Based on this empirical distribution, a significance threshold of p = 0.05 was used to determine if the PDC value was a real connection.

In this study, we used time window of (0.1, 0.6) s after stimulus onset to compute PDC, and the obtained PDC(i,j,f)value was averaged over the theta band (4–7 Hz) to obtain a 20 × 20 PDC effective connectivity matrix used as a candidate feature for each subject.

Feature Selection and Classification of Schizophrenia

Until now, we obtained a total of 570 features including 20 × (20 − 1)/2 = 190 functional connectivity features and 20 × (20 − 1) = 380 effective connectivity features. It is undesirable to use all the features that usually contain lots of irrelevant features for direct classification, because these irrelevant features will lead to a bad classification performance. In order to select relevant and key features for SZ classification, Fisher score was used as a feature selection criterion, which has been widely used in many previously published classification studies (Hwang et al., 2014; Zhang et al., 2015; Shim et al., 2016). A higher Fisher score means that this feature has a higher discriminative power than a feature with a lower Fisher score. Based on the Fisher score, three strategies were implemented to select appropriate brain connectivity features: (1) only functional connectivity features: for functional connectivity feature set, the top N features with the highest Fisher scores were selected for classification, and N varies from 1 to 50 to find optimal feature number. (2) Only effective connectivity features: for effective connectivity feature set, the top M features with the highest Fisher scores were selected for classification, and M also varies from 1 to 50 to search optimal feature number. (3) A combination of functional and effective connectivity features: the total number of features was set to L varying from 2 to 50, which contained the top n and m (L = n + m) features with the highest Fisher scores from functional and effective connectivity feature set, respectively. In order to search the optimal combination of these two kinds of features, n varies from 1 to L − 1, while m changed from L − 1 to 1.

The classification of SZ patients and HCs was performed using an SVM classifier with a linear kernel function. The SVM classifier used in the present study was provided by LIBSVM toolbox (Chang and Lin, 2011). The classification accuracy (ACC), specificity (SPE), sensitivity (SEN), and area under the receiver operating characteristic (AUC) curve were determined using the leave-one-out cross-validation (LOOCV).

Statistics

In the present study, we performed the comparison of time-frequency power maps between SZ patients and HCs, which involved a total of 15,350 comparisons (307 time points from −0.2 to 1.0 s, 50 frequency points from 3.9 to 40 Hz). Thus, we applied a cluster-based nonparametric permutation test to control multiple comparison problem (Maris and Oostenveld, 2007). Briefly, 10,000 random permutations of the time-frequency power maps between two groups were performed. For each permutation, t-test was carried out between the permuted time-frequency maps, all pixels with an uncorrected p value below 0.01 were formed into clusters, and the largest cluster (i.e., the cluster with the largest number of pixels) was stored. Therefore, we obtained a null distribution of the largest clusters after 10,000 random permutations. Finally, the original time-frequency maps of two groups were also compared using t-test, all pixels with p values below 0.01 were formed into clusters, and any clusters that were less than the (1 − α) percentile of the null distribution of largest clusters expected were removed. In our current study, the family-wise α level was set to 0.05.

The nonparametric permutation test is one of the most popular methods to access the statistical significance of the classification performance (Golland and Fischl, 2003; Ojala and Garriga, 2010). However, traditional permutation test for accessing how likely the observed classification ACC would be obtained by chance usually determines the null distribution by permuting the labels in the data. Therefore, a modified permutation test procedure that was proposed by Ojala and Garriga (2010) was applied in the present study. In brief, a randomized version of training data was obtained by applying independent permutation to the elements of each feature of the original training data, and then the LOOCV was performed on the permutated training data to produce a classification accuracy (denoted as ACC_rand). After the above process was repeated N times (N = 10,000 in current study), an empirical distribution of the ACC_rand under null hypothesis was obtained. The p value of nonparametric permutation test was defined as the ratio of the number of ACC_rand in the null distribution that was larger than the ACC obtained on the original training data to the total number of permutations.

In addition, the comparison of connectivity strength between two groups was carried out using nonparametric Wilcoxon signed rank test with false discovery rate (FDR) correction for controlling multiple comparison problem (Genovese et al., 2002).

Results

Time-Frequency Analysis

The time-frequency analysis result showed that SZ patients exhibited obvious theta/alpha band desynchronization, which began to occur at about 0.2 s after stimulus onset (Figure 1A). Unlike SZ patients, HCs showed obvious theta band synchronization that occurred immediately after stimulus onset and alpha/beta band desynchronization that started to occur at about 0.2 s after stimulus onset (Figure 1B). Statistical comparison between the two groups indicated that theta band power of HCs was stronger compared with that of SZ during the time window from about 0.17–0.6 s after stimulus onset (Figure 1C, p = 0.003). Therefore, it was reasonable to compute PDC and PLI brain connectivity as classification features over the theta band and time period of 0.1, 0.6 s after stimulus onset in our current study.

FIGURE 1

Figure 1. Time-frequency power maps for (A) schizophrenia (SZ) and (B) healthy controls (HCs). Time-frequency power maps were averaged over two channels (F3, F4) and plotted as 10log10 change over baseline (from −0.2 to −0.1 s). (C) The result of the statistical comparison. Red color indicates time–frequency regions with significant difference between two groups, and blue color indicates no significant difference.

Brain Functional and Effective Connectivity Patterns

Figures 2A,B show the group average PLI functional connectivity pattern of SZ patients and HCs, respectively, and we qualitatively observed that the main difference between two groups was that SZ patients exhibited much less and much weaker long-range connections between the frontal/temporal cortexes and the visual cortex than did HCs. However, the PDC effective connectivity pattern seemed to reveal the abnormal connections of SZ patients from another aspect, and we found that HCs showed obvious long-range down-top information flows from the parietal to frontal regions (Figure 2D), whereas there was almost no such directional information transmission for SZ patients (Figure 2C).

FIGURE 2

Figure 2. Group average phase lag index (PLI) functional connectivity pattern for schizophrenia (SZ) patients (A) and healthy controls (HCs) (B), and sparsity was set to 0.3 to obtain a better illustration. Group average partial directed coherence (PDC) effective connectivity pattern for SZ patients (C) and HCs (D), and sparsity was set to 0.15 to obtain a better illustration. Color bar shows the connectivity strength. Brain connectivity was visualized with the BrainNet Viewer toolbox (http://www.nitrc.org/project/bnv/).

Classification Result

In the present study, three feature sets were tested in order to get a good discrimination between SZ patients and HCs: (1) only functional connectivity features, (2) only effective connectivity features, and (3) a combination of functional and effective connectivity features. We repeated the LOOCV with a varying number of features. Table 1 shows the obtained best performance with the corresponding feature numbers. As shown in Table 1, a good classification performance (SPE = 88.89%, SEN = 76.92%, AUC = 0.825, and ACC = 83.87%, p < 0.0001) was achieved with feature number of 6 when using only PLI functional connectivity feature set. A better classification performance (SPE = 86.11%, SEN = 88.46%, AUC = 0.909, and ACC = 87.97%, p < 0.0001) was obtained with feature number of 14 when using only PDC functional connectivity. However, we obtained the best classification performance (SPE = 94.44%, SEN = 96.15%, AUC = 0.952, and ACC = 95.16%, p < 0.0001) when using the combination of PLI and PDC connectivity features, and the corresponding selected feature number was 15, which included 12 PDC and three PLI connectivity features. The above results indicated that the combination of functional and effective connectivity features could provide a better discrimination between SZ patients and HCs than single functional or effective connectivity. The spatial distribution of the selected 15 connectivity features (12 PDC and three PLI connectivity features) is shown in Figure 3. As shown in Figure 3A, the 12 PDC connectivity features with the highest discriminative power mainly contained four kinds of effective connectivity pattern: (1) long-range “top-down” and “down-top” interactions between frontal cortex and occipital/parietal cortexes (F8→P7, F8→P8, P4→F3, O1→FP1, and Oz→F7); (2) top-down links from the temporal/central cortexes to the visual/parietal cortexes (T7→P7, T7→P8, Cz→O2, and C4→P4); (3) top-down information flow from the frontal cortex to the central cortex (F3→C4); and (4) local information flows in the frontal and parietal cortexes (F3→F4 and P4→P8). Interestingly, the three selected PLI connectivity features with the highest discriminative power showed an asymmetric right hemisphere lateralized pattern (Figure 3B) and mainly exhibited long-range connections between the frontal/temporal cortexes and the visual cortex (F8-Oz, T8-Oz, and T8-O2).

TABLE 1

Table 1. The obtained best classification performance (ACC, SPE, SEN, and AUC) with the corresponding feature numbers on three different feature sets.

FIGURE 3

Figure 3. The spatial distribution of the 12 partial directed coherence (PDC) (A) and three phase lag index (PLI) (B) connectivity features selected from the combined feature set when the classification performance was optimal. Color bar represents the discriminative power i.e., Fisher score.

Table 2 summarizes the values of the selected 12 PDC and three PLI connectivity features and the results of statistical comparison between SZ patients and HCs. As shown in Table 2, SZ patients exhibited reduced down-top information flows from the occipital and parietal cortexes to the frontal cortex (P4→F3, O1→FP1, and Oz→F7) than did HCs. As for other PDC effective connectivity features, SZ patients showed abnormally enhanced values as compared with HCs. More interestingly, we found that SZ patients exhibited significantly decreased phase synchronization between the frontal/temporal cortexes and the visual cortex (F8-Oz, T8-Oz, and T8-O2) than did HCs. The result of statistical comparison implied that all the 15 selected brain connectivity features had statistical difference between two groups, which proved that the proposed feature selection method in our study could reliably find the connectivity with high discriminative power.

TABLE 2

Table 2. The brain connectivity values of the selected 12 PDC and three PLI connectivity features and the result of statistical comparison between SZ patients and HCs.

Discussion

In this study, we proved that simultaneous application of brain functional and effective connectivity could greatly improve overall classification performance in the ML-based discrimination of SZ. An excellent classification ACC of 95.16% was obtained when the combination of functional and effective connectivity feature was used, which was much higher than either the functional connectivity feature set (83.87%) or the effective connectivity feature set (87.97%). Studying the 15 connectivity features (12 PDC and three PLI connectivity features) selected from the combined feature set implied that brain effective and functional connectivity could reveal altered brain connectivity pattern of SZ from different aspects: all the down-top information flows from the occipital/parietal cortexes to the frontal cortex (P4→F3, O1→FP1, and Oz→F7) were reduced, whereas other effective connectivity was enhanced in SZ patients compared with HCs, and all the three PLI functional connectivity was reduced in SZ patients.

Advantages of Combination of Functional and Effective Connectivity for Classification

Different from brain functional connectivity that refers to the statistical dependence between two brain regions’ electrophysiological signals and usually does not imply the existence of causal influence of one region on another, brain effective connectivity measures the directional information flow among different brain areas and can be computed by a method based on Granger causality. Therefore, functional or effective connectivity can only capture one different aspect of interdependence, and their combination could provide complementary information. Actually, some previous studies have proved this (Dauwels et al., 2010; Blinowska et al., 2017). For example, Dauwels et al. (2010) computed more than 30 EEG-based measures of functional and effective connectivity, and they found that many of those measures were strongly correlated and seemed to provide little complementary information about EEG synchrony. They further observed that only Granger causality measure and some functional connectivity measures including phase synchrony indices were mutually uncorrelated, and the combination of Granger causality measure and functional connectivity was used as features to distinguish mild cognitive impairment (MCI) patients from HCs, yielding a relatively high classification ACC of 83%. Blinowska et al. (2017) demonstrated that the use of both effective and functional connectivity could significantly increase classification performance of Alzheimer’s disease (AD) individuals from HCs as compared with when only effective or functional connectivity was used.

The PLI functional connectivity pattern analysis showed that SZ patients exhibited much less and much weaker long-range connections between the frontal/temporal cortexes and the visual cortex than did HCs (Figures 2A,B), while PDC effective connectivity pattern analysis implied that there were almost no long-range down-top information flows from the parietal to frontal regions for SZ patients compared with HCs (Figures 2C,D). Therefore, we believed that the combination of effective and functional connectivity could provide complementary information about the altered brain connectivity pattern of SZ patients. However, to the best of our knowledge, the combination of functional and effective connectivity has not been used as features for ML-based classification of SZ. Our further result based on SVM classification indicated that the simultaneous application of brain functional and effective connectivity could greatly improve overall classification performance in the ML-based discrimination of SZ, which was consistent with the results of previous studies (Dauwels et al., 2010; Blinowska et al., 2017).

Altered Theta-Band Functional and Effective Connectivity in Schizophrenia Patients

Our results of time-frequency analysis revealed that only theta-band oscillatory activity of HCs was significantly stronger than that of SZ patients after negative feedback stimuli (Figure 1). Actually, a large number of studies have found that increased theta activity was often observed in SZ patients during the rest state whereas reduced theta activity in SZ patients during various cognition tasks in recent years (Moran and Hong, 2011; Başar and Güntekin, 2013). Therefore, we have enough reasons to use theta-band effective and functional connectivity as features to classify SZ patients from HCs in this study.

Our classification result based on the SVM classifier finally selected 15 brain connectivity features (12 PDC and three PLI connectivity features) with the highest discriminative power (Figure 3 and Table 2). The 12 PDC effective connectivity features were mainly located between the frontal/temporal/central lobes and visual/parietal lobes (Figure 3A), and the three PLI functional connectivity features were mainly located between the frontal/temporal cortexes and visual cortex of the right hemisphere (Figure 3B). Li et al. (2019) used phase locking value (PLV) to study the EEG functional connectivity of SZ patients during a P300 task and observed that long-range functional connections including the frontal-occipital, temporal-occipital, and frontal-parietal connectivity were significantly reduced in SZ patients compared with HCs. Ganella et al. (2017) used functional magnetic resonance imaging (fMRI) technique and found that functional connectivity strength predominantly involving fronto-temporal, fronto-occipital, and temporo-occipital connections were reduced in SZ patients. These previous results seemed to be consistent with our findings that the selected three PLI functional connectivity including frontal-occipital and temporal-occipital connections were significantly decreased in SZ patients. However, for the effective connectivity, we did not found any similar researches that studied the abnormal effective connectivity pattern in SZ patients during a cognitive task. Therefore, similar studies from other groups were needed to evaluate the consistency of our results of the PDC effective connectivity in the future.

Compared With Previous Studies

The comparison of the classification result in our study with that of previous EEG studies is illustrated in Table 3. As shown in Table 3, it was obvious that the classification accuracy obtained in this work was better than that of other studies except the literature (Oh et al., 2019) that obtained an excellent classification accuracy of 98.07% by applying an 11-layered deep CNN model to extract features automatically. A direct comparison of the present classification results with those of previous related studies was difficulty because different classifiers were used in different studies. However, when compared with the studies that also used the SVM classifier (Bose et al., 2016; Santos-Mayo et al., 2016; Shim et al., 2016; Chu et al., 2017; Jahmunah et al., 2019; Li et al., 2019), it was evident that our study exhibited the highest accuracy. We believed that the excellent classification performance in our study was mainly due to the simultaneous use of effective and functional connectivity as features that could reveal the abnormal brain alterations of SZ more comprehensively.

TABLE 3

Table 3. Comparison of the classification accuracy of SZ patients with other previous studies.

Limitations and Future Directions

Although the classification performance in the present study using a combination of effective and functional connectivity is encouraging, there are still some limitations that should be considered in the future. Firstly, the sample size of subjects in the present study is relatively small, and therefore a large independent dataset should be used to test our methods and confirm the results. Secondly, all the SZ patients were on medication, and thus, we could not control for possible confounding effects of the drugs. Moreover, our method in the present study is effective by using “task-state” effective and functional connectivity as features and only applied for the discrimination of SZ; therefore, it is necessary to test our method on “resting-state” EEG data and on other psychiatric disorders to prove its generalizability in the future.

Conclusion

In this study, a combination of brain effective and functional connectivity was proposed for ML-based discrimination of SZ. Results indicated that brain effective and functional connectivity analysis could reveal the altered brain connectivity pattern of SZ from different aspects, and the simultaneous use of PDC effective and PLI functional connectivity features could reliably differentiate SZ patients from HCs with a high classification accuracy of 95.16%, a specificity of 94.44%, and a sensitivity of 96.15%, which was better than most of the previously reported results.

Data Availability Statement

The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found below: https://zenodo.org/record/29601.

Ethics Statement

The studies involving human participants were reviewed and approved by University of Maryland Institutional Review Board. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

ZZ carried out the effective and functional connectivity computation and ML-based classification and wrote the first draft of the manuscript. JL and CW carried out the time-frequency power analysis. YY, QR, and QY undertook the statistical analysis. YX and JZ modified the manuscript. YN helped answer the reviewer’s questions, modify the manuscript according to the reviewers’ comments and check the proof. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by the Doctoral Scientific Research Foundation of Xinxiang Medical University (No. XYBSKYZZ201718) and the Natural Science Foundation of Henan Province (No. 202300410323).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Akar, S., Kara, S., Latifoglu, F., and Bilgiç, V. (2015). Estimation of nonlinear measures of schizophrenia patients’ EEG in emotional states. IRBM 36, 250–258. doi: 10.1016/j.irbm.2015.06.005

CrossRef Full Text | Google Scholar

Akar, S. A., Kara, S., Latifoğlu, F., and Bilgiç, V. (2016). Analysis of the complexity measures in the EEG of schizophrenia patients. Int. J. Neural Syst. 26, 263–274.

Google Scholar

Albrecht, M. A., Waltz, J. A., Cavanagh, J. F., Frank, M. J., and Gold, J. M. (2016). Reduction of pavlovian bias in schizophrenia: enhanced effects in clozapine-administered patients. PLos One 11:e0152781. doi: 10.1371/journal.pone.0152781

PubMed Abstract | CrossRef Full Text | Google Scholar

Andreas, S., Theisen, P., Mestel, R., Koch, U., and Schulz, H. (2009). Validity of routine clinical DSM-IV diagnoses (Axis I/II) in inpatients with mental disorders. Psychiatry Res. 170:255.

Google Scholar

Andreou, C., Leicht, G., Nolte, G., Polomac, N., Moritz, S., Karow, A., et al. (2015a). Resting-state theta-band connectivity and verbal memory in schizophrenia and in the high-risk state. Schizophrenia Res. 161, 299–307. doi: 10.1016/j.schres.2014.12.018

PubMed Abstract | CrossRef Full Text | Google Scholar

Andreou, C., Nolte, G., Leicht, G., Polomac, N., Hanganu-Opatz, I. L., Lambert, M., et al. (2015b). Increased resting-state gamma-band connectivity in first-episode schizophrenia. Schizophrenia Bull. 41, 930–939. doi: 10.1093/schbul/sbu121

PubMed Abstract | CrossRef Full Text | Google Scholar

Astolfi, L., Cincotti, F., Mattia, D., Babiloni, C., Carducci, F., Basilisco, A., et al. (2005). Assessing cortical functional connectivity by linear inverse estimation and directed transfer function: simulations and application to real data. Clin. Neurophysiol. 116, 920–932. doi: 10.1016/j.clinph.2004.10.012

PubMed Abstract | CrossRef Full Text | Google Scholar

BaAr, E., Schmiedt-Fehr, C., Mathes, B., Femir, B., Emek-Savaş, D. D., Tülay, E., et al. (2015). What does the broken brain say to the neuroscientist? Oscillations and connectivity in schizophrenia, Alzheimer\”s disease, and bipolar disorder. Int. J. Psychophysiol. 103, 135–148. doi: 10.1016/j.ijpsycho.2015.02.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Başar, E., and Güntekin, B. (2013). Review of delta, theta, alpha, beta, and gamma response oscillations in neuropsychiatric disorders. Suppl. Clin. Neurophysiol. Elsevier. 62, 303–341. doi: 10.1016/b978-0-7020-5307-8.00019-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Blinowska, K. J., Rakowski, F., Kaminski, M., De Vico Fallani, F., Del Percio, C., Lizio, R., et al. (2017). Functional and effective brain connectivity for discrimination between Alzheimer’s patients and healthy individuals: a study on resting state EEG rhythms. Clin. Neurophysiol. 128, 667–680. doi: 10.1016/j.clinph.2016.10.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Bose, T., Sivakumar, S. D., and Bhanu, K. (2016). Identification of schizophrenia using EEG alpha band power during hyperventilation and post-hyperventilation. J. Med. Biol. Eng. 36, 901–911. doi: 10.1007/s40846-016-0192-2

CrossRef Full Text | Google Scholar

Chang, C.-C., and Lin, C.-J. (2011). LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. (TIST) 2:27.

Google Scholar

Chu, W. L., Huang, M.-W., Jian, B. L., and Cheng, K. S. (2017). Analysis of EEG entropy during visual evocation of emotion in schizophrenia. Ann. Gene. Psychiatry 16:34.

Google Scholar

Chyzhyk, D., Gra?a, M., Öngür, D., and Shinn, A. K. (2015). Discrimination of schizophrenia auditory hallucinators by machine learning of resting-state functional MRI. Int. J. Neural Syst. 25:1550007. doi: 10.1142/s0129065715500070

PubMed Abstract | CrossRef Full Text | Google Scholar

Consortium, S. W. G. o. t. P. G. (2014). Biological insights from 108 schizophrenia-associated genetic loci. Nature 511, 421–427. doi: 10.1038/nature13595

PubMed Abstract | CrossRef Full Text | Google Scholar

Dauwels, J., Vialatte, F., Musha, T., and Cichocki, A. (2010). A comparative study of synchrony measures for the early diagnosis of Alzheimer’s disease based on EEG. NeuroImage 49, 668–693. doi: 10.1016/j.neuroimage.2009.06.056

PubMed Abstract | CrossRef Full Text | Google Scholar

Devia, C., Mayol-Troncoso, R., Parrini, J., Orellana, G., Ruiz, A., Maldonado, P. E., et al. (2019). EEG classification during scene free-viewing for schizophrenia detection. IEEE Trans. Neural. Syst. Rehabil. Eng. 27, 1193–1199. doi: 10.1109/tnsre.2019.2913799

PubMed Abstract | CrossRef Full Text | Google Scholar

Di, L. G., Andrea, D., Ferrentino, F., Santarnecchi, E., Ciabattini, F., Monaco, L., et al. (2015). Altered resting-state EEG source functional connectivity in schizophrenia: the effect of illness duration. Front. Hum. Neurosci. 9:234.

Google Scholar

Ding, M., Bressler, S. L., Yang, W., and Liang, H. (2000). Short-window spectral analysis of cortical event-related potentials by adaptive multivariate autoregressive modeling: data preprocessing, model validation, and variability assessment. Biol. Cybernetics 83, 35–45. doi: 10.1007/s004229900137

PubMed Abstract | CrossRef Full Text | Google Scholar

Doesburg, S. M., Julie, V., and Taylor, M. J. (2013). Reduced theta connectivity during set-shifting in children with autism. Front. Hum. Neurosci. 7:785.

Google Scholar

Ganella, E. P., Bartholomeusz, C. F., Seguin, C., Whittle, S., Bousman, C., Phassouliotis, C., et al. (2017). Functional brain networks in treatment-resistant schizophrenia. Schizophrenia Res. 184, 73–81. doi: 10.1016/j.schres.2016.12.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Genovese, C. R., Lazar, N. A., and Nichols, T. (2002). Thresholding of statistical maps in functional neuroimaging using the false discovery rate. NeuroImage 15, 870–878. doi: 10.1006/nimg.2001.1037

PubMed Abstract | CrossRef Full Text | Google Scholar

Golland, P., and Fischl, B. (2003). “Permutation tests for classification: towards statistical significance in image-based studies,” in Proceeding of the Biennial International Conference on Information Processing in Medical Imaging, (Springer).

Google Scholar

Gomez-Pilar, J., Lubeiro, A., Poza, J., Hornero, R., Ayuso, M., Valcárcel, C., et al. (2017). Functional EEG network analysis in schizophrenia: evidence of larger segregation and deficit of modulation. Prog. Neuro Psychopharmacol. Biol. Psychiatry 76, 116–123. doi: 10.1016/j.pnpbp.2017.03.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Hamilton, H. K., Williams, T. J., Ventura, J., Jasperse, L. J., Owens, E. M., Miller, G. A., et al. (2017). Clinical and cognitive significance of auditory sensory processing deficits in schizophrenia. Am. J. Psychiatry 175, 275–283. doi: 10.1176/appi.ajp.2017.16111203

PubMed Abstract | CrossRef Full Text | Google Scholar

Haro, J. M., Altamura, C., Corral, R., Elkis, H., Evans, J., Malla, A., et al. (2014). Understanding the impact of persistent symptoms in schizophrenia: cross-sectional findings from the pattern study. Schizophrenia Res. 24:S507.

Google Scholar

Hirano, Y., Oribe, N., Kanba, S., Onitsuka, T., Nestor, P. G., and Spencer, K. M. (2015). Spontaneous gamma activity in schizophrenia. JAMA Psychiatry 72, 813–821. doi: 10.1001/jamapsychiatry.2014.2642

PubMed Abstract | CrossRef Full Text | Google Scholar

Huang, D., Aifeng, R., Shang, J., Lei, Q., Zhang, Y., Yin, Z., et al. (2016). Combining partial directed coherence and graph theory to analyse effective brain networks of different mental tasks. Front. Hum. Neurosci. 10:235.

Google Scholar

Hunt, M. J., Kopell, N. J., Traub, R. D., and Whittington, M. A. (2017). Aberrant network activity in schizophrenia. Trends Neurosci. 40, 371–382. doi: 10.1016/j.tins.2017.04.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Hwang, H.-J., Lim, J.-H., Kim, D. W., and Im, C. H. (2014). Evaluation of various mental task combinations for near-infrared spectroscopy-based brain-computer interfaces. J. Biomed. Optics 19:077005. doi: 10.1117/1.jbo.19.7.077005

CrossRef Full Text | Google Scholar

Jahmunah, V., Oh, S. L., Rajinikanth, V., Ciaccio, E. J., Cheong, K. H., Arunkumar, N., et al. (2019). Automated detection of schizophrenia using nonlinear signal processing methods. Artif. Intell. Med. 100:101698. doi: 10.1016/j.artmed.2019.07.006

PubMed Abstract | CrossRef Full Text | Google Scholar

James, A., Joyce, E., Lunn, D., Hough, M., Kenny, L., Ghataorhe, P., et al. (2016). Abnormal frontostriatal connectivity in adolescent-onset schizophrenia and its relationship to cognitive functioning. Eur. Psychiatry 35, 32–38. doi: 10.1016/j.eurpsy.2016.01.2426

PubMed Abstract | CrossRef Full Text | Google Scholar

Ji, G.-J., Chen, X., Bai, T., Wang, L., Wei, Q., Gao, Y., et al. (2019). Classification of schizophrenia by intersubject correlation in functional connectome. Hum. Brain Mapp. 40, 2347–2357. doi: 10.1002/hbm.24527

PubMed Abstract | CrossRef Full Text | Google Scholar

Kam, J. W. Y., Bolbecker, A. R., O’Donnell, B. F., Hetrick, W. P., and Brenner, C. A. (2013). Resting state EEG power and coherence abnormalities in bipolar disorder and schizophrenia. J. Psychiat. Res. 47, 1893–1901. doi: 10.1016/j.jpsychires.2013.09.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Khandoker, A. H., Marzbanrad, F., Voss, A., Schulz, S., Kimura, Y., Endo, M., et al. (2019). Analysis of maternal–fetal heart rate coupling directions with partial directed coherence. Biomed. Signal Proc. Control 30, 25–30. doi: 10.1016/j.bspc.2016.06.010

CrossRef Full Text | Google Scholar

Kim, J. W., Lee, Y. S., Han, D. H., Min, K. J., Lee, J., and Lee, K. (2015). Diagnostic utility of quantitative EEG in un-medicated schizophrenia. Neurosci. Lett. 589, 126–131. doi: 10.1016/j.neulet.2014.12.064

PubMed Abstract | CrossRef Full Text | Google Scholar

Kubera, K. M., Thomann, P. A., Hirjak, D., Barth, A., Sambataro, F., Vasic, N., et al. (2017). Cortical folding abnormalities in patients with schizophrenia who have persistent auditory verbal hallucinations. Eur. Neuropsychopharmacol. 28, 297–306. doi: 10.1016/j.euroneuro.2017.12.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, F., Wang, J., Liao, Y., Yi, C., Jiang, Y., Si, Y., et al. (2019). Differentiation of schizophrenia by combining the spatial EEG brain network patterns of rest and task P300. IEEE Trans. Neural Syst. Rehabil. Eng. 27, 594–602. doi: 10.1109/tnsre.2019.2900725

PubMed Abstract | CrossRef Full Text | Google Scholar

Maris, E., and Oostenveld, R. (2007). Nonparametric statistical testing of EEG-and MEG-data. J. Neurosci. Methods 164, 177–190. doi: 10.1016/j.jneumeth.2007.03.024

PubMed Abstract | CrossRef Full Text | Google Scholar

Moghimi, P., Lim, K. O., and Netoff, I. (2018). Data driven classification of fMRI network measures: application to schizophrenia. Front. Neuroinf. 12:71.

Google Scholar

Moran, L. V., and Hong, L. E. (2011). High vs low frequency neural oscillations in schizophrenia. Schizophrenia Bull. 37, 659–663. doi: 10.1093/schbul/sbr056

PubMed Abstract | CrossRef Full Text | Google Scholar

Morf, M., Vieira, A., Lee, D. T. L., and Kailath, T. (1978). Recursive multichannel maximum entropy spectral estimation. IEEE Trans. Geosci. Electron. 16, 85–94. doi: 10.1109/tge.1978.294569

CrossRef Full Text | Google Scholar

Morris, S. E., Heerey, E. A., Gold, J. M., and Holroyd, C. B. (2008). Learning-related changes in brain activity following errors and performance feedback in schizophrenia. Schizophrenia Res. 99, 274–285. doi: 10.1016/j.schres.2007.08.027

PubMed Abstract | CrossRef Full Text | Google Scholar

Neuhaus, A. H., Popescu, F. C., Bates, J. A., Goldberg, T. E., and Malhotra, A. K. (2013). Single-subject classification of schizophrenia using event-related potentials obtained during auditory and visual oddball paradigms. Eur. Arch. Psychiatry Clin. Neurosci. 263, 241–247. doi: 10.1007/s00406-012-0326-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Nurjono, M., Lee, J., and Chong, S. A. (2012). A review of brain-derived neurotrophic factor as a candidate biomarker in schizophrenia. Clin. Psychopharmacol. Neurosci. 10, 61–70. doi: 10.9758/cpn.2012.10.2.61

PubMed Abstract | CrossRef Full Text | Google Scholar

Oh, S. L., Vicnesh, J., Ciaccio, E. J., Rajamanickam, Y., and Acharya, U. R. (2019). Deep convolutional neural network model for automated diagnosis of schizophrenia using EEG signals. Appl. Sci. 9:2870. doi: 10.3390/app9142870

CrossRef Full Text | Google Scholar

Ojala, M., and Garriga, G. C. (2010). Permutation tests for studying classifier performance. J. Mach. Learn. Res. 11, 1833–1863.

Google Scholar

Olejarczyk, E., and Jernajczyk, W. (2017). Graph-based analysis of brain connectivity in schizophrenia. PLos One 12:e0188629. doi: 10.1371/journal.pone.0188629

PubMed Abstract | CrossRef Full Text | Google Scholar

Pergola, G., Trizio, S., Di Carlo, P., Taurisano, P., Mancini, M., Amoroso, N., et al. (2016). Grey matter volume patterns in thalamic nuclei are associated with familial risk for schizophrenia. Schizophrenia Res. 180, 13–20. doi: 10.1016/j.schres.2016.07.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Phang, C.-R., Ting, C.-M., Numan, F., and Ombao, H. (2019). Classification of EEG-based brain connectivity networks in schizophrenia using a multi-domain connectome convolutional neural network. arXiv [preprint] arXiv:1903.08858,

Google Scholar

Phillips, K. G., and Uhlhaas, P. J. (2015). Neural oscillations as a translational tool in schizophrenia research: rationale, paradigms and challenges. J. Psychopharmacol. 29, 155–168. doi: 10.1177/0269881114562093

PubMed Abstract | CrossRef Full Text | Google Scholar

Santos-Mayo, L., San-José-Revuelta, L. M., and Arribas, J. I. (2016). A computer-aided diagnosis system with EEG based on the P3b wave during an auditory odd-ball task in schizophrenia. IEEE Trans. Biomed. Eng. 64, 395–407. doi: 10.1109/tbme.2016.2558824

PubMed Abstract | CrossRef Full Text | Google Scholar

Shim, M., Hwang, H.-J., Kim, D. W., Lee, S. H., and Im, C. H. (2016). Machine-learning-based diagnosis of schizophrenia using combined sensor-level and source-level EEG features. Schizophrenia Res. 176, 314–319. doi: 10.1016/j.schres.2016.05.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Silfverhuth, M. J., Hintsala, H., Kortelainen, J., and Seppänen, T. (2012). Experimental comparison of connectivity measures with simulated EEG signals. Med. Biol. Eng. Comput. 50, 683–688. doi: 10.1007/s11517-012-0911-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Stam, C. J., Nolte, G., and Daffertshofer, A. (2007). Phase lag index: assessment of functional connectivity from multi channel EEG and MEG with diminished bias from common sources. Hum. Brain Mapp. 28, 1178–1193. doi: 10.1002/hbm.20346

PubMed Abstract | CrossRef Full Text | Google Scholar

Sun, Y., Lim, J., Kwok, K., and Bezerianos, A. (2014a). Functional cortical connectivity analysis of mental fatigue unmasks hemispheric asymmetry and changes in small-world networks. Brain Cogn. 85, 220–230. doi: 10.1016/j.bandc.2013.12.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Sun, Y., Lim, J., Meng, J., Kwok, K., Thakor, N., and Bezerianos, A. (2014b). Discriminative analysis of brain functional connectivity patterns for mental fatigue classification. Ann. Biomed. Eng. 42, 2084–2094. doi: 10.1007/s10439-014-1059-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Van, O. J., and Kapur, S. (2009). Schizophrenia. Lancet 374, 359–372.

Google Scholar

Wang, J., Hirayasu, Y., Hiramatsu, K., Hokama, H., Miyazato, H., Ogura, C., et al. (2003). Increased rate of P300 latency prolongation with age in drug-naive and first episode schizophrenia. Clin. Neurophysiol. Official J. Int. Federation Clin. Neurophysiol. 114, 2029–2035. doi: 10.1016/s1388-2457(03)00207-4

CrossRef Full Text | Google Scholar

Wang, L., Wenhui, W., Yan, T., Song, J., Yang, W., Wang, B., et al. (2017). Beta-band functional connectivity influences audiovisual integration in older age: an EEG study. Front. Aging Neurosci. 9:239.

Google Scholar

Woodward, M. L., Gicas, K. M., Warburton, D. E., White, R. F., and Rauscher, A. (2018). Hippocampal volume and vasculature before and after exercise in treatment-resistant schizophrenia. Schizophrenia Res. 202, 158–165. doi: 10.1016/j.schres.2018.06.054

PubMed Abstract | CrossRef Full Text | Google Scholar

Yu, Y., Zhao, Y., Si, Y., Ren, Q., Ren, W., Jing, C., et al. (2016). Estimation of the cool executive function using frontal electroencephalogram signals in first-episode schizophrenia patients. Biomed. Eng. Online 15:131.

Google Scholar

Zhang, H., Chavarriaga, R., and del R Millán, J. (2015). Discriminant brain connectivity patterns of performance monitoring at average and single-trial levels. NeuroImage 120, 64–74.

Google Scholar

Zhao, Z., Cheng, Y., Li, Z., and Yu, Y. (2018). Altered small-world networks in first-episode schizophrenia patients during cool executive function task. Behav. Neurol. 2018:2191208.

Google Scholar

Zhao, Z., Wang, C., Yuan, Q., Zhao, J., Ren, Q., Xu, Y., et al. (2020). Dynamic changes of brain networks during feedback-related processing of reinforcement learning in schizophrenia. Brain Res. 1746:146979.

Google Scholar