Combination of resting state fMRI, DTI, and sMRI data to discriminate schizophrenia by N-way MCCA + jICA

Multimodal brain imaging data have shown increasing utility in answering both scientifically interesting and clinically relevant questions. Each brain imaging technique provides a different view of brain function or structure, while multimodal fusion capitalizes on the strength of each and may uncover hidden relationships that can merge findings from separate neuroimaging studies. However, most current approaches have focused on pair-wise fusion and there is still relatively little work on N-way data fusion and examination of the relationships among multiple data types. We recently developed an approach called “mCCA + jICA” as a novel multi-way fusion method which is able to investigate the disease risk factors that are either shared or distinct across multiple modalities as well as the full correspondence across modalities. In this paper, we applied this model to combine resting state fMRI (amplitude of low-frequency fluctuation, ALFF), gray matter (GM) density, and DTI (fractional anisotropy, FA) data, in order to elucidate the abnormalities underlying schizophrenia patients (SZs, n = 35) relative to healthy controls (HCs, n = 28). Both modality-common and modality-unique abnormal regions were identified in SZs, which were then used for successful classification for seven modality-combinations, showing the potential for a broad applicability of the mCCA + jICA model and its results. In addition, a pair of GM-DTI components showed significant correlation with the positive symptom subscale of Positive and Negative Syndrome Scale (PANSS), suggesting that GM density changes in default model network along with white-matter disruption in anterior thalamic radiation are associated with increased positive PANSS. Findings suggest the DTI anisotropy changes in frontal lobe may relate to the corresponding functional/structural changes in prefrontal cortex and superior temporal gyrus that are thought to play a role in the clinical expression of SZ.

Introduction

Multimodal brain imaging techniques are playing increasingly important roles in elucidating structural and functional properties in normal and diseased brains, as well as providing the conceptual glue to bind together data from multiple types or levels of analysis. The related computational methods are also valuable for clinical research on the mechanisms of disease progression. The goal of multimodal fusion is to capitalize on the strength of each imaging modality as well as their inter-relationships in a joint analysis, rather than to analyze separately.

Each imaging modality provides a different view of brain function or structure, and data fusion capitalizes on the strengths of each imaging modality/task and their inter-relationships in a joint analysis, creating an important tool to help unravel the black box of psychotic disorders, such as schizophrenia (SZ) (Calhoun et al., 2006; Sui et al., 2012a). Recent advances in data fusion include integrating multiple (task) fMRI data sets (Sui et al., 2009b, 2010; Kim et al., 2010) from the same participant to specify common versus specific sources of activity to a greater degree than traditional general linear model-based approaches. This can increase confidence when making conclusions about the functional significance of brain regions and activation changes in brain diseases. In addition, the combination of function and structure may provide more informative insights into both altered brain patterns and connectivity (McCarley et al., 2008; Michael et al., 2010; Sui et al., 2011). For example, a lower and different function–structure connection is often found in patients with SZs compared with healthy controls (HCs) (Zhou et al., 2008; Venkataraman et al., 2010; Camchong et al., 2011; Michael et al., 2011), while varied brain patterns are also identified frequently (Calhoun et al., 2008; Xu et al., 2009; Brown et al., 2012; Lu et al., 2012).

Why Go beyond Two Modalities?

However, most current approaches have focused on pair-wise fusion and there is still relatively little work on N-way data fusion and examination of the full relationships among multiple data types. Given the availability of more powerful MR scanners, there are typically more than two imaging modalities available for one participant. Hence, we believe the joint multivariate analysis of multiple data types (e.g., resting state fMRI, task-related fMRI, DTI, and sMRI) will improve our ability to understand brain diseases. We have proposed an N-way fusion model, “multi-set canonical correlation analysis (mCCA) + joint independent component analysis,” i.e., “mCCA + jICA,” which successfully identified both modal-common and modal-unique group-discriminative patterns for HCs and SZs via combination of task-related fMRI, DTI, and sMRI data (Sui et al., 2013). Considering the importance of the interpretation of multi-way features, the method and tool we propose will enable examination of full correspondence across N modalities by achieving reliable inter-modality associations and high decomposition accuracy together, thus making discoveries of changes in one modality causing related alterations in distant, but connected regions in other modalities possible.

To our knowledge, there have been only a few reports combining three or more types of brain imaging data to investigate brain disorders (e.g., Correa et al., 2009) examined changes that are related across fMRI, sMRI, and EEG data for SZ (Groves et al., 2011) compared Alzheimer’s patients and age-matched controls by combining gray matter (GM) density and three diffusion data measures [fractional anisotropy (FA), mean diffusivity, and tensor mode]. For resting state fMRI data, several pair-wise fusion applications have been reported (Teipel et al., 2010; Long et al., 2012; Segall et al., 2012); however, there has been no report that combine resting state fMRI with other two or more different types of brain imaging data to study SZ.

In this project, we applied the N-way fusion model, “mCCA + jICA” (Sui et al., 2013), to compare not only modality-common but also modality-unique abnormalities among resting state fMRI, sMRI, and DTI data, which is the first attempt to combine such three types of data to discriminate SZ patients (n = 35) from HCs (n = 28). N-way fusion of brain imaging data is more challenging than pair-wise combination, since many fusion applications rely on studying correlations between highly distilled measures (e.g., small regions of interest), while there is still relatively little examination of the full relationships among data types. The method and tools we propose will enable such an examination and can be potentially useful for identification of unique biomarkers of brain disorders. Furthermore, the high-dimensional neuroimaging data is typically very noisy and massive redundancy reduction is usually necessary to facilitate the identification of relationships among modalities. For this purpose, each modality is first reduced to a “feature” for each subject, which tends to be more tractable than working with the large-scale original data (Calhoun and Adali, 2009) and provides a simpler space to link the data (Smith et al., 2009), e.g., an fMRI contrast map from the general linear model, a GM segmentation image from the sMRI scan and voxel-wise DTI measures such as FA. For resting state fMRI data, we used the amplitude of low-frequency fluctuation (ALFF) as fusion input (Zang et al., 2007; Zou et al., 2008; Calhoun and Allen, 2013), which has been used previously for default mode or other applications in multiple papers (Calhoun et al., 2012; Turner et al., 2012; Yu et al., 2012b, 2013).

Materials and Methods

Theory Development

Existing multivariate fusion methods have different optimization priorities and limitations: some enable common as well as distinct levels of connection among modalities, such as mCCA (Correa et al., 2009) and partial least squares (PLS) (Lin et al., 2003; Chen et al., 2009), but their separated sources may not be sufficiently spatially sparse. For example, mCCA maximizes the inter-subject covariation across two sets of features and generates two linked variables, one from each dataset, i.e., canonical variants (CVs); which correlate with each other only on the same indices (rows) and their corresponding correlation values are called canonical correlation coefficients (CCC). This strategy allows for both common and distinct aspects of two features, but the brain maps of several components may look similar when the CCCs are not sufficiently distinct. Some approaches perform well in spatial decomposition, such as jICA (Calhoun et al., 2006) and linked ICA (Groves et al., 2012), which aim at maximizing the independence among estimated sources combining more than two modalities, but only allow a common mixing matrix. These two methods enable detection of features common to all modalities at the expense of features which may be distinct to one or more of them (a situation which becomes more likely when combining more than two modalities). Multiple previous studies that combined function and structure (Olesen et al., 2003; Rykhlevskaia et al., 2008; Camara et al., 2010; Sui et al., 2012b) provide support for the assumption that components decomposed from each modality have some degree of correlation between their mixing profiles among subjects. This motivates our data-driven model that is optimized for both flexibility in inter-modal associations and high capability on source separation.

The basic strategy of mCCA + jICA is shown in Figure 1. MCCA is first adopted to project the data in a space so that the correlations among mixing profiles (D _k, k = 1, 2, …, n) of n (n = 3 in this study) modalities are jointly maximized (in their sum of squared correlations). The resulting CVs D _k are sorted by correlation which provides a closer initial match to the potential highly or weakly correlated mixing profiles between components, which will make the subsequent application of jICA more reliable. At this time, the associated maps C _k may not be completely separated by mCCA. We then apply jICA on the concatenated maps (C_1,C_2,…, C _n) to obtain the final maximally independent source S _k. In other words, mCCA first relates multiple datasets with flexible linkages (correlation) in their mixing matrices, which matches well with the assumptions of jICA that is subsequently applied to the joint spatial maps. Hence, mCCA and jICA are complementary to one another, and can relax the limitations of each listed above if used together, generating both highly and weakly correlated joint components that are independent.

FIGURE 1

Figure 1. N-way mCCA + jICA fusion strategy of for real human data (n = 3 in this study).

We assume that the multimodal dataset X _k, is a linear mixture of M_k sources given by S _k, mixed with a non-singular mixing matrix (or loading parameters) A _k for each, k denotes modality.

$X_k=A_k{S}_{k}k=1,2,\dots ,n$ (1)

where X _k is a subjects-by-voxels feature matrix (we use voxels for our description but it could also be, e.g., time points or genes). The sources S _k, are distinct within each dataset, while the columns of A _i and A _j have higher correlation only on their corresponding indices, i, j ∈ {1, 2, …, n} i ≠ j are modality number. Given that there are N subjects, typically, the number of voxels L in X _k is much larger than N. Due to the high dimensionality and high noise levels in the brain imaging data, order selection is critical to avoid over fitting the data. Using the improved minimum description length (MDL) criterion (Li et al., 2007), the number of independent components M_k are estimated for each modality and we set the final component number for jICA as M = max(M₁, M₂, …, M_n). Dimension reduction is then performed on X _k using singular value decomposition to determine the signal subspace given by

$Y_k=X_k{E}_{k}k=1,2,\dots ,n$ (2)

where Y _k is in size of N × M and E _k contains eigenvectors corresponding to significant (the top M highest) singular values. Multi-set CCA (Li et al., 2009) is thus performed on Y _k, generating the CVs D _k = Y _k w _k by maximizing the sum-of-squares of all correlation values in the corresponding columns of D _k so that

$E\left\{D_k^T{D}_k\right\}=I;E\left\{D_k^T{D}_j\right\}\approx \mathrm{diag}\left(r_{k,j}^{\left(1\right)},r_{k,j}^{\left(2\right)}...r_{k,j}^{\left(M\right)}\right)$ (3)

where k, j ∈ {1, 2, …, n}, k ≠ j. Based on the linear mixture model, we simultaneously obtain the associated components C _k via X _k = D _k⋅C _k, C _k = pinv(D _k)⋅X _k. However, the performance of mCCA for blind source separation (BSS) may suffer when $r_{k,j}^{\left(1\right)},r_{k,j}^{\left(2\right)}...r_{k,j}^{\left(M\right)}$ are very close in values, which might occur in applications using real brain data, since the multimodal connection among components usually are not very high and could be similar in value (Sui et al., 2011). Therefore, C _k will typically be a set of sources that are not completely independent. Joint ICA is then implemented on the concatenated maps (C₁, C_2,…, C _n), to maximize the independence among joint components by reducing their second and higher order statistical dependencies, as in Eq. (4). ICA as a central tool for BSS has been studied extensively and we utilized Infomax (Bell and Sejnowski, 1995) in our work due to its good stability.

$\left[S_1,S_2...S_n\right]=W\cdot \left[C_1,C_2...C_n\right]$ (4)

Finally, n sets of independent components S _k are achieved, with their corresponding mixing matrices A _k linked via correlation. The proposed scheme “mCCA + jICA” can be summarized as shown in Figure 1.

$X_k=\left(D_k\cdot W^{-1}\right)\cdot S_k,A_k=D_k\cdot W^{-1}$ (5)

Multi-set canonical correlation analysis + jICA was compared with its alternatives in simulation in Sui et al. (2013), where results show that combination of mCCA and jICA mitigates the performance deficits of each and achieves more reliable and better separation on both sources and mixing matrices. Interestingly, when the estimated component number is higher than the ground truth, the source estimation performance continues to be high, while the estimation of mixing coefficients achieves best performance when M equals to true values.

Human Brain Data

Participants

Multi-set canonical correlation analysis + jICA was applied to DTI, resting state fMRI, and sMRI data of 63 subjects recruited as part of a multimodal SZ center for biomedical research excellence (COBRE) study at the Mind Research Network¹. Informed consent was obtained from all subjects according to institutional guidelines required by the Institutional Review Board at the University of New Mexico (UNM). Table 1 lists the demographic information. All subjects were screened and excluded if they had history of neurological disorder, history of mental retardation, history of severe head trauma with more than 5 min loss of consciousness, or history of substance abuse, or dependence within the last 12 months (except for nicotine). HCs were free from any Axis I disorder, as assessed with the SCID-NP (Structured Clinical Interview for DSM-IV-TR, Non-patient version). Patients met criteria for SZ defined by the DSM-IV-TR based on the SCID-P interview (First et al., 1995). All patients were on stable medication prior to the fMRI scan session. The two groups did not differ with regard to age, gender, and ethnicity, see Table 1. Symptom scores were determined based on the positive and negative syndrome scale (PANSS) (Kay et al., 1987).

TABLE 1

Table 1. Demographic information of the subjects.

Imaging parameters

All the data were collected on a 3-T Siemens Trio scanner with a 12-channel radio frequency coil at the Mind Research Network. The imaging parameters were as follows: fMRI: resting state data were collected with single-shot full k-space echo-planar imaging (EPI) with ramp sampling correction using the inter commissural line (AC/PC) (anterior commissure/posterior commissure) as a reference (TR = 2 s, TE = 29 ms, matrix size = 64 × 64, flip angle = 75 °, slice thickness = 3.5 mm, slice gap = 1.05 mm, field of view (FOV) 240 mm, matrix size = 64 × 64, voxel size = 3.75 mm × 3.75 mm × 4.55 mm. sMRI: a multi-echo MPRAGE sequence was used with the following parameters: TR/TE/TI = 2530/(1.64, 3.5, 5.36, 7.22, 9.08)/900 ms, flip angle = 7 °, FOV = 256 × 256 mm, slab thickness = 176 mm, matrix size = 256 × 256 × 176, Voxel size = 1 mm × 1 mm × 1 mm, Pixel bandwidth = 650 Hz, Total scan time = 6 min. DTI: data was collected along the AC/PC line, throughout the whole brain, FOV = 256 × 256 mm, slice thickness = 2 mm, NEX (number of excitations) = 1, TE = 84 ms, TR = 9,000 ms. A multiple channel radio frequency coil was used, with GRAPPA (generalized autocalibrating partially parallel acquisition) (×2), 30 gradient directions with a diffusion sensitivity, b = 800 s/mm². The b = 0 experiment was repeated five times, and equally inter-spread between the 30 gradient directions. All b = 0 images were registered to the first b = 0 image with a six degrees-of-freedom transformation. This was followed by registering the b = 800 s/mm² image to the b = 0 image immediately before it by an affine 12 degrees-of-freedom transformation. The two transformations were multiplied and then one transformation applied to the b = 800 s/mm² image to align it to the first b = 0 image. This resulted in all images being registered to the first b = 0 image. FLIRT (FMRIB’s Linear Image Registration Tool) was used for all registration steps.

Resting state fMRI

Resting-state scans were a minimum of 5 min, 4 s in duration (152 volumes). Subjects were instructed to keep their eyes open during the scan and stare passively at a foveally presented fixation cross, as this is suggested to facilitate network delineation compared to eyes-closed conditions and helps ensure that subjects are awake.

fMRI preprocessing

SPM8 software package² was employed to perform fMRI preprocessing. Slice timing was performed with the middle slice as the reference frame. Images were realigned using INRIalign, a motion correction algorithm that is unbiased by local signal changes (Freire et al., 2002). Data were then spatially normalized into the standard Montreal Neurological Institute (MNI) space (Friston et al., 1995) with affine transformation followed by a non-linear approach with 4 × 5 × 4 basis functions. Images (originally collected at 3.75 mm × 3.75 mm × 4.55 mm) were then slightly upsampled to 3 mm × 3 mm × 3 mm, resulting in a data cube of 53 × 63 × 46 voxels. Before smoothing, we further regress out the six motion parameters for each slice to remove the motion effect. Finally, data were spatially smoothed with a Gaussian kernel of full-width half maximum (FWHM) of 10 mm × 10 mm × 10 mm. For the rest fMRI, we extracted the voxel-wise ALFF to generate a map for each subject. The ALFF calculation consisted of computing the fast Fourier transform (FFT) of each voxel time series, taking the square root of the power spectrum to obtain amplitude, and averaging amplitude in (0.01, 0.1) Hz. Prior to computing ALFF, the original 4D fMRI data sets were divided by their global mean (over time and space) to normalize differences in scan intensity units. ALFF maps computed in this manner were used previously in a comparative classification analysis (Erhardt et al., 2011) and the use of ALFF maps in a “second-level” ICA has been previously studied (Calhoun and Allen, 2013).

DTI preprocessing

DTI data were preprocessed by FMRIB Software Library (FSL)³ and consisted of the following steps: (a) quality check, any gradient directions with excessive motion or vibration artifacts were identified and removed; (b) motion and eddy current correction; (c) correction of gradient directions for any image rotation done during the previous motion correction step; (d) calculation of diffusion tensor and scalar measures such as FA, which were then smoothed and resized to a final 53 × 63 × 46 matrix for each subject, see more details in Sui et al. (2011).

sMRI preprocessing

sMRI data were also preprocessed using the SPM8 software package which was used to segment the brain into white-matter (WM), GM, and cerebral spinal fluid with unmodulated normalized parameters via the unified segmentation method (Ashburner and Friston, 2005). After segmentation, the GM images were smoothed to a FWHM Gaussian kernel of 10 mm (White et al., 2001) and re-sliced to a matrix of 53 × 63 × 46 voxels. Subject outlier detection was further performed using a spatial Pearson correlation with the template image, to ensure that all subjects were properly segmented (for details, see Segall et al., 2009).

Normalization

After feature extraction (preprocessing), the 3D brain images of each subject were reshaped into a one-dimensional vector and stacked, forming a matrix with dimensions of 63 × number of voxels for each of the three modalities. These three feature matrices were then normalized to have the same average sum-of-squares (computed across all subjects and all voxels/locus for each modality) to ensure all modalities had the same ranges. Following normalization, the relative scaling (a normalization factor) within a given data type was preserved (i.e., 1.08, 0.24, 0.39 for ALFF, FA, GM respectively), but the normalized input units have the same voxel-wise mean square variance for all modalities. Next, the data was processed via the pipeline shown in Figure 1, i.e., dimension reduction → multi-set CCA → jICA → component analysis. The component number was estimated using modified MDL (Li et al., 2007) to be 10, 5, 8 for fMRI, DTI, and sMRI respectively. We thus choose M = 10 for the following analysis since we have found that a slight overestimation of the component number does not adversely affect the results in simulation (Sui et al., 2011). Note that the estimated IC number is lower than that used for 4D fMRI data typically, since mCCA + jICA works on extracted features of interests, instead of the original imaging data. However, a considerable amount of variance is retained for the M = 10 case, i.e., 95, 96, 99% for fMRI, DTI, and sMRI respectively.

Analyzing Generated Components and Mixing Coefficients

After applying the mCCA + jICA to the human brain data, independent component S _k and the mixing matrices A _k for each modality (k = 3 in this study) were generated, providing a variety of ways to analyze the inter-correlation between modalities as well as the group differences, as in Sui et al. (2011). In this paper, we are most interested in:

Shared/distinct abnormalities

Two-sample t-tests were performed on mixing coefficients of each IC for each modality (i.e., first 28 elements corresponding HC versus last 35 elements corresponding SZ from mth column of A _k for the mth IC of modality k), the results tell us which components are significantly abnormal in SZ. If the components of the same index show group differences in more than one modality, they are called modality-common (or joint) group-discriminative ICs. By contrast, if the component shows significant group difference only in a single modality, it is called a modality-unique group-discriminative IC. That are what we call shared or distinct abnormalities.

Inter-modality correlation

We also looked into the column-wise correlations between A₁, A₂, and A₃ pair wisely. It is likely that the joint group-discriminative components have a strong inter-modality correlation between their mixing coefficents, which indicates the interaction and correspondence among modalities.

Impact of clinical measures

The derived mixing coefficients also provide a way to investigate the relationships between the identified components and subjects’ clinical data, e.g., the correlation between mixing coefficients of patients for each component and antipsychotic medication doses [standardized as olanzapine equivalents (Gardner et al., 2010)] or PANSS scores. In this paper, we computed the correlation with PANSS (Kay et al., 1987), which rate the scale of severity of positive, negative, and general symptoms in SZ.

Potential use for classification

To test the potential use of the identified group-discriminative components (i.e., corresponding rows of S _k of modality k), we next used them to generate features (e.g., the Z map above certain threshold) and train a classifier, to see whether they are able to predict diagnosis or serve as potential biomarkers, which may prove the great significance for multimodal analysis.

For each modality, we transferred the group-discriminating components (for ALFF and GM, we use only two ICs with minimum p values) into Z values and thresholded at |Z| > 3.5, generating a mask from each component. The masks of the same modality were then combined and applied to the raw input matrix of each modality, which served as the input to the further classification based on uni-modal and multimodal features. Each individual was assigned one of two class memberships (SZ versus HC). We trained four different classification algorithms: linear support vector machine (LSVM) (Cortes and Vapnik, 1995), radial basis function support vector machine (RSVM) (Amari and Wu, 1999), k-nearest neighbor algorithm (KNN) (Geva and Sitte, 1991), and Gaussian naïve bayes (GNB) (McCallum and Nigam, 1998). Each algorithm was trained on 50% of the data (randomly chosen samples) with 10-fold cross validation, and tested on the other half for 1000 times, with the mean and maximal success rate recorded. Because this paper is not mainly focused on classification, we will not address the details of each algorithm. One limitation of this experiment is that the data set used to identify group-discriminating components is the same as the one which we did classification with, since we don’t have other similar resting fMRI-DTI-sMRI data at hand for cross validation and our main aim is to test whether mCCA + jICA is able to serve as an effective feature selection method for group prediction.

Results

Group Differences in Human Brain Data

Two-sample t-tests found both modality-common group-discriminative ICs (e.g., IC6 and IC7 in green frames, as shown in Figure 2) as well as modality-unique group-discriminative ICs, e.g., GM_IC5, ALFF_IC3 in our case. Interestingly, the modal-connection between joint-discriminative ICs indicate significant correlations (GM-ALFF IC6: r = 0.28, p = 0.025; FA-GM IC7: r = 0.38, p = 0.002; FA-ALFF IC7: r = 0.31, p = 0.015) between their mixing profiles.

FIGURE 2

Figure 2. Group-discriminating regions across three modalities, with a threshold of |Z| > 2.5. Two-sample t-tests were performed on mixing coefficients of each IC for each modality. If the components of the same index show group differences in more than one modality, they are called modality-common (or joint) group-discriminative ICs in green frames; otherwise, it is called a modality-unique group-discriminative IC, e.g., GM_IC5, ALFF_IC3 in red frames.

Correlation with PANSS Scores

There was no significant correlation regarding the antipsychotic medication doses. However, two ICs: FA_IC4 (anterior thalamic radiation, ATR and superior longitudinal fasciculus, SLF) and GM_IC4 (subregions of the default mode) were significantly correlated with positive PANSS scores, while there was no significant correlation with negative PANSS score. The scatter plots and linear trends are shown in Figure 3.

FIGURE 3

Figure 3. The scatter plots and linear trends of components with significant correlation between positive PANSS score and its loadings.

The specific identified regions of the components of interest and their abbreviations are summarized in Table 2 for resting state fMRI components (Talairach labels), Table 3 for DTI (WM tracts), and Table 4 for sMRI (MNI labels) respectively. For fMRI and sMRI, each IC is transformed into a Z map by dividing its standard deviation across all voxels, and the voxels above the threshold (|Z| > 2.5) were converted from MNI coordinates to Talairach coordinates and entered into a database to provide anatomic and functional labels for the right (R) and left (L) hemispheres. The volume of identified voxels in each area is provided in cubic centimeters (cm³). Within each area, the maximum Z value and its MNI coordinates are provided for all three tables. To summarize the WM results, we used the Johns Hopkins WM tractography atlas (from FSL) (Hua et al., 2008), from which 20 structures were identified; mostly large bundles. In Table 3, the WM tract labels, the identified volume (cc), and the percentage that indicates the overlap of the identified voxels with each WM tract are listed in detail.

TABLE 2

Table 2. Anatomic regions of the GM components of interest.

TABLE 3

Table 3. White-matter tract labels of the FA components of interest.

TABLE 4

Table 4. Anatomic regions of the group-discriminating fMRI components.

Classification Based on Selected Components

After transferring the group-discriminating components into Z values and thresholded at |Z| > 3.5, the mask from each component were generated and applied to the raw input matrix of each modality, resulting in three feature matrices in dimension of subject by voxels, i.e., FA: 63 × 312, ALFF 63 × 566, GM 63 × 1035, which served as the input to the further classification based on uni-modal and multimodal features.

Each individual was assigned one of two class memberships (SZ versus HC) and we have seven modal combinations (three single, three pair-wise, one three-way) as shown in Figure 4. After comparison, RSVM achieved the best classification accuracy among the four algorithms we trained with each of seven modal combinations; its mean, and maximum rates were summarized in Figure 4, where GM features obtained the highest accuracy in single modality, while FA + GM predict best among all seven modal combinations (mean 0.79, max 0.96).

FIGURE 4

Figure 4. Classification accuracy based on selected group-discriminative components from mCCA + jICA for seven modal combinations.

Discussion

In this paper we applied the mCCA + jICA model to three-way fusion of resting state fMRI, sMRI, and DTI data. The aim of the method is to identify precise correspondence among n data types and make possible the investigation of both shared and distinct abnormalities spanning multiple modalities for a specified brain disorder. Some abnormalities may occur in specific modalities, while others may be found in more than one modality simultaneously. Also, hidden linkages between components from different modalities may underlie in the data.

Group Differences

IC 7 significantly differentiated SZ from HC in all three modalities, suggesting the following abnormalities in SZ: (a) prefrontal cortex and left superior temporal gyrus (STG) (rest fMRI); (b) ATR, corticospinal tract (CSF), and forceps major (FMAJ; WM, DTI); and (c) regions of the motor cortex, medial/superior frontal cortex, and temporal gyrus (GM density). Furthermore, these identified affected regions may share some underlying relationship in SZ. The FA changes in ATR, CST, and FMAJ were previously associated with disconnectivity of brain networks in SZ in separate studies (Schlosser et al., 2007; Friedman et al., 2008; Sussmann et al., 2009). In particular, ATR projects from the anterior and medial regions of the thalamus to the frontal lobe, while CST subserves motor control. Accordingly, GM_IC7 shows strong alterations in motor cortex and, corresponding nicely to findings in Douaud et al. (2007) where the abnormalities in the primary sensorimotor and premotor cortices and in WM CST tracts were detected. Moreover, ALFF_IC7 implicates prefrontal cortex as abnormal, which plays an important role in the sensory integration and has been frequently reported dysfunction in SZ (Badcock et al., 2005; Hamilton et al., 2009; Yu et al., 2012a). These two pairs of components (FA-ALFF IC7, FA-GM IC7) depict a set of functional-anatomical “connected” regions. Note that both pairs have significant correlations (0.31/0.38) between their subject-mixing profiles as mentioned before, suggesting that disrupted WM connectivity may contribute to coordinated brain dysfunction, especially in the frontal and motor cortex, which is frequently hypothesized to be “disconnected” from other brain regions in SZ (Williams et al., 2004). Our results suggest that the anisotropy changes may relate to functional/structural changes in brain connectivity that are thought to play a central role in the clinical expression of SZ (Douaud et al., 2007).

Furthermore, GM-ALFF IC6 is another joint group-discriminative component, with middle/medial frontal cortex and thalamus (Woodward et al., 2012) indicated in ALFF map and temporal/frontal cortex shown in GM changes. The abnormality in each component have been previously found associated with the SZ deficits separately (Onitsuka et al., 2004; Zhou et al., 2007a; Edgar et al., 2012). Specifically, the result in Jayakumar et al. (2005) was in well accordance with our findings that SZ patients have significantly smaller global and regional GM volumes in inferior frontal, superior temporal, and parahippocampal gyri etc. Our results also suggest that functional disconnectivity associated with frontal lobe (also shown in ALFF_IC3) is present in SZ during rest (Hoptman et al., 2009). This is consistent with the notion that deregulation of medial frontal regions is associated with self-directed thoughts. This may lead to confusion between the source of internal and external stimuli, and may provide a neurophysiological basis for hallucinations (Whitfield-Gabrieli et al., 2009). This would have to be verified in future work.

We also identified ICs of interest showing significance only in one modality, such as GM_IC 5, 9, 10 and ALFF_IC3 (pink frame). The three structural components indicated regions including STG, precuneus, prefrontal cortex, insula, and thalamus, Hence, GM concentrations were significantly reduced in the above regions in the SZ group, consistent with other findings (Ha et al., 2004; Chua et al., 2007; Segall et al., 2009). Since structurally segregated and functionally specialized regions of the human cerebral cortex are interconnected by a dense network of cortico-cortical pathways (Hagmann et al., 2008; Segall et al., 2012), supporting the hypothesis that the SZ deficit may lie in aberrant structural changes and disconnectivity among different cortical areas.

Correlation with Positive Symptoms

Positive symptoms refer to an excess or distortion of normal psychological functions, e.g., hallucinations and delusions. In Figure 3, the higher positive symptoms were correlated with identified voxels in the middle/STG, precuneus, anterior cingulate, and the parahippocampal gyrus in GM_IC4. This is consistent with similar findings in fMRI (Garrity et al., 2007) where PANSS positive scores were associated with abnormal activation of STG, precuneus. Similarly GM volumes in anterior and posterior cingulate regions were correlated with positive symptoms (Choi et al., 2005; Yan et al., 2012). Additionally, in Meda et al. (2012), similar regions were also reported in resting state fMRI data, in which anterior default mode and frontal-occipital regions have significant correlation with the PANSS positive subscale in SZ. All these findings suggest a general hypothesis that psychotic symptoms derive from functionally disconnected brain circuits, e.g., the disintegrated brain connectivity between medial frontal/prefrontal and parietal networks in SZ (Zhou et al., 2007b). For FA_IC4, the FA values in left ATR and SLF showed a significant negative correlation with positive PANSS, consistent with (Caprihan et al., 2008; Cui et al., 2011), suggesting that deficits of WM integrity in left frontal-parietal lobe may also be involved in the pathophysiology of positive symptoms. Finally, this data also supports the hypothesis that the failure of left-hemisphere lateralization might be involved in the pathophysiology of SZ (Szeszko et al., 2005).

Classification Based on Selected ICs

The classification in Figure 4 shows that GM feature achieves the best classification among three single modalities, consistent with the fact that the selected GM components have much smaller p values than ALFF or FA. The most powerful prediction can be accomplished by using features from FA + GM, which is able to detail the multifaceted pathology that is likely to be present in SZ compared with single modality. Our results suggest that multimodal fusion of the selected group-discriminative components can improve the potential diagnosis prediction, in accordance with Sui et al. (2009a) and Yang et al. (2010), however, fusing as many modalities as possible in the training sample does not guarantee best classification rates, as we showed here and reported in Zhang et al. (2012); thus it would be helpful to compare a combination of uni-modal and multimodal results, as we did in Kim et al. (2010), to detect the potential biomarkers. We plan to pursue this possibility in future work by using larger data sets and various modalities, which aims to have bigger effect size and achieve higher accuracy.

Future Work

In this paper we develop and evaluate a novel multivariate method that can explore cross-information in multiple (more than two) data types and applied it to compare SZ patients to controls using an fMRI-DTI-sMRI combination. This is a novel attempt to perform a fusion of three different imaging modalities. The method described here could be applied straightforwardly to study other brain diseases (or subsets of a particular illness, such as psychotic or non-psychotic bipolar disorder). In addition, the choice of which multimodal data type to utilize is flexible, i.e., EEG, MEG, or genetic data, different features like fractional ALFF (fALFF) from fMRI (Kalcher et al., 2013) are also applicable. In a recent study, we found both ALFF and fALFF to be interesting and decided to start with ALFF (Turner et al., 2012), and will consider fALFF in future work. Finally the proposed method is very computationally efficient.

A limitation to the current study is that the subject number is not very high. Several statistical tests did not survive from the multiple comparisons, which may be complemented in future studies by including more subject samples or by multi-site recruitment. Additionally, mCCA + jICA operates on extracted features, rather than the original imaging data (e.g., using FA values instead of raw DTI data). Although some of the information is lost using this method, a “feature” tends to be more tractable than working with the large-scale original data due to the reduced number of dimensions (Calhoun and Adali, 2009) and provides a simpler space to link the data (Smith et al., 2009). Note that in our study we did not perform WM tractography but provided a type of summary statistic. A major strength of mCCA + jICA is that it can discover changes in one modality, e.g., which are related to alterations in distant, but connected regions in other modalities, without requiring a direct link.

Another point worth noting is that we did not collect physiologic data during the rest fMRI session as studies of patients tend to make this more difficult to collect. However it would be worth evaluating this in future work. With the advent of more rapid scanning (e.g., multiband sequences) which can adequately sample the cardiac noise, it is becoming much more feasible to characterize physiologic noise in large patient studies. We did not collect information on nicotine use either in these subjects, which may have potential effects on the imaging results, and would better be taken into account in the future. For example, recent studies indicated evidences of smoking effect in resting-state networks (Janes et al., 2012) and more prevalence in subjects with psychiatric disorder like SZ (Dickerson et al., 2013).

Multimodal fusion is an effective approach for analyzing biomedical imaging data that combines multiple data types in a joint analysis. It helps to identify the unique and shared variance associated with each imaging modality that underlies cognitive functioning in HCs and impairment in mental illness. In this real-world fusion application, we highlighted data from rest fMRI, WM tract, and GM concentration from SZ and healthy control subjects. We identified both modality-common and modality-unique group-discriminating aspects that verified the abnormalities in SZ, as well as replicated and extended previous findings. Such observations add to our understanding of the neural correlates of SZ. The proposed model promises a widespread utilization in the neuroimaging community and may be used to identify potential brain illness biomarkers.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This work was supported by the National Institutes of Health grants R01EB 006841, R01EB 005846, and 5P20RR021938 (to Vince D. Calhoun), and R01MH43775, R01MH074797, and R01MH077945 (to Godfrey D. Pearlson).

Footnotes

1. ^http://cobre.mrn.org
2. ^http://www.fil.ion.ucl.ac.uk/spm/software/spm8
3. ^www.fmrib.ox.ac.uk/fsl

References

Amari, S., and Wu, S. (1999). Improving support vector machine classifiers by modifying kernel functions. Neural Netw. 12, 783–789. doi:10.1016/S0893-6080(99)00032-5

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ashburner, J., and Friston, K. J. (2005). Unified segmentation. Neuroimage 26, 839–851. doi:10.1016/j.neuroimage.2005.02.018

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Badcock, J. C., Michiel, P. T., and Rock, D. (2005). Spatial working memory and planning ability: contrasts between schizophrenia and bipolar I disorder. Cortex 41, 753–763. doi:10.1016/S0010-9452(08)70294-6 doi:10.1016/S0010-9452(08)70294-6

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Bell, A. J., and Sejnowski, T. J. (1995). An information-maximization approach to blind separation and blind deconvolution. Neural Comput. 7, 1129–1159. doi:10.1162/neco.1995.7.6.1129

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Brown, P. L., Shepard, P. D., Elmer, G. I., Stockman, S., McFarland, R., Mayo, C. L., et al. (2012). Altered spatial learning, cortical plasticity and hippocampal anatomy in a neurodevelopmental model of schizophrenia-related endophenotypes. Eur. J. Neurosci. 36, 2773–2781. doi:10.1111/j.1460-9568.2012.08204.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Calhoun, V. D., and Adali, T. (2009). Feature-based fusion of medical imaging data. IEEE Trans. Inf. Technol. Biomed. 13, 711–720. doi:10.1109/TITB.2008.923773

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Calhoun, V. D., Adali, T., Giuliani, N. R., Pekar, J. J., Kiehl, K. A., and Pearlson, G. D. (2006). Method for multimodal analysis of independent source differences in schizophrenia: combining gray matter structural and auditory oddball functional data. Hum. Brain Mapp. 27, 47–62. doi:10.1002/hbm.20166

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Calhoun, V. D., and Allen, E. (2013). Extracting intrinsic functional networks with feature-based group independent component analysis. Psychometrika, 78, 243–259. doi:10.1007/s11336-012-9291-3

CrossRef Full Text

Calhoun, V. D., Eichele, T., Adali, T., and Allen, E. A. (2012). Decomposing the brain: components and modes, networks and nodes. Trends Cogn. Sci. (Regul. Ed.) 16, 255–256. doi:10.1016/j.tics.2012.03.008

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Calhoun, V. D., MacIejewski, P. K., Pearlson, G. D., and Kiehl, K. A. (2008). Temporal lobe and “default” hemodynamic brain modes discriminate between schizophrenia and bipolar disorder. Hum. Brain Mapp. 29, 1265–1275. doi:10.1002/hbm.20463

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Camara, E., Rodriguez-Fornells, A., and Munte, T. F. (2010). Microstructural brain differences predict functional hemodynamic responses in a reward processing task. J. Neurosci. 30, 11398–11402. doi:10.1523/JNEUROSCI.0111-10.2010

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Camchong, J., MacDonald, A. W. III, Bell, C., Mueller, B. A., and Lim, K. O. (2011). Altered functional and anatomical connectivity in schizophrenia. Schizophr. Bull. 37, 640–650. doi:10.1093/schbul/sbp131

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Caprihan, A., Pearlson, G. D., and Calhoun, V. D. (2008). Application of principal component analysis to distinguish patients with schizophrenia from healthy controls based on fractional anisotropy measurements. Neuroimage 42, 675–682. doi:10.1016/j.neuroimage.2008.04.255

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Chen, K., Reiman, E. M., Huan, Z., Caselli, R. J., Bandy, D., Ayutyanont, N., et al. (2009). Linking functional and structural brain images with multivariate network analyses: a novel application of the partial least square method. Neuroimage 47, 602–610. doi:10.1016/j.neuroimage.2009.04.053

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Choi, J. S., Kang, D. H., Kim, J. J., Ha, T. H., Roh, K. S., Youn, T., et al. (2005). Decreased caudal anterior cingulate gyrus volume and positive symptoms in schizophrenia. Psychiatry Res. 139, 239–247. doi:10.1016/j.pscychresns.2004.05.008

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Chua, S. E., Cheung, C., Cheung, V., Tsang, J. T., Chen, E. Y., Wong, J. C., et al. (2007). Cerebral grey, white matter and csf in never-medicated, first-episode schizophrenia. Schizophr. Res. 89, 12–21. doi:10.1016/j.schres.2006.09.009

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Correa, N. M., Li, Y. O., Adali, T., and Calhoun, V. D. (2009). “Fusion of fMRI, sMRI, and EEG data using canonical correlation analysis,” in ICASSP 2009 (Taipei: IEEE), 385–388

Cortes, C., and Vapnik, V. (1995). “Support-vector networks,” in Machine Learning (Boston: Kluwer Academic Publishers), 273–297

Cui, L., Chen, Z., Deng, W., Huang, X., Li, M., Ma, X., et al. (2011). Assessment of white matter abnormalities in paranoid schizophrenia and bipolar mania patients. Psychiatry Res. 194, 347–353. doi:10.1016/j.pscychresns.2011.03.010

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Dickerson, F., Stallings, C. R., Origoni, A. E., Vaughan, C., Khushalani, S., Schroeder, J., et al. (2013). Cigarette smoking among persons with schizophrenia or bipolar disorder in routine clinical settings, 1999-2011. Psychiatr. Serv. 64, 44–50. doi:10.1176/appi.ps.201200143

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Douaud, G., Smith, S., Jenkinson, M., Behrens, T., Johansen-Berg, H., Vickers, J., et al. (2007). Anatomically related grey and white matter abnormalities in adolescent-onset schizophrenia. Brain 130, 2375–2386. doi:10.1093/brain/awm184

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Edgar, J. C., Hunter, M. A., Huang, M., Smith, A. K., Chen, Y., Sadek, J., et al. (2012). Temporal and frontal cortical thickness associations with M100 auditory activity and attention in healthy controls and individuals with schizophrenia. Schizophr. Res. 140, 250–257. doi:10.1016/j.schres.2012.06.009

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Erhardt, E. B., Allen, E. A., Damaraju, E., and Calhoun, V. D. (2011). On network derivation, classification, and visualization: a response to Habeck and Moeller. Brain Connect. 1, 1–19. doi:10.1089/brain.2011.0022

CrossRef Full Text

First, M. B., Spitzer, R. L., Gibbon, M., and Williams, J. B. (1995). Structured Clinical Interview for DSM-IV axis I disorders. New York: New York State Psychiatric Institute.

Freire, L., Roche, A., and Mangin, J. F. (2002). What is the best similarity measure for motion correction in fMRI time series? IEEE Trans. Med. Imaging 21, 470–484. doi:10.1109/TMI.2002.1009383

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Friedman, J. I., Tang, C., Carpenter, D., Buchsbaum, M., Schmeidler, J., Flanagan, L., et al. (2008). Diffusion tensor imaging findings in first-episode and chronic schizophrenia patients. Am. J. Psychiatry 165, 1024–1032. doi:10.1176/appi.ajp.2008.07101640

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Friston, K. J., Ashburner, J., Frith, C. D., Poline, J. P., Heather, J. D., and Frackowiak, R. S. (1995). Spatial registration and normalization of images. Hum. Brain Mapp. 2, 165–189. doi:10.1002/hbm.460030303

CrossRef Full Text

Gardner, D. M., Murphy, A. L., O’Donnell, H., Centorrino, F., and Baldessarini, R. J. (2010). International consensus study of antipsychotic dosing. Am. J. Psychiatry 167, 686–693. doi:10.1176/appi.ajp.2009.09060802

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Garrity, A. G., Pearlson, G. D., McKiernan, K., Lloyd, D., Kiehl, K. A., and Calhoun, V. D. (2007). Aberrant “default mode” functional connectivity in schizophrenia. Am. J. Psychiatry 164, 450–457. doi:10.1176/appi.ajp.164.3.450

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Geva, S., and Sitte, J. (1991). Adaptive nearest neighbor pattern classification. IEEE Trans. Neural Netw. 2, 318–322. doi:10.1109/72.80344

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Groves, A. R., Beckmann, C. F., Smith, S. M., and Woolrich, M. W. (2011). Linked independent component analysis for multimodal data fusion. Neuroimage 54, 2198–2217. doi:10.1016/j.neuroimage.2010.09.073

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Groves, A. R., Smith, S. M., Fjell, A. M., Tamnes, C. K., Walhovd, K. B., Douaud, G., et al. (2012). Benefits of multi-modal fusion analysis on a large-scale dataset: life-span patterns of inter-subject variability in cortical morphometry and white matter microstructure. Neuroimage 63, 365–380. doi:10.1016/j.neuroimage.2012.06.038

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Ha, T. H., Youn, T., Ha, K. S., Rho, K. S., Lee, J. M., Kim, I. Y., et al. (2004). Gray matter abnormalities in paranoid schizophrenia and their clinical correlations. Psychiatry Res. 132, 251–260. doi:10.1016/j.pscychresns.2004.05.001

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hagmann, P., Cammoun, L., Gigandet, X., Meuli, R., Honey, C. J., Wedeen, V. J., et al. (2008). Mapping the structural core of human cerebral cortex. PLoS Biol. 6:e159. doi:10.1371/journal.pbio.0060159

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hamilton, L. S., Altshuler, L. L., Townsend, J., Bookheimer, S. Y., Phillips, O. R., Fischer, J., et al. (2009). Alterations in functional activation in euthymic bipolar disorder and schizophrenia during a working memory task. Hum. Brain Mapp. 30, 3958–3969. doi:10.1002/hbm.20820

CrossRef Full Text

Hoptman, M. J., Zuo, X. N., Butler, P. D., Javitt, D. C., D’Angelo, D., Mauro, C. J., et al. (2009). Amplitude of low-frequency oscillations in schizophrenia: a resting state fMRI study. Schizophr. Res. 117, 13–20. doi:10.1016/j.schres.2009.09.030

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Hua, K., Zhang, J., Wakana, S., Jiang, H., Li, X., Reich, D. S., et al. (2008). Tract probability maps in stereotaxic spaces: analyses of white matter anatomy and tract-specific quantification. Neuroimage 39, 336–347. doi:10.1016/j.neuroimage.2007.07.053

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Janes, A. C., Nickerson, L. D., Frederick Bde, B., and Kaufman, M. J. (2012). Prefrontal and limbic resting state brain network functional connectivity differs between nicotine-dependent smokers and non-smoking controls. Drug Alcohol Depend. 125, 252–259. doi:10.1016/j.drugalcdep.2012.02.020

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Jayakumar, P. N., Venkatasubramanian, G., Gangadhar, B. N., Janakiramaiah, N., and Keshavan, M. S. (2005). Optimized voxel-based morphometry of gray matter volume in first-episode, antipsychotic-naive schizophrenia. Prog. Neuropsychopharmacol. Biol. Psychiatry 29, 587–591. doi:10.1016/j.pnpbp.2005.01.020

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kalcher, K., Boubela, R. N., Huf, W., Biswal, B. B., Baldinger, P., Sailer, U., et al. (2013). RESCALE: voxel-specific task-fMRI scaling using resting state fluctuation amplitude. Neuroimage 70, 80–88. doi:10.1016/j.neuroimage.2012.12.019

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kay, S. R., Fiszbein, A., and Opler, L. A. (1987). The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr. Bull. 13, 261–276. doi:10.1093/schbul/13.2.261

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Kim, D. I., Sui, J., Rachakonda, S., White, T., Manoach, D. S., Clark, V. P., et al. (2010). Identification of imaging biomarkers in schizophrenia: a coefficient-constrained independent component analysis of the mind multi-site schizophrenia study. Neuroinformatics 8, 213–229. doi:10.1007/s12021-010-9077-7

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Li, Y. O., Adali, T., and Calhoun, V. D. (2007). Estimating the number of independent components for functional magnetic resonance imaging data. Hum. Brain Mapp. 28, 1251–1266. doi:10.1002/hbm.20359

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Li, Y. O., Adali, T., Wang, W., and Calhoun, V. D. (2009). Joint blind source separation by multi-set canonical correlation analysis. IEEE Trans. Signal Process. 57, 3918–3929. doi:10.1109/TSP.2009.2021636

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Lin, F. H., McIntosh, A. R., Agnew, J. A., Eden, G. F., Zeffiro, T. A., and Belliveau, J. W. (2003). Multivariate analysis of neuronal interactions in the generalized partial least squares framework: simulations and empirical studies. Neuroimage 20, 625–642. doi:10.1016/S1053-8119(03)00333-1

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Long, D., Wang, J., Xuan, M., Gu, Q., Xu, X., Kong, D., et al. (2012). Automatic classification of early Parkinson’s disease with multi-modal MR imaging. PLoS ONE 7:e47714. doi:10.1371/journal.pone.0047714

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Lu, L. H., Zhou, X. J., Keedy, S. K., Reilly, J. L., and Sweeney, J. A. (2012). White matter microstructure in untreated first episode bipolar disorder with psychosis: comparison with schizophrenia. Bipolar Disord. 13, 604–613. doi:10.1111/j.1399-5618.2011.00958.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

McCallum, A., and Nigam, K. (1998). “A comparison of event models for naive bayes text classification,” in AAAI/ICML-98 Workshop on Learning for Text Categorization, Technical Report WS-98-05. Madison: AAAI Press, 41–48.

McCarley, R. W., Nakamura, M., Shenton, M. E., and Salisbury, D. F. (2008). Combining ERP and structural MRI information in first episode schizophrenia and bipolar disorder. Clin. EEG Neurosci. 39, 57–60. doi:10.1177/155005940803900206

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Meda, S. A., Gill, A., Stevens, M. C., Lorenzoni, R. P., Glahn, D. C., Calhoun, V. D., et al. (2012). Differences in resting-state functional magnetic resonance imaging functional network connectivity between schizophrenia and psychotic bipolar probands and their unaffected first-degree relatives. Biol. Psychiatry 71, 881–889. doi:10.1016/j.biopsych.2012.01.025

CrossRef Full Text

Michael, A. M., Baum, S. A., White, T., Demirci, O., Andreasen, N. C., Segall, J. M., et al. (2010). Does function follow form? Methods to fuse structural and functional brain images show decreased linkage in schizophrenia. Neuroimage 49, 2626–2637. doi:10.1016/j.neuroimage.2009.08.056

CrossRef Full Text

Michael, A. M., King, M. D., Ehrlich, S., Pearlson, G., White, T., Holt, D. J., et al. (2011). A data-driven investigation of gray matter-function correlations in schizophrenia during a working memory task. Front. Hum. Neurosci. 5:71. doi:10.3389/fnhum.2011.00071

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Olesen, P. J., Nagy, Z., Westerberg, H., and Klingberg, T. (2003). Combined analysis of DTI and fMRI data reveals a joint maturation of white and grey matter in a fronto-parietal network. Brain Res. Cogn. Brain Res. 18, 48–57. doi:10.1016/j.cogbrainres.2003.09.003

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Onitsuka, T., Shenton, M. E., Salisbury, D. F., Dickey, C. C., Kasai, K., Toner, S. K., et al. (2004). Middle and inferior temporal gyrus gray matter volume abnormalities in chronic schizophrenia: an MRI study. Am. J. Psychiatry 161, 1603–1611. doi:10.1176/appi.ajp.161.9.1603

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Rykhlevskaia, E., Gratton, G., and Fabiani, M. (2008). Combining structural and functional neuroimaging data for studying brain connectivity: a review. Psychophysiology 45, 173–187. doi:10.1111/j.1469-8986.2007.00621.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Schlosser, R. G., Nenadic, I., Wagner, G., Gullmar, D., Von Consbruch, K., Kohler, S., et al. (2007). White matter abnormalities and brain activation in schizophrenia: a combined DTI and fMRI study. Schizophr. Res. 89, 1–11. doi:10.1016/j.schres.2006.09.007

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Segall, J. M., Allen, E. A., Jung, R. E., Erhardt, E. B., Arja, S. K., Kiehl, K., et al. (2012). Correspondence between structure and function in the human brain at rest. Front. Neuroinform. 6:10. doi:10.3389/fninf.2012.00010

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Segall, J. M., Turner, J. A., Van Erp, T. G., White, T., Bockholt, H. J., Gollub, R. L., et al. (2009). Voxel-based morphometric multisite collaborative study on schizophrenia. Schizophr. Bull. 35, 82–95. doi:10.1093/schbul/sbn150

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Smith, S. M., Fox, P. T., Miller, K. L., Glahn, D. C., Fox, P. M., Mackay, C. E., et al. (2009). Correspondence of the brain’s functional architecture during activation and rest. Proc. Natl. Acad. Sci. U.S.A. 106, 13040–13045. doi:10.1073/pnas.0905267106

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Sui, J., Adali, T., Pearlson, G., Yang, H., Sponheim, S. R., White, T., et al. (2010). A CCA+ICA based model for multi-task brain imaging data fusion and its application to schizophrenia. Neuroimage 51, 123–134. doi:10.1016/j.neuroimage.2010.01.069

CrossRef Full Text

Sui, J., Adali, T., Pearlson, G. D., and Calhoun, V. D. (2009a). An ICA-based method for the identification of optimal FMRI features and components using combined group-discriminative techniques. Neuroimage 46, 73–86. doi:10.1016/j.neuroimage.2009.01.026

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Sui, J., Adali, T., Pearlson, G. D., Clark, V. P., and Calhoun, V. D. (2009b). A method for accurate group difference detection by constraining the mixing coefficients in an ICA framework. Hum. Brain Mapp. 30, 2953–2970. doi:10.1002/hbm.20721

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Sui, J., Adali, T., Yu, Q., Chen, J., and Calhoun, V. D. (2012a). A review of multivariate methods for multimodal fusion of brain imaging data. J. Neurosci. Methods 204, 68–81. doi:10.1016/j.jneumeth.2011.10.031

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Sui, J., Yu, Q., He, H., Pearlson, G. D., and Calhoun, V. D. (2012b). A selective review of multimodal fusion methods in schizophrenia. Front. Hum. Neurosci. 6:27. doi:10.3389/fnhum.2012.00027

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Sui, J., He, H., Pearlson, G. D., Adali, T., Kiehl, K. A., Yu, Q., et al. (2013). Three-way (N-way) fusion of brain imaging data based on mCCA+jICA and its application to discriminating schizophrenia. Neuroimage 2, 119–132. doi:10.1016/j.neuroimage.2012.10.051

CrossRef Full Text

Sui, J., Pearlson, G., Caprihan, A., Adali, T., Kiehl, K. A., Liu, J., et al. (2011). Discriminating schizophrenia and bipolar disorder by fusing fMRI and DTI in a multimodal CCA+ joint ICA model. Neuroimage 57, 839–855. doi:10.1016/j.neuroimage.2011.05.055

CrossRef Full Text

Sussmann, J. E., Lymer, G. K., McKirdy, J., Moorhead, T. W., Munoz Maniega, S., Job, D., et al. (2009). White matter abnormalities in bipolar disorder and schizophrenia detected using diffusion tensor magnetic resonance imaging. Bipolar Disord. 11, 11–18. doi:10.1111/j.1399-5618.2008.00646.x

CrossRef Full Text

Szeszko, P. R., Ardekani, B. A., Ashtari, M., Kumra, S., Robinson, D. G., Sevy, S., et al. (2005). White matter abnormalities in first-episode schizophrenia or schizoaffective disorder: a diffusion tensor imaging study. Am. J. Psychiatry 162, 602–605. doi:10.1176/appi.ajp.162.3.602

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Teipel, S. J., Bokde, A. L., Meindl, T., Amaro, E. Jr., Soldner, J., Reiser, M. F., et al. (2010). White matter microstructure underlying default mode network connectivity in the human brain. Neuroimage 49, 2021–2032. doi:10.1016/j.neuroimage.2009.10.067

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Turner, J. A., Chen, H., Mathalon, D. H., Allen, E. A., Mayer, A. R., Abbott, C. C., et al. (2012). Reliability of the amplitude of low-frequency fluctuations in resting state fMRI in chronic schizophrenia. Psychiatry Res. 201, 253–255. doi:10.1016/j.pscychresns.2011.09.012

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Venkataraman, A., Rathi, Y., Kubicki, M., Westin, C. F., and Golland, P. (2010). Joint generative model for fMRI/DWI and its application to population studies. Med. Image Comput. Comput. Assist. Interv. 13, 191–199.

Pubmed Abstract | Pubmed Full Text

White, T., O’Leary, D., Magnotta, V., Arndt, S., Flaum, M., and Andreasen, N. C. (2001). Anatomic and functional variability: the effects of filter size in group fMRI data analysis. Neuroimage 13, 577–588. doi:10.1006/nimg.2000.0716

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Whitfield-Gabrieli, S., Thermenos, H. W., Milanovic, S., Tsuang, M. T., Faraone, S. V., McCarley, R. W., et al. (2009). Hyperactivity and hyperconnectivity of the default network in schizophrenia and in first-degree relatives of persons with schizophrenia. Proc. Natl. Acad. Sci. U.S.A. 106, 1279–1284. doi:10.1073/pnas.0809141106

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Williams, L. M., Das, P., Harris, A. W., Liddell, B. B., Brammer, M. J., Olivieri, G., et al. (2004). Dysregulation of arousal and amygdala-prefrontal systems in paranoid schizophrenia. Am. J. Psychiatry 161, 480–489. doi:10.1176/appi.ajp.161.3.480

CrossRef Full Text

Woodward, N. D., Karbasforoushan, H., and Heckers, S. (2012). Thalamocortical dysconnectivity in schizophrenia. Am. J. Psychiatry 169, 1092–1099. doi:10.1176/appi.ajp.2012.12010056

CrossRef Full Text

Xu, L., Pearlson, G., and Calhoun, V. D. (2009). Joint source based morphometry identifies linked gray and white matter group differences. Neuroimage 44, 777–789. doi:10.1016/j.neuroimage.2008.09.051

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Yan, H., Tian, L., Yan, J., Sun, W., Liu, Q., Zhang, Y. B., et al. (2012). Functional and anatomical connectivity abnormalities in cognitive division of anterior cingulate cortex in schizophrenia. PLoS One 7:e45659. doi:10.1371/journal.pone.0045659

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Yang, H., Liu, J., Sui, J., Pearlson, G. D., and Calhoun, V. D. (2010). A hybrid machine learning method for fusing fMRI and genetic data: combining both improves classification of schizophrenia. Front. Hum. Neurosci. 4:192. doi:10.3389/fnhum.2010.00192

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Yu, Q., Allen, E. A., Sui, J., Arbabshirani, M. R., Pearlson, G., and Calhoun, V. D. (2012a). Brain connectivity network in schizophrenia underlying resting state functional magnetic resonance imaging. Curr. Top. Med. Chem. 12, 2415–2425. doi:10.2174/156802612805289890

CrossRef Full Text

Yu, R., Chien, Y. L., Wang, H. L., Liu, C. M., Liu, C. C., Hwang, T. J., et al. (2012b). Frequency-specific alternations in the amplitude of low-frequency fluctuations in schizophrenia. Hum. Brain Mapp. (in press) doi:10.1002/hbm.22203

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Yu, Q., Sui, J., Liu, J., Plis, S. M., Kiehl, K. A., Pearlson, G., et al. (2013). Disrupted correlation between low frequency power and connectivity strength of resting state brain networks in schizophrenia. Schizophr. Res. 143, 165–171. doi:10.1016/j.schres.2012.11.001

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Zang, Y. F., He, Y., Zhu, C. Z., Cao, Q. J., Sui, M. Q., Liang, M., et al. (2007). Altered baseline brain activity in children with ADHD revealed by resting-state functional MRI. Brain Dev. 29, 83–91. doi:10.1016/j.braindev.2006.07.002

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Zhang, H., Liu, L., Wu, H., and Fan, Y. (2012). “Feature selection and SVM classification of multiple modality images for predicting MCI,” in OHBM, Beijing

Zhou, Y., Liang, M., Jiang, T., Tian, L., Liu, Y., Liu, Z., et al. (2007a). Functional dysconnectivity of the dorsolateral prefrontal cortex in first-episode schizophrenia using resting-state fMRI. Neurosci. Lett. 417, 297–302. doi:10.1016/j.neulet.2007.02.081

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Zhou, Y., Liang, M., Tian, L., Wang, K., Hao, Y., Liu, H., et al. (2007b). Functional disintegration in paranoid schizophrenia using resting-state fMRI. Schizophr. Res. 97, 194–205. doi:10.1016/j.schres.2007.05.029

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Zhou, Y., Shu, N., Liu, Y., Song, M., Hao, Y., Liu, H., et al. (2008). Altered resting-state functional connectivity and anatomical connectivity of hippocampus in schizophrenia. Schizophr. Res. 100, 120–132. doi:10.1016/j.schres.2007.11.039

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Zou, Q. H., Zhu, C. Z., Yang, Y., Zuo, X. N., Long, X. Y., Cao, Q. J., et al. (2008). An improved approach to detection of amplitude of low-frequency fluctuation (ALFF) for resting-state fMRI: fractional ALFF. J. Neurosci. Methods 172, 137–141. doi:10.1016/j.jneumeth

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text