A Comparative Study of Feature Selection Methods for the Discriminative Analysis of Temporal Lobe Epilepsy

It is crucial to differentiate patients with temporal lobe epilepsy (TLE) from the healthy population and determine abnormal brain regions in TLE. The cortical features and changes can reveal the unique anatomical patterns of brain regions from structural magnetic resonance (MR) images. In this study, structural MR images from 41 patients with left TLE, 34 patients with right TLE, and 58 normal controls (NC) were acquired, and four kinds of cortical measures, namely cortical thickness, cortical surface area, gray matter volume (GMV), and mean curvature, were explored for discriminative analysis. Three feature selection methods including the independent sample t-test filtering, the sparse-constrained dimensionality reduction model (SCDRM), and the support vector machine-recursive feature elimination (SVM-RFE) were investigated to extract dominant features among the compared groups for classification using the support vector machine (SVM) classifier. The results showed that the SVM-RFE achieved the highest performance (most classifications with more than 84% accuracy), followed by the SCDRM, and the t-test. Especially, the surface area and GMV exhibited prominent discriminative ability, and the performance of the SVM was improved significantly when the four cortical measures were combined. Additionally, the dominant regions with higher classification weights were mainly located in the temporal and the frontal lobe, including the entorhinal cortex, rostral middle frontal, parahippocampal cortex, superior frontal, insula, and cuneus. This study concluded that the cortical features provided effective information for the recognition of abnormal anatomical patterns and the proposed methods had the potential to improve the clinical diagnosis of TLE.

Introduction

Epilepsy, affecting approximately 50 million patients worldwide, has attracted increasing attention of many investigators. Among various epilepsy categories, temporal lobe epilepsy (TLE) is the most common drug-resistant category (1), which originates in the temporal lobe and involves structural and functional abnormalities of the brain (2–4). For patients with epilepsy for whom treatment with medicines is not effective, surgical treatment is usually considered as an eventual option. Thus, it is very important to determine effective and objective biomarkers to differentiate early TLE patients from normal controls (NC), so as to control seizures and prevent deteriorating.

During the past two decades, abundant voxel-based morphometry (VBM) studies have shown that structural changes in patients with TLE extend beyond the mesial temporal structures. There were differences in the extent of anatomical damage between hemispheres (5, 6) and the morphological abnormalities were more widespread in the left temporal lobe epilepsy (LTLE) with gray matter volume (GMV) loss, especially in the hippocampus, the parahippocampal gyrus, and the entorhinal cortex (7). Some resting-state fMRI studies suggested that significant decreases of the regional homogeneity (ReHo) were observed mainly in the default mode network (DMN), including the precuneus, the posterior cingulate gyrus, the bilateral inferior lateral parietal, and the mesial prefrontal cortex (8, 9). Similar changes were detected in the low-frequency fluctuation (ALFF) (10, 11). Functional magnetic resonance imaging (MRI) findings have indicated that patients with LTLE have significantly lower scores in verbal fluency than controls while compared to the right TLE (RTLE) (12). At the individual level, more patients with LTLE showed decreased or disrupted functional connectivity while a reverse pattern was found in RTLE (13). Neuropathological research has revealed various patterns of neuronal cell loss within the hippocampus and adjacent temporal lobe structures in the brain of the TLE patients (14). High-resolution MRI illustrated that the focal cortical dysplasia was a cause of medically intractable partial epilepsy (15). Morphological features of the cerebral cortex such as cortical thickness (CTh), surface area, GMV, and mean curvature (MCu) have been found to be associated with pathogenesis of either the LTLE or the RTLE. Previous studies indicated that there existed asymmetric reduction of cortical surface area (CSA) in the ipsilateral mesial and the anterior temporal lobe subregions (16), MCu abnormality in the bilateral insula (17), overall cortical thinning (18), and gray volume loss (19).

Based on the morphological and functional features of the brain, machine learning techniques such as the support vector machine (SVM), artificial neural network, clustering, and Bayesian networks, have been widely used in distinguishing patients with brain disorders from healthy controls (20–24). Existing studies have demonstrated that neuroimaging data were considered potential biomarkers for LTLE and RTLE diagnosis (25–30). For example, anatomical connectivity was reported for the separation of patients with LTLE from normal individuals with accuracy up to 93 and 90.0% for the RTLE (27), voxel-based MRI classification using diffusion tensor imaging (DTI), and T1-weighted MRI of the LTLE or RTLE patients could reach 90–100% (28, 29), the large-scale functional brain network measures as informative biomarkers for epilepsy pattern classification achieved a cross-validated classification accuracy of 83.9% (31). When a linear classifier was used to discriminate the LTLE from RTLE, hippocampal asymmetry could obtain 94% classification accuracy. More recently, the cortical measures were applied to automatically classify epilepsy patients with mesial temporal sclerosis (32).

In machine learning, feature selection is a vital step for extracting meaningful and relevant features to build an efficient classification model, reduce computation complexity, and boost its generalization ability (33–35). Recently, feature selection exhibited great advantages on biomedical studies like genomic (36) and proteomic studies (37), which usually had extremely large feature dimensionality and a small number of samples. In general, the feature selection methods can be divided into three categories according to the interaction with the estimation of the classification model: (1) the filter methods (38), the selection process is independent of classifiers and rank features according to the intrinsic properties, (2) the wrapper methods, which utilize the model’s predictive power to rank subsets of features (39), and (3) the embedded methods, where feature selection interacts with the machine learning process. The filter methods including the F-statistic, t-test, and principle component analysis, have the benefit of low computational cost, while the wrapper methods are more superior compared to the filter methods by taking discriminative power into consideration. Due to considering interaction among features, embedded methods such as the correlation-based feature selection (40) and support vector machine-recursive feature elimination (SVM-RFE) (41) showed excellent performance in pattern recognition research (32).

Although feature selection methods have been applied in some of the studies on the TLE classification from the neuroimaging data, it still remains uncertain to systematically explore effective feature selection strategies and perform discriminate analysis of cortical surface features from MRI among the LTLE and RTLE patients and the NC. In this study, we investigated feature selection methods to distinguish patients with LTLE and RTLE from the NC. Four morphological measures including the CTh, the CSA, the GMV, and the MCu, were explored for discriminative analysis. Three feature selection methods—the t-test filtering, the sparse-constrained dimensionality reduction model (SCDRM), and the SVM-RFE were compared and analyzed. Furthermore, the dominant cortical features and the corresponding brain regions with significant discriminative ability among compared groups were discussed.

Materials and Methods

Subjects and Data Acquisition

Forty-one LTLE patients, 34 RTLE patients, and 58 demographically matched NCs were enrolled in this study. Table 1 shows the demographic and clinical information of participants. The patients with seizure onset were determined by experienced physicians. All participant subjects were right-handed native Chinese cohorts recruited from Guangdong 999 hospital, and this study was approved by the Research Ethics Review Board of Guangdong 999 hospital. Written informed consent was obtained in accordance with the Helsinki Declaration prior to their inclusion. The inclusion criteria were as follows: (1) The seizure types and epileptic syndromes were diagnosed according to the classification of the International League Against Epilepsy and determined by the comprehensive evaluation standards including detailed history, neuropathological examination, electroencephalography (EEG) recordings, and MRI inspection; (2) continuous interictal-ictal scalp video EEG showed interictal epileptiform discharges of unilateral temporal origin.

TABLE 1

Table 1. Demographic and clinical information of participants.

All volumetric MRI were acquired with a 1.5T (n = 67) or 3.0T (n = 66) MRI scanner (1.5T Philips Intera; 3.0T GE Signa HDxT) using the T1-weighted three-dimension magnetization prepared rapid gradient echo (3D-MPRAGE) sequence according to following parameters: repetition time (TR) = 25 ms/8.84 ms, echo time (TE) = 4.6 ms/3.51 ms, matrix size = 256 × 256, voxel dimensions = 0.94 mm × 0.94 mm × 1.2 mm, flip angle = 30°, field of view (FOV) = 240 mm × 240 mm, and slice thickness = 1.2 mm. Approximately 140 isotropic images with voxel size of 1 mm × 1 mm × 1 mm were acquired.

Feature Extraction

3D T1-weighted DICOM images were converted to NIfTI (neuroimaging informatics technology initiative) format and their orientation was checked using the software MRIcron (http://people.cas.sc.edu/rorden). Cortical morphological measures were computed automatically using the FreeSurfer image analysis software (freely available at http://surfer.nmr.mgh.harvard.edu/). The detailed procedures are described elsewhere (42–45). Briefly, this processing included motion correction and averaging of multiple volumetric T1-weighted images, removal of non-brain tissue using a hybrid watershed and surface deformation procedure, automated Talairach transformation, segmentation of the subcortical white matter and deep gray matter, volumetric normalization of the structure intensity, tessellation of the gray matter and white matter boundaries, automated topology correction, and surface deformation following intensity gradients to optimally place the gray/white and gray/cerebrospinal fluid (CSF) borders at the location where the greatest shift in intensity defined the transition to the other tissue class. Once the cortical models are completed, several deformable procedures could be performed for further data processing and analysis including surface inflation, registration to a spherical atlas, which is based on individual cortical folding patterns to match cortical geometry across subjects, and parcelation of the cerebral cortex into units with respect to gyral and sulcal structure. This method uses both intensity and continuity information from the entire three-dimensional MRI volume in segmentation and deformation procedures to produce representations of the CTh and to calculate the closest distance from the gray/white boundary to the gray/CSF boundary at each vertex on the tessellated surface. After these steps, a mesh model of the cortical surface was generated and the measures were calculated according to the Desikan–Killiany atlas, which divided the cortical surface into 34 distinct cortical regions of interest in each hemisphere. Therefore, 68 features of each type of measure such as the CTh, the surface area, the GMV, and the MCu were obtained.

Feature Selection

The purpose of feature selection is to optimize the number of features in the subsequent machine learning model to enhance performance and generalizability. The original data usually contain redundant features, which provide no more information than currently selected features, and irrelevant features, which provide no useful information in any context; therefore, these features should be removed. Moreover, feature selection is also useful as a part of the data analysis process, which could determine the importance of each feature in classification or prediction, revealing the relationship between the features.

The univariate t-test method, as one of the most commonly used approaches was explored. A two-sample t-test was performed to compute the statistical significance value, and then the features with significant difference between the compared groups were extracted.

$T=\frac{\overline{X_1}-\overline{X_{\text{2}}}}{\sqrt{\frac{\text{(}{n}_{\text{1}}-\mathit{1}\text{)}{S}_{\text{1}}{}^{\text{2}}+\text{(}{n}_{\text{2}}-\mathit{1}\text{)}{S}_{\text{2}}{}^{\text{2}}}{n_{\text{1}}+n_{\text{2}}-\text{2}}\left(\frac{\text{1}}{n_{\text{1}}}+\frac{\text{1}}{n_{\text{2}}}\right)}}$ (1)

Where n₁ and n₂ denote the sample sizes, ${\overline{{\displaystyle X}}}_1$ and ${\overline{{\displaystyle X}}}_2$ denote the means, while $S_1{}^2$ and $S_2{}^2$ denote the SDs.

The sparsity-constrained dimensionality reduction was based on the least squares problems model, which was introduced a regularization term on model parameters to enforce sparsity using the l_2–1-norm (46).

${\displaystyle \underset{\mathbf{W}}{\text{min}}}{\displaystyle \sum _{i=1}^n{{\displaystyle \parallel {\mathbf{W}}^T{\mathbf{x}}_i-y_i{\parallel }_{2,1}^{}}}_{}}+{{\displaystyle \lambda R(W)}}_{}$ (2)

where $\left\{{\mathbf{x}}_1,{\mathbf{x}}_2,\cdots ,x_n\right\}\in R^d$ was the training set and $\left\{{\mathbf{y}}_1,{\mathbf{y}}_2,\cdots ,{\mathbf{y}}_n\right\}\in R^c$ denoted the corresponding class labels, W was the parameter matrix of size d × c, and $\Vert .\Vert$ represented the Frobenius norm of the matrix and λ∈[0,1] was a constant coefficient.

R(W) was defined as

$R(\mathbf{W})={\displaystyle \sum _{i=1}^d\Vert {\mathbf{w}}^i{\Vert }_{2,1}}$ (3)

Therefore, the model with the regularization term in Eq. 4 is given by

$\begin{array}{c}\underset{W}{{\displaystyle \min }}J(W)={\displaystyle \sum _{i=1}^n{{\displaystyle \parallel W^T{x}_i-y_i{\parallel }_{{}_{2,1}^{}}}}_{}}+{{\displaystyle \lambda R(W)}}_{}\\ ={{\displaystyle \parallel X^{T}W-Y\parallel }}_{2,1}+{{\displaystyle \lambda \mathrm{\parallel }W\parallel }}_{2,1}\end{array}$ (4)

The last term in Eq. 4 was a penalty or regularization term on the model parameters to enforce sparsity. An effective iterative algorithm could be used to solve the optimization problem in the model, and the coefficients in the parameters matrix represented the weights of the corresponding features.

The SVM methods were based on recursive feature elimination (SVM-RFE) (41), which eliminated features that were inessential or uninformative for discrimination, and retained the most discriminative features. A feature-ranking strategy based on correlation coefficients was used in the learning model, whereby each feature was given a weight, which represented its magnitude of separation; therefore, the features with the highest weights were the most informative.

SVM Classification

The SVM classifier, a supervised machine learning algorithm, was used to perform the classification. The LIBSVM toolbox was downloaded from https://www.csie.ntu.edu.tw/~cjlin/libsvm/. To verify the classifier, a nested cross-validation (CV) was applied in this study. In the inner CV loop, a fivefold cross validation and a grid search algorithm was leveraged to determine the optimal parameters for each classifier. During the outer CV loop, the leave-one-out cross validation strategy was employed to train and test each classifier. Three performance measures (accuracy, sensitivity, and specificity) were used to evaluate its performance, which was given according to the following equations:

$\text{Accuracy}=\frac{\text{TP}+\text{TN}}{\text{TP}+\text{FN}+\text{TN}+\text{FP}}$ (5)

$\text{Sensitivity}=\frac{\text{TP}}{\text{TP}+\text{FN}}$ (6)

$\text{Specificity}=\frac{\text{TN}}{\text{TN}+\text{FP}}$ (7)

where TP was the proportion of true positives, which denotes the number of patients correctly predicted; TN, was the proportion of true negatives, which corresponds to the number of subjects correctly classified as the NC; FN, namely, the false negatives, represented the number of individuals incorrectly identified as NC; and the false positives (FP) was the number of the healthy individuals incorrectly identified as patients.

To obtain the best performance and explore dominant features associated with core brain regions, the retained features extracted by the feature selection methods were sorted by the p-value from the t-test, coefficients in the SCDRM, and weights in the SVM-RFE, which represented the importance of each feature, and then the SVM added the feature one by one in order to evaluate their performance. Therefore, the features with the best performance were determined. To further evaluate the performance of the classifier using the extracted essential features, the corresponding receiver operating characteristic (ROC) curve was created and then the area under the curve (AUC) was computed.

Results

Mean Cortical Surface Feature Maps

To visualize four types of cortical surface features, the mean maps of the three compared groups were calculated and are shown in Figure 1.

FIGURE 1

Figure 1. Mean cortical surface feature maps in cortical thickness (CTh), cortical surface area (CSA), gray matter volume (GMV), and mean curvature (MCu) of the left TLE (LTLE), right TLE (RTLE) and normal controls (NC), respectively. Colors from blue to red denote the feature increase from the lowest value to the highest value.

Visual Inspection

During the diagnostic workup of epilepsy, the patients with seizure onset were comprehensively determined by clinical symptoms, neurobehavioral tests, EEG recordings, and MRI inspection. Despite enormous improvements in MRI scanners and sequences, MRI inspection in the TLE diagnosis was often reported to be unremarkable. In this study, there were 24.4% of LTLE and 23.5% of RTLE participants who did not exhibit distinctive signs in MRI visual analysis. More specifically, 10 of 41 LTLE and 8 of 34 RTLE patients were not identified in magnetic resonance (MR) scans. Besides, ipsilateral hippocampal sclerosis, focal cortical dysplasia, or both of them were visually identified in the other patients, which was one of the primary approaches for determining the lateralization of TLE.

Statistical Analysis

A tool within FreeSurfer called QDEC was applied to identify structural difference among LTLE, RTLE patients, and NC subjects on CTh, surface area, GMV, and MCu. Age and sex were used as nuisance factor and the statistical p-value was set at p < 0.01. Finally, the cortical feature differences between two groups patients were mapped onto a standard brain template called Desikan–Killiany atlas and displayed in Figure 2.

FIGURE 2

Figure 2. The statistical differences among left TLE (LTLE), right TLE (RTLE), and normal controls (NC) on cortical thickness (CTh), cortical surface area (CSA), gray matter volume (GMV), and mean curvature (MCu). The top line shows that the CTh in brain regions (listed in Table 2 and marked with blue color) reduced significantly in LTLE, RTLE group compared with NC group, while CTh value of a few of brain regions with red color increased significantly in LTLE group compared with RTLE group. Similarly, the second, third, and last lines represent the significant differences of CSA, GMV, and MCu in compared groups.

Furthermore, to investigate differences or changes of subcortical structures of brain among compared groups, we applied an automatic subcortical segmentation pipeline within FreeSurfer to assign each voxel of normalized brain volume into 40 labels (47). Therefore, the voxel number (volume) of 40 brain regions was obtained and then two sample t-test method was used to explore the statistical significance of the 40 features among LTLE-NC and RTLE-NC comparison. As listed in Table 2, the left hippocampus, left thalamus, CSF, posterior cerebral cortex, central cerebral cortex, left ventral diencephalon, and left ventricle changed significantly in the LTLE, while the RTLE patients were found abnormal in the right hippocampus, right thalamus, left thalamus, left cerebellum cortex, right lateral ventricle, right cerebellum cortex, third ventricle, right accumbens area, and left amygdala. Figure 3 shows three sectional images based on the atlas with 40 labeling regions of subcortical structures in LTLE, RTLE, and NC groups, respectively.

TABLE 2

Table 2. Brain regions with significant differences in subcortical structure of compared groups.

FIGURE 3

Figure 3. Sectional images of subcortical structures in left TLE (LTLE), right TLE (RTLE), and normal controls (NC) subjects.

Classification Performance

The results of various experiments using the three feature selection strategies and four cortical measures to the classification scenarios are shown in Table 3.

TABLE 3

Table 3. Classification performance.

From Table 3, we can observe that the SVM-REF achieved the highest performance (most classifications with more than 84% accuracy), followed by the SCDRM, and the t-test. It can be seen that the surface area and GMV exhibited the prominent discriminative ability. Moreover, when four types of measure were combined as input features of the SVM-RFE, the highest classification accuracy could reach up to 96.76%, and the optimal feature numbers with the highest performance of the LTLE-NC, RTLE-NC, and LTLE-RTLE classification were 45, 58, and 47, respectively.

In t-test feature selection process, the SVM classifier obtained the highest accuracy from 70 to 83% when applying a single measure among the LTLE-NC, RTLE-NC, LTLE-RTLE comparisons. While all of four measures were combined, the classification accuracy was increased to 80.71, 83.70, and 89.33%, respectively. Among the four measures, the CSA exhibited better discriminative ability than the other three measures.

With regard to the SCDRM feature selection method, the highest classification accuracies of the SVM classifier using the single measure were 70–84% among the LTLE-NC, RTLE-NC, and LTLE-RTLE comparisons. The performances of the SVM classifier were significantly improved with the combination of the four measures, such that the accuracies were increased to 86.87, 85.87, and 84.0% in LTLE-NC, RTLE-NC, and LTLE-RTLE comparisons.

Table 3 shows that the SVM-RFE achieved much better performance than the other two feature selection methods. The highest accuracies were from 81.3 to 92.0% in the single measure classification. As shown in Tables 3 and 4 and Figures 4 and 5, after the four measures were combined, the SVM classifier obtained much better performance such as the highest accuracies of 94.95% using 45 features in LTLE-NC, 96.76% using 58 features in RTLE-NC, and 96.0% using 47 features in LTLE-RTLE, respectively. The changes in classification performance with the increase in combined measures are shown in Figure 4. The ROC curves are shown in Figure 5.

TABLE 4

Table 4. The top 15 discriminative features for left TLE (LTLE)-normal controls (NC), right TLE (RTLE)-NC, and LTLE-RTLE classification selected by the support vector machine-recursive feature elimination classifier using combined measures, including cortical thickness (CTh), cortical surface area (CSA), gray matter volume (GMV), and mean curvature (MCu).

FIGURE 4

Figure 4. Classification performance among left TLE (LTLE)-normal controls (NC), right TLE (RTLE)-NC, and LTLE-RTLE comparisons using support vector machine-recursive feature elimination strategy by combining all four measures. Red dash lines present optimal classification feature number. Accuracy, sensitivity, and specificity fluctuation during feature selection process show in red, black, and blue straight line, respectively.

FIGURE 5

Figure 5. Corresponding receiver operating characteristic (ROC) curves of left TLE (LTLE)-normal controls (NC), right TLE (RTLE)-NC, and LTLE-RTLE comparisons using support vector machine-recursive feature elimination by combining all four measures. The areas under the curves were 0.9651, 0.9731, and 0.9749, respectively.

Features Contributing to Classification Performance

The most discriminative features determined by the SVM-RFE from the combined measures are illustrated in Table 3 and Figures 6 and 7. The top 15 features are listed in descending order by their weights (Table 3). The weights of features with prominent discriminative abilities were mapped into the brain template (Figure 6). To further illustrate the importance of all cortical surface features and compare their discriminative power, the weights of the features in classification are displayed in Figure 7.

FIGURE 6

Figure 6. Weights of the cortical surface regions with prominent discriminative abilities determined by the support vector machine-recursive feature elimination. The brain regions with red color have high discriminative powers, while brain regions with blue color possess low discriminative abilities.

FIGURE 7

Figure 7. Selected features and their weights in left TLE (LTLE)-normal controls (NC), right TLE (RTLE)-NC, LTLE-RTLE classification by the support vector machine-recursive feature elimination using combined measures. (A) LTLE-NC, (B) RTLE-NC, (C) LTLE-RTLE. Cortical thickness (CTh), cortical surface area (CSA), GMV, and mean curvature (MCu) are denoted with red, green, dark green, and blue colors, and feature weights of different regions are accumulated according to the weights of four types of feature.

Discussion

Previous group-level statistical analysis of neuroimaging data has revealed some neuroanatomical and functional alterations between the TLE and healthy people (4, 7, 48). However, those group-level findings exhibited limited clinical application. Advanced techniques based on machine learning devised for pattern recognition could be applied for single subject prediction, which showed significant potential for assisting disease diagnosis (28, 32) and predicting treatment outcome (25, 49) of the patients with TLE in the individual level. In this study, by using cortical morphological features derived from brain MR images, the SVM classifier with efficient feature selection strategies could effectively differentiate the LTLE, RTLE, and NC. When combining the four different cortical measures, the classifier exhibited stronger discriminative power than the single measure.

Figure 2 illustrates the results of the cortical features analysis using QDEC with significance threshold set at p < 0.01. Patients with LTLE had significantly thicker thickness than NC in the left middle temporal, left insula, left parahippocampal cortex, right inferior temporal, and right cuneus, while the patients with RTLE showed significantly thicker thickness in the right precentral, right entorhinal cortex, left lateral occipital, and the left precuneus. Keller and Roberts (7) have summarized that morphological abnormalities widely occurred in the LTLE and RLTE, especially in the hippocampus, the parahippocampal gyrus, and the entorhinal cortex, which were mostly consistent with the current t-test findings. Functional MRI findings suggested that significant decreases of the ReHo were observed mainly in the DMN, including the precuneus, the posterior cingulate gyrus, the bilateral inferior lateral parietal, and the mesial prefrontal cortex cortex (8, 9).

With regard to the subcortical features, it could be seen from Table 2 that the ipsilateral hippocampus and thalamus volume were changed greatly in the LTLE and RTLE patients, which was consistent with the previous findings (10, 50). Hippocampal sclerosis is the most frequent histopathologic abnormality occurred in the patients with TLE (14). VBM studies have found widespread gray volume atrophy in the medial temporal cortex, including the hippocampus, thalamus, and amygdala (7). Our experimental results showed that ipsilateral hippocampus and thalamus volume variability could be a sensitive biomarker to detect the LTLE or RTLE patients, and it suggested that ipsilateral hippocampus and thalamus is vulnerable in the TLE patients. In addition to this, patients with TLE experience significant changes of volume in some regions including posterior cerebral cortex, central cerebral cortex, amygdala.

Among three investigated feature selection methods, the SVM-RFE had the most promising results, followed by the SCDRM and the t-test filtering approach. This was ascribed to its special scoring strategy for each feature in the training process of the SVM-RFE, which ranked all the features according to the scores assigned by the classifier and iteratively eliminated the features with the lowest scores (36, 41). It exhibited excellent performance in recent discriminative research (51, 52). The SCDRM performed effective classification with more than 80% accuracies for most comparisons, which demonstrated that the SCDRM could extract salient features with strong discriminative abilities. The SVM classifier achieved about 80% accuracy using the selected features. Arbabshirani et al. (53) systematically studied the relationship between significance level of group difference measured by two-sample t-test and the importance of the features contributing to classification. It was concluded that features that were statistically significant in the univariate analysis did not have strong discriminative ability and vice versa.

The most discriminative measures were the CSA and the GMV. The GMV was calculated by the product of CTh and CSA (54, 55). Reduction of the GMV was attributed to the decline of thickness (56), surface area (16), or both. Many studies only focused on voxel-based gray matter or white matter classification (27, 28). However, investigation on CTh and surface area could facilitate further understanding of the morphological changes in the cortex of patients with TLE. Comparing with other features such as clinical and neuropsychological data (57) and anatomical connectivity difference (27), the cortical features had much stronger power in discriminative analysis.

Regarding the spatial distribution of the dominant regions in combined features, the entorhinal CSA of the left hemisphere, the rostral middle frontal CSA of the right hemisphere, and the cuneus CSA of the right hemisphere were the most discriminative features in LTLE-NC, RTLE-NC, and LTLE-RTLE comparison, respectively. The left parahippocampal cortex, left rostral middle frontal, left insula, and right superior frontal also played a vital role in the LTLE discrimination, while the left inferior parietal, left insula, right cuneus, right lateral occipital, and right superior parietal had stronger power in the RTLE discrimination. These results provided some perspective value for the clinical diagnosis of the LTLE or RTLE, which were similar to the findings reported in previous studies (2, 16, 18, 56). The entorhinal cortex and parahippocampal cortex, which are located in temporal lobe, exhibited stronger discriminative performance in this study. Intracranial EEG analysis (58) suggested that abnormity in the entorhinal cortex may result in seizure generation. Another study revealed that parahippocampal was probably associated with patients with LTLE (59). Previous researches hypothesized that the left temporal lobe may be more vulnerable than the right side due to perinatal vascular, morphological changes, which were more diffuse and bilateral in the LTLE (26). As shown in Figure 6, the features contributing to the LTLE discrimination had a higher weighting score of 7.29 in the left brain hemisphere compared to the right hemisphere with 4.35, while the dominant features of the RTLE discrimination performed 8.52 in the right hemisphere compared to 7.18 in the left hemisphere. It illustrates that these cortical features correlated to the ipsilateral seizure onsets. The left insula and right cuneus showed predominant contribution to classification in this study, and they provided effective information for early diagnosis on both LTLE and RTLE. The widespread alterations in cortical features that extend beyond the hippocampus were associated with cognitive, intellectual, and executive function of the TLE (32).

The MRI visual inspection has been a conventional radiological procedure for the TLE diagnosis. Beyond some apparent abnormality in the TLE that could be identified through visual inspection by experienced physicians, several patients with epileptic discharge recording still showed no discernible MRI findings. This study demonstrated that the detection of brain cortical measures associated with TLE can be improved with computer-aided approach. There were 24.4% of the patients with LTLE and 23.5% of the patients with RTLE who did not show distinctive signs of these conditions in visual MRI inspection. More specifically, 10 of 41 LTLE and 8 of 34 RTLE were incorrectly identified in visual analysis. Our implemented method could reduce the incorrect cases into five LTLE and two RTLE patients, which demonstrated significant potential to improve diagnostic assessment, especially for patients who exhibited unremarkable lesions in standard radiological inspection.

This method has the potential to supplement visual assessment, especially for patients who do not have any obvious lesions in standard radiological examinations. The absence of visible lesion is one of the greatest challenges in epilepsy surgery, and correct patient lateralization (left vs. right seizure onset) and seizure localization play an important role for the treatment outcome.

This study has several limitations necessary to be mentioned. First, small sample size is a common pitfall existing in most of the similar studies. When the number of features is large in comparison to the sample size, the classifier tends to overfit the sample data. Therefore, more samples need to be collected in the further studies. Second, MRIs were obtained from two different scanners, which might need more specific strategy for feature extraction. However, previous research has proved that the FreeSurfer software had good test–retest reliability across scanner manufactures and field strengths (60). Although MR data acquired from different scanners certainly added extra variability that might have weakened the classifier to differentiate subjects, it also helped boost the method generalizability to the real-world situation. Finally, only the cortical surface features based on T1-MR data were investigated. Multimodal data such as fMRI, DTI, PET, and brain connectivity remained to be explored so as to provide either complementary or efficient information for accurate recognition, lateralization, and focus location in patients with TLE.

Conclusion

In this study, three feature selection methods were investigated for detection of TLE using cortical surface features from brain structural MRI. The SVM-RFE strategy outperformed the other two methods (t-test and SCDRM) in discriminative analysis. The surface area and GMV had stronger discriminative ability than the CTh and MCu. Combined cortical surface features could improve the classification performance of the SVM classifier. Especially, the dominant regions with prominent discriminative power were mainly located in the temporal and in the frontal lobe, including the entorhinal cortex, rostral middle frontal, parahippocampal cortex, superior frontal, insula, and cuneus. The important brain features contributing to LTLE or RTLE discrimination mainly located in the ipsilateral hemisphere. It demonstrated that the cortical surface features could provide effective information in classification of left TLE, right TLE, and healthy subjects. The SVM-RFE could effectively extract the core features with strong discriminative ability and determine the salient brain regions associated with epilepsy.

Ethics Statement

This study was carried out in accordance with the recommendations of Research Ethics Review Board of Guangdong 999 Brain Hospital with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Guangdong 999 Brain Hospital.

Author Contributions

CL contributed to the image processing, statistical analysis, and wrote the manuscript. SG contributed to the design and conceptualization of the study and revised the manuscript. LC and WW were involved in the MRI collection and interpretation of pathological mechanisms.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Funding

This work was supported by National Natural Science Foundation of China (31371008), Science and Technology Planning Project of Guangdong Province (2015A02024006), Guangzhou Bureau of Science and Technology (201604020170), and the China Scholar Council.

References

1. Engel J, Wiebe S, French J, Sperling M, Williamson P, Spencer D, et al. Practice parameter: temporal lobe and localized neocortical resections for epilepsy – report of the quality standards subcommittee of the American Academy of Neurology, in association with the American Epilepsy Society and the American Association of Neurological Surgeons. Neurology (2003) 60(4):538–47.

Google Scholar

2. Scanlon C, Mueller SG, Cheong I, Hartig M, Weiner MW, Laxer KD. Grey and white matter abnormalities in temporal lobe epilepsy with and without mesial temporal sclerosis. J Neurol (2013) 260(9):2320–9. doi:10.1007/s00415-013-6974-3

CrossRef Full Text | Google Scholar

3. Lee JH, Kim SE, Park CH, Yoo JH, Lee HW. Gray and white matter volumes and cognitive dysfunction in drug-naive newly diagnosed pediatric epilepsy. Biomed Res Int (2015) 2015:923861. doi:10.1155/2015/923861

CrossRef Full Text | Google Scholar

4. Coan AC, Campos BM, Beltramini GC, Yasuda CL, Covolan RJM, Cendes F. Distinct functional and structural MRI abnormalities in mesial temporal lobe epilepsy with and without hippocampal sclerosis. Epilepsia (2014) 55(8):1187–96. doi:10.1111/epi.12670

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Fosi T, Chu C, Chong WK, Clark C, Scott RC, Boyd S, et al. Quantitative magnetic resonance imaging evidence for altered structural remodeling of the temporal lobe in West syndrome. Epilepsia (2015) 56(4):608–16. doi:10.1111/epi.12907

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Pail M, Marecek R, Hermanova M, Slana B, Tyrlikova I, Kuba R, et al. The role of voxel-based morphometry in the detection of cortical dysplasia within the temporal pole in patients with intractable mesial temporal lobe epilepsy. Epilepsia (2012) 53(6):1004–12. doi:10.1111/j.1528-1167.2012.03456.x

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Keller SS, Roberts N. Voxel-based morphometry of temporal lobe epilepsy: an introduction and review of the literature. Epilepsia (2008) 49(5):741–57. doi:10.1111/j.1528-1167.2007.01485.x

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Zeng HW, Pizarro R, Nair VA, La C, Prabhakaran V. Alterations in regional homogeneity of resting-state brain activity in mesial temporal lobe epilepsy. Epilepsia (2013) 54(4):658–66. doi:10.1111/epi.12066

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Qiu CJ, Liao W, Ding JR, Feng Y, Zhu CY, Nie XJ, et al. Regional homogeneity changes in social anxiety disorder: a resting-state fMIRI study. Psychiatry Res (2011) 194(1):47–53. doi:10.1016/j.pscychresns.2011.01.010

CrossRef Full Text | Google Scholar

10. Zhang ZQ, Lu GM, Zhong YA, Tan QF, Chen HF, Liao W, et al. FMRI study of mesial temporal lobe epilepsy using amplitude of low-frequency fluctuation analysis. Hum Brain Mapp (2010) 31(12):1851–61. doi:10.1002/hbm.20982

CrossRef Full Text | Google Scholar

11. Qi RF, Zhang LJ, Wu SY, Zhong JH, Zhang ZQ, Zhong Y, et al. Altered resting-state brain activity at functional MR imaging during the progression of hepatic encephalopathy. Radiology (2012) 264(1):187–95. doi:10.1148/radiol.12111429

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Bonelli SB, Powell R, Thompson PJ, Yogarajah M, Focke NK, Stretton J, et al. Hippocampal activation correlates with visual confrontation naming: fMRI findings in controls and patients with temporal lobe epilepsy. Epilepsy Res (2011) 95(3):246–54. doi:10.1016/j.eplepsyres.2011.04.007

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Bettus G, Guedj E, Joyeux F, Confort-Gouny S, Soulier E, Laguitton V, et al. Decreased basal fMRI functional connectivity in epileptogenic networks and contralateral compensatory mechanisms. Hum Brain Mapp (2009) 30(5):1580–91. doi:10.1002/hbm.20625

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Bluemcke I, Thom M, Aronica E, Armstrong DD, Bartolomei F, Bernasconi A, et al. International consensus classification of hippocampal sclerosis in temporal lobe epilepsy: a task force report from the ILAE commission on diagnostic methods. Epilepsia (2013) 54(7):1315–29. doi:10.1111/epi.12220

CrossRef Full Text | Google Scholar

15. Barkovich AJ, Dobyns WB, Guerrini R. Malformations of cortical development and epilepsy. Cold Spring Harb Perspect Med (2015) 5(5):a022392. doi:10.1101/cshperspect.a022392

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Alhusaini S, Doherty CP, Palaniyappan L, Scanlon C, Maguire S, Brennan P, et al. Asymmetric cortical surface area and morphology changes in mesial temporal lobe epilepsy with hippocampal sclerosis. Epilepsia (2012) 53(6):995–1003. doi:10.1111/j.1528-1167.2012.03457.x

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Ronan L, Alhusaini S, Scanlon C, Doherty CP, Delanty N, Fitzsimons M. Widespread cortical morphologic changes in juvenile myoclonic epilepsy: evidence from structural MRI. Epilepsia (2012) 53(4):651–8. doi:10.1111/j.1528-1167.2012.03413.x

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Ristic AJ, Dakovic M, Kerr M, Kovacevic M, Parojcic A, Sokic D. Cortical thickness, surface area and folding in patients with psychogenic nonepileptic seizures. Epilepsy Res (2015) 112:84–91. doi:10.1016/j.eplepsyres.2015.02.015

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Santana MTCG, Jackowski AP, Britto FD, Sandim GB, Caboclo LOSF, Centeno RS, et al. Gender and hemispheric differences in temporal lobe epilepsy: a VBM study. Seizure-Eur J Epilepsy (2014) 23(4):274–9. doi:10.1016/j.seizure.2013.12.006

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Salvatore C, Cerasa A, Castiglioni I, Gallivanone F, Augimeri A, Lopez M, et al. Machine learning on brain MRI data for differential diagnosis of Parkinson’s disease and progressive supranuclear palsy. J Neurosci Methods (2014) 222:230–7. doi:10.1016/j.jneumeth.2013.11.016

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Subasi A. Classification of EMG signals using PSO optimized SVM for diagnosis of neuromuscular disorders. Comput Biol Med (2013) 43(5):576–86. doi:10.1016/j.compbiomed.2013.01.020

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Shen H, Wang L, Liu Y, Hu D. Discriminative analysis of resting-state functional connectivity patterns of schizophrenia using low dimensional embedding of fMRI. Neuroimage (2010) 49(4):3110–21. doi:10.1016/j.neuroimage.2009.11.011

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Orru G, Pettersson-Yeo W, Marquand AF, Sartori G, Mechelli A. Using support vector machine to identify imaging biomarkers of neurological and psychiatric disease: a critical review. Neurosci Biobehav Rev (2012) 36(4):1140–52. doi:10.1016/j.neubiorev.2012.01.004

CrossRef Full Text | Google Scholar

24. Mwangi B, Ebmeier KP, Matthews K, Steele JD. Multi-centre diagnostic classification of individual structural neuroimaging scans from patients with major depressive disorder. Brain (2012) 135:1508–21. doi:10.1093/brain/aws084

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Bernhardt BC, Hong S-J, Bernasconi A, Bernasconi N. Magnetic resonance imaging pattern learning in temporal lobe epilepsy: classification and prognostics. Ann Neurol (2015) 77(3):436–46. doi:10.1002/ana.24341

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Besson P, Dinkelacker V, Valabregue R, Thivard L, Leclerc X, Baulac M, et al. Structural connectivity differences in left and right temporal lobe epilepsy. Neuroimage (2014) 100:135–44. doi:10.1016/j.neuroimage.2014.04.071

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Fang P, An J, Zeng L-L, Shen H, Chen F, Wang W, et al. Multivariate pattern analysis reveals anatomical connectivity differences between the left and right mesial temporal lobe epilepsy. Neuroimage Clin (2015) 7:555–61. doi:10.1016/j.nicl.2014.12.018

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Focke NK, Yogarajah M, Symms MR, Gruber O, Paulus W, Duncan JS. Automated MR image classification in temporal lobe epilepsy. Neuroimage (2012) 59(1):356–62. doi:10.1016/j.neuroimage.2011.07.068

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Hong S-J, Kim H, Schrader D, Bernasconi N, Bernhardt BC, Bernasconi A. Automated detection of cortical dysplasia type II in MRI-negative epilepsy. Neurology (2014) 83(1):48–55. doi:10.1212/WNL.0000000000000543

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Nikolova S, Bartha R, Parrent AG, Steven DA, Diosy D, Burneo JG. Functional MRI of neuronal activation in epilepsy patients with malformations of cortical development. Epilepsy Res (2015) 116:1–7. doi:10.1016/j.eplepsyres.2015.06.012

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Zhang J, Cheng W, Wang ZG, Zhang ZQ, Lu WL, Lu GM, et al. Pattern classification of large-scale functional brain networks: identification of informative neuroimaging markers for epilepsy. PLoS One (2012) 7(5):e36733. doi:10.1371/journal.pone.0036733

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Rudie JD, Colby JB, Salamon N. Machine learning classification of mesial temporal sclerosis in epilepsy patients. Epilepsy Res (2015) 117:63–9. doi:10.1016/j.eplepsyres.2015.09.005

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Xiang S, Nie F, Meng G, Pan C, Zhang C. Discriminative least squares regression for multiclass classification and feature selection. IEEE Trans Neural Netw Learn Syst (2012) 23(11):1738–54. doi:10.1109/TNNLS.2012.2212721

CrossRef Full Text | Google Scholar

34. Pal M, Foody GM. Feature selection for classification of hyperspectral data by SVM. IEEE Trans Geosci Remote Sens (2010) 48(5):2297–307. doi:10.1109/TGRS.2009.2039484

CrossRef Full Text | Google Scholar

35. Abeel T, Helleputte T, Van de Peer Y, Dupont P, Saeys Y. Robust biomarker identification for cancer diagnosis with ensemble feature selection methods. Bioinformatics (2010) 26(3):392–8. doi:10.1093/bioinformatics/btp630

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Mundra PA, Rajapakse JC. SVM-RFE with MRMR filter for gene selection. IEEE Trans Nanobioscience (2010) 9(1):31–7. doi:10.1109/TNB.2009.2035284

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Zhang XG, Lu X, Shi Q, Xu XQ, Leung HCE, Harris LN, et al. Recursive SVM feature selection and sample classification for mass-spectrometry and microarray data. BMC Bioinformatics (2006) 7:13. doi:10.1186/1471-2105-7-252

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Sanchez-Marono N, Alonso-Betanzos A, Tombilla-Sanroman M. Filter methods for feature selection – a comparative study. Lect Notes Comput Sci (2007) 4881:178–87. doi:10.1007/978-3-540-77226-2_19

CrossRef Full Text | Google Scholar

39. Kohavi R, John GH. Wrappers for feature subset selection. Artif Intell (1997) 97(1–2):273–324. doi:10.1016/S0004-3702(97)00043-X

CrossRef Full Text | Google Scholar

40. Hall MA, Smith LA. Feature selection for machine learning: comparing a correlation-based filter approach to the wrapper. Proc. 12th Int’l Florida Artificial Intelligence Research Soc. (1999). p. 235–9.

Google Scholar

41. Guyon I, Weston J, Barnhill S, Vapnik V. Gene selection for cancer classification using support vector machines. Mach Learn (2002) 46(1–3):389–422. doi:10.1023/A:1012487302797

CrossRef Full Text | Google Scholar

42. Fischl B, Dale AM. Measuring the thickness of the human cerebral cortex from magnetic resonance images. Proc Natl Acad Sci U S A (2000) 97(20):11050–5. doi:10.1073/pnas.200033797

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Segonne F, Dale AM, Busa E, Glessner M, Salat D, Hahn HK, et al. A hybrid approach to the skull stripping problem in MRI. Neuroimage (2004) 22(3):1060–75. doi:10.1016/j.neuroimage.2004.03.032

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Desikan RS, Segonne F, Fischl B, Quinn BT, Dickerson BC, Blacker D, et al. An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage (2006) 31(3):968–80. doi:10.1016/j.neuroimage.2006.01.021

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Fischl B. FreeSurfer. Neuroimage (2012) 62(2):774–81. doi:10.1016/j.neuroimage.2012.01.021

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Nie F, Huang H, Cai X, Ding CHQ. Efficient and robust feature selection via joint ℓ2, 1-norms minimization. In: Advances in Neural Information Processing Systems (NIPS). (2010). p. 1813–21.

Google Scholar

47. Fischl B, Salat DH, Busa E, Albert M, Dieterich M, Haselgrove C, et al. Whole brain segmentation: automated labeling of neuroanatomical structures in the human brain. Neuron (2002) 33(3):341–55. doi:10.1016/S0896-6273(02)00569-X

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Chiang S, Stern JM, Engel J Jr, Levin HS, Haneef Z. Differences in graph theory functional connectivity in left and right temporal lobe epilepsy. Epilepsy Res (2014) 108(10):1770–81. doi:10.1016/j.eplepsyres.2014.09.023

CrossRef Full Text | Google Scholar

49. Memarian N, Kim S, Dewar S, Engel J Jr, Staba RJ. Multimodal data and machine learning for surgery outcome prediction in complicated cases of mesial temporal lobe epilepsy. Comput Biol Med (2015) 64:67–78. doi:10.1016/j.compbiomed.2015.06.008

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Li JP, Zhang ZB, Shang HF. A meta-analysis of voxel-based morphometry studies on unilateral refractory temporal lobe epilepsy. Epilepsy Res (2012) 98(2–3):97–103. doi:10.1016/j.eplepsyres.2011.10.002

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Lu XB, Yang YZ, Wu FC, Gao MJ, Xu Y, Zhang Y, et al. Discriminative analysis of schizophrenia using support vector machine and recursive feature elimination on structural MRI images. Medicine (2016) 95(30):e3973. doi:10.1097/MD.0000000000003973

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Christian J, Kroll J, Strutzenberger G, Alexander N, Ofner M, Schwameder H. Computer aided analysis of gait patterns in patients with acute anterior cruciate ligament injury. Clin Biomech (2016) 33:55–60. doi:10.1016/j.clinbiomech.2016.02.008

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Arbabshirani MR, Plis S, Sui J, Calhoun VD. Single subject prediction of brain disorders in neuroimaging: promises and pitfalls. Neuroimage (2016) 145(Pt B):137–65. doi:10.1016/j.neuroimage.2016.02.079

CrossRef Full Text | Google Scholar

54. Rimol LM, Nesvag R, Hagler DJ, Bergmann O, Fennema-Notestine C, Hartberg CB, et al. Cortical volume, surface area, and thickness in schizophrenia and bipolar disorder. Biol Psychiatry (2012) 71(6):552–60. doi:10.1016/j.biopsych.2011.11.026

CrossRef Full Text | Google Scholar

55. Li S, Yuan X, Pu F, Li D, Fan Y, Wu L, et al. Abnormal changes of multidimensional surface features using multivariate pattern classification in amnestic mild cognitive impairment patients. J Neurosci (2014) 34(32):10541–53. doi:10.1523/JNEUROSCI.4356-13.2014

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Butler T, Blackmon K, McDonald CR, Carlson C, Barr WB, Devinsky O, et al. Cortical thickness abnormalities associated with depressive symptoms in temporal lobe epilepsy. Epilepsy Behav (2012) 23(1):64–7. doi:10.1016/j.yebeh.2011.10.001

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Armananzas R, Alonso-Nanclares L, DeFelipe-Oroquieta J, Kastanauskaite A, de Sola RG, DeFelipe J, et al. Machine learning approach for the outcome prediction of temporal lobe epilepsy surgery. PLoS One (2013) 8(4):e62819. doi:10.1371/journal.pone.0062819

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Bonilha L, Martz GU, Glazier SS, Edwards JC. Subtypes of medial temporal lobe epilepsy: influence on temporal lobectomy outcomes? Epilepsia (2012) 53(1):1–6. doi:10.1111/j.1528-1167.2011.03298.x

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Carone DA, Green P, Drane DL. Word memory test profiles in two cases with surgical removal of the left anterior hippocampus and parahippocampal gyrus. Appl Neuropsychol Adult (2014) 21(2):155–60. doi:10.1080/09084282.2012.755533

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Han X, Jovicich J, Salat D, van der Kouwe A, Quinn B, Czanner S, et al. Reliability of MRI-derived measurements of human cerebral cortical thickness: the effects of field strength, scanner upgrade and manufacturer. Neuroimage (2006) 32(1):180–94. doi:10.1016/j.neuroimage.2006.02.051

PubMed Abstract | CrossRef Full Text | Google Scholar