Self-Reported Complaints as Prognostic Markers for Outcome After Mild Traumatic Brain Injury in Elderly: A Machine Learning Approach

Self-reported complaints are common after mild traumatic brain injury (mTBI). Particularly in the elderly with mTBI, the pre-injury status might play a relevant role in the recovery process. In most mTBI studies, however, pre-injury complaints are neither analyzed nor are the elderly included. Here, we aimed to identify which individual pre- and post-injury complaints are potential prognostic markers for incomplete recovery (IR) in elderly patients who sustained an mTBI. Since patients report many complaints across several domains that are strongly related, we used an interpretable machine learning (ML) approach to robustly deal with correlated predictors and boost classification performance. Pre- and post-injury levels of 20 individual complaints, as self-reported in the acute phase, were analyzed. We used data from two independent studies separately: UPFRONT study was used for training and validation and ReCONNECT study for independent testing. Functional outcome was assessed with the Glasgow Outcome Scale Extended (GOSE). We dichotomized functional outcome into complete recovery (CR; GOSE = 8) and IR (GOSE ≤ 7). In total 148 elderly with mTBI (median age: 67 years, interquartile range [IQR]: 9 years; UPFRONT: N = 115; ReCONNECT: N = 33) were included in this study. IR was observed in 74 (50%) patients. The classification model (IR vs. CR) achieved a good performance (the area under the receiver operating characteristic curve [ROC-AUC] = 0.80; 95% CI: 0.74–0.86) based on a subset of only 8 out of 40 pre- and post-injury complaints. We identified increased neck pain (p = 0.001) from pre- to post-injury as the strongest predictor of IR, followed by increased irritability (p = 0.011) and increased forgetfulness (p = 0.035) from pre- to post-injury. Our findings indicate that a subset of pre- and post-injury physical, emotional, and cognitive complaints has predictive value for determining long-term functional outcomes in elderly patients with mTBI. Particularly, post-injury neck pain, irritability, and forgetfulness scores were associated with IR and should be assessed early. The application of an ML approach holds promise for application in self-reported questionnaires to predict outcomes after mTBI.

Introduction

Traumatic brain injury (TBI) is one of the most important causes of morbidity and mortality in adults (1). Mild TBI (mTBI) accounts for 85% of the cases (2, 3). Most patients who sustained mTBI report post-injury complaints within the first weeks after injury, which can involve cognitive (e.g., forgetfulness and poor concentration), emotional (e.g., irritability and anxiety), and/or physical domains (e.g., headaches and fatigue). Although post-injury complaints usually resolve within 3 months after injury, they may persist for months to years in a subgroup (≈20%), which is often referred to as post-concussion syndrome (PCS) (4). According to the International Classification of Diseases (ICD)-10 definition, the diagnosis of PCS requires the presence of three or more post-injury complaints for at least 4 weeks (5). However, individual complaints are non-specific to PCS and can also be found in healthy individuals and across a variety of clinical populations (6–8).

Post-injury complaints are frequently assessed through self-reported questionnaires that quantify the relative severity of the current complaints compared to the pre-injury level (9–12). Thus, pre-injury status must be recalled by the patients when filling in the questionnaires and their subjective perception can affect the rating of self-reported complaints (13). It has been suggested that individuals might tend to underestimate past problems after a traumatic event, such as sustaining an mTBI, which is referred to as “good-old-days” bias (14). In the elderly, additionally, a higher rate of pre- or co-morbid complaints is expected (15). Nevertheless, few studies have investigated the effects of pre-injury complaints on long-term disability after mTBI (16), and particularly information concerning the elderly population is lacking (15).

Increasing age has been identified as an independent predictor of worse outcomes after mTBI (17). As the world population continues to age, the number of elderly sustaining mTBI will further increase. The elderly with mTBI are more likely to suffer from comorbidities than their younger counterparts and constitute a vulnerable group, which is more prone to develop PCS (15, 18). Therefore, the geriatric population has been defined as a priority research area for mTBI prognostic studies (19, 20). Nonetheless, few studies have investigated the prognosis of geriatric mTBI so far (1, 15, 21, 22).

Identifying prognostic markers of incomplete recovery (IR) can assist clinicians in offering optimal, personalized clinical care for those patients at risk (1, 16). Injury-related functional impairments can be assessed using the Glasgow Outcome Scale Extended (GOSE), which is the most used outcome measure in TBI studies (23, 24). This scale is often combined with the number and severity of post-injury complaints in the (sub)acute phase after mTBI to indicate IR (25, 26). In mTBI research, the lack of specificity of post-injury complaints and the presence of comorbidities, particularly, in the elderly are examples of problems that hamper prognostic studies. Machine learning (ML) algorithms are powerful tools for pattern recognition that are increasingly used due to their capacity to deal with issues present in complex data, such as a high number of (potentially dependent) predictors and non-linear relationships in the data, that more classical statistical approaches, such as (logistic) regression, are not well-suited for. In the medical field, ML techniques can be applied for identifying risk factors and for developing outcome prediction models (27–29).

In this study, we aimed to identify which individual (pre- and post-injury) complaints are potential prognostic markers for long-term functional impairments (i.e., IR) in elderly patients with mTBI. To achieve this aim, we used an interpretable ML-based approach employing a support vector machine (SVM).

Methods

Study Population

Data were obtained as part of two independent larger prospective follow-up studies. Both were conducted at a university level 1 trauma center. Patients with mTBI were consecutively included at the emergency department (ED) between March 2013 and February 2015 for the UPFRONT study and between November 2018 and September 2019 for the ReCONNECT study. The diagnosis of mTBI was based on the following criteria: attending the hospital with an mTBI defined by a Glasgow Coma Scale (GCS) score of 13–15, loss of consciousness ≤ 30 min, and/or post-traumatic amnesia of ≤ 24 h (30). For both studies, comprehension of the Dutch language was necessary. The age of inclusion for the UPFRONT study was 16 years or older and for the RECONNECT study, 60 years or older. Here, we only analyzed data from participants aged 60 years or older for whom the 6-month outcome assessment (i.e., GOSE score) was available, and missing data for predictors were no higher than 5% (i.e., maximum 2 out of 40 questions left unanswered). Exclusion criteria were a history of drug or alcohol abuse, and/or a major psychiatric or neurologic disorder as identified by the attending or ward physician. Patients without a permanent home address were also excluded. Both studies were approved by the local Medical Ethics Committee, and written informed consent was obtained from all participants. All procedures were performed according to the declaration of Helsinki (2013).

Measurements

Complaints

The Head Injury Symptoms Checklist (HISC) is a 21-item post-traumatic questionnaire (9), derived from the Rivermead Post-concussion Symptoms Questionnaire (RPQ) (10). Patients are asked to score each complaint both on the pre-injury level and on the current level with values ranging from 0 to 2 (never = 0, sometimes = 1, and often = 2). For each complaint, the post-injury score is calculated by subtracting the pre-injury score from the current score. In this manner, complaints that were new or increased in severity after mTBI could be identified. Both post-injury scores (i.e., current complaints scores corrected for pre-injury level) and pre-injury scores were used as features for classification. The HISC was administered to patients within the first 2 weeks after injury. One of the 21 symptoms (intolerance to alcohol) was excluded from the analysis as patients are usually refraining from alcohol consumption in the (sub)acute phase after injury.

Outcome

The GOSE measures functional outcome by TBI-related changes relative to pre-injury abilities on a scale that ranges from 1 (death) to 8 (complete recovery, CR). Questions, where there has been no change in comparison to pre-injury functioning status, were ignored for the definitive scoring in accordance with the manual (31). The outcome was dichotomized into CR (GOSE = 8) and IR (GOSE ≤ 7) (32). The GOSE was administered 6 months after the injury for both UPFRONT- and ReCONNECT studies. Previous studies indicate that functional outcome stabilizes around 3 months after injury. At 6 months, further improvements in the functional status are therefore unlikely for mTBI (33).

ML Approach

Predicting the outcome of a patient with mTBI (i.e., distinguishing between CR and IR) can be considered a binary classification problem.

In the context of ML, SVMs provide one of the commonly used supervised learning algorithms that perform well on binary classification problems and minimize overfitting (34). In supervised methods, the model is trained using input data with known labels that are provided by the user.

In this study, we designed an SVM-based classification model to separate patients with IR and patients with CR (class labels defined as 1 and −1, respectively).

Although SVM is a well-established ML method, employing it to determine prognostic markers based on self-reported complaints is, to the best of our knowledge, a novel approach. Therefore, to validate our approach, we performed a preliminary analysis on a similar, but supposedly simpler, classification problem, which consisted of discriminating patients in the (sub)acute phase after injury from healthy controls based on self-reported complaints. The details of this preliminary analysis are presented in the Supplementary Material.

Data Preparation

Classification features: The feature vector x(k), where k is the total number of features, consisted of the 20 pre-injury and the 20 post-injury scores measured with the HISC questionnaire, thus k = 40. The rows of the classification feature matrix X(m, k) consisted of the feature vectors from all observations, where m is the total number of participants. The class labels vector y(m) contained the class for each feature vector in X (CR or IR).

Cross-validation training, external validation, and independent testing subsets: Data of the UPFRONT study were used for cross-validation training and external validation. The cross-validation training and the external validation subsets consisted of a random, stratified selection of 80 and 20% of the UPFRONT study dataset (35), respectively, and were obtained by using the function “cvpartition” (MATLAB 2020). The external validation dataset was held out from the cross-validation training process and left unseen until external validation. All data from the ReCONNECT study dataset were used for independent testing (i.e., confirmatory analysis) and were left unseen during the cross-validation training and external validation. The datasets from ReCONNECT and UPFRONT studies were not combined to avoid adding heterogeneity to the training dataset, which could reduce the efficiency of the feature selection process.

Model Training and Features Selection

We built a script based on an SVM training algorithm (“fitcsvm,” MATLAB 2020) and a recursive backward feature selection procedure. The SVM algorithm returns trained SVM classifiers for binary classification based on a matrix with data from predictive features (X) and a vector (y) with the associated known labels. X and y were used as input to the SVM training algorithm using a linear kernel function. Hyperparameters were defined as: kernel scale (default, 1) and Box Constraint (default, 1).

A recursive backward selection method was used for the subset selection of features. The order of the features (complaints) followed the same order as they are listed in the original questionnaire. At the first step, all features (individual complaint scores as assessed by the HISC questionnaire) were selected. At each subsequent step, the number of features was reduced by one. Additionally, at each subsequent step, all observations were initially included and observations with missing data (coded as “NaN”; Not a Number) for any of the selected features were (stepwise) identified and excluded from the dataset to maximize data usage per participant. The feature to be removed is the one that results in the smallest loss of classification performance (as measured by average the area under the receiver operating characteristic curve [ROC-AUC]) when removed. The average ROC-AUC for a given subset was calculated based on r-repeated k-fold cross-validation (r = 10, k = 5) (35). In the case of multiple occurrences with an identical result, the first occurrence was selected. The procedure was repeated until only two features were left. The selected (locally) optimal number of features correspond to the smallest number of features that attained the maximum average classification performance (i.e., maximum average ROC-AUC based on 10-repeated 5-fold cross-validation). The selected (locally) optimal subset of features corresponds to the subset with the selected number of features that attained the maximum classification performance (i.e., maximum ROC-AUC based on 10-repeated 5-fold cross-validation).

After the feature selection process, the final model was trained using the optimal subset of features. This approach reduces the risk of overfitting.

Statistical Analysis and Performance Assessment

We verified whether there were any differences between (UPFRONT and ReCONNECT) datasets in age and 6-month outcome by independent-samples median tests and Chi-square tests, respectively (as variables were non-normally distributed). Additionally, we verified whether there were any differences in age between groups (CR and IR), per dataset, by independent-samples median tests.

For the final model, we calculated the ROC-AUC to assess model performance (36). Secondary performance measures were sensitivity, specificity, F1-score, and total accuracy [please refer to Sokolova et al. (37) for details about their calculation]. For the feature weights and the performance measures (ROC-AUC, sensitivity, specificity, and total accuracy), mean and 95% CIs were calculated based on 5-fold cross-validation (35).

Statistical significance of mean weights being different from the statistical distribution under the null hypothesis was calculated using permutation tests (38, 39). For 10,000 permutations, new models were built after randomly permuting class labels to empirically estimate the statistical distribution of the weights under the null hypothesis. The p-values were calculated as the proportion in the distribution under the null hypothesis that is greater than or equal to the absolute value of the mean weight value obtained by using the original (non-permutated) data.

Results

Participant Characteristics

In total, 148 elderly with mTBI were included in this study (median age: 67 years, IQR: 9 years; UPFRONT: n = 115; ReCONNECT: n = 33).

The summary of characteristics per (UPFRONT and ReCONNECT) dataset of patients with mTBI and for all patients (both datasets combined) is presented in Table 1. Independent-samples median tests neither indicated any significant difference in age between datasets nor between groups (IR vs. CR) as assessed per dataset.

TABLE 1

Table 1. Patient characteristics per dataset.

The prevalence of post-injury complaints per group (CR or IR) for both datasets combined is presented in Figure 1. Additionally, the prevalence of pre-injury complaints per group (CR or IR) is presented in Figure 2.

FIGURE 1

Figure 1. Overview of the prevalence of self-reported post-injury complaints (%) in patients with mTBI (UPFRONT and ReCONNECT datasets) within 2 weeks after injury per outcome group (IR: incomplete recovery or CR: complete recovery). Complaints ordered by prevalence in the IR group from highest to lowest. There were no differences between IR and CR groups (Mann-Whitney U test, p < 0·05, Bonferroni corrected for multiple comparisons). mTBI, mild traumatic brain injury.

FIGURE 2

Figure 2. Overview of the prevalence of self-reported pre-injury complaints (%) in patients with mTBI within 2 weeks after injury per outcome group (IR: incomplete recovery or CR: complete recovery). Complaints ordered by prevalence in the IR group from highest to lowest. There were no differences between groups (Mann Whitney U test, p < 0.05, Bonferroni corrected for multiple comparisons). mTBI, mild traumatic brain injury.

Feature Selection and Performance Assessment

The ML feature selection procedure showed that the maximum average ROC-AUC of 0.77 was achieved with 8 out of the 40 available features. Subsequently, the best model with eight features was determined that achieve the maximum ROC-AUC of 0.81 (see Figure 3).

FIGURE 3

Figure 3. Boxplots of the average area under the receiver operating characteristic curve (ROC-AUC) per number of features in the subset during the feature selection stage using the cross-validation dataset.

The selected subset containing eight features for prediction, ranked by order of exclusion, is detailed in Table 2. The ML feature selection procedure identified 3 out of 40 assessed pre- and post-injury complaint scores as significant predictors of IR, namely, post-injury neck pain score (weight: 1.38 [95% CI: 1.09, 1.68], p = 0.001), post-injury irritability score (weight: 1.91 [95% CI: 1.61, 2.21], p = 0.011), and post-injury forgetfulness score (weight: 1.28 [95% CI: 1.02, 1.54], p = 0.035) as shown in Table 2. Additionally, post-injury arm pain (Weight: −1.02 [95% CI: −1.15, −0.88], p = 0.021) was the only complaint score identified as a significant predictor of CR.

TABLE 2

Table 2. Optimal subset of features for prediction ranked by order of exclusion if selection process continued until the last feature (1: last feature to be excluded).

The final trained, cross-validated model yielded an average ROC-AUC of 0.80 [95% CI: 0.74, 0.86] (see Figure 4A). The confusion matrices displaying the classification results for the cross-validation training, for the external validation, and the independent testing are also shown in Figures 4B–D.

FIGURE 4

Figure 4. (A) Receiver operating characteristic curve for prediction of group classification—complete recovery (CR; class label: 1) vs. incomplete recovery (IR; class label: −1)—based on a linear support vector machine (SVM) model built with the optimal subselection of discriminating features and 5-fold cross-validation. Dashed lines indicate results per fold. (B–D) Confusion matrices for the predictions of the SVM model vs. true labels on cross-validation dataset [(B); 80% of the UPFRONT dataset], external validation on unseen data [(C); 20% of the UPFRONT dataset], and testing on an independent dataset [(D); ReCONNECT dataset].

Secondary performance measures are detailed in Table 3. The classifier achieved slightly higher performance for sensitivity in the external validation (Sensitivity_EV = 0.73) and in the independent testing (Sensitivity_IT = 0.68) than in the cross-validation training (Sensitivity_CV = 0.65 [95% CI 0.46, 0.83]). The performance for specificity, however, was lower in the independent testing (Specificity_IT = 0.57) than in the cross-validation training (Specificity_CV = 0.76 [95% CI 0.61, 0.92]) and in the external validation (Specificity_EV = 0.78).

TABLE 3

Table 3. Performance of selected predictive support vector machine (SVM) model on cross-validation dataset and external validation on an independent dataset.

Discussion

The main goal of this study was to identify which individual pre- and post-injury complaints in the elderly during the (sub)acute phase after mTBI are the strongest predictors (i.e., potential prognostic markers) for IR. We used an ML approach employing an SVM to robustly deal with correlated predictors, achieve high classification performance and optimal interpretability of results. This methodology allowed us to assess how different complaints relate to functional outcomes after mTBI.

The classification model achieved a good performance (ROC-AUC 0.80) based on a subset of 8 (out of 40) self-reported pre- and post-injury complaints. From this subset, three post-injury scores (which are corrected for pre-injury level) were identified as significant predictors for IR and one as a significant predictor for CR, whereas none of the pre-injury scores were identified as significant predictors for IR/CR, independently. We will detail and discuss our findings below.

Our results indicated post-injury neck pain, irritability, and forgetfulness scores as the most relevant predictors for IR in the elderly population. Importantly, these three aforementioned scores represent post-injury complaints that are part of, respectively, physical, emotional, and cognitive domains, corroborating with the notion that the involvement of a higher number of domains among post-injury complaints makes PCS (4, 5) and thus IR more likely.

In comparison to the cross-validation, the external validation achieved similar sensitivity and specificity levels, indicating that the model generalized for unseen data. The independent testing achieved similar sensitivity levels, but lower specificity levels, meaning that predictors of CR did not generalize well to a different population and should be interpreted with additional caution. Perhaps complaints among patients that recover well are less homogeneous than complaints among patients with IR. To the goal of this study (i.e., identifying potential prognostic markers for elderly patients at risk of IR), sensitivity is more important than specificity, however. In other words, for this study, missing a patient at risk of IR would be more costly than mistakenly classifying a patient that will probably recover well as being at risk of IR. Therefore, we will focus on discussing the predictors of IR.

Concerning the role of cognitive impairments in the recovery process in older adults after mTBI, we found that the self-assessment of increased forgetfulness after injury entails prognostic value for determining IR. Currently, there is increased awareness of a possible association between sustaining an mTBI at an older age and the early onset of neurodegenerative diseases, such as mild cognitive impairment and dementia (40–42). Although this possible association suggests that mTBI may act as a stressor in the normal aging process (42, 43), a clear mechanism relating (pre- and post-injury) cognitive complaints to functional outcome after mTBI has not been established. Here, the possibility that (some of the) patients were facing pre-existing memory deficits due to reasons that are unrelated to mTBI and/or misattributed increased forgetfulness to the injury cannot be ruled out, although patients with either pre-diagnosed or suspected neurodegenerative diseases were not included in this study. Moreover, it is known that several older patients with mTBI are able to cope with pre- and post-injury cognitive impairments and recover well (15). Perhaps our and previous results could be elucidated by the theoretical concept of cognitive reserve, which attempts to explain individual differences in cognitive functioning after brain damage (44). According to this theoretical framework, the elderly might cope with brain damage without clinical manifestation as long as cognitive reserve allows them to maintain cognitive performance (44, 45). We encourage future studies to disentangle the mechanisms behind cognitive functioning/decline, the development of cognitive complaints, and the role of cognitive reserve in the recovery process after mTBI at the older age.

With regard to the emotional domain, we found a post-injury irritability score as a predictor of IR after sustaining an mTBI at an older age. Previous studies on adults with mTBI identified irritability among post-traumatic complaints that can persist for longer than 1 year (46, 47) and are associated with its severity (48). Interestingly, Yang et al. (49) studied adults with mild- and moderate-to-severe TBI and found that annoyance (i.e., an aspect of irritability) was self-reported by patients with mTBI but remained unnoticed by their caregivers. Moreover, post-injury irritability might be associated with impaired cognitive capabilities, such as information processing in adults after mTBI (49). Our results add to evidence that both increases in irritability and forgetfulness from pre- to post-injury (as self-reported in the subacute phase) indicate an increased risk for worse outcomes in the elderly mTBI population. Therefore, the early assessment of subjective complaints regarding alterations in cognitive capabilities and their interaction warrants further investigation.

Regarding the role of physical impairments in the recovery process, we identified the post-injury neck pain score as a predictor for IR in elderly patients. Previous studies suggested that acceleration/deceleration of the head during mTBI can cause concomitant damage to cervical spine tissues, which might be reflected by (acute) neck pain (50). Further, previous studies indicated that a possible interplay (between mTBI and concomitant cervical involvement) might lead to increased risk for the development of PCS (50–52) and/or incomplete functional recovery in patients with mTBI. Our findings suggest that cervical involvement in the injury might be a risk factor for IR in elderly patients, as well.

Additionally, we identified post-injury arm pain score as a predictor for CR. Perhaps pain (outside neck/head region) during the subacute phase after an injury is found across patients regardless of their outcome, reflecting merely contusional lesions to muscles and joints that mostly resolve well over time and, in the absence of (a combination of) post-injury neck pain and/or emotional and cognitive complaints, do not affect the course of recovery (53?). However, the prediction of CR did not generalize well to the independent dataset and we therefore refrain from further interpreting this result.

Overall, our findings suggest an interplay between physical, emotional, and cognitive limitations interfering with long-term recovery in elderly with mTBI. Notably, the most predictive factors for IR (i.e., forgetfulness, irritability, and neck pain) were not the most prevalent (i.e., fatigue, headache, and dizziness). As such, certain somatic complaints might be typical, perhaps transient complaints that are found in the early phase after mTBI (at an older age), with reduced value for determining long-term outcome. Another interesting finding is that not a high number but a selection of complaints can yield more prognostic information.

Clinical implications of the aforementioned findings include the potential of improving the identification of the subgroup of older adults at risk of experiencing an IR trajectory after mTBI. Additionally, information from questionnaires could be used more efficiently, either by reducing the number of questions to be asked or by increasing interpretability (e.g., developing additional (sub)scores, according to the purpose of the assessment). Nevertheless, the mechanisms underlying this interaction between older age and mTBI sequelae, comprising cognitive, emotional, and physical complaints, among other factors, are still largely unknown. Although the self-assessment of complaints can provide valuable and predictive information, the possibility that (some of the) patients might misattribute complaints as being new or having their severity increased after mTBI exists, which hampers the identification of mTBI sequelae at an individual level. Future studies are encouraged to assess a selection of individual self-reported complaints and include clinically diagnosed pre-injury factors and/or mTBI symptoms in their analysis to advance the knowledge in the field.

Moreover, we identified four post-injury scores (which are controlled for pre-injury complaints status) as significant predictors for IR/CR, whereas none of the pre-injury scores were separately identified as significant predictors. Therefore, the subjective assessment of complaints as being new or having their severity increased after mTBI (which requires pre-injury assessment) has a predictive value, although the subjective assessment of pre-injury complaints (independently) did not yield predictive information, possibly due to inaccuracies. Of note, the HISC questionnaire measures the (pre- and post-injury) severity of a list of commonly occurring complaints after TBI and is not designed to provide a complete clinical assessment of pre-injury status. Therefore, other pre-injury factors (e.g., physical activity level, social engagement, and comorbidities under medical treatment or not) were not analyzed in this study but could potentially be predictors for IR/CR, as well.

Further, our outcome classification model (IR vs. CR) achieved a good performance (ROC-AUC 0.80) based on only a subset of self-reported complaints. Nonetheless, the performance of our classifier is comparable or even superior to previous predictive models that were built based on a broad variety of predictors (e.g., clinical characteristics, injury mechanism, and among others) (54), suggesting a benefit in applying our ML approach. We chose to focus solely on self-reported complaints to explore this specific aspect of the mTBI clinical assessment in elderly patients. At an older age, complaints can widely vary in number and severity both pre- and post-injury making them difficult to be processed and disentangled. Therefore, if prognostic markers can be identified from questionnaires, the burden for the clinicians when assessing large amounts of patient information to yield information for prognosis could be reduced by an ML approach. Possibly our findings can improve the early identification of the subgroup of older patients at risk of IR after mTBI, increase the usefulness of questionnaires, and lead to new insights toward personalized treatment.

Methodological Considerations, Strengths, and Limitations

In our study, we opted for a linear SVM approach combined with backward feature selection. In comparison to classical statistical approaches, this method deals adequately with variables that (highly) correlate with each other (i.e., multicollinearity) and is less affected by the presence of outliers in the data. The application of ML for the analysis of individual complaints after mTBI is, to the best of our knowledge, a new approach. Therefore, to explore the validity of this ML approach, we performed a preliminary analysis on a similar, presumably simpler, classification problem (patients with mTBI vs. healthy controls; see Supplementary Material). In the preliminary analysis, we achieved an excellent performance (ROC-AUC 0.91) based on a relatively small set (9 out of 20) of self-reported complaints. Although the results of this preliminary analysis have low clinical value (clinicians can already distinguish a patient with mTBI from a healthy control), this step allowed us to confirm the validity of the proposed approach.

Additionally, the linear SVM variant allows reasonably straightforward interpretability, which is highly desirable in exploratory analysis and for the translation of results to the clinical setting. However, although linear SVM can robustly deal with outliers and some level of non-linearity, linear methods are most likely suboptimal if dealing with heterogeneous datasets, which is frequently the case when analyzing clinical populations (such as an mTBI population).

Regarding the optimization procedure, which is performed to increase performance by selecting the most important features for classification, the following should be taken into consideration. This procedure discards not solely the features with poor predictive power, but also those features that are highly correlated to the selected ones.

Some limitations to this study should be mentioned. First, self-reported complaints are one of several aspects that determine outcome after mTBI. While focusing on self-reported complaints allowed to gain knowledge on this aspect of outcome determination, complaints are more likely to be associated with IR than with CR. Therefore, the identification and inclusion of prognostic factors to the model that is likely to be associated with CR (e.g., higher education level) would possibly lead to higher classification performance.

Second, as previously mentioned, self-reported information entails subjectivity and does not provide a complete assessment of pre- and post-injury status. It is possible that (some of the) patients have other comorbidities interfering with the recovery trajectory, have misattributed complaints to the mTBI, and/or have developed complaints about reasons that are unrelated to the mTBI. Therefore, it is not possible to demonstrate causal relationships between mTBI and any (combination of) the assessed complaints. Nonetheless, self-reported complaints are easy to obtain and, frequently, are one of a few sources of information about the pre-injury status. With the present study, we demonstrate that self-reported complaints, despite the subjectivity involved, entail prognostic value for IR after mTBI at the older age. Future studies are encouraged to investigate underlying mechanisms to explain a possible association of post-traumatic complaints with outcomes.

Third, factors that are either indirectly related or unrelated to the brain injury itself might partly contribute to long-term IR. As an example, it has been previously identified that psychological factors (i.e., emotional distress and maladaptive coping experienced early after injury) in combination with pre-injury mental health problems, education, and age are important predictors for recovery in adult mTBI populations (32). In the elderly, more complex interactions between mTBI sequelae, (pre-existing) factors, and/or co-occurring injuries are likely to play a role in the recovery process. Hence, the identification of functional impairments that are specific to mTBI sequelae becomes even more challenging in this group, which might in fact render the term post-concussive syndrome inapplicable for this population. In sum, it is possible that some of the patients with IR (i.e., GOSE ≤ 7), while being fully recovered from any neurological sequelae from mTBI, were still experiencing functional impairments at 6 months after injury due to other pre-existing factors or co-occurring injuries.

Finally, although we performed testing using an independent dataset to verify the generalizability of the model, our sample size was relatively small. Future confirmatory studies with the use of this ML approach are encouraged both to identify new individual prognostic factors (e.g., injury characteristics and personality factors) that could increase the performance of outcome prediction models and to verify the potential of the identified prognostic markers for generalizing to other populations.

Conclusion

A subset of individual pre- and post-injury complaints self-reported in the (sub)acute phase after mTBI can predict long-term functional outcomes in the elderly population. ML is a valuable approach for the identification of potential prognostic markers. The early assessment of physical complaints that indicate cervical spine involvement, subjective irritability, and (both pre- and post-injury) memory functioning impairments might facilitate the identification of elderly at risk of poor outcome, improving the rehabilitation process. Further studies are needed to confirm the validity of the identified self-reported complaints as prognostic markers and to clarify the mechanisms that associate them with the recovery process after mTBI.

Data Availability Statement

The datasets presented in this article are not readily available because patients did not consent to sharing data for purposes other than the current study. Requests to access the datasets should be directed to Mayra Bittencourt, m.bittencourt.villalpando@umcg.nl.

Ethics Statement

The studies involving human participants were reviewed and approved by Medical Ethical Committee of the University Medical Center Groningen. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

MB: conceptualization, methodology, software, formal analysis, investigation, writing–original draft. SB-S: investigation, data curation, and writing—review and editing. NM: supervision and writing—review & editing. JN: supervision, writing—review and editing, and funding acquisition. All authors critically revised and made substantial contributions to the final document.

Funding

This study was partly funded by the Dutch Brain Foundation (grant no. Ps2012-06).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

The authors thank Brigit Klever at the University Medical Center Groningen for her contribution to data acquisition for the ReCONNECT study.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fneur.2021.751539/full#supplementary-material

References

1. Carroll LJ, Cancelliere C, Côté P, Hincapié CA, Kristman VL, Holm LW, et al. Systematic review of the prognosis after mild traumatic brain injury in adults: cognitive, psychiatric, and mortality outcomes: results of the international collaboration on mild traumatic brain injury prognosis. Arch Phys Med Rehabil. (2014) 95:S152–73. doi: 10.1016/j.apmr.2013.08.300

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Levin HS, Diaz-Arrastia RR. Diagnosis, prognosis, and clinical management of mild traumatic brain injury. Lancet Neurol. (2015) 14:506–17. doi: 10.1016/S1474-4422(15)00002-2

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Cassidy JD, Carroll L, Peloso P, Borg J, von Holst H, Holm L, et al. Incidence, risk factors and prevention of mild traumatic brain injury: results of the who collaborating centre task force on mild traumatic brain injury. J Rehabil Med. (2004) 36:28–60. doi: 10.1080/16501960410023732

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Dwyer B, Katz DI. Postconcussion syndrome. Handb Clin Neurol. (2018) 158:163–78. doi: 10.1016/B978-0-444-63954-7.00017-3

PubMed Abstract | CrossRef Full Text | Google Scholar

5. World Health Organization. The ICD-10 classification of mental and behavioural disorders: Diagnostic criteria for research. Washington, DC: WHO (1993).

Google Scholar

6. Iverson GL. Outcome from mild traumatic brain injury. Curr Opin Psychiatry. (2005) 18:301–17. doi: 10.1097/01.yco.0000165601.29047.ae

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Voormolen DC, Cnossen MC, Polinder S, Gravesteijn BY, Von Steinbuechel N, Real RGL, et al. Prevalence of post-concussion-like symptoms in the general population in Italy, The Netherlands and the United Kingdom. Brain Inj. (2019) 33:1078–86. doi: 10.1080/02699052.2019.1607557

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Lagarde E, Salmi L-R, Holm LW, Contrand B, Masson F, Ribéreau-Gayon R, et al. Association of symptoms following mild traumatic brain injury with posttraumatic stress disorder vs. postconcussion syndrome. JAMA Psychiatry. (2014) 71:1032–40. doi: 10.1001/jamapsychiatry.2014.666

PubMed Abstract | CrossRef Full Text | Google Scholar

9. de Koning ME, Gareb B, El Moumni M, Scheenen ME, van der Horn HJ, Timmerman ME, et al. Subacute posttraumatic complaints and psychological distress in trauma patients with or without mild traumatic brain injury. Injury. (2016) 47:2041–7. doi: 10.1016/j.injury.2016.04.036

PubMed Abstract | CrossRef Full Text | Google Scholar

10. King NS, Crawford S, Wenden FJ, Moss NEG, Wade DT. The rivermead post concussion symptoms questionnaire: a measure of symptoms commonly experienced after head injury and its reliability. J Neurol. (1995) 242:587–92. doi: 10.1007/BF00868811

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Jennett B, Snoek J, Bond MR, Brooks N. Disability after severe head injury: observations on the use of the Glasgow Outcome Scale. J Neurol Neurosurg Psychiatry. (1981) 44:285–93. doi: 10.1136/jnnp.44.4.285

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Wilde EA, Whiteneck GG, Bogner J, Bushnik T, Cifu DX, Dikmen S, et al. Recommendations for the use of common outcome measures in traumatic brain injury research. Arch Phys Med Rehabil. (2010) 91:1650–60.e17. doi: 10.1016/j.apmr.2010.06.033

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Voormolen DC, Cnossen MC, Spikman J, Polinder S, Iverson GL, de Koning M, et al. Rating of pre-injury symptoms over time in patients with mild traumatic brain injury: the good-old-days bias revisited. Brain Inj. (2020) 34:1001–9. doi: 10.1080/02699052.2020.1761563

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Iverson GL, Lange RT, Brooks BL, Lynn Ashton Rennison V. “Good old days” bias following mild traumatic brain injury. Clin Neuropsychol. (2010) 24:17–37. doi: 10.1080/13854040903190797

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Gardner RC, Dams-O'Connor K, Morrissey MR, Manley GT. Geriatric traumatic brain injury: epidemiology, outcomes, knowledge gaps, and future directions. J Neurotrauma. (2018) 35:889–906. doi: 10.1089/neu.2017.5371

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Cassidy JD, Cancelliere C, Carroll LJ, Côté P, Hincapié CA, Holm LW, et al. Systematic review of self-reported prognosis in adults after mild traumatic brain injury: results of the international collaboration on mild traumatic brain injury prognosis. Arch Phys Med Rehabil. (2014) 95:S132–51. doi: 10.1016/j.apmr.2013.08.299

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Jacobs B, Beems T, Stulemeijer M, van Vugt AB, van der Vliet TM, Borm GF, et al. Outcome prediction in mild traumatic brain injury: age and clinical variables are stronger predictors than CT abnormalities. J Neurotrauma. (2010) 27:655–68. doi: 10.1089/neu.2009.1059

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Dams-O'Connor K, Gibbons LE, Landau A, Larson EB, Crane PK. Health problems precede traumatic brain injury in older adults. J Am Geriatr Soc. (2016) 64:844–48. doi: 10.1111/jgs.14014

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Kristman VL, Borg J, Godbolt AK, Salmi LR, Cancelliere C, Carroll LJ, et al. Methodological issues and research recommendations for prognosis after mild traumatic brain injury: results of the international collaboration on mild traumatic brain injury prognosis. Arch Phys Med Rehabil. (2014) 95:S265–77. doi: 10.1016/j.apmr.2013.04.026

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Carroll L, Cassidy JD, Peloso P, Borg J, von Holst H, Holm L, et al. Prognosis for mild traumatic brain injury: results of the who collaborating centre task force on mild traumatic brain injury. J Rehabil Med. (2004) 36:84–105. doi: 10.1080/16501960410023859

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Thompson HJ, McCormick WC, Kagan SH. Traumatic brain injury in older adults: epidemiology, outcomes, and future implications. J Am Geriatr Soc. (2006) 54:1590–5. doi: 10.1111/j.1532-5415.2006.00894.x

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Kristman VL, Brison RJ, Bédard M, Reguly P, Chisholm S. Prognostic markers for poor recovery after mild traumatic brain injury in older adults: a pilot cohort study. J Head Trauma Rehabil. (2016) 31:E33–43. doi: 10.1097/HTR.0000000000000226

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Shukla D, Devi BI, Agrawal A. Outcome measures for traumatic brain injury. Clin Neurol Neurosurg. (2011) 113:435–41. doi: 10.1016/j.clineuro.2011.02.013

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Jennett B, Bond M. Assessment of outcome after severe brain damage. A practical scale. Lancet. (1975) 305:480–4. doi: 10.1016/S0140-6736(75)92830-5

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Hiploylee C, Dufort PA, Davis HS, Wennberg RA, Tartaglia MC, Mikulis D, et al. Longitudinal study of postconcussion syndrome: not everyone recovers. J Neurotrauma. (2017) 34:1511–23. doi: 10.1089/neu.2016.4677

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Ducos Y, Aghakhani N. Prognostic factors for unfavorable outcome after mild traumatic brain injury. A review of literature. Neurochirurgie. (2021) 67:259–64. doi: 10.1016/j.neuchi.2020.04.129

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Hahn T, Nierenberg AA, Whitfield-Gabrieli S. Predictive analytics in mental health: applications, guidelines, challenges and perspectives. Mol Psychiatry. (2017) 22:37–43. doi: 10.1038/mp.2016.201

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Mourao-Miranda J, Reinders AATS, Rocha-Rego V, Lappin J, Rondina J, Morgan C, et al. Individualized prediction of illness course at the first psychotic episode: a support vector machine MRI study. Psychol Med. (2012) 42:1037–47. doi: 10.1017/S0033291711002005

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Yu JS, Xue AY, Redei EE, Bagheri N. A support vector machine model provides an accurate transcript-level-based diagnostic for major depressive disorder. Transl Psychiatry. (2016) 6:e931. doi: 10.1038/tp.2016.198

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Vos PE, Alekseenko Y, Battistin L, Ehler E, Gerstenbrand F, Muresanu DF, et al. Mild traumatic brain injury. Eur J Neurol. (2012) 19:191–8. doi: 10.1111/j.1468-1331.2011.03581.x

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Wilson JTL, Pettigrew LEL, Teasdale GM. Structured interviews for the glasgow outcome scale and the extended glasgow outcome scale: guidelines for their use. J Neurotrauma. (1998) 15:573–80. doi: 10.1089/neu.1998.15.573

PubMed Abstract | CrossRef Full Text | Google Scholar

32. van der Naalt J, Timmerman ME, de Koning ME, van der Horn HJ, Scheenen ME, Jacobs B, et al. Early predictors of outcome after mild traumatic brain injury (UPFRONT): an observational cohort study. Lancet Neurol. (2017) 16:532–40. doi: 10.1016/S1474-4422(17)30117-5

PubMed Abstract | CrossRef Full Text | Google Scholar

33. McMahon PJ, Hricik A, Yue JK, Puccio AM, Inoue T, Lingsma HF, et al. Symptomatology and functional outcome in mild traumatic brain injury: results from the prospective TRACK-TBI study. J Neurotrauma. (2014) 31:26–33. doi: 10.1089/neu.2013.2984

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Vapnik VN. The Nature of Statistical Learning Theory. New York, NY: Springer Verlag (2000). doi: 10.1007/978-1-4757-3264-1

CrossRef Full Text | Google Scholar

35. Varoquaux G, Raamana PR, Engemann DA, Hoyos-Idrobo A, Schwartz Y, Thirion B. Assessing and tuning brain decoders: Cross-validation, caveats, and guidelines. Neuroimage. (2017) 145:166–79. doi: 10.1016/j.neuroimage.2016.10.038

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Bradley AP. The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognit. (1997) 30:1145–59. doi: 10.1016/S0031-3203(96)00142-2

CrossRef Full Text | Google Scholar

37. Sokolova M, Japkowicz N, Szpakowicz S. Beyond accuracy, F-score and ROC: a family of discriminant measures for performance evaluation. In: Sattar A, Kang B, editors. AI 2006: Advances in Artificial Intelligence. AI 2006. Lecture Notes in Computer Science, vol 4304. Berlin; Heidelberg; Boston, MA: Springer (2006) 1015–21. doi: 10.1007/11941439_114

CrossRef Full Text | Google Scholar

38. Mourão-Miranda J, Bokde ALW, Born C, Hampel H, Stetter M. Classifying brain states and determining the discriminating activation patterns: support vector machine on functional MRI data. Neuroimage. (2005) 28:980–95. doi: 10.1016/j.neuroimage.2005.06.070

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Ojala M, Garriga GC. Permutation tests for studying classifier performance. J Mach Learn Res. (2010) 11:1833–63. doi: 10.1109/ICDM.2009.108

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Rapoport MJ, Feinstein A. Outcome following traumatic brain injury in the elderly: a critical review. Brain Inj. (2000) 14:749–61. doi: 10.1080/026990500413777

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Godbolt AK, Cancelliere C, Hincapié CA, Marras C, Boyle E, Kristman VL, et al. Systematic review of the risk of dementia and chronic cognitive impairment after mild traumatic brain injury: results of the international collaboration on mild traumatic brain injury prognosis. Arch Phys Med Rehabil. (2014) 95:S245–56. doi: 10.1016/j.apmr.2013.06.036

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Gardner RC, Burke JF, Nettiksimmons J, Kaup A, Barnes DE, Yaffe K. Dementia risk after traumatic brain injury vs nonbrain trauma. JAMA Neurol. (2014) 71:1490. doi: 10.1001/jamaneurol.2014.2668

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Johnson VE, Stewart W. Traumatic brain injury: age at injury influences dementia risk after TBI. Nat Rev Neurol. (2015) 11:128–30. doi: 10.1038/nrneurol.2014.241

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Stern Y, Arenaza-Urquijo EM, Bartrés-Faz D, Belleville S, Cantilon M, Chetelat G, et al. Whitepaper: Defining and investigating cognitive reserve, brain reserve, and brain maintenance. Alzheimer's Dement. (2020) 16:1305–11. doi: 10.1016/j.jalz.2018.07.219

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Stern Y. Cognitive reserve: a catalyst for research. J Neurol Sci. (2009) 283:256. doi: 10.1016/j.jns.2009.02.069

CrossRef Full Text | Google Scholar

46. Yang CC, Hua MS, Lin WC, Tsai YH, Huang SJ. Irritability following traumatic brain injury: Divergent manifestations of annoyance and verbal aggression. Brain Inj. (2012) 26:1185–91. doi: 10.3109/02699052.2012.666374

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Deb S, Lyons I, Koutzoukis C. Neurobehavioural symptoms one year after a head injury. Br J Psychiatry. (1999) 174:360–5. doi: 10.1192/bjp.174.4.360

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Timmer ML, Jacobs B, Schonherr MC, Spikman JM, van der Naalt J. The spectrum of long-term behavioral disturbances and provided care after traumatic brain injury. Front Neurol. (2020) 11:246. doi: 10.3389/fneur.2020.00246

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Yang CC, Huang SJ, Lin WC, Tsai YH, Hua MS. Divergent manifestations of irritability in patients with mild and moderate-to-severe traumatic brain injury: perspectives of awareness and neurocognitive correlates. Brain Inj. (2013) 27:1008–15. doi: 10.3109/02699052.2013.794975

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Marshall CM, Vernon H, Leddy JJ, Baldwin BA. The role of the cervical spine in post-concussion syndrome. Phys Sportsmed. (2015) 43:274–84. doi: 10.1080/00913847.2015.1064301

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Kennedy E, Quinn D, Tumilty S, Chapple CM. Clinical characteristics and outcomes of treatment of the cervical spine in patients with persistent post-concussion symptoms: a retrospective analysis. Musculoskelet Sci Pract. (2017) 29:91–8. doi: 10.1016/j.msksp.2017.03.002

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Leddy JJ, Baker JG, Merchant A, Picano J, Gaile D, Matuszak J, et al. Brain or strain? Symptoms alone do not distinguish physiologic concussion from cervical/vestibular injury. Clin J Sport Med. (2015) 25:237–42. doi: 10.1097/JSM.0000000000000128

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Venkatesan UM, Rabinowitz AR, Bernier RA, Hillary FG. Cognitive reserve in individuals aging with traumatic brain injury. J Head Trauma Rehabil. (2021). doi: 10.1097/HTR.0000000000000697. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Kornblith ES, Langa KM, Yaffe K, Gardner RC. Physical and functional impairment among older adults with a history of traumatic brain injury. J Head Trauma Rehabil. (2020) 35:E320–9. doi: 10.1097/HTR.0000000000000552

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Cnossen MC, van der Naalt J, Spikman JM, Nieboer D, Yue JK, Winkler EA, et al. Prediction of persistent post-concussion symptoms after mild traumatic brain injury. J Neurotrauma. (2018) 35:2691–8. doi: 10.1089/neu.2017.5486

PubMed Abstract | CrossRef Full Text | Google Scholar