Changes in EEG Activity Following Live Z-Score Training Predict Changes in Persistent Post-concussive Symptoms: An Exploratory Analysis

A specific variant of neurofeedback therapy (NFT), Live Z-Score Training (LZT), can be configured to not target specific EEG frequencies, networks, or regions of the brain, thereby permitting implicit and flexible modulation of EEG activity. In this exploratory analysis, the relationship between post-LZT changes in EEG activity and self-reported symptom reduction is evaluated in a sample of patients with persistent post-concussive symptoms (PPCS). Penalized regressions were used to identify EEG metrics associated with changes in physical, cognitive, and affective symptoms; the predictive capacity of EEG variables selected by the penalized regressions were subsequently validated using linear regression models. Post-treatment changes in theta/alpha ratio predicted reduction in pain intensity and cognitive symptoms and changes in beta-related power metrics predicted improvements in affective symptoms. No EEG changes were associated with changes in a majority of physical symptoms. These data highlight the potential for NFT to target specific EEG patterns to provide greater treatment precision for PPCS patients. This exploratory analysis is intended to promote the refinement of NFT treatment protocols to improve outcomes for patients with PPCS.

Introduction

A ubiquitous pathophysiological consequence of mild traumatic brain injury (mTBI) is dysfunctional neural activity (1–3). For many individuals who sustain mTBI, this neural dysfunction and its associated cognitive, physical, and behavioral symptoms are transient and resolve within a matter of weeks or months. However, a minority of patients–5–30%—experience persistent post-concussive symptoms (PPCS) beyond the typical recovery period (4–6) and these symptoms are often associated with residual neural dysfunction (7–10).

Research efforts are underway to identify and validate effective treatments to improve a range of neurobehavioral symptoms in patients with PPCS. One promising treatment is neurofeedback therapy (NFT), with several studies reporting that NFT reduces symptoms in patients with TBI (11–17). NFT is a non-pharmacological treatment that uses operant conditioning to train patients to autonomously modulate neural activity (18, 19). The general premise of NFT is that patients receive positive feedback when their EEG activity achieves a desired pattern; through this process, patients implicitly learn to regulate their EEG activity to achieve desired behavioral goals. There are numerous variations of NFT that target different aspects of neural activity, including specific oscillatory frequencies (20, 21) and/or specific networks or regions of the brain (22, 23).

One type of NFT, Live Z-Score Training (LZT), is unique from common variants of NFT for its real-time, continuous calculation of Z-scores of EEG activity (24, 25). Z-scores are calculated for six EEG metrics in ten frequency bands over all channels included in the EEG montage; in a 19-channel montage, LZT produces 5,700 Z-scores (Table 1). Participants receive positive feedback when a pre-specified percentage (e.g., 60%) of all computed Z-scores falls within a pre-specified range (e.g., ±1 SD) of the normative mean (24, 25). Training can be configured to either target specific EEG activity or remain agnostic to what EEG metrics are targeted. With this latter approach, patients are trained to normalize activity in some but not all frequency bands and/or brain regions, as normalization of only a subset of EEG metrics is required for reinforcement. Because any combination of Z-scores can satisfy the criteria to receive positive feedback, there can be substantial variability between patients in the strategies they implicitly and flexibly choose for normalizing neural activity (24, 25). For example, a patient who normalizes brain-wide alpha activity and a patient who normalizes theta and beta activity over central and parietal regions may be equally successful at completing LZT. However, it is unknown how various strategies for EEG normalization, when not specified and directly targeted by NFT, correspond to symptom improvements.

TABLE 1

Table 1. Live Z-score training EEG measurement parameters.

Consistent with previous studies of NFT, a feasibility analysis recently conducted in our laboratory demonstrated that LZT was associated with symptom reduction in a sample of active duty service members and veterans with PPCS (26). Here we report results of an exploratory analysis of data from the feasibility study to (1) characterize changes in resting state EEG activity in patients with PPCS following completion of LZT and (2) evaluate how changes in EEG activity are associated with changes in symptomatology. Results from this analysis may be helpful to identify EEG activity that can be targeted by NFT to customize symptom-specific treatment in PPCS patients.

Methods

Design

This study was approved by the Madigan Army Medical Center's Institutional Review Board. Here we provide a summary of study methods; full details are provided in Hershaw et al. (26). This study used a single-group design wherein participants received LZT. Participants completed pre-treatment (T1), post-treatment (T2) and follow-up (T3) evaluations that included self-report symptom questionnaires, a full neuropsychological battery, a physiological stress test, and resting state EEG recording. T1 evaluations were conducted no more than 4 weeks prior to the start of LZT; T2 evaluations were conducted between 1 and 4 weeks following treatment; and T3 evaluations were conducted between 11 and 15 weeks following treatment.

Participants

Participants were recruited from Fort Carson Army Post and the surrounding Colorado Springs, CO area using posted recruitment material and provider referrals. To be eligible for study participation, individuals had to have active duty or veteran status, be 18–50 years old, have a history of mild to moderate TBI as defined by the American Congress of Rehabilitation Medicine (27) 3 months to 5 years prior to enrollment, and currently endorse post-concussive symptoms to include emotion dysregulation. Individuals with unstable medical or psychiatric conditions, who failed symptom validity tests, endorsed alcohol or substance abuse, or used medications known to interfere with EEG recordings were excluded from participation. A total of 38 individuals met eligibility requirements and agreed to undergo LZT.

LZT Protocol

LZT training stimuli were delivered via a video chosen by participants from the BrainMaster BrainAvatar software system (Bedford, OH: BrainMaster Technologies). As the video played, EEG data were recorded using the BrainMaster Discovery 24E amplifier using a 19-channel montage adhering to the 10–20 international electrode placement system. EEG data were decomposed into 10 frequency bands to obtain measures of absolute power, relative power, and power ratios at each electrode, and asymmetry, phase delay, and coherence for every electrode pair (Table 1). Given every possible combination of metrics and electrode pairs, 5,700 EEG metrics were computed and compared continuously in real-time to a normative database [Applied NeuroScience BrainMaster Z-Score Dynamic Link Library (Largo, FL: Applied Neuroscience, Inc.)] to derive Z-scores. When participants' EEG activity approximated normative values (as defined by pre-specified criteria described below), they received positive feedback in the form of visual or audio cues in the video.

To receive positive feedback, the following three criteria must have been met: (1) Z-scores had to fall within a “target window” of ±0.9 standard deviations of the normative mean; (2) based on a titrated threshold, a certain percentage of the 5,700 Z-scores had to fall within the target window 40–60% of time; and (3) a variable percentage of Z-scores that were outside of the target window (“outliers”) had to move toward the target window. The required percentage of outlier Z-scores moving toward the target window was adjusted continuously, as needed, such that participants received 10–15 reinforcements per minute. The target window was held constant throughout treatment, but the threshold for percentage of Z-scores needing to fall within that window varied depending on performance. The duration of each treatment session increased from 10 min at the first session to 30 min by session 6 or 7. LZT treatment took place over a period of 6 weeks. Treatment completion was defined as completing at least 15 treatment sessions; however, participants were encouraged to complete as many sessions as possible, up to 20.

Self-Reported Symptom Questionnaires

Given that many functional domains can be affected in patients with PPCS, self-report symptom questionnaires were selected to assess a broad range of symptoms. The following symptoms were assessed: (1) post-traumatic stress symptoms [PTSD Checklist-Military version, PCL-M; (28)]; (2) depressive symptoms [Patient Health Questionnaire-9, PHQ-9; (29)]; (3) neurobehavioral symptoms [Neurobehavioral Symptom Inventory, NSI; (30)]; (4) sleep quality [Medical Outcomes Study Sleep Scale, MOS-Sleep; (31)]; (5) pain [Chronic Pain Grade questionnaire, CPG; (32)]; and (6) migraines [Migraine Disability Assessment Scale, MIDAS; (33)]. The following symptom scores were derived from these questionnaires in accordance with their respective scoring manuals to be included in analyses: PCL-M total; PHQ-9 total; NSI somatic subscale; NSI affective subscale; NSI cognitive subscale; NSI vestibular subscale; MOS-Sleep total; CPG pain intensity subscale; CPG pain-related disability subscale; and MIDAS total. A decrease in all scores represents symptom improvements.

Resting State EEG Data Acquisition and Pre-processing

As part of the pre- and post-treatment evaluations, eyes open and eyes closed resting state EEG data were recorded. Data were recorded at a 1,000 Hz sampling rate using NeuroScan acquisition system (Victoria, Australia: Compumedics Neuroscan) with a 64-channel montage adhering to the 10–20 international electrode placement system. Data were processed offline using EEGlab (34). Data were downsampled to 256 Hz and a 1–100 Hz bandpass filter was applied. Data were then cleaned using the EEGlab clean_rawdata function (35), referenced to the common average, and submitted to an independent components analysis to identify and remove horizontal and vertical ocular artifacts from the data. Cleaned data were then submitted to the bandpower function in MATLAB (Version 2014b, Natick, MA: Mathworks), whereby spectral power density is estimated using a periodogram method.

EEG Variable Quantification

In an effort to reduce the computational burden and the risk of committing Type 1 errors in this exploratory analysis, we analyzed eyes open resting state data; the eyes open condition was chosen due to its properties of having fewer ocular artifacts (36), reduced large-amplitude alpha rhythm that is dominant during eyes closed resting periods (36), and greater sensitivity to neural dysfunction compared to eyes closed EEG activity (37). Resting state EEG data was quantified to maximize fidelity to the EEG metrics for which Z-scores were computed as part of LZT training. The system used to record resting state EEG data had a 64-channel montage; the system used to deliver LZT had a 19-channel montage (see Table 1). We limited our quantification of resting state EEG data to those channels in the 64-channel system that were also represented in the 19-channel system; however, channels T3, T4, T5, and T6 were included in the 19-channel system but not the 64-channel system. As a substitute for these channels, we included channels T7 and T8 in our quantification, as these channels were included in the 64-channel system but not in the 19-channel system. This resulted in 17 channels included in the resting state EEG analysis.

As a further effort to reduce the computational burden and the risk of committing Type 1 errors in this exploratory analysis, we analyzed a subset of the 5,700 EEG variables that are included in LZT training. Resting state EEG data were decomposed into the same 10 frequency bands that were included in LZT training (see Table 1), as well as total power (1–30 Hz). We quantified absolute power, relative power, power ratios, and asymmetry metrics. Absolute power was measured for each of the 10 frequency bands and total power. Relative power was measured as the absolute power in each of the 10 frequency bands divided by total power. Five absolute power ratios were computed: theta/beta, theta/alpha, alpha/beta, delta/theta, and delta/alpha. The first four ratios were selected because they are the most commonly reported power ratios (38) and the latter ratio was selected in light of evidence that it is related to rehabilitation following TBI (39). All power metrics were quantified for each of the 17 channels, resulting in 442 power metrics. Additionally, we computed lateral asymmetry ratios for frontal (FP2–FP1), parietal (P4–P3), and temporal (T8–T7) regions for each of the 10 frequency bands and total power. These asymmetry-based metrics brought the total number of EEG variables to 475.

Statistical Analysis

For the purpose of this analysis, we limited our comparison to pre- (T1) and post-treatment (T2) EEG metrics. EEG metrics obtained during follow-up (T3) were not included in the analysis because of existing evidence that changes in EEG activity following NFT are often not sustained and may even rebound, despite observing sustained clinical effects (40). This dissociation is attributed to a homeostatic mechanism that permits “re-normalization” of EEG activity while maintaining behavioral gains (41, 42); thus long-term measures of EEG activity may not validly represent the association between NFT-induced EEG alterations and symptom resolution.

To characterize post-treatment changes in EEG power, we compared EEG variables between T1 and T2 using paired samples t-tests. Because of the exploratory nature of this analysis, we report results both without a correction for multiple comparisons [see (43), for a thorough discussion on this topic] and with a Benjamini-Hochberg to correct for a False Discovery Rate (FDR) of 0.05 (44).

Change scores (T2–T1) were computed for all EEG variables (n = 475) and all symptom scores (n = 10). To identify what, if any, changes in EEG activity predict changes in symptomatology, a separate lasso regression was conducted for each symptom change score with all EEG change scores entered as predictor variables. Lasso regressions are optimal for variable selection when there is a large number of predictor variables (e.g., more than the number of observations) and high levels of multicollinearity. Lasso regression models impose a penalty, λ, on the value of coefficients that do not contribute to the prediction of an outcome variable, thereby shrinking to zero the coefficients of all non-predictive variables. The optimal value of λ is that which minimizes the lasso function in Equation 1.1. Thus, any variable with a non-zero coefficient is a significant predictor of the outcome. The result is a parsimonious model that includes only unrelated variables that significantly contribute to the prediction of the outcome. In effect, it is a solution to the problem of overfitting when there are a large number of predictors entered into a model.

$\begin{array}{ll}{\displaystyle \sum _{i=1}^n}{(Y_i-{\displaystyle \sum _j}{X}_{ij}{\beta }_j)}^2+\lambda {\displaystyle \sum _{j=1}^p}\left|{\beta }_j\right|& (1)\end{array}$

The degree of coefficient shrinkage increases as λ, the penalty parameter, increases for a given value of α (here, α = 1). To determine the value of λ that would produce the optimal degree of shrinkage, we used 10-fold cross-validation for each of the lasso regressions, wherein values of λ were tested on 10 random subsets of the dataset to identify the value that yielded the smallest mean square error (MSE) of prediction. The predictor coefficients that are non-zero at this optimal value of λ reflect the EEG variables selected by the model as significant predictors of the outcome. Following each lasso regression, EEG variables that were selected by the regressions were then submitted to a multiple linear regression on the symptom change score using the full dataset to further validate their predictive capacity. All regression models were tested and are reported using standardized (z-score) variables.

Results

Twenty-seven participants completed LZT treatment. One participant was excluded from analyses for unusable resting state EEG data. The final sample included in this exploratory analysis consisted of 22 males and four females with a mean age of 35.54 years (SD = 7.20). The mean time since injury to study enrollment ranged from 3 to 45 months (M = 14.16, SD = 12.82).

Of the 475 EEG variables tested, 31 (6.5%) changed significantly between T1 and T2 (Table 2); when a correction for FDR was applied, 27 (5.7%) of these variables remained significant. In the frontal and temporal regions, significant differences were noted almost exclusively for alpha metrics; in contrast, changes in additional frequency bands were observed throughout central and parietal regions.

TABLE 2

Table 2. EEG variables with significant pre-post treatment difference.

Descriptive statistics for symptom change scores are provided in Supplementary Table A. Improvements in pain intensity (Table 3) were predicted with a small margin of error by changes in lateral frontal and occipital theta/alpha ratio and occipital theta; 25% of the variance in pain reduction was accounted for by these EEG metrics. Outcomes related to emotion regulation, including PTSD-related symptoms (Table 4) and affective symptoms (Table 5), were predicted with high accuracy: nearly 20% of variance in changes in PTSD-related symptoms and almost 63% of the variance in changes in affective symptoms were explained by changes in several EEG metrics, predominantly beta-related metrics. Improvements in self-reported cognitive symptoms (Table 6) were predicted by theta/alpha ratio over the medial parietal region, with these metrics accounting for 31% of the variance in symptom reduction. Figure 1 depicts the amount of variance in each of these four outcomes that is accounted for by post-treatment changes in specific frequency bands and ratios. Descriptive statistics of change scores for all EEG variables that were selected as predictors by the lasso regressions are provided in Supplementary Table B.

TABLE 3

Table 3. Regressions on CPG intensity.

TABLE 4

Table 4. Regressions on PCL-M.

TABLE 5

Table 5. Regressions on NSI affective.

TABLE 6

Table 6. Regressions on NSI cognitive.

FIGURE 1

Figure 1. Variance in post-treatment changes in pain intensity, post-traumatic stress symptoms, affective symptoms, and cognitive symptoms accounted for by post-treatment changes in EEG activity in specific frequency bands and power ratios.

The remaining six outcome variables tested were not predicted by any of the 475 EEG variables entered in the LASSO regression models (Table 7). These outcomes included pain-related disability (CPG disability scale), somatic symptoms (NSI somatic subscale), vestibular symptoms (NSI vestibular subscale), sleep quality (MOS Sleep scale), depressive symptoms (PHQ-9), and migraine (MIDAS).

TABLE 7

Table 7. Model fits for outcomes not predicted by EEG metrics.

Discussion

In the current study, the delivery of LZT enabled us to investigate changes in EEG activity following an NFT protocol that is agnostic to specific EEG patterns or metrics. Given that our approach to LZT was unrestrained in terms of frequency bands and brain regions targeted during treatment, we sought to describe changes in EEG activity and how they relate to symptom improvements following LZT. The comparison of post-treatment to pre-treatment EEG activity revealed that patients most consistently increased frontal and central alpha activity. Our results also showed that changes in EEG activity differentially predict changes in specific symptoms. Due to the exploratory nature of this analysis and large number of comparisons, theoretical interpretations regarding these findings should be reserved for future confirmatory studies. However, these data generate important considerations regarding the development and refinement of NFT protocols for PPCS.

Our data suggest that when patients are free to implicitly modulate any oscillatory frequencies in any regions, individuals largely increase alpha activity brain-wide. Two-thirds of the EEG metrics that changed significantly following treatment were some measure of alpha activity—either relative alpha or a ratio including alpha—across the cortex. This suggests that modulation of alpha is implicitly favored over other frequencies in this specific NFT protocol. Clinicians using NFT protocols that target specific EEG patterns should consider that some frequencies in this population may be easier to modulate than others and accordingly adjust treatment parameters.

The predictive association observed between EEG activity and pain intensity has been previously reported by other authors (45). While our regression model was not statistically significant (likely due to limited power), it accounted for over 25% of the variance in the reduction in pain intensity, suggesting that these metrics contribute meaningfully to improving the subjective experience of pain in patients with PPCS. It is notable that of the 475 EEG variables entered into the model, three of the four selected as significant predictors of pain reduction were theta/alpha ratio, albeit in different regions. This pattern provides support for targeting theta/alpha ratio to treat pain in patients with PPCS. Currently, there is no evidence implicating a direct link between theta/alpha ratio and pain modulation; however, theta/alpha ratio is implicated in deep relaxation, hypnagogic states (46), and mood (47), which may mediate the relationship between NFT and pain modulation.

The predictive association between beta-related metrics and affective symptom improvements observed in our cohort is in agreement with extant evidence supporting the association of theta/beta ratio and beta power with emotion regulation (48), behavioral inhibition in emotional contexts (49), and affective processing (50). In consideration of this body of evidence, our results indicate that PPCS patients who report affective symptoms and/or emotion dysregulation may benefit from NFT that directly targets beta activity.

While it seems counterintuitive that a single EEG metric (theta/alpha ratio over the medial parietal region) accounted for 31% of the variance in cognitive symptom reduction, causal relationships between theta-targeted NFT and improved attention and working memory (51) as well as alpha-targeted NFT and improvements in spatial reasoning, executive function, and cognitive control, have been previously reported in non-clinical and non-PPCS cohorts. In conjunction with this previous evidence, our findings suggest that global cognitive improvement in this population may also be facilitated by targeting theta and alpha activity.

The six symptoms that were not predicted by EEG metrics tested in this analysis represent a variety of physical manifestations of mild TBI, including pain-related disability, somatic, vestibular, and depressive symptoms, poor sleep, and migraines. In contrast, we demonstrated that reduced pain intensity was predicted by changes in EEG activity following LZT. This dissociation suggests that NFT, while potentially effective for reducing pain, may not be sufficient to promote improvement for a broad range of physical symptoms for patients with PPCS; physical symptoms in this population may be treated more effectively using an integrative, multi-modal approach that includes NFT (52–54).

Limitations and Future Directions

The feasibility study reported here used a single-group design; accordingly, changes in outcomes between pre- and post-treatment evaluations cannot be conclusively attributed to treatment effects. Future clinical trials with a control group are necessary to ascertain that the changes we observed are, in fact, due to LZT. Similarly, we cannot infer a causal relationship between changes in EEG activity and improvements in outcomes. Our regression-based analysis can only establish that changes in symptomatology are predicted by changes in EEG activity; experimental studies comparing outcomes following NFT that targets different EEG patterns may provide additional insight into causality.

Many of the linear regressions conducted in this study revealed high proportions of variance accounted for by the selected EEG metrics while the overall models did not achieve statistical significance (p < 0.05). The small sample size of this study, in combination with the reduction in power with increasing model complexity, likely impeded our ability to detect significant effects. Despite these statistical limitations, the proportions of variance explained by several of our models is large. Additionally, the distinction between statistical significance and clinical significance is of critical importance when evaluating the effects of NFT. Whereas, we highlighted the association between changes in EEG activity and reduced symptomatology, future research is warranted to examine the magnitude of change in EEG activity needed to produce clinically meaningful symptom reduction.

Conclusions

The symptoms reported in PPCS are multifaceted and variable between patients; therefore, outcomes may be improved with greater treatment precision. Overall, we found evidence that changes to certain EEG metrics predict improvements in specific self-reported symptoms; taken together with previous findings that EEG activity in these specific frequency bands is abnormal in PPCS patients (55–57), our data corroborate existing evidence that these frequency bands may serve as critical targets for treating specific symptoms of PPCS. Specifically, we found evidence that changes in EEG activity following LZT—and NFT in general—are related to improvements in cognitive and affective symptoms, but less so for physical symptoms.

Our results describe changes in specific EEG metrics that predict reductions in specific self-reported symptoms, which indicates that unique symptom profiles of PPCS may be successfully targeted with individualized NFT protocols. This report ideally will promote further hypothesis testing of NFT for treating PPCS and promote the development and refinement of individualized, symptom-specific NFT protocols for patients with PPCS. Future research would benefit from stratifying patients in accordance with their symptom profiles or other phenotypical classifications to investigate the effects of LZT in distinct subtypes of patients and symptom clusters. In contrast, owing to its generalized and agnostic approach to targeting abnormal EEG activity, LZT has merit for treating PPCS in environments where generalized treatment is prioritized over individualized treatment, for example, if an NFT-trained clinician is unavailable or if specific symptoms cannot be dissociated or validly measured; however its superiority over more traditional and targeted forms of NFT remains to be demonstrated and warrants clinical trials (58).

Data Availability Statement

The authors are ethically and legally restricted by the Madigan Army Medical Center Internal Review Board from sharing data because participants did not consent to this form of data sharing. However, we are able to provide data access to interested individuals pending Internal Review Board approval of individual data requests. Future interested individuals may contact the lead author, Jamie N. Hershaw, for queries and requests for data.

Ethics Statement

The studies involving human participants were reviewed and approved by Madigan Army Medical Center IRB. The patients/participants provided their written informed consent to participate in this study. In the conduct of research where humans are the subjects, the investigators adhered to the policies regarding the protection of human subjects as prescribed by Code of Federal Regulations (CFR) Title 45, Volume 1, Part 46; Title 32, Chapter 1, Part 219; and Title 21, Chapter 1, Part 50 (Protection of Human Subjects).

Author Contributions

CH-P contributed to the conception and design of the study and managed all study procedures and organized the database. JH performed the statistical analysis and wrote the manuscript. All authors contributed to manuscript revision and approved the submitted version.

Funding

This research was funded by the Traumatic Brain Injury Center of Excellence (formerly Defense and Veterans Brain Injury Center) and the AMEDD Advanced Technology Initiative (AAMTI) through the Telemedicine and Advanced Technology Research Center (TATRC).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fneur.2022.714913/full#supplementary-material

References

1. Bigler ED, Maxwell WL. Neuropathology of mild traumatic brain injury: relationship to neuroimaging findings. Brain Imaging Behav. (2012) 6:108–36. doi: 10.1007/s11682-011-9145-0

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Laskowski RA, Creed JA, Raghupathi R. Pathophysiology of mild TBI: implications for altered signaling pathways. In: Kobeissy FH, editor. Brain Neurotrauma: Molecular, Neuropsychological, and Rehabilitation Aspects. CRC Press/Taylor & Francis (2015). doi: 10.1201/b18126-6

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Signoretti S, Lazzarino G, Tavazzi B, Vagnozzi R. The pathophysiology of concussion. PM R. (2011) 3:S359–68. doi: 10.1016/j.pmrj.2011.07.018

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Boake C, McCauley SR, Levin HS, Pedroza C, Contant CF, Song JX, et al. Diagnostic criteria for postconcussional syndrome after mild to moderate traumatic brain injury. J Neuropsychiatry Clin Neurosci. (2005) 17:350–6. doi: 10.1176/jnp.17.3.350

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Boyle E, Cancelliere C, Hartvigsen J, Carroll LJ, Holm LW, Cassidy JD. Systematic review of prognosis after mild traumatic brain injury in the military: results of the International Collaboration on Mild Traumatic Brain Injury Prognosis. Arch Phys Med Rehabil. (2014) 95:S230–7. doi: 10.1016/j.apmr.2013.08.297

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Røe C, Sveen U, Alvsåker K, Bautz-Holter E. Post-concussion symptoms after mild traumatic brain injury: influence of demographic factors and injury severity in a 1-year cohort study. Disabil Rehabil. (2009) 31:1235–43. doi: 10.1080/09638280802532720

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Bigler ED. Neuropsychology and clinical neuroscience of persistent post-concussive syndrome. J Int Neuropsychol Soc. (2008) 14:1–22. doi: 10.1017/S135561770808017X

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Kirov II, Tal A, Babb JS, Reaume J, Bushnik T, Ashman TA, et al. Proton MR spectroscopy correlates diffuse axonal abnormalities with post-concussive symptoms in mild traumatic brain injury. J Neurotrauma. (2013) 30:1200–4. doi: 10.1089/neu.2012.2696

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Mayer AR, Hanlon FM, Dodd AB, Ling JM, Klimaj SD, Meier TB. A functional magnetic resonance imaging study of cognitive control and neurosensory deficits in mild traumatic brain injury. Hum Brain Mapp. (2015) 36:4394–406. doi: 10.1002/hbm.22930

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Munia TT, Haider A, Schneider C, Romanick M, Fazel-Rezai R. A novel EEG based spectral analysis of persistent brain function alteration in athletes with concussion history. Sci Rep. (2017) 7:17221. doi: 10.1038/s41598-017-17414-x

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Bhandari T, Thompson L, Reid-Chung A. Treating postconcussion syndrome using neurofeedback: a case study. Biofeedback. (2013) 41:174–82. doi: 10.5298/1081-5937-41.4.03

CrossRef Full Text | Google Scholar

12. Gray SN. An overview of the use of neurofeedback biofeedback for the treatment of symptoms of traumatic brain injury in military and civilian populations. Med Acupunct. (2017) 29:215–9. doi: 10.1089/acu.2017.1220

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Koberda J. LORETA Z-score neurofeedback-effectiveness in rehabilitation of patients suffering from traumatic brain injury. J Neurol Neurobiol. (2015) 1:1–9. doi: 10.16966/2379-7150.113

CrossRef Full Text | Google Scholar

14. Linden M. The effects of QEEG-guided neurofeedback on postconcussion syndrome. Biofeedback. (2015) 43:42–4. doi: 10.5298/1081-5937-43.1.08

CrossRef Full Text | Google Scholar

15. May G, Benson R, Balon R, Boutros N. Neurofeedback and traumatic brain injury: a literature review. Ann Clin Psychiatry. (2013) 25:289–96.

PubMed Abstract | Google Scholar

16. Thomas JL, Smith ML. Neurofeedback for traumatic brain injury: current trends. Biofeedback. (2015) 43:31–7. doi: 10.5298/1081-5937-43.1.05

CrossRef Full Text | Google Scholar

17. Thompson M, Thompson L, Reid-Chung A. Treating postconcussion syndrome with LORETA Z-score neurofeedback and heart rate variability biofeedback: neuroanatomical/neurophysiological rationale, methods, case examples. Biofeedback. (2015) 43:15–26. doi: 10.5298/1081-5937-43.1.07

CrossRef Full Text | Google Scholar

18. Gaume A, Vialatte A, Mora-Sanchez A, Ramdani C, Vialatte F-B. A psychoengineering paradigm for the neurocognitive mechanisms of biofeedback and neurofeedback. Neurosci Biobehav Rev. (2016) 68:891–910. doi: 10.1016/j.neubiorev.2016.06.012

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Thibault RT, Lifshitz M, Raz A. The self-regulating brain and neurofeedback: experimental science and clinical promise. Cortex. (2016) 74:247–61. doi: 10.1016/j.cortex.2015.10.024

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Enriquez-Geppert S, Huster RJ, Scharfenort R, Mokom ZN, Zimmermann J, Herrmann CS. Modulation of frontal-midline theta by neurofeedback. Biol Psychol. (2014) 95:59–69. doi: 10.1016/j.biopsycho.2013.02.019

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Zoefel B, Huster RJ, Herrmann CS. Neurofeedback training of the upper alpha frequency band in EEG improves cognitive performance. Neuroimage. (2011) 54:1427–31. doi: 10.1016/j.neuroimage.2010.08.078

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Haller S, Kopel R, Jhooti P, Haas T, Scharnowski F, Lovblad O, et al. Dynamic reconfiguration of human brain functional networks through neurofeedback. Neuroimage. (2013) 81:243–52. doi: 10.1016/j.neuroimage.2013.05.019

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Peeters F, Ronner J, Bodar L, van Os J, Lousberg R. Validation of a neurofeedback paradigm: manipulating frontal EEG alpha-activity and its impact on mood. Int J Psychophysiol. (2014) 93:116–20. doi: 10.1016/j.ijpsycho.2013.06.010

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Collura TF. Live Z-score neurofeedback. Biofeedback. (2016) 44:212–7. doi: 10.5298/1081-5937-44.4.01

CrossRef Full Text | Google Scholar

25. Collura TF, Guan J, Tarrant J, Bailey J, Starr F. EEG biofeedback case studies using live Z-score training and a normative database. J Neurother. (2010) 14:22–46. doi: 10.1080/10874200903543963

CrossRef Full Text | Google Scholar

26. Hershaw JN, Hill-Pearson CA, Arango JI, Souvignier AR, Pazdan RM. Semi-automated neurofeedback therapy for persistent postconcussive symptoms in a military clinical setting,: a feasibility study. Milit Med. (2020) 185:e457–65. doi: 10.1093/milmed/usz335

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Kay T, Harrington DE, Adams R, Anderson T, Berrol S, Cicerone K, et al. Definition of mild traumatic brain injury. J Head Trauma Rehabil. (1993) 8:86–7. doi: 10.1097/00001199-199309000-00010

CrossRef Full Text | Google Scholar

28. Wilkins KC, Lang AJ, Norman SB. Synthesis of the psychometric properties of the PTSD checklist (PCL) military, civilian, and specific versions. Depress Anxiety. (2011) 28:596–606. doi: 10.1002/da.20837

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. (2001) 16:606–13. doi: 10.1046/j.1525-1497.2001.016009606.x

PubMed Abstract | CrossRef Full Text | Google Scholar

30. King PR, Donnelly KT, Donnelly JP, Dunnam M, Warner G, Kittleson C, et al. Psychometric study of the neurobehavioral symptom inventory. J Rehabil Res Dev. (2012) 49:879–88. doi: 10.1682/JRRD.2011.03.0051

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Hays RD, Martin SA, Sesti AM, Spritzer KL. Psychometric properties of the medical outcomes study sleep measure. Sleep Med. (2005) 6:41–4. doi: 10.1016/j.sleep.2004.07.006

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Elliott AM, Smith BH, Smith WC, Chambers WA. Changes in chronic pain severity over time: the chronic pain grade as a valid measure. Pain. (2000) 88:303–8. doi: 10.1016/S0304-3959(00)00337-7

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Stewart WF, Lipton RB, Dowson AJ, Sawyer J. Development and testing of the Migraine Disability Assessment (MIDAS) Questionnaire to assess headache-related disability. Neurology. (2001) 56:S20–8. doi: 10.1212/WNL.56.suppl_1.S20

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Delorme A, Makeig S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods. (2004) 134:9–21. doi: 10.1016/j.jneumeth.2003.10.009

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Kothe CA, Makeig S. BCILAB: a platform for brain–computer interface development. J Neural Eng. (2013) 10:056014. doi: 10.1088/1741-2560/10/5/056014

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Barry RJ, De Blasio FM. EEG differences between eyes-closed and eyes-open resting remain in healthy ageing. Biol Psychol. (2017) 129:293–304. doi: 10.1016/j.biopsycho.2017.09.010

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Miraglia F, Vecchio F, Bramanti P, Rossini PM. EEG characteristics in “eyes-open” versus “eyes-closed” conditions: small-world network architecture in healthy aging and age-related brain degeneration. Clin Neurophysiol. (2016) 127:1261–8. doi: 10.1016/j.clinph.2015.07.040

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Thatcher RW, North D, Biver C. EEG and intelligence: relations between EEG coherence, EEG phase delay and power. Clin Neurophysiol. (2005) 116:2129–41. doi: 10.1016/j.clinph.2005.04.026

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Leon-Carrion J, Martin-Rodriguez JF, Damas-Lopez J, y Martin JMB, Dominguez-Morales MR. Delta–alpha ratio correlates with level of recovery after neurorehabilitation in patients with acquired brain injury. Clin Neurophysiol. (2009) 120:1039–45. doi: 10.1016/j.clinph.2009.01.021

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Gruzelier JH. EEG-neurofeedback for optimising performance. III: a review of methodological and theoretical considerations. Neurosci Biobehav Rev. (2014) 44:159–82. doi: 10.1016/j.neubiorev.2014.03.015

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Kluetsch RC, Ros T, Théberge J, Frewen PA, Calhoun VD, Schmahl C, et al. Plastic modulation of PTSD resting-state networks and subjective wellbeing by EEG neurofeedback. Acta Psychiatr Scand. (2014) 130:123–36. doi: 10.1111/acps.12229

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Sitaram R, Ros T, Stoeckel L, Haller S, Scharnowski F, Lewis-Peacock J, et al. Closed-loop brain training: the science of neurofeedback. Nat Rev Neurosci. (2017) 18:86. doi: 10.1038/nrn.2016.164

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Bender R, Lange S. Adjusting for multiple testing—when and how? J Clin Epidemiol. (2001) 54:343–9. doi: 10.1016/S0895-4356(00)00314-0

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Ser B. (1995) 57:289–300. doi: 10.1111/j.2517-6161.1995.tb02031.x

CrossRef Full Text | Google Scholar

45. Koberda JL, Koberda P, Bienkiewicz AA, Moses A, Koberda L. Pain management using 19-electrode Z-score LORETA neurofeedback. J Neurother. (2013) 17:179–90. doi: 10.1080/10874208.2013.813204

CrossRef Full Text | Google Scholar

46. Batty MJ, Bonnington S, Tang B-K, Hawken MB, Gruzelier JH. Relaxation strategies and enhancement of hypnotic susceptibility: EEG neurofeedback, progressive muscle relaxation and self-hypnosis. Brain Res Bull. (2006) 71:83–90. doi: 10.1016/j.brainresbull.2006.08.005

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Raymond J, Varney C, Parkinson LA, Gruzelier JH. The effects of alpha/theta neurofeedback on personality and mood. Cogn Brain Res. (2005) 23:287–92. doi: 10.1016/j.cogbrainres.2004.10.023

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Zotev V, Phillips R, Yuan H, Misaki M, Bodurka J. Self-regulation of human brain activity using simultaneous real-time fMRI and EEG neurofeedback. Neuroimage. (2014) 85:985–95. doi: 10.1016/j.neuroimage.2013.04.126

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Putman P, van Peer J, Maimari I, van der Werff S. EEG theta/beta ratio in relation to fear-modulated response-inhibition, attentional control, affective traits. Biol Psychol. (2010) 83:73–8. doi: 10.1016/j.biopsycho.2009.10.008

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Güntekin B, Tülay E. Event related beta and gamma oscillatory responses during perception of affective pictures. Brain Res. (2014) 1577:45–56. doi: 10.1016/j.brainres.2014.06.029

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Wang J-R, Hsieh S. Neurofeedback training improves attention and working memory performance. Clin Neurophysiol. (2013) 124:2406–20. doi: 10.1016/j.clinph.2013.05.020

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Hillinger MG, Wolever RQ, McKernan LC, Elam R. Integrative medicine for the treatment of persistent pain. Prim Care. (2017) 44:247–64. doi: 10.1016/j.pop.2017.02.008

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Madsen C, Vaughan M, Koehlmoos TP. Use of integrative medicine in the United States military health system. Evid Based Compl Altern Med. (2017) 2017:9529257. doi: 10.1155/2017/9529257

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Tan G, Dao TK, Smith DL, Robinson A, Jensen MP. Incorporating complementary and alternative medicine (CAM) therapies to expand psychological services to veterans suffering from chronic pain. Psychol Serv. (2010) 7:148. doi: 10.1037/a0020304

CrossRef Full Text | Google Scholar

55. Lewine JD, Plis S, Ulloa A, Williams C, Spitz M, Foley J, et al. Quantitative EEG biomarkers for mild traumatic brain injury. J Clin Neurophysiol. (2019) 36:298–305. doi: 10.1097/WNP.0000000000000588

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Slobounov S, Sebastianelli W, Hallett M. Residual brain dysfunction observed one year post-mild traumatic brain injury: combined EEG and balance study. Clin Neurophysiol. (2012) 123:1755–61. doi: 10.1016/j.clinph.2011.12.022

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Thatcher RW, Walker RA, Gerson I, Geisler FH. EEG discriminant analyses of mild head trauma. Electroencephalogr Clin Neurophysiol. (1989) 73:94–106. doi: 10.1016/0013-4694(89)90188-0

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Coben R, Hammond DC, Arns M. 19 channel Z-score and LORETA neurofeedback: does the evidence support the hype? Appl Psychophysiol Biofeedback. (2019) 44:1–8. doi: 10.1007/s10484-018-9420-6

PubMed Abstract | CrossRef Full Text | Google Scholar