Ambulatory Assessment in Concussion Clinical Care and Rehabilitation

Concussion is a mild traumatic brain injury that is characterized by a wide range of physical, emotional, and cognitive symptoms as well as neurocognitive, vestibular, and ocular impairments that can negatively affect daily functioning and quality of life. Clinical consensus statements recommend a targeted, clinical profile-based approach for management and treatment. This approach requires that clinicians utilize information obtained via a clinical interview and a multi-domain assessment battery to identify clinical profile(s) (e.g., vestibular, mood/anxiety, ocular, migraine, cognitive fatigue) and prescribe a corresponding treatment/rehabilitation program. Despite this comprehensive approach, the clinical picture can be limited by the accuracy and specificity of patient reports (which often conflate timing and severity of symptomology), as well as frequency and duration of exposure to symptom exacerbating environments (e.g., busy hallways, sitting in the back seat of a car). Given that modern rehabilitation programs leverage the natural environment as a tool to promote recovery (e.g., expose-recover approach), accurate characterization of the patient clinical profile is essential to improving recovery outcomes. Ambulatory assessment methodology could greatly benefit concussion clinical care by providing a window into the symptoms and impairments experienced by patients over the course of their daily lives. Moreover, by evaluating the timing, onset, and severity of symptoms and impairments in response to changes in a patient's natural environment, ambulatory assessments can provide clinicians with a tool to confirm clinical profiles and gauge effectiveness of the rehabilitation program. In this perspective report, we review the motivations for utilizing ambulatory assessment methodology in concussion clinical care and report on data from a pilot project utilizing smart phone-based, ambulatory assessments to capture patient reports of symptom severity, environmental exposures, and performance-based assessments of cognition for 7 days following their initial evaluation.

Introduction

Concussion, or mild traumatic brain injury (mTBI), is characterized by a heterogenous presentation of signs, symptoms, and impairments that can negatively affect physical, emotional, cognitive, and sleep-related functioning. A clinical care approach that includes a multi-domain assessment to inform an active, targeted treatment plan is key to mitigating poor recovery outcomes. Concussion recovery time is variable, with most patients recovering within 30 days of injury and a smaller percentage of patients exhibiting a longer, protracted recovery (1–4). Several patient- (e.g., concussion history) and injury-related risk factors (e.g., on-field dizziness) are associated with protracted recovery and warrant consideration by the clinician (5, 6). The current standard of care for concussion includes a patient interview (e.g., health history, injury details) and a comprehensive battery of symptom, neurocognitive, vestibular, ocular, and mood assessments (7, 8). Recent clinical and expert consensus (7, 9–11) advocate for a clinical profiles-based care approach that utilizes subjective reports from patients and objective, multi-domain assessment data to identify a concussion clinical profile that can be matched to a targeted treatment and rehabilitation strategy [e.g., a vestibular profile would warrant referral to vestibular therapy; (12, 13)]. The timeline for concussion patient care includes an initial clinical visit (ideally within the first week of injury) where a concussion clinical profile(s) is determined, and a treatment plan is established. Follow-up visits to the clinic occur ~every 1–2 weeks where the clinician will gauge progress and recovery until medical clearance.

The centerpiece of the concussion clinical evaluation involves ascertaining the patient's lived injury experience. The accuracy of this information is critical for the clinician to generate a clear picture of the injury and establish a targeted treatment plan (8, 14). Self-reported symptoms are the primary driver of clinical care, and these reports are associated with determining injury severity, clinical profile and treatment, and risk for protracted recovery (3, 15–18). Beyond the diversity and severity of reported symptoms, the impact of patients' everyday life environments on these symptoms is also very informative to the clinician. For example, dizziness may be a patient's chief complaint; however, the clinician may inquire how different daily environments and activities affect dizziness (e.g., riding in a car, shopping at a store). The value of this information has prompted researchers to modify current symptom scales to be more inclusive of the interaction between the patient's environment and their symptoms (e.g., the Clinical Profiles Screen: CP Screen) (19).

While patient reports are essential to constructing the clinical picture, encapsulation of this information within the clinical interview has its limitations. Patient reported outcomes may lack ecological validity, as they may be influenced by a number of patient attributes [e.g., somatization (20, 21), sport ethic (22), parental influence (23)], and performance-based assessments are obtained under controlled testing conditions (e.g., quiet, clinical testing rooms often with parents or caregivers present). Moreover, patients may struggle to provide accurate, retrospective accounts of the type, frequency, and severity of their concussion symptoms over the course of several clinic visits due to recall bias, lack of education, awareness, and short-lived experience with their concussion. These limitations often require clinicians to estimate the burden of symptoms and impairments in their patients, which may influence treatment selection and recovery outcomes.

Ambulatory Assessment in Acute Concussion Management

Our approach intends to improve assessment and monitoring by conducting short surveys and ultra-brief cognitive assessments (typically <60 s) on smart phones in a measurement burst design (24–27). By delivering these assessments remotely, and over the course of participants' daily lives, we hope to capture a more accurate representation of patients' everyday experiences including the frequency and duration of exposures to various, potentially symptom-modifying, environments (28, 29). By assessing symptom severity in close temporal proximity to experience, we aim to reduce the influence of biases associated with patient reported outcomes [e.g., retrospective biases (29, 30)]. An advantage of conducting ambulatory cognitive assessments in such an intensive, longitudinal study design protocol is the ability to distribute an equivalent or greater amount of overall testing that would be completed in the clinic, over the course of a week between visits. Separating performance-based assessments from the clinical environment allows patients to complete the assessments in more familiar surroundings (e.g., home or school), increasing the ecological validity of the testing context (24, 27). For example, some patients exhibit significant anxiety related to completing neurocognitive testing for their concussion and discussing their symptoms with the clinician. The administration of performance-based assessments in the patient's daily life and environments may not only reduce anxiety, but possibly increase the clinical accuracy of these assessments and self-reported data from patients. In addition, intensive repeated assessment may help reveal the timecourse of recovery by generating a semi-continuous timeseries of symptom and cognitive performance data.

The Mobile Neurocognitive Health Project

The Mobile Neurocognitive Health project is a study of middle and high school athletes with sport-related concussion. Participants were recruited based on the following inclusion criteria: 13+ years of age, medically diagnosed concussion occurring within 7 days of enrollment, no history of a prior concussion within the last 6 months, and willing to use personal smart phone for data collection. Participants were excluded if there was evidence of more severe brain injury (e.g., positive neuroimaging finding, loss of consciousness ≥20 min), they were not fluent in English, or had significant visual impairments that would prevent use of a smart phone. International consensus criteria for concussion were used to standardize injury diagnosis for all participants (7). All participants were enrolled in the study protocol during acute injury presentation (<7 days of injury). Participants in MNCH are asked to download our study application (the Mobile Monitoring of Cognitive Change, M2C2, smart phone application, under development as part of the forthcoming, NIH Mobile Toolbox) onto their own phones and respond to 4 assessment notifications daily, for a period of 7 days following their initial clinic visit. Two feasibility questions addressed by MNCH include: (a) whether students with a sport-related concussion would comply with a measurement burst design protocol, and (b) whether cognitive exertion associated with completing the surveys and ultra-brief cognitive assessments might exacerbate symptoms.

Conducting a measurement burst design study involving student athletes requires tailoring of the assessment protocol to the demands of student life, while also providing space for day to day variation in schedule and availability (e.g., weekday vs. weekend). Accordingly, assessments are scheduled during four 2.5-h time windows throughout each day (approximating the start of the school day, lunch, the end of school day, and evening prior to bedtime; Figure 1A). Data from the first 60 participants suggest that 75% of patients provided evaluable data (completing >50% expected assessments), including a 78.6% median compliance rate to the ambulatory protocol (equivalent to completing ~22 of the 28 assessments administered). Initial results from a single item administered at the end of each assessment (“Completing the assessments made my symptoms worse.” 100-pt visual analog slider: Not at All to Extremely) suggest that the MNCH ambulatory assessments are generally well-tolerated (M = 34 pts out of 100, SD = 24.6). Overall, our initial observation is that ambulatory assessment of exposures, symptoms, and cognition in a measurement burst design may be acceptable in this population, among which compliance and exertion are important considerations.

FIGURE 1

Figure 1. MNCH Study Protocol. (A) Recruitment study infographic depicting the measurement burst design. (B) Each assessment contained short surveys of various environmental exposures, symptom severity ratings, and performance-based cognitive tasks. The Symbol Search task, measuring processing speed and attention, is depicted. (C) Response time profiles during Symbol Search performance over the 7-day measurement burst. Patient A and Patient B exhibit nearly identical response times during the first 3 administrations of the Symbol Search task. Thereafter, Patient A exhibits high intra-individual variation and generally no improvement in task performance over 7 days. Patient B, by comparison, exhibits exponential improvement early in the measurement burst, lower intra-individual variation, and evidence of incremental improvements throughout the measurement burst.

Performance-Based Ambulatory Cognitive Assessments

In-person computerized or paper-and-pencil neuropsychological assessment represents the current standard for performance-based assessment of cognition during concussion management and rehabilitation (7). However, periodic, clinic-based assessments provide a relatively low resolution timecourse of neurocognitive recovery. Distributed ultra-brief cognitive assessments can provide a timecourse of data describing change over multiple timescales (moment-to-moment, day-to-day, over the course of a week). Multiple features of this timecourse may provide clinically-meaningful information about the recovery trajectory.

Figure 1B depicts one trial of the M2C2 Symbol Search task and Figure 1C depicts median response times (RT) for each administration (“session”), over the 7-day measurement burst, for 2 participants. While Patient A and Patient B exhibit nearly identical performance during the first 3 sessions, highly differentiable patterns of performance unfold over the remainder of the measurement burst. Of note, Patient A exhibits greater session-to-session, intra-individual variation than Patient B. Inconsistency in speeded performance in other populations (e.g., older adults) has been identified as a risk factor for cognitive impairment (31–33). Recently, in a study using M2C2, we found that older adults with mild cognitive impairment exhibit significantly greater session-to-session and day-to-day variation in Symbol Search task performance than cognitively-unimpaired older adults (34). A primary aim of the MNCH project is to determine if RT inconsistency is a clinically-relevant marker of cognitive impairment following concussion.

A second feature that differentiates Patient A from Patient B is the apparent lack of improvement in performance over the 7-day burst. Retest effects are common among performance-based assessments and may reflect procedural learning (e.g., familiarity with timing and task procedures and adaptation to response modality) or explicit learning of testing materials (35–38). M2C2 assessments such as the Symbol Search task are designed to discourage explicit learning as stimuli are randomly drawn from large stimulus banks on each trial. Nonetheless, we have observed substantial retest effects during Symbol Search task performance that are well-captured by process models of learning and proceduralization [negative exponential modeling (39, 40)]. An important consideration for monitoring of concussion is that improvements are likely to reflect the joint contribution of retest learning and recovery from the injury. That is, concussion recovery is associated with improvements in processing speed (2, 41–45), and retest effects should trend in the same direction. Application of cognitive process models may help dissociate retest- from recovery-related improvements. Notwithstanding, an overall lack of improvement in RT (as seen with Patient A) is reported to be a predictive marker for protracted recovery [see Future Directions (42, 44)].

Everyday Exposures and Symptom Severity Ratings

Exposure to potentially symptom-modifying environments and activities is cataloged during each assessment in MNCH. We surveil 9 environments and activities that student athletes are commonly exposed to during an average school day and weekend (19). During each assessment participants are asked to report the approximate amount of time they spent in each of these environments and activities. Figure 1B displays an example of an exposure assessment related to school activities. From this data stream we can extrapolate both the frequency of exposure to each environment and activity, as well as the average duration, over the course of the 7-day burst period. Following exposure and activity reporting, participants in MNCH are asked to rate the severity of 15 commonly reported concussion symptoms (46). Symptom severity ratings are made on a 100 pt visual analog scale using a slider element. Figure 1B displays an example symptom severity rating of perseverative thought.

Figure 2A displays average symptom severity ratings for the same two patients depicted in Figure 1. Patient A reports a cluster of symptoms centered around feelings of tiredness, headache, and stress, which are thought to be indicative of a primary post-traumatic migraine profile (3, 47). In contrast, while Patient B reports an overlapping pattern of tiredness, headache, and stress, they also report experiencing symptoms consistent with an anxiety/mood profile (perseverative thoughts, anxiety) as well as vestibular impairments (experience of slow dizziness and nausea). These characteristics of Patient B are consistent with the frequent and overlapping of the anxiety and vestibular clinical profiles of concussion (12). An important aim of MNCH will be to compare ambulatory symptom severity ratings with results of clinical interviews to potentially validate putative injury profiles. Moreover, modeling the degree to which symptoms co-occur, vary over time, and/or operate in a lead-lag fashion over daily life will help highlight symptoms or symptom clusters that might be viable targets for precision concussion rehabilitation (see Future Directions). Finally, similar to data derived from the ambulatory cognitive assessments, symptom timecourses may provide an invaluable monitoring system for tracking recovery as well as the efficacy of rehabilitation programs.

FIGURE 2

Figure 2. Symptom Severity, Exposure and Activity Reports, and Context-dependent Symptom Severity Change. (A) Average symptom ratings for Patient A and Patient B over the burst. (B) Frequency and duration of the 9 everyday environments and activities Patient A reported as exposures. Consistent with the daily life a student athlete, Patient B's exposure pattern was very similar to Patient A and not displayed here. (C) Exposure-specific symptom reactivity to two common everyday exposures for participants enrolled in the MNCH project.

Reactivity to Everyday Exposures

Although the influence of everyday exposures on concussion symptoms and impairments is central to the clinical concussion evaluation, empirical reports on this topic are scant. To date researchers have focused primarily on the exposure to academic (48, 49) and physical activity (50–52) environments. Natural exposure to the academic environment (i.e., attending a day of school with no accommodations for concussion) is linked to increased symptoms and prolonged recovery (48, 53), which has prompted the adoption of a gradual “return to learn” care plan or “pacing” model (54, 55). The usage and frequency of accommodations have been described, but this literature is limited by cross-sectional, cohort studies. Aerobic-based, expose-recover approaches (50, 56) that focus on sub-symptom physical activity is linked to improved concussion outcomes, but these exposures occur during bouts of supervised exercise, not measured in the patient's everyday life. Critically, it remains poorly understood whether the same exposures consistently modify the same symptoms across patients in their natural environment. Precision rehabilitation will require an understanding of which environments and activities affect which patients in which ways. Figure 2C shows how two frequent everyday exposures influence each symptom monitored in MNCH. Standardized differences in symptom severity ratings are plotted for moments where patients report being exposed to physical activity and school activity compared with moments where these exposures are not reported. While physical activity and school activities are associated with increases in overlapping symptoms, the highlighted region shows how feelings of fast dizziness and motion sickness are specific to physical activity, which is consistent with clinical anecdotes suggesting that clusters of symptoms may be closely tied to specific everyday exposures.

Future Directions

Ambulatory assessments represent a new tool for both basic science and clinical management of concussion. As is highlighted by individual data from the MNCH study, ambulatory assessments can be leveraged to provide complementary information to in-clinic evaluation and generate real-time feedback on symptom response to everyday environmental and activity exposures. This information could be used to corroborate the clinical impression (e.g., confirm clinical profile) and to monitor the efficacy of treatment plans. A key contribution may also be the use of ambulatory assessment to optimize, and gauge adherence to, post-concussion management recommendations and rehabilitation prescriptions. For example, behavioral management recommendations are often used as a first-step management approach for concussion and includes proper sleep hygiene, nutrition, hydration, physical activity, and stress management (12). Post-concussion rehabilitation strategies may include vestibular and/or ocular motor therapy referral, which involve both in-clinic and at-home therapy sessions that involve exposure to symptom-provoking environments in a controlled manner [i.e., expose-recover (57)]. Higher density data streams could be leveraged to document adherence and effectiveness of behavioral regulation strategies and determine optimum dosage (i.e., duration) of expose-recover rehabilitation strategies.

Ambulatory assessments may also be leveraged as a digital biomarker of injury severity and recovery. Neurocognitive impairments are one concussion hallmark, and a digital biomarker of concussion injury and recovery will require sensitivity to change over multiple timescales. Our approach is designed to maximize sensitivity to several aspects of cognitive change including changes in level and variation in performance. We anticipate that short-term dynamics (absence of short-term improvement/high intra-individual variation) might be a valuable predictive digital biomarker of protracted recovery. Despite exhibiting overall greater symptom burden and severity than Patient A, Patient B exhibited a pattern of performance on the Symbol Search task consistent with retest learning (exponential improvements) and lower moment-to-moment variation over the measurement burst (Patient A exhibited no overall improvements). Although both patients, similar in age (15 and 17 years) and both female, began the measurement burst within 4 days of their injury; Patient B was medically cleared after 10 days compared to 54 days for Patient A. A primary aim of the MNCH project is to determine whether these short-term neurocognitive outcomes predict protracted recovery and document how these outcomes might be associated with or modified by other established risk factors (e.g., symptom burden). In addition, we aim to establish the utility of performance-based ambulatory cognitive assessments as a digital biomarker of recovery. This effort will require establishing norms and expected range of performance (and variation) in healthy individuals as well as patients during the course of the injury. Against this backdrop, the efficacy of novel rehabilitation programs could be tested and evaluated at multiple timescales (i.e., acute vs. longer-term outcomes). Finally, performance-based, ambulatory cognitive assessments offer an opportunity to confirm the temporality of impairments (e.g., evidence of impairments after episodes of cognitive exertion/school in patients with a cognitive/fatigue profile) and disruption of regulatory systems (e.g., changes in diurnal patterns).

The remote nature of ambulatory assessments creates new avenues both early in the injury and into longer-term recovery. Health care disparities due to geographical or economic circumstances could be reduced by delivering these assessments and rehabilitation prescriptions over pervasive technology (i.e., smart phones), reaching individuals without access to specialty care. In addition, ambulatory assessment protocols that require limited proctoring or onboarding could be a tool for athletic trainers to deploy in regular practice, which may help refine the referral process to primary or specialty care (i.e., support triage based on remote screening). Finally, the ability to conduct longer-term, remote monitoring (i.e., telemedicine) might help better define the bounds of medical clearance/clinical recovery as ambulatory assessments offer a low-burden, remote approach to repeated assessment during follow-up.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by WCG IRB. Written informed consent to participate in this study was provided by the participants' legal guardian/next of kin.

Author Contributions

JH, RE, and MW contributed to the conception of the study design. CD coordinated the study and conducted all data collection. DE conducted all data processing and analyses. JH and RE wrote the first draft of the manuscript. MW and SF wrote sections of the manuscript. All authors contributed to manuscript revision, editing, and approved the submitted version.

Funding

This study was supported by the National Institute on Aging of the National Institutes of Health under award numbers R00AG056670 and U2CAG060408. JH's effort on the project was supported by R00AG056670. Funding for the study costs was supported by the Penn State College of Medicine through research startup funding provided to JH. Support for development and instrumentation of the ambulatory assessment platform was provided by U2CAG060408. The content is solely the responsibility of the authors and does not necessarily represent the official views of these granting agencies.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Collins MW, Lovell MR, Iverson GL, Ide T, Maroon J. Examining concussion rates and return to play in high school football players wearing newer helmet technology: a three-year prospective cohort study. Neurosurgery. (2006) 58:275–86. doi: 10.1227/01.NEU.0000200441.92742.46

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Henry LC, Elbin RJ, Collins MW, Marchetti G, Kontos AP. Examining recovery trajectories after sport-related concussion with a multimodal clinical assessment approach. Neurosurgery. (2016) 78:232–41. doi: 10.1227/NEU.0000000000001041

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Kontos AP, Elbin RJ, Sufrinko A, Marchetti G, Holland CL, Collins MW. Recovery following sport-related concussion: integrating pre- and postinjury factors into multidisciplinary care. J Head Trauma Rehabil. (2019) 34:394–401. doi: 10.1097/HTR.0000000000000536

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Rohling ML, Larrabee GJ, Millis SR. The “Miserable Minority” following mild traumatic brain injury: who are they and do meta-analyses hide them? Clin Neuropsychol. (2012) 26:197–213. doi: 10.1080/13854046.2011.647085

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Elbin RJ, Covassin T, Gallion C, Kontos AP. Factors inflencing risk and recovery from sport-related concussion: reviewing the evidence. Persp Neurophysiol Neuro Speech Lang Dis. (2015) 25:4–16. doi: 10.1044/nnsld25.1.4

CrossRef Full Text | Google Scholar

6. Lau BC, Collins MW, Lovell MR. Sensitivity and specificity of subacute computerized neurocognitive testing and symptom evaluation in predicting outcomes after sports-related concussion. Am J Sports Med. (2011) 39:1209–16. doi: 10.1177/0363546510392016

PubMed Abstract | CrossRef Full Text | Google Scholar

7. McCrory P, Meeuwisse WH, Dvorak J, Echemendia RJ, Engebretsen L, Feddermann-Demont N, et al. 5th international conference on concussion in sport (Berlin). Brit J Sports Med. (2017) 51:837. doi: 10.1136/bjsports-2017-097878

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Paquin H, Taylor A, Meehan WP 3rd. Office-based concussion evaluation, diagnosis, management: pediatric. Handbook Clin Neurol. (2018) 158:107–17. doi: 10.1016/B978-0-444-63954-7.00011-2

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Collins MW, Kontos AP, Okonkwo DO, Almquist J, Bailes J, Barisa M, et al. Statements of agreement from the targeted evaluation and active management (TEAM) approaches to treating concussion meeting held in pittsburgh, October 15-16, 2015. Neurosurgery. (2016) 79:912–29. doi: 10.1227/NEU.0000000000001447

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Collins MW, Kontos AP, Reynolds E, Murawski CD, Fu FH. A comprehensive, targeted approach to the clinical care of athletes following sport-related concussion. Knee Surg Sports Traumatol Arthr. (2014) 22:235–46. doi: 10.1007/s00167-013-2791-6

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Lumba-Brown A, Teramoto M, Bloom OJ, Brody D, Chesnutt J, Clugston JR, et al. Concussion guidelines step 2: evidence for subtype classification. Neurosurgery. (2020) 86:2–13. doi: 10.1093/neuros/nyz332

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Kontos AP, Sufrinko A, Sandel N, Emami K, Collins MW. Sport-related concussion clinical profiles: clinical characteristics, targeted treatments, preliminary evidence. Curr Sports Med Rep. (2019) 18:82–92. doi: 10.1249/JSR.0000000000000573

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Sandel N, Reynolds E, Cohen PE, Gillie BL, Kontos AP. Anxiety and mood clinical profile following sport-related concussion: from risk factors to treatment. Sport Exerc Perform Psychol. (2017) 6:304–23. doi: 10.1037/spy0000098

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Charek DB, Collins M, Kontos A. Office-based concussion evaluation, diagnosis, management: adult. Handbook Clin Neurol. (2018) 158:91–105. doi: 10.1016/B978-0-444-63954-7.00010-0

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Casson IR, Sethi NK, Meehan WP 3rd. Early symptom burden predicts recovery after sport-related concussion. Neurology. (2015) 85:110–1. doi: 10.1212/WNL.0000000000001700

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Cohen PE, Sufrinko A, Elbin RJ, Collins MW, Sinnott AM, Kontos AP. Do initial symptom factor scores predict subsequent impairment following concussion? Clin J Sport Med. (2018) 30, S61–S68. doi: 10.1097/JSM.0000000000000581

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Howell DR, O'Brien MJ, Beasley MA, Mannix RC, Meehan WP 3rd. Initial somatic symptoms are associated with prolonged symptom duration following concussion in adolescents. Acta Paediatrica. (2016) 105:e426–32. doi: 10.1111/apa.13486

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Meehan WP 3rd, Mannix RC, Stracciolini A, Elbin RJ, Collins MW. Symptom severity predicts prolonged recovery after sport-related concussion, but age and amnesia do not. J Pediatr. (2013) 163:721–5. doi: 10.1016/j.jpeds.2013.03.012

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Kontos AP, Elbin RJ, Trbovich A, Womble M, Said A, Sumrok VF, et al. Concussion clinical profiles screening (CP Screen) tool: preliminary evidence to inform a multidisciplinary approach. Neurosurgery. (2020) 87, 348–56. doi: 10.1093/neuros/nyz545

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Nelson LD, Tarima S, LaRoche AA, Hammeke TA, Barr WB, Guskiewicz K, et al. Preinjury somatization symptoms contribute to clinical recovery after sport-related concussion. Neurology. (2016) 86:1856–63. doi: 10.1212/WNL.0000000000002679

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Root JM, Zuckerbraun NS, Wang L, Winger DG, Brent D, Kontos A, et al. History of somatization is associated with prolonged recovery from concussion. J Pediatr. (2016) 174:39–44 e31. doi: 10.1016/j.jpeds.2016.03.020

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Kroshus E, Garnett B, Hawrilenko M, Baugh CM, Calzo JP. Concussion under-reporting and pressure from coaches, teammates, fans, and parents. Soc Sci Med. (2015) 134:66–75. doi: 10.1016/j.socscimed.2015.04.011

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Gagnon I, Teel E, Gioia G, Aglipay M, Barrowman N, Sady M, et al. Parent-child agreement on postconcussion symptoms in the acute postinjury period. Pediatrics. (2020) 146. doi: 10.1542/peds.2019-2317

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Hakun J, Roque N, Gerver CR, Cerino ES. Ultra-Brief Ambulatory Assessment of Working Memory Capacity Via Smart Phones (2021). doi: 10.31234/osf.io/ewzg7

CrossRef Full Text | Google Scholar

25. Nesselroade JR. The Warp and the Woof of the Developmental Fabric. Mahwah: Lawrence Erlbaum Associates, Inc (1991).

Google Scholar

26. Sliwinski MJ. Measurement-burst designs for social health research. Soc Person Psychol Compass. (2008) 2:245–61. doi: 10.1111/j.1751-9004.2007.00043.x

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Sliwinski MJ, Mogle JA, Hyun J, Munoz E, Smyth JM, Lipton RB. Reliability and validity of ambulatory cognitive assessments. Assessment. (2018) 25:14–30. doi: 10.1177/1073191116643164

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. Ann Rev Clin Psychol. (2008) 4:1–32. doi: 10.1146/annurev.clinpsy.3.022806.091415

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Smyth JM, Stone AA. Ecological momentary assessment research in behavioral medicine. Ann Rev Clin Psychol. (2003) 4:1–32. doi: 10.1023/A:1023657221954

CrossRef Full Text | Google Scholar

30. Shiffman S. Ecological momentary assessment (EMA) in studies of substance use. Psychol Assess. (2009) 21:486. doi: 10.1037/a0017074

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Bielak AA, Hultsch DF, Strauss E, Macdonald SW, Hunter MA. Intraindividual variability in reaction time predicts cognitive outcomes 5 years later. Neuropsychology. (2010) 24:731–41. doi: 10.1037/a0019802

PubMed Abstract | CrossRef Full Text | Google Scholar

32. MacDonald SWS, Stawski RS. Longitudinal changes in response time mean and inconsistency exhibit predictive dissociations for risk of cognitive impairment. Neuropsychology. (2020) 34:264–75. doi: 10.1037/neu0000608

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Stawski RS, MacDonald SWS, Brewster PWH, Munoz E, Cerino ES, Halliday DWR. A comprehensive comparison of quantifications of intraindividual variability in response times: a measurement burst approach. J Gerontol Series B Psychol Sci Soc Sci. (2019) 74:397–408. doi: 10.1093/geronb/gbx115

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Cerino ES, Katz MJ, Wang C, Qin J, Gao Q, Hyun J, et al. Variability in cognitive performance on mobile devices is sensitive to mild cognitive impairment: results from the einstein aging study. Frontier Digital Health (2021) 3:758031. doi: 10.3389/fdgth.2021.758031

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Ferrer E, Salthouse TA, Stewart WF, Schwartz BS. Modeling age and retest processes in longitudinal studies of cognitive abilities. Psychol Aging. (2004) 19:243–59. doi: 10.1037/0882-7974.19.2.243

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Scharfen J, Jansen K, Holling H. Retest effects in working memory capacity tests: a meta-analysis. Psychon Bull Rev. (2018) 25:2175–99. doi: 10.3758/s13423-018-1461-6

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Sliwinski MJ, Hoffman L, Hofer S. Modeling retest and aging effects in a measurement burst design. In: Molenaar PCM, Newell KM, editorsă Individual Pathways of Change: Statistical Models for Analyzing Learning and Development, (Washington, DC: American Psychological Association) 37–50. doi: 10.1037/12140-003

CrossRef Full Text | Google Scholar

38. Woods SP, Delis DC, Scott JC, Kramer JH, Holdnack JA. The california verbal learning test–second edition: test-retest reliability, practice effects, and reliable change indices for the standard and alternate forms. Arch Clin Neuropsychol. (2006) 21:413–20. doi: 10.1016/j.acn.2006.06.002

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Munoz E, Sliwinski MJ, Scott SB, Hofer S. Global perceived stress predicts cognitive change among older adults. Psychol Aging. (2015) 30:487. doi: 10.1037/pag0000036

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Oravecz Z, Roque N, Sliwinski MJ. Bayesian modeling of cognitive impairment in the presence of retest effects. Innov Aging. (2020) 4:566. doi: 10.1093/geroni/igaa057.1870

CrossRef Full Text | Google Scholar

41. Iverson GL, Lovell MR, Collins MW. Validity of ImPACT for measuring processing speed following sports-related concussion. J Clin Exp Neuropsychol. (2005) 27:683–9. doi: 10.1081/13803390490918435

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Lau B, Lovell MR, Collins MW, Pardini J. Neurocognitive and symptom predictors of recovery in high school athletes. Clin J Sport Med. (2009) 19:216–21. doi: 10.1097/JSM.0b013e31819d6edb

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Lau BC, Kontos AP, Collins MW, Mucha A, Lovell MR. Which on-field signs/symptoms predict protracted recovery from sport-related concussion among high school football players? Am J Sports Med. (2011) 39:2311–8. doi: 10.1177/0363546511410655

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Lau BC, Collins MW, Lovell MR. Cutoff scores in neurocognitive testing and symptom clusters that predict protracted recovery from concussions in high school athletes. Neurosurgery. (2012) 70:371–9. doi: 10.1227/NEU.0b013e31823150f0

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Sandel NK, Lovell MR, Kegel NE, Collins MW, Kontos AP. The relationship of symptoms and neurocognitive performance to perceived recovery from sports-related concussion among adolescent athletes. Appl Neuropsychol Child. (2013) 2:64–9. doi: 10.1080/21622965.2012.670580

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Lovell MR, Iverson GL, Collins MW, Podell K, Johnston KM, Pardini D, et al. Measurement of symptoms following sports-related concussion: reliability and normative data for the post-concussion scale. Appl Neuropsychol. (2006) 13:166–74. doi: 10.1207/s15324826an1303_4

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Kontos AP, Elbin RJ, Lau B, Simensky S, Freund B, French J, et al. Posttraumatic migraine as a predictor of recovery and cognitive impairment after sport-related concussion. Am J Sports Med. (2013) 41:1497–504. doi: 10.1177/0363546513488751

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Sady MD, Vaughan CG, Gioia GA. School and the concussed youth: recommendations for concussion education and management. Phys Med Rehabil Clin North Am. (2011) 22:701–19, ix. doi: 10.1016/j.pmr.2011.08.008

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Swanson MW, Weise KK, Dreer LE, Johnston J, Davis RD, Ferguson D, et al. Academic difficulty and vision symptoms in children with concussion. Optometry Vision Sci. (2017) 94:60–7. doi: 10.1097/OPX.0000000000000977

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Leddy JJ, Willer B. Use of graded exercise testing in concussion and return-to-activity management. Curr Sports Med Rep. (2013) 12:370–6. doi: 10.1249/JSR.0000000000000008

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Silverberg ND, Iverson GL, McCrea M, Apps JN, Hammeke TA, Thomas DG. Activity-related symptom exacerbations after pediatric concussion. JAMA Pediatr. (2016) 170:946–53. doi: 10.1001/jamapediatrics.2016.1187

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Thomas DG, Apps JN, Hoffmann RG, McCrea M, Hammeke T. Benefits of strict rest after acute concussion: a randomized controlled trial. Pediatrics. (2015) 135:213–23. doi: 10.1542/peds.2014-0966

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Ransom DM, Vaughan CG, Pratson L, Sady MD, McGill CA, Gioia GA. Academic effects of concussion in children and adolescents. Pediatrics. (2015) 135:1043–50. doi: 10.1542/peds.2014-3434

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Holmes A, Chen Z, Yahng L, Fletcher D, Kawata K. Return to learn: academic effects of concussion in high school and college student-athletes. Front Pediatr. (2020) 8:57. doi: 10.3389/fped.2020.00057

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Kirelik SB, McAvoy K. Acute concussion management with remove-reduce/educate/adjust-accommodate/pace (REAP). J Emerg Med. (2016) 50:320–24. doi: 10.1016/j.jemermed.2015.02.054

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Leddy JJ, Kozlowski K, Donnelly JP, Pendergast DR, Epstein LH, Willer B. A preliminary study of subsymptom threshold exercise training for refractory post-concussion syndrome. Clin J Sport Med. (2010) 20:21–7. doi: 10.1097/JSM.0b013e3181c6c22c

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Kontos AP, Deitrick JM, Collins MW, Mucha A. Review of vestibular and oculomotor screening and concussion rehabilitation. J Athl Train. (2017) 52:256–61. doi: 10.4085/1062-6050-51.11.05

PubMed Abstract | CrossRef Full Text | Google Scholar