The Stability of Retrospective Pre-injury Symptom Ratings Following Pediatric Concussion

Objective: To determine the stability of children's retrospective ratings of pre-injury levels of symptoms over time following concussion.

Methods: Children and adolescents (n = 3,063) between the ages of 5–17 diagnosed with a concussion by their treating pediatric emergency department (PED) physician within 48 h of injury completed the Post-Concussion Symptom Inventory (PCSI) at the PED and at 1, 2, 4, 8, and 12-weeks post-injury. At each time point, participants retrospectively recalled their pre-injury levels of post-injury symptoms. The PCSI has three age-appropriate versions for children aged 5–7 (PCSI-SR5), 8–12 (PCSI-SR8), and 13–18 (PCSI-SR13). Total scale, subscales (physical, cognitive, emotional, and sleep), and individual items from the PCSI were analyzed for stability using Gini's mean difference (GMD).

Results: The mean GMD for total score was 0.31 (95% CI = 0.28, 0.34) for the PCSI-SR5, 0.19 (95% CI = 0.18, 0.20) for the PCSI-SR8, and 0.17 (95% CI = 0.16, 0.18) for the PCSI-SR13. Subscales ranged from mean GMD 0.18 (physical) to 0.31 (emotional) for the PCSI-SR8 and 0.16 (physical) to 0.31 (fatigue) for the PCSI-SR13. At the item-level, mean GMD ranged from 0.13 to 0.60 on the PCSI-SR5, 0.08 to 0.59 on the PCSI-SR8, and 0.11 to 0.41 on the PCSI-SR13.

Conclusions: Children and adolescents recall their retrospective pre-injury symptom ratings with good-to-perfect stability over the first 3-months following their concussion. Although some individual items underperformed, variability was reduced as items were combined at the subscale and full-scale level. There is limited benefit gained from collecting multiple pre-injury symptom queries.

Clinical Trial Registration: Clinicaltrials.gov through the US National Institute of Health/National Library of Medicine. (NCT01873287; http://clinicaltrials.gov/ct2/show/NCT01873287).

Introduction

Concussions are a prevalent injury among children and adolescents throughout North America. In the US, concussions account for one in 220 PED visits, accumulating over 700,000 PED visits annually (1–3). Rates of reporting to the emergency department are even higher in Canada, where concussions are responsible for one in every 70 PED visits (4). Currently, an objective biomarker for concussion diagnosis and subsequent recovery is elusive. Instead, health care providers must rely upon a clinical examination in combination with symptom, balance, and cognitive assessments for concussion diagnosis and management (5, 6). This can be particularly difficult when evaluating children and adolescents, especially elementary-aged children, as current clinical assessments are developed for use with adults and are often not validated or developmentally appropriate for young patients (7).

Concussions can result in a number of somatic, emotional, cognitive, or sleep-related symptoms (8), with headache, imbalance/dizziness, and fatigue the most commonly reported post-concussion symptoms (8, 9). Most clinicians do not have access to a concussion symptom checklist completed by the child pre-injury to serve as a baseline level of normal functioning. One alternative approach is to rely on the injured child to retrospectively report pre-injury levels of their current post-injury symptoms. As concussion management requires serial assessment, and potentially oversight from multiple health care providers (10), it may require children to provide retrospective pre-injury ratings multiple times throughout recovery.

The Post-Concussion Symptom Inventory (PCSI) is a commonly used pediatric symptom checklist that is sensitive to concussion (8). The PCSI has three, developmentally appropriate versions for children aged 5–7, 8–12, and 13–18 years old. Several psychometric properties of the PCSI have been previously reported, including its factor structure, internal consistency, rater concordance, and convergent validity (8, 11, 12). The test-retest reliability of the PCSI has been assessed in healthy (i.e., non-injured) children and interclass correlation coefficients ranged from moderate-to-high (0.65 < ICC < 0.89) (8). However, the presence of concussion may alter the stability of this assessment, particularly when retrospectively recalling pre-injury ratings. Adults with concussion are suspected to engage in a “good-old-days” bias, whereby they idealize retrospective pre-injury levels compared to uninjured controls (13, 14). Recall bias in self-reported symptom outcomes is also highlighted in the scientific literature, with patient age and time since event named as important factors (15, 16). Therefore, it is critical to formally evaluate the stability of retrospective pre-injury ratings throughout pediatric concussion recovery, since pre-injury status can have important implications on clinical decision making following injury.

The primary purpose of this study is to determine the stability of retrospectively provided pre-injury ratings from children and adolescents over the first 3-months following a diagnosed concussion. Due to the relatively short follow-up period (12-weeks), we anticipate the effect of a “good-old-days” or recall bias will be relatively minimal and hypothesize that retrospective pre-injury ratings will be highly stable overall. However, we further hypothesize that individuals who report higher retrospective pre-injury symptom ratings will be less stable than individual who report few or no pre-injury symptoms.

Materials and Methods

Participants

Data collection methods have been previously described in detail in other studies (17, 18). Briefly, children presenting to nine PEDs across Canada were eligible if they were between 5 and 17 years old, were diagnosed with a concussion using the definition reported by the 2012 Zurich consensus statement (19), presented to the PED within 48 h of injury, and were proficient in either English or French. Children were excluded if they had: (1) A Glasgow Coma Scale score ≤ 13; (2) Any abnormal findings on standard neuroimaging (if neuroimaging was clinically indicated); (3) Neurosurgical operative intervention, intubation, or intensive care stay, (4) Multisystem injuries with treatment requiring hospital stay; (5) Severe developmental delay resulting in communication difficulties; (6) Intoxication at time of PED presentation; (7) No clear primary mechanism of trauma (e.g., sports-related injury, motor vehicle accident, fall) for the current head injury; or (8) Previously enrolled in the same study. Research assistants were present at each PED from 12:00 to 22:00 h to screen for potential participants, assess eligibility, and explain study procedures to eligible patients. This study was carried out in accordance with the recommendations of the Research Ethics Board of each participating institution with written informed consent from all subjects. Written consent was obtained from all eligible and willing parents as well as children and adolescents capable of consenting on their own behalf in accordance with the Declaration of Helsinki. The protocol was approved by the Research Ethics Board of each participating institution.

Methods and Outcomes

Initial PED Visit

Once enrolled, a research assistant collected data from both the parent and child with an electronic survey in their primary language (English or French) on a portable computer tablet to gather demographic information. Along with demographic data, the parent and the child were then asked to complete the age-appropriate version PCSI prior to leaving the PED. Symptoms on the PCSI were rated once scoring the child's current level of post-injury symptoms and once retrospectively recalling the child's pre-injury levels of those same symptoms. In addition to data collection, each treating physician provided a routine clinical assessment of the child, provided discharge instructions, and gave follow-up information as per the normal standard-of-care procedures of that PED site. Prior to leaving the PED, contact information was obtained for all families.

Study Follow-Up

Parents chose to complete follow up measures either online using a web-based platform (REDCap) or over the phone (Email: n = 2,776, Phone: n = 276, Missing: n = 11). Families choosing to complete the follow-up sessions over the phone were called by a research assistant, who read out all survey questions and recorded the participant's answers. For families choosing the web-based option, a link to a secure web-based survey was sent to the parent via email. If the survey was not completed within 48 h of the initial reminder email, phone calls were initiated. A maximum of five phone calls per patient was attempted. Once the patient was contacted, ratings of current symptoms and retrospective report of pre-injury levels were collected at all time points (1, 2, 4, 8, and 12-weeks post-enrollment).

Post-concussion Symptom Inventory (PCSI)

The PCSI is a developmentally-appropriate concussion symptom scale that was made for children and adolescents. The PCSI has three age-based versions (PCSI-SR5: Ages 5–7, 13-items, 3-point Guttman Scale; PCSI-SR8: Ages 8–12, 25-items, 3-point Guttman Scale; PCSI-SR13: Ages 13–18, 26-items, 7-point Guttman Scale). As the PCSI uses different scaling based on the age-appropriate version, the PCSI-SR5, and the PCSI-SR8 were transformed by multiplying each item by three to make the scale range equivalent to the PCSI-SR13 which used a 7-pt Guttman scale most commonly found in the literature. The PCSI was chosen because it is one of only two symptom scales with age-appropriate version for younger children, with appropriate psychometric data published in the literature (8, 11, 12). Additionally the PCSI-SR8 and PCSI-SR13 can be evaluated as physical, cognitive, emotional, and sleep subscales (8). The list of symptoms and which subscales they contribute to can be found in Table 1.

TABLE 1

Table 1. List of symptoms provided on each version of the PCSI.

Statistical Analysis

All analyses were completed in R version 3.3.2 (Vienna, Austria) (20). Stability was evaluated by quantifying the mean absolute discrepancy of the retrospective pre-injury ratings for each patient, a quantity also known as the Gini's Mean Difference (GMD), and post-injury symptom ratings were not analyzed in this study. GMD is a descriptive outcome calculated by taking the average of the absolute value for all pair-wise comparisons between pre-injury ratings at all study time points (i.e., pre-injury rating at the PED and 1, 2, 4, 8, and 12-weeks follow-ups). GMD was chosen because no reference time-point must be declared (i.e., no single time-point was assumed to be more accurate than others) and it is free from distributional assumptions (i.e., remain valid under non-normal distributions). With six total time points throughout this study, a maximum of 15 pairwise comparisons were possible for each individual item. To be included in the analysis, individuals were required to complete the retrospective pre-injury ratings on the PCSI at least twice. Individuals who only completed the PCSI at one time point were excluded as GMD could not be calculated. Additional information about the amount of missing data present throughout this study can be found in the Supplementary Material.

A GMD value was first calculated using the rating provided for each individual item listed on the PCSI. At the subscale and full-scale level, the GMD value was calculated by taking the average of all constituent items involved (i.e., if the subscale had three items, the GMD values calculated for those three individual-items were averaged to create a GMD value for the entire subscale). This approach allowed all GMD values reported in this study to range from 0 (perfect stability) to 6 (worst possible stability), providing consistency between all levels of analysis (full scale, subscale, and individual-items) and more appropriate comparisons for situations where all individual items were not completed at all time points. To calculate 95% confidence intervals for the GMD values, estimates were provided by percentile bootstrap based on 100 repetitions. As the number of items differs on each version of the PCSI, we chose to report GMD values separately for the PCSI-SR5, PCSI-SR8, and the PCSI-SR13. There is no literature available to define what level of stability is needed in clinical use, but some level of variability is to be expected. We defined an acceptable level of stability to be good-to-perfect to be a GMD <1, which is 1/6th (16.7%) of the possible value given the PCSI range.

Previous concussion studies have shown that ~40% of individuals report no pre-injury level of symptoms (21). Individuals who consistently report no pre-injury levels of problems across all study time-points would have perfect stability (GMD = 0); a large proportion of individuals with perfect stability may mask clinically important findings. Therefore, we completed a subgroup analysis by removing all individuals who reported no pre-injury symptoms at all six time points studied and re-analyzed stability only with individuals reporting some retrospective pre-injury symptom presence throughout the study (+ Pre-Injury Ratings). Stability results for the entire sample and for the + Pre-Injury Ratings subgroup are presented throughout the study.

Results

A total of 3,063 children aged 5–17 were recruited through the Predicting and Preventing Postconcussive Problems in Pediatrics (5P) study. Of those 3,063 children, 2,866 (93.6%) completed the retrospective pre-injury PCSI rating during at least two time points and were included into the analysis (Ages 5–7: N = 499, Ages 8–12: N = 1,203, Ages 13–17: N = 1,164). In our sample, 207 (41.5%) participants aged 5–7, 343 (28.5%) participants aged 8–12, and 252 (21.6%) participants aged 13–17 retrospectively rated all pre-injury levels as zero throughout the entire study period (GMD = 0) and were not included in the subgroup analysis. Descriptive statistics for the study sample can be found in Table 2.

TABLE 2

Table 2. Descriptive statistics for the 5–7, 8–12, and 13–18-year-old age groups.

PCSI-SR5

Full Sample

The full scale for the PCSI-SR5 was highly stable, with 94% of children displaying good-to-perfect stability. The mean GMD for the full 5–7-year-old sample ranged from 0.13 to 0.60 at the item-level. Dizziness was the most stable individual item and difficulty concentrating showed the lowest stability.

+ Pre-injury Ratings

When looking only at individuals who retrospectively reported at least one non-zero pre-injury rating throughout the study period, the overall stability was reduced but trends were similar to the full sample. The full scale remained highly stable for the + Pre-Injury sample, with 89% of children continuing to show good-to-perfect. Balance problems and difficulty concentrating remained the most and least stable individual items, respectively. Overall results for the PCSI-SR5 can be found in Table 3.

TABLE 3

Table 3. Mean (95% CI), median (95% CI), and percentage of individuals with good-to-perfect stability for the PCSI-SR5 (Ages 5–7).

PCSI-SR8

Full Sample

Overall, the full scale for the PCSI-SR8 was highly stable (GMD = 0.19, 95% CI: 0.18, 0.20). The physical subscale showed the highest relative stability, with moving slowly and vision problems encompassing the most stable individual items. The emotional subscale showed the lowest stability among the four subscales. Headache had the lowest relative stability at the item-level, followed by difficulty concentrating.

+ Pre-injury Ratings

The PCSI-SR8 remained highly stable for the + Pre-Injury sample, but different trends emerged in this group. The physical domain continued to be the most stable subscale, but nausea and headache had the highest relative stability of individual-items. In disagreement with the full sample, fatigue (subscale), and vision problems (item-level) were the least stable relative to other outcomes. Full results for the PCSI-SR8 can be found in Table 4.

TABLE 4

Table 4. Mean (95% CI), median (95% CI), and percentage of individuals with good-to-perfect stability for the PCSI-SR8 (Ages 8–12).

PCSI-SR13

Full Sample

The full scale for the PCSI-SR13 had high overall stability. Vision problems, answer slowly, and nausea were the most stable individual items and the physical domain was the most stable subscale. Conversely, fatigue, headache, and the fatigue subscale had the lowest relative stability.

+ Pre-injury Ratings

The overall stability for the full scale remained high for the + Pre-Injury sample. The physical subscale and dizziness (item) had the highest relative stability. On the low end, fatigue (subscale) and headache (item) had the lowest stability. Overall results for the PCSI-SR13 can be found in Table 5.

TABLE 5

Table 5. Mean (95% CI), median (95% CI), and percentage of individuals with good-to-perfect stability for the PCSI-SR13 (Ages 13–18).

Discussion

The results of this study confirm our hypothesis that the retrospective recall of pre-injury ratings is stable in children and adolescents. At the full-scale level, all mean GMD values were <5% of the scale range, suggesting little dispersion, and high stability in the retrospective recall of pre-injury levels in the 3-months post-concussion. Averaging over all items, including those with very low mean GMDs, and non-consistent retrospective reporting in opposing direction (i.e., retrospectively rating one symptom 1-point higher than previous reports while rating a different symptom 1-point lower than previous reports) likely contribute to the high stability seen across both the full scale and sub-scale level. The wider 95% confidence intervals and smaller percentage of individuals displaying good-to-perfect stability indicate that the youngest children (ages 5–7) may have slightly less stability in their retrospective symptom recall, in agreement with previous literature suggesting young children have the least reliable recall (15). Our results support the high overall stability of retrospective pre-injury ratings despite large, non-linear changes in post-injury symptoms ratings, which have been shown previously in this sample of concussed children and adolescents (22).

Every individual item on each version of the PCSI had a mean GMD ≤ 1/10th of the scale range and a median GMD of 0, indicating little dispersion. Headache, fatigue, and difficulty concentrating were among the least stable individual items across all versions of the PCSI. These particular symptoms are not specific to concussion and are among the most commonly reported baseline symptoms in healthy children and adolescents for their respective age groups (8, 23). Additionally, the severity at which these problems occur if present often fluctuates over time, which can account for the higher variability (less stability) seen in these items. This rationale is supported by previous research in children with headache history, which shows that children can accurately recall headache frequency but struggle with recalling headache intensity and duration (16). Conversely, the items with the highest individual stability (vision, balance/dizziness, and move/answer slowly) are among the least frequently endorsed baseline symptoms in our study, which agrees with previous literature (8).

The PCSI was highly stable as hypothesized. Previous studies show that the “good-old-days” and recall biases can elevate pre-injury symptom reports, which would theoretically decrease the stability of the assessment. However, reports of these biases are traditionally found over longer periods of time than evaluated in this study and in individuals with on-going symptoms (13, 15). The main objective of this study was to describe the overall stability of retrospective symptoms rating on the PCSI. Therefore, stability was examined only as a collective whole over the follow-up period (i.e., not specific to time intervals within the follow-up period) and with pre-injury symptom ratings [post-injury symptom scores and their progression over the follow-up period in this sample have been previously reported (22)]. As such, our methodology does not allow for a conclusive determine regarding the presence of the “good-old-days” in our sample, which should be evaluated further in future studies.

The primary strength of this study lies in the large, diverse sample. Our pediatric participants comprised a wide age range (5–17), included various mechanisms of injury, and embraced patients with behavioral, learning, and psychological problems [more specific information about our sample can be found in Zemek et al. (18)]. Our broad inclusion criteria increase the generalizability of findings. The use of a validated concussion symptom checklist, which has three, developmentally appropriate versions for children and adolescents, is an additional strength. Lastly, the analytical method of GMD is considered a strength due its ability to tolerate non-normally distributed data and handle missing data (only two retrospectively PCSI rating were needed for inclusion). Even though missing data may mean different time intervals between assessments, there is no natural bias toward stability when this is the case which allows the inclusion of more patients and increases generalizability of our results.

Although the psychometric properties of the PCSI are a strength, the number of items and rating scales for the three different versions are not equivalent and represent a limitation. All reported pre-injury rating responses (0–2 scale) on the PCSI-SR5 and PCSI-SR8 were multiplied by three to make all response outcomes range of GMD 0–6. However, this induces more instability into the PCSI-SR5 and PCSI-SR8 and assumes a 1-point discrepancy on the 0–2 scale is equal to a 3-point discrepancy on a 0–6 scale. The multiple rating scales utilized in the PCSI complicates comparisons across age groups; however, rescaling prevents an inaccurate misinterpretation that younger children have higher stability because the scale range is smaller. Original GMDs values for the PCSI-SR5 and PCSI-SR8 can be found by dividing values reported in this manuscript by three. IN addition, GMD values reported are group averages. Therefore, some individuals may have had large changes (low stability) in their retrospective pre-injury recall. Clinicians should be aware of this when working on the individual patient-level.

Symptom reporting is limited by both floor and ceiling effects, floor effects being a larger concern in our study evaluating pre-injury ratings. The GMD looks at the absolute value of differences in pre-injury reporting over multiple time points. Patients who consistently report no symptoms would have a GMD of zero, thus increasing the stability of the group analysis. However, even patients at the floor (total score of zero) can have a large GMD if they report higher pre-injury ratings at subsequent assessments. Recovery status is not considered in this study. Some participants will have reached full concussion recovery during the follow-up period while other were still experiencing post-concussion symptoms and deficits. How recovery status may have affected retrospective symptom reporting is unknown. Nearly all patients preferred email follow-ups, but a minority of patients (n = 276) requested phone follow-ups. Similarly, very young children may need their parent to help read or understand the symptom checklist. It is possible that interacting with a research assistant via phone follow-ups or parent may influence the patient's response.

While some items underperformed relative to other and some minor differences in stability between the full sample and the + Pre-Injury Ratings reporters were apparent at the item-level, the stability for both groups was high at both the subscale and full-scale level. Based on the results of this study, there appears to be no additional clinical benefit to having patients retrospectively recall their pre-injury symptom levels multiple times throughout the recovery period and this can reduce burden on the healthcare provider and the injured child. If multiple retrospective pre-injury ratings are needed (e.g., seeing multiple providers), clinicians can be assured that children and adolescent will retrospectively rate pre-injury levels with relatively high stability. As long as clinical decisions are based on the subscale or the full-scale level, multiple pre-injury ratings will not have meaningful differences and provide equivalent results.

Children and adolescents retrospectively rate pre-injury levels of post-concussion symptoms with good-to-perfect stability over the first 3-months following their injury. Although some individual items underperformed, variability was reduced as items were combined at the subscale and full-scale level. There is limited benefit to multiple pre-injury symptom queries.

Data Availability

The dataset for this study can be requested through Ontario Brain Institute (OBI) BRAIN-CODE after registering using the following link: https://www.braincode.ca/user/register

Ethics Statement

This study was carried out in accordance with the recommendations of the Research Ethics Board of each participating institution with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Research Ethics Board of each participating institution.

Author Contributions

ET, RZ, GG, and IG were involved in the conception of this project. KT made substantial contributions to data analysis. ET, KT, MS, and CV were involved in the interpretation of the findings. ET drafted this manuscript and all authors critically revised the manuscript for important intellectual content. All authors approved the final version and agree to be accountable for all aspects of the work.

Funding

This study was supported by a Canadian Institutes of Health Research (CIHR) operating grant (MOP 126197); a CIHR–Ontario Neurotrauma Foundation Mild Traumatic Brain Injury team grant (TM1 127047); and CIHR planning grant (MRP 119829).

Group Information

Additional Pediatric Emergency Research Canada (PERC) Concussion Team members include: Martin H. Osmond, MDCM (Department of Pediatrics, Children's Hospital of Eastern Ontario, University of Ottawa, Ottawa, Ontario, Canada); Nick Barrowman, Ph.D (Children's Hospital of Eastern Ontario Research Institute, Ottawa, Ontario, Canada); Stephen B. Freedman, MDCM, M.Sc (Department of Pediatrics, Alberta Children's Hospital, Alberta Children's Hospital Research Institute, University of Calgary, Alberta, Canada); Jocelyn Gravel, MD (Department of Pediatrics, Hospital Ste. Justine, University of Montreal, Montreal, Quebec, Canada); Gurinder Sangha, MD (Department of Pediatrics, Children's Hospital of Western Ontario, Western University, London, Ontario, Canada); Kathy Boutis, MD (Department of Pediatrics, Hospital for Sick Children, Toronto, Ontario, Canada); Darcy Beer, MD (Department of Pediatrics, Manitoba Children's Hospital, Winnipeg, Manitoba, Canada); William Craig, MDCM (Department of Pediatrics, Stollery Children's Hospital, Edmonton, Alberta, Canada); Emma Burns, MD (Department of Pediatrics, IWK Health Centre, Halifax, Nova Scotia, Canada); and Ken J. Farion, MD (Department of Pediatrics, Children's Hospital of Eastern Ontario, University of Ottawa, Ottawa, Ontario, Canada).

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The handling editor declared a past co-authorship with one of the authors IG.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fneur.2019.00672/full#supplementary-material

References

1. Meehan WP III, Mannix R. Pediatric concussions in United States emergency departments in the years 2002 to 2006. J Pediatr. (2010) 157:889–93. doi: 10.1016/j.jpeds.2010.06.040

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Control NCfIPa. Traumatic Brain Injury in the United States: Emergency Department Visits, Hospitalizations and Deaths 2002-2006. Atlanta, GA: Centers for Disease Control and Prevention (2010).

Google Scholar

3. Emery C, Kang J, Shrier I, Goulet C, Hagel B, Benson B, et al. Risk of injury associated with bodychecking experience among youth hockey players. Can Med Assoc J. (2011) 183:1249–56. doi: 10.1503/cmaj.101540

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Langlois JA, Rutland-Brown W, Wald MM. The epidemiology and impact of traumatic brain injury: a brief overview. J Head Trauma Rehabil. (2006) 21:375–8. doi: 10.1097/00001199-200609000-00001

PubMed Abstract | CrossRef Full Text | Google Scholar

5. McCrory P, Meeuwisse W, Dvorak J, Aubry M, Bailes J, Broglio S, et al. Consensus statement on concussion in sport—the 5 international conference on concussion in sport held in Berlin, October 2016. Br J Sports Med. (2017) 51:838–47. doi: 10.1136/bjsports-2017-097699

CrossRef Full Text | Google Scholar

6. Broglio SP, Cantu RC, Gioia GA, Guskiewicz KM, Kutcher J, Palm M, et al. National Athletic Trainers' Association position statement: management of sport concussion. J Athletic Train. (2014) 49:245–65. doi: 10.4085/1062-6050-49.1.07

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Gioia G, Schneider J, Vaughan C, Isquith P. Which symptom assessments and approaches are uniquely appropriate for paediatric concussion? Br J Sports Med. (2009) 43(Suppl. 1):i13–22. doi: 10.1136/bjsm.2009.058255

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Sady MD, Vaughan CG, Gioia GA. Psychometric characteristics of the postconcussion symptom inventory in children and adolescents. Arch Clin Neuropsychol. (2014) 29:348–63. doi: 10.1093/arclin/acu014

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Guskiewicz KM, McCrea M, Marshall SW, Cantu RC, Randolph C, Barr W, et al. Cumulative effects associated with recurrent concussion in collegiate football players: the NCAA Concussion Study. JAMA. (2003) 290:2549–55. doi: 10.1001/jama.290.19.2549

PubMed Abstract | CrossRef Full Text | Google Scholar

10. McLeod TCV, Perrin DH, Guskiewicz KM, Shultz SJ, Diamond R, Gansneder BM. Serial administration of clinical concussion assessments and learning effects in healthy young athletes. Clin J Sport Med. (2004) 14:287–95. doi: 10.1097/00042752-200409000-00007

CrossRef Full Text | Google Scholar

11. Schneider J, Gioia G. Psychometric properties of the Post-Concussion Symptom Inventory (PCSI) in school age children. Dev Neurorehabil. (2007) 10:282.

Google Scholar

12. Vaughan C, Gioia G, Vincent D. Initial examination of self-reported post-concussion symptoms in normal and mTBI children ages 5 to 12. J Int Neuropsych Soc. (2008) 14(Suppl. 1):207.

Google Scholar

13. Iverson GL, Lange RT, Brooks BL, Lynn Ashton Rennison V. “Good old days” bias following mild traumatic brain injury. Clin Neuropsychol. (2010) 24:17–37. doi: 10.1080/13854040903190797

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Lange RT, Iverson GL, Rose A. Post-concussion symptom reporting and the “good-old-days” bias following mild traumatic brain injury. Arch Clin Neuropsychol. (2010) 25:442–50. doi: 10.1093/arclin/acq031

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Schmier JK, Halpern MT. Patient recall and recall bias of health state and health status. Expert Rev Pharmacoeconom Outcomes Res. (2004) 4:159–63. doi: 10.1586/14737167.4.2.159

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Van den Brink M, Bandell-Hoekstra E, Abu-Saad HH. The occurrence of recall bias in pediatric headache: a comparison of questionnaire and diary data. Headache. (2001) 41:11–20. doi: 10.1046/j.1526-4610.2001.111006011.x

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Zemek R, Osmond MH, Barrowman N. Predicting and preventing postconcussive problems in paediatrics (5P) study: protocol for a prospective multicentre clinical prediction rule derivation study in children with concussion. BMJ Open. (2013) 3:e003550. doi: 10.1136/bmjopen-2013-003550

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Zemek R, Barrowman N, Freedman SB, Gravel J, Gagnon I, McGahern C, et al. Clinical risk score for persistent postconcussion symptoms among children with acute concussion in the ED. JAMA. (2016) 315:1014–25. doi: 10.1001/jama.2016.1203

PubMed Abstract | CrossRef Full Text | Google Scholar

19. McCrory P, Meeuwisse WH, Aubry M, Cantu B, Dvorák J, Echemendia RJ, et al. Consensus statement on concussion in sport: the 4th International Conference on Concussion in Sport held in Zurich, November 2012. Br J Sports Med. (2013) 47:250–8. doi: 10.1136/bjsports-2013-092313

PubMed Abstract | CrossRef Full Text | Google Scholar

20. R Core Team. R: A Language and Environment for Statistical Computing. Vienna: R Foundation for Statistical Computing (2016).

PubMed Abstract | Google Scholar

21. Lovell MR, Iverson GL, Collins MW, Podell K, Johnston KM, Pardini D, et al. Measurement of symptoms following sports-related concussion: reliability and normative data for the post-concussion scale. Appl Neuropsychol. (2006) 13:166–74. doi: 10.1207/s15324826an1303_4

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Ledoux A-A, Tang K, Yeates KO, Pusic MV, Boutis K, Craig WR, et al. Natural progression of symptom change and recovery from concussion in a pediatric population. JAMA Pediatri. (2019) 173:e183820-e. doi: 10.1001/jamapediatrics.2018.3820

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Covassin T, Swanik CB, Sachs M, Kendrick Z, Schatz P, Zillmer E, et al. Sex differences in baseline neuropsychological function and concussion symptoms of collegiate athletes. Br J Sports Med. (2006) 40:923–7. doi: 10.1136/bjsm.2006.029496

PubMed Abstract | CrossRef Full Text | Google Scholar