Brad Donohue
Jesse Scott1
Grace Goodwin
Kimberly A. Barchard
Daniel N. Allen- 1Department of Psychology, University of Nevada, Las Vegas, Las Vegas, NV, United States
- 2The Chicago School of Professional Psychology, Chicago, IL, United States
Introduction: There is a need to psychometrically develop assessment instruments capable of screening mental health disorders in athlete populations. The current study was conducted to determine reliability, validity and clinical utility of the Mental Health Disorders Screening Instrument for Athletes (MHDSIA).
Methods and results:: 259 collegiate athletes completed the MHDSIA. Factor analysis determined a single factor with good internal consistency, and this factor was positively correlated with an established measure of psychiatric symptomology (Symptom Checklist 90-R), demonstrating its concurrent validity. An optimum clinical cutoff score (i.e., 32) was determined using Receiver Operating Characteristic (ROC) analyses to assist appropriate mental health referrals.
Discussion: Results suggest the MHSIA is a reliable, valid, and relatively quick and easy to interpret screen for the broad spectrum of mental health disorders in collegiate athletes. As expected, NCAA athletes reported lower MHDSIA scores than club and intramural athletes, while males reported similar severity scores as females.
Introduction
Mental health symptomology in collegiate athletes
Annually, in excess of 10,000,000 college students play sports in the U.S. alone (Dugan et al., 2014; NCAA Sport Science Institute, 2016). Although exercise is generally associated with substantial benefits, athletes are potentially exposed to distinct stressors when compared with non-athletes (Arnold and Fletcher, 2012); which increases their risk of developing mental health-related problems (Sudano and Miles, 2017). While the actual prevalence of psychological problems within athletes has been debated, the notion that athletes are protected against mental health problems has been shown to be inaccurate (Wolanin et al., 2015). The rates of psychological disorders found in student-athletes and non-athletes tends to be very similar. Approximately 20% of adults suffer from significant mental-health problems on an annual basis (Hedden et al., 2015). However, the emergence of social media has created a platform for athletes to be publicly criticized (Kristiansen et al., 2012), and pressure to win competitions, entertain fans, and please coaches and families can lead to chronic feelings of stress, as athletes tend to display depressive symptoms after failing to achieve goals (Hammond et al., 2013) or losing competitions (Jones and Sheffield, 2008). The emerging adult years are associated with peak athletic performance, and are the years athletes are at highest risk for the onset of mental health disorders (Gulliver et al., 2012). Financial concerns, travel commitments, and maintaining academic eligibility requirements are common stressors within the student-athlete population (Rice et al., 2016). Student-athletes are expected to be successful in their classes, just as other students, but they are also expected to excel in athletic competition. Athletes often experience both physical and mental fatigue and report feeling socially isolated (Parham, 1993). In many cases, athletes must maintain superior levels of physical fitness and adhere to excessive sport-related time demands (Broughton and Neyer, 2001). There is evidence showing the daily stress associated with athletic participation may contribute to depression (Serido et al., 2004). Athletes who are subject to extreme training regimens tend to report higher levels of stress and depression than those who train less (Nixdorf et al., 2015), and intense physical activity has been shown to compromise physical well-being and increase symptoms of depression and anxiety in athletes, often due to injury, overtraining, or burnout (Peluso and Andrade, 2005). Meeusen et al. (2013) found that overtraining combined with insufficient recovery time may elevate athletes’ risk of developing a psychological disorder.
Student-athletes often report personal relationship difficulties, in addition to reduced energy and, and low levels of motivation (Parham, 1993). Living in an unfamiliar environment, reduced support networks due to relocation, and poor adjustment to life after sports are all associated with depression in athletes (Fletcher and Wagstaff, 2009). Over 40% of athletes in one study demonstrated low quantity of sleep and/or the quality of their sleep was poor (Mah et al., 2018). Chronically restricted sleep has important implications for potential injury (Luke et al., 2011), susceptibility to infectious illness (Prather et al., 2015), and the accuracy of concussion assessment (Silverberg et al., 2016). Additionally, restricted sleep has been shown to reduce reaction times (Dinges et al., 1997) and impair sport execution (Reyner and Horne, 2013), which both may have a negative impact on the athletes’ performance and, subsequently, their overall mental health.
The intense mental and physical demands of sport participation have been associated with risky behaviors (Hughes and Leavey, 2012). Nearly half of collegiate athletes are likely to experience an injury preventing them from participating in their sport, for at least a brief period of time (Meeuwisse et al., 2003), and injuries have been shown to increase athlete vulnerability to a variety of mental illnesses (Gulliver et al., 2015).
Investigations into the mental health of sub-groups of athletes are exceedingly rare. Researchers in France found female elite athletes were 1.3 times more likely to receive a diagnosis of a mental health disorder as compared to male athletes (Schaal et al., 2011). Many athletes display depression or anxiety related to adjusting to life outside of sports after their playing careers are over (Giannone et al., 2017). As compared to the general population, retired male athletes from team sports have consistently endorsed higher rates of distressing psychological symptoms (Van Ramele et al., 2017). Even athletes simply contemplating or preparing for retirement have shown elevations in self-reported symptoms of depression than those who are not considering retirement (Beable et al., 2017).
Evidence suggests high-level athletes demonstrate a similar risk of developing anxiety disorders as compared to the general population (Gulliver et al., 2015). Similarly, 17% to 21% of college-aged males and females meet criteria for depression in both athlete and non-athlete populations (Yang et al., 2007; Weigand et al., 2013) and nearly one-quarter of females in this age group, including athletes, endorse subthreshold symptoms of disordered eating (Greenleaf et al., 2009). One of the key concerns for mental health providers who treat college athletes is the high rate of comorbidity of psychological problems in this population. For example, student-athletes who demonstrate symptoms of eating disorders are likely to endorse elevated anxiety levels (Vardar et al., 2007), and athletes who endorse symptoms of depression display higher rates of alcohol abuse and dependence (Miller et al., 2002).
Depression is a major contributor to preventable hospitalizations (Davydow et al., 2015), associated with higher rates of substance use disorders, and linked with increased suicide risk [U.S. Department of Health and Human Services, National Institutes of Health, National Institute of Mental Health (NIMH), 2015]. In another study involving more than 200 athletes, almost half reported symptoms of depression or anxiety (Gulliver et al., 2015).
For providers in the most frequent contact with student-athletes, monitoring athlete mental health should be treated with the utmost importance, as the relationship between mental and physical health has been demonstrated repeatedly (Yang et al., 2014). For example, student-athletes experiencing symptoms of depression are more likely to exhibit a decline in sport performance or to suffer a sport-related injury. The same injury risk and performance-related problems have been linked to athletes who are diagnosed with eating disorders or alcohol abuse (Mountjoy et al., 2014).
Between 2008 and 2012, nearly one-third of the males and half of the females who participated in National Collegiate Athletic Association (NCAA) sports endorsed experiencing depressive or anxiety-related symptoms (Brown et al., 2014). Substance use is also a significant problem in university athletic populations (Donohue et al., 2018). In addition, student-athletes demonstrate an elevated risk of developing sleep problems (Brown et al., 2014), and disordered eating (Bratland-Sanda and Sundgot-Borgen, 2013.
Student-athletes have repeatedly demonstrated similar rates of clinical depression (Yang et al., 2007; Weigand et al., 2013) and eating disorders (Greenleaf et al., 2009), when compared to non-athlete peers. However, some findings suggest athletes may be particularly susceptible to these two disorders (Rice et al., 2016). It is possible certain subgroups of athletes demonstrate an elevated risk for eating disorders. For example, athletes who need a lean body shape to compete in their particular sport and female athletes in a variety of sports tend to be at higher risk than individuals in the general population (Byrne and Mclean, 2002; Sundgot-Borgen and Torstveit, 2010).
Athletes have also demonstrated a higher rate of alcohol use than the general population in some studies, possibly due to a binge pattern of consumption during the offseason or vacations (Mastroleo et al., 2018). Student-athletes report more problematic alcohol use and riskier drinking patterns than non-athletes, which could be related to the elevated physiological and psychological stress associated with sport participation (Brenner and Swanik, 2007). Male athletes as well as Caucasian athletes have been shown to drink alcohol more excessively than other groups (Leichliter et al., 1998). Although, this may not be unique to athletes as these same factors have been shown to be more predictive of drinking levels among college students in general (Baer, 2002). In another study, weekly alcohol use doubled the risk of injury among participants (O’Brien and Lyons, 2000).
There is some evidence to suggest male athletes use more illicit substances than their female counterparts; yet, rates of self-reported illicit drug use are typically low within the elite athletic population (Buckman et al., 2013). In a study examining the drug and alcohol use of over 1,000 student-athletes at Division I schools, over one-third of respondents endorsed taking banned performance-enhancing drugs, but less than 5 percent of them said they would disclose their use of drugs to coaches or other health care providers. Similarly, almost half of the athletes endorsed drinking more than five drinks in a week, yet only 3 percent said they would openly admit to doing so (Druckman et al., 2015). This provides some evidence that self-underreporting may be a problem when it comes to the assessment of drug and alcohol use among college athletes (Buckman et al., 2013). In a treatment outcome study requiring collegiate student athletes to evidence alcohol or illicit substance use in the past 4 months, more than 50% evidenced a current mental health disorder and 80% evidenced a current or past mental health disorder, most often a substance use disorder (Donohue et al., 2018).
Challenges in the identification of mental health symptomology in athletes
Most universities are ill-equipped to assess the psychological concerns of student-athletes due to a lack of athlete-specific assessment tools (Donohue et al., 2019; Hussey et al., 2019). Additionally, universities that do offer mental health screening options use screens that are difficult to interpret for those who are likely to administer these instruments (Watson, 2006). Identifying psychological disorders, especially in the case of subthreshold presentations, is a difficult undertaking for individuals who are not trained in mental health. One major barrier to the identification of mental health issues by untrained providers is that it can be challenging to differentiate between common athlete behaviors and symptoms of psychological disorders (Sundgot-Borgen and Torstveit, 2010). For instance, dedicated athletes may adhere to strict eating and exercise regimens that may appear to be disordered eating behaviors (Thompson and Sherman, 1999). Similarly, many athletes complain of fatigue, which could be a natural result of excessive sport training or, potentially, a symptom of depression (Esfandiari et al., 2011). Concussions have also been found to produce a negative impact on a student-athletes’ cognitive functioning for an extended period of time (Thoma et al., 2015). Concussion-related cognitive issues may be difficult to distinguish from poor concentration, which is a common symptom of multiple mental health disorders. Anxiety related to athletic performance, maintaining eligibility requirements, or lack of playing time may be challenging to discern from generalized anxiety. In difficult cases like these, a screening tool that aids in the early detection of potential psychological problems could differentiate between athletes who may benefit from mental health treatment and those who are not currently in need of services. Although, screening measures are not sufficient to diagnose mental health disorders, identifying dysfunction beyond what might be expected in sport settings can be the first step toward diagnosis and effective intervention.
Variation in assessment resources/practices in the United States
The mental health resources provided by universities for student-athletes vary significantly across the United States (Gallagher, 2012). Schools with larger athletic departments (i.e., Division I) tend to have larger mental health budgets than smaller schools (i.e., Division II or III; Matheson et al., 2012) and are more likely to have comprehensive athlete mental health screening programs (Kroshus, 2016). However, even in Division I programs, less than one-third of schools require their athletes to participate in annual mental health screening (Watson, 2006) and many universities fail to implement mental health screening of athletes altogether, possibly due to a shortage of qualified mental health providers or, perhaps more importantly, a lack of efficient and validated screening measures (Matheson et al., 2012). In a survey including 127 head athletic trainers of NCAA Division I schools, it was found only 42.5% of those schools utilized a mental health-screening instrument (Sudano and Miles, 2017). Moreover, the instruments currently in use do not assess mental health symptoms or these instruments have not been validated within athlete populations (Kroshus, 2016). The predominate mental health assessments currently in use include the Patient Health Questionnaire (PHQ-9; Kroenke et al., 2001), the Generalized Anxiety Disorder (GAD-7; Spitzer et al., 2006), Symptom Checklist 90 revised (SCL-90-R; Derogatis, 1994), and the Pre-Participation Physical Examination (PPE; Smith and Laskowski, 1998). Although these assessment measures have been psychometrically validated in non-athlete populations (Derogatis, 1994; Kroenke et al., 2001; Martin et al., 2006), in the case of the PHQ-9 and GAD-7 respectively, they only offer an assessment of one mental health issue (e.g., depression or anxiety) and neglect to examine the wide range of possible mental health concerns faced by athletes. The PPE is widely used to screen athletes for physical or medical concerns (i.e., respiratory problems, vision, pre-existing health conditions), but does not assess for the presence of psychological issues (Cottone, 1999). The Symptom Checklist 90-Revised (SCL-90-R; Derogatis, 1994) is currently one of the most frequently implemented tools for assessing global psychological symptomatology in non-athlete populations. The SCL-90-R is a 90-item questionnaire designed to assess nine distinct dimensions of mental health functioning. The SCL-90-R’s reliability and validity have also been examined in multicultural settings (Martin et al., 2006). This measure has also been utilized to examine the psychometric properties of a wide variety of screening instruments (Øiesvold et al., 2011). However, its length may make it less economical to administer.
Compounding the problem of inadequate screening, there is reason to believe athletes’ responses on these measures may not be accurate, perhaps because athletes tend to be unaware of the relationship between athletic performance and mental health (Buckman et al., 2013; Neal et al., 2013), and because they may be concerned with potential negative consequences for reporting mental health dysfunction. Due to this lack of awareness, student-athletes are considered a unique population for which sport-focused mental health assessment may be more effective (Comeaux et al., 2017). There are three sport-specific mental health screens. The Sport Interference Checklist (SIC; Donohue et al., 2007) is a validated measure that identifies mental health interferences in sport competition and training. Examinations of the SIC have demonstrated its strong factor structure and concurrent validity (Donohue et al., 2007), and it is capable of reliably screening symptoms of mental health disorders in athletes (Donohue et al., 2018). However, it does not assess the extent to which mental health symptoms may interfere in life outside of sports. The Student Athlete Relationship Inventory (SARI; Donohue et al., 2007) is another reliable and valid sport-specific method of screening mental health in athletes (Donohue et al., 2007). Its items assess sport-specific problems experienced by athletes with their coaches, teammates, family. However, SARI item stems are not directly relevant to mental health symptoms, thus it lacks face validity. Lastly, the Athlete Psychological Strain Questionnaire (APSQ; Rice et al., 2020) is a 10 item screening instrument for mental health problems, and includes 3 subscales (i.e., Self-regulation, Performance, External Coping). This scale has demonstrated excellent reliability and validity in elite athletes. Although these measures have successfully identified athletes who are at-risk for mental health treatment, the item stems do not explicitly assess the wide array of mental health disorders.
Understaffing and lack of financial resources make effective screening tools important in university athletic systems, particularly in smaller schools, with presumably lower budgets allocated to athlete mental health. Some have advocated for the utilization of brief validated screening measures that do not require professional interpretation (Eisenberg et al., 2007). Identifying psychological disorders may be profoundly impactful in addressing the mounting financial strain on colleges and athletic departments. Consequently, interest is rising among university administrations to improve mental health screening instruments; thus, reducing long-term mental health care costs (Neal et al., 2013; Rao et al., 2015; Wolanin et al., 2015).
A major obstacle in connecting athletes with mental health services is the lack of mental health assessment tools that have been validated in athlete populations, are psychometrically valid, and are capable of identifying both clinical and subclinical mental health symptomology. A measure examined explicitly in athletes, easy to administer, and that reliably assesses mental health problems could help better assist athletes in their receipt of mental health services when needed. The aforementioned sport-specific screens do not comprehensively and explicitly assess mental symptomology factors outside of sport (Bar and Markser, 2013). Indeed, while it is important for student athletes to perform well in sport settings, stakeholders of athletes should also work to ensure student-athletes are able to perform well in their life outside of sport (Lazarus, 2000).
Importantly, investigators have yet to examine how severity of athletes’ mental health symptomology is related to their performance in life outside of sports. This is an important oversight given the increased interest sport organizations have expressed in treating athletes holistically. Along these lines, existing sport-specific mental health assessments have demonstrated reliability and validity in athlete samples, and the factor scores of these items have been shown to accurately predict mental health symptomology within sport settings. However, these scales do not explicitly assess the wide range of mental health disorders that are evidenced in athlete populations.
The purpose of the present study was to psychometrically examine the utility of a novel mental health screening measure in a sample of collegiate athletes (i.e., Mental Health Disorders Screening Instrument for Athletes; MHDSIA), including an evaluation of this measure’s factor structure, internal consistency, convergent validity, and clinical cutoff scores to assist in determining collegiate athletes who may benefit from mental health referrals.
Methods
Participants
Participants were 259 undergraduate students enrolled at a National Collegiate Athletic Association (NCAA) Division I university in the United States. Participants were at least 18 years of age; competed in NCAA, club, or intramural sports; consumed at least one alcoholic drink or non-prescribed drug during the previous 4 months according to self-report, and expressed interest in receiving goal-oriented psychologically-based intervention in an outcome study funded by the National Institutes of Health (NIH) in the United States.
Measures
Mental health disorders screening instrument for athletes
Eight collegiate students with competitive sport experience created the measure in a focus group with a licensed clinical psychologist who acted as moderator. The goal was to create one inventory specific to mental health factors, with each item stem encapsulating a mental health disorder in the DSM-5. The moderator facilitated discussion (Krueger and Casey, 2000), while maintaining minimal control over discussion content, consistent with Millward’s (1995) approach. The moderator’s other main responsibility was to ensure efficiency and depth of discussion. Item stems were derived to fit within the following context: “How often does [item stem] interfere with performance during life outside of sports?” This was conceptually important because in the DSM-5 almost all mental health disorders require functional impairment.
Symptom frequency has proven to be a reliable indicator of functional impairment; thus, it was determined that a frequency response format would be appropriate for this measure. According to Weng (2004), a 7-point frequency scale is the preferred response scale in college student samples. Consequently, a 7-point frequency scale was selected for this measure, ranging from 1 (never) to 7 (always). Specifically, the athletes are asked to respond how often the item content interferes with performance outside of sports. A copy of this scale is provided in Appendix.
Symptom checklist 90-revised
The Symptom checklist 90-revised (SCL-90-R; Derogatis, 1994) is a 90-item self-report measure, assessing nine distinct dimensions of mental health functioning. The SCL-90-R prompts participants to rate the severity of 90 symptoms over the past week using a 5-point scale from 0 to 4, on which a 0 indicates the participant has not experienced the symptom in the past week and a 4 indicates the participant has been extremely distressed by the symptom over the past week. The Global Severity Index (GSI) is this measure’s assessment of general psychological impairment. The GSI has been shown to have strong psychometric properties in a number of studies (Derogatis, 1994). The GSI was used in the present study to evaluate whether general psychological impairment is associated with psychological interference with the athletes’ performance in their lives outside of their respective sport. In previous studies, both college athletes at various levels of competition (i.e., NCAA, recreational) have exhibited significantly lower GSI scores as compared to their non-athlete peers (Donohue et al., 2004).
Procedure
The current study occurred within the context of baseline assessment in a randomized clinical trial. Participants in this study were recruited to determine their interest in being randomized to one of two goal-oriented programs (i.e., traditional campus counseling and psychological services, The Optimum Performance Program in Sports; see Donohue et al., 2018). Participants were recruited in several ways, including course credit, Athletics Department referral consequent to problematic alcohol use or illicit drug use, flyers describing the study that were circulated throughout campus, after study presentations during athletic team meetings and performance workshops. After study consent, participants completed study measures. The study was approved by the university’s institutional review board (IRB; #1206-4,177 M). No adverse events occurred during the course of this study. The participants completed the study within the context of a certificate of confidentiality issued by the NIH, which protects the participants’ records from being released due to court mandate.
Study hypotheses
H1: Based on the results of factor analysis, each item from the experimental measure will load onto a single factor because all items are psychiatric constructs impacting life performance outside of sports.
H2: Items within the resulting factor will evidence high internal consistency.
H3: The hypothesized factor score (MHDSIA) will evidence a positive correlation with the SCL-90-R Global Severity Index (GSI), demonstrating its concurrent validity.
H4: Males will demonstrate significantly lower scores on the MHDSIA and GSI, as compared with females.
H5: NCAA athletes will demonstrate lower scores on the MHDSIA and GSI than club and intramural athletes.
H6: A cutoff score will distinguish between non-clinical and clinical level mental health concerns.
Statistical analysis
Principal component analysis will be performed on the MHDSIA inventory. The number of factors will be established according to the results of parallel analysis (Horn, 1965) and Minimum Average Partial (MAP, Velicer, 1974) tests. A sample size of 100 to 200 participants has been shown to produce a reasonably stable factor structure, although 300 participants are recommended (Guadagnoli and Velicer, 1988).
Cronbach’s alpha will be applied to calculate the internal consistency of all resulting factors. Feldt’s (1965) method of calculating confidence intervals for Cronbach’s alpha will be used to calculate confidence intervals for ICC (A,k). Total scores will be calculated for all recognized factors to establish a numerical representation of each participant’s total pattern of symptomology. Participants’ total scale scores will be calculated by summing their responses to items in the experimental measure. Correlations will also be calculated between the MHDSIA and SCL-90-R Global Severity Index (GSI) to determine the convergent validity of the measure. A positive correlation is expected between the scores of the experimental measure and the SCL-90-R GSI because mental health symptom severity would presumably be associated with an increase in psychological interference outside of sports.
To determine if participants of different genders and varying sport levels exhibited similar response patterns on the MHDSIA and SCL-90-R, a multivariate analyses of variance (MANOVA) will be performed using the MHDSIA and SCL-90-R as dependent variables and athlete type (NCAA, club, recreational) as the independent variable, and a similar MANOVA will be performed with gender as the independent variable. Post-hoc analyses will be performed for significant MANOVAs.
To explore whether the MHDSIA scale can be used to assist in the referral of athletes to mental health services, Receiver Operating Characteristic (ROC) analyses will be used to predict MHDSIA scores that are significantly associated with clinically elevated scores on the SCL-90-R GSI. ROC analysis will be used to generate empirical cut-off scores for the MHDSIA. Factor scores on the MHDSIA corresponding to a T score of 60 on the SCL-90-R GSI will be considered clinically relevant and can be considered cues for referral to appropriate mental health services. Cut off scores will be identified using Youden’s Index, which equals sensitivity + specificity – 1, and thus maximizes overall correct classification.
Results
Participant demographic and assessment variables
Table 1 displays the participants’ demographic information and Table 2 displays the means and standard deviations for the primary study variables.
Table 1. Demographic information (N = 259).
Table 2. Means and standard deviations of study variables (n = 259).
Outliers
Mahalanobis distance analysis indicated the associated for value of p for 257 of the sets was greater than 0.001 and, therefore, these sets are not considered outliers (Tabachnick and Fidell, 2013). Two sets of scores did meet the 0.001 alpha outlier criterion; however, after examination of the scores it was determined these extreme scores were likely not due to error and both indicated fewer self-identified mental health symptoms endorsed on the SCL-90-R as compared to the experimental measure. Due to the minimal number of outliers and the expected variability within a sample of this size, no cases were selected for removal.
MHDSIA
A principal components analysis revealed a one-factor solution for the MHDSIA, comprising 36.2% of the total variance. As displayed in Table 3, items loaded onto one factor related to general mental health symptoms. The scale items demonstrated high internal consistency (Cronbach’s α = 0.86, 95% CI [0.83, 0.88]) and standardized alpha (0.86). The intraclass correlation for absolute agreement [ICC (A,k)] was also strong (0.83, 95% CI [0.79, 0.86]). Corrected item-total correlations and alphas-if-item-deleted were also performed on the items of the MHDSIA. The results are summarized in Table 4. As no corrected item-total correlations were below 0.3, the deletion of any items would not improve internal consistency. All alpha-if-item-deleted values fell between 0.84 and 0.85. As the coefficient alpha of the 14-item measure is 0.86, the removal of any items would not improve internal consistency of this measure. Convergent validity for the MHDSIA was assessed by correlating the total factor score with the SCL-90-R Global severity index (GSI). The correlation between this scale and the GSI was significant (r(25) = 0.62, p < 0.008, 95% CI [0.53, 0.68]) and demonstrates a strong positive association between the two measures, which indicates that as athletes experience an increase in mental health symptoms, they experience related interference with their lives outside of sport.
Table 3. First principal component (MHDSIA).
Table 4. Item-total statistics (MHDSIA).
Analysis of potential effects due to gender and athlete type
Gender
It was predicted that males would exhibit more severe mental health symptomology on the MHDSIA and SCL-90R (see Table 5 for means and standard deviations). To examine this hypothesis, a MANOVA was performed utilizing the MHDSIA and the SCL-90-R as dependent variables. There was a statistically significant difference in gender, F(2, 256) = 3.73, p < 0.05, Wilk’s Λ = 0.972. To determine the source of these differences, one-way ANOVAs were performed using the MHDSIA scale and the SCL-90-R as dependent variables and gender as the independent variable for analyses. There were no significant differences between genders on the MHDSIA scale [F(1,257) = 0.004, p > 0.05]; however, there was a difference between genders on the SCL-90-R [F(1,257) = 4.69, p < 0.05]. Specifically, female athletes endorsed the presence of more overall mental health symptoms than male athletes, p < 0.05.
Table 5. Screening measure scores across gender and sport level.
Athlete type
It was also expected that different types of athletes (intramural, club, NCAA) would exhibit similar response patterns on the MHDSIA and SCL-90-R (see means and standard deviations for these scale in Table 5). To test this hypothesis, a MANOVA was performed utilizing the MHDSIA total score and the SCL-90-R GSI as dependent variables and sport type as the independent variable. There was a statistically significant difference in responses based on whether the participant was an NCAA, club, or intramural athlete F(4, 508) = 3.27, p < 0.005, Wilk’s Λ = 0.95. To determine the source of these differences, one-way ANOVAs were performed using the MHDSIA scale and the SCL-90-R as dependent variables and sport type as the independent variable for each analysis.
There were significant differences between athlete types on the MHDSIA scale [F(2,255) = 5.37, p < 0.01] and the SCL-90-R [F(2,255) = 8.40, p < 0.01]. A Tukey post hoc test revealed that NCAA athletes scores on the MHDSIA were significantly lower than club athletes and intramural athletes. There was a statistically significant difference between groups on the SCL-90-R scores [F(2,255) = 4.613, p < 0.05]. A Tukey post hoc test revealed that NCAA athletes responded significantly lower than club athletes on this scale (p < 0.01).
ROC results
For the ROC analysis, athletes were divided into two groups, in which 80 athletes demonstrated subclinical to clinical levels of mental health concerns (T-scores > 60) and 179 athletes demonstrated nonclinical levels of concern (T-scores < 60). For the MHDSIA, the variances in sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were evaluated. The PPV and NPV are accuracy statistics that indicate the number of identified positive cases that actually are of High Risk vs. those that are actually of Low Risk (compared to false positive and false negative classifications; Stojanovic et al., 2014). The MHDSIA’s ability to distinguish between athletes of High and Low-Risk for mental health difficulties was measured using the Area under the ROC curve (AUC), with an AUC of 0.50 indicating chance classification, and AUC of 1.00 indicating a perfect classification rate (Hosmer et al., 2013). AUCs were compared using the method of Hanley and McNeil (1983). Table 6 presents details on the AUCs for the MHDSIA. Cut off scores were ascertained using Youden’s Index (sensitivity + specificity – 1), thus, maximizing overall correct classification. Figure 1 presents the ROC curve for the MHDSIA, and Table 7 presents sensitivity, specificity, PPV, NPV, correct classifications, and diagnostic likelihood ratios (DLR) for each analysis. Results indicate that the MHDSIA identified High Risk athletes significantly better than chance, with an AUC of 0.86.
Table 6. Receiver operating characteristic (ROC) area under the curve (AUC) for the MHDSIA for classification of SCL-90-R global severity index scores.
Figure 1. ROC curve for MHDSIA.
Table 7. Classification accuracy statistics for SCL-90-R global severity index.
Given that NCAA and Club athletes had significantly different mean scores on the SCL-90-R and theMHDSIA, ROC analyses were performed on each athlete group separately to determine whether a cutoff score of 32 would be appropriate across these groups. The results of each ROC analysis indicated that a cutoff score of 32 is optimal regardless of athlete type. Indeed, in the case of NCAA and Club athletes, sensitivity and specificity both increase when groups are examined independently. For NCAA athletes, a cutoff score of 32 produces a sensitivity of 0.85 and a sensitivity of 0.91, while the AUC increases to 0.94. For Club athletes, a cutoff score of 32 produces a sensitivity of 0.88 and a specificity of 0.86, while the AUC increases to 0.96. Lastly, for intramural athletes, a cutoff score of 32 offers the highest diagnostic likelihood ratio, although sensitivity decreases to 0.72 and specificity decreases to 0.69.
Discussion
There is a great need to empirically develop mental health screening tools in athlete samples. Indeed, the extant measures that have been evaluated in the assessment of athletes’ mental health tend to be focused on the impact of psychological issues experienced during sport performance competition or training (Donohue et al., 2018, 2019; Hussey et al., 2019; Rice et al., 2019, 2020), potentially neglecting assessment of symptoms manifesting in athletes’ lives outside of sports. Along this vein, mental health-related problems outside of sport must be considered carefully when working with student-athletes. According to the APA (2013), mental health disorders are often marked by functional impairments in multiple settings. In the assessment of athletes’ mental health, a decline in sport performance due to outside stressors is relatively common. However, these stressors are often not openly discussed with team physicians or coaches (Reardon and Factor, 2010). Therefore, assessing athletes’ mental health symptomology as an interference with performance, as compared with a pathological framework (i.e., dysfunction), may be an effective strategy to improve identification of treatment targets by reducing potential stigma. The current study was designed to examine the factor structure, reliability, and validity of a mental health screening measure explicitly for use in collegiate athlete populations (i.e., MHDSIA).
Results indicated that each individual item on the MHDSIA evidenced a salient loading on a single factor, and the MHDSIA demonstrated strong convergent validity with the a widely established measure of psychiatric symptomology (i.e., SCL-90-R GSI), as predicted. Although many items included in the MHDSIA overlap with items in the SCL-90-R (e.g., depressed mood, anxiety), the MHDSIA offers unique advantages. Indeed, the MHDSIA is relatively brief, and its items prompt athletes to explore how the respective content impacts functioning outside of sports. Males endorsed significantly fewer items on the SCL-90-R than females but did not demonstrate significant differences from females in their responses to the MHDSIA. This is an important finding due to the tendency of males to endorse fewer psychological symptoms than females (Grant et al., 2005) due, in part, to men having greater perceptions of stigma associated with reporting mental health symptoms. Thus it is possible the MHDSIA may have lowered perceived mental health stigma that is often found in male athlete populations.
MHDSIA scores differentiated elite athletes (i.e., NCAA) from club and intramural athletes, providing support for its discriminate validity. This is an important finding as athletes competing at various levels of competition have demonstrated similar psychiatric symptomology (Donohue et al., 2004), and have evidenced similar psychiatric improvements subsequent to mental health treatment (Donohue et al., 2018). Nevertheless, examining MHDSIA responses of NCAA, intramural, and club athlete groups separately may be an area of focus in future research.
ROC analysis indicated the MHDSIA is an effective tool for detecting athletes who may be at-risk for clinically significant mental health concerns and that identified cut-off scores should be used to prompt appropriate referrals. These findings are in line with recommendations from the National Athletic Trainer’s Association that in order to promote the health and well-being of student-athletes, universities should utilize empirically validated mental health screening instruments (Conley et al., 2014; Kroshus, 2016). Results of this study suggest scores over 32 on this measure are predictive of significant, and potentially referable, mental health concerns (T-score greater than 60 on the SCL-90 R); which is considered by the NCAA to be a crucial aspect of the screening process (NCAA Sport Science Institute, 2016). Independent athlete-type ROC analyses also support a cut-off score of 32. Although, this cutoff score was selected because it provided high levels of both sensitivity and specificity, higher or lower scores may be used if higher specificity or sensitivity, respectfully, is desired (cutoff scores are outlined in Table 7). In addition, the standard error of measurement was 4.69 on the MHDSIA and because the items may not be parallel, the SEM may underestimate how far the observed scores are from true scores. Due to its predictive capability, the MHDSIA may be useful in linking student-athletes to warranted mental health services who may have otherwise been missed. Along these lines, they may be more likely to seek treatment if they realize the effect improvements in mental health difficulties have on their performance (Gulliver et al., 2015; Gouttebarge et al., 2017).
In regards to feasibility, the results of this study suggest the MHDSIA may be cost-effectively implemented with athletes who are determined to be at-risk for mental health disorders by licensed mental health experts working with trained personnel (e.g., athletic trainers), or broadly implemented across teams, universities or leagues by athletic administrations or schools using confidentially-protected computer-based programs that are appropriately monitored by licensed mental health professionals. Administration of this scale may be completed in under 2 min and items appear to measure what they are intended to measure. The clinical utility of the MHDSIA is enhanced by cutoff scores that help to identify individuals who may benefit from a referral to mental health services.
Implementation of such strategies facilitates athletes’ awareness of mental health functioning outside of sports, potentially improving the pursuit of mental health care in athletes. Future outcome research might examine whether discussion of MHDSIA responses is more effective than discussion of responses to other general mental health screeners in the improvement of motivation for mental healthcare (Pinkerton et al., 1989; Donohue et al., 2016). The MHDSIA provides an opportunity to establish specific treatment goals by identifying elevations in performance interference in particular domains outside of sport. Supporting this contention, high specificity of treatment goals can help provide clarity concerning which specific interventions are likely to be beneficial (Miller and Rollnick, 2013).
Lastly, the sample size (259) was probably enough to yield a stable factor structure. However, a sample size of 300 is recommended when evaluating this type of data to prevent uninterpretable or unstable factor matrices (Tabachnick and Fidell, 2013). Future research is needed to confirm the factor structure that was found in the current sample.
Data availability statement
The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.
Ethics statement
The studies involving human participants were reviewed and approved by the University of Nevada, Las Vegas International Review Board. Written informed consent to participate in this study was provided by the participants.
Author contributions
All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.
Funding
This research study was supported by a grant from the National Institutes of Health in the United States (National Institute on Drug Abuse award #1R01 DA031828). Publishing costs were funded by the University Libraries Open Article Fund and the College of Liberal Arts at the University of Nevada, Las Vegas. This study was funded by the National Institute on Drug Abuse; NIDA; 1 R01 DA031828.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Arnold, R., and Fletcher, D. (2012). A research synthesis and taxonomic classification of the organizational stressors encountered by sport performers. J. Sport Exerc. Psychol. 34, 397–429. doi: 10.1123/jsep.34.3.397
Baer, J. (2002). Student factors: understanding individual variation in college drinking. J. Stud. Alcohol 14, 40–53. doi: 10.15288/jsas.2002.s14.40
Bar, K. J., and Markser, V. (2013). Sport specificity of mental disorders: the issue of sport psychiatry. Eur. Arch. Psychiatry Clin. Neurosci. 263, 205–210. doi: 10.1007/s00406-013-0458-4
Beable, S., Fulcher, M., Lee, A. C., and Hamilton, B. (2017). SHARPSports mental health awareness research project: prevalence and risk factors of depressive symptoms and life stress in elite athletes. J. Sci. Med. Sport 20, 1047–1052. doi: 10.1016/j.jsams.2017.04.018
Bratland-Sanda, S., and Sundgot-Borgen, J. (2013). Eating disorders in athletes: Overview of prevalence, risk factors and recommendations for prevention and treatment. European Journal of Sport Science 13(5), 499–508. doi: 10.1080/17461391.2012.740504
Brenner, J., and Swanik, K. (2007). High-risk drinking characteristics in collegiate athletes. J. Am. Coll. Health 56, 267–272. doi: 10.3200/JACH.56.3.267-272
Broughton, E., and Neyer, M. (2001). Advising and counseling student athletes. New Dir. Stud. Serv. 2001, 47–53. doi: 10.1002/ss.4
Brown, G. T., Hainline, B., Kroshus, E., and Wilfert, M. (Eds..). (2014). Mind, body and sport: Understanding and supporting student-athlete mental wellness. http://www.ncaa.org/health-and-safety/sport-science-institute/introductionmind-body449and-sport
Buckman, J. F., Farris, S. G., and Yusko, D. A. (2013). A national study of substance use behaviors among NCAA male athletes who use banned performance enhancing substances. Drug Alcohol Depend. 131, 50–55. doi: 10.1016/j.drugalcdep.2013.04.023
Byrne, S., and Mclean, N. (2002). Elite athletes: effects of the pressure to be thin. J. Sci. Med. Sport 5, 80–94. doi: 10.1016/S1440-2440(02)80029-9
Comeaux, E., Bachman, T., Burton, R. M., and Aliyeva, A. (2017). Undergraduate experiences of division I athlete science, technology, engineering, and mathematics (STEM) graduates. J. Sci. Educ. Technol. 26, 24–32. doi: 10.1007/s10956-016-9648-y
Conley, K. M., Bolin, D. J., Carek, P. J., Konin, J. G., Neal, T. L., and Violette, D. (2014). National Athletic Trainers’ association position statement: pre-preparation physical examinations and disqualifying conditions. J. Athl. Train. 49, 102–120. doi: 10.4085/1062-6050-48.6.05
Cottone, J. (1999). Preparticipation physical examinations. Athletic Ther Today 4, 55–60. doi: 10.1123/att.4.5.55
Davydow, D. S., Fenger-Gron, M., Ribe, A. R., Pedersen, H. S., Prior, A., Vedsted, P., et al. (2015). Depression and risk of hospitalisations and rehospitalisations for ambulatory care-sensitive conditions in Denmark: a population-based cohort study. Br. Med. J. 5, e009878–e009819. doi: 10.1136/bmjopen-2015-009878
Derogatis, L. R. (1994). SCL-90-R: Administration, scoring, and procedures manual (3rd Edn.). Minneapolis, MN: Derogatis.
Dinges, D. F., Pack, F., Williams, K., Gillen, K. A., Powell, J. W., Ott, G. E., et al. (1997). Cumulative sleepiness, mood disturbance, and psychomotor vigilance performance decrements during a week of sleep restricted to 4-5 hours per night. Sleep 20, 267–277. doi: 10.1093/sleep/20.4.267
Donohue, B., Dickens, Y., Lancer, K., Covassin, T., Hash, A., Miller, A., et al. (2004). Improving athletes’ perspectives of sport psychology consultation: a controlled evaluation of two interview methods. Behav. Modif. 28, 182–193. doi: 10.1177/0145445503259399
Donohue, B., Dowd, A., Phillips, C., Plant, C. P., Loughran, T., and Gavrilova, Y. (2016). Controlled evaluation of a method to assist recruitment of participants into treatment outcome research and engage student-athletes into substance abuse intervention. J. Clin. Sport Psychol. 10, 272–288. doi: 10.1123/jcsp.2015-0022
Donohue, B., Galante, M., Maietta, J., Lee, B., Paul, N., Perry, J. E., et al. (2019). Empirical development of a screening method to assist mental health referrals in collegiate athletes. J. Clin. Sport Psychol. 13, 561–579. doi: 10.1123/jcsp.2018-0070
Donohue, B., Gavrilova, Y., Galante, M., Gavrilova, E., Loughran, T., Scott, J., et al. (2018). Controlled evaluation of an optimization approach to mental health and sport performance. J. Clin. Sport Psychol. 12, 234–267. doi: 10.1123/jcsp.2017-0054
Donohue, B., Silver, N. C., Dickens, Y., Covassin, T., and Lancer, K. (2007). Development and psychometric evaluation of the sport interference checklist. Behav. Modif. 31, 937–957. doi: 10.1177/0145445507303827
Druckman, J. N., Gilli, M., Klar, S., and Robison, J. (2015). Measuring drug and alcohol use among college student-athletes. Soc. Sci. Q. 96, 369–380. doi: 10.1111/ssqu.12135
Dugan, J., Torrez, M., and Turman, N. T. (2014). Leadership in intramural sports and club sports: examining influences to enhance educational impact. Available at: https://www.researchgate.net/publication/301293309
Eisenberg, D. E., Golberstein, E. E., and Gollust, S. E. (2007). Help-seeking and access to mental health care in a university student population. Med. Care 45, 594–601. doi: 10.1097/MLR.0b013e31803bb4c1
Esfandiari, A., Broshek, D. K., and Freeman, J. R. (2011). Psychiatric and neuropsychological issues in sports medicine. Clin. Sports Med. 30, 611–627. doi: 10.1016/j.csm.2011.03.002
Feldt, L. S. (1965). The approximate sampling distribution of Kuder-Richardson reliability coefficient twenty. Psychometrika 30, 357–370. doi: 10.1007/BF02289499
Fletcher, D., and Wagstaff, C. R. (2009). Organizational psychology in elite sport: its emergence, application and future. Psychol. Sport Exerc. 10, 427–434. doi: 10.1016/j.psychsport.2009.03.009
Gallagher, R. P. (2012). Thirty years of the national survey of counseling center directors: a personal account. J. Coll. Stud. Psychother. 26, 172–184. doi: 10.1080/87568225.2012.685852
Giannone, Z. A., Haney, C. J., Kealy, D., and Ogrodniczuk, J. S. (2017). Athletic identity and psychiatric symptoms following retirement from varsity sports. Int. J. Soc. Psychiatry 63, 598–601. doi: 10.1177/0020764017724184
Gouttebarge, V., Jonkers, R., Moen, M., Verhagen, E., Wylleman, P., and Kerkhoffs, G. (2017). The prevalence and risk indicators of symptoms of common mental disorders among current and former Dutch elite athletes. J. Sports Sci. 35, 2148–2156. doi: 10.1080/02640414.2016.1258485
Grant, B. F., Hasin, D. S., Stinson, F. S., Dawson, D. A., Ruan, W. J., Goldstein, R. B., et al. (2005). Prevalence, correlates, co-morbidity, and comparative disability of DSM-IV generalized anxiety disorder in the USA: results from the National Epidemiologic Survey on alcohol and related conditions. Psychol. Med. 35, 1747–1759. doi: 10.1017/S0033291705006069
Greenleaf, C., Petrie, T. A., Carter, J., and Reel, J. J. (2009). Female collegiate athletes: prevalence of eating disorders and disordered eating behaviors. J. Am. Coll. Health 57, 489–496. doi: 10.3200/JACH.57.5.489-496
Guadagnoli, E., and Velicer, W. F. (1988). Relation to Sample Size to the Stability of Component Patterns. Psychological Bulletin. 103, 265–275. doi: 10.1037/0033-2909.103.2.265
Gulliver, A., Griffiths, K., and Christiansen, H. (2012). Barriers and facilitators to mental health help-seeking for young elite athletes: a qualitative study. BMC Psychiatry 12, 157–165. doi: 10.1186/1471-244X-12-157
Gulliver, A., Griffiths, K. M., Mackinnon, A., Batterham, P. J., and Stanimirovic, R. (2015). The mental health of Australian elite athletes. J. Sci. Med. Sport 18, 255–261. doi: 10.1016/j.jsams.2014.04.006
Hammond, T., Gialloreto, C., Kubas, H., and Davis, H. (2013). The prevalence of failure-based depression among elite athletes. Clin. J. Sport Med. 23, 273–277. doi: 10.1097/JSM.0b013e318287b870
Hanley, J. A., and Mcneil, B. J. (1983). A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 148, 839–843. doi: 10.1148/radiology.148.3.6878708
Hedden, S. L., Kennet, J., Lipari, R., Medley, G., Tice, P., Copello, E. A., et al. (2015). Behavioral health trends in the United States: results from the 2014 national survey on drug use and health. Available at: https://www.samhsa.gov/data/sites/default/files/NSDUH-FRR1-2014/NSDUH-FRR1-2014.pdf
Horn, J. L. (1965). A rationale and test for the number of components from the matrix of partial correlations. Psychometrika 30, 179–185. doi: 10.1007/BF02289447
Hosmer, D. W., Lemeshow, S., and Sturdivant, R. X. (2013). “Introduction to the logistic regression model” in Applied logistic regression. eds. D. Hosmer, S. Lemeshow, and R. Sturdivant. 3rd ed (Hoboken, NJ: John Wiley & Sons), 1–30.
Hughes, L., and Leavey, G. (2012). Setting the bar: athletes and vulnerability to mental illness. Br. J. Psychiatry 200, 95–96. doi: 10.1192/bjp.bp.111.095976
Hussey, J., Donohue, B., Barchard, K., and Allen, D. N. (2019). Family contributions to sport performance and their utility in predicting appropriate referrals to mental health optimization programmes. Eur. J. Sport Sci. 19, 972–982. doi: 10.1080/17461391.2019.1574906
Jones, M. V., and Sheffield, D. (2008). The impact of game outcome on the well-being of athletes. Int J Sport Exerc Psychol 5, 54–65. doi: 10.1080/1612197X.2008.9671812
Kristiansen, E., Halvari, H., and Roberts, G. C. (2012). Organizational and media stress among professional football players: testing an achievement goal theory model. Scand. J. Med. Sci. Sports 22, 569–579. doi: 10.1111/j.1600-0838.2010.01259.x
Kroenke, K., Spitzer, R. L., and Williams, J. B. (2001). The PHQ-9. J. Gen. Intern. Med. 16, 606–613. doi: 10.1046/j.1525-1497.2001.016009606.x
Kroshus, E. (2016). Variability in institutional screening practices related to collegiate student-athlete mental health. J. Athl. Train. 51, 389–397. doi: 10.4085/1062-6050-51.5.07
Krueger, R. A., and Casey, M. A. (2000). Focus groups: A practical guide for applied research. Thousand Oaks, CA: Sage Publications.
Lazarus, R. S. (2000). How emotions influence performance in competitive sports. Sport Psychol. 14, 229–252. doi: 10.1123/tsp.14.3.229
Leichliter, J. S., Meilman, P. W., Presley, C. A., and Cashin, J. R. (1998). Alcohol use and related consequences among students with varying levels of involvement in college athletics. J. Am. Coll. Health 46, 257–262. doi: 10.1080/07448489809596001
Luke, A., Lazaro, R. M., Bergeron, M. F., et al. (2011). Sports-related injuries in youth athletes: is overscheduling a risk factor? Clinical Journal of Sport Medicine. Sleep Health 21(4), 307–314.
Mah, C. D., Kezirian, E. J., Marcello, B. M., and Dement, W. C. (2018). Poor sleep quality and insufficient sleep of a collegiate student-athlete population. Sleep Health 4, 251–257. doi: 10.1016/j.sleh.2018.02.005
Martin, A., Rief, W., Klaiberg, A., and Braehler, E. (2006). Validity of the brief patient health questionnaire mood scale (PHQ-9) in the general population. Gen. Hosp. Psychiatry 28, 71–77. doi: 10.1016/j.genhosppsych.2005.07.003
Mastroleo, N. R., Barnett, N. P., and Bowers, K. M. (2018). Association between sex, race/ethnicity, season, day of week and alcohol use and related risks in college student athletes and nonathletes. J. Am. Coll. Health 67, 422–432. doi: 10.1080/07448481.2018.1484367
Matheson, V. A., O’connor, D. J., and Herberger, J. H. (2012). The bottom line: accounting for revenues and expenditures in intercollegiate athletics. Int. J. Sport Financ. 7, 30–45.
Meeusen, R., Duclos, M., Foster, C., Fry, A., Gleeson, M., Nieman, D., et al. (2013). Prevention, diagnosis and treatment of the overtraining syndrome: joint consensus statement of the European College of Sport Science (ECSS) and the American College of Sports Medicine (ACSM). Eur. J. Sport Sci. 13, 1–24. doi: 10.1080/17461391.2012.730061
Meeuwisse, W. H., Selmer, R., and Hagel, B. E. (2003). Rates and risks of injury during intercollegiate basketball. Am. J. Sports Med. 31, 379–385. doi: 10.1177/03635465030310030901
Miller, B. E., Miller, M. N., Verhegge, R., Linville, H. H., and Pumariega, A. J. (2002). Alcohol misuse among college athletes: self-medication for psychiatric symptoms? J. Drug Educ. 32, 41–52. doi: 10.2190/JDFM-AVAK-G9FV-0MYY
Miller, W. R., and Rollnick, S. (2013). Motivational interviewing: Helping people change., 3rd Edn. New York, NY: Guilford Press.
Millward, L. J. (1995). “Focus groups” in Research methods in psychology. eds. G. M. Breakwell, S. Hammond, and C. Fife-Schaw (London: Sage Ltd), 274–292.
Mountjoy, M., Sundgot-Borgen, J., Burke, L., Carter, S., Constantini, N., Lebrun, C., et al. (2014). The IOC consensus statement: beyond the female athlete triad--relative energy deficiency in sport (RED-S). Br. J. Sports Med. 48, 491–497. doi: 10.1136/bjsports-2014-093502
NCAA Sport Science Institute. (2016). Mental health best practices: inter-association consensus document. Available at: http://www.ncaa.org/sites/default/files/HS_Mental-Health-Best-Practices_20160317.pdf
Neal, T. L., Diamond, A. B., Goldman, S., Klossner, D., Morse, E. D., Pajak, D. E., et al. (2013). Inter-association recommendations for developing a plan to recognize and refer student-athletes with psychological concerns at the collegiate level: an executive summary of a consensus statement. J. Athl. Train. 48, 716–720. doi: 10.4085/1062-6050-48.4.13
Nixdorf, I., Frank, R., and Beckmann, J. (2015). An explorative study on major stressors and its connection to depression and chronic stress among German elite athletes. Adv. Physiol. Educ. 5, 255–262. doi: 10.4236/ape.2015.54030
O’Brien, C., and Lyons, F. (2000). Alcohol and the athlete. Sports Med. 29, 295–300. doi: 10.2165/00007256-200029050-00001
Øiesvold, T., Bakkejord, T., and Sexton, J. A. (2011). Concurrent validity of the health of the nation outcome scales compared with a patient-derived measure, the symptom Checklist-90-revised in out-patient clinics. Psychiatry Res. 187, 297–300. doi: 10.1016/j.psychres.2010.10.023
Parham, W. D. (1993). The intercollegiate athlete: A 1990s profile. The Counseling Psychologist, 21, 411–412.
Peluso, M. A., and Andrade, L. H. (2005). Physical activity and mental health: the association between exercise and mood. Clinics 60, 61–70. doi: 10.1590/S1807-59322005000100012
Pinkerton, R. S., Hinz, L. D., and Barrow, J. C. (1989). The college student-athlete: psychological considerations and interventions. J. Am. Coll. Health 37, 218–226. doi: 10.1080/07448481.1989.9939063
Prather, A. A., Janicki, D., Hall, M. H., and Cohen, S. (2015). Behaviorally assessed sleep and susceptibility to the common cold. Sleep 38, 1353–1359. doi: 10.5665/sleep.4968
Rao, A. L., Asif, I. M., Drezner, J. A., Toresdahl, B. G., and Harmon, K. G. (2015). Suicide in National Collegiate Athletic Association (NCAA)athletes. Sports Health 7, 452–457. doi: 10.1177/1941738115587675
Reardon, C. L., and Factor, R. M. (2010). Sport psychiatry: a systematic review of diagnosis and medical treatment of mental illness in athletes. Sports Med. 40, 961–980. doi: 10.2165/11536580-000000000-00000
Reyner, L. A., and Horne, J. A. (2013). Sleep restriction and serving accuracy in performance tennis players, and effects of caffeine. Physiol. Behav. 120, 93–96. doi: 10.1016/j.physbeh.2013.07.002
Rice, S., Olive, L., Gouttebarge, V., Parker, A. G., Clifton, P., Harcourt, P., et al. (2020). Mental health screening: severity and cut-off point sensitivity of the athlete psychological strain questionnaire in male and female elite athletes. BMJ Open Sport Exerc. Med. 6:e000712. doi: 10.1136/bmjsem-2019-000712
Rice, S. M., Parker, A. G., Mawren, D., Clifton, P., Harcourt, P., Lloyd, M., et al. (2019). Preliminary psychometric validation of a brief screening tool for athlete mental health among male elite athletes. Int J Sport Exerc Psychol 18, 850–865. doi: 10.1080/1612197X.2019.1611900
Rice, S. M., Purcell, R., De Silva, S., Mawren, D., McGorry, P. D., and Parker, A. G. (2016). The mental health of elite athletes: a narrative systematic review. Sports Med. 46, 1333–1353. doi: 10.1007/s40279-016-0492-2
Schaal, K., Tafflet, M., Nassif, H., Thibault, V., Pichard, C., Alcotte, M., et al. (2011). Psychological balance in high level athletes: gender-based differences and sport-specific patterns. PLoS One 6:e19007. doi: 10.1371/journal.pone.0019007
Serido, J., Almedia, D. M., and Wethington, E. (2004). Chronic stressors and daily hassles: unique and interactive relationships with psychological distress. J. Health Soc. Behav. 45, 17–33. doi: 10.1177/002214650404500102
Silverberg, N. D., Berkner, P. D., Joseph, A. E., Ross, Z. L., and Iverson, G. L. (2016). Relationship between short sleep duration and preseason concussion testing. Clin. J. Sport Med. 26, 226–231. doi: 10.1097/JSM.0000000000000241
Smith, J., and Laskowski, E. R. (1998). The preparticipation physical examination: Mayo Clinic experience with 2,739 examinations. Mayo Clin. Proc. 73, 419–429. doi: 10.1016/S0025-6196(11)63723-3
Spitzer, R. L., Kroenke, K., Williams, J. B. W., and Lowe, B. (2006). A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch. Intern. Med. 166, 1092–1097. doi: 10.1001/archinte.166.10.1092
Stojanovic, M., Apostolovic, M., Stojanovic, D., Milosevic, Z., Toplaovic, A., Lakusic, V. M., et al. (2014). Understanding sensitivity, specificity, and predictive values. Pract Advic Physic 71, 1062–1065. doi: 10.2298/VSP1411062S
Sudano, L. E., and Miles, C. M. (2017). Mental health services in NCAA division I athletics: a survey of head ATCs. Sports Health 9, 262–267. doi: 10.1177/1941738116679127
Sundgot-Borgen, J., and Torstveit, M. K. (2010). Aspects of disordered eating continuum in elite high-intensity sports. Scand. J. Med. Sci. Sports 20, 112–121. doi: 10.1111/j.1600-0838.2010.01190.x
Tabachnick, B. G., and Fidell, L. S. (2013). Using multivariate statistics (6th Edn.). Boston, MA: Allyn & Bacon/Pearson Education.
Thoma, R. J., Cook, J. A., Mcgrew, C., King, J. H., Mayer, A. R., Lewine, J. D., et al. (2015). The effect of days since last concussion and number of concussions on cognitive functioning in division I athletes. Brain Inj. 29, 633–638. doi: 10.3109/02699052.2014.999352
Thompson, R. A., and Sherman, R. T. (1999). “Good athlete” traits and characteristics of anorexia nervosa: are they similar? Eat. Disord. 7, 181–190. doi: 10.1080/10640269908249284
U.S. Department of Health and Human Services, National Institutes of Health, National Institute of Mental Health (NIMH). (2015). Depression (NIH publication no. 15–3561). Washington, DC: Government Printing Office.
Van Ramele, S., Aoki, H., Kerkhoffs, G., and Gouttebarge, V. (2017). Mental health in retired professional football players: 12-month incidence, adverse life events and support. Psychol. Sport Exerc. 28, 85–90. doi: 10.1016/j.psychsport.2016.10.009
Vardar, E., Vardar, S. A., and Kurt, C. (2007). Anxiety of young female athletes with disordered eating behaviors. Eat. Behav. 8, 143–147. doi: 10.1016/j.eatbeh.2006.03.002
Velicer, W. F. (1974). An empirical comparison of the stability of factor analysis, principal component analysis, and image analysis. Educational and Psychological Mcasurement, 34, 563–572.
Watson, J. C. (2006). Student-athletes and counseling: factors influencing the decision to seek counseling services. Coll. Stud. J. 40, 35–42.
Weigand, S., Cohen, J., and Merenstein, D. (2013). Susceptibility for depression in current and retired student athletes. Sports Health 5, 263–266. doi: 10.1177/1941738113480464
Weng, L. (2004). Impact of the number of response categories and anchor labels on coefficient alpha and test-retest reliability. Educ. Psychol. Meas. 64, 956–972. doi: 10.1177/0013164404268674
Wolanin, A., Gross, M., and Hong, E. (2015). Depression in athletes: prevalence and risk factors. Curr. Sports Med. Rep. 14, 56–60. doi: 10.1249/JSR.0000000000000123
Yang, J., Cheng, G., Zhang, Y., Covassin, T., Heiden, E. O., and Peek-Asa, C. (2014). Influence of symptoms of depression and anxiety on injury hazard among collegiate American football players. Res. Sports Med. 22, 147–160. doi: 10.1080/15438627.2014.881818
Yang, J. D., Peek-Asa, C. T., Corlette, J. T., Cheng, G. T., Foster, D. T., and Albright, J. T. (2007). Prevalence of and risk factors associated with symptoms of depression in competitive collegiate student athletes. Clin. J. Sport Med. 17, 481–487. doi: 10.1097/JSM.0b013e31815aed6b
Appendix
MHDSIA
Below is a list of things that sometimes occur with athletes during their lives outside of sports. Please circle the number that represents how often each of these things interferes with your life outside of sports.
Please use the following scale for items 1 through 14:
1 = Never, 2 = Very Seldom, 3 = Seldom, 4 = Sometimes, 5 = Often, 6 = Very Often, 7 = Always.
Keywords: sport, athlete, mental health screen, mental health assessment, mental health disorders screening instrument for athletes, referral, clinical utility, factor analysis
Citation: Donohue B, Scott J, Goodwin G, Barchard KA, Bohall G and Allen DN (2023) Initial examination of the mental health disorders: screening instrument for athletes. Front. Psychol. 14:1029229. doi: 10.3389/fpsyg.2023.1029229
Edited by:
Mark Uphill, Canterbury Christ Church University, United KingdomReviewed by:
Manh Van Pham, Hai Duong Medical Technical University, VietnamAna Ruivo Alves, University of Beira Interior, Portugal
Copyright © 2023 Donohue, Scott, Goodwin, Barchard, Bohall and Allen. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Brad Donohue, bradley.donohue@unlv.edu