Autism Spectrum Screening Checklist (ASSC): The Development of a Scale to Identify High-Risk Individuals Within the Children's Mental Health System

Autism Spectrum Disorder (ASD) is a complex childhood onset neurodevelopmental disorder that has become the fastest growing developmental disability. Due to the increased demand for diagnostic assessments and subsequent increased wait times, standardized screening as part of regular clinical practice is needed. More specifically, there is an important need for the development of a more streamlined screening tool within an existing assessment system to identify those at greatest risk of having ASD. The current study utilized data from ~17,000 assessments obtained within the province of Ontario, based on the interRAI Child and Youth Mental Health (ChYMH) and Child and Youth Mental Health and Developmental Disability (ChYMH-DD), to develop a scale to identify children who have a higher likelihood of having autism. The scale was then tested on a trial population with data from the interRAI Early Years instrument. Further analyses examined the predictive validity of the scale. The Autism Spectrum Screening Checklist (ASSC) was found to be a good predictor of ASD with a sensitivity of 0.73 and specificity of 0.62, at the recommended cut-point of 2+. The results were consistent across several age ranges, specifically from 2 to 21 years of age. The ASSC scale provides an initial screen to help identify children and youth at heightened risk for autism within larger populations being assessed as part of routine practice. The main goal for the development and implementation of the ASSC scale is to harness the power of the existing interRAI assessment system to provide a more efficient, effective screening and referral process. This will ultimately help improve patient outcomes through needs-based care.

Introduction

Autism spectrum disorder (ASD) is a complex, lifelong, neurodevelopmental disorder that is characterized by impairment in social communication and the presence of restricted repetitive behaviors (1, 2). ASD has a variety of causes, such as those at the genetic, biological, and environmental level (3). Some of the earliest signs reported by parents include lower levels of social communication and attention, increased repetitive behavior, and temperament dysregulation (4). However, there is also substantial heterogeneity in its presentation and overlap with other developmental disorders, especially in the first few years of life, thereby adding to the complexity of the diagnostic process (5, 6). Autism is also highly co-morbid with intellectual disability, with estimates around 50–70% being reported in the literature (7). Notably, ASD is associated with substantial disability across the lifespan, which is only exacerbated when interventions are not provided early on (8, 9).

Autism is one of the most common childhood onset neurodevelopmental disorders. It is estimated that the current prevalence rate is ~1–1.5% of the world's population (10). With respect to the pediatric population in particular, recent estimates indicate that 1 in 68 school-aged children has been identified with ASD in the United States (11). Autism has also been deemed the fastest growing developmental disability, with a steady increase in reported prevalence over the past decade (12). Increased prevalence rates have led to increased demand for diagnostic assessments, which typically exceeds available resources and results in increased wait times. Importantly, this waiting time occurs during a critical period of brain development, and so lengthy wait times may delay intervention and decrease its effectiveness (8, 13). Indeed, there is substantive research showing that early intervention is key to achieving better prognostic outcomes (14, 15). This waiting period also represents a highly stressful time for families (16, 17). As such, studies have found that many parents are frustrated and dissatisfied with the diagnostic process and experience it as slow, stressful, and poorly managed (18, 19). Three recent ASD guidelines have recommended a maximal wait time of 3 to 6 months, yet the average wait time between parents' first concerns around their child's developmental progress and diagnosis is 2 to 4 years (20). This discrepancy can help explain the consistent finding that the average age of diagnosis is 4–5 years-old despite the fact that ASD can be reliably detected within the second year of life (20).

Various factors account for the more than 2 year difference between parents noticing the early signs of autism to receiving a diagnosis. Some of these barriers include time-consuming evaluations as well as a lack of providers who are able to administer the diagnostic assessments (21, 22). Other important barriers include inappropriate referrals (or more specifically, over-referrals), and a lack of effective screening tools (23–25). In a comprehensive review of early autism screening, the authors concluded that their findings emphasize the need for “a more efficient, intelligent, and innovative ASD screening tool” (p. 24) (25). Some instruments: (1) are time-consuming to administer, (2) have an unacceptable level of sensitivity (e.g., 40%), and (3) are not comprehensive in terms of the population served; for example, some of the screening tools were only intended to be used on infants from 16 to 36 months, whereas others were strictly meant for adolescents/adults.

Finally, an overarching barrier to early identification and diagnosis is ineffective care pathways (23). Improved care pathways are needed to reduce waiting times for an ASD diagnostic assessment and direct each child to more appropriate services. More explicitly, there is a critical need for the development of a more streamlined, easily implemented, resource-effective screening method to identify those at greatest risk of having ASD and require a more comprehensive follow-up. This will help facilitate earlier diagnosis and, as a result, earlier intervention and better patient outcomes.

The aim of the present study was to develop a methodology for identifying children who are at greatest risk of having autism within the children's mental health system in the province of Ontario, Canada. Since no effective, easily implemented screening method exists, an effort was launched to develop a new scale for identifying individuals who have an increased likelihood of autism. The Autism Spectrum Screening Checklist (ASSC) was created to assist service providers in determining whether a toddler, child or adolescent is at higher risk of having ASD. This scale is embedded in an assessment-to-intervention system that is already used as standard practice across Ontario in most child and youth mental health agencies to foster effective, evidence-informed care pathways. The aim of this study is to describe the development of the ASSC scale.

Methods

Sample

Data came from assessments of children and youth receiving mental health services in Ontario, Canada. The derivation sample came from individuals aged 4–21 years assessed with the Child and Youth Mental Health (ChYMH) (26) or the Child and Youth Mental Health and Developmental Disability (ChYMH-DD) (27) instruments, as part of regular clinical practice from 54 agencies from 2012 to 2020. An additional sample of 2 and 3 year-old children assessed with the interRAI Early Years instrument (28) was used for a trial application, and collected from 15 agencies from 2017 to 2020. These assessment instruments are described below. Assessed individuals were referred to these agencies through a variety of sources including family and specialty physicians, school personnel, other allied health professionals, or parents/primary caregivers. Assessment information is used for a variety of purposes, including standardized care planning, as well as the use of items and calculated outcome measures to inform decision making and to track individual change.

There were 16,955 individuals in the derivation dataset, using the first assessment if an individual had more than one. The mean age was 11.95 years (SD 3.50) and 55.9% were male. There were 724 individuals in the trial application dataset of 2 and 3 year-old children, where the mean age was 2.48 (SD 0.52) and 68.8% were male. To examine predictive validity, a sub-sample of 318 individuals was used from the original derivation sample. The mean age was 11.02 years (SD 4.00) and 64.5% were male.

Assessors completed a 2.5 day training of each of the three interRAI Child and Youth instruments: ChYMH, ChYMH-DD, and the interRAI Early Years. The trained child/youth mental health professionals included psychologists, nurses, psychiatrists, speech and language therapists, child and youth workers, developmental social service workers, and social workers. All available sources of information are utilized to complete the assessment (i.e., family members, community members, document review, and clinical observations).

Secure web-based software was utilized to record assessment information, requiring responses of the proper form for all essential items before the record can be authorized as complete. Before making the data available for analysis, personal identifiers were removed. Western University's ethics board granted approval for the secondary analysis of data collected in various agencies throughout the province of Ontario (REB #106415).

Measures

The interRAI Early Years is a new instrument within the interRAI child/youth suite and has been designed for young children under the age of 4 years who are referred for assessment due to mental health, relational and/or developmental concerns (28). It provides unique information tailored to early identification and intervention (e.g., prenatal complications; family and social relations; temperamental characteristics; risks related to development and mental health). It also provides a comprehensive assessment of individual needs with applications that can be used to support decisions related to care planning and outcome measurement. There are compatible items in use across care domains that share design features such as a specified observation period or time frame, a focus on observable behaviors, the use of a few, powerful questions to assess areas of need, and the use of professional judgment to integrate multiple sources of information. The interRAI Early Years is compatible with other interRAI instruments across services and sectors (e.g., mental health, education, adult sectors), relevant for all age groups across the lifespan.

The interRAI ChYMH and ChYMH-DD are comprehensive, clinician-rated, standardized, and multi-sectoral mental health instruments for children and youth (26, 27). These instruments include over 400 items and build a comprehensive picture of the child's strengths, needs, functioning, and areas of risk to inform care-planning for clients with mental health needs. The clinician creates a clinical profile of children based on a collection of reports, observations, and judgments made from interactions with the family, the children themselves, and service providers with appropriate consent. Each instrument contains evidence-based items, scales, and domains relevant to the population used in this study. While the ChYMH was designed for children 4–18 years of age with potential mental health issues, the ChYMH-DD covers a range of common issues in children with global developmental delays or intellectual disabilities from 4–21 years of age. The items are tailored to the needs of children and youth with intellectual and developmental disabilities and mental health concerns in inpatient and outpatient settings as part of standard of care. Assessors rate a child/youth on a number of demographic variables, family, mental health, and physical health indicators.

For all three instruments, clinicians receive an item-by-item interpretation guide to the interRAI instruments with information regarding intent, definition, process, and proper coding method of each item to ensure accurate and uniform assessment of children/youth across multiple mental healthcare settings. The ChYMH, ChYMH-DD and interRAI Early Years include a subsection called “Diagnostic and other health information,” which collects diagnostic information on 12 provisional categories, including ASD, as determined by a psychiatrist, psychologist, or attending physician.

These instruments were designed to provide a comprehensive assessment to support enhanced individualized care planning (29–32), while providing clinical decision-support algorithms (33–36) to foster evidence-based prioritization/triaging. Notably, the relatively new interRAI Early Years provides 17 care planning protocols pertinent to specific areas of need (37); for example, attachment, sleep, caregiver distress, gross and fine motor skills, and sensory issues.

Strong reliability and validity for the scales and algorithms on the interRAI ChYMH and ChYMH-DD have been found (38–42). These instruments have several applications including outcome measurement, resource allocation, and case-mix systems (34, 35, 43–46).

Analysis

We sought to create a calculated scale that would explain a diagnosis of autism. The dependent variable was “Autism Spectrum Disorder” as a provisional diagnosis, for which the assessor records if a psychiatrist, psychologist, or attending physician has made this diagnosis. It is important to note that the assessor is not acting in any diagnostic capacity and is merely consulting all available sources of information to determine if such a diagnosis has been made. The recorded item requires the assessor to rank any of a number of provisional diagnoses by importance (most, second most, etc.); for our dependent variable, we collapsed a diagnosis of autism of any importance to be one, otherwise zero.

A list of potential explanatory variables was generated. All items in the ChYMH or ChYMH-DD instruments were considered by a clinical expert for their potential association with autism, and seven candidate items were selected (e.g., narrowly restricted range of interest and excessive preoccupation with activity or routine). Furthermore, bivariate associations between an autism diagnosis and other items in the derivation dataset were used to identify a small number of additional items that offered statistical strength. However, these additional items were not pursued, either because they were not available in the interRAI Early Years assessment instrument, or they were considered problematic for use with very young children (e.g., positive symptoms or having at least one friend). The seven candidate items were all binary. The last 3 days was the reference timeframe.

Using multivariable logistic regression, these seven items were tested together to assess their ability to independently predict an autism diagnosis and to remove non-contributing items. A series of tests using the count of retained items was then applied, with logistic regression of the count of the items to assess goodness of fit, and correlation analysis to provide a Cronbach alpha value for the individual items to inform internal consistency of the contributing items. Sensitivity and specificity for different summed scale cut-points were considered. The selected scale was tested in a trial population using the interRAI Early Years cases where it could be calculated. Furthermore, using a sample of 318 ChYMH or ChYMH-DD assessments, the ASSC scale was calculated, and longitudinal analyses related to predictive validity were conducted. Specifically, the sample included individuals for whom provisional diagnostic assignment (for all DSM-IV diagnoses, including ASD) had not been completed at the time of initial assessment, and a follow-up assessment within 365 days where it had been subsequently done. This allowed the scale to operate more like a predictive measure where a child or youth not yet subjected to diagnostic assignment at the time the scale is assigned is subsequently diagnosed, either positively or negatively, for autism.

Results

The seven candidate items and their distribution by autism diagnosis are summarized in Table 1, along with sample characteristics. One item, “lack of interest in social interaction”, was dropped because of a weaker association and also because it was not available in this form in the interRAI Early Years instrument. Progression of the analysis is summarized in Table 2. Step 1 used the six items, giving a Cronbach alpha of 0.702 that would be increased to 0.723 if the item “self-injurious behavior” was removed. When this was done (step 2), the model fit did not decrease, making the five-item construction superior to the six-item one. As one additional variation, the weakest item of these five, “difficulty adapting to even minor change”, was dropped, resulting in a four-item sum (step 3) with a small drop in internal consistency but no change in model fit. Distribution of the summed items, odds ratios of each sum total, as well as sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) for sum cut-points are also provided in Table 2. Receiver-Operator Characteristics (ROC) curve plot of the five-item sum is shown in Figure 1.

TABLE 1

Table 1. Sample characteristics.

TABLE 2

Table 2. Sequential steps and results: Derivation sample.

FIGURE 1

Figure 1. Receiver Operator Characteristics (ROC) Curve: Five-item scale.

The five-item version was ultimately selected based on it having more parsimony than the six-item version, and the best internal consistency of the three options. However, all three versions are quite similar in performance with strong goodness of fit; c-statistics are >0.82, which is considered to be a strong result (47).

The five-item summative scale was applied to the interRAI Early Years assessments, where 9.5% of the cases had a diagnosis of autism; the results are summarized in Table 3. Distribution of the summed items and sensitivity, specificity, PPV, and NPV for sum cut-points are also provided in Table 3. The c-statistic is slightly higher than that in the derivation cases, and the Cronbach alpha value is slightly lower. Regarding the distribution, the interRAI Early Years cases tended to be in the lower risk categories, consistent with this group having a lower likelihood of an autism diagnosis.

TABLE 3

Table 3. Five-item scale in trial population (age 3 and younger).

Using a sub-sample from the derivation dataset, the predictive validity of the ASSC score was investigated. Distribution of the summed items and percentage with autism diagnosis at follow-up are provided in Table 4, along with sensitivity, specificity, PPV, and NPV for sum cut-points. In utilizing this approach, cut-points of 1+ and 2+ would provide PPVs of 29.5 and 36.7%, respectively. However, it should be noted that these higher achieved PPVs relate directly to the higher prevalence of a future autism diagnosis at 23.3%, compared to a 12.5 and 9.5% prevalence rate in the derivation sample and trial population, respectively.

TABLE 4

Table 4. Longitudinal analysis: High risk for future autism diagnosis by Autism Spectrum Screening Checklist (ASSC) score at baseline.

Discussion

High risk for ASD was predicted by five contributing items, namely narrowly restricted range of interest, excessive preoccupation with activity or routine, lack of social/emotional conventions when socializing, excessive or unusual reaction to sensory stimuli, and difficulty adapting to even minor change. The contributing items are all well-known signs and symptoms of autism (1). Moreover, several of these items represent some of the earliest behavioral symptoms in ASD. For example, studies have found that some of the signs that are often noticed and reported first by parents include repetitive interests and behaviors, atypical social emotional responses, and extremes of behavioral activity (4, 48, 49). Furthermore, several studies that examined and coded family home videos found differences in repetitive behaviors, social behaviors, and sensory oriented behaviors between those with ASD and typically developing children; these differences were detectable as early as 12 months-old (50, 51). Therefore, in addition to the contributing items of the ASSC scale representing many of the typical symptoms of ASD, research has found that they are also some of the most commonly reported initial concerns.

Use and Utility of ASSC

Based on the findings, ASSC provides an empirically based score that may be used to identify toddlers, children, and youth who present with signs and symptoms that are known to increase one's likelihood of being diagnosed with ASD. Findings indicate that the ASSC is a good predictor of autism and has reasonable sensitivity and specificity at the designated cut-point of 2+. As a result, it will allow service providers to make more systematic evaluations in determining whether an individual is at greater risk of having ASD, which ultimately helps facilitate prioritization and triaging.

While the ASSC scale has reasonable sensitivity (0.73) and specificity (0.62) at the recommended cut-point of 2+, its PPV at this level is 0.367. This means that ~37% of children and adolescents above the designated cut-point will likely be diagnosed with autism. Although this percentage seems somewhat low, there are a couple of important considerations to take into account. First, both PPV and NPV depend on prevalence, with PPV being directly proportional to the prevalence of autism. This is exemplified in the current study whereby as prevalence of autism increased, for example, from within the trial population dataset to the longitudinal dataset (i.e., 9.5 to 23.3%), the PPV also increased (i.e., from 0.226 to 0.295 for 1+ and to 0.367 for the designated cut-point of 2+). Therefore, utilizing NPV and PPV when prevalence is low should be done with caution, given that one would expect a low PPV. Second, other studies utilizing administrative datasets from real world settings to screen for autism have also reported low PPV (e.g., 0.11) on previously validated instruments (52–55).

Service providers who have completed the interRAI ChYMH, ChYMH-DD, or interRAI Early Years assessment can obtain the ASSC results automatically from the software in which the scale is embedded. It is important to emphasize that the results are meant to assist healthcare providers in identifying how to best support each child's care planning needs based on the ASSC score. Thus, the scale is not meant to be used as an automated decision-making system, without any clinical judgment, but in conjunction with all of the other information collected during the assessment process. Lastly, it is important to always consult with the individual child and the family to ensure that their strengths and needs are considered throughout the process.

Subsequent assessment and care planning steps will be determined, in part, by whether the child's ASSC score falls within the upper or lower range. For example, if the child's score falls within the lower range, it is advised that the healthcare clinicians discuss whether the ASSC score is fitting in light of all of the information that has been gathered. However, if the child's score falls within the upper range, it is advised that the healthcare clinicians consider the individual to be at higher risk for having autism and conduct an in-depth evaluation specifically designed for this sub-population. Ultimately, the key advantage of implementing the ASSC scale would be that toddlers, children, and youth with higher levels of risk should be receiving a timelier comprehensive follow-up evaluation compared to those with lower-level risk. Notably, future research will be conducted to assign ASSC scores to ascending risk categories to determine what labels are best utilized for specific scores on the scale (e.g., highly probable), as well as how they relate to clinical referrals and specific actions that are recommended.

It is important to note that diagnosing autism is a complex process and requires multiple steps, such as: (a) reviewing records; (b) interviewing parents, family members, and other caregivers; (c) assessing for core features through interactions with the child to examine social interaction and communication abilities; (d) utilizing ASD-specific diagnostic tools; and (e) conducting a physical examination and additional investigations (20). Given that the ASSC scale is a brief 5-item measure of key signs and symptoms of autism, it should only be used as an initial screening tool as part of routine practice within a population-based sample. In jurisdictions where the ChYMH or ChYMH-DD is done routinely, the ASSC scale is available at no added cost to provide an additional point of evidence that the team can weigh in the decision-making process. Best practice for a diagnosis of autism is often conducted within a team-based approach utilizing a multi-modal assessment process. In many Canadian jurisdictions, an inter-disciplinary or multidisciplinary specialized team comprised of various health care practitioners work collaboratively in an integrated and coordinated fashion to establish an ASD diagnosis (56), as well as consider differential diagnoses and co-occurring conditions (20).

While the ASSC scale provides an initial screening as opposed to a diagnostic tool, it still has significant utility for individuals, families, clinicians, and the system as a whole. For example, this effective screening method could reduce wait times, allowing for more efficient referrals, and thus quicker access to health, social support, and education services. Notably, earlier access to intervention can foster appropriate development in social interactions, communication, and behavior (57). Therefore, improving access to early diagnosis for young children will capitalize on key developmental windows, increasing the effectiveness of interventions, thereby enhancing prognostic outcomes (8).

Families will also benefit from implementation of the ASSC; more specifically, utilizing an existing instrument that is already in use across most mental health agencies can facilitate expedited triaging resulting in a reduction in the waiting period for families. Such an approach would not only reduce the stress level of the family when navigating the service system, but also foster an increased likelihood of more appropriate referrals to specialized autism services. Research has shown that most families experience the “diagnostic odyssey” as overwhelmingly negative due to long waiting periods. For example, Lappé et al. (58) found that these lengthy wait times caused parents to experience feelings of frustration and a profound sense of uncertainty. Each obstacle and delay within the diagnostic journey has the potential to erode the trust the family has in the healthcare system and their willingness to interact with it (59). Furthermore, as parents continue to wait for an assessment and their stress levels increase, they may be more likely to seek alternative non-evidence-based treatments for their child (60).

With respect to clinical utility, the ASSC provides an opportunity for initial screening for children and youth who are referred for mental health services. Consequently, children who are at greater risk of having autism can be identified through this initial screening approach, resulting in reduced cost and time on behalf of the clinician, agency, as well as the client and their family. Additionally, if a child is determined to be at high risk of having autism and is eventually referred for a more comprehensive diagnostic assessment, a substantial amount of background information will already be available from the interRAI assessment, thereby reducing assessor burden. Interestingly, one study examined factors influencing wait times for an ASD diagnosis and found that the most important predictor of assessment duration was the amount of information available in relation to the child prior to the assessment (61), exemplifying positive downstream effects of the interRAI assessment-to-intervention approach. Thus, increased efficiency will improve early identification, prioritization, and triaging, which will improve the referral-assessment-diagnostic and care pathway as a whole.

At the systems level, comprehensive assessment of ASD is associated with more healthcare costs and resources. Proper screening and triaging utilizing the ASSC can aid in more expedient and efficient use of resources. Having said that, use and utility may differ depending on the resources available as well as the challenges of operationalizing services that are more or less precious. For example, trading off wait times for follow-up diagnosis may look different if a child is seeking services in an urban center in comparison to a remote area or developing nation.

In addition to individualized care planning, the ASSC scale can also provide comprehensive, standardized data across large catchment areas, which: (1) enhances early identification of children with possible autism across the system and (2) provides the ability to examine the prevalence of these symptoms across jurisdictions. Furthermore, this streamlined screening method may help decrease disparities in access to diagnostic services, providing more equitable care at the population-level (62). In addition to the potential impact the ASSC scale can have on patient outcomes, it can also be more cost-effective to the healthcare system as a whole (8, 63, 64). Therefore, our initial screening tool has the potential to make a meaningful impact at both the individual and societal level.

While the current study has many notable strengths, it also has some limitations. For example, given that the present study was not conducted within the controlled setting of a rigorous research study, reliance on broad signals related to associations between items and an autism diagnosis is required, thereby limiting psychometric precision. As such, these results suggest that augmented assessment approaches are needed to reduce disparities to enhance early detection (53). To improve screening and diagnostic precision, new and innovative approaches could be developed that integrate the ASSC with video models (65), new technological advancements, as well as machine learning (52, 66, 67).

Another limitation is that the results of the study may not be generalizable to a community-based sample. This is due to the fact that the young persons assessed were receiving services from inpatient and outpatient mental health agencies. Therefore, future research will explore whether the results of the current study are similar when the young persons assessed are comprised of a community sample. Furthermore, future research will also examine the scale's utility at different cut points once further implementation of the instruments are done both nationally and internationally.

Conclusion and Implications

There is an overwhelming call for an evolution of the systems of care built to identify those with ASD, as the current wait time to receive a diagnosis is unacceptable to the individuals and families we serve (20, 22, 68). This critical need necessitates the development of an easily accessible and effective screening method. The ASSC scale provides an initial screen for larger populations being assessed as part of routine practice to help identify children and youth at heightened risk for autism. Overall, the main goal for the development and implementation of the ASSC scale is to harness the power of the existing interRAI assessment system to provide a more streamlined screening and referral process. This approach to screening can contribute to earlier identification and intervention, ultimately leading to better patient outcomes at the individual level, and more effective care pathways at the systems level.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author/s.

Ethics Statement

The studies involving human participants were reviewed and approved by Western University's ethics review board (REB #106415). Written informed consent from the participants' legal guardian/next of kin was not required to participate in this study in accordance with the national legislation and the institutional requirements.

Author Contributions

SS and JP developed the analytical strategy. JP performed the statistical analysis. All authors contributed to the formulation of the ideas presented in the study, provided critical feedback to the manuscript, and were involved in the writing and reviewing of the final manuscript.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The handling editor JF declared a past co-authorship/collaboration with one of the authors SS.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

The authors would like to sincerely thank the children, youth, and their families, along with the service providers within the agencies for their participation in the research.

References

1. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 5th ed. Washington, DC (2013).

Google Scholar

2. Ruzich E, Allison C, Smith P, Watson P, Auyeung B, Ring H, et al. Measuring autistic traits in the general population: a systematic review of the Autism-Spectrum Quotient (AQ) in a nonclinical population sample of 6,900 typical adult males and females. Mol Autism. (2015) 6:2. doi: 10.1186/2040-2392-6-2

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Waye MM, Cheng HY. Genetics and epigenetics of autism: a review. Psychiatry Clin Neurosci. (2018) 72:228–44. doi: 10.1111/pcn.12606

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Zwaigenbaum L, Bauman ML, Stone WL, Yirmiya N, Estes A, Hansen RL, et al. Early identification of autism spectrum disorder: recommendations for practice and research. Pediatrics. (2015) 136(Suppl. 1):S10–40. doi: 10.1542/peds.2014-3667C

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Malvy J, Barthélémy C, Damie D, Lenoir P, Bodier C, Roux S. Behaviour profiles in a population of infants later diagnosed as having autistic disorder. Eur Child Adolesc Psychiatry. (2004) 13:115–22. doi: 10.1007/s00787-004-0374-2

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Mitchell S, Cardy JO, Zwaigenbaum L. Differentiating autism spectrum disorder from other developmental delays in the first two years of life. Dev Disabil Res Rev. (2011) 17:130–40. doi: 10.1002/ddrr.1107

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Matson JL, Shoemaker M. Intellectual disability and its relationship to autism spectrum disorders. Res Dev Disabil. (2009) 30:1107–14. doi: 10.1016/j.ridd.2009.06.003

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Dawson G. Early behavioral intervention, brain plasticity, and the prevention of autism spectrum disorder. Dev Psychopathol. (2008) 20:775–803. doi: 10.1017/S0954579408000370

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Zwaigenbaum L, Penner M. Autism spectrum disorder: advances in diagnosis and evaluation. BMJ. (2018) 361:k1674. doi: 10.1136/bmj.k1674

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Baxter AJ, Brugha TS, Erskine HE, Scheurer RW, Vos T, Scott JG. The epidemiology and global burden of autism spectrum disorders. Psychol Med. (2015) 45:601–13. doi: 10.1017/S003329171400172X

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Christensen DL, Braun KWN, Baio J, Bilder D, Charles J, Constantino JD, et al. Prevalence and characteristics of autism spectrum disorder among children aged 8 years – autism and developmental disabilities monitoring network, 11 sites, United States, (2012). MMWR Surveill Summ. (2018) 65:1–23. doi: 10.15585/mmwr.ss6513a1

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Centers for Disease Control and Prevention. Prevalence of autism spectrum disorder among children aged 8 years – autism and developmental disabilities monitoring network, 11 sites, United States, 2010. MMWR Surveill Summ. (2014) 63:1–21.

Google Scholar

13. Perry A, Cummings A, Geier JD, Freeman NL, Hughes S, Managhan T, et al. Predictors of outcome for children receiving intensive behavioral intervention in a large, community-based program. Res Autism Spectr Disord. (2011) 5:592–603. doi: 10.1016/j.rasd.2010.07.003

CrossRef Full Text | Google Scholar

14. Constantino JD, Charman T. Diagnosis of autism spectrum disorder: reconciling the syndrome, its diverse origins, and variation in expression. Lancet Neuro. (2016) 15:279–91. doi: 10.1016/S1474-4422(15)00151-9

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Pickles A, Le Couteur A, Leadbitter K, Salomone E, Cole-Fletcher R, Tobin H, et al. Parent-mediated social communication therapy for young children with autism (PACT): long-term follow-up of a randomised controlled trial. Lancet. (2016) 388:2501–09. doi: 10.1016/S0140-6736(16)31229-6

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Howlin P, Moore A. Diagnosis in autism: a survey of over 1200 patients in the UK. Autism. (1997) 1:135–62. doi: 10.1177/1362361397012003

CrossRef Full Text | Google Scholar

17. Moh TA, Magiati I. Factors associated with parental stress and satisfaction during the process of diagnosis of children with Autism Spectrum Disorders. Res Autism Spectr Disord. (2012) 6:293–303. doi: 10.1016/j.rasd.2011.05.011

CrossRef Full Text | Google Scholar

18. Mansell W, Morris K. A survey of parents' reactions to the diagnosis of an autistic spectrum disorder by a local service: access to information and use of services. Autism. (2004) 8:387–407. doi: 10.1177/1362361304045213

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Osborne LA, Reed P. Parents' perceptions of communication with professionals during the diagnosis of autism. Autism. (2008) 12:309–24. doi: 10.1177/1362361307089517

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Brian JA, Zwaigenbaum L, Ip A. Standards of diagnostic assessment for autism spectrum disorder. Paediatr Child Health. (2019) 24:444–51. doi: 10.1093/pch/pxz117

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Crane L, Chester JW, Goddard L, Henry LA, Hill E. Experiences of autism diagnosis: a survey of over 1000 parents in the United Kingdom. Autism. (2016) 20:153–62. doi: 10.1177/1362361315573636

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Gordon-Lipkin E, Foster J, Peacock G. Whittling down the wait time: exploring models to minimize the delay from initial concern to diagnosis and treatment of autism spectrum disorder. Pediatr Clin North Am. (2016) 63:851–59. doi: 10.1016/j.pcl.2016.06.007

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Rutherford M, McKenzie K, Forsyth K, McCartney D, O'Hare A, McClure I, et al. Why are they waiting? Exploring professional perspectives and developing solutions to delayed diagnosis of autism spectrum disorder in adults and children. Res Autism Spectr Disord. (2016) 31:53–65. doi: 10.1016/j.rasd.2016.06.004

CrossRef Full Text | Google Scholar

24. Monteiro SA, Spinks-Franklin A, Treadwell-Deering D, Berry L, Sellers-Vinson S, Smith E, et al. Prevalence of autism spectrum disorder in children referred for diagnostic autism evaluation. Clin Pediatr. (2015) 54:1322–27. doi: 10.1177/0009922815592607

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Thabtah F, Peebles D. Early autism screening: a comprehensive review. Int J Environ Res Public Health. (2019) 16:3502. doi: 10.3390/ijerph16183502

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Stewart SL, Hirdes JP, Curtin-Telegdi N, Perlman C, MacLeod K, Ninan A, et al. interRAI Child and Youth Mental Health (ChYMH) Assessment Form and User's Manual. Version 9.3. Washington, DC: interRAI (2015).

Google Scholar

27. Stewart SL, LaRose L, Gleason K, Nicolson R, McKnight M, Knott W, et al. interRAI Child and Youth Mental Health — Developmental Disabilities (ChYMH-DD) Assessment Form and User's Manual. Version 1. Washington, DC: interRAI (2015).

Google Scholar

28. Stewart SL, Iantosca JM, Klassen JA, Cloutier S, Tucker M, Fisman S, et al. interRAI Early Years Assessment Form and User's Manual. Version 10.0. Washington, DC: interRAI: in pilot.

Google Scholar

29. Stewart SL, Theall LA, Morris JN, Berg K, Björkgren M, Declercq A, et al. interRAI Child and Youth Mental Health Collaborative Action Plans (CAPs) for Use With the interRAI Child and Youth Mental Health (ChYMH) Assessment Instrument. Version 9.3, Standard Edition. Washington, DC: interRAI (2015).

Google Scholar

30. Stewart SL, Theall LA, Morris JN, Berg K, Björkgren M, Declercq A, et al. interRAI Child and Youth Mental Health Collaborative Action Plans (CAPs) for Use With the interRAI Child and Youth Mental Health Developmental Disability (ChYMH-DD) Assessment Instrument. Version 1, Standard Edition. Washington, DC: interRAI (2016).

Google Scholar

31. Arbeau K, Stewart SL, Fisman S, Neufeld E, Rabinowitz T, Theall L, et al. Suicidality and purposeful self-harm. In: Stewart SL, Theall LA, Morris JN, Berg K, Björkgren M, Declercq A, et al., editors. interRAI (CAPs) for Use With the interRAI (ChYMH) Assessment Instrument, Research Version 1 Standard Edition. Washington, DC: interRAI (2015). p. 54–62.

Google Scholar

32. Stewart SL, Kam C, Marshman ME, Leschied AW, Perlman CM, Brown GP, et al. Harm to others. In: Stewart SL, Theall LA, Morris JN, Berg K, Björkgren M, Declercq A, et al. interRAI (CAPs) for Use With the interRAI (ChYMH) Assessment Instrument, Research Version 1 Standard Edition. Washington, DC: interRAI (2015). p. 31–8.

Google Scholar

33. Stewart SL, Hirdes JP, McKnight M, Curtin-Telegdi N, Perlman CM, MacLeod K, et al. interRAI Child and Youth Mental Health Screener (ChYMH-S) Assessment Form and User's Manual. Version 9.3. Washington, DC: interRAI (2017).

Google Scholar

34. Stewart SL, Poss JW, Thornley E, Hirdes JP. Resource intensity for children and youth: the development of an algorithm to identify high service users in children's mental health. Health Serv Insights. (2019) 12:1178632919827930. doi: 10.1177/1178632919827930

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Stewart SL, Celebre A, Hirdes JP, Poss JW. Risk of suicide and self-harm in kids: the development of an algorithm to identify high-risk individuals within the children's mental health system. Child Psychiatry Hum Dev. (2020) 51:913–24. doi: 10.1007/s10578-020-00968-9

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Stewart SL, Morris JN, Asare-Bediako YA, Toohey A. Examining the structure of a new pediatric measure of functional independence using the interRAI Child and Youth Mental Health assessment system. Dev Neurorehabil. (2020) 23:526–33. doi: 10.1080/17518423.2019.1698070

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Stewart SL, Morris JN, Berg K, Björkgren M, Declercq A, Finne-Soveri H, et al. interRAI Early Years Collaborative Action Plans (CAPs) for Use With the interRAI Early Years Assessment Instrument. Version 9.3, Standard Edition. Washington, DC: interRAI (in press).

Google Scholar

38. Lau C, Stewart SL, Saklofske DH, Hirdes J. Development and psychometric validation of the interRAI ChYMH externalizing subscale. Clin Child Psychol Psychiatry. (2021) 26:295–305. doi: 10.1177/1359104520963143

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Lau C, Stewart SL, Saklofske DH, Tremblay PF, Hirdes J. Psychometric evaluation of the interRAI Child and Youth Mental Health Disruptive/Aggression Behaviour Scale (DABS) and Hyperactive/Distraction Scale (HDS). Child Psychiatry Hum Dev. (2018) 49:279–89. doi: 10.1007/s10578-017-0751-y

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Li Y, Babcock SE, Stewart SL, Hirdes JP, Schwean VL. Psychometric evaluation of the Depressive Severity Index (DSI) among children and youth using the interRAI Child and Youth Mental Health (ChYMH) assessment tool. Child Youth Care Forum. 50:611–30. doi: 10.1007/s10566-020-09592-z

CrossRef Full Text | Google Scholar

41. Stewart SL, Babcock SE, Li Y, Dave HP. A psychometric evaluation of the interRAI Child and Youth Mental Health instruments (ChYMH) Anxiety Scale in children with and without developmental disabilities. BMC Psychiatry. (2020) 20:1–14. doi: 10.1186/s12888-020-02785-9

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Stewart SL, Klassen J, Tohver GC. Validation of the interrail risk of school disruption scale: a measurement of school disengagement and disruption. Pers Ind Diff. (2016) 100:518. doi: 10.1016/j.paid.2016.05.311

CrossRef Full Text | Google Scholar

43. Stewart SL, Celebre A, Head MJ, James ML, Martin L, Fries BE. A case-mix system for children and youth with developmental disabilities. Health Serv Insights. (2020) 13:1–7. doi: 10.1177/1178632920977899

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Phillips CD, Patnaik A, Moudouni DK, Naiser E, Dyer JA, Hawes C, et al. Summarizing activity limitations in children with chronic illnesses living in the community: a measurement study of scales using supplemented interRAI items. BMC Health Serv Res. (2012) 12:19. doi: 10.1186/1472-6963-12-19

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Stewart SL, Hamza CA. The Child and Youth Mental Health assessment (ChYMH): an examination of the psychometric properties of an integrated assessment developed for clinically referred children and youth. BMC Health Serv Res. (2017) 17:82. doi: 10.1186/s12913-016-1970-9

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Stewart SL, Hirdes JP. Identifying mental health symptoms in children and youth in residential and in-patient care settings. Healthc Manage Forum. (2015) 28:150–6. doi: 10.1177/0840470415581240

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Hosmer DW, Lemeshow S. Applied Logistic Regression. 2nd ed. New York, NY: John Wiley & Sons (2000).

Google Scholar

48. De Giacomo A, Fombonne E. Parental recognition of developmental abnormalities in autism. Eur Child Adolesc Psychiatry. (1998) 7:131–6. doi: 10.1007/s007870050058

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Young RL, Brewer N, Pattison C. Parental identification of early behavioural abnormalities in children with autistic disorder. Autism. (2003) 7:125–43. doi: 10.1177/1362361303007002002

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Baranek GT. Autism during infancy: a retrospective video analysis of sensory-motor and social behaviors at 9–12 months of age. J Autism Dev Disord. (1999) 29:213–24.

Google Scholar

51. Maestro S, Muratori F, Cavallaro MC, Pei F, Stern D, Golse B, et al. Attentional skills during the first 6 months of age in autism spectrum disorder. J Am Acad Child Adolesc Psychiatry. (2002) 41:1239–45. doi: 10.1097/00004583-200210000-00014

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Guthrie W, Wallis K, Bennett A, Brooks E, Dudley J, Gerdes M, et al. Accuracy of autism screening in a large pediatric network. Pediatrics. (2019) 144:e20183963. doi: 10.1542/peds.2018-3963

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Kim SH, Joseph RM, Frazier JA, O'Shea TM, Chawarska K, Allred EN, et al. Predictive validity of the Modified Checklist for Autism in Toddlers (M-CHAT) born very preterm. J Pediatr. (2016) 178:101–7. doi: 10.1016/j.jpeds.2016.07.052

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Stenberg N, Bresnahan M, Gunnes N, Hirtz D, Hornig M, Lie KK, et al. Identifying children with autism spectrum disorder at 18 months in a general population sample. Paediatr Perinat Epidemiol. (2014) 28:255–62. doi: 10.1111/ppe.12114

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Toh TH, Tan VW, Lau PS, Kiyu A. Accuracy of Modified Checklist for Autism in Toddlers (M-CHAT) in detecting autism and other developmental disorders in community clinics. J Autism Dev Disord. (2018) 48:28–35. doi: 10.1007/s10803-017-3287-x

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Penner M, Anagnostou E, Ungar WJ. Practice patterns and determinants of wait time for autism spectrum disorder diagnosis in Canada. Mol Autism. (2018) 9:1–3. doi: 10.1186/s13229-018-0201-0

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Thabtah F. An accessible and efficient autism screening method for behavioural data and predictive analyses. Health Informatics J. (2019) 25:1739–55. doi: 10.1177/1460458218796636

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Lappé M, Lau L, Dudovitz RN, Nelson BB, Karp EA, Kuo AA. The diagnostic odyssey of autism spectrum disorder. Pediatrics. (2018) 141(Suppl. 4):S272–9. doi: 10.1542/peds.2016-4300C

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Harrington JW, Patrick PA, Edwards KS, Brand DA. Parental beliefs about autism: implications for the treating physician. Autism. (2006) 10:452–62. doi: 10.1177/1362361306066609

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Harrington JW, Rosen L, Garnecho A, Patrick PA. Parental perceptions and use of complementary and alternative medicine practices for children with autistic spectrum disorders in private practice. J Dev Behav Pediatr. (2006) 27(Suppl. 2):S156–61. doi: 10.1097/00004703-200604002-00014

PubMed Abstract | CrossRef Full Text | Google Scholar

61. McKenzie K, Forsyth K, O'Hare A, McClure I, Rutherford M, Murray A, et al. Factors influencing waiting times for diagnosis of autism spectrum disorder in children and adults. Res Dev Disabil. (2015) 45–46:300–6. doi: 10.1016/j.ridd.2015.07.033

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Herlihy LE, Brooks B, Dumont-Mathieu T, Barton ML, Fein D, Chen C, et al. Standardized screening facilitates timely diagnosis of autism spectrum disorders in a diverse sample of low-risk toddlers. J Dev Behav Pediatr. (2014) 35:85–92. doi: 10.1097/DBP.0000000000000014

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Penner M, Rayar M, Bashir N, Roberts SW, Hancock-Howard R, Coyte PC. Cost-effectiveness analysis comparing pre-diagnosis autism spectrum disorder (ASD)-targeted intervention with Ontario's autism intervention program. J Autism Dev Disord. (2015) 45:2833–47. doi: 10.1007/s10803-015-2447-0

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Peters-Scheffer N, Didden R, Korzilius H, Matson J. Cost comparison of early intensive behavioral intervention and treatment as usual for children with autism spectrum disorder in the Netherlands. Res Dev Disabil. (2012) 33:1763–72. doi: 10.1016/j.ridd.2012.04.006

PubMed Abstract | CrossRef Full Text | Google Scholar

65. Janvier YM, Coffield CN, Harris JF, Mandell DS, Cidav Z. The developmental check-in: development and initial testing of an autism screening tool targeting young children from underserved communities. Autism. (2019) 23:689–98. doi: 10.1177/1362361318770430

PubMed Abstract | CrossRef Full Text | Google Scholar

66. Kanne SM, Carpenter LA, Warren Z. Screening in toddlers and preschoolers at risk for autism spectrum disorder: evaluating a novel mobile-health screening tool. Autism Res. (2018) 11:1038–49. doi: 10.1002/aur.1959

PubMed Abstract | CrossRef Full Text | Google Scholar

67. Campbell K, Carpenter KL, Hashemi J, Espinosa S, Marsan S, Borg JS, et al. Computer vision analysis captures atypical attention in toddlers with autism. Autism. (2019) 23:619–28. doi: 10.1177/1362361318766247

PubMed Abstract | CrossRef Full Text | Google Scholar

68. Smith-Young J, Chafe R, Audas R. “Managing the wait”: parents' experiences in accessing diagnostic and treatment services for children and adolescents diagnosed with autism spectrum disorder. Health Serv Insights. (2020). 13:1178632920902141. doi: 10.1177/1178632920902141

PubMed Abstract | CrossRef Full Text | Google Scholar