The interRAI Child and Youth Suite of Mental Health Assessment Instruments: An Integrated Approach to Mental Health Service Delivery

Various biological, social, psychological, and environmental factors impact children and youth living with mental health problems across their lifespan. To meet the wide-ranging challenges of mental illness, service system integration is needed to improve efficiencies and reduce fragmentation. Unfortunately, the mental health system has been plagued by the lack of coordination across services. There is a general consensus that mental health service delivery must ensure a child or youth’s needs are addressed in a collaborative, coordinated, and seamless manner. A key element to successful integration is the development of a comprehensive standardized screening and assessment system. Numerous assessments have been developed to assess child mental health and functioning, but they typically have a very narrow focus with limited use and utility. Not only does this reduce the ability to take a life course perspective to mental health, but this uncoordinated approach also results in redundancies in information collected, additional resources, and increased assessor burden for children, youth, and their families. The interRAI child and youth mental health assessment suite was developed in response to the need for an integrated mental health system for young persons. This suite includes screening and assessment instruments for in-patient and community settings, emergency departments, educational settings, and youth justice custodial facilities. The instruments form a mental health information system intentionally designed to work in an integrated fashion beginning in infancy, and incorporate key applications such as care planning, outcome measurement, resource allocation, and quality improvement. The design of these assessment tools and their psychometric properties are reviewed. Data is then presented using examples related to interpersonal trauma, illustrating the use and utility of the integrated suite, along with the various applications of these assessment systems.

Introduction

Mental health and wellness begins in infancy, as the persistence, stability, and negative long-term consequences of early mental health problems are evident across the lifespan (1). Infants who experience serious adversity in the first few years of life, such as exposure to violence or trauma, are more likely to demonstrate emotional and behavioral problems (2). Young persons with mental health issues are at increased risk for poor academic achievement, underemployment, substance abuse, and impaired social and emotional functioning (3, 4). Additionally, childhood mental illness has been associated with suicide, homicide, criminal activity, as well as institutionalization and incarceration (5, 6). Despite approximately 10–20% of children and youth experiencing a mental health problem, the majority go undiagnosed and untreated (7, 8).

Untreated mental health problems in childhood can persist into adulthood with approximately 70% of adult mental health issues beginning in childhood and adolescence (9). While there has been a substantial amount of research examining the natural progression of child psychopathology, studies exploring the continuity of mental health symptoms across the lifespan are limited, often due to issues regarding appropriate measurement. Some researchers have found that while the overall prevalence of disorders may be comparable, the patterns of specific disorders vary based on age (10–12).

Theoretical frameworks to conceptualize children and youth mental health have been investigated over several decades. A comprehensive approach to understanding children’s mental health is appropriate considering the strengths and needs of young persons are shaped by interactions of intrapersonal and interpersonal factors over time (13). Examples include, but are not limited to, physical health, disability, social relationships with family and peers, and education. Mental health concerns in young persons are becoming increasingly prevalent with approximately 10% of those referred for services presenting with increasing complexity (14, 15). These children account for a significant and disproportionate amount of mental healthcare spending and often require ‘episodic, chronic, and ongoing care’ from multiple service sectors, including costly residential and in-patient services (12, 16–21).

Mental health disorders have a societal and economic impact, with direct and indirect costs exceeding $59 billion annually (22). Given the sequalae of untreated mental health problems, coupled with the corresponding economic costs, it is critical for children, youth, and their families to be identified early and have access to timely and integrated services (23, 24). To improve the responsivity of mental health services, it is imperative to adopt an orientation addressing all aspects implicit in shaping a child’s educational, physical, socio-emotional, and developmental well-being (13)¹.

Early identification of mental health needs is essential, as it fosters early intervention and prevention, thereby reducing the need for intensive resources and crisis services (26, 27). Investments made before birth have a higher return, highlighting that prevention is more cost effective than later remediation (28, 29). Early identification and intervention have the potential to decrease impairment, diminish distress, and reduce the number of young persons who require treatment services later in life. As a result, these children and youth are afforded a greater opportunity to be productive contributors within their communities.

The mental health system has been plagued by high fragmentation and low funding, particularly in children’s mental health (26). At the jurisdictional level, mental health programs and services are typically delivered with little coordination among governmental agencies (8, 30). As the number of service providers involved increases, it becomes more difficult to coordinate services due to a lack of communication and integration (21). Tragically, these compounding factors create steep barriers for families to access appropriate mental health services (31). To alleviate some of the stress experienced by families navigating the system, the development of an integrated standardized mental health assessment system that uses a common language linking community agencies, hospitals, child welfare, youth justice facilities, and educational settings while providing opportunities for transitional care into adulthood is needed (32).

In response to the call for an effective mental health information system for adults, interRAI has developed a number of assessment and screening instruments [e.g., (33)]. For an in-depth review of the adult mental health suite of instruments, please see our previously published work (34). Despite advances within the adult system, children’s mental health has continued to lack a harmonized approach to service delivery, in part, due to the lack of psychometrically sound instruments spanning service sectors. While various assessments have been developed to assess child mental health and functioning, these measures typically have a narrow focus with limited utility. As a result, multiple assessments are often required to evaluate the child or youth’s strengths and needs in various settings and to capture the wide array of features typically associated with the presenting problem (35). This uncoordinated approach to assessment often results in redundancies in information collected, additional resources, and increased assessor burden on the child/youth and their families (35, 36). Critically, previous assessment measures have yet to create a coordinated, cross-sector approach to children’s mental healthcare, or facilitate a lifespan approach to mental health service delivery.

This paper provides an overview of the interRAI child and youth suite of mental health assessment instruments, which was developed in response to the need for an integrated mental health system for young persons. These instruments were designed to facilitate a standardized, comprehensive, and coordinated approach to the delivery of mental health services for infants, children, and youth, ultimately supporting a life course approach to assessment. The paper begins with a discussion of the various factors that must be considered to support a person-centered approach to care. The remainder of the paper describes the design, psychometric properties, and applications of interRAI child/youth mental health assessments using examples related to interpersonal trauma.

Physical Health and Disability

There is a bi-directional relationship between mental and physical health problems. Particularly, children with chronic health conditions are more likely to have mental health issues, and those with mental health problems are more likely to experience physical health problems (37, 38). For example, a large genomic cohort study revealed young persons with physical health problems (i.e., an autoimmune/inflammatory or central-nervous system condition) were more likely to be diagnosed with a mood or neurodevelopmental disorder (38). Furthermore, children with a serious mental health problem were 13% more likely to have a chronic health condition compared to children with no mental health problem (37). While research suggests that comorbid mental and physical health problems persist into adulthood, some studies have found that interventions targeting mental health issues in childhood improve physical health outcomes (39, 40). Taken altogether, the increase in disability and severity in children and youth with comorbid mental and physical health problems suggests the need for a comprehensive mental health assessment and greater coordination between all service providers, particularly medical and mental health professionals, in order to successfully address this “hidden morbidity” (41, 42).

Family and Social Relationships

There is also a reciprocal relationship between family dynamics and children’s mental health. Specifically, a child or youth’s mental health issue might play a role in family conflict with caregivers and siblings, potentially contributing to the development of ineffective or inappropriate parenting techniques and detrimental family interactions (43). In turn, high family conflict and poor parenting practices can worsen the child’s mental health issue (44–47).

Parenting practices have a direct impact on attachment in young children. Attachment to a primary caregiver is a key developmental goal in early childhood for survival, safety, and security. However, disruptions in attachment behavioral patterns can have a detrimental impact on the physical and mental health of children (e.g., brain development and long-term functioning) (48–50). Earlier assessment and intervention of attachment issues can significantly reduce the negative sequalae associated with these relationship difficulties and improve overall socio-emotional functioning. Family financial security and socioeconomic status can also have a significant impact on mental health outcomes. For example, a systematic review examining the relationship between socioeconomic inequalities and mental health problems in children and adolescents reported that low parental education and household income had a greater impact on mental health problems compared to parental unemployment or low occupational status (51).

With respect to social relationships, children who struggle in their peer relations often experience mental health issues in adulthood (52). A longitudinal study reported predictive links between early social isolation, poor peer acceptance, and perceptions of social incompetence with subsequent internalizing problems, while early aggression and peer rejection were predictive of subsequent externalizing problems (53–56). Social support can mitigate the negative impact of mental illness as peer connectedness may be a strong protective factor against anxiety, depression, and suicide, thereby bolstering positive self-esteem and general mental well-being (57). Assessment of factors in relation to parenting, family relationships, attachment, financial security, as well as social relations are all crucial in identifying the strategies for effective intervention and treatment. Interactions between social and familial factors often occur simultaneously, creating a more complex approach to address assessment and system needs (58).

Substance Use

Children and youth with poor family and social relationships are at greater risk for substance use (59, 60). Substance use among young people is a significant public health concern, as it is one of the most commonly cited reasons for admission into a mental health setting (61). The overall prevalence of co-occurring problems in adolescents is just under 3%, with young people between the ages of 15–24 years being more likely to experience substance use disorders (SUD) and/or mental illness compared to any other age group (62, 63). Studies have shown that individuals who suffer from mental health or addiction issues are more likely to die prematurely, with mental illness decreasing a person’s life expectancy by 10–20 years (64).

There are a variety of elements influencing substance use among children and youth, such as peer pressure, trauma, an insecure attachment with caregivers, as well as the presence of a mental health problem (65–68). Youth with both substance use and mental health problems are at risk of serious short- and long-term consequences. For instance, youth with co-occurring disorders are more likely to have impaired functioning, a history of criminal activity, and are typically less responsive to treatment (6, 69–71).

The relationship between substance use and mental illness is a complex one. While numerous theories have been proposed over the years attempting to explain this comorbidity, such as self-medication, the reverse-causal pathway, and shared vulnerability, there is ultimately no simple cause-effect relationship (72). Thus, comprehensive assessment is needed to obtain a better understanding of the interplay between substance use and mental health symptoms, in order to provide effective treatment. Furthermore, many of these youth receive services across multiple service sectors, and so, they require enhanced planning to support transitions across systems of care.

Transitions

Due to transitions being a lengthy and complex process, it is unsurprising that up to 60% of youth in mental health services lose access to treatment as they transition to adult services (73). Unfortunately, youth who present with a number of complex needs, cultural differences, and/or general distrust of the healthcare system are at greatest risk for ineffective transitions, and therefore, are most likely to experience gaps in their care (74–76).

There are numerous barriers to successful transitions such as the lack of communication between service providers, long wait lists, lack of a common language across sectors, and differing levels of involvement for families in decision-making processes (77, 78). Even small transitions can be difficult for young persons who have difficulty adapting, with particularly challenging transitions across educational placements and service transitions (76). Children and youth who do not have the needed support systems in place often require additional resources to support their care. Because transitions often co-occur with changes in professional relationships, it is quite common for youth to experience feelings of abandonment from their service providers during this time (79, 80). By recognizing these issues and bridging service gaps, positive transitions to support proper care planning can be facilitated, thereby increasing the chances of successful outcomes (76).

Education

A greater focus on mental health functioning in educational settings may promote learning and prevent the adverse outcomes linked to untreated mental health issues (81). Studies have reported that learning problems predict anxiety, depression, and substance use during adolescence (82–84). In addition, children and youth with mental health concerns are at increased risk of academic struggles (85–87). When children have both mental health and academic problems, they are at greater risk of negative distal outcomes, including a higher likelihood of mental health service utilization, poor academic achievement, special education placement, and school disengagement (88). Notably, over 50% of students (14 years of age or above) who have emotional and behavioral difficulties drop out of high school (89). Overall, mental health problems are significant barriers to learning and school success, and positive behavioral and emotional health are related to academic achievement (90, 91). Therefore, early identification of mental health concerns among students is of significant importance to ensure appropriate access to community resources, effectively providing young persons with a continuum of care.

Youth Justice

Youth involved in the juvenile justice system experience high rates of mental health problems and trauma (92–94). Approximately 65% of justice-involved youth meet criteria for a mental health disorder, compared to 10–25% of youth in the general population (93). Furthermore, justice-involved youth experience a high level of comorbidity and are 5-times more likely to have one or more mental health disorders compared to the community sample (95).

There is substantive overlap between the mental health needs of youth receiving mental health services and those involved with the justice system (6). Unfortunately, many young persons with mental health concerns are often directed toward the juvenile justice pathway, particularly those with oppositional defiant disorder (ODD), conduct disorder (CD), and substance use issues (96, 97). Ultimately, many justice-involved youth do not receive the mental health care they need – a problem exacerbated by the incarceration of juveniles in adult prisons, which do not provide mental health services designed for youth.

Traumatic Life Events

Traumatic life events can have debilitating lifelong consequences for children and youth (98, 99). The Adverse Childhood Experiences (ACE) Study examined the impact of family dysfunction and childhood abuse on numerous health outcomes later in life and found that exposure to traumatic events in childhood was strongly associated with mental and physical health problems in adulthood (e.g., depression and heart disease) (100–102).

Traumatic events can be categorized as interpersonal or non-interpersonal in nature (103). Interpersonal traumatic events include those that are directly “human induced [and] involve a malicious perpetrator, one who consciously intends to harm another human being,” (e.g., sexual or physical abuse) (p. 2502) (104). In contrast, non-interpersonal traumatic events lack a malicious perpetrator (e.g., car accidents), or the effects of a malicious perpetrator indirectly impact the individual (e.g., being the victim of crime) (104). Children and youth who have experienced interpersonal trauma are more likely to have internalizing and externalizing problems, such as depression and aggression (105), as well as poor self-image, difficulties in self-regulation, and increased risk of self-harm (106, 107). Overall, interpersonal trauma has a negative effect on psychological, social, and emotional well-being.

Extant research has shown that chronic activation of a child’s stress response system can have an adverse impact on a wide-range of aspects of typical development, such as the development of secure attachments and higher order cognitive skills (108, 109). Notably, many children who experience traumatic events often have a diagnosable mental health disorder, namely Attention Deficit/Hyperactivity Disorder (ADHD), anxiety disorders, ODD, reactive attachment disorder (RAD), and communication disorders (108). Due to the plethora of adverse consequences of traumatic life events, comprehensive assessment is needed to understand how trauma relates to mental health concerns across the lifespan.

The interRAI Child and Youth Suite of Mental Health Instruments: An Integrated Assessment System

interRAI is a not-for-profit collaborative of over 100 clinicians, researchers, and policy experts from over 35 countries. Although its initial focus in the early 1990s was on geriatrics, the scope of interRAI’s research broadened to include vulnerable persons with complex needs of all ages, including infants, toddlers, and school-aged children.

interRAI’s primary aim is to improve quality of life and care through the development and implementation of an integrated suite of comprehensive assessment, screening, and care planning systems that span the continuum of care. The development of the child/youth suite of instruments in the last decade currently comprises 11 comprehensive assessments, screeners, and supplements (6 finalized, 5 in final stages of refinement). The following section provides a brief description of the child/youth suite using interpersonal trauma as a common clinical theme to show how these instruments provide insights into a young person’s preferences, strengths, and needs.

Before outlining the assessment and screening instruments within the child and youth suite, the paper will define some important key terms. An “infant” refers to an individual from birth to 12 months of age; a “toddler” refers to an individual over 12 months (12 months and 1 day) to 3 years of age; and a “preschooler” refers to an individual 3 years and 1 day to 3 years and 11 months. Furthermore, a “child” refers to an individual over 4 years (4 years and 1 day) to 12 years of age; and a “youth” refers to an individual 12 years and 1 day to 21 years of age.

Assessment and Screening Instruments for Children and Youth

Published Assessment Tools

interRAI Child and Youth Mental Health Instrument and Adolescent Supplement

The interRAI Child and Youth Mental Health Instrument (ChYMH) is the primary tool in the child and youth suite of instruments (110). It is used in mental health settings at admission, discharge (if more than 7 days after admission), every 90 days (for longer term patient stays), or when there is a clinically significant change in the young person’s status that potentially requires care plan modifications. The target audience is children and youth between the ages of 4–18 years. The ChYMH is intended to support quality indicators, outcome measurement, case-mix classification, and comprehensive care planning in 30 domains, including social and peer relationships, education, traumatic life events, sleep disturbance, and life skills.

The ChYMH has 31 summary scales (e.g., externalizing and internalizing) and four algorithms (i.e., assessments of harm to self, harm to others, caregiver distress, and resource intensity) embedded within it to support symptom monitoring and care planning. The tool uses specific look-back periods to provide valid and reliable measures of clinical characteristics that represent the young person’s preferences, strengths, and needs. Most items employ a 3-day observation period; however, some items use a 7-day, 30-day, or 90-day window, or lifetime estimates depending on the nature of the issue. Notably, some items address the frequency and/or recency of symptoms prior to and within the last 3 days. The tool also provides the option to indicate that symptoms are present but not exhibited within the last 3 days. An example item from the ChYMH that falls under the ‘Mental State Indicators: Mood Disturbance’ section is “sad, pained, or worried facial expression.” This item can be coded from 0 (not present) to 4 (exhibited daily in last 3 days, 3 or more episodes or continuously). The instrument is typically completed by mental health professionals, such as social workers or nurses, and takes approximately 1 h to complete (dependent on case complexity).

The ChYMH assessment has two versions available, the in-patient and community-based forms (see Table 1). The in-patient version is used for young persons residing in a psychiatric unit/facility or residential facility, whereas the community-based version is used for young persons who reside within the community. Importantly, in the event that a child or youth is transferred from an in-patient to community-based setting, the discharge assessment is shared to support continuity of care. Finally, both versions of the ChYMH include an Adolescent Supplement, which is completed for children and youth who are 12 years of age or above. The supplement can be used for children below 12 years of age if they are engaging in mature behaviors such as substance use or sexual activity.

TABLE 1

Table 1. Item counts by domain area and interRAI mental health system for child/youth populations (ages 0–18).

The ChYMH was piloted in Ontario and has been adopted by 90 children’s agencies within the province. Additionally, several organizations in other provinces (e.g., British Columbia, Newfoundland and Labrador, Quebec, Prince Edward Island, and Nova Scotia) have now implemented the interRAI children’s mental health system. The instrument was published in 2015 and is currently available in both English and Canadian-French. To date, over 20,000 assessments have been completed; notably, this data has been utilized in a number of research studies [e.g., (111–113)].

interRAI Child and Youth Mental Health Instrument for Developmental Disabilities and Adolescent Supplement

The interRAI Child and Youth Mental Health Instrument for Developmental Disabilities (ChYMH-DD) is intended to be used with children and youth 4–21 years-old with intellectual or developmental disabilities [e.g., autism, Down syndrome; (114)]. The ChYMH-DD assessment has two versions available; the in-patient and community-based forms, supporting the same broad range of applications as the ChYMH. The instrument supports care plan development in 23 domains, including accessibility and mobility, injurious behavior, continence, educational support, modified nutrition intake, life skills, and social relations.

The ChYMH-DD inpatient and community-based forms are typically completed by developmental service workers. The tool mirrors the ChYMH with respect to the scales and algorithms embedded within it, its specific observation periods, and approximate time for completion. Further, the ChYMH-DD similarly has an Adolescent Supplement. The ChYMH-DD was piloted within Ontario and Finland and is now standard of care in Newfoundland and Labrador, Prince Edward Island, and Nova Scotia. The instrument was published in 2015 and is currently available in both English and Canadian-French. To date, just over 1,000 assessments have been completed, with several interRAI papers examining this particularly vulnerable population [e.g., (115–117)].

interRAI Child and Youth Mental Health Screener

The interRAI Child and Youth Mental Health Screener (ChYMH-S) is a brief standardized assessment tool that is intended to complement the full ChYMH and ChYMH-DD assessments (118). The ChYMH-S was designed to support decision-making related to triaging, placement, and service urgency for young persons with mental health issues. This basic screening tool has the same target audience as the previously described instruments, namely children and youth between the ages of 4–18 years. Further, the tool can be used in various settings, such as in-patient and out-patient programs and educational environments.

The ChYMH-S is comprised of items largely selected from the ChYMH instrument, with some additional items specific for screening purposes. The tool has similar look-back periods to that of the ChYMH and ChYMH-DD and takes approximately 15–20 min to complete by trained clinical staff. It is important to note that while the ChYMH-S informs immediate care triaging, it is not an alternative to or a replacement for the full ChYMH or ChYMH-DD; more pointedly, it is intended to identify those young persons who are in need of a more comprehensive mental health assessment. Furthermore, the ChYMH-S does not support care plan development. Lastly, the tool has three clinical scales and three algorithms (i.e., assessments of harm to self, harm to others, and service urgency) embedded within it. The algorithm related to service urgency is called the Children’s Algorithm for Mental Health and Psychiatric Services (ChAMhPs). ChAMhPs provides a score ranging from 0–6, with higher scores indicative of more urgent and emergent cases. A score of three or higher indicates a full ChYMH should be completed based on case complexity.

The ChYMH-S was initially piloted within Ontario and China. The tool was published shortly after the comprehensive ChYMH and ChYMH-DD assessments in 2017 and is now used as part of standard of care across most mental health agencies in the Province of Ontario. It is currently available in English, Canadian-French, and Simplified Chinese. Over 80,000 assessments have been completed thus far, with a number of research studies utilizing the ChYMH-S data [e.g., (119–121)].

interRAI Pediatric Home Care

The interRAI Pediatric Home Care Assessment (PEDS-HC) is a clinical tool intended for children and youth between 4–20 years of age who receive home-based medical care due to their complex health needs (122). The instrument is predominantly used to support decision-making with respect to the allocation of funding for children and youth who are medically complex and require home-based services. It supports several wide-ranging applications (e.g., quality indicators and outcome measurement), and employs similar specific observation periods.

The PEDS-HC is comprised of over 400 items with scales, algorithms, and care plans in development. The PEDS-HC was developed in the United States and has been piloted within Ontario. It is currently used in Ontario, Nebraska, Texas, and Maryland. The tool was published in 2014, is currently available in English, and almost 500 assessments have been completed to date.

Self-Reported Quality of Life – Child and Youth Mental Health

The interRAI Self-Reported Quality of Life – Child and Youth Mental Health (QoL-ChYMH) is a self-report survey that assesses the perception of well-being and life satisfaction of young persons 7–18 years of age prior to and after receiving mental health services (123). The tool fosters child and youth engagement and involvement in treatment planning and goal setting, thus providing young persons with an opportunity to impact service delivery. The purpose of the QoL-ChYMH is to identify areas of strengths and needs of young clients, with the ultimate goal of maximizing quality of life.

The QoL-ChYMH is comprised of four major domains, which are then further subdivided into 10 categories based on protective factors and indicators of positive mental health that are well-established in the literature: (1) Basic Needs (living conditions, food, safety, and privacy), (2) Social (friends and activities, respect from others, family), (3) Individual (autonomy, health), and (4) Services (school, treatment). Three harmonized versions of the tool have been developed, including pre-service for in-patient and out-patient programs, post-service for in-patient programs, and post-service for out-patient programs. The QoL-ChYMH can be used in conjunction with the ChYMH and Family Quality of Life tool to gain insight into the young person’s perspective and inform treatment planning. The tool takes around 15 min to complete. Approximately 1,000 assessments have been completed to date, and research studies have begun to utilize the QoL-ChYMH data [e.g., (124)]. Similar to the other instruments, the QoL-ChYMH was piloted in Ontario and has been recommended as best practice by Accreditation Canada.

Family Quality of Life – Child and Youth Mental Health

The interRAI Family Quality of Life – Child and Youth Mental Health (FamQoL) is a survey that assesses the perception of the family’s well-being and life satisfaction prior to and after their child receives mental health services (123). Similar to the self-report tool, the FamQoL assists with the promotion of family engagement by involving an adult family member (preferably the primary caregiver) in the child or youth’s treatment planning. The purpose of the FamQoL is to help service providers identify areas of strengths and difficulties from the family’s perspective, in order to maximize treatment outcomes and quality of life of both the child/youth and their family.

Similar to the QoL-ChYMH, the FamQoL was developed in collaboration with expert clinicians well-versed in family dynamics within the context of children’s mental health, and in line with well-established protective factors and indicators of positive mental health. The FamQoL tool consists of seven domains, namely Safety, Informal Support, Formal Support, Community Interaction and Leisure, Family Relationships and Interactions, Life Circumstances, and Interpersonal Challenges. Two versions of the tool have been developed, including the pre-service and post-service for in-patient/out-patient programs. Importantly, the FamQoL can be used in combination with the ChYMH and QoL-ChYMH to gain insight into the family’s perspective on strengths/needs as well as service satisfaction. This QoL tool takes approximately 10–15 min to complete. The FamQoL was piloted in Ontario and approximately 1,200 assessments have been completed thus far.

Assessments at Pilot Stage

interRAI Youth Justice Custodial Facilities

The interRAI Youth Justice Custodial Facilities (YJCF) is a comprehensive standardized instrument for youth between the ages of 12–18 years who currently reside in custodial facility settings². The YJCF has several wide-ranging applications, including quality indicators, resource allocation, and comprehensive care planning in 26 domains (e.g., criminality prevention, sexual offending, harm to others, self-harm, family functioning). It is available in both English and Canadian-French.

The YJCF utilizes a standard 3-day look back period across several areas. A 3-year Ontario pilot study from 2015–2018 yielded 90 completed assessments and was used to provide descriptive profiles of youth in custody compared to those in mental health settings (6, 70).

interRAI Early Years

The interRAI Early Years is a comprehensive standardized tool that is intended to be used with infants, toddlers, and children between 0–47 months who are demonstrating developmental, emotional, social, or behavioral concerns³. The tool has a similar breadth of scope regarding its applications, with a key one being comprehensive care planning. The 17 care planning protocols triggered by the instrument address issues related to attachment, caregiver distress, nutritional intake, traumatic life events, gross/fine motor, and toilet training readiness. Further, the tool currently has three scales and algorithms, with others in development.

The interRAI Early Years assesses five developmental milestones, namely cognition, socio-emotional development, expressive and receptive language, gross motor, and fine motor skills. The tool has similar specific observation periods as the other instruments previously described. The interRAI Early Years is also designed to be directly compatible with other interRAI assessments for young persons such as the ChYMH and ChYMH-DD. The synchronization among the tools within the child/youth suite sets the foundation for a seamless transition between services for infants, toddlers, and school-aged children (4–18 years). Importantly, service providers can monitor a child’s progress through consistent outcome measurement from infancy to adulthood. The interRAI Early Years was piloted over a 3-year period, from 2016–2019, within Ontario. Notably, over 1,000 assessments have been completed thus far with publication anticipated by 2022.

interRAI Child and Youth Emergency Screener for Psychiatry

The interRAI Child and Youth Emergency Screener for Psychiatry (ESP-CY) evaluates the needs of young persons with mental health issues who present to crisis or emergency services, such as psychiatric emergency departments, general emergency departments, or mobile crisis teams⁴. Like the ChYMH, the ESP-CY is typically completed by mental health professionals, such as nurses and social workers. Designed to inform decision-making related to patient safety, placement, and service utilization, the ESP-CY can also provide important information and valuable insights at the beginning of an in-patient mental health episode.

The ESP-CY is based on, and complements, the full mental health assessments (i.e., ChYMH and ChYMH-DD). Considering the ESP-CY has a different clinical focus, namely an emphasis on patient safety and acute symptoms, the basic time frame is set to the last 24 h unless otherwise indicated. Consequently, some items address the frequency and recency of symptoms prior to and within the last 24 h.

The target audience of the ESP-CY is young persons between 4–18 years-old. The tool can be used in a variety of settings, including both in-patient and community-based services and programs. The average time for completion is 15–20 min, although it can vary depending on the acute nature of the child or youth’s symptoms and availability of other informants, such as family members. The ESP-CY has two care planning protocols and three basic risk appraisal algorithms that assess risk of harm to self, risk of harm to others, and inability to care for self. Importantly, while the instrument informs immediate safety planning, it is not an alternative to or replacement for the full ChYMH or ChYMH-DD assessments. The ESP-CY is now being piloted in Ontario.

interRAI Education

The interRAI Education (EDU) is intended to be used with young persons referred to school-based psychological or mental health services⁵. The tool has the same target audience as the PEDS-HC, which is children and youth between 4–20 years of age. It is designed to support a similar broad range of applications, including case-mix classification, quality indicators, outcome measurement, and comprehensive care planning. Particularly, the tool supports care plan development in 30 domains, such as communication, attention and learning supports, vision and hearing impairment, strengths, classroom management/discipline, and is intended to support school engagement (129–131).

The interRAI EDU essentially provides an assessment of key domains of function, educational needs, mental and physical health, and social support. Certain items on the tool can also identify those students who are at higher risk for specific problems related to well-being, health, and functioning, and may require further evaluation. Importantly, the EDU is compatible with other instruments in the interRAI child/youth suite. The tool also mirrors the ChYMH and ChYMH-DD with respect to specific observation periods (i.e., default set to 3-days), and its Adolescent Supplement. Similar to the interRAI ChYMH and ChYMH-DD, there are several decision-support algorithms within the instrument.

interRAI Adolescent Addictions Supplement

The interRAI Adolescent Addictions Supplement is an ancillary clinical tool that is intended to be used with young persons who struggle with addictive behaviors⁶. The supplement is designed for use in conjunction with the ChYMH and ChYMH-DD. Specific scales and care planning protocols for this supplement are in development. Youth can fall into four different addictive behavior streams to support intervention and treatment planning (i.e., video gaming, gambling, tobacco use, and substance use). The supplement employs the same basic time frame (3 days) unless otherwise indicated.

The Adolescent Addictions Supplement should always be completed for young persons who are receiving treatment for addictive behaviors. When triggered, it is strongly advised to complete it shortly after the full mental health assessment (i.e., preferably the same day). When the supplement is not triggered, it can still be completed based on clinician discretion. A number of items are taken into account when deciding whether the supplement should be completed, including “smokes tobacco daily,” “number of days in last 30 days consumed alcohol to point of intoxication,” “gambled excessively or uncontrollably,” “problem video gaming in last 90 days,” among others.

Data Holdings Utilized to Demonstrate interRAI Applications

Before outlining the factors that make the child/youth suite an integrated system, it is important to provide an overview of the interRAI data holdings that will be used to illustrate concepts in this and subsequent sections. Analytic data for the results presented are drawn from assessments collected in Ontario mental health settings. The interRAI Early Years assessment is used as part of regular clinical assessment practice in 15 mental health agencies serving very young children from October 2016 to August 2020. The interRAI ChYMH-S, ChYMH, and ChYMH-DD are used by 62, 59, and 13 agencies, respectively, as part of regular clinical assessment practice. The ChYMH and ChYMH-DD are drawn from assessments done between September 2012 and August 2020, while ChYMH-S records were completed between April 2014 and August 2020. The YJCF data is drawn from a pilot implementation in 9 agencies between November 2014 and August 2015. Finally, OMHRS is a mandated implementation of the interRAI-MH in all hospital-based adult in-patient psychiatry units. All residents with stays of 72 h or longer are to receive a comprehensive admission assessment, and these were used if the patient was 21 years of age or younger at the time of the assessment. For these six instruments, an encrypted individual level identifier was available to select the first assessment of individuals if there was more than one. Of note, for the baseline and follow-up treatment used in one set of outcomes, pairs of assessments for individuals were selected between 30 and 365 days apart. Please see the Appendix for additional information regarding data holdings.

An Integrated Health Information System

There are a number of key features that make the child/youth suite of instruments an integrated health information system. All these instruments have a primary clinical focus on comprehensive functional assessment of strengths and needs to support care planning and outcome measurement across diverse groups [e.g., (133)]. The intended clinical use is not for diagnosis; rather, the instruments capture existing medical and psychiatric diagnoses. All instruments also have a common conceptual emphasis on care planning protocols. These collaborative action plans (CAPs) use empirically derived triggering algorithms to flag areas of potential need and provide evidence-informed guidelines for engaging youth and their support network in a shared decision-making approach while incorporating individual strengths, preferences, and needs (see text foonote 1, 134).

This integrated system also provides a common language with consistent terminology to define common concepts across all settings (e.g., mental health, youth justice, and education), as well as transitions throughout the lifespan (e.g., from infancy to adulthood), thereby improving the effectiveness of communication. Importantly, items only differ between the child/youth and adult suite when it makes sense to do so from a developmental perspective. Also, some items appear in some, but not all, instruments because they are relevant only in specific developmental stages.

The interRAI system uses standardized data collection methods with a detailed training manual that includes intent, definitions, and coding rules for each item. Assessors typically complete a 2.5-day training for the comprehensive interRAI Child and Youth instruments (e.g., the ChYMH, ChYMH-DD, and interRAI Early Years) and a full day training for the briefer instruments (e.g., ChYMH-S). The trained child/youth mental health professionals include psychologists, nurses, psychiatrists, speech and language therapists, child and youth workers, developmental social service workers, and social workers. All available sources of information are utilized to complete the assessment (i.e., family members, community members, document review, and clinical observations). Utilizing multiple forms of information, the assessments do not use fixed narrative questions, but rather employ a semi-structured interview format, thereby providing assessors flexibility in how data is gathered. The interRAI child/youth suite also has a set of core items shared across tools that allow for population-level analyses of issues that are pertinent to children’s mental health.

Psychometric Properties of interRAI Child and Youth Mental Health Instruments

One of the many benefits of adopting research-based mental health instruments for decision-making when providing services to children, youth and their families is the ability to consistently and accurately measure the constructs of interest. Reliability and validity are two psychometric properties that should be considered upon administration of an instrument and interpretation of the collected data (135). The interRAI child and youth mental health instruments and their associated elements, such as scales and algorithms, have gone through extensive reliability and validity testing to ensure their use across multiple service sectors [e.g., (12, 20, 112, 113, 120, 136–145)].

Reliability

Inter-Rater Reliability

It is imperative that assessments conducted by different trained clinicians result in consistent outcomes. Inter-rater reliability or inter-observer agreement is the consistency of results taken from assessments across trained clinicians (135). interRAI assessments have undergone rigorous inter-rater reliability testing with both children and adults (34, 146). One approach to obtain inter-rater reliability strictly utilizes vignettes, which can result in inflated values; however, interRAI takes a more rigorous approach, in that it is conducted as part of typical clinical practice (34). This is exemplified in the obtainment of inter-rater reliability for our newest instrument, the interRAI Early Years. Here, assessors familiar with the tool independently conducted a review of the case file, collateral contacts, and related information while conducting an assessment. Further, they documented their findings with the young child and their family. Results indicated strong precision between raters with values of ICC = 0.98 ([95% CI, 0.97, 0.99], p < 0.001) for expressive and receptive language scales and ICC = 0.87 ([95% CI, 0.72, 0.94], p < 0.001) for the gross motor scale (146). Similarly, several items within the child and youth instruments are shared across the entire interRAI suite, which have demonstrated strong inter-rater reliability [see (141, 144, 147)].

Item Reliability or Internal Consistency

Another form of reliability is item reliability or internal consistency, which is the extent that items in a single assessment measure the same construct (135). Item reliability is routinely measured by Cronbach’s alpha (135, 148). Within the interRAI suite, this form of reliability has been used to evaluate new scales; moreover, it has also been used to help with quality assurance of the data obtained from widespread implementation (34). Numerous studies have been conducted examining the internal consistency of scales and algorithms derived from the interRAI child and youth suite of instruments (138–141, 144, 146). Table 2 describes several clinical summary scales that are available in the child/youth suite, and Table 3 summarizes the internal consistency results for symptoms and behaviors of interest across various instruments. Almost all the scales meet or exceed an alpha level of 0.70 or 0.80 indicating fair or good/moderately high reliability, respectively (135).

TABLE 2

Table 2. Summary of scales and algorithms in interRAI child/youth mental health instruments.

TABLE 3

Table 3. Internal consistency of clinical scales derived from interRAI child/youth mental health instruments.

Validity

Face and Content Validity

A reliable instrument is necessary, but not sufficient, to prompt the adoption of an assessment system for clinical practice. Validity demonstrates that the assessment measures what it intends to capture, with face or content validity referring to the extent to which the assessment’s items represent the construct being investigated (135). During the development of each assessment tool, interRAI establishes face or content validity through consultation with clinicians and researchers, as well as a thorough review of the current literature (34, 144). For example, as part of the developmental efforts of the internalizing and externalizing scales for the interRAI ChYMH, a panel of experts in the field were tasked with evaluating the content representativeness. These efforts resulted in scales with excellent content validity [e.g., (112, 137)].

The interRAI ChYMH consists of over 400 clinically relevant items that have shown strong face validity in evaluating a child or youth’s strengths, needs, and functioning related to presenting psychiatric, medical, and social issues (110). Strong face and content validity have been demonstrated throughout the test development phase with extensive international feedback. Notably, many items from the ChYMH are shared across other child and youth instruments (i.e., ChYMH-S, ChYMH-DD, YJCF, EDU, PEDS-HC, and Early Years). Additionally, certain items from the child/youth suite are similar to those found in the adult suite to aid with the sharing of information between clinicians in different service sectors and across age groups (34). Such an approach fosters continuity of care across service sectors as well as transitions into adult services. All of the interRAI scales, algorithms, and associated features of the assessment systems are created in a similar, rigorous manner.

Construct Validity

Once the construct has been defined and key items have been selected for inclusion in the assessment tool, it is important to test whether the items are associated in a way that would be expected. There are two forms of construct validity, namely convergent and discriminant validity. Convergent validity examines whether similar factors of a construct are shown to be related to each other, whereas discriminant validity highlights when two dissimilar items are shown to be unrelated (148). Items within the child/youth suite of instruments have demonstrated strong construct validity (112, 136, 138, 139, 141–143).

Lau and colleagues (136) assessed the ability of the Disruptive/Aggressive Behavior Scale (DABS) and Hyperactive/Distraction Scale (HDS) to discriminate between two groups that are known to differ on the elements of interest. This was completed through the comparison of mean scores of DABS and HDS for children and youth with and without disorders of interest based on the Diagnostic and Statistical Manual of Mental Disorders – Fourth Edition (DSM-IV) criteria. Furthermore, comparisons were made between the values of the area under the curves. Results suggested that DABS could differentiate between disruptive behavior disorders (DBD) and ADHD. Similarly, HDS was able to differentiate between an ADHD diagnosis relative to a DBD diagnosis with area under the curve values of over 0.70, indicating good discrimination for the measures. These results coincide with our previous work that found scores on the Social Disengagement Scale, which measures the frequency of symptoms related to anhedonia, were strongly associated with a mood disorder diagnosis (143). Similarly, the Depressive Severity Index (DSI) has been found to be highly related to a mood disorder diagnosis, while the Anxiety Scale has been found to be related to anxiety disorders in children and youth with and without intellectual disabilities (138, 142).

With respect to Activities of Daily Living (ADLs), Stewart and colleagues (113) developed and validated two ADL summary scales for both children and youth with normative intellect and developmental disabilities; both scales have shown strong psychometric properties. Results from this study highlighted that no clear hierarchical structure was observed, particularly when stratified by age. However, children and youth with developmental disabilities exhibited a higher level of dependence in daily tasks. Furthermore, both ADL summary scales had excellent internal consistency. A study conducted by Phillips and colleagues (139) investigated the psychometric properties of two scales measuring the activity limitations of a non-clinical sample of children with chronic illnesses. The results of the study supported discriminant validity with Pearson’s r = −0.0174 (p = 0.46) and −0.025 (p = 0.28) for the PEDS-HC Activities of Daily Living Limitations Scale and the Hands-on Needs Scale, respectively.

Criterion-Related Validity

Criterion-related validity compares the scores from an assessment against a particular outcome; two types include concurrent and predictive validity (135). Concurrent validity demonstrates the correlation between the scores under investigation with scores belonging to another assessment that evaluates the same construct (139). In contrast, predictive validity is based on correlations of scores on one assessment with scores on a criterion measure obtained in the future (135, 148).

Concurrent Validity

Numerous studies suggest that the tools in the child/youth suite have strong concurrent validity (112, 137, 138, 141, 142). A common approach to this type of validity is to compare the assessment tool under investigation with a tool that is recognized as a “gold standard” for the construct of interest (34). For example, Lau and colleagues (112, 137) found that the internalizing and externalizing subscales from the interRAI ChYMH had strong concurrent validity by comparing them with relevant criterion measures from other validated assessment instruments: Beck Youth Inventories, Social Skills Improvement System (SSIS), Child and Adolescent Functional Assessment Scale (CAFAS), Child Behavior Checklist (CBCL), and the Brief Child and Family Phone Interview (BCFPI). The strongest correlations were exhibited with the SSIS internalizing and externalizing behaviors and CBCL internalizing and externalizing scales, respectively, as resultant Pearson’s Rho Bayesian Correlations were greater than 0.60. Additionally, Stewart and Babcock (141) conducted a similar study examining scales on the ChYMH-S (i.e., anxiety, distractibility/hyperactivity, internalizing, and externalizing). Their findings suggested that the ChYMH-S scales were significantly and positively correlated with relevant criterion scales from The Behavior Assessment System for Children, Third Edition (BASC-3). More recently, Li and colleagues (148) examined the Depressive Severity Index against the CBCL’s Internalizing Scale and the BASC-3’s Depression Scale and found evidence supporting the DSI’s concurrent validity with a resultant Pearson’s correlation of r = 0.450, p = < 0.001 and r = 0.613, p < 0.001, respectively.

Predictive Validity

One of the most difficult forms of validity to establish is predictive validity (34). A number of studies have shown that various scales within the interRAI child and youth suite have strong predictive validity (138, 139, 143). For example, data from over 5,000 children and youth placed in adult psychiatric settings in Ontario suggested that the Aggressive Behavior Scale was predictive of multiple control intervention use, such as use of restraints. For each unit increase on the Aggressive Behavior Scale, there was a 54%, 62%, and 34% increase in the odds of having received a mechanical restraint, physical restraint, or been held in seclusion, respectively (143). The Severity of Self-Harm (SOS) scale has been useful in predicting admission for risk of self-harm in youth between the ages of 10–17 years, as well as suicide in inpatient settings several years later. Specifically, Hirdes found that individuals who were considered to be high-risk on the SOS were 6.82-times more likely to die by suicide (unpublished data). Similarly, the DSI has been found to significantly predict self-injurious attempts and suicide intent (138). This form of validity is extremely important as one of the main goals of clinical work is to have a positive influence on the child or youth’s developmental trajectory (34). The child and youth Caregiver Distress algorithm was recently developed and validated to identify factors associated with, and predictive of, new or ongoing distress among caregivers referred for mental health services (149). Utilizing longitudinal data, it was found to predict new or ongoing caregiver distress in parents of treatment-seeking children and youth, providing valuable clinical information to prevent future family breakdown. Additionally, the Risk of Injury to Others (RIO) is a decision-support tool developed and validated in order to identify children and youth at increased risk for violence (150). Findings indicated that it was a strong predictor of harmful behavior toward others, and it also predicted increased likelihood of future aggressive behaviors. These decision-support algorithms can be utilized to support strategic prevention and early intervention efforts for these vulnerable youth to circumvent negative sequelae when many of these features present early in life.

Table 4 also highlights the predictive validity of a selected set of scales within the suite. Here, it is evident that an increase in odds ratios for provisional psychiatric diagnoses is related to higher scale values; this means that individuals who scored higher on a particular scale were more likely to have a provisional diagnosis of the associated disorder. For example, those who scored higher on the Hyperactive/Distraction scale were more likely to have a provisional diagnosis of ADHD. Furthermore, goodness of fit ranged from 0.71 to 0.83 after adjusting for age group and sex, suggesting good/moderately high predictive validity.

TABLE 4

Table 4. Odds ratios (95% CL) for provisional psychiatric diagnoses by associated symptoms.

Use and Utility of the interRAI Child and Youth Suite

Some of the key fundamental measures that are available in all or most of the six instruments previously described in the section outlining our data holdings are summarized in Table 5. Considering the different target ages and clinical needs that these instruments are designed for, the ability to measure and report across populations and in a valid and comparable way is a key feature of this family of instruments. For example, about 1 in 10 individuals assessed with the interRAI Early Years instrument report a family member feeling overwhelmed, and this is approximately 2 in 3 among those assessed with the YJCF or ChYMH-DD instrument. This table also highlights that fewer than 10% of those children/youth assessed with the ChYMH, ChYMH-DD, or ChYMH-S had a lifetime suicide attempt, whereas this number jumps to 22% of those assessed in youth justice and 30% of those assessed in inpatient psychiatry. Experiencing abuse is notably higher in the youth justice sample, particularly physical and emotional abuse. Furthermore, younger persons assessed in inpatient psychiatry had the highest prevalence of sexual abuse, at nearly 1 in 5.

TABLE 5

Table 5. Summary of fundamental measures available across child/youth instruments.

Figure 1A shows lifetime interpersonal trauma, which uses three items that record if the child or youth has been a victim of sexual assault or abuse, physical assault or abuse, or emotional abuse at any time. Figure 1B shows any interpersonal trauma in the last year, to illustrate the extent to which it is not merely the accumulation of more years of exposure that produces increased prevalence with age. As such, both Figures 1A,B show proportions of interpersonal trauma are low among younger ages, and then it generally increases by year. This is evident regardless of which instrument is used, with the exception of younger adults who are assessed with the interRAI-MH as part of the OMHRS system for inpatient psychiatry beds. Here, it is interesting to note that the proportion with interpersonal trauma declines around 17 years of age. Both figure panels also show that young persons assessed with the ChYMH-DD tend to have lower levels of interpersonal trauma, whereas those assessed with the YJCF have exceptionally high levels. It is possible that, due to low communication or verbal skills, children with intellectual disabilities are less likely to disclose such interpersonal trauma. Furthermore, children in regular inpatient units age 13 and younger show more interpersonal trauma, and this is even more apparent when considering the last year only. Finally, Figure 1A highlights that the large number of individuals assessed with the ChYMH-S show lower levels of interpersonal trauma compared to those assessed with the more comprehensive ChYMH; however, this pattern is not similarly seen in Figure 1B.

FIGURE 1

Figure 1. (A) Lifetime trauma: victim of sexual assault or abuse, physical assault or abuse, or emotional abuse at any time. (B) Trauma in the last year: victim of sexual assault or abuse, physical assault or abuse, or emotional abuse within the last year.

Applications of interRAI Instruments

All instruments within the child/youth suite are intended to be used as part of standard clinical practice and they each serve a number of different functions. Given the high prevalence of interpersonal trauma within children’s mental health (i.e., approximately 40% depending on the interRAI instrument used), this construct was utilized to illustrate the differences in applications, specifically across the variety of scales/algorithms, care planning protocols, outcome measurement, and quality indicators.

Care Planning

A unique feature of interRAI assessments is that they integrate a comprehensive evaluation of an individual’s areas of strengths and needs within a series of collaborative action plans. Within each instrument, these care planning guidelines have been developed to inform clinical decision-making. Each CAP is comprised of the same five elements: (1) a description of the clinical issue, (2) goals of care, (3) an overview of the various triggering levels, which are based on certain items from the assessment and are associated with different approaches to care, (4) guidelines to assist with care planning based on international best practice, and (5) additional resources related to that particular clinical issue, including references to peer-reviewed publications.

As noted, specific items within each assessment tool serve as “triggers” for certain CAPs, which are subsequently used to help provide evidence-informed care. It is important to emphasize that the purpose of CAPs is to support needs-based care planning; hence, they are not prescriptive in nature. In addition, it is essential that CAPs are utilized within a client-centered approach to care. This means that they are used to help facilitate shared decision-making. More specifically, the child/youth and their family are an integral part of the discussion with regard to how the main areas of need identified by the CAPs will be addressed moving forward. These CAPs have been extensively reviewed by over 150 experts in their respective fields across at least three continents.

Table 6 shows 12 selected CAPs across the ChYMH, ChYMH-DD, and YJCF. The ChYMH cases show the highest triggering rates for medication adherence, whereas the ChYMH-DD cases show the highest triggering rates for communication, medication review, and strengths. Furthermore, YJCF cases show the highest triggering rates for substance use, hazardous fire risk, self-harm, harm to others, interpersonal conflict, transitions, and traumatic life events. Finally, cases across all three instruments show similar triggering rates for sleep disturbance, which is approximately 40% of assessed individuals.

TABLE 6

Table 6. List of selected CAPs and their triggering rates across three child/youth mental health instruments.

Figure 2A shows the triggering rates for the Suicidality and Purposeful Self-Harm CAP by interpersonal trauma and instrument. It is quite evident that a greater proportion of those who have experienced interpersonal trauma trigger the Suicidality and Purposeful Self-Harm CAP, regardless of instrument/cohort. This link between interpersonal trauma and increased risk of suicidality and self-harm is well-supported by the literature [e.g., (151–153)]. Figure 2B shows the triggering rates for the Harm to Others CAP by interpersonal trauma and instrument. These results show a similar pattern as the Suicidality and Purposeful Self-Harm CAP, whereby a greater proportion of those with a history of interpersonal trauma trigger the Harm to Others CAP across instruments/cohorts. Previous research has also found a relationship between these traumatic experiences and risk of harm to others [e.g., (154–156)].

FIGURE 2

Figure 2. (A) Triggering rates for the Suicidality and Purposeful Self-Harm CAP, by trauma and instrument. (B) Triggering rates for the Harm to Others CAP, by trauma and instrument.

Outcome Measurement

Scales embedded within interRAI instruments help capture the complexity of the areas of need at a given point in time. These scales provide a clinical summary of an individual’s status across key domains (e.g., cognitive functioning, depression) and are automatically calculated upon completion of the assessment. Across all instruments, higher scores indicate greater symptom severity, problems related to functioning, or frequency of occurrences. Overall, interRAI scales are useful in describing the individual’s current level of functioning and supporting care planning; furthermore, when examined longitudinally, they provide insights about response to intervention and changes in strengths and needs over time.

Scales can be utilized to highlight differences in mental health severity across in-patient and out-patient services, and to examine outcomes to determine treatment effectiveness (157). An example within the child/youth suite is the Depressive Severity Index, which sums 5 items to produce a scale from 0 to 15, where higher values indicate greater depressive symptoms. In Figure 3, the DSI outcome scale is used to show higher depressive symptoms among those with a lifetime history of interpersonal trauma, compared to others without such a history, across 5 instruments used in child/youth mental health populations.

FIGURE 3

Figure 3. Percentage distributions of Depressive Severity Index score of 4 or greater, by trauma and instrument.

Quality Indicators

Quality indicators (QIs) are summary measures that can provide a comprehensive understanding of quality of care. While outcome scales can be used to track changes in strengths and needs over time at the individual level, longitudinal data can be compiled to track performance at the population level. QIs serve multiple functions; for example, they can be used by agencies to monitor care and facilitate internal quality improvement. Furthermore, QIs can be used to assist the government with accountability and public awareness. These indicators often look at two main outcomes, namely improvement in symptoms or failure to improve/worsening of symptoms between admission and follow-up.

Figure 4 depicts the proportion of children and youth showing improvement at follow-up across five selected scales, by interpersonal trauma status. Based on the findings, the presence of interpersonal trauma history negatively impacts recovery. Specifically, it is less likely for measures of anxiety, depressive symptoms, externalizing symptoms, risk of self-harm and injury to others to improve at follow-up, compared to those without a history of interpersonal trauma. Furthermore, it is important to note that baseline rates are higher in those who have experienced trauma, which means they have a greater opportunity to improve by at least the threshold points, and yet they do not.

FIGURE 4

Figure 4. Change from initial to follow-up assessment, by scale and trauma.

The interRAI child/youth suite is currently in the process of further developing QIs to reflect changes in patterns of symptom levels and domains of functioning. A more thorough examination of the relationship between individual characteristics and differential outcomes will help determine which sub-populations respond well to certain interventions, compared to those that do not. As a result, this will help facilitate differential triaging and, in turn, effective resource allocation. Finally, it can also help identify areas that would benefit from the development of innovative approaches to intervention, including novel approaches to trauma-informed care.

Resource Allocation

interRAI assessment systems can also be used to inform decisions with respect to resource allocation at the individual and societal level. For example, within the adult suite of instruments, the interRAI-MH instrument was used to develop a level of care framework to support resource allocation. Notably, such a framework or decision support algorithm had not previously been available for pediatric populations. Specifically, within the children’s mental health setting, decisions about resource allocation were often based on unstandardized instruments and subjective interpretations, thereby reducing the likelihood that resources were allocated fairly and effectively. In response to this need, interRAI launched an effort to develop a decision support algorithm for resource allocation among younger populations and created the Resource Intensity for Children and Youth (RIChY) (20).

Resource Intensity for Children and Youth is an empirically based decision-support tool that may be used to inform the nature and intensity of scope of service needs for children and youth needing facility- or community-based services. RIChY is based on item responses on full comprehensive assessments. The RIChY algorithm divides children and youth into three age groups: 7 and under, 8 to 11, and 12 and older. The algorithm provides a score ranging from 0–5 based on levels of need for intensive resources; however, not all ages populate all categories. Specifically, the algorithm ranges from 0–3 for children 7 and under, whereas it ranges from 0–5 for children 8 to 11 years and those 12 and older. Higher scores on the algorithm indicate a greater priority for intensive services.

The child or youth can fall into a given level via a number of pathways that represent different combinations of risk factors, such as intimidation of others or threatened violence, external circumstances (e.g., traumatic life events, family dysfunction, or lack of close friends), and risk of harm to self or others. While the algorithm is a decision-support tool, it is important to emphasize that it is ultimately the responsibility of the clinical team to use professional judgment to decide if the RIChY score accurately reflects the young person’s need for intensive services, given all available information.

Figure 5 shows the distribution of the RIChY scale by interpersonal trauma status across the ChYMH and ChYMH-DD. For both ChYMH and ChYMH-DD cases, expected resource intensity is higher in those with a history of interpersonal trauma compared to those without such a history (i.e., there is a greater proportion in high resource groups, and a lower proportion in low resource groups).

FIGURE 5

Figure 5. Distribution of the Resource Intensity for Children and Youth (RIChY) scale.

Another tool that can be used to inform decisions regarding the allocation of resources within the field of healthcare are case-mix classification systems. Case-mix modeling utilizes information about individual characteristics in conjunction with resource data to create groupings based on similar resource requirements. Hence, case-mix classification systems typically describe the comparative resource needs of different groups, with payment systems then attaching a dollar value to these various case-mix groups. Within the adult mental health sector, a case-mix system was developed and implemented across the Province of Ontario, namely the System for Classification of In-Patient Psychiatry (SCIPP) (34). Unfortunately, most service sectors typically utilize funding models based on a standard rate per client rather than the complexity of their needs. To address this gap within the children’s mental health sector, the Arkansas Division of Developmental Disability Services (ARDDS) collaborated with interRAI to develop a case-mix classification system to inform decisions regarding the allocation of resources for children and youth with intellectual and developmental disabilities (IDD). This effort was part of a system-wide payment reform initiative for Medicaid. As a result of this effort, the interRAI Child and Youth Resource Index (ChYRI) was created (shown in Figure 6).

FIGURE 6

Figure 6. The interRAI Child and Youth Resource Index (ChYRI). Ovals represent terminal ChYRI groups.

Child and Youth Resource Index is an empirically based decision-support tool that can be used to inform resource allocation among young persons with IDD (158). It serves as a useful guideline to aid in decision-making around allocation of resources and planning for appropriate services. The classification system incorporates 8 distinct groups and explains 30% of the variance in per diem costs. A number of variables are included within the system, such as age, motor skills, violence to others, supportive relationship with the family, foster placement, and certain health conditions (e.g., paralysis, bladder continence). There is a 3-to-1 range in case-mix indexes (CMIs) across the groups. Overall, ChYRI can be utilized to improve equitability in the allocation of limited resources within vulnerable populations, centered around stability and fairness.

Future Directions

Over the past decade, the child and youth mental health suite of instruments has made significant contributions within both clinical and research domains. Nevertheless, in order to continue to make a meaningful impact, some of the limitations of the suite need to be addressed. For the suite’s potential to be fully realized, it is important for the leaders of mental health teams to emphasize the clinical utility of the system rather than the aspects related to data collection. It is also important to emphasize the value of effective communication and collaboration across various service sectors and healthcare providers in order to maximize the utility of this integrated system. Additionally, ensuring there is a strong, well-designed software system to support data collection and uptake of the scales, algorithms, and care plans to ensure proper use and utility of the interRAI suite is key.

Notably, the interRAI instruments exemplify a needs-based approach to care, which supports a more equitable approach to service delivery, and consequently, reduces disparities in the provision of mental health services (159). This approach will only become increasingly vital as the social inequities around the world continue to rise, and we are tasked with navigating new global challenges, as evidenced by the COVID-19 pandemic [see (160–162)]. Therefore, the interRAI integrated assessment system is both well-equipped and well-positioned to help inform equitable service delivery to ensure that those who are most vulnerable and in greatest need have increased access to services and receive the appropriate resources in an efficient, effective manner.

The implementation of the interRAI instruments has been highly effective in Ontario with implementation across other provinces currently underway. Future research is needed across Canada as well at the international level. It will be essential to continue to build partnerships with health leaders from different countries, including those from low-, middle-, and high-income nations. Furthermore, the development of additional self-report tools is presently underway, which will be especially useful in serving parts of the world where there are a lack of mental health professionals and significant barriers to accessing mental health services. These instruments will also support new implementation efforts within primary care.

While many studies have already been published examining the validity and reliability of the child/youth mental health instruments, additional research in this area is needed including international studies investigating the areas of predictive validity, criterion validity, and inter-rater reliability. It will also be important to consider developing case-mix systems for in-patient and community mental health instruments for children and youth in general, as the ChYRI is only intended for those with IDD.

Finally, as the child/youth suite of mental health instruments continues to grow and develop, it will create a plethora of new and exciting opportunities that will be transformative in nature. For example, these longitudinal datasets comprised of rich clinical information will create opportunities for applying artificial intelligence tools, thereby expanding possibilities related to the development of novel applications for personalized health care systems. Most critically, a comprehensive mental health assessment system that spans from infancy to adulthood will provide an extraordinary opportunity to examine the impact of mental health and illness on all ages and stages, and develop innovative solutions to help each individual maximize their quality of life.

Author Contributions

SS and JP developed the analytical strategy. JP performed the statistical analysis. All authors contributed to the formulation of the ideas presented in the study and were involved in the writing and reviewing of the final manuscript.

Funding

This research was funded by the Public Health Agency of Canada (#1617-HQ-000050).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

The authors would like to sincerely thank the children, youth, and their families, along with the service providers within the agencies for their participation in the research.

Footnotes

1. ^ Stewart SL, Toohey A. <arttitle>Screening and assessment of mental health problems in students: utilizing an integrated assessment-to-intervention approach to enhance integrated care. In: Andrews J, Shaw S, Domene J, McMorris C editors. School, Clinical, and Counseling Psychology: Mental Health Assessment, Prevention, and Intervention. New York, NY: Springer (under review).
2. ^ Stewart SL, Leschied AW, Hirdes J, Mathias K, Currie M, McKnight M, et al. interRAI Youth Justice Custodial Facilities (YJCF) Assessment Form and User’s Manual. Washington, DC: interRAI (in pilot).
3. ^ Stewart SL, Iantosca JM, Klassen JA, Cloutier S, Tucker M, Fisman S, et al. interRAI Early Years Assessment Form and User’s Manual. Version 10.0. Washington, DC: interRAI (in pilot).
4. ^ Stewart SL, Hirdes J, Morris JN, Berg K, Björkgren M, Declercq A, et al. interRAI Emergency Screener in Psychiatry for Child and Youth (ESP-CY) Assessment Form and User’s Manual. Version 1. Washington, DC: interRAI (in pilot).
5. ^ Stewart SL, McKnight M, Currie M, Gilpin M, Evans B, Richards P, et al. interRAI Education (EDU) Assessment Form and User’s Manual. Version 1. Washington, DC: interRAI (in pilot).
6. ^ Stewart SL, Perlman C, Hirdes J, Curtin-Telegdi N, Berger J, Ferris J, et al. interRAI Adolescent Addictions Supplement to the Child and Youth Mental Health (ChYMH), Child and Youth Mental Health-Developmental Disabilities (ChYMH-DD) and Child and Youth Emergency Screener for Psychiatry (CY-ESP). Version 10. Washington, DC: interRAI (in pilot).

References

1. McLeod GFH, Horwood LJ, Fergusson DM. Adolescent depression, adult mental health and psychosocial outcomes at 30 and 35 years. Psychol Med. (2016) 46:1401–12. doi: 10.1017/S0033291715002950

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Scheeringa MS, Gaensbauer TJ. Posttraumatic stress disorder. 2nd ed. In: Zeanah CH Jr editor. Handbook of Infant Mental Health. New York, NY: The Guilford Press (2000). p. 369–81.

Google Scholar

3. Breslau J, Miller E, Breslau N, Bohnert K, Lucia V, Schweitzer J. The impact of early behavior disturbances on academic achievement in high school. Pediatrics. (2009) 123:1472–6. doi: 10.1542/peds.2008-1406

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Kirby M, Keon WJ. Out of the Shadows at Last: Transforming Mental Health, Mental Illness, and Addiction Service in Canada. Ottawa, ON: Senate of Canada (2006).

Google Scholar

5. Pumariega AJ, Winters NC, Huffine C. The evolution of systems of care for children’s mental health: forty years of community child and adolescent psychiatry. Community Ment Health J. (2003) 39:399–425. doi: 10.1023/a:1025808902438

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Stewart SL, Thornley E, Lapshina N, Erickson P, Vingilis E, Hamilton H, et al. Sex differences in youth with mental health problems in inpatient, outpatient and youth justice settings. BMC Psychiatry. (2020) 20:16. doi: 10.1186/s12888-019-2413-z

CrossRef Full Text | Google Scholar

7. Kessler RC, Angermeyer M, Anthony JC, De Graaf RON, Demyttenaere K, Gasquet I, et al. Lifetime prevalence and age-of-onset distributions of mental disorders in the World Health Organization’s world mental health survey initiative. World Psychiatry. (2007) 6:168–76.

PubMed Abstract | Google Scholar

8. Waddell C, McEwan K, Shepherd CA, Offord DR, Hua JM. A public health strategy to improve the mental health of Canadian children. Can J Psychiatry. (2005) 50:226–33. doi: 10.1177/070674370505000406

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Government of Canada.The Human Face of Mental Health and Mental Illness in Canada. (2006). Available online at: https://www.phac-aspc.gc.ca/publicat/human-humain06/pdf/human_face_e.pdf (accessed November 15, 2020).

Google Scholar

10. Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the national comorbidity survey replication. Arch Gen Psychiatry. (2005) 62:593–602. doi: 10.1001/archpsyc.62.6.593

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Kessler RC, Birnbaum HG, Shahly V, Bromet E, Hwang I, McLaughlin KA, et al. Age differences in the prevalence and co-morbidity of DSM-IV major depressive episodes: results from the WHO world mental health survey initiative. Depress Anxiety. (2010) 27:351–64. doi: 10.1002/da.20634

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Stewart SL, Hirdes JP. Identifying mental health symptoms in children and youth in residential and in-patient care settings. Healthc Manage Forum. (2015) 28:150–6. doi: 10.1177/0840470415581240

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Stiffman AR, Stelk W, Horwitz SM, Evans ME, Outlaw FH, Atkins M. A public health approach to children’s mental health services: possible solutions to current service inadequacies. Adm Policy Ment Health. (2010) 37:120–4. doi: 10.1007/s10488-009-0259-2

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Epstein MH, Kutash KE, Duchnowski AE. Outcomes for Children and Youth with Emotional and Behavioral Disorders and their Families: Programs and Evaluation best Practices. Austin, TX: PRO-ED (1998).

Google Scholar

15. Reid GJ, Cunningham CE, Tobon JI, Evans B, Stewart M, Brown JB, et al. Help-seeking for children with mental health problems: parents’ efforts and experiences. Adm Policy Ment Health. (2011) 38:384–97. doi: 10.1007/s10488-010-0325-9

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Reid GJ, Brown JB. Money, case complexity, and wait lists: perspectives on problems and solutions at children’s mental health centers in Ontario. J Behav Health Serv Res. (2008) 35:334–46. doi: 10.1007/s11414-008-9115-5

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Reid G, Stewart SL, Zaric GS, Carter JR, Neufeld RWJ, Tobon JI, et al. Defining episodes of care in children’s mental health using administrative data. Adm Policy Ment Health. (2015) 42:737–47. doi: 10.1007/s10488-014-0609-6

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Reid GJ, Stewart SL, Barwick M, Carter J, Leschied A, Neufeld RW, et al. Predicting patterns of service utilization within children’s mental health agencies. BMC Health Serv Res. (2019) 19:993. doi: 10.1186/s12913-019-4842-2

CrossRef Full Text | Google Scholar

19. Reid GJ, Stewart SL, Barwick M, Cunningham C, Carter J, Evans B, et al. Exploring patterns of service utilization within children’s mental health agencies. J Child Fam Stud. (2021) 30:556–74. doi: 10.1007/s10826-020-01859-2

CrossRef Full Text | Google Scholar

20. Stewart SL, Poss JW, Thornley E, Hirdes JP. Resource intensity for children and youth: the development of an algorithm to identify high service users in children’s mental health. Health Serv Insights. (2019) 12:1–11. doi: 10.1177/1178632919827930

CrossRef Full Text | Google Scholar

21. Tobon JI, Reid GJ, Brown JB. Continuity of care in children’s mental health: parent, youth and provider perspectives. Community Ment Health J. (2015) 51:921–30. doi: 10.1007/s10597-015-9873-5

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Lim KL, Jacobs P, Ohinmaa A, Schopflocher D, Dewa CS. A new population-based measure of the economic burden of mental illness in Canada. Chronic Dis Can. (2008) 28:92–8. doi: 10.24095/hpcdp.28.3.02

CrossRef Full Text | Google Scholar

23. Merikangas KR, He JP, Burstein M, Swanson SA, Avenevoli S, Cui L, et al. Lifetime prevalence of mental disorders in US adolescents: results from the national comorbidity survey replication–adolescent supplement (NCS-A). J Am Acad Child Adolesc Psychiatry. (2010) 49:980–9.

Google Scholar

24. Smith JP, Smith GC. Long-term economic costs of psychological problems during childhood. Soc Sci Med. (2010) 71:110–5. doi: 10.1016/j.socscimed.2010.02.046

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Kirby M, Keon WJ. Mental Health, Mental Illness, and Addiction: Issues and Options for Canada. Ottawa, ON: Senate of Canada (2004).

Google Scholar

26. Koot HM. Longitudinal studies of general population and community samples. In: Verhulst FC Koot HM editors. The Epidemiology of Child and Adolescent Psychopathology. New York, NY: Oxford University Press (1995).

Google Scholar

27. Doyle O, Harmon CP, Heckman JJ, Tremblay RE. Investing in early human development: timing and economic efficiency. Econ Hum Biol. (2009) 7:1–6. doi: 10.1016/j.ehb.2009.01.002

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Heckman JJ. The economics, technology, and neuroscience of human capability formation. Proc Natl Acad Sci USA. (2007) 104:13250–5. doi: 10.1073/pnas.0701362104

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Health Canada.Celebrating Success: A Self-Regulating Service Delivery System for Children and Youth. Ottawa, ON: Health Canada (2000).

Google Scholar

30. Boydell KM, Pong R, Volpe T, Tilleczek K, Wilson E, Lemieux S. Family perspectives on pathways to mental health care for children and youth in rural communities. J Rural Health. (2006) 22:182–8. doi: 10.1111/j.1748-0361.2006.00029.x

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Schraeder KE, Barwick M, Cairney J, Carter J, Kurdyak P, Neufeld RWJ, et al. Re-accessing mental health care after age 18: a longitudinal cohort study of youth involved with community-based child and youth mental health agencies in Ontario. J Can Acad Child Adolesc Psychiatry. (2021) 30:12–24.

PubMed Abstract | Google Scholar

32. Hirdes JP, Smith TF, Rabinowitz T, Yamauchi K, Pérez E, Telegdi NC, et al. The resident assessment instrument-mental health (RAI-MH): inter-rater reliability and convergent validity. J Behav Health Serv Res. (2002) 29:419–32. doi: 10.1007/BF02287348

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Hirdes JP, van Everdingen C, Ferris J, Franco-Martin M, Fries BE, Heikkilä J, et al. The interRAI suite of mental health assessment instruments: an integrated system for the continuum of care. Front Psychiatry. (2020) 10:926. doi: 10.3389/fpsyt.2019.00926

CrossRef Full Text | Google Scholar

34. Kazdin AE. Evidence-based assessment for children and adolescents: issues in measurement development and clinical application. J Clin Child Adolesc Psychol. (2005) 34:548–58. doi: 10.1207/s15374424jccp3403_10

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Neisworth JT, Bagnato SJ. The mismeasure of young children: the authentic assessment alternative. Infant Young Child. (2004) 17:198–212.

Google Scholar

36. Combs-Orme T, Heflinger CA, Simpkins CG. Comorbidity of mental health problems and chronic health conditions in children. J Emot Behav Disord. (2002) 10:116–25.

Google Scholar

37. Merikangas KR, Calkins ME, Burstein M, He J, Chiavacci R, Lateef T, et al. Comorbidity of physical and mental disorders in the neurodevelopmental genomics cohort study. Pediatrics. (2015) 135:e927–38. doi: 10.1542/peds.2014-1444

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Anderson SE, Cohen P, Naumova EN, Must A. Association of depression and anxiety disorders with weight change in a prospective community-based study of children followed up into adulthood. Arch Pediatr Adolesc Med. (2006) 160:285–91. doi: 10.1001/archpedi.160.3.285

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Anderson SE, Cohen P, Naumova EN, Must A. Relationship of childhood behavior disorders to weight gain from childhood into adulthood. Ambul Pediatr. (2006) 6:297–301. doi: 10.1016/j.ambp.2006.06.002

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Cadman D, Boyle M, Szatmari P, Offord DR. Chronic illness, disability, and mental and social well-being: findings of the Ontario child health study. Pediatrics. (1987) 79:805–13.

PubMed Abstract | Google Scholar

41. Costello EJ, Costello AJ, Edelbrock C, Burns BJ, Dulcan MK, Brent D, et al. Psychiatric disorders in pediatric primary care: prevalence and risk factors. Arch Gen Psychiatry. (1988) 45:1107–16. doi: 10.1001/archpsyc.1988.01800360055008

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Ma N, Roberts R, Winefield H, Furber G. The quality of family relationships for siblings of children with mental health problems: a 20-year systematic review. J Fam Stud. (2017) 23:309–32.

Google Scholar

43. Arbeau A, Stewart SL, Jordan P, Smith T, Theall L, Hirdes JP, et al. Social and peer relationships. In: Stewart SL, Theall LA, Morris JN, Berg K, Björkgren M, Declercq A et al editors. interRAI (CAPs) for use with the interRAI (ChYMH) Assessment Instrument, Research Version 1 Standard Edition. Washington, DC: interRAI (2015).

Google Scholar

44. Arbeau A, Stewart SL, Theall L, DeOliveira CA, Rupert K, Sheppard P, et al. Parenting. In: Stewart SL, Theall LA, Morris JN, Berg K, Björkgren M, Declercq A et al. editors. interRAI (CAPs) for use with the interRAI (ChYMH) Assessment Instrument, Research Version 1 Standard Edition. Washington, DC: interRAI (2015).

Google Scholar

45. Klahr AM, McGue M, Iacono WG, Burt SA. The association between parent–child conflict and adolescent conduct problems over time: results from a longitudinal adoption study. J Abnorm Psychol. (2011) 120:46–56. doi: 10.1037/a0021350

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Stewart SL, Theall L, Marshman ME, Moody K, Martin L. Social relations. In: Stewart SL, Theall LA, Morris JN, Berg K, Björkgren M, Declercq A et al. editors. interRAI (CAPs) for use with the interRAI Developmental Disability (ChYMH-DD) Assessment Instrument, Research Version 1 Standard Edition. Washington, DC: interRAI (2016).

Google Scholar

47. DeOliveira CA, Rupert K, Arbeau K, Stewart SL, Theall LA. Attachment. In: Stewart SL, Theall LA, Morris JN, Berg K, Björkgren M, Declercq A et al. editors. interRAI (CAPs) for use with the interRAI (ChYMH) Assessment Instrument, Research Version 1 Standard Edition. Washington, DC: interRAI (2015).

Google Scholar

48. Kaiser S, Zimmet M, Fraser J, Liddle K, Roberts G. Recognition of attachment difficulties and developmental trauma is the responsibility of all paediatricians. J Paediatr Child Health. (2018) 54:1110–6. doi: 10.1111/jpc.14154

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Zeanah PD, Larrieu JA, Zeanah CH. Training in infant mental health. 2nd ed. In: CH Zeanah <suffix>Jr</suffix> editor. Handbook of Infant Mental Health. New York, NY: The Guilford Press (2000). p. 548–58.

Google Scholar

50. Reiss F. Socioeconomic inequalities and mental health problems in children and adolescents: a systematic review. Soc Sci Med. (2013) 90:24–31. doi: 10.1016/j.socscimed.2013.04.026

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Lacey RE, Kumari M, Bartley M. Social isolation in childhood and adult inflammation: evidence from the national child development study. Psychoneuroendocrinology. (2014) 50:85–94. doi: 10.1016/j.psyneuen.2014.08.007

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Hymel S, Rubin KH, Rowden L, LeMare L. Children’s peer relationships: longitudinal prediction of internalizing and externalizing problems from middle to late childhood. Child Dev. (1990) 61:2004–21.

Google Scholar

53. Rubin KH, Burgess K, Kennedy L, Stewart SL. Social withdrawal in childhood. In: Mash E, Barkley R editors. Child Psychopathology. New York, NY: Guilford Press (2003). p. 372–406.

Google Scholar

54. Rubin KH, Stewart SL. Social withdrawal in childhood. In: Mash E, Barkley R editors. Child Psychopathology. New York, NY: Guilford Press (1996). p. 277–307.

Google Scholar

55. Rubin KH, Stewart SL, Chen X. The parents of aggressive and withdrawn children. In: M Bornstein editor. Handbook of Parenting. Mahwah, NJ: Lawrence Erlbaum Associates, Inc (1995). p. 255–84.

Google Scholar

56. Roach A. Supportive peer relationships and mental health in adolescence: an integrative review. Issues Ment Health Nurs. (2018) 39:723–37. doi: 10.1080/01612840.2018.1496498

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Whitesell M, Bachand A, Peel J, Brown M. Familial, social, and individual factors contributing to risk for adolescent substance use. J Addict. (2013) 2013:1–9. doi: 10.1155/2013/579310

CrossRef Full Text | Google Scholar

58. Van Ryzin MJ, Fosco GM, Dishion TJ. Family and peer predictors of substance use from early adolescence to early adulthood: an 11-year prospective analysis. Addict Behav. (2012) 37:1314–24. doi: 10.1016/j.addbeh.2012.06.020

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Zhou Q, King KM, Chassin L. The roles of familial alcoholism and adolescent family harmony in young adults’ substance dependence disorders: mediated and moderated relations. J Abnorm Psychol. (2006) 115:320–31. doi: 10.1037/0021-843X.115.2.320

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Patel V, Flisher AJ, Hetrick S, McGorry P. Mental health of young people: a global public-health challenge. Lancet. (2007) 369:1302–13. doi: 10.1016/S0140-6736(07)60368-7

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Pearson C, Janz T, Ali J. Mental and Substance use Disorders in Canada. Statistics Canada Catalogue, Health at a Glance. (2013). Available online at: https://www150.statcan.gc.ca/n1/en/pub/82-624-x/2013001/article/11855-eng.pdf?st=6memgq3p (accessed November 16, 2020).

Google Scholar

62. Winstanley EL, Steinwachs DM, Stitzer ML, Fishman MJ. Adolescent substance abuse and mental health: problem co-occurrence and access to services. J Child Adolesc Subst Abuse. (2012) 21:310–22. doi: 10.1080/1067828X.2012.709453

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Chesney E, Goodwin GM, Fazel S. Risks of all-cause and suicide mortality in mental disorders: a meta-review. World Psychiatry. (2014) 13:153–60. doi: 10.1002/wps.20128

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Henderson J, Stewart SL, Theall L, Selby P, Owen-Anderson A, Perlman C, et al. Substance use. In: Stewart SL, Theall LA, Morris JN, Berg K, Björkgren M, Declercq A et al. editors. interRAI (CAPs) for use with the interRAI (ChYMH) Assessment Instrument, Research Version 1 Standard Edition. Washington, DC: interRAI (2015).

Google Scholar

65. Lemyre A, Gauthier-Légaré A, Bélanger RE. Shyness, social anxiety, social anxiety disorder, and substance use among normative adolescent populations: a systematic review. Am J Drug Alcohol Abuse. (2019) 45:230–47. doi: 10.1080/00952990.2018.1536882

PubMed Abstract | CrossRef Full Text | Google Scholar

66. McDonough MH, Jose PE, Stuart J. Bi-directional effects of peer relationships and adolescent substance use: a longitudinal study. J Youth Adolesc. (2016) 45:1652–63. doi: 10.1007/s10964-015-0355-4

PubMed Abstract | CrossRef Full Text | Google Scholar

67. Schindler A. Attachment and substance use disorders – theoretical models, empirical evidence, and implications for treatment. Front Psychiatry. (2019) 10:727. doi: 10.3389/fpsyt.2019.00727

CrossRef Full Text | Google Scholar

68. Abram KM, Teplin LA, McClelland GM, Dulcan MK. Comorbid psychiatric disorders in youth in juvenile detention. Arch Gen Psychiatry. (2003) 60:1097–108. doi: 10.1001/archpsyc.60.11.1097

PubMed Abstract | CrossRef Full Text | Google Scholar

69. Vingilis E, Stewart SL, Hamilton HA, Seeley J, Einarson KM, Kolla NJ, et al. Pilot study of mental health and substance use of detained youths in Ontario, Canada. Child Youth Serv Rev. (2020) 116:105156. doi: 10.1016/j.childyouth.2020.105156

CrossRef Full Text | Google Scholar

70. National Institute on Drug Abuse.Comorbidity: Addiction and other Mental Illnesses. National Institutes of Health, US Department of Health and Human Services. (2010). Available online at: https://www.drugabuse.gov/sites/default/files/rrcomorbidity.pdf (accessed November 16, 2020).

Google Scholar

71. Deady M, Teesson M, Brady KT. Impact of Substance use on the Course of Serious Mental Disorders. San Diego, CA: Elsevier Academic Press (2013).

Google Scholar

72. Davidson SI, Cappelli M. We’ve got Growing up to do: Transitioning Youth from Child and Adolescent Mental Health Services to Adult Mental Health Services. Ontario, ON: Centre of Excellence for Child and Youth Mental Health (2011).

Google Scholar

73. Lugasi T, Achille M, Vaillancourt I, Stevenson M, Bell L, Clermont MJ, et al. The impact of self-determination variables on patients’ adherence to medication and readiness to transition to adult health care. J Adolesc Health. (2009) 44:S34.

Google Scholar

74. Pottick KJ, Bilder S, Vander Stoep A, Warner LA, Alvarez MF. US patterns of mental health service utilization for transition-age youth and young adults. J Behav Health Serv Res. (2008) 35:373–89. doi: 10.1007/s11414-007-9080-4

PubMed Abstract | CrossRef Full Text | Google Scholar

75. Harris B, Targett PS, Sukhera J, Westelaken K, Theall L, Stewart SL, et al. Transitions. In: Stewart SL, Theall LA, Morris JN, Berg K, Björkgren M, Declercq A et al. editors. interRAI (CAPs) for use with the interRAI (ChYMH) Assessment Instrument, Research Version 1 Standard Edition. Washington, DC: interRAI (2015).

Google Scholar

76. Paul M, Street C, Wheeler N, Singh SP. Transition to adult services for young people with mental health needs: a systematic review. Clin Child Psychol Psychiatry. (2015) 20:436–57. doi: 10.1177/1359104514526603

PubMed Abstract | CrossRef Full Text | Google Scholar

77. Randall GE, Mulvale GM, Wakefield PA, Embrett MG, Barr NG, Miatello AM, et al. Mapping the Policy to Practice Landscape for Youth Mental Health in Ontario. “Youth to Adult Transitions in Health Care- The Case of Mental Health Services in Ontario” Research Team. Hamilton, ON: McMaster University (2016).

Google Scholar

78. Broad KL, Sandhu VK, Sunderji N, Charach A. Youth experiences of transition from child mental health services to adult mental health services: a qualitative thematic synthesis. BMC Psychiatry. (2017) 17:380. doi: 10.1186/s12888-017-1538-1

CrossRef Full Text | Google Scholar

79. Singh SP, Tuomainen H. Transition from child to adult mental health services: needs, barriers, experiences and new models of care. World Psychiatry. (2015) 14:358–61. doi: 10.1002/wps.20266

PubMed Abstract | CrossRef Full Text | Google Scholar

80. Ringeisen H, Henderson K, Hoagwood K. Context matters: schools and the “research to practice gap” in children’s mental health. School Psych Rev. (2003) 32):153–68.

Google Scholar

81. Fleming CB, Harachi TW, Cortes RC, Abbott RD, Catalano RF. Level and change in reading scores and attention problems during elementary school as predictors of problem behavior in middle school. J Emot Behav Disord. (2004) 12:130–44.

Google Scholar

82. Huntington DD, Bender WN. Adolescents with learning disabilities at risk? Emotional well-being, depression, suicide. J Learn Disabil. (1993) 26:159–66. doi: 10.1177/002221949302600303

PubMed Abstract | CrossRef Full Text | Google Scholar

83. Mammarella IC, Ghisi M, Bomba M, Bottesi G, Caviola S, Broggi F, et al. Anxiety and depression in children with nonverbal learning disabilities, reading disabilities, or typical development. J Learn Disabil. (2016) 49:130–9. doi: 10.1177/0022219414529336

PubMed Abstract | CrossRef Full Text | Google Scholar

84. Masten AS, Roisman GI, Long JD, Burt KB, Obradović J, Riley JR, et al. Developmental cascades: linking academic achievement and externalizing and internalizing symptoms over 20 years. Dev Psychol. (2005) 41:733–46. doi: 10.1037/0012-1649.41.5.733

PubMed Abstract | CrossRef Full Text | Google Scholar

85. Pearce J, Stewart SL, Theall L, Gilpin M, Kehyayan V, St. Pierre J, et al. Education. In: Stewart SL, Theall LA, Morris JN, Berg K, Björkgren M, Declercq A et al. editors. interRAI (CAPs) for use with the interRAI (ChYMH) Assessment Instrument, Research Version 1 Standard Edition. Washington, DC: interRAI (2015).

Google Scholar

86. Zendarski N, Mensah F, Hiscock H, Sciberras E. Trajectories of emotional and conduct problems and their association with early high school achievement and engagement for adolescents with ADHD. J Atten Disord. (2021) 25:623–35. doi: 10.1177/1087054719833167

PubMed Abstract | CrossRef Full Text | Google Scholar

87. Darney D, Reinke WM, Herman KC, Stormont M, Ialongo NS. Children with co-occurring academic and behavior problems in first grade: distal outcomes in twelfth grade. J School Psychol. (2013) 51:117–28. doi: 10.1016/j.jsp.2012.09.005

PubMed Abstract | CrossRef Full Text | Google Scholar

88. Child and Adolescent Health Measurement Initiative, Data Resource Center for Child & Adolescent Health.National Survey of Children with Special Health Care Needs. Data Resource Center for Child and Adolescent Health Supported by the U.S. Department of Health and Human Services, Health Resources and Services Administration (HRSA), Maternal and Child Health Bureau (MCHB). (2005/2006). Available online at: www.childhealthdata.org (accessed November 24, 2020).

Google Scholar

89. O’Connor M, Cloney D, Kvalsvig A, Goldfeld S. Positive mental health and academic achievement in elementary school: new evidence from a matching analysis. Educ Res. (2019) 48:205–16. doi: 10.3102/0013189x19848724

CrossRef Full Text | Google Scholar

90. Porche MV, Costello DM, Rosen-Reynoso M. Adverse family experiences, child mental health, and educational outcomes for a national sample of students. Sch Ment Health. (2016) 8:44–60.

Google Scholar

91. Fazel S, Doll H, Långström N. Mental disorders among adolescents in juvenile detention and correctional facilities: a systematic review and metaregression analysis of 25 surveys. J Am Acad Child Adolesc Psychiatry. (2008) 47:1010–9. doi: 10.1097/CHI.ObO13e31817eecf3

PubMed Abstract | CrossRef Full Text | Google Scholar

92. Teplin LA, Abram KM, McClelland GM, Dulcan MK, Mericle AA. Psychiatric disorders in youth in juvenile detention. Arch Gen Psychiatry. (2002) 59:1133–43.

Google Scholar

93. Abram KM, Teplin LA, Charles DR, Longworth SL, McClelland GM, Dulcan MK. Posttraumatic stress disorder and trauma in youth in juvenile detention. Arch Gen Psychiatry. (2004) 61:403–10. doi: 10.1001/archpsyc.61.4.403

PubMed Abstract | CrossRef Full Text | Google Scholar

94. Ulzen TPM, Hamilton H. The nature and characteristics of psychiatric comorbidity in incarcerated adolescents. Can J Psychiatry. (1998) 43:57–63. doi: 10.1177/070674379804300106

PubMed Abstract | CrossRef Full Text | Google Scholar

95. Maschi T, Hatcher SS, Schwalbe CS, Rosato NS. Mapping the social service pathways of youth to and through the juvenile justice system: a comprehensive review. Child Youth Serv Rev. (2008) 30:1376–85.

Google Scholar

96. Teplin LA, Abram KM, McClelland GM, Washburn JJ, Pikus AK. Detecting mental disorder in juvenile detainees: who receives services. Am J Public Health. (2005) 95:1773–80. doi: 10.2105/AJPH.2005.067819

PubMed Abstract | CrossRef Full Text | Google Scholar

97. Stewart SL, Theall L, Perry B, MacLeod K, Smith C, Mathias K, et al. Traumatic life events. In: Stewart SL, Theall LA, Morris JN, Berg K, Björkgren M, Declercq A et al. editors. interRAI (CAPs) for use with the interRAI (ChYMH) Assessment Instrument, Research Version 1 Standard Edition. Washington, DC: interRAI (2015).

Google Scholar

98. Stewart SL, Toohey A, Lapshina N. Childhood maltreatment and risk of harm to self and others: the role of sex and polyvictimization. Int J Child Adolesc Resil. (2020) 7:108–22.

Google Scholar

99. Anda RF, Brown DW, Dube SR, Bremner JD, Felitti VJ, Giles WH. Adverse childhood experiences and chronic obstructive pulmonary disease in adults. Am J Prev Med. (2008) 34:396–403. doi: 10.1016/j.amepre.2008.02.002

PubMed Abstract | CrossRef Full Text | Google Scholar

100. Dube SR, Felitti VJ, Dong M, Giles WH, Anda RF. The impact of adverse childhood experiences on health problems: evidence from four birth cohorts dating back to 1900. Prev Med. (2003) 37:268–77. doi: 10.1016/s0091-7435(03)00123-3

PubMed Abstract | CrossRef Full Text | Google Scholar

101. Felitti VJ, Anda RF, Nordenberg D, Williamson DF, Spitz AM, Edwards V, et al. Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults: the adverse childhood experiences (ACE) study. Am J Prev Med. (1998) 14:245–58. doi: 10.1016/s0749-3797(98)00017-8

CrossRef Full Text | Google Scholar

102. Howard Sharp KM, Schwartz LE, Barnes SE, Jamison LE, Miller-Graff L, Howell KH. Differential influence of social support in emerging adulthood across sources of support and profiles of interpersonal and non-interpersonal potentially traumatic experiences. J Aggress Maltreat Trauma. (2017) 26:736–55.

Google Scholar

103. Lilly MM, Valdez CE, Graham-Bermann S. The mediating effect of world assumptions on the relationship between trauma exposure and depression. J Interpers Violence. (2011) 26:2499–516. doi: 10.1177/0886260510383033

PubMed Abstract | CrossRef Full Text | Google Scholar

104. Leeb RT, Lewis T, Zolotor AJ. A review of physical and mental health consequences of child abuse and neglect and implications for practice. Am J Lifestyle Med. (2011) 5:454–68. doi: 10.1016/j.jflm.2020.101930

PubMed Abstract | CrossRef Full Text | Google Scholar

105. Bailey HN, Moran G, Pederson DR. Childhood maltreatment, complex trauma symptoms, and unresolved attachment in an at-risk sample of adolescent mothers. Attach Hum Dev. (2007) 9:139–61. doi: 10.1080/14616730701349721

PubMed Abstract | CrossRef Full Text | Google Scholar

106. Spinazzola J, Ford JD, Zucker M, van der Kolk, Bessel A, Silva S, et al. Survey evaluates complex trauma exposure, outcome, and intervention among children and adolescents. Psychiatr Ann. (2005) 35:433–9.

Google Scholar

107. Cook A, Spinazzola J, Ford J, Lanktree C, Blaustein M, Cloitre M, et al. Complex trauma in children and adolescents. Psychiatr Ann. (2005) 35:390–8.

Google Scholar

108. Shonkoff JP, Garner AS, Siegel BS, Dobbins MI, Earls MF, McGuinn L, et al. The lifelong effects of early childhood adversity and toxic stress. Pediatrics. (2012) 129:e232–46.

Google Scholar

109. Stewart SL, Hirdes JP, Curtin-Telegdi N, Perlman C, MacLeod K, Ninan A, et al. interRAI Child and Youth Mental Health (ChYMH) Assessment Form and User’s Manual. Version 9.3. Washington, DC: interRAI (2015).

Google Scholar

110. Armiento J, Hamza CA, Stewart SL, Leschied A. Direct and indirect forms of childhood maltreatment and nonsuicidal self-injury among clinically-referred children and youth. J Affect Disord. (2016) 200:212–7. doi: 10.1016/j.jad.2016.04.041

PubMed Abstract | CrossRef Full Text | Google Scholar

111. Lau C, Stewart SL, Saklofske DH, Hirdes J. Scale development and psychometric properties of internalizing symptoms: the interRAI child and youth mental health internalizing subscale. Psychiatry Res. (2019) 278:235–41. doi: 10.1016/j.psychres.2019.06.013

PubMed Abstract | CrossRef Full Text | Google Scholar

112. Stewart SL, Morris JN, Asare-Bediako Y, Toohey A. Examining the structure of a new pediatric measure of functional independence using the interRAI child and youth mental health assessment system. Dev Neurorehabil. (2020) 23:526–33. doi: 10.1080/17518423.2019.1698070

PubMed Abstract | CrossRef Full Text | Google Scholar

113. Stewart SL, LaRose L, Gleason K, Nicolson R, McKnight M, Knott W, et al. interRAI Child and Youth Mental Health – Developmental Disabilities (ChYMH- DD) Assessment Form and User’s Manual. Version 1. Washington, DC: interRAI (2015).

Google Scholar

114. Billawala AS, Hamza CA, Stewart SL. Risk factors for complex special needs among male children seeking mental health services. J Dev Disabl. (2018) 23:17–26.

Google Scholar

115. Lapshina N, Stewart SL. Examining service complexity in children with intellectual disability and mental health problems who receive inpatient or outpatient services. J Intellect Dev Disabil. (2019) 44:464–73.

Google Scholar

116. Redquest BK, Stewart SL, Bryden PJ, Fletcher PC. Assessing the wellbeing of individuals with autism using the interRAI child and youth mental health (ChYMH) and the interRAI child and youth mental health – developmental disabilities (ChYMH-DD) tools. Pediatr Nurs. (2020) 46: 83–91.

Google Scholar

117. Stewart SL, Hirdes JP, McKnight M, Curtin-Telegdi N, Perlman CM, MacLeod K, et al. interRAI Child and Youth Mental Health Screener (ChYMH-S) Assessment Form and User’s Manual. Version 9.3. Washington, DC: interRAI (2017).

Google Scholar

118. Marshall C, Semovski V, Stewart SL. Exposure to childhood interpersonal trauma and mental health service urgency. Child Abuse Negl. (2020) 106:10. doi: 10.1016/j.chiabu.2020.104464

PubMed Abstract | CrossRef Full Text | Google Scholar

119. Stewart SL, Celebre A, Hirdes JP, Poss JW. Risk of suicide and self-harm in kids: the development of an algorithm to identify high-risk individuals within the children’s mental health system. Child Psychiatry Hum Dev. (2020) 51:913–24. doi: 10.1007/s10578-020-00968-9

PubMed Abstract | CrossRef Full Text | Google Scholar

120. Semovski V, King C, Stewart SL. Mental health service urgency in children’s mental health: factors impacting the need for expedited services. Child Psychiatry Hum Dev. (2021) 1–11. [Online ahead of print]. doi: 10.1007/s10578-021-01161-2

CrossRef Full Text | Google Scholar

121. Phillips CD, Hawes C, Morris JN, Berg K, Björkgren M, Declercq A, et al. interRAI Pediatric Home Care (PEDS-HC) Assessment Form and User’s Manual. Version 9.2. Washington, DC: interRAI (2014).

Google Scholar

122. Stewart SL, Theall LA, Morris JN, Berg K, Björkgren M, Declercq A, et al. interRAI Self-Report Quality of Life - Child and Youth Mental Health Instrument (QoL-ChYMH) Surveys and User’s Manual. interRAI Standard Edition. Version 10.0.0. Washington, DC: interRAI (2020).

Google Scholar

123. Celebre A, Stewart SL, Theall L, Lapshina N. An examination of correlates of quality of life in children and youth with mental health issues. Front Psychiatry. (2021) 12:709516. doi: 10.3389/fpsyt.2021.709516

CrossRef Full Text | Google Scholar

124. Sun L, Semovski V, Stewart SL. A study of risk factors predicting school disruption in children and youth living in Ontario. Can J Sch Psychol. (2021) 36:191–205. doi: 10.1177/0829573521991421

CrossRef Full Text | Google Scholar

125. Klassen J, Hamza CA, Stewart SL. An examination of correlates for adolescent engagement in nonsuicidal self-injury, suicidal self-injury, and substance use. J Res Adolesc. (2017) 28:342–53. doi: 10.1111/jora.12333

CrossRef Full Text | Google Scholar

126. Klassen JA, Stewart SL. School disengagement and mental health service intensity need among clinically referred children. Front Psychiatry. (2021) 12:690917. doi: 10.3389/fpsyt.2021.690917

CrossRef Full Text | Google Scholar

127. Stewart SL, Van Dyke JN, Poss J. Examining the mental health presentations of treatment-seeking transgender and gender nonconforming (TGNC) youth. Child Psychiatry Hum Dev. (2021). [Online ahead of print]. doi: 10.1007/s10578-021-01289-1

CrossRef Full Text | Google Scholar

128. Stewart SL, Lapshina N, Semovski V. Interpersonal polyvictimization: addressing the care planning needs of traumatized children and youth. Child Abuse Negl. (2021) 114:104956. doi: 10.1016/j.chiabu.2021.104956

CrossRef Full Text | Google Scholar

129. Sattler JM. Assessment of Children: Cognitive Foundations and Applications. 6th ed. Las Mesa, CA: Sattler Publisher, Inc (2018).

Google Scholar

130. Lau C, Stewart SL, Saklofske DH, Tremblay PF, Hirdes J. Psychometric evaluation of the interRAI child and youth mental health disruptive/aggression behaviour scale (DABS) and hyperactive/distraction scale (HDS). Child Psychiatry Hum Dev. (2018) 49:279–89. doi: 10.1007/s10578-017-0751-y

PubMed Abstract | CrossRef Full Text | Google Scholar

131. Lau C, Stewart SL, Saklofske DH, Hirdes J. Development and psychometric validation of the interRAI ChYMH externalizing subscale. Clin Child Psychol Psychiatry. (2021) 26:295–305. doi: 10.1177/1359104520963143

PubMed Abstract | CrossRef Full Text | Google Scholar

132. Li Y, Babcock SE, Stewart SL, Hirdes JP, Schwean VL. Psychometric evaluation of the depressive severity index (DSI) among children and youth using the interRAI child and youth mental health (ChYMH) assessment tool. Child Youth Care Forum. (2021) 50:611–30. doi: 10.1007/s10566-020-09592-z

CrossRef Full Text | Google Scholar

133. Phillips CD, Patnaik A, Moudouni DK, Naiser E, Dyer JA, Hawes C, et al. Summarizing activity limitations in children with chronic illnesses living in the community: a measurement study of scales using supplemented interRAI items. BMC Health Serv Res. (2012) 12:19. doi: 10.1186/1472-6963-12-19

CrossRef Full Text | Google Scholar

134. Phillips CD, Hawes C. The interRAI pediatric home care (PEDS HC) assessment: evaluating the long-term community-based service and support needs of children facing special healthcare challenges. Health Serv Insights. (2015) 8:17–24. doi: 10.4137/HSI.S30775

PubMed Abstract | CrossRef Full Text | Google Scholar

135. Stewart SL, Babcock SE. interRAI child and youth mental health-screener (ChYMH-S): a psychometric evaluation and validation study. Child Psychiatry Hum Dev. (2020) 51:769–80. doi: 10.1007/s10578-020-01003-7

PubMed Abstract | CrossRef Full Text | Google Scholar

136. Stewart SL, Babcock SE, Li Y, Dave HP. A psychometric evaluation of the interRAI child and youth mental health instruments (ChYMH) anxiety scale in children with and without developmental disabilities. BMC Psychiatry. (2020) 20:390. doi: 10.1186/s12888-020-02785-9

CrossRef Full Text | Google Scholar

137. Stewart SL, Currie M, Arbeau K, Leschied A, Kerry A. Assessment and planning for community and custodial services: the application of interRAI assessment in the youth justice system. In: Corrado R, Whatley J, Leschied A, Lussier P editors. Serious and Violent Young Offenders and Youth Criminal Justice: A Canadian Perspective. Burnaby, BC: Simon Fraser University Publications (2015). p. 355–97.

Google Scholar

138. Stewart SL, Hamza CA. The child and youth mental health assessment (ChYMH): An examination of the psychometric properties of an integrated assessment developed for clinically referred children and youth. BMC Health Serv Res. (2017) 17:82. doi: 10.1186/s12913-016-1970-9

CrossRef Full Text | Google Scholar

139. Stewart SL, Klassen J, Tohver GC. Validation of the interrail risk of school disruption scale: a measurement of school disengagement and disruption. Personal Ind Diff. (2016) 100:518. doi: 10.1016/j.paid.2016.05.311

CrossRef Full Text | Google Scholar

140. Iantosca JM. Construction and Preliminary Validation of the interRAI 0-3 Developmental Domains. Doctoral dissertation. Electronic Thesis and Dissertation Repository, Ph.D. thesis. London: The University of Western Ontario (2020).

Google Scholar

141. Hirdes JP, Ljunggren G, Morris JN, Frijters DHM, Finne-Soveri H, Gray L, et al. Reliability of the interRAI suite of assessment instruments: a 12-country study of an integrated health information system. BMC Health Serv Res. (2008) 8:277. doi: 10.1186/1472-6963-8-277

CrossRef Full Text | Google Scholar

142. Cohen L, Manion L, Morrison K. Research Methods in Education. 8th ed. London: Routledge (2018).

Google Scholar

143. Stewart SL, Toohey A, Poss JW. iCCareD: the development of an algorithm to identify factors associated with distress among caregivers of children and youth referred for mental health services. Front Psychiatry. (2021) 12:737966. doi: 10.3389/fpsyt.2021.737966

CrossRef Full Text | Google Scholar

144. Stewart SL, Celebre A, Hirdes JP, Poss JW. Risk of injury to others: the development of an algorithm to identify children and youth at high-risk of aggressive behaviours. Front Psychiatry. (2022) 12:750625. doi: 10.3389/fpsyt.2021.750625

CrossRef Full Text | Google Scholar

145. Ásgeirsdóttir HG, Valdimarsdóttir UA, þorsteinsdóttir þK, Lund SH, Tomasson G, Nyberg U, et al. The association between different traumatic life events and suicidality. Eur J Psychotraumatol. (2018) 9:1510279. doi: 10.1080/20008198.2018.1510279

CrossRef Full Text | Google Scholar

146. Madge N, Hawton K, McMahon EM, Corcoran P, Leo D, Wilde EJ, et al. Psychological characteristics, stressful life events and deliberate self-harm: findings from the child & adolescent self-harm in Europe (CASE) study. Eur Child Adolesc Psychiatry. (2011) 20:499–508. doi: 10.1007/s00787-011-0210-4

CrossRef Full Text | Google Scholar

147. Ystgaard M, Hestetun I, Loeb M, Mehlum L. Is there a specific relationship between childhood sexual and physical abuse and repeated suicidal behavior? Child Abuse Negl. (2004) 28:863–75. doi: 10.1016/j.chiabu.2004.01.009

PubMed Abstract | CrossRef Full Text | Google Scholar

148. Al Odhayani A, Watson WJ, Watson L. Behavioural consequences of child abuse. Can Fam Physician. (2013) 59:831–6.

PubMed Abstract | Google Scholar

149. Fagan AA. The relationship between adolescent physical abuse and criminal offending: support for an enduring and generalized cycle of violence. J Fam Violence. (2005) 20:279–90. doi: 10.1007/s10896-005-6604-7

CrossRef Full Text | Google Scholar

150. Lansford JE, Miller-Johnson S, Berlin LJ, Dodge KA, Bates JE, Pettit GS. Early physical abuse and later violent delinquency: a prospective longitudinal study. Child Maltreat. (2007) 12:233–45. doi: 10.1177/1077559507301841

PubMed Abstract | CrossRef Full Text | Google Scholar

151. Theall L, Arbeau K, Willoughby K, St. Pierre J, Ng G, Stewart SL. Assessing needs and outcomes of children and youth receiving intensive services. Resid Treat Child Youth. (2022) 39:16–33.

Google Scholar

152. Stewart SL, Celebre A, Head M, James M, Martin L, Fries BE. A case-mix system for children and youth with developmental disabilities. Health Serv Insights. (2020) 13:1178632920977899. doi: 10.1177/1178632920977899

CrossRef Full Text | Google Scholar

153. Sawhill IV, Karpilow Q. How Much Could we Improve Children’s Life Chances by Intervening Early and Often? Center on Children and Families at Brookings. Washington, DC: Brookings (2015).

Google Scholar

154. Stewart SL, Vasudeva A, Van Dyke J, Poss JW. Following the epidemic waves: child and youth mental health assessments in Ontario through multiple pandemic waves. Front Psychiatry. (2021) 12:730915. doi: 10.3389/fpsyt.2021.730915

CrossRef Full Text | Google Scholar

155. Stewart SL, Vasudeva A, Van Dyke J, Poss JW. Child and youth mental health needs and service utilization during COVID-19. Traumatology. (2021). [Epub ahead of print]. doi: 10.1037/trm0000334

CrossRef Full Text | Google Scholar

156. Stewart SL, Toohey A, Celebre A, Poss JW. Abuse, mental state, and health factors pre and during the COVID-19 pandemic: a comparison among clinically referred adolescents in Ontario, Canada. Int J Environ Res Public Health. (2021) 18:10184. doi: 10.3390/ijerph181910184

CrossRef Full Text | Google Scholar

Appendix

Supplemental Data Description

All data used were from extracts of assessments captured by secure software platforms used by the assessing agencies. If an individual was assessed more than once by an agency using the same instrument, the first observation was used.

Data description, by instrument:

For Figure 4, individuals with two consecutive assessments (ChYMH or ChYMH-DD or ChYMH-Screener) between 30 and 365 days apart were identified. Assessment setting was required to be the same on both assessments, i.e., inpatient for both or outpatient for both. There were 11,552 pairs: 5,791 ChYMH, 124 ChYMH-DD, and 5,637 ChYMH-Screener. We report the proportion of these pairs that achieved a 2-point improvement (Anxiety, DSI, and Externalizing scales) or a 1-point improvement (RiSsK and RIO scales) among cases that had a baseline scale score of 2 more, or 1 or more, respectively. Note that higher scale values reflect higher acuity, so an improvement is a decline in score value.

These N’s were:

Anxiety Scale: 8,176 (3,376 ineligible).

DSI: 8,853 (2,699 ineligible).

Externalizing Scale: 8,661 (2,891 ineligible).

RiSsK Scale: 5,749 (5,803 ineligible).

RIO Scale: 5,992 (5,560 ineligible).