iCCareD: The Development of an Algorithm to Identify Factors Associated With Distress Among Caregivers of Children and Youth Referred for Mental Health Services

Caregiver well-being plays an important role in children's development and a number of factors have been found to impact distress levels among caregivers of children and youth referred for mental health services. Further, caregiver distress impacts youth psychopathology, its acuity as well as related mental health interventions. The purpose of this study was to develop and validate an algorithm for identifying caregivers who are at greatest risk of experiencing caregiver distress. This algorithm was derived from, and will be embedded in, existing comprehensive interRAI child and youth instruments. Ontario data based on the interRAI Child and Youth Mental Health assessment instruments (ChYMH and ChYMH-DD) were analyzed to identify predictors of distress among caregivers of children and youth ages 4–18 years. Starting with proactive aggression, the algorithm uses 40 assessment items to assign one of 30 nodes that are grouped into five levels of risk. The interRAI ChYMH Caregiver Distress (iCCareD) algorithm was validated using longitudinal data from mental health agencies across Ontario and was found to be a good predictor among this sample with a c-statistic of 0.71 for predicting new or ongoing caregiver distress and 65% for both sensitivity and specificity using algorithm values of 3 or greater. This algorithm provides an evidence-based decision-support tool embedded within a comprehensive assessment tool that may be used by clinicians to inform their selection of supports and services for families.

Introduction

The well-being of caregivers has significant implications for the healthy development of children and youth (hereafter referred to as children) (1). Although caregiver distress is expected and typical, it can become problematic if caregivers have difficulty identifying and responding to their children's needs. Research has demonstrated that a number of child, caregiver, and family/environmental characteristics are related to caregivers' stress levels in relation to parenting. Given the important role caregivers play in the development and nurturing of children, understanding the factors that contribute to their distress is crucial.

Much of the research investigating distress in relation to caring for children has focused on child characteristics. Research has consistently shown that children's physical health as well as the behavioral and emotional difficulties they experience, significantly impact their caregivers. With respect to physical health, research has shown that severity and duration of physical illness (2–6) as well as early onset physical health problems of their child (7), are all predictive of greater caregiver distress. That is, caregivers who have children with physical health problems are perceived as more severe or have been ongoing for a longer period of time, experience greater levels of distress in relation to their role as a caregiver. Research findings have also demonstrated that child sleep problems are predictive of greater caregiver distress (5).

In addition to being predicted by physical health factors, there is a well-established relationship between child behavioral and emotional difficulties and caregiver distress (5, 6, 8–11). That is, parents of children who experience greater levels of both externalizing (e.g., aggression) and internalizing (e.g., anxiety, depression) symptomology experience greater levels of caregiver distress. Researchers have found that children with executive functioning problems (i.e., sustaining attention, switching from one task to another, initiating, regulating behavior) have parents who experience more stress (12, 13). Further, those parents whose children were neuro-atypical with co-occurring emotional or behavioral problems reported the highest levels of parenting stress (13). Taken together, these findings highlight the impact of child physical and mental health characteristics on stress levels of their caregivers.

In addition to child-specific characteristics, extant literature examining caregiver distress has underscored the importance of the caregiver themselves as well as the family environment. The relationship between caregiver mental health and distress level is well-established in the literature. Research has shown that caregivers with depression (7) or greater psychiatric co-morbidity (4) experience significantly more caregiver distress. Further, research has shown that caregivers' psychological distress and anxiety are positively correlated with caregivers' feelings of burden and may, in turn, disrupt family well-being (14). In a recent review of the literature, researchers found that parental mental health problems are associated not only with greater caregiver distress, but also lower parenting satisfaction, poorer parent-child relationships, and more frequent use of negative parenting practices (15).

Extant literature has also demonstrated a clear link between financial struggles and caregiver distress. In fact, low family income [e.g., (5, 7)] and other financial stressors have been significantly linked with caregiver distress, accounting for up to 42% of the variance in overall stress reported (8).

In addition to financial difficulties, a perceived lack of social support has also consistently predicted caregiver distress in the literature [e.g., (3, 16)] with stress levels varying by marital quality and perceived level of support (6). Similarly, parents of adolescents with executive functioning issues report greater feelings of isolation, more conflict with their partner, and greater feels of guilt and incompetence (11), all of which are related to their overall caregiver distress levels. Research has also demonstrated that parents with intellectual disabilities are at greater risk of experiencing financial disadvantages and social isolation [e.g., (17, 18)]. Further, research has demonstrated that a lack of social support is related to increased parenting stress for mothers with intellectual disabilities as well as increased behavioral problems for their school-aged children (19). These results highlight that there are a number of caregiver and family environment characteristics that impact caregiver distress and underscore possible interactions of these factors can have an even greater impact on the distress experienced by caregivers.

Though each of these characteristics can individually be associated with child maladjustment, there appears to also be a cumulative effect of caregiver distress and family instability on child well-being (20). Further, the transactional model of development highlights that child and caregiver characteristics impact one another (21). This theoretical framework is supported by research underscoring the dynamic interactions among child, caregiver, and family environment factors that impact caregiver distress. Notably, extant literature has demonstrated that caregiver distress impacts family functioning, child psychopathology, mental health interventions for children [e.g., (1)], and risk of adverse childhood experiences [ACEs; (22)].

Findings from a recent review of parenting stress measures outline that there are a number of well-studied measures of parenting stress which have the ability to assess parenting stress accurately and reliably (1). Instruments have been utilized to assess the burden and strain of parenting children and youth presenting with mental health issues (1). For example, the Parenting Stress Scale (23) was designed to capture the parent's perception of the parental role rather than sources of stress (24). Similarly, the Caregiver Strain Questionnaire (25) has been utilized to assess the burden of caring for children and youth with serious emotional and behavioral issues in order to support clinical interventions and treatment outcomes. Also, the Parenting Stress Index (26) is an inventory that evaluates levels of stress within the parenting role that includes life stress, child characteristics as well as parenting factors. While the ability for identified measures to assess parenting distress accurately and reliably has been strong (1), to our knowledge, no previous research has developed a unique algorithm to predict on-going or future caregiver distress utilizing a comprehensive assessment system. These existing measures are stand-alone and are meant to be used in conjunction with a number of other child-focused assessment measures in order to gain an understanding of child and caregiver functioning. Further, none of these measures include psychometrics with respect to discriminative validity (1), underscoring the need to understand the combination of factors that identify high risk families earlier to provide appropriate and necessary supports and prevent possible ACEs. Given that caregiver distress is relevant to the development, continuation and treatment of childhood mental health issues, the ability to predict which caregivers are at greater risk of experiencing distress would be helpful for clinicians working with children and their families for prevention and early intervention efforts.

The purpose of this study is to develop a new decision-support algorithm embedded within existing assessment tools (the interRAI Child and Youth Mental Health suite). Developing such an algorithm will assist clinicians who are already administering the ChYMH or ChYMH-DD with identifying caregivers at greatest risk of caregiving-related distress. Based on the literature extant, it was anticipated that certain child, parent and environmental factors would be associated with increased levels of caregiver distress. With respect to child behaviors, it was anticipated that sleep problems, emotion dysregulation, aggression, and self-harm behaviors would increase levels of parental distress. Furthermore, it was anticipated that parents who are struggling with family issues, major life stressors, marital problems, domestic violence, as well as difficulties related to their own health and well-being would be at heightened risk for caregiver distress.

Materials and Methods

Sample

Participants in this study consisted of children aged 4–18 years who received services from over 50 mental health agencies across Ontario as part of standard of care. Assessments completed between January 2015 and February 2020 were abstracted to an analytic dataset, removing any personal identifiers. These children were referred to the agencies through family physicians, pediatricians, school personnel, parents, or other allied health professionals. After removing records where the client age was outside of the range of 4–18 years, as well as assessments where there was evidence that a parent or other primary caregiver was not present in the child's life, our sample consisted of 30, 210 eligible assessment records. In order to evaluate the data longitudinally, consecutive pairs of assessments for clients were then selected where the two assessments fell between 30 and 365 days apart, and where the first assessment of the pair was not designated as a discharge assessment. If more than one such pair could be constructed for a client, the first one was selected, resulting in 7,182 pairs for analysis. Given that assessment responses are entered by clinicians into computer forms that require all items to be completed to have responses that are of the proper form, no missing data are encountered. Western University's ethics board granted approval for the secondary analysis of data collected in various agencies throughout the Province of Ontario (REB #106415).

Measures

The Child and Youth Mental Health [ChYMH; (27)] and Child and Youth Mental Health and Developmental Disability [ChYMH-DD; (28)] assessments are comprehensive, clinician-rated standardized, multi-sectoral, and multidisciplinary mental health measures for children and youth, ages 4–18 years. These tools are used as the standard assessment instruments and administered in regular clinical practice in over 50 children's mental health agencies across the province of Ontario. The ChYMH includes over 400 clinical items related to a variety of domains (e.g., mental state indicators, behavior, independence in daily activities, communication, family, and social relations) and the ChYMH-DD includes roughly 65 additional population-specific items. Only the items that are available and identical in both instruments were used to develop the algorithm presented here.

The interRAI ChYMH and ChYMH-DD assessments include items related to the needs, areas of risk, functioning, and strengths of children and youth. These assessments are administered as semi-structured interviews by trained assessors involving the child/youth, guardians, family members as well as collateral contacts (e.g., teachers, therapists). Additionally, information from report cards, academic assessments, medical records, and relevant clinical documents is also reviewed. The tools are intended to support comprehensive care planning, outcome measurement, quality indicators, and case mix classification to estimate relative resource intensity (Stewart and Toohey, under review)¹.

The result is a valid and reliable set of information that can be used individually for case documentation and to inform program planning as well as collectively for system reporting and secondary research purposes. The interRAI ChYMH and ChYMH-DD are part of an integrated health information system in which psychometrically sound scales and algorithms are embedded within the instrument to support clinical decision making (29–39). A detailed manual supports the instrument and provides coding rules for the items.

Items within the instruments employ specific observation periods in order to provide reliable and valid measures of clinical characteristics that reflect the child's strengths, preferences, and needs. The basic time frame for assessment was set at 3 days unless otherwise indicated. Importantly, some items address the recency and frequency of symptoms prior to and within the last 3 days. There is also an option to indicate that symptoms are present but not exhibited within the last 3 days. Responses for items are constrained, almost always binary or ordinal, and are well-suited to completion using a computer entry format. These tools include a number of scales in order to support care planning.

Dependent Variable

The target variable of the modeling was the item “parent/primary caregiver expresses feelings of distress, anger, or depression,” coded as no or yes, for the assessment completed on the follow-up assessment. Since the baseline assessment also recorded caregiver distress, the result was to model two variations on this outcome. The first, where caregiver distress is absent at baseline, makes the target outcome that of a newly developed state of caregiver distress. The second, where caregiver distress is present at baseline, makes it one of ongoing distress. By effectively combining this, the model explores factors related to new caregiver distress that might emerge, or if currently present, whether it fails to improve. This so-called “double-barrel” perspective has been applied elsewhere, for example in quality indicator formulation (40).

Independent Variables

Explanatory items came from the child's baseline assessment. All individual clinical items were collected, gathering information about the child's well-being and development across a variety of domains, including information related to parenting discipline, monitoring, distress in relation to caregiving, and whether the child's parent has a mental health, developmental, or substance use issue. In addition to individual items, a number of computed scales were used as predictors. These included:

• Proactive aggression (7 items: stealing, elopement attempts/threats, bullying peers, preoccupation with violence, violence to others, intimidation of others or threatened violence, violent ideation; 30)

• Reactive aggression [5 items: impulsivity, physical abuse, outbursts of anger, defiant behavior, argumentativeness; (29)]

• Parenting strengths scale (6 items, 0–12, higher values indicate greater parenting strength)

• Disruptive/aggressive behavior scale [5 items, 0–20: verbal abuse, physical abuse, socially inappropriate/disruptive, destructive behavior toward property, outbursts of anger; (31)]

• ChYMH aggressive behavior scale [4 items, 0–12: verbal abuse, physical abuse, socially inappropriate/disruptive, resists care; (2)]

• Family functioning composite score (modified)—(4 items, 0–4: strong/supporting relationship with family, family persistently hostile/critical of child, parent has developmental/mental health/addiction issues, sibling has developmental/mental health/addiction issues).

Procedure

Seventy-five percent of assessments were randomly assigned to a derivation subset and the remaining 25% were assigned for validation. The analytic method used was an interactive decision tree tool supported by SAS Enterprise Miner. A decision tree starts with all observations that are then sub-divided at branch points into two or more groups based on the value of a clinical item, with these groups subsequently sub-divided using other clinical items until a terminal node is reached. Decision trees have the strength of identifying natural interactions of clinical items (41), such that an item may have a strong association with the outcome in one subset of the population, while different items are important among another. With this tool, the analyst is presented with a list of clinical items and their associated strength of association with the outcome (caregiver distress). A large number of possible variations of trees are possible, with the initial split being particularly important. Several options for the initial split were explored, including characteristics of the family, age of the child/youth, and a summary measure of child/youth aggression. Construction of a candidate tree continues until the number of remaining cases becomes too small, or there are no clinical items that reach a statistical threshold for a split.

The terminal nodes from the completed tree are subsequently gathered to form a number of discrete groups. A design goal was to have a compact scale of five levels. K-means clustering was used to inform the grouping of the decision tree terminal nodes into the five groups. Five candidate trees representing different first splits and designs were fully developed after considerable exploration, and it was felt that additional options would not yield better candidates from a clinical or performance perspective. These five options were assessed, guided by the scale distribution, mean values of the outcome at each level, and goodness of fit using logistic regression. Feedback from content experts was sought leading to additional refinement of the best candidate until a final tree model was selected. Additional subgroup testing by age groups and by sex was conducted.

Results

Table 1 outlines characteristics of the sample at the baseline assessment. Statistical tests for Table 1 were chi-squares, except for age which is a student t-test. Please note that multiple test correction was not completed for the data presented in Table 1. Group comparisons presented in Table 1 highlight that in families where caregivers report feelings of distress, there are poorer interpersonal relationships, higher rates of caregiver mental health/developmental issues, and higher rates of caregivers reporting that they are unwilling to continue caring for the child. Further, in these families, it is more likely that caregivers have made economic trade-offs, have experienced a major life stressor in the last 3 months, and that the physical home environment is cause for concern.

TABLE 1

Table 1. Sample characteristics at baseline.

The candidate tree that used proactive aggression as the initial split produced a tree with better performance than others considered, including those split initially by age, sex, other aggregations of parent/caregiver strengths/function or child/youth behavior. Figures 1, 2 show the chosen tree diagram for the assignment of the 30 nodes, and their subsequent assignment to a scale value. The first split is into three groups: those cases with none of 7 proactive aggressive items, those with 1 or 2, and those with 3 or more.

FIGURE 1

Figure 1. First branch of ChYMH Caregiver Distress Tree Algorithm.

FIGURE 2

Figure 2. Second and third branches of ChYMH Caregiver Distress Tree Algorithm.

In the chosen tree, all nodes are conditioned on characteristics of the child/youth as the first split, and by characteristics of the family, parent, or primary caregiver which is used as the second split in each of the three branches. There are 40 different items drawn from these eight sections of the assessment: mental state indicators, harm to self or others, behavior, strengths and resilience, executive functioning, health conditions, family/social relations, and stress/trauma.

Overall, 28.1% of the parent/primary caregivers were recorded as having new or ongoing distress, 13.4% new distress (among those without distress at baseline), and 48.7% had ongoing distress (among those with distress at baseline). The distribution and results of the five-point scale are shown in Table 2. Across the range of the scale, there is an 8-fold difference in rates of new or ongoing distress, 10-fold difference for new distress, and a 2.3-fold difference for ongoing distress.

TABLE 2

Table 2. ChYMH Caregiver Distress Algorithm results.

Goodness of fit, the area under the curve or C-statistic from logistic regression models, is presented in Table 3. C-statistic values of 0.5 represent performance no better than chance, while values of 1.0 represent performance where the algorithm predicts all observations perfectly. Considering the dependent variable that was modeled, new or ongoing distress, the reserved validation sample is somewhat lower, which is expected for this kind of modeling where there is a tendency to over-fit the derivation data, compared to the validation data. The difference in fit (0.740 vs. 0.706) here is to be noted, and with future data the degree of over-fitting could be gauged more precisely. Fit was slightly better for female clients than for males, and slightly poorer for the youngest clients. Goodness of fit was not as strong for the related, but not directly modeled, outcomes of new distress as well as ongoing distress. Values for sensitivity and specific, using algorithm values of 3 or greater, were found to be 69 and 67% in the derivation sample, and 65 and 65% in the validation sample.

TABLE 3

Table 3. ChYMH Caregiver Distress Algorithm goodness of Fit C-statistics^*.

There are no instances of reversal (i.e., where a higher scale value results in a lower proportion of the outcome) for the dependent variable among the age and the sex stratifications. In addition, there are no reversals observed for the outcomes of new distress and of ongoing distress. Fewer than 2.5% of the assessment pairs came from the Development Disabilities version of the ChYMH, too few to conduct sub-group analyses specific to these cases.

Discussion

Our results indicate that nearly half of the caregivers who were experiencing distress at baseline continued to experience distress at time two. Further, 13.4% of caregivers who were not experiencing distress initially were experiencing distress at the follow-up assessment. Given the negative consequences associated with caregiver distress for both caregiver and child well-being, these findings highlight the need for clinicians working with children and their families to evaluate and care plan for caregiver distress. Overall, our model highlights that greater proactive aggressive behavior is related to increased likelihood of caregiver distress. This result is consistent with the well-established relationship in the literature between externalizing behaviors and caregiver distress [e.g., (15, 42)]. These results highlight the importance of checking in with caregivers as to their well-being when children present with difficulties related to stealing, elopement, bullying, and violence.

Of note, with respect to new or ongoing distress, our model was a slightly better fit for child/caregiver dyads where the child was 8 years of age or older than for children 4–7 years old. This finding is not surprising given the extant literature highlighting that age of onset of mental health issues is negatively related to time to initial professional help-seeking [e.g., (43, 44)]. Moreover, results from a large-scale study found that the average delay among those with anxiety disorders who eventually made treatment contact was 6–8 years and 9–23 years for mood disorders (44). Taken together with the fact that the age of onset for anxiety disorders tends to be younger than that for mood, behavioral, and substance use disorders (45), it is possible that younger children (and their caregivers) are more unpredictable in their future progression with mental health services than older children and adolescents.

Consistent with previous literature, characteristics of both the child and the caregiver/caregiving environment both predicted caregiver distress. That is, our results demonstrate that among a large sample of children referred for mental health services, the model that best fit the data to predict caregiver distress included aggressive behavior of the child at the first level, followed by parenting strengths or family functioning at the second level. Our algorithm demonstrates that parents/caregivers who did not communicate effectively with their child, had difficulty assisting their child with emotion regulation and who did not demonstrate warmth and support to their child were less likely to utilize appropriate disciplinary practices and were more likely to experience increased distress. These findings are consistent with research demonstrating that caregiver distress is related to both parenting competence and parental responsiveness (46). Previous research has shown that parents who report elevated stress and perceive their child as being “difficult,” typically lack warmth and responsiveness in their interactions with their child, have developmentally inappropriate expectations for their child's behavior, and use inconsistent discipline practices [e.g., (47)]. The nature of the relationship between caregivers, and children and the level of support caregivers provide their children with, are clear indicators of the level of distress these caregivers are experiencing. Our findings also echo previous research findings suggesting that negative parent/child interactions (i.e., lack of emotional warmth, feelings of rejection) as well as experiencing major life stressors are significantly related to parenting stress (48). Further, families in which there are high levels of involvement and cohesion report lower levels of parenting stress (49). Given the impact of caregivers' perceptions of their own efficacy and enjoyment of parenting on their levels of distress, it is not surprising that those caregivers who experience negative relationships with their children, and who recently experienced a major stressful life event, would experience greater caregiver distress. Further, research has demonstrated that controlling and hostile parenting and interaction styles (50), and poor family social cohesion (51, 52) have all been associated with child internalizing disorders. Findings in our model underscore the important role that family and parent/child dynamics, as well as both child and caregiver well-being play with regards to caregiver distress.

With respect to characteristics of the child, our findings indicate that a child's high pain frequency predicted increased likelihood of caregiver distress, a finding consistent with previous research highlighting that severity and duration of physical health problems among children significantly predicts greater caregiver distress [e.g., (6, 9)]. Similar to other literature extant, child sleep problems were also related to increased risk of caregiver distress (9) as was compulsive child behaviors [e.g., (12, 13)]. It was not surprising that executive functioning difficulties in children were associated with increased caregiver distress given that such difficulties require ongoing supports from caregivers to remain on-task, resulting in the need for constant reminders and prompts for redirection (17). Finally, with respect to child characteristics, self-injurious attempts, and disruptive/aggressive behavior were also associated with caregiver distress, a consistent finding throughout the literature (53–56).

Consistent with previous literature [e.g., (8, 11)], for those children who demonstrated the most proactive aggression, in conjunction with those caregivers who exhibited developmental, mental health, or substance use issues, high levels of distress were noted. Similarly, parents with children with intellectual disabilities were more likely to experience financial difficulties and lack social support (22), both of which are related to increased caregiver distress.

Research has demonstrated the negative consequences of caregiver distress for children's development. More specifically, higher caregiver distress has been related to internalizing difficulties among children (57), lack of social competence, as well as externalizing behaviors (58). Further, research has also demonstrated that reducing parenting stress is related to less coercive, harsh parenting practices (59), thereby reducing the possible adverse consequences that elevated caregiver distress has on children and families. Identifying factors associated with caregiver distress can help facilitate the implementation of strategies to target those factors, thereby potentially reducing a variety of negative sequelae including risk of ACEs (22).

Use and Utility

Based on the findings, the ChYMH Caregiver Distress Algorithm is an empirically based decision-support tool that may be used to identify those who are at greater risk of experiencing caregiver distress. Service providers who have completed the interRAI ChYMH or ChYMH-DD assessment can obtain the iCCareD scores automatically from the software in which the algorithm is embedded, and these results can then provide insight as to the service needs for the child and family. It should be noted however, that the intent is not to use the ChYMH Caregiver Distress algorithm as an automated decision-making system. Rather, the iCCareD score, along with other information obtained during the assessment process, should be used to assist the clinical team when determining the level and kinds of support needed by the family. That is, if the iCCareD score is in the upper range, the clinical team could consider collecting additional information about the caregiver's well-being in order to further support care planning for the caregiver specifically (1), or provide referrals for the caregiver to receive additional individual supports. The clinical team should use their professional and clinical judgement to determine whether the score accurately reflects those with distress that may be persistent, or those without evident distress who may be at greater risk of developing it. If a score is in the upper range, it is recommended that the clinical team consider the caregiver to be at high need for support in order to reduce their distress level or to reduce the likelihood that the caregiver will develop distress due to their heightened risk level. If the score is in the lower range, it is recommended that further discussion occur to determine whether the level of distress is appropriate. In all situations, the child's caregiver should be involved in the decision-making process and consider their preferences as well as their strengths and needs (60). For example, a parent with a very high score on the iCCareD algorithm may not require intensive services because adequate supports and other family members are able to address their current needs.

In addition to being used for individualized caregiver support decisions, the iCCareD algorithm can also provide standardized, comprehensive data across agencies, allowing for the identification of needs across the system. Similar to other interRAI algorithms, populations can be stratified according to the Caregiver Distress levels and then be used to compare the performance of mental health agencies with respect to outcomes of care within the Caregiver Distress levels. The main benefit of implementing the iCCareD algorithm is that those individuals who are at greatest risk of experiencing distress will be identified more quickly and accurately and will be provided with more focused services and supports. At the same time, this is not meant to limit the supports provided for caregivers who are at the lower level.

Caregiver distress levels can also be evaluated at the regional, organization, national, and international levels to develop a benchmarking system (61) that can be used to identify jurisdictions where caregiver distress levels are higher than in other regions. This would allow comparisons of similar populations and can be used to inform policy development and planning. Additionally, caregiver distress levels at intake can be used to examine variations across regions with respect to how services are used based on level of need.

While this study has numerous strengths, including its relatively large sample size and longitudinal approach, the findings of this study are not without limitations. First, the results reported herein may not be generalizable to a community-based sample because the sample consisted of parents/caregivers of children who were accessing mental health services. Second, items chosen in the algorithm were selected based on both clinical relevance and statistical power, but do not represent all possible factors that may drive caregiver distress.

Data Availability Statement

The datasets presented in this article are not readily available due to the highly sensitive and confidential nature of the data, as well as the ethical requirements required for use, data will not be made freely available. Moreover, participating mental health agencies required that data not be made freely accessible. Requests to access the datasets should be directed to Shannon L. Stewart, sstewa24@uwo.ca.

Ethics Statement

Ethics approval for secondary data analyses of interRAI data gathered by other organizations was obtained through Western University REB (106415).

Author Contributions

The procedure and execution of the study was completed by SS. Data analysis was performed by JP in consultation with SS. The first draft of the manuscript was written by AT. SS and AT provided revised versions the manuscript. All authors contributed to the conception of this study, read, and approved the final manuscript.

Funding

This work was partially supported by the Public Health Agency of Canada under grant #1617-HQ-000050 and the Canadian Institutes of Health Research under grant #412154.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We wish to thank all participants as well as clinicians who collected this data.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2021.737966/full#supplementary-material

Footnotes

1. ^Stewart SL, Toohey A. Screening assessment of mental health problems in students: using an integrated assessment-to-intervention approach to enhance integrated care. In: Jacques JS, Shaw S, Domene J, McMorris C, editors. Mental Health Assessment, Prevention, and Intervention: Promoting Child and Youth Well-Being to be Published in the Springer Series on Human Exceptionality (under review).

References

1. Holly L, Fenley A, Kritikos T, Merson R, Abidin R, Langer D. Evidence-base update for parenting stress measures in clinical samples. J Clin Child Adoles Psychol. (2019) 48:685–705. doi: 10.1080/15374416.2019.1639515

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Annunziato R, Lee S, Galici E, Arnon R. Children with hepatitis C: the impact of disease and treatment on patients, caregivers and families. J Pediatr Nurs. (2017) 32:8–11. doi: 10.1016/j.pedn.2016.09.005

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Bemister T, Brooks B, Dyck R, Kirton A. Predictors of caregiver depression and family functioning after perinatal stroke. BMC Pediatr. (2015) 15:75. doi: 10.1186/s12887-015-0397-5

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Dada M, Okewole N, Ogun O, Bello-Mojeed M. Factors associated with caregiver burden in a child and adolescent psychiatric facility in Lagos, Nigeria: a descriptive cross sectional study. BMC Pediatr. (2011) 11:110. doi: 10.1186/1471-2431-11-110

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Edmond S, Graves P, Whiting S, Karlson C. Emotional distress and burden among caregivers of children with oncological/hematological disorders. Fam Syst Health. (2016) 34:166–71. doi: 10.1037/fsh0000181

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Pinquart M. Parenting stress in caregivers of children with chronic physical condition-a meta-analysis. Stress Health. (2017) 34:197–207. doi: 10.1002/smi.2780

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Choi E, Yoon S, Kim J, Park H, Kim J, Yu E. Depression and distress in caregivers of children with brain tumors undergoing treatment: psychosocial factors as moderators. Psycho Oncol. (2015) 25:544–50. doi: 10.1002/pon.3962

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Baldwin K, Brown R, Milan M. Predictors of stress in caregivers of attention deficit hyperactivity disordered children. Am J Fam Ther. (1995) 23:149–60. doi: 10.1080/01926189508251345

CrossRef Full Text | Google Scholar

9. Kelley S, Whitley D, Campos P. Behavior problems in children raised by grandmothers: the role of caregiver distress, family resources, and the home environment. Child Youth Serv Rev. (2011) 33:2138–45. doi: 10.1016/j.childyouth.2011.06.021

CrossRef Full Text | Google Scholar

10. Stewart M, McGillivray J, Forbes D, Austin D. Parenting a child with an autism spectrum disorder: a review of parent mental health and its relationship to a trauma-based conceptualisation. Adv Mental Health. (2016) 15:4–14. doi: 10.1080/18387357.2015.1133075

CrossRef Full Text | Google Scholar

11. Wiener J, Biondic D, Grimbos T, Herbert M. Parenting stress of parents of adolescents with attention-deficit hyperactivity disorder. J Abnorm Child Psychol. (2015) 44:561–74. doi: 10.1007/s10802-015-0050-7

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Hutchison L, Feder M, Abar B, Winsler A. Relations between parenting stress, parenting style, and child executive functioning for children with ADHD or autism. J Child Fam Stud. (2016) 25:3644–56. doi: 10.1007/s10826-016-0518-2

CrossRef Full Text | Google Scholar

13. Craig F, Operto F, De Giacomo A, Margari L, Frolli A, Conson M, et al. Parenting stress among parents of children with Neurodevelopmental Disorders. Psychiatry Res. (2016) 242:121–9. doi: 10.1016/j.psychres.2016.05.016

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Calderon C, Gomez-Lopez L, Martinez-Costa C, Borraz S, Moreno-Villares J, Pedron-Giner C. Feeling of burden, psychological distress, and anxiety among primary caregivers of children with home enteral nutrition. J Pediatr Psychol. (2010) 36:188–95. doi: 10.1093/jpepsy/jsq069

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Christie H, Hamilton-Giachritsis C, Alves-Costa F, Tomlinson M, Halligan S. The impact of parental posttraumatic stress disorder on parenting: a systematic review. Eur J Psychotraumatol. (2019) 10:1550345. doi: 10.1080/20008198.2018.1550345

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Hsiao Y. Pathways to mental health-related quality of life for parents of children with autism spectrum disorder: roles of parental stress, children's performance, medical support, and neighbor support. Res Autism Spectr Disord. (2016) 23:122–30. doi: 10.1016/j.rasd.2015.10.008

CrossRef Full Text | Google Scholar

17. Emerson E, Brigham P. The developmental health of children of parents with intellectual disabilities: cross sectional study. Res Dev Disabil. (2014) 35:917–21. doi: 10.1016/j.ridd.2014.01.006

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Wade C, Llewellyn G, Matthews J. Parent mental health as a mediator of contextual effects on parents with intellectual disabilities and their children. Clin Psychol. (2015) 19:28–38. doi: 10.1111/cp.12055

CrossRef Full Text | Google Scholar

19. Meppelder M, Hodes M, Kef S, Schuengel C. Parents with intellectual disabilities seeking professional parenting support: the role of working alliance, stress and informal support. Child Abuse Negl. (2014) 38:1478–86. doi: 10.1016/j.chiabu.2014.04.006

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Casanueva C, Dozier M, Tueller S, Dolan M, Smith K, Webb M, et al. Caregiver instability and early life changes among infants reported to the child welfare system. Child Abuse Negl. (2014) 38:498–509. doi: 10.1016/j.chiabu.2013.07.016

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Sameroff A. The transactional model. In: A. Sameroff, editor. The Transactional Model of Development: How Children and Contexts Shape Each Other. American Psychological Association (2009). p. 3–21. doi: 10.1037/11877-001

CrossRef Full Text | Google Scholar

22. Crouch E, Radcliff E, Brown M, Hung P. Exploring the association between parenting stress and a child's exposure to adverse childhood experiences (ACEs). Child Youth Serv Rev. (2019) 102:186–92. doi: 10.1016/j.childyouth.2019.05.019

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Berry JD, Jones WH. The Parental Stress Scale: initial psychometric evidence. J Soc Pers Relat. (1995) 12:463–72.

Google Scholar

24. Nærde A, Sommer Hukkelberg S. An examination of validity and reliability of the Parental Stress Scale in a population based sample of Norwegian parents. PLoS ONE. (2020) 15:e0242735. doi: 10.1371/journal.pone.0242735

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Brannan AM, Heflinger CA. The caregiver strain questionnaire: measuring the impact on the family of living with a child with serious emotional disturbance. J Emot Behav Disord. (1997) 5:212–22.

Google Scholar

26. Abidin RR. Parenting Stress Index: Professional Manual. 3rd ed. Odessa, TX: Psychological Assessment Resources, Inc. (1995).

PubMed Abstract

27. Stewart SL, Hirdes JP, Curtin-Telegdi N, Perlman C, MacLeod K, Ninan A, et al. interRAI Child and Youth Mental Health (ChYMH) Assessment Form and User's Manual. Version 9.3. Washington, DC: interRAI (2015).

Google Scholar

28. Stewart SL, LaRose L, Gleason K, Nicolson R, McKnight M, Knott W, et al. interRAI Child and Youth Mental Health — Developmental Disabilities (ChYMH-DD) Assessment Form and User's Manual. Version 1. Washington, DC: interRAI (2015).

Google Scholar

29. Lau C, Stewart S, Saklofske D, Hirdes J. Development and psychometric validation of the interRAI ChYMH externalizing subscale. Clin Child Psychol Psychiatry. (2020) 26:295–305. doi: 10.1177/1359104520963143

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Stewart S, Hamza C. The Child and Youth Mental Health Assessment (ChYMH): an examination of the psychometric properties of an integrated assessment developed for clinically referred children and youth. BMC Health Serv Res. (2017) 17:82. doi: 10.1186/s12913-016-1970-9

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Lau C, Stewart SL, Saklofske DH, Tremblay PF, Hirdes J. Psychometric evaluation of the interRAI child and youth mental health disruptive/aggression behaviour scale (DABS) and Hyperactive/Distraction Scale (HDS). Child Psychiatry Hum Dev. (2018) 49:279–89. doi: 10.1007/s10578-017-0751-y

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Lau C, Stewart SL, Saklofske DH, Hirdes J. Scale development and psychometric properties of internalizing symptoms: the interRAI Child and Youth Mental Health internalizing subscale. Psychiatry Res. (2019) 278:235–41. doi: 10.1016/j.psychres.2019.06.013

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Li Y, Babcock SE, Stewart SL, Hirdes JP, Schwean VL. Psychometric evaluation of the Depressive Severity Index (DSI) among children and youth using the interRAI Child and Youth Mental Health (ChYMH) assessment tool. Child Youth Care Forum. (2021) 50:611–30. doi: 10.1007/s10566-020-09592-z

CrossRef Full Text | Google Scholar

34. Stewart SL, Hirdes JP. Identifying mental health symptoms in children and youth in residential and in-patient care settings. Healthc Manage Forum. (2015) 28:150–56. doi: 10.1177/0840470415581240

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Stewart SL, Celebre A, Hirdes JP, Poss JW. Risk of suicide and self-harm in kids: the development of an algorithm to identify high-risk individuals within the children's mental health system. Child Psychiatry Hum Dev. (2020) 51:913–24. doi: 10.1007/s10578-020-00968-9

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Stewart SL, Babcock SE, Li Y, Dave HP. A psychometric evaluation of the interRAI Child and Youth Mental Health instruments (ChYMH) anxiety scale in children with and without developmental disabilities. BMC Psychiatry. (2020) 20:390. doi: 10.1186/s12888-020-02785-9

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Stewart SL, Klassen J, Hamza C. Emerging mental health diagnoses and school disruption: an examination among clinically referred children and youth. Exceptionality Educ Int. (2016) 26:5–20. doi: 10.5206/eei.v26i2.7738

CrossRef Full Text | Google Scholar

38. Stewart SL, Morris JN, Asare-Bediako YA, Toohey A. Examining the structure of a new pediatric measure of functional independence using the interRAI Child and Youth Mental Health assessment system. Dev Neurorehabil. (2020) 23:526–33. doi: 10.1080/17518423.2019.1698070

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Stewart SL, Poss JW, Thornley E, Hirdes JP. Resource intensity for children and youth: the development of an algorithm to identify high service users in children's mental health. Health Serv Insights. (2019) 12:1–11. doi: 10.1177/1178632919827926

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Jones R, Hirdes J, Poss J, Kelly M, Berg K, Fries B, et al. Adjustment of nursing home quality indicators. BMC Health Serv Res. (2010) 10:96. doi: 10.1186/1472-6963-10-96

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Kass G. An exploratory technique for investigating large quantities of categorical data. Appl Stat. (1980) 29:119. doi: 10.2307/2986296

CrossRef Full Text | Google Scholar

42. Stewart S, Falah Hassani K, Poss J, Hirdes J. The determinants of service complexity in children with intellectual disabilities. J Intellect Disabil Res. (2017) 61:1055–68. doi: 10.1111/jir.12423

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Christiana J, Gilman S, Guardino M, Mickelson K, Morselli P, Olfson M, et al. Duration between onset and time of obtaining initial treatment among people with anxiety and mood disorders: an international survey of members of mental health patient advocate groups. Psychol Med. (2000) 30:693–703. doi: 10.1017/s0033291799002093

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Wang P, Berglund P, Olfson M, Pincus H, Wells K, Kessler R. Failure and delay in initial treatment contact after first onset of mental disorders in the National Comorbidity Survey Replication. Archiv Gen Psychiatry. (2005) 62:603. doi: 10.1001/archpsyc.62.6.603

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Kessler RC, Angermeyer M, Anthony JC, DE Graaf R, Demyttenaere K, Gasquet I, et al. Lifetime prevalence and age-of-onset distributions of mental disorders in the World Health Organization's world mental health survey initiative. World Psychiatry. (2007) 6:168–176.

PubMed Abstract | Google Scholar

46. Chi P, Li X, Tam C, Du H, Zhao G, Zhao J. Parenting mediates the impact of caregivers' distress on children's well-being in families affected by HIV/AIDS. AIDS Behavior. (2015) 19:2130–9. doi: 10.1007/s10461-015-1104-0

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Crawford A, Manassis K. Familial predictors of treatment outcome in childhood anxiety disorders. J Am Acad Child Adolesc Psychiatry. (2001) 40:1182–9. doi: 10.1097/00004583-200110000-00012

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Platt R, Williams S, Ginsburg G. Stressful life events and child anxiety: examining parent and child mediators. Child Psychiatry Hum Dev. (2015) 47:23–34. doi: 10.1007/s10578-015-0540-4

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Anderson L. Predictors of parenting stress in a diverse sample of parents of early adolescents in high-risk communities. Nurs Res. (2008) 57:340–50. doi: 10.1097/01.NNR.0000313502.92227.87

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Kochanska G, Clark A, Goldman M. Implications of mothers' personality for their parenting and their young children's developmental outcomes. J Pers. (1997) 65:387–420. doi: 10.1111/j.1467-6494.1997.tb00959.x

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Leve L, Kim H, Pears K. Childhood temperament and family environment as predictors of internalizing and externalizing trajectories from ages 5 to 17. J Abnorm Child Psychol. (2005) 33:505–20. doi: 10.1007/s10802-005-6734-7

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Lucia V, Breslau N. Family cohesion and children's behavior problems: a longitudinal investigation. Psychiatry Res. (2006) 141:141–9. doi: 10.1016/j.psychres.2005.06.009

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Choe D, Olson S, Sameroff A. Effects of early maternal distress and parenting on the development of children's self-regulation and externalizing behavior. Dev Psychopathol. (2013) 25:437–53. doi: 10.1017/S0954579412001162

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Curtis S, Thorn P, McRoberts A, Hetrick S, Rice S, Robinson J. Caring for young people who self-harm: a review of perspectives from families and young people. Int J Environ Res Public Health. (2018) 15:950. doi: 10.3390/ijerph15050950

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Kazdin A, Whitley M. Treatment of parental stress to enhance therapeutic change among children referred for aggressive and antisocial behavior. J Consult Clin Psychol. (2003) 71:504–15. doi: 10.1037/0022-006X.71.3.504

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Waals L, Baetens I, Rober P, Lewis S, Van Parys H, Goethals E, et al. The NSSI family distress cascade theory. Child Adolesc Psychiatry Ment Health. (2018) 12:52. doi: 10.1186/s13034-018-0259-7

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Mäntymaa M, Puura K, Luoma I, Latva R, Salmelin R, Tamminen T. Predicting internalizing and externalizing problems at five years by child and parental factors in infancy and toddlerhood. Child Psychiatry Hum Dev. (2011) 43:153–70. doi: 10.1007/s10578-011-0255-

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Anthony L, Anthony B, Glanville D, Naiman D, Waanders C, Shaffer S. The relationships between parenting stress, parenting behaviour and preschoolers' social competence and behaviour problems in the classroom. Infant Child Dev. (2005) 14:133–54. doi: 10.1002/icd.385

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Lin C, Wiley A. The relationship of relative child care and parenting behaviors in fragile families. Child Youth Serv Rev. (2017) 82:130–8. doi: 10.1016/j.childyouth.2017.09.022

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Hirdes J, Poss J, Curtin-Telegdi N. The Method for Assigning Priority Levels (MAPLe): a new decision-support system for allocating home care resources. BMC Med. (2008) 6:9. doi: 10.1186/1741-7015-6-9

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Hirdes J, van Everdingen C, Ferris J, Franco-Martin M, Fries B, Heikkilä J, et al. The interRAI suite of mental health assessment instruments: an integrated system for the continuum of care. Front Psychiatry. (2020) 10:926. doi: 10.3389/fpsyt.2019.00926

PubMed Abstract | CrossRef Full Text | Google Scholar