Young Patients With Persistent and Complex Care Needs Require an Integrated Care Approach: Baseline Findings From the Multicenter Youth Flexible ACT Study

Background: The Multicenter Youth Flexible ACT Study is an ongoing observational prospective cohort study that examines the effects of Youth Flexible ACT (Assertive Community Treatment) on young people with complex care needs who are difficult to engage in traditional (office-based) mental health services. However, a clear and detailed description of this patient group is lacking. In the current paper, we present baseline characteristics and psychosocial outcomes of the Youth Flexible ACT target group and explore the existence of underlying specific patient subgroups.

Methods: Sixteen Youth Flexible ACT teams from seven mental healthcare institutes in the Netherlands participated in the study. Research participants were monitored for 18 months and administered questionnaires measuring psychiatric- and social functioning every 6 months, yielding four measurements. Baseline data were obtained from 199 adolescents, their mental health workers, and parents/carers. Latent Class Analysis based on HoNOSCA scores (measuring psychosocial and daily functioning) was conducted to identify underlying subgroups.

Results: The target group of Youth Flexible ACT mainly consisted of patients older than 15 years of age with a history of (specialized) mental healthcare. They face many complex problems, including trauma; developmental, mood, and anxiety disorders; and problems with school attendance, family life, and peer relationships. Other frequently reported difficulties were substance misuse, the involvement of the legal system or police, problems with intellectual functioning, and personal finance. Patients were classified into four distinct classes: the “internalizing,” “externalizing,” “non-specific,” and the “overly impulsive” subgroup. Each subgroup had its unique pattern of difficulties and focus, respectively, high levels of depression and anxiety, disruptive behavior, unspecific difficulties, and substance misuse.

Conclusions: As expected, patients in Youth Flexible ACT experienced many severe problems, rendering them vulnerable to fragmented and, thus, ineffective care. Our findings underscore the need for an integrated care approach with a multidisciplinary team of skilled professionals that can bridge these wide-ranging psychosocial problems, as each class of participants experienced a different set of difficulties. Youth Flexible ACT teams need to adjust their care services accordingly.

Introduction

It is estimated that about 5% of Dutch children and adolescents have mental illnesses leading to functional impairment and hindered development (1, 2). For the majority, these problems are often temporary. For a small subgroup, however, these problems can be rather severe and persistent (1–5). Adolescents in need of longer-term mental healthcare usually face complex psychiatric and social problems in everyday life, including difficulties with education, employment, peer relationships, family, housing, finances, health, substance abuse, and the criminal justice system (2, 6–10). These young people are often raised in families unable to provide proper education and/or have family members who suffer from psychiatric, financial, or addiction problems (6, 7, 11, 12). Multiple life challenges and harsh living conditions affect their psychiatric difficulties and vice versa (8).

Unfortunately, appropriate and accessible care for adolescents with complex psychiatric and social problems is sparse. The well-known traditional outpatient (office-based) care and psychiatric inpatient care offer limited services to this specific patient group (13–16). Limited access and commitment to care increase drop-out (17–20). Reasons for treatment disengagement include the lack of stability in daily life, difficulty trusting services, growing tired of services, treatment discontinuity, and a fragmented healthcare system (11, 17–25). When adolescents face complex care needs, they often need multiple mental health and social services for their various problems. To navigate these various services requires an active role and a fairly high level of knowledge of the healthcare system. This is often too challenging for these adolescents and their parents, resulting in fragmented and discontinued care. Integrated care approaches are needed to bridge these wide-ranging problems that these adolescents and their families face and provide alternative options to psychiatric inpatient and traditional office-based care.

In the Netherlands, Youth Flexible Assertive Community Treatment (Flexible ACT) teams were set up to meet the needs of children and adolescents with complex care problems and to tackle the problems of fragmented care services. Youth Flexible ACT is the youth variant (0–24 years of age) of Flexible ACT for adults. It is the Dutch adaptation and elaboration of Assertive Community Treatment (ACT), which originated in the United States in the 1970s (26). Flexible ACT is a client-centered service delivery model in which integrated teams provide long-term assertive outreach care consisting of both treatment for psychiatric symptoms and practical assistance with daily living needs, rehabilitation, and recovery support. Youth Flexible ACT is particularly focused on collaboration with adolescents, families, and their (in)formal networks. This results in shared goals aimed at improving the functioning in multiple life domains, improving patients' participation in the community, and enhancing their quality of life (7, 9). A multidisciplinary team of professionals delivers a complete range of services on a continuum of care. Nowadays, ~60 teams are active or under development in the Netherlands (27). A detailed portrayal of Youth Flexible ACT is outlined in the Youth Flexible ACT model description (7) and our study protocol (28).

In our ongoing Multicenter Youth Flexible ACT Study (28), we examine the effect of the care model by investigating (1) improvement in treatment outcomes throughout the Flexible ACT care and (2) associations between (elements of) Youth Flexible ACT model fidelity and treatment outcomes. Theoretically, Youth Flexible ACT targets a specific patient group with specific care needs. As the first step in this line of research, this study investigates whether characteristics of patients in our study population match with the intended target population as specified by the model description (7). Thus, the current paper provides a detailed description of adolescents in 16 Youth Flexible ACT teams in the Netherlands. Investigating the Youth Flexible ACT target group will contribute to organizing, adjusting, and developing appropriate care for these adolescents. Specifically, the aim of this paper is to characterize the composition of the study population. We describe baseline characteristics and psychosocial outcomes of the overall population of 199 participants and explore potential patient subgroups.

Materials and Methods

Design

The Multicenter Youth Flexible ACT Study is an ongoing observational prospective cohort study (2017–2021) of 16 Youth Flexible ACT teams from 7 mental healthcare institutes in the Netherlands. Patients were monitored for 18 months and administered questionnaires every 6 months, yielding 4 measurements (T0, T1, T2, T3). In this current paper we described cross-sectional data from the baseline measurement (T0).

Participants

According to the Youth Flexible ACT model description (7) young people <24 years of age eligible for Youth Flexible ACT if they adhere to the following criteria. They (1) are diagnosed with a mental health disorder (or presumptive diagnosis); (2) experience difficulties in multiple areas of daily life (for example, problems with education, employment, peer relationships, housing, personal finance, health, substance abuse, and issues with the criminal justice system); (3) face family system problems and/or parenting issues; (4) have difficulty accessing and remaining in traditional outpatient care or the traditional care proves to be unfruitful; and (5) live in the district of the Youth Flexible ACT team. Additionally, the following research inclusion criteria were used: patients had to be between 12 and 24 years of age, have sufficient knowledge of the Dutch language, and provide written informed consent (along with parent/caregivers' consent).

Setting and Data Collection

The 16 Youth Flexible ACT teams are located in both the city and the rural areas in the Netherlands and are working according to the Youth Flexible ACT model guidelines (7). An official audit was performed by the Centre for Certification ACT and Flexible ACT (CCAF) to determine the degree to which each team complies with the Youth Flexible ACT model. Fidelity scores showed that all teams had implemented the model successfully. We will describe these model fidelity scores more throroughly in our future studies.

Team members asked eligible adolescents to participate in the study during the intake process. Participants, their parents/carers, and the mental healthcare worker were asked to complete a baseline measurement consisting of questionnaires within 12 weeks after signing informed consent. The baseline measurement was completed during a regular appointment by a familiar mental health worker or by patients independently in their own time. Both paper and online versions were available, although online versions were preferred to minimize missing data. Adolescent participants received a remuneration of €10. An online data system was used to collect the data. Confidentiality of the data was guaranteed through a two-factor authentication login procedure. Each institution had a unique digital environment. Before conducting baseline assessments, mental health workers were trained in administering the questionnaires. In particular, mental health workers received a HoNOSCA training based on the official HoNOSCA training (29, 30).

Study Outcome Measures

The overall baseline characteristics and psychosocial outcomes were collected using questionnaires (Table 1) that assess general psychological functioning, specific diagnostic characteristics, and daily functioning of the participants. The employed set of questionnaires together reflect the multiple life domains in which Youth Flexible ACT operates. The questionnaires and the calculations of scores are described in detail in our study protocol (28). Additionally, sociodemographic and clinical characteristics (age, sex, clinical diagnoses) were collected from electronic patient records and via a sociodemographic questionnaire completed by mental health workers. To check whether research participants were representative of the Youth Flexible ACT population, age, sex and diagnoses of patients registered with the same Youth Flexible ACT teams during the same inclusion period were also collected. Furthermore, we conducted Latent Class Analysis (LCA) to identify the existence of specific patient subgroups based on their psychosocial- and daily functioning, as measured with the HoNSOCA (Health of the National Outcome Scales for Children and Adolescents) (30). We selected 10 out of the 13 HoNOSCA items that display the multiple life domains addressed by Youth Flexible ACT.

TABLE 1

Table 1. Overview of outcome assessments at baseline.

Participant Response Rates

During a 20-month enrolment period (October 2016–June 2018), 199 eligible patients, hereafter referred to as the monitoring sample, gave informed consent to participate, and completed baseline questionnaires on average 2.6 months after the start of Youth Flexible ACT care (SD = 2.38; range 0–14). Almost three quarters of the patients (74.9%) completed the baseline assessment within 16 weeks after starting with Youth Flexible ACT. In addition, age, sex, and diagnoses were obtained from 1,034 patients registered with the same Youth Flexible ACT teams during the same inclusion period, hereafter referred to as the non-monitoring sample. Demographic variables showed no significant differences between the monitoring and non-monitoring samples in terms of age (F_{(1, 1231)} = 0.99, p = 0.32) and sex (χ² (1) = 1.82, p = 0.18). Regarding clinical diagnoses, only anxiety disorders were more prevalent in the monitoring sample than in the non-monitoring sample (24.6 vs. 15.6%, respectively, χ² (1) = 9.27, p = 0.002, φ = −0.09). Yet, as the effect size was small, this difference was not substantial. This is an indication that the monitoring sample reflects the wider Youth Flexible ACT population in the Netherlands.

Data Analysis

LCA was conducted to explore the patient composition, as measures with the HoNOSCA, using the statistical package Mplus version 7.2 (31). Missing values were 0 (4 items), 1 (4 items), 2 (1 item), and 8 (1 item). The Full Information Maximum Likelihood (FIML)-estimator was used as a default procedure to handle missing values. With this procedure, all available information in the data was used to estimate latent classes. Furthermore, patients were clustered within the 16 teams. Therefore, we used the TYPE = COMPLEX procedure in Mplus to correct parameter estimates for the dependency of patients within teams. Data management, descriptive statistics, and other statistical analyses were performed using SPSS version 25 (32).

Latent Class Analysis With Mplus

The 10 selected items of the HoNOSCA were used as indicators to classify patients into distinct subgroups (33). The responses to these items were dichotomized into “no problem/no action required” (scores 0 and 1) and “mild or severe problem/action required” (scores 2, 3, and 4) to facilitate model estimation (29). The starting moment of LCA is a 1-class solution followed by 2−, 3−, …, k+1 class solutions. Four criteria were used to identify the number of k classes. The first criterion is based on one or more information-theoretic criteria, like the Bayesian Information Criterion (BIC) (34), Akaike Information Criterion (AIC) (35), or the Sample Size Adjusted BIC (SSABIC) (36). We used the SSABIC as a metric for model performance, as it was shown to outperform both the BIC and AIC for small sample sizes (N < 200) (37). Lower SSABIC-values indicate improvement of model fit. If SSABIC-values for a model with k+1 classes increase, then a model with k classes is optimal. The second criterion is based on the classification quality of a model. High posterior probabilities indicate how well individuals are classified into their class, and high values of the entropy measure (a standardized index based on all posterior probabilities varying between 0 and 1) (38) are preferred. The third criterion concerns the likelihood ratio tests; in our case, we used the Bootstrap Likelihood Ratio Test (BLRT). Non-significant p-values (>0.05) of the k+1 class solution means that the significant k-solution (with p < 0.05) is superior. The fourth criterion for a k-class solution is based on practical and theoretical considerations (39). After deciding on a k-class solution, we labeled the classes based on the patients' HoNOSCA response patterns. The patients were allocated to one of the k classes based on their highest subgroup posterior probability (probability of class membership for each patient).

Follow-Up Analyses With SPSS

We then tested whether the identified subgroups differed significantly on clinical characteristics and psychosocial outcomes using ANOVA for interval variables and Fisher's exact test for the nominal variables, followed by Bonferroni corrected post-hoc tests to test which classes differed from each other. To correct class uncertainty, the individual posterior probabilities were used as a weight variable in additional analyses.

Results

Baseline Characteristics of the Youth Flexible ACT Sample

The monitoring sample (n = 199) comprised 101 girls (50.8%) and 98 boys (49.2%) with a mean age of 18.6 years (SD = 2.49; range 12–24 years of age). Most patients were between 15 and 22 years of age (87.9%), and a large group of the patients was 18 years of age or older (69.3%; range 12–24 years of age). The participants were mainly born in the Netherlands (95%). On average, teams indicated that 17.0% of the caseloads had a parent of foreign origin (non-Dutch). The most frequent clinical diagnoses (Table 2) were trauma and stressor-related disorders (27.1%; of which 80.9% were posttraumatic stress disorders), mood disorders (27.1%), autism spectrum disorder (26.1%), anxiety disorders (23.6%), and attention deficit hyperactivity disorder (ADHD; 21.6%). Most patients had two or more clinical disorders (62.8%) and faced additional problems (62.3%), the so-called V-code or “other conditions that might be a focus of clinical attention” in Diagnostic and Statistical Manual of Mental Disorders (DSM5) (40).

TABLE 2

Table 2. Clinical diagnosis and care characteristics of the monitoring sample (n = 199).

A large group of patients was referred by a specialized mental health care organization (45.8%), mostly within the same organization that deployed the Youth Flexible ACT team (70.3%). Accordingly, most patients received care from a specialized mental health care organization (46.3%) in the 6 months preceding the Youth Flexible ACT care. A small percentage (8%) did not receive care within the 6 months before the Youth Flexible ACT care. A few patients (8%) already received Youth Flexible ACT care in the past. In 11.6% of the cases, participants had a family member who also received Flexible ACT care.

Subgroups Within the Youth Flexible ACT Sample

We aimed to explore the existence of underlying subgroups within the overall Youth Flexible ACT population. LCA revealed four subgroups of patients (n = 199; Table 3). The SSABIC-values decreased from 2,313 to 2,239 for a 4-class solution and then increased again for a 5-class solution, indicating that a 4-class solution is preferred. The entropy-values of the 3-, 4, and 5- class solutions did not show large differences (0.86 −0.88), and the posterior probabilities were high (>0.90), except for class 4 in the 4-class solution and classes 4 and 5 in the 5-class solution. Deciding on a specific class-solution based on entropy and posterior probabilities was difficult. The Bootstrap Likelihood Ratio Test (BLRT) showed a p-value of 0.113 for the 4-class solution and a p-value of 0.500 for the 5-class solution, indicating a 3-class solution is preferred for statistical reasons. The decision between a 3-class (BLRT) and a 4-class solution (SSABIC) is, therefore, rather arbitrary. The first three classes of the 4-class solution are the same as the three classes of the 3-class solution. We, therefore, first described these classes (Figure 1) before adding a fourth class. The first class (n = 87) had a high prevalence (>80%) of emotional symptoms, family life problems, peer relationship problems, and poor school attendance. The second class (n = 65) had a high prevalence of disruptive/aggressive behavior, peer relationship problems, emotional symptoms, and family life problems. The most important difference between class 1 and class 2 concerned disruptive/aggressive behavior, with a prevalence of 0% in class 1 and 100% in class 2. We labeled class 1 as the “internalizing” subgroup and class 2 as the “externalizing” subgroup. The third class, labeled as the “non-specific” subgroup (n = 32), had a prevalence of 4.6–52.3% across the 10 items and had a relatively high prevalence of family life problems, overactivity/attention difficulties, emotional symptoms, and peer relationship problems. The fourth class, labeled as “overly impulsive” (n = 15), had a high prevalence of family life problems, overactivity/attention difficulties, and disruptive/aggressive behavior. All members of this class scored maximally on substance misuse. This latter behavior sets this class apart from the first three. We, therefore, opted for the 4-class solution.

TABLE 3

Table 3. Latent class models with up to 5 classes.

FIGURE 1

Figure 1. Probability of having a mild/severe problem per class on 10 HoNOSCA items.

Differences Between Subgroups

We next examined differences between the classes in age, sex, and clinical diagnoses. Statistical results are described in Table 4, and any differences described in this paragraph were all significant at p < 0.05 after Bonferroni correction. A one way ANOVA indicated no significant differences in age between the four classes, F_{(3, 181)} = 0.62, p = 0.602. For the other (nominal) variables, Fisher's exact test was used. Post-hoc tests with Bonferroni correction were applied only if the p-value of Fisher's exact test was significant. Latent class membership was associated with sex and the prevalence of anxiety and mood disorders, disruptive, impulse-control, and conduct disorders, ADHD, and psychotic disorders. The “internalizing” subgroup consisted mostly of girls (64.4%) who had to deal primarily with anxiety and mood disorders (63.2%). The “overly impulsive” subgroup had a higher prevalence of ADHD (53.3%) compared to the “internalizing” subgroup (13.8%) and the “non-specific” subgroup (15.6%) and a higher prevalence of conduct disorders (26.7%) compared to the “internalizing” subgroup (3.4%). In contrast to the other subgroups, the “non-specific” subgroup was characterized by a more or less even pattern of problems across all domains without clear highlights. This subgroup had the highest prevalence of psychotic disorders (15.6%) compared to the other groups, although the absolute number was still relatively low.

TABLE 4

Table 4. Mean age, sex, and clinical diagnoses of the overall sample and per class.

Life Domains Outcomes for the Youth Flexible ACT Sample and Its Subgroups

To provide a thorough overview of the Youth Flexible ACT sample and to further clarify its subgroups, we measured a broad range of psychosocial and daily functioning outcomes. We allocated these outcomes into the psychological functioning, daily functioning, and intellectual functioning domains. Below, we first provide the descriptives of the total population for each theme and next describe differences between the classes. Numerical results (descriptives and statistical results) are presented in Table 5 (patient-reported), Table 6 (parent-reported), and Table 7 (clinician-reported). In Table 5, interval variables were tested with one-way ANOVA and nominal variables with Fisher's exact test. Both tests were followed by post-hoc tests with Bonferroni correction if the main results are significant. All differences described in the text below were significant at p < 0.05 after Bonferroni correction. As we did not gather enough responses from parents/carers for each subgroup, additional analyses to compare the classes on parent-reported outcomes could not be performed.

TABLE 5

Table 5. Descriptives of patient-reported outcomes.

TABLE 6

Table 6. Descriptives of parent-reported outcomes.

TABLE 7

Table 7. Descriptives of the HoNOSCA completed by mental health workers (n = 199 patients).

Psychological Functioning

Patient-reported

The average total scores of the patient-reported mental health questionnaires SDQ (psychosocial well-being) (41, 42), CDI-2 (depressive symptoms) (43, 44), and PQ-16 (psychosis risk screening) (45, 46) were substantially higher compared to norm scores of the general population [SDQ: (42, 47–49); CDI-2: (43, 50); PQ-16: (51)] and comparable to the scores of patients in a clinical setting [SDQ: (47, 49, 52, 53); CDI-2: (43, 50, 54); PQ-16 (45, 46, 55)].

Most of the “internalizing” subgroup scored in the clinical range of depression (74.7%; CDI-2 ≥14) and had a significantly higher average total depression score (CDI-2) and emotional difficulties score (emotional problems subscale; SDQ) compared to the other subgroups. The “internalizing” subgroup showed more emotional problems, whereas the “externalizing” and “overly impulsive” subgroups showed more conduct problems (conduct problems subscale; SDQ). On average, all subgroups indicated that their psychosocial problems interfere with their everyday life; however, this problem was more severe for the “internalizing” subgroup compared to the “non-specific” and “overly impulsive” subgroups (SDQ impact score).

Parent-reported

The average total score on the parent-reported SDQ (SDQ-P) (41) was comparable to the scores of clinical samples (49, 52, 56). Most parents (69.0%) observed an increased level of psychosocial problems in their child (cutoff ≥14).

Clinician-reported

Regarding the HoNOSCA scores, in many cases (80.9%), mental health workers reported the item “Problems with emotional and related symptoms” as “problematic” (mild problem to very severe problem). In more than half of the cases (56.7%), the item “Problems with overactivity, attention, or concentration” was scored as “problematic” while the item “Problems with disruptive, antisocial, or aggressive behavior” was “problematic” in fewer than half of the cases (39.2%). “Problems with alcohol, substance/solvent misuse,” “Non-accidental self-injury,” and “Problems associated with hallucinations, delusions, or abnormal perceptions” were less frequently reported but still evident (respectively 26.7, 21.5, and 20.1%). HoNOSCA scores per class were described previously. They are displayed in Figure 1.

Daily Functioning

Daytime activities

In summary, 34.7% of the adolescents in the monitoring sample reported going to school, 19.1% reported a combination of school and work, and 12.6% were employed. Nearly all of the adolescents who went to school were in secondary education (94.4%), with most of them (69.3%) in (pre)vocational education. However, about a third of the adolescents (33.7%) did not have school or employment, and more than half (59.7%) of these adolescents did not have any form of organized daily activities. Just above half of the monitoring sample (58.3%) rated their satisfaction with school and/or employment situation higher than six out of 10. In addition, mental health workers indicated problems with school attendance in 62.3% of the cases, as scored by the HoNOSCA. Besides the school and/or employment situation, most participants (64.8%) rated their satisfaction with leisure time higher than six out of 10. The “non-specific” subgroup was more satisfied with leisure time compared to the “internalizing” subgroup (84.4 vs. 50.6%).

Family life and housing situation

Regarding their housing situation, most adolescents were living with both their biological parents (34.2%), with one biological parent (28.1%), or with blended families (5.0%). Almost one-fifth of the adolescents (18.6%) were in assisted living. Others lived on their own, with friends, or had another form of housing situation. Mental health workers indicated that, on average, a small percentage of their caseload (8.0%) lived in a residential facility (including psychiatric hospitalization). Adolescents themselves were slightly satisfied with their housing situation, with 69.8% scoring above 6 out of 10. The “internalizing” subgroup was more satisfied with their housing situation compared to the “externalizing” subgroup (78.2 vs. 58.5%). Although adolescents were quite satisfied with their housing situation, mental health workers noticed “Problems with family life and relationships” in most cases (83.4%; HoNOSCA). In addition, a large group of parents (57.1%) experienced a considerable degree of parenting stress (cut-off ≥22; PSQ-S) (57, 58). The reported parenting stress total score was substantially higher compared to norm scores for a non-clinical sample (57) and comparable with scores for a clinical sample (59). Moreover, almost half of the parents (46.4%) reported experiencing psychological distress according to Dutch norms (cutoff ≤60; MHI-5) (60).

Peer relationships

On average, adolescents reported experiencing enough quality of social interaction with peers [Social Support and Peers' subscale of the Kidscreen-52; (61)] compared to the norm scores from both the general population and clinical samples (62, 63). Quite a large group (75.3%) scored above six out of 10 and indicated satisfaction with their peer relationships. Although on average, each subgroup experienced enough quality of interaction with their peers, the “overly impulsive” and “non-specific” subgroups were more satisfied compared to the “internalizing” and “externalizing” subgroups. A major contrast to these results is that mental health workers noted “Problems with peer relationships” of the HoNOSCA as “problematic” in 74.8% of the cases.

Personal finance

One-fifth of the total sample (21.6%) reported having debts or financial problems in the past 6 months. This percentage is similar to that for the subgroups.

Legal system and police

Almost a fourth of the adolescents (23.1%) reported having been in contact with the legal system/police in the past 6 months. The “externalizing” and “overly impulsive” subgroups (respectively 35.4 and 46.7%) reported having been more frequently in contact with the legal system/police compared to the “internalizing” subgroup (13.8%). Mental health workers reported that only a small number of their caseload (0.6%) was in juvenile detention.

Health-related quality of life

On average, adolescents reported satisfactory quality of life on the KIDSCREEN-10 according to European norms (61, 63). However, when examining the frequencies, almost half of the adolescents (47.7%) indicated experiencing poor quality of life (cut-off <40). Additionally, more than half of the parents (57.7%) indicated that their child had a poor quality of life (cut-off <40) (61).

Empowerment

Many adolescents reported having control over decisions and actions (79.8%), as scored on the EMPO 3.1 (58, 64). The remaining 20.2% indicated that empowerment needs to be a subject of attention during the treatment (cut-off <35). No significant difference in the quality of life and empowerment were observed between classes.

Intellectual Functioning

Intellectual functioning can be divided into Borderline Intellectual Functioning (IQ 70–85) and Mild Intellectual Disability (IQ 50–69) (40). Intelligence Quotient (IQ) scores were collected in 41.7% of the cases (21.6% <2 years; 20.1% >2 years ago; 20.6% missing scores). The collected scores showed an average IQ score of 92.6 (SD = 17.89). About one third (33.8%) had an IQ score below 85. Mental health workers indicated that ~18.6% of the total sample had to deal with (suspected) below average intellectual functioning. No significant differences between the classes were observed.

Discussion

In this paper, we aimed to characterize adolescents in Youth Flexible ACT care. We presented the baseline findings of our prospective cohort study, including sociodemographic-, clinical characteristics, and psychosocial outcomes. To our knowledge, this is the first characterization of the Youth Flexible ACT target group.

The Youth Flexible ACT target group consisted mainly of adolescents aged 15 and 22 who already received prior (specialized) mental healthcare. As expected, they face many severe psychiatric and social problems associated with a significant amount of trauma and developmental, mood, and anxiety disorders. Their development in multiple life domains is hindered, especially since one third does not attend a school or a job, and almost all adolescents show problems with family life and peer relationships. About half reported experiencing poor quality of life. Other frequently reported difficulties are substance misuse, the involvement of the legal system or police, problems with intellectual functioning, and personal finance. On a positive note, most adolescents indicated feeling empowered: the feeling that they can control their decisions and actions, which affect their health (58).

Our results showed that the Youth Flexible ACT target group could be divided into four subgroups: the (1) “internalizing,” (2) “externalizing,” (3) “non-specific,” and (4) the “overly impulsive” subgroup. The “internalizing,” “externalizing,” and “overly impulsive” subgroups showed severe problems in multiple life domains, each with its unique difficulties. The “non-specific” subgroup displayed a milder and diffuse pattern of problems across all domains without clear highlights. This subgroup had the highest percentage of psychotic disorders and autism spectrum disorders. The existence of this generic subgroup could be explained by problems remaining under the radar for some adolescents at the start of Youth Flexible ACT. While some adolescents apply for Youth Flexible ACT on their own volition, other adolescents are registered by other concerned parties, such as parents, caregivers, educators, health care professionals, or law enforcement. In these latter cases, adolescents probably may not (yet) recognize or acknowledge the problems they encounter, which can (temporarily) result in “meddling care” (intervention without the clear consent of the person). In these cases, time is initially devoted to building and maintaining trust and motivation, while problem diagnostic is secondary. Another explanation for this generic subgroup comprising a relatively large number of patients with mild psychotic symptoms could be the presence of “early intervention in psychosis” teams (65) in the region. These teams work with young adults who deal with psychosis. In this case, young adults with somewhat milder psychotic symptoms will then get treatment and support from Youth Flexible ACT teams or traditional outpatient care.

Since each subgroup has its own set of difficulties, appropriate care should be embedded in the Youth Flexible ACT team care services. For example, expert knowledge about developmental disorders and psychosis must be available in the teams. Additionally, family and pedagogical interventions and addiction interventions are highly needed. Considering the disruptive behavior and impulse control problems, interventions targeting self-control, aggression regulation, social and problem-solving skills are relevant. Although these services are already mentioned in the Youth Flexible ACT model description (7), our findings reiterate the importance of their presence in the teams. Taken together, our findings underscore the need for an integrated care approach. These teams should consist of a team of multidisciplinary professionals who can address the multiple needs across psychiatric and social services. For future research, it would be interesting to explore different improvement trajectories of these subgroups through longitudinal tracking of psychosocial outcomes.

Findings in Perspective

In general, an integrated care approach to treating adolescents with persistent complex care needs is vital. Although the Youth Flexible ACT model is prominent in the Netherlands (27), other youth-friendly mental health and integrated services are on the rise in other countries (14, 66–68). International examples include IMYOS (Intensive Mobile Youth Outreach Service) in Australia, Jigsaw in Ireland, Forward Thinking Birmingham in the United Kingdom, and ACCESS Open Minds in Canada (14, 21, 24, 68, 69). The patient group of the IMYOS is similar (e.g., poor school attendance, problematic upbringing, history of mainstream mental health services before referral) yet slightly more focused on “high risk” youth compared to the Youth Flexible ACT patients (70, 71). In contrast to the Youth Flexible ACT approach, most other community-based centers provide (mental health) services for youth in the primary care setting or focus specifically on particular diagnoses, often psychotic disorders. The Youth Flexible ACT model is unique because it encompasses a multi-agency care approach delivering long-term assertive outreach care for wide-ranging problems. Multidisciplinary teams provide psychiatric interventions in a specialized mental health care setting in close collaboration with adolescents (and their families) and professionals from other youth services, thereby delivering a range of services while maintaining continuity of care. In subsequent (inter)national studies, it would be interesting: (1) to see how the comparable models and target groups relate to each other and (2) to establish the benefits and relevance of integrated care.

In addition, most patients of our research sample were between 15 and 22 years of age, also referred to as transitional age youth. Continuity of care in transitional age youth has a high priority in the global mental healthcare. In the Netherlands, the Flexible ACT model for adults is an established model, whereas the youth version is a more recent development. Our findings show that the Youth Flexible ACT caseload is substantially different compared to the Adult Flexible ACT patient group, which predominantly focuses on treating severe and enduring mental disorders, such as psychotic-, personality-, and substance use disorders (72–74). The transition of 18+ Youth Flexible ACT patients to Adult Mental Health Services (AMHS) is complicated because of the variable organization of service provision and the limited access to the required services in AMHS (67, 68, 75, 76). By incorporating a broad age group, Youth Flexible ACT attempts to prevent transitional age youth from falling through the gap between child and adolescent mental health services and AMHS. In addition, as the Youth Flexible ACT model is composed of multiple professionals and areas of expertise, it addresses the age-related concerns of both young children and young adults.

Furthermore, we note that 17% of youths had at least one parent of foreign origin. This is slightly lower than the Dutch national average of 27% (77). This could be due to the fact that this percentage is generally lower in rural areas, and that our teams are predominantly located in rural areas. Alternatively, it might be possible that youths with a migration background have additional difficulty in navigating the Dutch mental health care system, or potentially seek less help due to cultural differences, in turn leading to underrepresentation in the specialized mental health care settings and thus Youth Flexible ACT care. However, there are no current national figures that outlines the number of children and adolescents from either migrant or Dutch origin in specialized youth mental health care. Yet, the percentage of migrants in the Youth Flexible ACT sample is comparable with that of other studies with young patients from clinical populations in the Netherlands (46, 50, 54).

Strengths and Limitations

An important strength of the current study is its observational and naturalistic character, which improves the external validity because the data closely represents everyday practice. The study examined Youth Flexible ACT in daily practice across multiple Youth Flexible ACT teams and mental health care centers throughout the Netherlands. In addition, the study included a relatively large study sample representative of the Youth Flexible ACT population in the Netherlands, which is a difficult patient group to study. Furthermore, using both a broad set of questionnaires (both psychiatric and social functioning) and data from multiple informants, this study provides a comprehensive view from multiple vantage points. This in-depth framework can be used by other researchers to perform comparisons.

Several limitations of our study should be noted as well. First, mental health workers were not always able to obtain a complete overview of the problems for some patients due to the difficult-to-engage nature of the patient group. This can lead to underestimation of diagnoses and problems. Second, the downside of including a broad age group is that the employed questionnaires have not been extensively validated with older adolescents. This should be considered when interpreting the results. Third, our results showed that problems related to psychosocial well-being and peer relationships were reported more often by clinicians and parents than by patients themselves. This is in line with other observations that parents and mental health workers generally identify more problems than the patients themselves (47, 52, 78–80). Finally, although the study population was sufficiently large to perform LCA analysis, it should be noted that the individual subgroup sizes were sometimes relatively small (minimally n = 15). Finding statistically significant differences in small sample sizes is only possible if the differences have large effect sizes. As we found multiple statistically significant effects (with sufficient power to describe our study population), this indicates that our effect sizes were large. For finding significant differences with medium or small effect sizes some subgroups were too small indicating that replication of these results with larger samples is recommended before policies regarding targeted and specialized care toward one of the subgroups is to be set in place.

In conclusion, the multitude of psychiatric and social problems found in the current study highlights the vulnerability of the population requiring specialized forms of care that can bridge the wide-ranging problems. We await the results of our longitudinal study that will show whether Youth Flexible ACT is indeed successful in engaging and helping adolescents with persistent complex problems.

Data Availability Statement

The datasets generated for this article are not readily available because the datasets analyzed for this study will not be made publicly available due to ethical, legal, and privacy restrictions. Requests to access the datasets should be directed to m.broersen@ggzoostbrabant.nl.

Ethics Statement

The study involving human participants was reviewed and approved by the Trimbos Ethics Committee. Written informed consent to participate in this study was provided by adolescents and parents or legal guardians.

Author Contributions

All authors contributed to the article and approved the submitted version. MB was responsible for data collection, data management, and reporting the study results. MB and AV were responsible for conducting and reporting data analysis. NF, DC, and HK read the manuscript and provided suggestions for improvement. NF, DC, and HK also served as supervisors.

Funding

This study received external funding from the Olim Foundation, a non-profit mental health foundation to improve the quality of mental healthcare in the east part of North Brabant, the Netherlands. The Olim Foundation provided the funding after reviewing and approving the study protocol. The Olim Foundation played no role in the collection, analysis, and interpretation of data, or the writing of manuscripts. Additionally, each of the six participating institutions paid a small contribution (€7,600) to cover research materials, including an online data managing system.

Disclaimer

All opinions presented in this article belong to the authors alone.

Conflict of Interest

HK was a board member of the Centre for Certification ACT and Flexible ACT (CCAF). The CCAF executed peer audits using model fidelity scales to ensure the quality of ACT and Flexible ACT.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors thanked the participated mental healthcare institutions (Accare, GGZ Oost Brabant, GGZ Noord-Holland-Noord, Kenter Jeugdhulp, Lucertis, Mondriaan-Gastenhof, and Triversum) for valuable collaboration in collecting baseline measurements.

Abbreviations

ACT, Assertive Community Treatment; ADHD, Attention Deficit Hyperactivity Disorder; AIC, Akaike Information Criterion; AMHS, Adult Mental Health Services; ANOVA, Analysis of variance; BIC, Bayesian Information Criterion; BLRT, Bootstrap Likelihood Ratio Test; CCAF, Centre for Certification ACT and Flexible ACT; CDI-2, Child Depression Inventory 2; DSM, Diagnostic and Statistical Manual of Mental Disorders; EMPO 3.1, Empowerment questionnaire 3.1; HoNOSCA, Health of the National Outcome Scales for Children and Adolescents; IMYOS, Intensive Mobile Youth Outreach Service; IQ, Intelligence Quotient; LCA, Latent Class Analysis; MHI-5, Mental Health Inventory 5; PSQ-S, Parenting Stress Questionnaire – short form; PQ-16, The Prodromal Questionnaire 16; SDQ, The Strengths and Difficulties Questionnaire; SDQ-P, Strengths and Difficulties Questionnaire – Parent version; SPSS, Statistical Package for the Social Sciences; SSABIC, Sample Size Adjusted BIC; STROBE, Strengthening the reporting of observational studies in epidemiology.

References

1. GGZ Nederland. Jeugd-ggz: Investeren in de toekomst. Ambities voor 2011–2014. (2011). Available online at: https://www.canonsociaalwerk.eu/1928_MOB/jeugdggzinvesteren-in-de-toekomst.visiedoc.pdf (accessed January 13, 2020).

2. Sytema S, Gunther N, Reelick F, Drukker M, Pijl B, van't Land H. Verkenningen in de kinder- en Jeugdpsychiatrie. Een bijdrage uit de psychiatrische casusregisters. Utrecht: Trimbos-instituut (2006). Available online at: https://www.trimbos.nl/docs/0ce13506-b12c-4ff3–9372-8d2c0f743fd2.pdf (accessed January 13, 2020).

3. Veerbeek M, Knispel A, Nuijen J. GGZ in tabellen 2013 - 2014. Utrecht: Trimbos Instituut (2015). Available online at: https://www.trimbos.nl/docs/cfe358cd-59cc-4033-ac68–50711f835768.pdf (accessed January 13, 2020).

Google Scholar

4. Delespaul PH, Consensusgroep EPA. Consensus over de definitie van mensen met een ernstige psychische aandoening (EPA) en hun aantal in Nederland. Tijdschrift voor Psychiatrie. (2013) 55:427–38.

PubMed Abstract | Google Scholar

5. Delespaul PH. (Flexible) ACT is voor ernstige psychiatrische aandoening (EPA). In: van Veldhuizen R, Polhuis D, Bähler M, Mulder N, Kroon H, editors. Handboek (Flexible) ACT. Utrecht: De Tijdstroom (2015). p. 20–30.

6. De Koning N, Storm A. (Flexible) ACT voor kinderen en jongeren. In: van Veldhuizen R, Polhuis D, Bähler M, Mulder N, Kroon H, editors. Handboek (Flexible) ACT. Utrecht: De Tijdstroom (2015). p. 435–41.

Google Scholar

7. Hendriksen-favier A. Modelbeschrijving FACT Jeugd. Herziene versie. Utrecht: Trimbosinstituut (2013). Available online at: https://ccaf.nl/wp-content/uploads/sites/2/2016/02/Hendriksen-Favier-A-2013-Modelbeschrijving-FACT-Jeugd-Utrecht-Trimbosinstituut.pdf (accessed February 5, 2020).

Google Scholar

8. Delespaul PH. Graag meer onderzoek naar ernstige psychische aandoeningen. Tijdschrift voor Psychiatrie. (2019) 61:443–4.

PubMed Abstract | Google Scholar

9. Storm A, Frieswijk N, Hendriksen-Favier A. FACT als organisatiemodel voor langdurig zorgafhankelijke kinderen en jongeren. Kind en Adolescent Praktijk. (2013) 12:52–61. doi: 10.1007/s12454-013-0018-1

CrossRef Full Text | Google Scholar

10. Van Houtem-Solberg D, Chatrou E, Werrij M, van Amelsvoort T. Jeugd-F-ACT: problematiek van een bijzondere populatie in kaart gebracht. Tijdschrift voor Psychiatrie. (2015) 57:892–6.

PubMed Abstract | Google Scholar

11. Zoon M, van Rooijen K, Berg-le Clerqe T. Wat werkt in de hulp aan gezinnen met meervoudige en complexe problemen? NJI instituut (2020). Available online at: https://www.nji.nl/nl/Download-NJi/Wat-werkt-in-hulp-aan-gezinnen-met-meervoudige-en-compleze-problemen.pdf (accessed March 24, 2020).

Google Scholar

12. Holwerda A, Jansen DEMC, Reijneveld SA. De effectiviteit van hulpverlening aan multiprobleemgezinnen: een overzicht. Groningen: Universitair Medisch Centrum Groningen (2014). Available online at: https://instituutgak.nl/onderzoek/kennisbank/de-effectiviteit-van-hulpverlening-aan-multiprobleemgezinnen-een-overzicht/ (accessed February 5, 2020).

13. Lamb CE. Alternatives to admission for children and adolescents: providing intensive mental healthcare services at home and in communities: what works? Curr Opin Psychiatry. (2009) 22:345–50. doi: 10.1097/YCO.0b013e32832c9082

PubMed Abstract | CrossRef Full Text | Google Scholar

14. McGorry P, Bates T, Birchwood M. Designing youth mental health services for the 21st century: examples from Australia, Ireland and the UK. Br J Psychiatry. (2013) 22:30–5. doi: 10.1192/bjp.bp.112.119214

PubMed Abstract | CrossRef Full Text | Google Scholar

15. O'Brien A, Fahmy R, Singh SP. Disengagement from mental health services: a literature review. Soc Psychiatry Psychiatr Epidemiol. (2009) 44:558–68. doi: 10.1007/s00127-008-0476-0

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Thornicroft G, Tansella M. What Are the Arguments for Community-Based Mental Health Care? Copenhagen: World Health Organization (European Region) Health Evidence Network (2003). Available online at: https://www.euro.who.int/__data/assets/pdf_file/0019/74710/E82976.pdf (accesssed March 26, 2020).

17. Block AM, Greeno CG. Examining outpatient treatment dropout in adolescents: a literature review. Child Adolesc Soc Work. (2011) 28: 393–420. doi: 10.1007/s10560-011-0237-x

CrossRef Full Text | Google Scholar

18. Lavik KO, Veseth M, Frøysa H, Binder P-E, Moltu C. ‘Nobody else can lead your life': what adolescents need from psychotherapists in change processes. Couns Psychother Res. (2018) 18:262–73. doi: 10.1002/capr.12166

CrossRef Full Text | Google Scholar

19. De Haan AM, Boon AE, de Jong JTVM, Hoeve M, Vermeiren RRJM. A meta-analytic review on treatment dropout in child and adolescent outpatient mental health care. Clin Psychol Rev. (2013) 33:698–711. doi: 10.1016/j.cpr.2013.04.005

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Gibson K, Cartwright C, Kerrisk K, Campbell J, Seymour F. What young people want: a qualitative study of adolescents' priorities for engagement across psychological services. J Child Fam Stud. (2016) 25:1057–65. doi: 10.1007/s10826-015-0292-6

CrossRef Full Text | Google Scholar

21. Settipani CA, Hawke L, Cleverley K, Chaim G, Cheung A, Mehra K. Key attributes of integrated community-based youth service hubs for mental health: a scoping review. Int J Ment Health Syst. (2019) 13:52. doi: 10.1186/s13033-019-0306-7

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Loos S, Walia N, Becker T, Puschner B. Lost in transition? Perceptions of health care among young people with mental health problems in Germany: a qualitative study. Child Adolesc Psychiatry Ment Health. (2018) 12:41. doi: 10.1186/s13034-018-0249-9

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Smith TE, Easter A, Pollock M, Pope LG, Wisdom J. Disengagement from care: perspectives of individuals with serious mental illness and of service providers. Psychiatr Serv. (2013) 64:770–5. doi: 10.1176/appi.ps.201200394

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Schley C, Yuen K, Fletcher K, Radovini A. Does engagement with an intensive outreach service predict better treatment outcomes in ‘high-risk'youth? Early Interv Psychiatry. (2012) 6:176–84. doi: 10.1111/j.1751-7893.2011.00338.x

PubMed Abstract | CrossRef Full Text | Google Scholar

25. O'Keeffe S, Martin P, Target M, Midgley N. ‘I just stopped going': a mixed methods investigation into types of therapy dropout in adolescents with depression. Front Psychol. (2019) 10:75. doi: 10.3389/fpsyg.2019.00075

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Van Veldhuizen R. FACT: a Dutch version of ACT. Community Ment Health J. (2007) 43:421–33. doi: 10.1007/s10597-007-9089-4

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Centre for Certification ACT and Flexible ACT. (2020). Available online at: https://ccaf.nl (accessed March 26, 2020).

28. Broersen M, Creemers HM, Frieswijk N, Vermulst AA, Kroon H. Investigating the critical elements and psychosocial outcomes of Youth Flexible Assertive Community Treatment: a study protocol for an observational prospective cohort study. BMJ Open. (2020) 10:4. doi: 10.1136/bmjopen-2019-035146

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Gowers SG, Harrington RC, Whitton A, Beevor AS, Lelliott P, Wing JK. HoNOSCA Health of the Nation Outcome Scales (Child & Adolescent Mental Health). Trainer's Guide. Chester: University of Liverpool (1998). Available online at: https://www.anq.ch/wp-content/uploads/2017/12/ANQ_PSY_KJP_HoNOSCA_Trainers-Guide_Gowers.pdf (accessed March 30 2020).

Google Scholar

30. Gowers SG, Harrington RC, Whitton A, Beevor AS, Lelliott P, Wing JK. Brief scale for measuring the outcomes of emotional and behavioural disorders in children Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA). Br J Psychiatry. (1999) 174:413–6. doi: 10.1192/bjp.174.5.413

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Muthén LK, Muthén BO. Mplus User's Guide. 8th ed. Los Angeles, CA: Muthén & Muthén (1998–2017).

Google Scholar

32. IBM Corp. IBM SPSS Statistics for Windows, Version 25.0. Armonk, NY: IBM Corp (2017).

33. Collins LM, Lanza ST. Latent Class and Latent Transition Analysis: With Applications in the Social, Behavioral and Health Sciences. Hoboken, NJ: Wiley (2010).

Google Scholar

34. Schwarz G. Estimating the dimension of a model. Ann Stat. (1978) 6:176–84. doi: 10.1214/aos/1176344136

CrossRef Full Text | Google Scholar

35. Akaike H. A New look at the statistical model identifaction. IEEE Trans Automat Contr. (1974) 19:716–23. doi: 10.1109/TAC.1974.1100705

CrossRef Full Text | Google Scholar

36. Sclove LS. Application of model-selection criteria to some problems in multivariate analysis. Psychometrika. (1987) 52:333–43. doi: 10.1007/BF02294360

CrossRef Full Text | Google Scholar

37. Nylund KL, Asparouhov T, Muthen BO. Deciding on the number of classes in latent class analysis and growth mixture modeling: a Monte Carlo simulation study. Struct Equ Modeling. (2007) 14:535–69. doi: 10.1080/10705510701575396

CrossRef Full Text | Google Scholar

38. Celeux G, Soromenho G. An Entropy criterion for assessing the number of clusters in a mixture model. J Classif . (1996) 13:195–212. doi: 10.1007/BF01246098

CrossRef Full Text | Google Scholar

39. Porcu M, Giambona F. Introduction to latent class analysis with applications. J Early Adolesc. (2017) 37:129–58. doi: 10.1177/0272431616648452

CrossRef Full Text | Google Scholar

40. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 5th ed. Arlington, VA: American Psychiatric Association (2013). doi: 10.1176/appi.books.9780890425596

CrossRef Full Text | Google Scholar

41. Goodman R. Psychometric properties of the strengths and difficulties questionnaire. J Am Acad Child Adolesc Psychiatry. (2001) 40:1337–45. doi: 10.1097/00004583-200111000-00015

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Van Widenfelt BM, Goedhart AW, Treffers PDA, Goodman R. Dutch version of the Strengths and Difficulties Questionnaire (SDQ). Eur Child Adolesc Psychiatry. (2003) 12:281–9. doi: 10.1007/s00787-003-0341-3

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Bodden D, Braet, Stikkelbroek Y. CDI-2: Screeningsvragenlijst voor depressie bij kinderen en jongeren (Nederlandse bewerking). Amsterdam: Hogrefe Uitgevers B.V (2016).

44. Kovacs M. The Child Depression Inventory 2 (CDI-2). Toronto: Multi Health Systems (2011).

Google Scholar

45. Ising HK, Veling W, Loewy RL, Rietveld MW, Rietdijk J, Dragt S. The validity of the 16-item version of the prodromal questionnaire (PQ-16) to screen for ultra high risk of developing psychosis in the general help-seeking population. Schizophr Bull. (2012) 38:1288–96. doi: 10.1093/schbul/sbs068

PubMed Abstract | CrossRef Full Text | Google Scholar

46. De Jong Y, Mulder CL, Boon AE, Deen M, van‘t Hof M, van der Gaag M. Screening for psychosis risk among adolescents in child and adolescent mental health services: a description of the first step with the 16-item version of the Prodromal Questionnaire (PQ-16). Early Interv Psychiatry. (2016) 12:4. doi: 10.1111/eip.12362

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Goodman R, Meltzer H, Bailey V. The strengths and difficulties questionnaire: a pilot study on the validity of the self-report version. Eur Child Adolesc Psychiatry. (1998) 7:125–30. doi: 10.1007/s007870050057

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Theunissen MHC, de Wolff MS, Reijneveld SA. The strengths and difficulties questionnaire self-report: a valid instrument for the identification of emotional and behavioral problems. Acad Pediatr. (2019) 19:471–6. doi: 10.1016/j.acap.2018.12.008

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Malmberg M, Rydell A-M, Smedje H. Validity of the Swedish version of the Strengths and Difficulties Questionnaire (SDQ-Swe). Nord J Psychiatry. (2003) 57:357–63. doi: 10.1080/08039480310002697

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Van den Heuvel MWH, Stikkelbroek YAJ, Bodden DHM, van Baar AL. Coping with stressful life events: cognitive emotion regulation profiles and depressive symptoms in adolescents. Dev Psychopathol. (2019) 32:985–95. doi: 10.1017/S0954579419000920

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Chen F, Wang L, Wang J, Heeramun-Aubeeluck A, Yuan J, Zhao X. Applicability of the Chinese version of the 16-item Prodromal Questionnaire (CPQ-16) for identifying attenuated psychosis syndrome in a college population. Early Interv Psychiatry. (2014) 10:3–16. doi: 10.1111/eip.12173

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Becker A, Hagenberg N, Roessner V, Woerner W, Rothenberger A. Evaluation of the selfreported SDQ in a clinical setting: do self-reports tell us more than ratings by adult informants? Eur Child Adolesc Psychiatry. (2004) 13:17–24. doi: 10.1007/s00787-004-2004-4

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Vugteveen J, De Bildt A, Hartman C, Timmerman M. Using the Dutch multi-informant Strengths and Difficulties Questionnaire (SDQ) to predict adolescent psychiatric diagnoses. Eur Child Adolesc Psychiatry. (2018) 27:1347–59. doi: 10.1007/s00787-018-1127-y

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Wijnhoven LA, Niels-Kessels H, Creemers DH, Vermulst AA, Otten R, Engels RC. Prevalence of comorbid depressive symptoms and suicidal ideation in children with autism spectrum disorder and elevated anxiety symptoms. J Child Adolesc Ment Health. (2019) 31:77–84. doi: 10.2989/17280583.2019.1608830

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Power L, Polari AR, Yung AR, McGorry PD, Nelson B. Distress in relation to attenuated psychotic symptoms in the ultra-high-risk population is not associated with increased risk of psychotic disorder. Early Interv Psychiatry. (2016) 10:258–62. doi: 10.1111/eip.12233

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Maurice-Stam H, Haverman L, Splinter A, van Oers HA, Schepers SA, Grootenhuis MA. Dutch norms for the Strengths and Difficulties Questionnaire (SDQ) – parent form for children aged 2–18 years. Health Qual Life Outcomes. (2018) 16:1. doi: 10.1186/s12955-018-0948-1

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Vermulst A, Kroes G, de Meyer R, Nguyen L, Veerman JW. Handleiding OBVL. Nijmegen: Praktikon (2015). Available online at: https://www.praktikon.nl/resources/vragenlijsten/handleiding_obvl_2015__1_.pdf (accessed January 8, 2020).

Google Scholar

58. Damen H, Veerman JW, Vermulst AA, Nieuwhoff R, de Meyer RE, Scholte RHJ. Parental empowerment: construct validity and reliability of a Dutch Empowerment Questionnaire (EMPO). J Child Fam Stud. (2017) 26:424–36. doi: 10.1007/s10826-016-0568-5

CrossRef Full Text | Google Scholar

59. De Greef M, Van Hattum MJC, Granger KL, McLeod BD, Pijnenburg HM, Scholte RHJ. Predictors of parent-professional alliance in home-based parenting support. Child Youth Serv Rev. (2018) 89:171–8. doi: 10.1016/j.childyouth.2018.04.028

CrossRef Full Text | Google Scholar

60. Driessen M. Geestelijke ongezondheid in Nederland in kaart gebracht. Een beschrijving van de MHI-5 in de gezondheidsmodule van het Permanent Onderzoek Leefsituatie. Den Haag: Centraal Bureau voor de Statistiek (2011). Available online at: https://www.werkhoezithet.nl/doc/kennisbank/geestelijke_ongezondheid_Ned_2011.pdf (accessed January 8, 2020).

Google Scholar

61. Ravens-Sieberer U, the European KIDSCREEN Group. The KIDSCREEN Questionnaires - Quality of Life Questionnaires for Children and Adolescents - Handbook. Lengerich: Pabst Science Publishers (2006).

PubMed Abstract | Google Scholar

62. Clark BG, Magill-Evans JE, Koning CJ. Youth with autism spectrum disorders: self- and proxy-reported quality of life and adaptive functioning. Focus Autism Other Dev Disabl. (2015) 30:57–64. doi: 10.1177/1088357614522289

CrossRef Full Text | Google Scholar

63. Erhart M, Ottova V, Gaspar T, Jericek H, Schnohr C, Alikasifoglu M. Measuring mental health and well-being of school-children in 15 European countries: results from the KIDSCREEN-10 Index. Int J Public Health. (2009) 54:160–6. doi: 10.1007/s00038-009-5407-7

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Damen H, Veerman JW, Vermulst AA, Nieuwhoff R, de Meyer RE, Scholte RHJ. Empowerment vragenlijst voor jongeren (Youth Empowerment Questionnaire; versie 3.1). Praktikon & Pactum (2017). Available online at: https://www.praktikon.nl/wat-we-doen/vragenlijsten/empo (accessed January 8, 2020).

65. Aceituno D, Vera N, Prina AM, McCrone P. Cost-effectiveness of early intervention in psychosis: systematic review. Br J Psychiatry. (2019) 215:388–94. doi: 10.1192/bjp.2018.298

PubMed Abstract | CrossRef Full Text | Google Scholar

66. Firn M, White SJ, Hubbeling D, Jones B. The replacement of assertive outreach services by reinforcing local community teams: a four-year observational study. J Ment Health. (2018) 27:4–9. doi: 10.3109/09638237.2016.1139073

PubMed Abstract | CrossRef Full Text | Google Scholar

67. McGorry PD, Mei C. Early intervention in youth mental health: progress and future directions. Evid Based Ment Health. (2018) 21:182–4. doi: 10.1136/ebmental-2018-300060

PubMed Abstract | CrossRef Full Text | Google Scholar

68. Fusar-Poli P. Integrated mental health services for the developmental period (0 to 25 years): a critical review of the evidence. Front Psychiatry. (2019) 10:355. doi: 10.3389/fpsyt.2019.00355

PubMed Abstract | CrossRef Full Text | Google Scholar

69. Hetrick SE, Bailey AP, Smith KE, Malla A, Mathias S, Singh S. Integrated (one-stop shop) youth health care: best available evidence and future directions. Med J Aust. (2017) 207:5–18. doi: 10.5694/mja17.00694

PubMed Abstract | CrossRef Full Text | Google Scholar

70. Schley C, Ryall V, Crothers L, Radovini S, Fletcher K, Marriage K. Early intervention with difficult to engage, ‘high-risk' youth: evaluating an intensive outreach approach in youth mental health. Early Interv Psychiatry. (2008) 2:195–200. doi: 10.1111/j.1751-7893.2008.00079.x

PubMed Abstract | CrossRef Full Text | Google Scholar

71. Schley C, Radovini A, Halperin S, Fletcher K. Intensive outreach in youth mental health: description of a service model for young people who are difficult-to-engage and ‘high-risk'. Child Youth Serv Rev. (2011) 33:1506–14. doi: 10.1016/j.childyouth.2011.03.013

CrossRef Full Text | Google Scholar

72. Nieuwenhuis JG, Smits HJH, Noorthoorn EO, Mulder CL, Penterman EJM, Nijman HLI. Not recognized enough: the effects and associations of trauma and intellectual disability in severely mentally ill outpatients. Eur Psychiatry. (2019) 58:63–9. doi: 10.1016/j.eurpsy.2019.02.002

PubMed Abstract | CrossRef Full Text | Google Scholar

73. Seelen-de Lang BL, Smits HJH, Penterman BJM, Noorthoorn EO, Nieuwenhuis JG, Nijman HLI. Screening for intellectual disabilities and borderline intelligence in Dutch outpatients with severe mental illness. J Appl Res Intellect Disabil. (2019) 32:1096–102. doi: 10.1111/jar.12599

PubMed Abstract | CrossRef Full Text | Google Scholar

74. Kortrijk H, Schaefer B, van Weeghel J, Mulder CL, Kamperman A. Trajectories of patients with severe mental illness in two-year contact with Flexible Assertive Community Treatment teams using Routine Outcome Monitoring data: an observational study. PLoS ONE. (2019) 14:e0207680. doi: 10.1371/journal.pone.0207680

PubMed Abstract | CrossRef Full Text | Google Scholar

75. Brimblecombe N, Knapp M, Murguia S. The role of youth mental health services in the treatment of young people with serious mental illness: 2-year outcomes and economic implications. Early Interv Psychiatry. (2017) 11:393–400. doi: 10.1111/eip.12261

PubMed Abstract | CrossRef Full Text | Google Scholar

76. Nguyen T, Embrett MG, Barr NG, Mulvale GM, Vania DK, Randall GE. Preventing youth from falling through the cracks between child/adolescent and adult mental health services: a systematic review of models of care. Community Ment Health J. (2017) 53:375–82. doi: 10.1007/s10597-017-0098-7

PubMed Abstract | CrossRef Full Text | Google Scholar

77. StatLine Jeugdmonitor. Centraal Bureau voor de Statistiek. (2020). Available online at: https://jmopendata.cbs.nl/#/JM/nl/dataset/71009ned/table?ts=1603186893413 (accessed October 19, 2020).

Google Scholar

78. Hansen-Bauer K, Langsrud Ø, Kvernmo S, Heyerdahl S. Clinician-rated mental health in outpatient child and adolescent mental health services: associations with parent, teacher and adolescent ratings. Child Adolesc Psychiatry Ment Health. (2010) 4:29. doi: 10.1186/1753-2000-4-29

CrossRef Full Text | Google Scholar

79. Cleridou K, Patalay P, Martin P. Does parent–child agreement vary based on presenting problems? Results from a UK clinical sample. Child Adolesc Psychiatry Ment Health. (2017) 11:22. doi: 10.1186/s13034-017-0159-2

PubMed Abstract | CrossRef Full Text | Google Scholar

80. Van der Meer M, Dixon A, Rose D. Parent and child agreement on reports of problem behaviour obtained from a screening questionnaire, the SDQ. Eur Child Adolesc Psychiatry. (2008) 17:491–7. doi: 10.1007/s00787-008-0691-y

PubMed Abstract | CrossRef Full Text | Google Scholar