Effects of Illness Management and Recovery: A Multicenter Randomized Controlled Trial

There have been inconsistent findings in the literature with respect to the efficacy of Illness Management and Recovery (IMR) in the psychosocial treatment of people with schizophrenia or other severe mental illnesses. This study aimed to comprehensively investigate the effectiveness of IMR, including the impact of completion and fidelity. In this randomized controlled trial (RCT), 187 outpatients received either IMR plus care as usual (CAU) or only CAU. Multilevel modeling was implemented to investigate group differences over an 18-month period, comprising 12 months of treatment and six months of follow-up. The primary outcome was overall illness management, which was assessed using the client version of the IMR scale. Secondary outcomes included measures regarding illness management, clinical, personal, and functional recovery, and hospitalizations. The interviewers were blinded to group allocation. This clinical trial was registered with the Netherlands Trial Register (NL4931, NTR5033). Patients who received IMR showed statistically significant improvement in self-reported overall illness management (the primary outcome). Moreover, they showed an improvement in self-esteem, which is a component of personal recovery. There were no effects within the other questionnaires. There were also no statistically significant between-group differences in terms of hospitalizations. Patients in both groups showed statistically significant improvement in clinician-rated overall illness management, social support, clinical and functional recovery, and self-stigma over time. IMR completion was associated with stronger effects. High IMR fidelity was associated with self-esteem. This study confirms the efficacy of IMR in overall illness self-management. To our knowledge, this is the first RCT on IMR to explore the impact of fidelity on treatment efficacy. Future studies should further establish efficacy in personal recovery. To improve efficacy, it appears important to promote IMR completion and fidelity.

Introduction

Patients with schizophrenia and other severe mental illnesses (SMIs) face major challenges in achieving personal goals and fully participating in society. This is due to their recurring symptomatology, cognitive impairment, loss of social support, and societal barriers such as stigma (1, 2). Although treatment with psychopharmacological drugs facilitates reductions in symptom severity and relapse, there is a need for effective psychosocial interventions to support participants in illness self-management. The aim of these interventions is to develop fulfilling and valued workplace roles and social connections, to obtain housing, and to facilitate self-determination and well-being (1, 3–6). Illness management programs, including Illness Management and Recovery (IMR), have been developed to support individuals with SMIs in addressing the physical, social, and emotional impact of their persistent condition. These programs seek to improve the course of illness (1, 7, 8).

IMR is a structured psychosocial program that promotes illness self-management in people with schizophrenia and other SMIs. It was developed based on an empirical literature review with regard to teaching illness self-management strategies (9). IMR combines psychoeducation, behavioral tailoring for medication adherence, relapse-prevention training, and cognitive-behavioral training in social and coping skills (3). The various individual components of the IMR program are not new; however, the novelty of IMR results from offering these components as an integrated, introductory package (6). The working mechanisms underlying IMR have been suggested in its previously published conceptual framework (Figure 1), indicating that progress toward recovery may be achieved by combining better illness management with the pursuit of personal goals (3). Recovery is a multidimensional concept comprised of three subtypes that are not mutually exclusive (10–14): clinical or symptomatic recovery (4, 15); functional or objective recovery (3), which is defined as the degree of vocational and social functioning (14, 16–18), and personal or subjective recovery (3, 19), whose key elements are summarized via the acronym CHIME: connectedness, hope, identity, meaning in life, and empowerment (20). IMR is currently implemented in various countries, including the US, the Netherlands, Denmark, Norway, Sweden, Spain, Japan, and Singapore.

FIGURE 1

Figure 1. Conceptual framework of illness management and recovery (3).

A recent systematic review and meta-analysis of 37 studies on self-management interventions found that these interventions reduced symptoms and the length of hospital stay, as well as enhancing functioning and quality of life. Moreover, there were statistically significant positive effects on subjective measures of recovery, self-rated recovery, and self-efficacy (7). Although this review indicated the relevance of self-management interventions, it only included nine (24%) studies on IMR. Therefore, there is a need for more IMR-specific experimental evidence.

In September 2018, we conducted a literature search that yielded 65 studies on IMR, including six randomized controlled trials (RCTs) (21–28). These six RCTs yielded inconsistent results. Regarding our primary outcome—illness self-management measured using the client version of the IMR scale—three RCTs reported positive results for IMR as compared with the control group (22, 23, 28), while the remaining three RCTs reported null results (21, 24, 25, 27). Regarding illness management measured using the clinician version of the IMR scale, four RCTs reported positive results for IMR (21–23, 28), while one other RCT reported null results (25, 27). Three RCTs reported that IMR had positive effects on reducing psychiatric symptoms (22, 23, 28), while the other RCTs identified in the literature search reported null results (22, 24, 26, 27). Regarding hospitalizations, only one RCT reported positive results (28); the remaining RCTs reported null results (21–24, 26). Three RCTs reported no effect on personal recovery (23–25, 27) and one RCT reported no effect on employment (22). Two RCTs did not observe any effects of IMR (24–26).

Differences in results between these studies may be related to variations in patient populations, sample sizes, control group characteristics, the duration of IMR, drop-out rates, and model fidelity (6, 29). The present study is focused on two implementation aspects that may have contributed to the inconsistency in the results of earlier RCTs: model fidelity and IMR completion. Fidelity is defined as the degree of adherence to the standards and principles of a program model (30, 31). There are indications that higher fidelity to empirically supported mental health program models, notably evidence-based practices (EBPs), is predictive of better client outcomes for people with SMIs (32). With respect to this predictive validity, there is some evidence regarding several interventions for this target group: Assertive Community Treatment (ACT) (33–37), Individual Placement and Support (IPS) (38), and IMR (39).

Before conducting the current study, we conducted a pilot study to explore the feasibility of a randomized controlled trial. This pilot study indicated that an RCT appeared feasible. However, the results likewise indicated a 50% dropout rate from the treatment regimen (30). The relevance of IMR completion (22, 40), including with respect to both fidelity and completion (27), has been suggested in prior research. Therefore, the current study explored the impact of fidelity and IMR completion rate on the efficacy of this intervention. The study design was customized to facilitate completion analysis: a priori, we chose to assign more clients to the experimental condition (IMR) than to the CAU group (6).

Although IMR appeared promising for implementation in the Netherlands, the inconsistent findings within previous studies indicated the need for further thorough research on IMR efficacy and outcomes. Therefore, the present study aimed to compare IMR plus care as usual (CAU) with CAU alone in patients with schizophrenia or other SMIs. Using multiple outcome measures of illness self-management, illness outcomes, and recovery, the present study sought to thoroughly assess the effects of IMR. We formulated multiple hypotheses as follows: (1) IMR + CAU allows for better illness management as well as reducing symptoms and relapses as compared with CAU alone; (2) IMR + CAU allows for better personal and functional recovery as compared with CAU alone; and (3) IMR+CAU-related improvement is associated with the fidelity of IMR implementation (6). The adaptation of the study design to facilitate completion analysis was based on the proposition that a higher rate of completion is associated with stronger effects.

Methods

Study Design

This multicenter, parallel-group, single-blinded RCT was conducted at two mental health care institutions in the Netherlands (6). Ethical approval was obtained at the Erasmus University Medical Center (METC registration number NL38605.078.12). The inclusion criteria were as follows: age 18–65 years; diagnosis with an SMI such as schizophrenia or a persistent mood disorder with or without comorbid disorders (i.e., substance abuse and personality disorders); and receiving outpatient treatment in the greater Rotterdam area. Exclusion criteria comprised previous participation in IMR training and insufficient Dutch language skills. Participants were recruited through clinician referrals from 14 participating community mental health teams.

Measures

The primary outcome measure was the IMR scale (client version) (6). Both the client and clinician versions of the IMR scale allow for the overall measurement of illness self-management, and both scales are composite measures of various self-management components. The 15 scale items comprise the key elements of IMR training, including progress toward goals, knowledge regarding mental illness, relapse-prevention planning, involvement with significant others, coping with symptoms, medication adherence, substance abuse, and symptom distress (3, 41–43).

Moreover, we sought to determine whether individually assessing the scale components had any effect on study results and interpretation. Therefore, in addition to the clinician version of the IMR scale and measuring instruments with respect to clinical, functional, and personal recovery, we included five separate scales regarding components of illness self-management as secondary outcome measures in the current study (6).

Secondary outcome measures included the IMR Scale (clinician version) (43) and five scales assessing specific aspects of illness management as follows: coping (Coping Self-Efficacy Scale) (44), social support (Multidimensional Scale of Perceived Social Support) (45), medication adherence (treatment-adherence-subscale of the clinician-rated Service Engagement Scale) (46), problems with alcohol or drugs (item 24 of the Addiction Severity Index) (47), and psychiatric insight (Insight Scale) (48). Clinical recovery was analyzed using the Brief Symptom Inventory (49). Hospitalizations were assessed and operationalized as the occurrence of hospitalizations and the lengths of hospital stay. Functional recovery was assessed using the Social Functioning Scale (50). Personal recovery was assessed using the Mental-Health Recovery Measure (MHRM), a composite personal recovery scale that measures “self-empowerment,” “learning and new potentials,” and “spirituality” (51), the Internal Stigma of Mental Illness scale (ISMI) (52), and the Self-Esteem Rating Scale-Short Form (SERS-SF) (6, 53). The IMR Fidelity Scale was used to measure fidelity to IMR program implementation at the group level (31, 54). Details regarding all scales have been described previously (6, 55).

Procedures

During interviews with an assistant researcher, the caseloads of all clinicians on the community mental health teams were reviewed by the clinicians themselves for their potential suitability for IMR. The clinicians then asked the selected clients about participating in IMR and about their willingness to be informed about the study. If clients expressed interest and agreed to be contacted by an assistant researcher with more detailed information, the assistant researcher explained the study objectives, the randomization procedure, and the three measurement moments. The clients were then asked if they were willing to participate in the study. After written informed consent was given, the baseline interview was followed by randomization (6). For allocation of participants to receive IMR plus CAU (i.e., the intervention group) or CAU only (i.e., the control group), we used computer-generated random permuted blocks of five stratified across the treatment teams. Blocks were independently constructed by a research assistant using a randomization plan (http://www.randomization.com). The results of our pilot study suggested an expected 50% dropout rate from treatment within the experimental condition (55). Therefore, to facilitate completer analyses, we allocated more participants to the intervention group than to the control group with a ratio of 3:2. Written informed consent was obtained from all participants prior to participation.

Since research assistants were blinded to group allocation, the data collection procedure was single-blinded (11). Before each assessment, participants and clinicians were instructed through letter or e-mail not to disclose the treatment assignment. At the start of each session, the participants were verbally reminded of this requirement of the investigation.

Assessments were performed at baseline (T1), after 12 months (post-treatment, T2), and after 18 months (six months of follow-up, T3). The research assistants, who were centrally instructed, met the study participants at several institute branches to conduct the assessments.

The principal researcher and one of the two co-auditors assessed IMR fidelity in 15 IMR groups within one-day site visits near the end of the treatment course. Both assessors performed independent scoring following a standard procedure (54). In the case of differences or disagreements, the assessors established a consensus score. The principal investigator was trained on how to use the IMR fidelity scale by two American specialists (6). The two trained co-auditors were a psychologist and an advanced nurse practitioner. In addition to semi-structured interviews with participants and trainers, assessments involved one observation session and checking forms, such as anonymized IMR Goal Tracking Sheets (6). All IMR elements were reviewed with all respondents during the interviews. This included questioning about elements that were not used during the observed session. Thus, we were able to obtain a good overall picture of the fidelity within each group and we were able to assess all 13 items of the IMR fidelity scale. Periodically, the researchers provided feedback regarding the assessment results in the supervision groups.

To assess the impact of fidelity, patients who received IMR were assigned the fidelity score of their group. This score indicates the fidelity level of the IMR model as applied to the participant.

IMR Content and Services

All study participants received CAU comprising standard outpatient psychiatric care (including case management, multidisciplinary psychiatric and psychological treatment, and rehabilitation services). The treatment contacts for individual case management involved face-to-face meetings with a mental health nurse, a social worker, or another psychiatric counselor at two-week intervals. Additionally, a psychiatrist was consulted at least once a year as appropriate. Typical contacts with case managers involved individualized recovery support and structuring conversations on all life domains.

In the intervention group, these usual services were provided together with IMR, a comprehensive structured training program for people with SMI. IMR was provided in a group format in weekly 90-min sessions. In accordance with the IMR fidelity scale, these groups included up to eight people. However, the exact group size varied per group and per session. The IMR intervention comprised eleven modules as well as practitioner guides and handouts for the participants. The eleven modules included Recovery Strategies, Basic Facts about Mental Illness, the Stress-Vulnerability Model, Building Social Support, Using Medication Effectively, Drug and Alcohol Use, Reducing Relapses, Coping with Stress, Coping with Persistent Symptoms, Getting Your Needs Met in the Mental-Health System, and Healthy Lifestyles (56). The original American text was translated into Dutch and was adapted to the Dutch context as appropriate. Each group of patients who received IMR was guided by two trainers, who used a combination of motivation-enhancement strategies (including conveying confidence and exploring the pros and cons of change), educational strategies (including interactive teaching, breaking down information, and checking for understanding), and cognitive-behavioral techniques (including shaping, modeling, and role-playing) (3). Peer-group support, coping, and social skills training are key IMR elements. Workbooks and homework assignments were accessible through an e-health module (6). During the first module—Recovery Strategies—the participants identified their personal goals during the program. For half of each session, some participants worked toward these goals. During the other half of each session, all participants worked on the module subjects with the help of the handouts. Our pilot study demonstrated that completing the modules required an average of three to four sessions (55).

IMR was mostly provided by mental health nurses or social workers, recruited from among the group of case managers in the CMHC teams who were interested in providing the IMR intervention. All case managers were given the opportunity to provide IMR. Most case managers had years of professional experience and had previously been trained in psychiatric rehabilitation methodologies, including the Boston University Approach to Psychiatric Rehabilitation (57). As part of the institutes' regular quality care program, all IMR trainers underwent two days of training in IMR and underwent additional four-hour training sessions twice annually. The IMR trainers underwent two hours of group supervision by a senior counselor at two-week intervals.

Statistical Analysis

Based on the effect sizes observed in the three RCTs on IMR published at the time of designing the current study (21–23), we anticipated a moderate effect size of 0.40 with respect to the primary outcome variable (i.e., the self-rated IMR scale) (41, 42). Based on power analyses with three measurement times (mixed models), equal allocation to the experimental and control groups, a power of 0.80, alpha set at 0.05, and an effect size of 0.40, we determined that it would be necessary to randomize 148 clients: 74 to the experimental condition and 74 to the CAU control group (6, 58, 59). Due to the planned 3:2 randomization we aimed for 111 participants (74 × 3:2) in the experimental condition.

Linear mixed models (LMM) were implemented to investigate between-group differences (IMR vs. CAU) over an 18-month period, comprising 12 months of treatment and six months of follow-up (59). To assess treatment effects, we adopted the following analysis protocol for all outcomes (6). We first determined the best-fitting model using a stepwise modeling procedure. Subsequently, time and conditions were entered into the model; a time × condition interaction term was included only if the inclusion of this term improved the model based on the Akaike Information Criterion. A statistically significant time × condition interaction was considered to represent a treatment effect. For all outcomes, the estimated marginal means and 95% confidence intervals (CI) were calculated for the three measurement moments. To assess between-group effect sizes, we calculated Cohen's ds (0.2–0.3, small; 0.5, medium; >0.8, large) (60). As LMMs use the full data set, this methodology retains people with missing values; thus, imputation is not necessarily beneficial in this context (61).

For all outcomes, we followed the same procedure to examine the robustness of our findings within a sensitivity analysis among IMR completers. A “completer” was defined as a patient who had attended ≥50% of all scheduled sessions. To explore the possibility of selective IMR non-completion, the patients' attendance was analyzed via logistic regression using 12 variables representing baseline characteristics.

Hospitalization was analyzed in two ways. First, to examine between-group differences in the odds of hospitalization during the year following T2, we performed logistic regression analysis with respect to hospitalization (dichotomized), with T2 as the dependent variable, the treatment group as the independent variable, and hospitalization (dichotomized) one year prior to T1 as a confounder. Second, we determined difference scores with respect to the length of stay (days) at one year before and after treatment. Between-group differences were tested using the Mann–Whitney test.

In our study protocol, we hypothesized that IMR-related improvement would be associated with the fidelity of IMR implementation (6). Therefore, we explored the impact of fidelity on outcomes. This analysis included all participants with fidelity scores who had attended at least ten IMR sessions, indicating consistent engagement with the IMR program. Participants were divided into high and moderate-to-low fidelity groups, with mixed models used to assess differences in time effects between the control group and the two experimental subgroups. The control group was considered the reference group. The model components were comprised of the three measurement moments, the three subgroups (high fidelity, moderate/low fidelity, and control), and their interactions. In cases where there was an overall statistically significant interaction effect, we interpreted the parameter estimates for these interaction effects as appropriate. The cutoff scores for high, moderate, and low fidelity within the IMR fidelity scale were ≥4.0, 3.0–4.0, and <3.0, respectively (54).

Statistical analyses were performed using the Statistical Package for the Social Sciences (version 25.0; SPPS, Inc., Chicago, IL, USA). Multiplicity adjustments, including Bonferroni and Benjamini–Hochberg corrections (62), are often applied. However, these methodologies have substantial limitations (63, 64). This study applied 11 secondary outcomes, selected carefully a priori, in order to explore the effects of various components of the conceptual framework for the IMR program (3). Our analyses followed a pre-specified protocol (6) and we present a qualitative interpretation of our findings. Additionally, as a sensitivity analysis, we present the impact of Benjamini–Hochberg corrections on our results.

Due to an administrative oversight, the prospective trial registration was initially overlooked; however, this omission was noticed halfway through the data collection period and the trial was registered with the Netherlands Trial Register (NL4931, NTR5033). Details of our initial ethical approval and protocol are provided in the Supplementary Material.

Results

Between October 2012 and May 2014, we randomly assigned 187 participants (3:2 ratio) to receive IMR + CAU (n = 116) or CAU alone (n = 71; Figure 2 shows the CONSORT flow chart for study enrollment). There were no between-group differences in baseline demographic or medical characteristics (Table 1), and only 9.4% missing data.

FIGURE 2

Figure 2. CONSORT Flowchart. *Percentage of participants allocated to condition.

TABLE 1

Table 1. Demographic and clinical characteristics of study participants at baseline.

On average, participants in the experimental group (n = 116) attended 23.57 IMR sessions [standard deviation (SD) = 21.09], representing 44.6% of all scheduled sessions (SD = 37.3%). IMR completers (n = 57, 49%) attended an average of 42.4 sessions (SD = 12.64), representing 80.1% of the total number of scheduled sessions (SD = 13.05%).

Primary Outcome

As compared with the control group, the IMR group showed a statistically significant improvement in the primary outcome measure (illness self-management measured with the client version of the IMR scale; p = 0.048) (Table 2). The largest improvement, though still with a small effect size, was shown at follow-up (Table 3).

TABLE 2

Table 2. Fixed effects of IMR on all outcomes from final linear mixed model, in the Intention to treat sample.

TABLE 3

Table 3. Estimated outcomes as a function of IMR, estimated from linear mixed models in the Intention to treat sample.

Secondary Outcomes

Compared with the control group, the IMR group showed a statistically significant improvement in self-esteem (a component of personal recovery), which was measured using the SERS-SF (p = 0.01) (Table 2).

There were no effects on illness self-management measured via the clinician version of the IMR scale, on the illness management components of coping, social support, medication adherence, insight, and addiction, or on clinical, functional, and personal recovery measured via the MHRM (a composite measure) and the ISMI (a scale assessing self-stigma) (Table 2). The observed effect on self-esteem did not remain following Benjamini–Hochberg correction.

For all outcome measures, Table 3 shows the estimated marginal means (95% CI) from the LMM analyses. The results, including reported effect sizes (Cohen's d), concern the differences between the experimental and control condition (IMR vs. CAU) at different time points. Post-treatment, all measures showed improvement in favor of IMR with the exception of the IS and the ASI (Table 3). At follow-up, this improvement continued not only within the client version of the IMR scale, but also within the MHRM and SERS-SF personal recovery scales; those two scales demonstrated small and moderate effect-sizes, respectively.

Both the experimental and control group showed statistically significant improvement over time with respect to overall illness management, as measured using the clinician version of the IMR scale, as well as with respect to social support, clinical and functional recovery, and reduced self-stigma (Table 2).

After adjusting for hospitalization a year prior to T1, the treatment condition was not found to be a statistically significant predictor for hospitalization within the year following T2 [odds ratio (OR) = 1.22, 95% CI = 0.44, 3.42, p = 0.71). Regarding total days of hospitalization one year before and after treatment, there were no statistically significant between-group differences as measured by difference scores (IMR group, M = −7.72, SD = 51.19 vs. the control group, M = −4.46, SD = 44.23; U = 3,928.50, p = 0.50).

Impact of Fidelity

Fidelity assessments via the IMR fidelity scale were conducted in 15 IMR groups, which enrolled 68 study participants who had attended ≥10 IMR sessions. This yielded a mean fidelity score of 3.94 (SD = 0.29). Eight groups (n = 39, 57%) had scores of ≥4 (range 4.00–4.54), which indicated high (54) or good (31) fidelity. Seven groups (n = 29, 43%) had scores <4 and >3 (range 3.46–3.92), which indicated moderate (54) or fair (31) fidelity. Lower fidelity scores were partly due to the fact that role-playing, a key element of cognitive-behavioral techniques and coping skills training, was practiced at a low rate within the evaluated interventions.

Assessment of the impact of fidelity on the results of the client version of the IMR scale as well as SERS-SF scores revealed no effects on overall IMR scale scores [F_{(4, 271)} = 1.42, p = 0.23]; however, we did observe a statistically significant overall interaction effect for group and time with respect to self-esteem [F_{(4, 237)} = 3.11, p = 0.02].

Examination of the fixed coefficients for the client version of the IMR scale revealed that there were no statistically significant differences in improvement between the high-fidelity and control groups at post-treatment (B = 0.08, p = 0.46) or at follow-up (B = 0.16, p = 0.08). Moreover, there were no statistically significant differences in improvement between the moderate fidelity and control groups at post-treatment (B = 0.06, p = 0.48); however, at follow-up, we did observe a statistically significant between-group difference in overall improvement (B = 0.20, p = 0.04) (Table 4).

TABLE 4

Table 4. Fixed coefficients for linear mixed model (IMR client version) on time effects for high and moderate IMR fidelity.

Regarding the fixed coefficients for the SERS-SF score, the high-fidelity subgroup showed a statistically significant improvement as compared with the control group at post-treatment (B = 7.22, p = 0.04) but not at follow-up (B = 4.96, p = 0.19). At post-treatment, the moderate fidelity subgroup did not improve at the level of statistical significance as compared with the control group (B = 2.65, p = 0.53); however, there was a statistically significant improvement at follow-up (B = 9.06, p = 0.01) (Table 5).

TABLE 5

Table 5. Fixed coefficients for linear mixed model (SERS-SF) on time effects for high and moderate IMR fidelity.

IMR Completion

Analysis of overall illness management in completers using the client version of the IMR scale revealed a greater effect than that observed in the intention-to-treat analysis (p = 0.016), with small and moderate effect sizes at T2 and T3, respectively (for all outcomes for completers, see Tables 6, 7). Unlike analysis with the intention-to-treat principle, completer analysis within the clinician version of the IMR scale revealed a statistically significant result (p = 0.012), with large and moderate effect sizes at T2 and T3, respectively. For IMR completers, there were no effects on the five measured illness management components.

TABLE 6

Table 6. Fixed effects of IMR on all outcomes from final linear mixed model, in the completers sample.

TABLE 7

Table 7. Estimated outcomes as a function of IMR, estimated from linear mixed models in the completers sample.

With respect to self-esteem, similar to the findings of the intention-to-treat analysis IMR completers showed statistically significant improvement as compared with the control group (p = 0.03) with moderate effect sizes at T2 and T3. Unlike in the intention-to-treat analysis, IMR completers showed a statistically significant improvement in overall personal recovery measured via the MHRM (p = 0.03), with small effect sizes at T2 and T3 (for all statistically significant outcomes at 18 months, see Figure 3). For IMR completers, there was no effect on self-stigma. Further, similar to the intention-to-treat analysis, IMR completion did not show effects on clinical and functional recovery.

FIGURE 3

Figure 3. Significant Outcomes at 18 Months. 0 = baseline measurement, 1 = 12-months measurement, 2 = 18-months measurement. IMR client, client version of the Illness Management and Recovery scale; IMR clinician, clinician version of the Illness Management and Recovery scale; SERS-SF, Self Esteem Rating Scale-Short Form; MHRM, Mental Health Recovery Measure.

The possibility of selective IMR non-completion was explored using 12 baseline characteristics. It was found that selective non-completion of IMR was unlikely, as suggested by multivariate logistic regression analysis [omnibus test: ${\chi }_{(13)}^2$ = 11.64, p = 0.56; Supplementary Table 1].

Discussion

Discussion of the Results

Our positive findings regarding the client version of the IMR scale, an overall measure of illness self-management and our primary outcome measure, confirms the effectiveness of IMR as compared with CAU. Additionally, our findings in the sensitivity analysis for IMR completers regarding the effects on both the client and clinician versions of the IMR scale support this positive finding.

Although our results with respect to these IMR scales indicate an overall improvement in illness management, the questionnaire scores on specific components of illness management revealed no statistically significant effects. These questionnaire scales assessed social support, coping, medication adherence, insight, and addiction. Although the IMR scales capture the overall changes in these components, IMR training may have insufficient specificity for generating meaningful effects on individual measures within these separate domains.

Additionally, as compared with the control group, the intervention group showed a greater improvement in self-esteem. This positive outcome is supported by the findings for the sensitivity analysis among IMR completers. Compared with the control group, IMR completers had superior outcomes with respect to self-esteem and the MHRM score, a composite measure of personal recovery. These combined results indicate the potential efficacy of IMR with respect to personal recovery.

In contrast, the IMR intervention did not show a statistically significant effect on psychiatric symptoms. This is consistent with the findings of a recent meta-analysis that indicated a small to moderate association between clinical and personal recovery in patients with schizophrenia-spectrum disorders (15). Furthermore, clinical recovery does not appear to be necessary for personal recovery (11, 15).

Previous studies have confirmed the relevance of improvement in personal recovery by demonstrating the importance of the therapeutic benefits of managing mental illness, which facilitates personal well-being and self-perceived growth. Additionally, studies have highlighted the value of living a satisfying life despite the presence of enduring symptoms (13, 19, 65, 66). Therefore, changes should be measured based on both symptomatology and consumer-defined recovery (5, 20, 65, 67). Apart from symptoms (13), loss of self-esteem is considered to greatly affect the life and self-image of patients with schizophrenia (53, 57, 68). The concept of self-esteem corresponds with the concept of identity (i.e., a positive sense of self), which is one of the five processes within the CHIME conceptual framework of personal recovery (20).

The impact of fidelity was analyzed within the two scales, revealing statistically significant outcomes in the intention-to-treat analysis. There was no effect of fidelity with respect to the client version of the IMR scale. However, the fidelity of IMR training affected SERS-SF scores. This suggests that fidelity is relevant to the effectiveness of IMR with respect to self-esteem. However, there is a need to further research the predictive validity of the IMR fidelity scale within high-powered gold-standard investigations.

Although our positive results regarding the client version of the IMR scale are consistent with the results of three RCTs on IMR (22, 23, 28), they differ from those of three other RCTs (21, 24–27). Moreover, our negative results regarding the clinician version of the IMR scale are consistent with those of a previous RCT (25, 27); however, they differ from the results of four previous RCTs reporting positive outcomes (21–23, 28, 29). Our positive findings regarding personal recovery differ from those of three previous RCTs (23–25, 27). This inconsistency in results may be attributed at least in part to differing completion rates between the investigations. We thereby suggest that the differing attendance rates within IMR may have affected the findings of this and other RCTs with respect to IMR outcomes.

A previous RCT that reported no statistically significant effects was conducted among a cohort with an IMR attendance of only 28% (24). Two of three RCTs with IMR completion rates of ~50% (including our study) reported positive effects (22), while the third RCT reported null results (25–27). Two RCTs with completion rates of 100% reported positive effects on psychiatric symptoms as well as on both IMR scales (23, 28). Additionally, Levitt et al. found that, with respect to the intention-to-treat analysis, the effect size for completers increased from 0.36 to 0.75 and from 0.39 to 0.59 on the client and clinician versions of the IMR scale, respectively (22). This is consistent with our findings, wherein effect sizes on both IMR scales substantially increased in the completer subgroup.

A minimum threshold of exposure to IMR could be required for treatment effects to occur (24). Therefore, in addition to the percentage of participants exposed to >50% of the scheduled sessions, the total number of sessions attended is also relevant in evaluating treatment efficacy. In the current study, the mean attendance for the IMR group was 23.57 sessions (SD = 21.09), with completers attending an average of 42.40 sessions (SD = 12.53). A Danish RCT (25–27) that did not observe any effects had an average attendance of 16.4 sessions; moreover, completers (defined as participants exposed to >10 sessions) had only attended an average of 26.1 sessions. Therefore, the results of the two negative trials on IMR (24–27) could be partly attributed to their respective low completion rates.

The importance of a higher completion rate with respect to the efficacy of IMR suggests the relevance of increasing the motivation to continue IMR among patients. In addition to reminders and phone calls, IMR trainers should employ other strategies for facilitating attendance (24). For example, to promote the efficient use of available places at our study sites, the current study had rolling admission into IMR groups; however, peer support—and thus participation—may be promoted if groups have a closed enrollment format (24). At one site in our study, IMR groups had lunch together at each meeting in order to promote group cohesion and thus attendance. In addition, to promote completion, the two respective RCTs on IMR that had the highest percentages of completers selected participants based on earlier treatment adherence (23, 24) and applied the total IMR curriculum using home-visits (28). However, choosing one or more of these options will not be preferred or feasible at all practices and centers.

Although our study demonstrated positive results, the observed effects were small. We observed a statistically significant effect on illness self-management within the client version of the IMR scale (p = 0.048). Although we found effects on personal recovery, there were limited effects on other secondary outcome measures in the current study. These limitations could be attributed to several potential reasons. First, IMR training may not be sufficiently specific for generating statistically significant effects on the separate components of illness self-management. This limitation may be associated with the multiplicity of IMR objectives, as shown in the previously published conceptual framework for IMR (3). Second, participants in both experimental conditions showed improvement in five domains over time as follows: clinician-rated overall illness management, social support, clinical and functional recovery, and self-stigma. These findings may be attributed to the positive effects of CAU, given that many clinicians in the Netherlands have been trained in psychiatric rehabilitation methodologies, including the Boston University Approach to Psychiatric Rehabilitation (69). Additionally, standard outpatient treatment and care have evolved into the FACT model. FACT (Function ACT) is a rehabilitation-oriented clinical case management model; a flexible version of ACT (70). Third, in our study, most of the IMR practitioners (community mental health nurses and social workers) had no specific training in the empirically supported strategies underlying IMR, including cognitive-behavioral approaches and skills training. Thus, while the facilitators may have had training in rehabilitation skills and attitudes, other specific clinical skills required for the successful implementation of IMR were often insufficient. Moreover, with respect to IMR group training, it may be challenging to provide these EBPs based on the required protocols. This challenge was partly reflected in the observed fidelity scores.

Our study observed improved outcomes in only one of the eleven secondary outcomes in the intervention group. Sensitivity analyses for completers revealed improved outcomes in the intervention group for four of the eleven secondary outcomes. These effects were not observed following Benjamini-Hochberg correction for multiple testing. However, multiple adjustment methodologies have been criticized within the statistical, epidemiological, and medical literature (63, 71). For example, some researchers have indicated that multiple correction methodologies mechanize, and therefore trivialize, interpretative problems. In addition, multiplicity adjustments are not considered appropriate for more exploratory secondary analyses (63, 71). Therefore, we suggest that the present interpretation of our findings may be relevant.

Strengths and Limitations

Our study has five main strengths. First, our sample size was relatively large and we used a complete set of outcomes. This allowed for rigorous measurement of the IMR effects. Second, we thoroughly investigated the impact of completion rates. Third, to our knowledge, this is the first RCT on IMR to assess the impact of fidelity. Fourth, this is the first RCT on IMR to find effects on personal recovery. Finally, given the natural setting of the current investigation, our results can be assumed to have good generalizability.

In addition to these substantial strengths, this study had several limitations. First, there was suboptimal IMR implementation, with approximately half of the IMR-participants completing the program; furthermore, IMR fidelity was only fair to moderate for almost half of the IMR participants. For example, skills training using role play was applied at a low rate within the evaluated interventions. This suboptimal IMR implementation may have led to an underestimation of the outcomes. Second, since we aimed to comprehensively explore the effects of IMR, we examined numerous secondary outcome measures. Therefore, with reference to alpha inflation, the statistically significant result regarding self-esteem can be disputed. Third, when completing the clinician version of the IMR scale and the Service Engagement Scale, clinicians were not blinded to their patients' experimental conditions. Since we used self-score questionnaires, the patients were not blinded to the treatment condition. However, all interviewers were blinded to the condition. Fourth, in measuring the impact of fidelity, we could only utilize the fidelity scores of the 68 participants in the 15 IMR groups. This limits the generalizability of our results. Fifth, most of the enrolled participants had relatively few problems in the three domains of addiction, medication adherence, and insight at baseline; therefore, there was little room for improvement within these domains.

Conclusions

In the current study, we observed positive results with respect to the client version of the IMR scale. Our results support the effectiveness of IMR in overall illness self-management. This finding was confirmed within our secondary analysis among IMR completers. However, we observed negative results for five specific components of illness management. Therefore, our findings suggest that IMR is a non-specific intervention for illness self-management.

This study provides indications regarding the efficacy of IMR on components of personal recovery. However, we found no effects on clinical and functional recovery. There is a need for further research to confirm these findings.

We suggest that challenges in implementing the multiple specific clinical skills required for IMR may impede the achievement of specific outcomes. Therefore, more training as well as more specific and comprehensive training in different IMR elements is recommended based on the results of this research. However, within IMR, protocolized application of the EBPs underlying IMR may present a logistical challenge.

The quality of IMR implementation appeared to be relevant given the association of IMR fidelity with effects on self-esteem; moreover, greater exposure to IMR appeared to enhance these effects. Therefore, based on the findings of the current research as well as the overall literature to date, we propose more comprehensive initial and continuing education for IMR trainers in order to improve high-fidelity IMR implementation, as well as an emphasis on new and innovative strategies focusing on promoting IMR completion.

Data Availability Statement

The datasets presented in this article are not readily available because Parnassia Groep Mental Health Care needs to consent to data access. Requests to access the datasets should be directed to Bert-Jan Roosenschoon, Yi5yb29zZW5zY2hvb24mI3gwMDA0MDtwYXJuYXNzaWFncm9lcC5ubA==.

Ethics Statement

The studies involving human participants were reviewed and approved by the Research Ethics Committee (METC) of the Erasmus MC University Medical Center Rotterdam. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

B-JR, CM, JW, and MD conceived and designed the study. B-JR managed recruitment, data collection, data administration, and drafted the article, which was critically revised by all authors. EE coordinated data collection and supervised the quality of IMR at Yulius Mental Health. B-JR and MD did statistical analysis. B-JR, CM, JW, MD, and AK had full access to interim reports on statistics, analyses, and tables. All authors contributed to the article and approved the submitted version.

Funding

This work was supported mainly by the Parnassia Groep Psychiatric Institute (The Hague/ Rotterdam, the Netherlands); it was also supported by Janssen-Cilag B. V. with additional funding of an unconditional educational grant (1-4H1ZDZ67890). The funding organizations had no role in the study design, data collection, data analysis, data interpretation, writing of the report, or the decision to publish the study.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We thank the patients, research assistants, clinicians, and staff at Parnassia Groep and Yulius Mental Health (especially Fred Termeer) who participated in this study.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2021.723435/full#supplementary-material

Supplementary Table 1. Coefficients, odds ratios and CI's, predicting non-completion from baseline characteristics.

Supplementary Data Sheet 1. Positive decision Erasmus MC.

Supplementary Data Sheet 2. Original study Protocol for the METC.

References

1. Schaub A, Mueser KT, von Werder T, Engel R, Moller HJ, Falkai P. A randomized controlled trial of group coping-oriented therapy vs supportive therapy in schizophrenia: results of a 2-year follow-up. Schizophr Bull. (2016) 42(Suppl. 1):S71–80. doi: 10.1093/schbul/sbw032

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Morgan C, Burns T, Fitzpatrick R, Pinfold V, Priebe S. Social exclusion and mental health: conceptual and methodological review. Br J Psychiatry. (2007) 191:477–83. doi: 10.1192/bjp.bp.106.034942

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Mueser KT, Meyer PS, Penn DL, Clancy R, Clancy DM, Salyers MP. The illness management and recovery program: rationale, development, and preliminary findings. Schizophr Bull. (2006) 32(Suppl. 1):S32–43. doi: 10.1093/schbul/sbl022

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Slade M, Amering M, Farkas M, Hamilton B, O'Hagan M, Panther G, et al. Uses and abuses of recovery: implementing recovery-oriented practices in mental health systems. World Psychiatry. (2014) 13:12–20. doi: 10.1002/wps.20084

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Andresen R, Caputi P, Oades LG. Do clinical outcome measures assess consumer-defined recovery? Psychiatry Res. (2010) 177:309–17. doi: 10.1016/j.psychres.2010.02.013

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Roosenschoon BJ, Mulder CL, Deen ML, van Weeghel J. Effectiveness of illness management and recovery (IMR) in the Netherlands: a randomised clinical trial; Study Protocol. BMC Psychiatry. (2016) 16:11. doi: 10.1186/s12888-016-0774-0

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Lean M, Fornells-Ambrojo M, Milton A, Lloyd-Evans B, Harrison-Stewart B, Yesufu-Udechuku A, et al. Self-management interventions for people with severe mental illness: systematic review and meta-analysis. Br J Psychiatry. (2019) 214:260–8. doi: 10.1192/bjp.2019.54

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Mueser KT, Roe D. Schizophrenia disorders. In: Norcross JC VG, Freedheim DK, Pole N, editor. APA Handbooks in Psychology APA Handbook of Clinical Psychology: Psychopathology and Health. Washington, DC: APA Books (2016). p. 225–51.

Google Scholar

9. Mueser KT, Corrigan PW, Hilton DW, Tanzman B, Schaub A, Gingerich S, et al. Illness management and recovery: a review of the research. Psychiatr Serv. (2002) 53:1272–84. doi: 10.1176/appi.ps.53.10.1272

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Slade M, Leamy M, Bacon F, Janosik M, Le Boutillier C, Williams J, et al. International differences in understanding recovery: systematic review. Epidemiol Psychiatr Sci. (2012) 21:353–64. doi: 10.1017/S2045796012000133

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Roosenschoon BJ, Kamperman AM, Deen ML, Weeghel JV, Mulder CL. Determinants of clinical, functional and personal recovery for people with schizophrenia and other severe mental illnesses: a cross-sectional analysis. PLoS ONE. (2019) 14:e0222378. doi: 10.1371/journal.pone.0222378

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Jaaskelainen E, Juola P, Hirvonen N, McGrath JJ, Saha S, Isohanni M, et al. A systematic review and meta-analysis of recovery in schizophrenia. Schizophr Bull. (2013) 39:1296–306. doi: 10.1093/schbul/sbs130

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Bellack AS. Scientific and consumer models of recovery in schizophrenia: concordance, contrasts, and implications. Schizophr Bull. (2006) 32:432–42. doi: 10.1093/schbul/sbj044

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Whitley R, Drake RE. Recovery: a dimensional approach. Psychiatr Serv. (2010) 61:1248–50. doi: 10.1176/ps.2010.61.12.1248

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Van Eck RM, Burger TJ, Vellinga A, Schirmbeck F, de Haan L. The relationship between clinical and personal recovery in patients with schizophrenia spectrum disorders: a systematic review and meta-analysis. Schizophr Bull. (2018) 44:631–42. doi: 10.1093/schbul/sbx088

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Robinson DG, Woerner MG, McMeniman M, Mendelowitz A, Bilder RM. Symptomatic and functional recovery from a first episode of schizophrenia or schizoaffective disorder. Am J Psychiatry. (2004) 161:473–9. doi: 10.1176/appi.ajp.161.3.473

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Harvey PD, Bellack AS. Toward a terminology for functional recovery in schizophrenia: is functional remission a viable concept? Schizophr Bull. (2009) 35:300–6. doi: 10.1093/schbul/sbn171

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Carrion RE, McLaughlin D, Goldberg TE, Auther AM, Olsen RH, Olvet DM, et al. Prediction of functional outcome in individuals at clinical high risk for psychosis. JAMA Psychiatry. (2013) 70:1133–42. doi: 10.1001/jamapsychiatry.2013.1909

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Deegan P. Recovery as a self-directed process of healing and transformation. Occup Ther Mental Health. (2002) 17:5–21. doi: 10.1300/J004v17n03_02

CrossRef Full Text | Google Scholar

20. Leamy M, Bird V, Le Boutillier C, Williams J, Slade M. Conceptual framework for personal recovery in mental health: systematic review and narrative synthesis. Br J Psychiatry. (2011) 199:445–52. doi: 10.1192/bjp.bp.110.083733

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Hasson-Ohayon I, Roe D, Kravetz S. A randomized controlled trial of the effectiveness of the illness management and recovery program. Psychiatr Serv. (2007) 58:1461–6. doi: 10.1176/ps.2007.58.11.1461

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Levitt AJ, Mueser KT, Degenova J, Lorenzo J, Bradford-Watt D, Barbosa A, et al. Randomized controlled trial of illness management and recovery in multiple-unit supportive housing. Psychiatr Serv. (2009) 60:1629–36. doi: 10.1176/ps.2009.60.12.1629

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Fardig R, Lewander T, Melin L, Folke F, Fredriksson A. A randomized controlled trial of the illness management and recovery program for persons with schizophrenia. Psychiatr Serv. (2011) 62:606–12. doi: 10.1176/ps.62.6.pss6206_0606

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Salyers MP, McGuire AB, Kukla M, Fukui S, Lysaker PH, Mueser KT. A randomized controlled trial of illness management and recovery with an active control group. Psychiatr Serv. (2014) 65:1005–11. doi: 10.1176/appi.ps.201300354

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Dalum HS, Waldemar AK, Korsbek L, Hjorthoj C, Mikkelsen JH, Thomsen K, et al. Participants' and staffs' evaluation of the illness management and recovery program: a randomized clinical trial. J Ment Health. (2016) 27:30–7. doi: 10.1080/09638237.2016.1244716

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Dalum HS, Waldemar AK, Korsbek L, Hjorthoj C, Mikkelsen JH, Thomsen K, et al. Illness management and recovery: clinical outcomes of a randomized clinical trial in community mental health centers. PLoS ONE. (2018) 13:e0194027. doi: 10.1371/journal.pone.0194027

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Jensen SB, Dalum HS, Korsbek L, Hjorthoj C, Mikkelsen JH, Thomsen K, et al. Illness management and recovery: one-year follow-up of a randomized controlled trial in Danish community mental health centers: long-term effects on clinical and personal recovery. BMC Psychiatry. (2019) 19:65. doi: 10.1186/s12888-019-2048-0

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Tan CHS, Ishak RB, Lim TXG, Marimuthusamy P, Kaurss K, Leong JJ. Illness management and recovery program for mental health problems: reducing symptoms and increasing social functioning. J Clin Nurs. (2017) 26:3471–85. doi: 10.1111/jocn.13712

PubMed Abstract | CrossRef Full Text | Google Scholar

29. McGuire AB, Kukla M, Green A, Gilbride D, Mueser KT, Salyers MP. Illness management and recovery: a review of the literature. Psychiatr Serv. (2014) 65:171–9. doi: 10.1176/appi.ps.201200274

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Bond GR, Drake RE, McHugo GJ, Rapp CA, Whitley R. Strategies for improving fidelity in the National evidence-based practices project. Res Soc Work Pract. (2009) 19:569–81. doi: 10.1177/1049731509335531

CrossRef Full Text | Google Scholar

31. Egeland KM, Heiervang KS, Landers M, Ruud T, Drake RE, Bond GR. Psychometric properties of a fidelity scale for illness management and recovery. Adm Policy Ment Health. (2019) 47:885–93. doi: 10.1007/s10488-019-00992-5

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Bond GR, Drake RE. Assessing the fidelity of evidence-based practices: history and current status of a standardized measurement methodology. Adm Policy Ment Health. (2020) 47:874–84. doi: 10.1007/s10488-019-00991-6

PubMed Abstract | CrossRef Full Text | Google Scholar

33. McGrew JH, Bond GR, Dietzen L, Salyers M. Measuring the fidelity of implementation of a mental health program model. J Consult Clin Psychol. (1994) 62:670–8. doi: 10.1037/0022-006X.62.4.670

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Priebe S, Fakhoury W, Watts J, Bebbington P, Burns T, Johnson S, et al. Assertive outreach teams in London: patient characteristics and outcomes. Pan-London Assertive Outreach Study, part 3. Br J Psychiatry. (2003) 183:148–54. doi: 10.1192/bjp.183.2.148

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Latimer EA. Economic impacts of assertive community treatment: a review of the literature. Can J Psychiatry. (1999) 44:443–54. doi: 10.1177/070674379904400504

PubMed Abstract | CrossRef Full Text | Google Scholar

36. van Vugt MD, Kroon H, Delespaul PA, Dreef FG, Nugter A, Roosenschoon BJ, et al. Assertive community treatment in the Netherlands: outcome and model fidelity. Can J Psychiatry. (2011) 56:154–60. doi: 10.1177/070674371105600305

PubMed Abstract | CrossRef Full Text | Google Scholar

37. McHugo GJ, Drake RE, Teague GB, Xie H. Fidelity to assertive community treatment and client outcomes in the New Hampshire dual disorders study. Psychiatr Serv. (1999) 50:818–24. doi: 10.1176/ps.50.6.818

PubMed Abstract | CrossRef Full Text | Google Scholar

38. de Winter L, Couwenbergh C, van Weeghel J, Bergmans C, Bond GR. Fidelity and IPS: does quality of implementation predict vocational outcomes over time for organizations treating persons with severe mental illness in the Netherlands? Soc Psychiatry Psychiatr Epidemiol. (2020) 55:1607–17. doi: 10.1007/s00127-020-01890-0

PubMed Abstract | CrossRef Full Text | Google Scholar

39. McGuire AB, White DA, Bartholomew T, Flanagan ME, McGrew JH, Rollins AL, et al. The relationship between provider competence, content exposure, and consumer outcomes in illness management and recovery programs. Adm Policy Ment Health. (2017) 44:81–91. doi: 10.1007/s10488-015-0701-6

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Monroe-DeVita M, Morse G, Mueser KT, McHugo GJ, Xie H, Hallgren KA, et al. Implementing illness management and recovery within assertive community treatment: a pilot trial of feasibility and effectiveness. Psychiatr Serv. (2018) 69:562–71. doi: 10.1176/appi.ps.201700124

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Salyers MP, Godfrey JL, Mueser KT, Labriola S. Measuring illness management outcomes: a psychometric study of clinician and consumer rating scales for illness self management and recovery. Community Ment Health J. (2007) 43:459–80. doi: 10.1007/s10597-007-9087-6

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Goossens PJJ, Beentjes TAA, Knol S, Salyers MP, de Vries SJ. Investigating the reliability and validity of the Dutch versions of the illness management and recovery scales among clients with mental disorders. J Ment Health. (2017) 26:489–95. doi: 10.3109/09638237.2015.1124398

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Mueser KT, Gingerich S, Salyers MP, McGuire AB, Reyes RU, Cunningham H. The Illness Management and Recovery (IMR) Scales, Client and Clinician Versions. Concord, NH: New Hampshire-Dartmouth Psychiatric Research Center (2004).

PubMed Abstract | Google Scholar

44. Chesney MA, Neilands TB, Chambers DB, Taylor JM, Folkman S. A validity and reliability study of the coping self-efficacy scale. Br J Health Psychol. (2006) 11:421–37. doi: 10.1348/135910705X53155

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Zimet GD, Powell SS, Farley GK, Werkman S, Berkoff KA. Psychometric characteristics of the multidimensional scale of perceived social support. J Pers Assess. (1990) 55:610–7. doi: 10.1207/s15327752jpa5503&amp;4_17

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Tait L, Birchwood M, Trower P. A new scale (SES) to measure engagement with community mental health services. J Ment Health. (2002) 11:191–8. doi: 10.1080/09638230020023570-2

PubMed Abstract | CrossRef Full Text | Google Scholar

47. McLellan AT, Luborsky L, Woody GE, O'Brien CP. An improved diagnostic evaluation instrument for substance abuse patients. The Addiction Severity Index. J Nerv Ment Dis. (1980) 168:26–33. doi: 10.1097/00005053-198001000-00006

PubMed Abstract | CrossRef Full Text

48. Birchwood M, Smith J, Drury V, Healy J, Macmillan F, Slade M. A self-report insight scale for psychosis: reliability, validity and sensitivity to change. Acta Psychiatr Scand. (1994) 89:62–7. doi: 10.1111/j.1600-0447.1994.tb01487.x

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Derogatis LR, Melisaratos N. The brief symptom inventory: an introductory report. Psychol Med. (1983) 13:595–605. doi: 10.1017/S0033291700048017

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Birchwood M, Smith J, Cochrane R, Wetton S, Copestake S. The social functioning scale. The development and validation of a new scale of social adjustment for use in family intervention programmes with schizophrenic patients. Br J Psychiatry. (1990) 157:853–9. doi: 10.1192/bjp.157.6.853

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Young SL, Bullock WA. The Mental Health Recovery Measure. Toledo, OH: University of Toledo, Department of Psychology (2003).

52. Ritsher JB, Otilingam PG, Grajales M. Internalized stigma of mental illness: psychometric properties of a new measure. Psychiatry Res. (2003) 121:31–49. doi: 10.1016/j.psychres.2003.08.008

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Lecomte T, Corbiere M, Laisne F. Investigating self-esteem in individuals with schizophrenia: relevance of the self-esteem rating scale-short form. Psychiatry Res. (2006) 143:99–108. doi: 10.1016/j.psychres.2005.08.019

PubMed Abstract | CrossRef Full Text | Google Scholar

54. McHugo GJ, Drake RE, Whitley R, Bond GR, Campbell K, Rapp CA, et al. Fidelity outcomes in the National implementing evidence-based practices project. Psychiatr Serv. (2007) 58:1279–84. doi: 10.1176/ps.2007.58.10.1279

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Roosenschoon BJ, van Weeghel J, Bogaards M, Deen ML, Mulder CL. Illness Management & Recovery (IMR) in the Netherlands; a naturalistic pilot study to explore the feasibility of a randomized controlled trial. BMC Psychiatry. (2016) 16:391. doi: 10.1186/s12888-016-1096-y

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Gingerich S, Mueser KT. Illness Management and Recovery: Personalized Skills and Strategies for Those With Mental Illness. 3rd ed. Center City, MN: Hazelden (2011).

PubMed Abstract

57. Barrowclough C, Tarrier N, Humphreys L, Ward J, Gregg L, Andrews B. Self-esteem in schizophrenia: relationships between self-evaluation, family attitudes, and symptomatology. J Abnorm Psychol. (2003) 112:92–9. doi: 10.1037/0021-843X.112.1.92

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Guanghan L, Kung-Yee L. Sample size calculations for studies with correlated observations. Biometrics. (1997) 53:937–47. doi: 10.2307/2533554

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Twisk JWR. Applied Longitudinal Data Analysis for Epidemiology - A Practical Guide. Newyork, NY: Cambridge University Press (2003).

Google Scholar

60. Cohen JM. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Hillsdale, NJ: Erlbaum (1988).

PubMed Abstract | Google Scholar

61. Twisk J, de Boer M, de Vente W, Heymans M. Multiple imputation of missing values was not necessary before performing a longitudinal mixed-model analysis. J Clin Epidemiol. (2013) 66:1022–8. doi: 10.1016/j.jclinepi.2013.03.017

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Thissen D, Steinberg L, Kuang D. Quick and easy implementation of the Benjamini-Hochberg procedure for controlling the false positive rate in multiple comparisons. J Educ Behav Stat. (2002) 27:77–83. doi: 10.3102/10769986027001077

CrossRef Full Text | Google Scholar

63. Schulz KF, Grimes DA. Multiplicity in randomised trials I: endpoints and treatments. Lancet. (2005) 365:1591–5. doi: 10.1016/S0140-6736(05)66461-6

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Perneger TV. What's wrong with Bonferroni adjustments. BMJ. (1998) 316:1236–8. doi: 10.1136/bmj.316.7139.1236

PubMed Abstract | CrossRef Full Text | Google Scholar

65. Canacott L, Moghaddam N, Tickle A. Is the Wellness Recovery Action Plan (WRAP) efficacious for improving personal and clinical recovery outcomes? A systematic review and meta-analysis. Psychiatr Rehabil J. (2019) 42:372–81. doi: 10.1037/prj0000368

PubMed Abstract | CrossRef Full Text | Google Scholar

66. Slade M, Amering M, Oades L. Recovery: an international perspective. Epidemiol Psichiatr Soc. (2008) 17:128–37. doi: 10.1017/S1121189X00002827

PubMed Abstract | CrossRef Full Text | Google Scholar

67. van Os J, Guloksuz S, Vijn TW, Hafkenscheid A, Delespaul P. The evidence-based group-level symptom-reduction model as the organizing principle for mental health care: time for change? World Psychiatry. (2019) 18:88–96. doi: 10.1002/wps.20609

PubMed Abstract | CrossRef Full Text | Google Scholar

68. Markowitz FE. Modeling processes in recovery from mental illness: relationships between symptoms, life satisfaction, and self-concept. J Health Soc Behav. (2001) 42:64–79. doi: 10.2307/3090227

PubMed Abstract | CrossRef Full Text | Google Scholar

69. Anthony WA, Cohen M, Farkas M, Gagne C. Psychiatric Rehabilitation. 2 ed. Boston, MA: Center of Psychiatric Rehabilitation Boston University (2002).

Google Scholar

70. van Veldhuizen JR. FACT: a Dutch version of ACT. Community Ment Health J. (2007) 43:421–33. doi: 10.1007/s10597-007-9089-4

PubMed Abstract | CrossRef Full Text | Google Scholar

71. Rothman KJ. No adjustments are needed for multiple comparisons. Epidemiology. (1990) 1:43–6. doi: 10.1097/00001648-199001000-00010

CrossRef Full Text | Google Scholar