Cost Effectiveness and Budget Impact of the Boston University Approach to Psychiatric Rehabilitation for Increasing the Social Participation of Individuals With Severe Mental Illnesses

Background: The purpose of this study was to investigate the cost-effectiveness and budget impact of the Boston University Approach to Psychiatric Rehabilitation (BPR) compared to an active control condition (ACC) to increase the social participation (in competitive employment, unpaid work, education, and meaningful daily activities) of individuals with severe mental illnesses (SMIs). ACC can be described as treatment as usual but with an active component, namely the explicit assignment of providing support with rehabilitation goals in the area of social participation.

Method: In a randomized clinical trial with 188 individuals with SMIs, BPR (n = 98) was compared to ACC (n = 90). Costs were assessed with the Treatment Inventory of Costs in Patients with psychiatric disorders (TIC-P). Outcome measures for the cost-effectiveness analysis were incremental cost per Quality Adjusted Life Year (QALY) and incremental cost per proportional change in social participation. Budget Impact was investigated using four implementation scenarios and two costing variants.

Results: Total costs per participant at 12-month follow-up were € 12,886 in BPR and € 12,012 in ACC, a non-significant difference. There were no differences with regard to social participation or QALYs. Therefore, BPR was not cost-effective compared to ACC. Types of expenditure with the highest costs were in order of magnitude: supported and sheltered housing, inpatient care, outpatient care, and organized activities. Estimated budget impact of wide BPR implementation ranged from cost savings to €190 million, depending on assumptions regarding uptake. There were no differences between the two costing variants meaning that from a health insurer perspective, there would be no additional costs if BPR was implemented on a wider scale in mental health care institutions.

Conclusions: This was the first study to investigate BPR cost-effectiveness and budget impact. The results showed that BPR was not cost-effective compared to ACC. When interpreting the results, one must keep in mind that the cost-effectiveness of BPR was investigated in the area of social participation, while BPR was designed to offer support in all rehabilitation areas. Therefore, more studies are needed before definite conclusions can be drawn on the cost-effectiveness of the method as a whole.

Introduction

Individuals with severe mental illnesses (SMIs) experience serious social participation problems. Frequently reported issues are high unemployment rates (1–3) and difficulties participating in daytime activities such as education, unpaid employment or activities outside the home (4, 5). Participation in these activities is important because it facilitates recovery and is desired by most people with SMIs (6–8). Furthermore, competitive employment increases income and alleviates poverty (9). For instance, Levinson et al. (10) found that individuals with SMIs earned -on average- a third less than median earnings in over 19 countries in the Americas, Europe, the Middle East, Africa, Asia and New Zealand.

The low employment rates of people with SMIs not only have personal implications, but also implications for society at large. In the Netherlands and the UK, people with mental health problems represent the largest group (30–40%) of disability benefits claimants (11, 12). Therefore, increasing the social participation of individuals with SMIs may not only facilitate their recovery and improve life satisfaction, but also increase their productivity and reduce benefit payments, which is beneficial for society. Considering these problems, people with SMIs need structured supports with their social participation and this could be achieved through psychiatric rehabilitation. The aim of psychiatric rehabilitation is to “help persons with psychiatric disabilities increase their ability to function successfully and be satisfied in the environment of their choice with the least amount of ongoing professional intervention (13)”. It differs from standard care approaches with regard to its focus on personal goal attainment and (re)gaining various social and personal roles.

Several rehabilitation approaches have been developed to support individuals with SMIs with rehabilitation goals such as increasing their social participation. Some approaches target one aspect of rehabilitation, such as Individual Placement and Support (IPS) for competitive employment (14). Others, such the Strengths model or the Boston University Approach to Psychiatric Rehabilitation (BPR) (13, 15), focus on diverse rehabilitation areas.

Several trials have confirmed that BPR effectively improves social participation and functioning (16–19), and the approach has been implemented in many Dutch Mental Health Care (MHC) facilities. Nevertheless, dissemination is slow and BPR availability remains limited. An important reason for limited implementation is the fear that BPR will be time-consuming and expensive. Cost insights appear essential to further promote implementation, particularly as many MHC institutions are dealing with ongoing budget cuts.

To date, no studies on the cost-effectiveness of BPR have been conducted. Therefore, a randomized controlled trial (RCT) was conducted to investigate the cost-effectiveness and budget impact of BPR compared to an active control condition (ACC) for individuals with SMIs who have a wish for change regarding social participation. An active control condition was chosen to ensure that both conditions received equal amounts of attention and support.

It was hypothesized that BPR would be cost-effective compared to ACC because: (1) based on earlier studies, larger increases in social participation were expected combined with few additional costs, and (2) the methodology's positive effects on functioning would lower MHC costs.

Methods

Design

In brief, from 2014–2017, an RCT comparing BPR to ACC was accompanied by quality of life and resource use measurement. A trial-based economic evaluation was conducted alongside this RCT.

Resources were costed using guideline-based unit prices, and cost-effectiveness over the trial follow-up time of 12 months was calculated. For the Budget impact analysis, four scenarios were developed and total costs calculated over a 4 year time horizon. A 4-year time horizon was chosen based on the Dutch and international guidelines (20, 21).

Randomization occurred through block randomization by an independent researcher, and participants were stratified by center and previous work experience. There was no consumer choice. Detailed descriptions of the study protocol and results on BPR effectiveness compared to ACC are presented elsewhere (22, 23).

Participants

In total, 188 participants with SMIs were recruited at two organizations offering supported and sheltered housing, at three regional MHC centers offering outpatient care [Functional Assertive Community Teams (F-ACT); van Veldhuizen (24)], a team for patients with enduring eating disorders, and an outpatient team for substance addiction. Participants needed to have severe mental illness (a DSM-V diagnosis, long duration of service contact and functional impairments which substantially interfere with or limit major life activities), be between 18–64 years old, and have a wish for change in social participation. Participants were excluded if hospitalized during enrolment, except when treated for severe eating disorders. Participants were randomized to receive either BPR (98 participants) or ACC (90 participants), and were assessed at baseline, and at 6 and 12 months after enrollment.

Interventions

BPR

BPR is a systematic psychiatric rehabilitation approach designed to help individuals with SMIs achieve and retain rehabilitation goals with regard to housing, education, work and social contacts. The approach consists of four phases: exploring, choosing, getting and keeping rehabilitation goals (13, 25). Each phase consists of several techniques that can be used by BPR practitioners to optimally support their patients. BPR is characterized by its person-centered focus: the patient sets the goal and directs the pace of the rehabilitation process and the practitioner has a facilitating role.

BPR was delivered by 28 BPR-practitioners: mental health professionals from different vocational backgrounds who had all received additional training in BPR from the Dutch training institute for BPR, R92 or internal training from BPR trainers licensed by R92.

ACC

ACC can be described as treatment as usual with an active component. The mental health practitioners in the ACC group did not work according to a systematic rehabilitation method, however they did receive the explicit assignment of providing support with rehabilitation goals in the area of social participation. Participants in the ACC condition were also offered at least one session every 2 weeks. ACC was delivered by 55 practitioners who also offered active rehabilitative support with participants' personal goals, but without using a structured rehabilitation methodology. These mental health professionals came from vocational backgrounds comparable to the BPR practitioners but had not received specialized training.

In both conditions, participants were offered at least one session every 2 weeks without a predetermined minimum or maximum number of sessions to be completed. Furthermore, practitioners in both conditions were allowed to involve additional resources such as specialized vocational services that are available in most Dutch municipalities for people with problems regarding work participation, including facilities specialized for unemployed persons with mental health problems and a distance to the labor market.

There were no differences between practitioners in ACC and BPR with regard to educational level but ACC practitioners had significantly more years of experience than BPR practitioners [BPR: M (SD) = 15,66 (9,75); ACC: M (SD) = 20,09 (12,22); p = 0.007].

Measures

Costs

Costs were assessed according to the Dutch guidelines for economic evaluation (21). Healthcare resource use was obtained at baseline, 6 months and 12 months, using the Treatment Inventory of Costs in Patients with psychiatric disorders (TIC-P) (26), adapted to match the specific context of the target population. For medication costs, prices from the year 2017 based on price information from the National Health Care Institute were applied (27). For other costs, unit prices were taken from the cost manual by Hakkaart-van Roijen et al. (28) [an appendix to the Dutch guidelines for economic evaluation (21)], which were based on 2014 and indexed to 2017 using the national consumer price index (29). Full details on unit prices per type of costs can be found in Appendix 1. The friction cost method was applied for productivity losses (28). The friction cost method is a way of estimating the costs of production loss from the absence of an employee during the period that is needed to fully replace that employee by another one (the friction period). The underlying assumption is that the previous level of productivity will be resumed by the new employee. Standardized friction periods and hourly rates are available from the Dutch guideline for economic evaluations (21). In case of partial absenteeism, the number of absent hours was multiplied by the hourly wage rate. In case of full absenteeism, the job size in hours was multiplied by the hourly rate, up to a maximum of 85 days (the 2014 friction period). In accordance with the guideline, the same procedure was adopted for unpaid work using corresponding prices from the cost manual (28).

Intervention costs were based on the number of sessions and costs per hour for practitioners. Sometimes participants were referred to external job coaches. These are job coaches that are not part of the patients' team or mental health organization but employees that work with a specialized vocational service outside the mental health organization. These external job coaches are supported by local employment services that are available for people with a distance to the labor market in most Dutch municipalities, and also support unemployed persons with mental health difficulties. Because no detailed data on the exact hours spent with these external job coaches were available, the average duration and price for job coaches reimbursed by the Employee Insurance Agency (UWV) was used (8 × € 92 = € 736) (30). The training that BPR practitioners received (initial training) was already provided before the start of the study and practitioners had been working along the lines of this approach with patients other than those participating in the trial. Therefore, the per patient costs of the initial training are low and uncertain and were not included in the total intervention costs. Travel costs were not measured and not included in the analysis since they were deemed small compared to other cost elements.

Effectiveness

Generic health status was measured at baseline, 6 months and 12 months, with the 12-item Short Form Health Survey (SF-12) (31), which was transformed into SF-6D scores using the methodology developed by Brazier and Roberts (32). The SF-6D utility scores were subsequently used to derive Quality Adjusted Life Years (QALYs) at 6 and 12 months (33). Social participation over the past 6 months was assessed using the Occupation and Employment subscale of the Birchwood Social Functioning Scale (SFS_OE) (34). The raw SFS_OE score was dichotomized in no employment (scores 0–6) vs. employment (scores = >7).

Data Analysis

Cost Effectiveness Analysis (CEA)

The CEA was performed from a societal perspective. The study had a 12-month time horizon; costs and health outcomes were not discounted (33). Cost-effectiveness outcomes were expressed as incremental costs per percentage change in the dichotomized SFS_OE subscale. Furthermore, a cost utility analysis was conducted resulting in incremental costs per QALY gained. Missing values were estimated using Multiple Imputation in SPSS-25 and aggregated using Rubin's rule (35). Analyses were repeated on individuals with complete data. All CEAs were conducted in SPSS v25 (36).

Budget Impact Analyses (BIA)

The BIA was conducted according to the guidelines of Sullivan et al. (20), from a societal perspective and using unit prices as in the CEA. Trial results were extrapolated to a time horizon of up to 4 years and to the entire Dutch population of people with SMIs. Data of participants with complete data were used. The results of the cost-effectiveness analysis were combined with epidemiological data on the size of the target population, and data on BPR program scale and implementation. The BIA was performed for four scenarios: (1) a maximum implementation scenario assuming participation of all eligible patients with SMIs between 18–64 years; (2) An optimistic trial scenario assuming inclusion criteria and participation rates as observed in the study, but adjusting for drop-out, (3) A conservative trial scenario assuming inclusion criteria, participation rates and dropout rates as observed in the study, 4) A scenario assuming participation of only those who receive outpatient care from F-ACT (24) and are between 18–64 years.

Two costing variants were applied, exploring different perspectives and assumptions regarding costs per participant: In variant one, mean resource use per participant was taken from the trial results, and valued from a societal perspective. In variant two, a third party payer (health insurance) perspective was used, and all resource use was valued using actual reimbursement rules.

In contrast to the CEA, for the BIA, initial training costs were included as one-time up-front spending for the BPR scenarios, to reflect the investments needed for introducing it into current care.

Uncertainty Analysis

In a dedicated spreadsheet tool, bootstrapping with 1000 replications was performed to assess uncertainty surrounding the cost-effectiveness and cost-utility ratios, and around the differences in mean costs per participant for the BIA. The results are presented in incremental cost-effectiveness planes (CE-planes), and cost-effectiveness acceptability curves (CEACs), while for the BIA the 95%-interquantile ranges were used to present a range of possible values for the total costs of large scale implementation.

Results

All 188 participants were included in the economic evaluation. There were no significant baseline differences on any of the sociodemographic variables measured (Table 1).

TABLE 1

Table 1. Sociodemographic characteristics of study participants at baseline.

Costs

The average number of sessions did not significantly differ between conditions (BPR: 16.15 (SD = 11.37); ACC: 13.87 (SD = 12.81); t = −1.30, df = 186, P = 0.197). A detailed overview of costs is provided in Table 2. Total intervention costs after 12 months were €551 in BPR and €492 in ACC. Total costs including intervention costs did not significantly differ between BPR and ACC at 12 months (BPR: €12,886; ACC €12,012; 95% CI: €−3.59–€11.43). Types of expenditure in which most costs were made were in order of magnitude: supported and sheltered housing, inpatient care, outpatient care, and organized activities.

TABLE 2

Table 2. Differences in participation scores and QALYs (top part of table) and total pooled average costs across assessment points by type of expenditure in euros (price level 2017).

Effectiveness

The results on the effectiveness of BPR compared to ACC have been published elsewhere (23), and showed that participants in BPR as well as ACC significantly improved their social participation during the study period. However, BPR was not more effective than ACC on any of the primary or secondary outcome measures. The difference in participation score at 12 months (SFS) was not significant (BPR: 0.55; ACC: 0.62; 95% CI SFS: −0.20–0.08). The difference in QALYs at 12 months was very small and not significant (BPR: 0.68; ACC: 0.68; 95% CI QALY: −0.04–0.03), indicating that the intervention did not have a lasting effect on the quality of life when compared to ACC in this study.

Cost-Effectiveness

For both effectiveness measures, BPR was not cost-effective compared to ACC at 12-month follow-up. This is highlighted in the incremental CE planes (Figure 1). Because of the minimal difference in QALYs and the negative difference in social participation, it was not informative to calculate incremental cost-effectiveness ratios (ICERs).

FIGURE 1

Figure 1. Cost-effectiveness planes (costs are reported in euros). (A) Cost per increase in participation at 12 months; (B) Cost per increase in QALY at 12 months.

Figure 2 shows the CEACs. With the SFS as outcome (3a), the probability that BPR is cost-effective varies between 20–31%. With the QALY as outcome, the probability of BPR being cost-effective varies between 36–38%. This underlines the slim chance of BPR being cost-effective compared to ACC in this study sample.

FIGURE 2

Figure 2. Cost effectiveness acceptability curves for pooled imputed and complete data (costs are reported in euros). (A) Bootstrap SFS_OE at 12 months; (B) Bootstrap QALY at 12 months.

Complete Data

Complete data on costs and QALYs were available for 77 participants in BPR and 69 in ACC. Complete data on costs and SFS were available for 78 participants in BPR and 70 in ACC. Completers on costs per QALY had significantly fewer psychiatric symptoms than non-completers (t = 2.06, df = 54.99, p = 0.044). Completers on costs per increase in participation (SFS) were significantly more often male than non-completers χ²(1) = 5.00, p = 0.025). In both sets of completers there were no significant differences between completers and non-completers concerning duration in MHC, educational level, psychiatric diagnosis, or having paid employment.

Outcomes for completers still indicated that BPR was not cost-effective compared to ACC (dashed lines in Figures 2A,B).

Budget Impact Analysis (BIA)

Table 3 presents the results for the BIA. The estimated number of participants for the whole of the Netherlands in the various scenarios varied from 2,000 to 16,000 for follow-up years. In the first year it was substantially higher, since it was assumed all persons known with SMIs would be treated during this year, while in the follow-up years only new cases were included for treatment.

TABLE 3

Table 3. Results for budget impact analysis (price level 2017, in mln euros).

From the Third party payer perspective, no separate rate for guidance toward participation could be identified, and BPR and ACC were considered part of normal rates of care for this group of patients. Hence, from a health insurer perspective, no additional costs would occur if specialized mental health organizations started to use BPR more widely.

The BIA showed that from a societal perspective in scenario 1 (most optimistic regarding uptake of BPR), the expected budget impact in year 1 would be €101 million (95% IQR -€641to €870 million), and €22 million, (95%IQR -€140 million to €190 million) in year 2 and further. In scenario 2 (uptake as in the trial), the expected budget impact in year 1 would be €13 million (95% IQR -€81 million to €110 million) and €2.8 million (95% IQR -€18 million to €24 million) in year 2 and further. The two remaining scenarios had budget impact estimates in between these two extremes.

Results were mostly affected by differences in healthcare costs other than intervention costs of €685 (95% IQR -€7,794 to €9,535) and the difference in productivity costs of €665 (95% IQR -€670 to €1975). Differences in intervention costs were minor at around 2 euro per participant (95% IQR−130 to €129). All cost differences were highly uncertain, as reflected by the wide interquantile ranges, from cost savings to large additional costs.

Discussion

Although BPR effectiveness regarding social participation has been confirmed in a number of RCT's when compared to standard care or an active control condition, dissemination and availability remain limited. This may be due to a lack of insight into the associated costs for MHC services providing this support, and into the benefits for society in terms of potential increases in quality of life and -if paid employment is obtained or kept- in productivity. Therefore, this study examined the cost effectiveness and budget impact of BPR in the area of social participation.

The main study investigating the effectiveness of BPR compared to ACC for individuals with SMIs in the area of social participation showed that both interventions effectively improved social participation after 12 months but no difference between the two interventions could be shown. Therefore, a priori expectations of BPR being cost-effective compared to ACC were low.

Results confirmed this, and BPR had a <50% probability of being cost-effective compared to ACC in the study population. Because the difference in QALYs was so small, the effects were not only statistically, but also clinically insignificant.

When costs collected over a 12 month follow-up period were divided into different types of expenditures, it became apparent that the highest total costs in both the intervention and control group were related to supported and sheltered housing, followed by inpatient care, outpatient care, and engaging in organized activities. The costs of supported and sheltered housing were considerably higher than those related to the other categories.

For the budget impact, the difference in intervention costs was small and non-significant, while differences in other healthcare costs and productivity costs were substantial, but highly uncertain. This resulted in an estimated budget impact that could range from cost savings to €190 million (see Table 3), depending on assumptions regarding uptake such as assuming participation of all eligible patients with SMIs aged 18–64, or only those who receive care from F-ACT (and variants in between). As there is no separate rate for support aimed at improving social participation, there would be no additional costs if BPR was implemented on a wider scale in mental health care institutions, from a health insurer perspective. This implies that any additional costs will have to be accommodated for by the mental healthcare organizations.

As this is the first study to investigate the cost-effectiveness of BPR in any goal area, it is difficult to compare the results to existing studies. Only studies targeted at paid employment have been conducted, and they have shown mixed results. For instance, Dixon et al. (37) compared IPS to enhanced vocational rehabilitation (EVR). The results were inconclusive as to whether IPS was more cost-effective than EVR. On the other hand, a study comparing IPS to standard vocational services (38) found that IPS had a high probability of being cost-effective. The general lack of cost effectiveness studies regarding psychiatric rehabilitation methods, and the mixed results of existing studies, clearly indicate that more research is needed in this area.

A number of strong points are associated with our study. First, an active control condition was used in which practitioners were explicitly instructed to offer support with rehabilitation goals. Many studies that investigate the effectiveness of protocolled rehabilitation methods use a control condition in which standard care but no active support is offered, which may overestimate the effect of protocolled rehabilitation methods. Second, studies on the cost-effectiveness of psychiatric rehabilitation programs often do not measure and analyze costs from a societal perspective, whereas we did. Including costs within as well as outside of the healthcare sector is important to providing appropriate incentives to take into account when making decisions on allocation or reimbursement of health-related resources (39, 40).Third, as far as we know, this is the first study to investigate the cost-effectiveness of a psychiatric rehabilitation approach focused on attaining a wide range of social participation goals as opposed to just competitive employment. Not all people with SMIs are willing to engage or capable of engaging in paid employment. It is therefore important to incorporate other forms of social activities in this type of study, such as unpaid employment. Fourth, the attrition rate was very low compared to other studies involving people with SMIs.

A limitation was the collection of cost data through interviews with service users, which could be considered less reliable than administratively recorded data or data obtained from public registries. However, production loss involving paid or unpaid employment is also difficult to calculate from registries. Furthermore, a study by Patel et al. (41) showed congruence between data obtained with self-reports and case records, and the TIC-P is a widely used self -report measure for the assessment of costs (42). A second limitation was the follow-up period of 12 months. A previous study on BPR effectiveness by a member of the current study group showed that rehabilitation goal attainment almost doubled in BPR from 12 to 24 months after enrolment (16). It should be noted that this study already found better results for BPR compared to ACC at 12 months. Perhaps more time was needed for psychiatric rehabilitation support, and perhaps BPR may need more time to show its full potential. RCTs in mental healthcare often have short follow-up periods for financial reasons. This may interfere with a reliable assessment of costs. Sharfstein and Clark (43) emphasize the importance of assessing costs over a sufficiently long period of time in order to include both short- and long-term outcomes. This may particularly apply to psychiatric rehabilitation methods as their effects may not be limited to the short-term but may become apparent with some delay. The National Health Care Institute even recommends a life-long time horizon for economic evaluations in healthcare (21). On the other hand, it could also be argued that generic health status is a relatively stable construct, which is not easily influenced by an increase in social participation. As data on this subject are lacking, it is difficult to predict what the effect of a longer follow-up period would be. A final limitation is that QALYs were derived from the SF-6D, which measures generic health and is not particularly tailored to individuals with SMIs (44, 45). In a study by Mulhern et al. (46), the SF-6D and the EuroQol-5D were found to be reliable in populations with common mental disorders, but possibly less sensitive to change for use among individuals with schizophrenia. On the other hand, a generic measure of quality of life allows broad comparison of results to other interventions and target groups.

Cost-effectiveness studies on psychiatric rehabilitation programs are scarce, while information on their effectiveness is needed in order to make decisions on further implementation. This is particularly important since MHC organizations continue to be faced with financial constraints. Future studies on psychiatric rehabilitation programs should therefore incorporate cost-effectiveness and budget impact analyses within a sufficiently long time horizon to fully capture the relevant costs and benefits. Future research should also focus on further exploring differences between subgroups to identify specific subgroups that may particularly benefit from BPR. Examples of such subgroup comparisons are unemployed or working in sheltered workshops compared to competitive employment or supported and sheltered housing compared to Independent living. Furthermore, the approach of “social return on investment” as the ratio of benefits (clients' work earnings) to total investment for each client, expressed as a percentage could be an interesting angle to explore.

The current study showed that BPR was not cost-effective compared to ACC. People in both conditions considerably increased their social participation, however at comparable costs. It would be unfair to discard the value of BPR based on a single cost-effectiveness study in a specific goal area. There were many factors that influenced the effectiveness of BPR compared to ACC such as the fact that ACC has become increasingly focused on rehabilitation in the past few years, and that most participants in our study sample wanted to obtain paid employment, which is the goal that proved the most difficult to obtain. These and other factors are extensively discussed in the main article on BPR effectiveness from the same study population (23). A process evaluation could have shed more light on the active components of successful psychiatric rehabilitation, and would be recommended for future studies.

As this was the first study to investigate the cost-effectiveness of BPR, and because we limited BPR to the area of social participation for which it was not specifically designed, more studies are needed before definite conclusions can be drawn on the cost-effectiveness of the method as a whole.

Data Availability Statement

The datasets presented in this article are not readily available because restrictions apply to the availability of these data, which were used under license for this study. The data that support the findings of this study are available from the author WS, senior researcher, with the permission of Altrecht Institute for Mental Health Care. Requests to access the datasets should be directed to w.swildens@altrecht.nl

Ethics Statement

The studies involving human participants were reviewed and approved by University Medical Center Groningen Ethical approval reference number 2013/70. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

SS, TF, WS, JvB, JvW, and TvA contributed to the study conception and design and commented on previous versions of the manuscript. SS, WS, JvB, TF, and TvA performed material preparation and data collection and analysis. SS wrote the first draft of the manuscript. All authors contributed to the article and approved the submitted version.

Funding

This research was supported by a grant from the Netherlands Organization for Health Research and Development (ZonMw) (project number 837002006).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We thank Nico Rosenkrantz for his contribution on writing the SPSS syntaxes, and the guidance committee for their advice on the study. We also thank all study participants, participating organizations, data managers, and clinical and administrative staff who made the study possible.

References

1. Marwaha S, Johnson S, Bebbington P, Stafford M, Angermeyer MC, Brugha T, et al. Rates and correlates of employment in people with schizophrenia in the Uk, France and Germany. Br J Psych. (2007) 191:30–7. doi: 10.1192/bjp.bp.105.020982

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Salkever DS, Karakus MC, Slade EP, Harding CM, Hough RL, Rosenheck RA, et al. Measures and predictors of community-based employment and earnings of persons with schizophrenia in a multisite study. Psychiatr Serv. (2007) 58:315–24. doi: 10.1176/ps.2007.58.3.315

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Cook JA. Employment barriers for persons with psychiatric disabilities: a report for the President's new freedom commission. Psychiatr Serv. (2006) 57:1391–405. doi: 10.1176/ps.2006.57.10.1391

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Leufstadius C, Eklund M. Time use among individuals with persistent mental illness: identifying risk factors for imbalance in daily activities. Scand J Occup Ther. (2008) 15:23–33. doi: 10.1080/11038120701253428

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Burns-Lynch B, Brusilovskiy E, Salzer MS. An empirical study of the relationship between community participation, recovery, and quality of life of individuals with serious mental illnesses. Isr J Psychiatry Relat Sci. (2016) 53:46-54.

PubMed Abstract | Google Scholar

6. Mueser KT, Salyers MP, Mueser PR. A Prospective analysis of work in schizophrenia. Schizophr Bull. (2001) 27:281–96. doi: 10.1093/oxfordjournals.schbul.a006874

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Seebohm P, Secker J. What do service users want. In: Grove B, Secker J, Seebohm P, editors. New Thinking About Mental Health and Employment. Oxford: Radcliffe Publishing (2005). p. 11–8.

Google Scholar

8. Drake RE, Whitley R. Recovery and severe mental illness: description and analysis. Can J Psychiatry. (2014) 59:236–42. doi: 10.1177/070674371405900502

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Stuart H. Mental illness and employment discrimination. Curr Opin Psychiatry. (2006) 19:522–6. doi: 10.1097/01.yco.0000238482.27270.5d

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Levinson D, Lakoma MD, Petukhova M, Schoenbaum M, Zaslavsky AM, Angermeyer M, et al. Associations of serious mental illness with earnings: results from the who world mental health surveys. Br J Psych. (2010) 197:114–21. doi: 10.1192/bjp.bp.109.073635

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Burns T, Catty J, Becker T, Drake RE, Fioritti A, Knapp M, et al. The effectiveness of supported employment for people with severe mental illness: a randomised controlled trial. Lancet. (2007) 370:1146–52. doi: 10.1016/S0140-6736(07)61516-5

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Einerhand M, Ravesteijn B. Psychische Klachten Op De Arbeidsmarkt [Psychiatric complaints on the labor market]. ESB. (2017) 102:481–3.

Google Scholar

13. Anthony WA, Farkas MD. The Essential Guide to Psychiatric Rehabilitation Practice. Boston, MA: Boston University Center for Psychiatric Rehabilitation (2011). p. 73.

PubMed Abstract | Google Scholar

14. Becker MDR, Drake RE. Individual placement and support: a community mental health center approach to vocational rehabilitation. Community Ment Health J. (1994) 30:193–206. doi: 10.1007/BF02188630

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Rapp CA, Goscha RJ. The Strengths Model: A Recovery-Oriented Approach to Mental Health Services: New York, NY: Oxford University Press (2011).

Google Scholar

16. Swildens W, van Busschbach JT, Michon H, Kroon H, Koeter MW, Wiersma D, et al. Effectively working on rehabilitation goals: 24-month outcome of a randomized controlled trial of the Boston Psychiatric Rehabilitation approach. Can J Psychiatry. (2011) 56:751–60. doi: 10.1177/070674371105601207

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Svedberg P, Svensson B, Hansson L, Jormfeldt H. A 2-Year Follow-up study of people with severe mental illness involved in psychosocial rehabilitation. Nord J Psychiatry. (2014) 68:401–8. doi: 10.3109/08039488.2013.851737

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Gigantesco A, Vittorielli M, Pioli R, Falloon IR, Rossi G, Morosini P, et al. Memorial tribute: the vado approach in psychiatric rehabilitation: a randomized controlled trial. Psychiatr Serv. (2006) 57:1778–83. doi: 10.1176/ps.2006.57.12.1778

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Shern DL, Tsemberis S, Anthony W, Lovell AM, Richmond L, Felton CJ, et al. Serving street-dwelling individuals with psychiatric disabilities: outcomes of a psychiatric rehabilitation clinical trial. Am J Public Health. (2000) 90:1873–8. doi: 10.2105/AJPH.90.12.1873

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Sullivan SD, Mauskopf JA, Augustovski F, Caro JJ, Lee KM, Minchin M, et al. Budget impact analysis–principles of good practice: report of the ISPOR 2012 budget impact analysis good practice II task force. Value Health. (2014) 17:5–14. doi: 10.1016/j.jval.2013.08.2291

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Zorginstituut Nederland. Richtlijn Voor Het Uitvoeren Van Economische Evaluaties in De Gezondheidszorg [Guideline for Conducting Economic Evaluations in Healthcare] (2016).

Google Scholar

22. Sanches SA, Swildens WE, van Busschbach JT, Stant AD, Feenstra TL, van Weeghel J. Cost effectiveness and budgetary impact of the boston university approach to psychiatric rehabilitation for societal participation in people with severe mental illness: a randomised controlled trial protocol. BMC Psychiatry. (2015) 15:217. doi: 10.1186/s12888-015-0593-8

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Sanches SA, Swildens WE, Schaefer B, Moerbeek M, Feenstra TL, van Asselt T, et al. Effectiveness of the Boston University approach to psychiatric rehabilitation in improving social participation in people with severe mental illnesses: a randomized controlled trial. Front Psychiatry. (2020) 11:970. doi: 10.3389/fpsyt.2020.571640

PubMed Abstract | CrossRef Full Text | Google Scholar

24. van Veldhuizen JR. Fact: A Dutch version of act. Community Ment Health J. (2007) 43:421–33. doi: 10.1007/s10597-007-9089-4

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Korevaar L, Droës J, Van Wel T. Methodebeschrijving Individuele Rehabilitatiebenadering [Manual for the Boston University Approach to Psychiatric Rehabilitation]. Utrecht: Stichting Rehabilitatie (2010). p. 73.

Google Scholar

26. Hakkaart-van Roijen L, Straten Av, Tiemens B, Donker MCH. Handleiding Trimbos/Imta Questionnaire for Costs Associated with Psychiatric Illness (Tic-P). Rotterdam, Erasmus University (2002).

Google Scholar

27. National Health Care Institute. Pharmacy Price Information. (2020). Available online at: www.medicijnkosten.nl (accessed December 31, 2017).

Google Scholar

28. Hakkaart-van Roijen L, Van der Linden N, Bouwmans C, Kanters T, Tan SS. Kostenhandleiding. Methodologie Van Kostenonderzoek En Referentieprijzen Voor Economische Evaluaties in De Gezondheidszorg. In Opdracht Van Zorginstituut Nederland. Geactualiseerde Versie [Costs Manual]. (2015).

Google Scholar

29. Cental Bureau of Statistics StatLine. Consumer Price Index. (2020). Available online at: https://opendata.cbs.nl/statline (accessed June 1, 2021).

Google Scholar

30. Schuring M, Reeuwijk K, Burdorf A. Het Monitoren Van Arbeidstoeleiding Van Mensen Met Grote Afstand Tot De Arbeidsmarkt En Gezondheidsproblemen [Monitoring Job Placement of People with a Large Distance to the Labour Market and Health Problems]: Afdeling van Maatschappelijke Gezondheidszorg Erasmus Universitair Medisch Centrum (2016).

Google Scholar

31. Ware JE, Kosinski M, Keller SD. Sf-12: How to Score the Sf-12 Physical and Mental Health Summary Scales. Lincoln, RI: QualityMetric Incorporated: Health Institute, New England Medical Center (1998).

PubMed Abstract | Google Scholar

32. Brazier JE, Roberts J. The estimation of a preference-based measure of health from the Sf-12. Med Care. (2004) 42:851–9. doi: 10.1097/01.mlr.0000135827.18610.0d

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Drummond MF. Methods for the Economic Evaluation of Health Care Programmes. 4th ed. New York, NY: Oxford University Press (1988).

Google Scholar

34. Birchwood M, Smith JO, Cochrane R, Wetton S, Copestake SONJ. The social functioning scale. the development and validation of a new scale of social adjustment for use in family intervention programmes with schizophrenic patients. Br J Psych. (1990) 157:853–9. doi: 10.1192/bjp.157.6.853

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Rubin DB, Schenker N. Multiple imputation in health-are databases: an overview and some applications. Stat Med. (1991) 10:585–98. doi: 10.1002/sim.4780100410

PubMed Abstract | CrossRef Full Text | Google Scholar

36. IBM Corp. Ibm Spss Statistics for Windows Version 25.0. Armonk, NY: IBM Corp., (2017).

Google Scholar

37. Dixon L, Hoch JS, Clark R, Bebout R, Drake R, McHugo G, et al. Cost-effectiveness of two vocational rehabilitation programs for persons with severe mental illness. Psychiatr Serv. (2002) 53:1118–24. doi: 10.1176/appi.ps.53.9.1118

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Knapp M, Patel A, Curran C, Latimer E, Catty J, Becker T, et al. Supported employment: cost-effectiveness across six European sites. World J Psychiatry. (2013) 12:60–8. doi: 10.1002/wps.20017

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Johannesson M, Jönsson B, Jönsson L, Kobelt G, Zethreaus N. Why Should Economic Evaluations of Medical Technologies Have a Societal Perspective. OHE. (2009). doi: 10.2139/ssrn.2640123

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Jönsson B. Ten Arguments for a Societal Perspective in the Economic Evaluation of Medical Innovations. New york, NY:Springer. (2009). doi: 10.1007/s10198-009-0173-2

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Patel A, Rendu A, Moran P, Leese M, Mann A, Knapp M, et al. Comparison of two methods of collecting economic data in primary care. Fam Pract. (2005) 22:323–7. doi: 10.1093/fampra/cmi027

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Andersson E, Ljótsson B, Smit F, Paxling B, Hedman E, Lindefors N, et al. Cost-effectiveness of internet-based cognitive behavior therapy for irritable bowel syndrome: results from a randomized controlled trial. BMC Public Health. (2011) 11:215. doi: 10.1186/1471-2458-11-215

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Sharfstein SS, Clark HW. Economics and the chronic mental patient. Schizophr Bull. (1978) 4:399. doi: 10.1093/schbul/4.3.399

CrossRef Full Text | Google Scholar

44. Knapp M, Mangalore R. The trouble with QALYs…. Epidemiol Psychiatr Sci. (2007) 16:289–93. doi: 10.1017/S1121189X00002451

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Neumann PJ, Cohen JT. Qalys in 2018—Advantages and Concerns. Jama. (2018) 319:2473–4. doi: 10.1001/jama.2018.6072

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Mulhern B, Mukuria C, Barkham M, Knapp M, Byford S, Brazier J. Using Generic Preference-Based Measures in Mental Health: Psychometric Validity of the Eq-5d and Sf-6d. Br J Psychiatry. (2014) 205:236–43. doi: 10.1192/bjp.bp.112.122283

PubMed Abstract | CrossRef Full Text | Google Scholar

47. OECD European Union, UNESCO Institute for, Statistics,. Isced 2011 Operational Manual: Guidelines for Classifying National Education Programmes Related Qualifications: OECD Publishing (2015). Available online at: http://dx.doi.org/10.1787/9789264228368-en doi: 10.1787/9789264228368-en (accessed June 1, 2021).

CrossRef Full Text | Google Scholar

Appendix

TABLE A1

Table A1. Unit prices per type of costs in euros¹.