Reconstructing the value puzzle in health technology assessment: a pragmatic review to determine which modelling methods can account for additional value elements

Health technology assessment (HTA) has traditionally relied on cost-effectiveness analysis (CEA) as a cornerstone of evaluation of new therapies, assessing the clinical validity and utility, the efficacy, and the cost-effectiveness of new interventions. The current format of cost-effectiveness analysis, however, does not allow for inclusion of more holistic aspects of health and, therefore, value elements for new technologies such as the impact on patients and society beyond its pure clinical and economic value. This study aimed to review the recent modelling attempts to expand the traditional cost-effectiveness analysis approach by incorporating additional elements of value in health technology assessment. A pragmatic literature review was conducted for articles published between 2012 and 2022 reporting cost-effectiveness analysis including value aspects beyond the clinical and cost-effectiveness estimates; searches identified 13 articles that were eligible for inclusion. These expanded modelling approaches mainly focused on integrating the impact of societal values and health equity in cost-effectiveness analysis, both of which were championed as important aspects of health technology assessment that should be incorporated into future technology assessments. The reviewed cost-effectiveness analysis methods included modification of the current cost-effectiveness analysis methodology (distributional cost-effectiveness analysis, augmented cost-effectiveness analysis, extended cost-effectiveness analysis) or the use of multi-criteria decision analysis. Of these approaches, augmented cost-effectiveness analysis appears to have the most potential by expanding traditional aspects of value, as it uses techniques already familiar to health technology assessment agencies but also allows space for incorporation of qualitative aspects of a product’s value. This review showcases that methods to unravel additional value elements for technology assessment exist, therefore, patient access to promising technologies can be improved by moving the discussion from “if” to “how” additional value elements can inform decision-making.

Introduction

Value in health technology assessment (HTA), which is the foundation upon which decision-making regarding new drugs and health technologies is made in several healthcare systems, has been primarily based on balancing the clinical benefits to patients and/or economic costs involved by introducing the new technology to the healthcare system. Expansion of the concept of value in HTAs has been the subject of recent research and debate mainly driven by patients, carers and clinicians who recognize that the value of a new technology is multidimensional (Caro et al., 2019; Reed et al., 2019). This multidimensional nature is reflected in the latest definition of HTA provided by the Professional Society for Health Economics and Outcomes Research (ISPOR) Task Force which, in part, notes that dimension of value for a health technology may be assessed “by examining the intended and unintended consequences of using a health technology” and that this evaluation should encompass a comprehensive array of factors, including ethical, social, and cultural issues (O'Rourke et al., 2020a; O'Rourke et al., 2020b).

To this end, several organizations and research groups have developed value-based frameworks as an attempt to address the limitations of current HTA decision tools (Zhang et al., 2022). The ISPOR Strategic Task Force is leading an effort to reshape the future of HTA by examining the definition of a technology’s value and encouraging the integration of additional elements of value not currently included in the technology submissions. The findings regarding new concepts of value have been summarized in the ISPOR Task Force’s “Value Flower” (Lakdawalla et al., 2018). Some of the proposed elements beyond the traditional clinical and cost-effectiveness analyses include the value of: the reduction in uncertainty surrounding a disease, the fear of contagion, the value of insurance, the severity of disease, the value of hope, real option value, health equity, and scientific spillovers (Lakdawalla et al., 2018). Indeed, the Second Panel on Cost-effectiveness (Sanders et al., 2016) has recommended the incorporation of reference cases in each cost-effectiveness analysis (CEA) and an “impact inventory,” i.e., a cataloguing of consequences of analysis decisions both inside and outside of the healthcare sector. Previous research has also shown that even though these expanded value-based frameworks (generic or disease specific) provide the possibility of incorporating additional benefits that technologies may bring to patients and society and contextual factors to be considered through deliberative processes, there are practical limitations for their implementation in routine HTA decision-making (Willke et al., 2018; Breslau et al., 2023). One of the main barriers for the wider implementation of these value-based frameworks in decision-making, especially when CEA is the pillar of HTA, is the lack of consensus on how reliably and consistently these elements can be applied across different disease indications and technologies (Willke et al., 2018; Reed et al., 2019). Additionally, the lack of consensus regarding methods to address these concerns, the concerns of double counting of outcomes or interdependent variables raised by this lack of consensus and the historically narrow remit of HTA agencies (i.e., costs and benefits are assessed from a healthcare systems or payer perspective) represent significant barriers to widespread adoption (Fornaro et al. 2021; Hendriks and Pearson, 2021; Garrison et al., 2020). As a result, little traction has been gained thus far for their wider implementation.

Traditionally, CEA evaluates the value of an intervention from a clinical and cost perspective, determining value as a trade-off between cost and health benefit (Canadian Institute for Health Information (CIHI), 2022; Guidelines for the economic, 2017). The structured nature of CEA contributes to its appeal, as it offers a quantitative and reproducible method of analysis standardized across different disease areas and technologies for decision-makers, who are concerned largely with extracting the maximum value for treatments provided for any given condition. Herein, however, lies one of the major drawbacks of the traditional approach to CEA: its restrictive nature fails to capture the additional elements of values that do not fall precisely within these standard, well-defined parameters (Willke et al., 2018; Garrison et al., 2019; Neumann et al., 2022). The quality-adjusted life year (QALY), which is considered by many in the HTA field as the cornerstone of traditional CEA and one of the two drivers of CEA results (along with survival), is seen by others as inherently flawed and based largely on sometimes unfounded assumptions; as such, this compromises its ability to provide a meaningful calculation of an intervention’s value (Nord et al., 2009; Torbica et al., 2018; Caro et al., 2019; Rand and Kesselheim, 2021).

With this background in mind, this research aimed to identify if the additional elements of value previously described for consideration in HTAs have been proposed in modified economic modelling techniques or other deliberative approaches. In theory, various methodologies have been suggested to remedy the drawbacks of the current CEA approach. These methodologies range from slight alterations to QALY modifiers that consider additional elements of value without dramatically altering the current structure to completely new methodologies that attempt to maintain the objectivity of the CEA approach while incorporating expanded concepts of value (Asaria et al., 2016; Garrison et al., 2019). A commentary by Caro et al., 2019 (Caro et al., 2019) provided a critical summary of alternative approaches to QALYs that expand the measure of benefit/value of new technologies and help further deliberations on determining aspects of technology’s value. To supplement the arguments noted in this commentary, and to continue the discussion on how the new HTA era should focus on creating an equitable, efficient, and high-quality health system (O'Rourke et al., 2020a), this review aimed to identify and describe the expanded economic analyses beyond the traditional CEA approach by incorporating additional elements of a technology’s value in modelling approaches.

Materials and methods

A pragmatic literature review using reproducible criteria was conducted to capture relevant peer-reviewed articles. Reporting was guided by the Preferred Reporting Items for Systematic Review and Meta-Analyses statement (Page et al., 2021). The research question followed the Sample, Phenomenon of Interest, Design, Evaluation, Research (SPIDER) format (Library UoC, 2022): how have assessments of value (beyond clinical and cost estimates) for health technologies been incorporated in recent modelling approaches and deliberative processes? A structured database search for publicly available literature published in English from 2012 to the present was conducted in Embase and MEDLINE on 24 March 2023 (see Appendix for complete search strategy). As the HTA process is rapidly evolving across many countries and “value” may be defined differently across cultures and healthcare systems, the review was not restricted by geography. Prior to commencing screening, a calibration exercise among reviewers was conducted on a random sample of 50 articles. Screening of titles/abstracts was carried out in the DistillerSR platform (Evidence Partners Incorporated; Ottawa, Canada) by a single reviewer with a second reviewer screening 15% of excluded articles as a quality check. The same approach was used for full-text screening. Eligible studies were required to meet all the following criteria: published following a peer-review process; discussed current HTA value frameworks in the context of CEA; provided new or expanded definitions of value; and discussed new modelling approach (es) to HTA. Studies that focused on disease-specific, value-based frameworks, solely on patient experiences, or strictly on economic modelling with no reference on how additional value elements can be incorporated were excluded. No grey literature sources or commentaries/editorials were considered for inclusion. Conference abstracts were excluded given the limited information provided.

Data extraction of included studies was carried out in a pre-specified template by a single reviewer and validated by a second reviewer. Data were extracted on publication characteristics, key themes, limitations in existing CEA approaches, and new recommendations for incorporating value within CEA. Each eligible article was evaluated based on three broad criteria: did the article comment on the suitability of the current HTA methodology; did the article discuss what aspects could or should be added to the current approach; and did the article discuss new methods for evaluating therapies? Included studies were categorized based on their recommendations for HTA agencies. The three main areas of methodology were: modifications to the current CEA approach (modified CEA: mCEA), which can include variations such as multi-criteria decision analysis (MCDA); and alternate approaches, such as discrete choice experiments (DCE). Within these frameworks, several sub-methodologies exist, such as distributional CEA (DCEA), augmented CEA (ACEA) and extended CEA (ECEA) within the mCEA framework; and different variations of current MCDA methods (Table 1).

TABLE 1

TABLE 1. Summary of current traditional and expanded modelling approaches.

Results

Study eligibility

The database searches returned 3,614 records and after removing duplicates, 2,528 unique records were screened at the title/abstract level, and 132 were selected for full-text screening. Thirteen peer-reviewed studies that provided recommendations on new approaches to HTA in the context of CEA were included. Figure 1 shows the literature selection procedure.

FIGURE 1

FIGURE 1. PRISMA diagram detailing literature search results and subsequent review process.

Description of included publications

Ten of the 13 included articles were literature review articles offering expert opinion while three were reports from multistakeholder workshops or committees. Six articles specifically discussed at least one additional element to the current HTA (clinical and cost-effectiveness) value paradigm. Societal values and health equity were identified as the top two core pillars where the current CEA paradigm is wanting, with authors across these publications generally recommending an expansion of the definition of value within CEA to include these broad aspects (Figure 2).

FIGURE 2

FIGURE 2. The “Value Puzzle” illustrates the existing aspects of CEA (clinical efficacy, economic value, disease modifiers and unmet treatment needs) but also highlights the missing aspects of the current system (health equity and societal values).

Societal values were the most identified elements, mentioned in four articles (Dionne et al., 2015; Phelps and Madhavan, 2017; Pearson et al., 2019; Diaby et al., 2021). Societal aspects encompass a relatively broad spectrum of elements, but all authors agreed that the impact of disease on the patient is central to these societal considerations. For example, in the context of potentially curative treatments (Pearson et al., 2019), considerations of disease severity, its rarity, and the potential for a cure to extend life or limit the burden of illness (especially in children), as well as the value of hope and real option value offered by these treatments should be considered. The impact of productivity is also considered an important aspect to add to CEA (Dionne et al., 2015), as patients’ contributions to society are directly related to their health and wellness. Above all, the perspectives of all relevant parties, labelled as the “5Ps” (patients, providers, payers, producers, and planners) are encouraged to be considered by decision-makers (Dionne et al., 2015; Phelps and Madhavan, 2017).

Health equity was identified in four articles (Dionne et al., 2015; Cookson et al., 2017; Goetghebeur and Cellier, 2018; Diaby et al., 2021) as an important factor that is largely lacking in the existing HTA frameworks. Equity is defined by the World Health Organization as: “the absence of unfair, avoidable or remediable differences among groups of people, whether those groups are defined socially, economically, demographically, or geographically or by other dimensions of inequality (e.g., sex, gender, ethnicity, disability, or sexual orientation)” (Organization WH, 2022; Sarri, 2022). Inequity is thus evident in circumstances where a deficit in one of these areas affects access to affordable care, which is limited in one or more marginalized groups. Cookson et al., 2017 (Cookson et al., 2017) made equity the core of their argument for new approaches in HTA analyses, focusing on the trade-offs required to ensure health equity and the net equity impact of HTA decisions. They argued that the tools for health equity analysis do exist (i.e., who gains and who loses in policy decisions) and that assessing the equity trade-offs should be incorporated into existing CEA methods. Similarly, Goetghebeur et al., 2018 (Goetghebeur and Cellier, 2018) framed equity as central to an approach based on the application of compassionate care concepts, where ethical considerations are contemplated by decision-makers to maximize equity and sustainability. Diaby et al., 2021 (Diaby et al., 2021) and Dionne et al., 2015 (Dionne et al., 2015) discussed equity from the patient’s perspective, with patient demographics and a lack of patient heterogeneity in clinical studies mainly contributing to inequity in health assessment. The low representation of minority groups in clinical studies is suggested to under-represent the effect of therapies on these populations, thus contributing to decreased availability of treatments for these patients. Consideration of individual patient needs (i.e., patient preferences) and fairness in how health-economic decisions are made (i.e., balancing population and individual priorities while considering patient age, alternate treatments, and equity across different jurisdictions and populations) are additional dimensions of health equity domain (Dionne et al., 2015). In summary, researchers have long argued for societal values and equity considerations to be incorporated into existing HTA frameworks. In the context of societal values, it was argued that the impact of disease and its characteristics on patients and their productivity should also factor into decision-making. Similarly, patient preferences, addressing the needs of underrepresented groups, and ensuring access to affordable care are central to including equity considerations in HTA frameworks (Dionne et al., 2015; Diaby et al., 2021; Sarri, 2022).

Summary of mCEA or new modelling approaches

All eligible studies provided recommendations on new or modified approaches to HTA and CEA, which are mainly grouped as follows: mCEA; adoption of MCDA methods for CEA; and methods taking alternate approaches, such as DCE (Figure 3). The main theses and conclusions of the included peer-reviewed articles are summarized in Table 2.

FIGURE 3

FIGURE 3. Overview of approaches recommended in the literature.

TABLE 2

TABLE 2. Summary of eligible articles.

mCEA

Four articles (Kristensen et al., 2017; Pearson et al., 2019; Diaby et al., 2021; Zamora et al., 2021) recommended mCEA as an expanded CEA method to incorporate additional value elements, albeit their suggestions differed considerably. Kristensen et al., 2017 (Kristensen et al., 2017) summarized the results of a decade-long analysis of HTA methods by the European Network for HTA (EUnetHTA), which recommends a tiered system that accounts for typical domains such as effectiveness, safety, and health economics but also includes domains addressing social, patient, legal, and organizational elements. EUnetHTA identified nine core elements that should be considered by HTA agencies and, as part of its tiered approach, delineated between a rapid relative effectiveness assessment (REA) for interventions requiring a short turnaround and a full, comprehensive assessment for all other interventions. The REA would focus on basic clinical elements (e.g., health problem identification, intervention description, safety, and clinical effectiveness), while the full assessment would add elements such as costs, ethical analysis, organization impact, patient/societal aspects, and legal aspects. The core elements are designed to be interconnected, such that the costs/economics domain can draw information from other domains (e.g., organizational or patients/social aspects) to expand the calculation of cost and affordability. This allows for a more comprehensive and nuanced analysis that better incorporates non-traditional elements.

Diaby et al., 2021 (Diaby et al., 2021) offered two recommendations on mCEA methods: the DCEA, and a two-part appraisal that augments incremental cost-effectiveness ratio (ICER) with a comprehensive benefits and value (CBV) score. DCEA refers to a blanket approach to technology assessment that models health distributions as a comparison of health gains vs. disease burden. This approach allows for analysis of health interventions at the population (i.e., societal) and demographic subgroup (e.g., sex, race) levels, enabling an analysis of health gains in the context of sociodemographic variables, which inevitably incorporates elements of equity as defined by these variables. Health gains and losses can thus be analyzed based on individual sociodemographic variables and/or by disease category. Under this proposed approach, decision-makers are asked to make trade-offs between decisions that would improve the overall health of the population and those that would reduce inequity in healthcare availability among specific patient subgroups. The two-part approach combines ICER threshold with a CBV score, allowing for a more robust analysis that considers quantitative and qualitative assessment factors. The CBV score is a composite, qualitative rating calculated using elements such as innovativeness, disease severity, and unmet need (Goldman et al., 2010; Diaby et al., 2021) which provides a more holistic assessment of the non-economic aspects of a given intervention.

Zamora et al., 2021 (Zamora et al., 2021) examined the potential of ACEA to incorporate additional individual value elements such as insurance value, option value, and the value of hope to the traditional ICER approach. Any health gains from new elements are measured in equivalent or risk-adjusted QALYs. A hierarchical approach is then used to calculate the final aggregate impact of an intervention, beginning with the incremental QALY and then incorporating QALY equivalents for new elements. Ultimately, final decisions on technologies’ value are made through consideration of the trade-offs among the elements, as gains in one area may be associated with losses in another. The ability to quantify benefits/losses of an intervention in a common unit of measure (QALY) creates a single denominator in the calculation, which maintains an objective framework while incorporating elements that may traditionally be considered subjective. Additionally, the authors compared the ACEA and MCDA approaches and found them to be fairly similar, such that the choice between the two was largely pragmatic and thus their research question was left unresolved.

Finally, Pearson et al., 2019 (Pearson et al., 2019) discussed a more intervention-specific method that does not explicitly incorporate non-economic factors. They suggested several modifications, such as a sliding scale for the ICER, specifically for curative treatments, where interventions can be compared against a lower CEA threshold based on the projected short-term budget impact of the intervention. They further recommended adaptations including disallowing full credit for cost offsets for any interventions no longer required after a condition is cured, if that intervention itself was not cost effective; capping costs based on patient willingness to pay; and using shared savings, such that cost savings realized by curative treatments are shared between the innovator and the healthcare system. Fundamentally, their approach seeks to modify the calculations made during HTA but maintain the ability to objectively calculate costs and cost-savings, an approach not unlike that of MCDA in its desire to maintain a level of objectivity in decision-making.

MCDA

Nine articles (Dionne et al., 2015; Angelis and Kanavos, 2017; Baltussen et al., 2017; Phelps and Madhavan, 2017; Goetghebeur and Cellier, 2018; Krahn et al., 2018; DiStefano and Levin, 2019; Diaby et al., 2021; Zamora et al., 2021) recommended the adoption of some version of MCDA as a method for future CEA. MCDA methods allow for consideration of multiple perspectives and sources of input during decision-making with the aim to maintain an objective framework where scoring and weighting can be used to guide the process (Koksalan et al., 2011). Krahn et al., 2022 (Krahn et al., 2018), summarized the findings of the Ontario Health Technology Advisory Committee which recommend a framework that includes four key attributes: consideration of overall clinical benefit, consistency with societal/ethical values, value for money, and feasibility. It is within this framework that they advocate for the use of MCDA methods, citing the Evidence and Values Impact on DEcision Making framework (EVIDEM) (Goetghebeur et al., 2008) as an approach that is being increasingly explored and should be considered in future HTAs. Goetghebeur (Goetghebeur and Cellier, 2018) extended this work to suggest the use of a reflective MCDA approach, where decision-makers can reflect on and weight the motivations of their decisions, bearing in mind that decisions should be made in a patient-centric manner, with an eye toward the delivery of compassionate care. Angelis et al., 2017 (Angelis and Kanavos, 2017) outlined a new value framework using MCDA methods as a foundation, which incorporates several key aspects such as burden of disease, therapeutic considerations, safety, innovation, and socioeconomic considerations. Their resulting decision-tree approach considers each of these elements, with subsequent downstream decisions made based on each one; the final decision is based ultimately on the cumulative impact of each element and decision. Baltussen et al., 2017 (Baltussen et al., 2017) combined MCDA methods with accountability for reasonableness concepts as part of a new approach that they refer to as an “evidence-informed deliberative process.” They categorized the traditional elements assessed by HTA agencies (e.g., safety, effectiveness, budget impact) as “general criteria” and advocated for the additional consideration of “contextual criteria,” which encompass more patient-centric or societal considerations. They recommended consultation with the public on what contextual elements may be important during an HTA. As such, a combination of quantifiable criteria and non-quantifiable (i.e., qualitative) criteria should be considered, with both ultimately being used as inputs into the deliberative process. Phelps, et al., 2017 (Phelps and Madhavan, 2017), Dionne et al., 2015 (Dionne et al., 2015), and Zamora et al., 2021 (Zamora et al., 2021) also advocated for the use of MCDA methods as a way to maintain objectivity in decision-making, while still taking into account societal elements such as patient preference/autonomy and health equity. Diaby et al., 2021 (Diaby et al., 2021) similarly suggested using MCDA methods as part of an “equitable MCDA” approach, one that is transparent and results in clear and consistent decisions. They suggested the importance of both the consideration of multiple criteria as well as the impact of a given treatment on health equality. Finally, DiStefano et al., 2019 (DiStefano and Levin, 2019) stressed the importance of qualitative MCDA (Baltussen et al., 2019) which, by forgoing aggregation of scores, allows decisions to include deliberation among decision-makers regarding the relative value of non-economic elements, thus maintaining transparency and allowing for more subjective incorporation of elements such as equity, rather than integration of those elements into more traditional or mCEA. This aims to maintain the objective nature of MCDA while allowing for subjective consideration in the decision-making process.

DCE

One included publication specifically looked at the use of the DCE model for future HTA decisions (Ngorsuraches, 2021). The authors suggested that DCE allows for a qualitative analysis as part of the decision-making process and, in the context of HTA, incorporates patient preferences while all stakeholders involved (patients, providers, policymakers) are asked to choose between two hypothetical alternatives from a number of choice sets. As such, the prevalence and importance of equity to each stakeholder is determined in the initial stage, after which those equity elements can be incorporated into a choice model which, when applied, can be used to establish the value of equity. Two variations on the DCE approach include one which utilizes a latent class model to address concerns about health equity in value assessment, and another which utilizes a quantum choice model. The author did not express a preference for one method over another but noted that the use of either methods would address the inadequacies of current methodologies and help address health disparities and underrepresented patient populations.

Discussion

In most countries, HTA remains anchored by CEAs, the cornerstone analysis when considering reimbursement of new therapies. The objective nature of the traditional CEA is seen as a benefit that lends itself to impartial decision-making although this method did not entirely prevent discrepancies in decision-making; however, there is increasing sentiment that the objective approach in fact marginalizes the subjective aspects of the healthcare assessment. The definition of “value” is a prime example of the drawbacks of the current system, as there is a growing opinion that value in HTA should be viewed through more than simply an economic lens. Health gains are not straightforwardly assessed, and several approaches have been proposed to define additional elements of value beyond the clinical and cost gains. Lately, there is an increased discontent with the inability of ICERs and QALYs to sufficiently capture the benefits valued by patients and societies overall when a new health technology is introduced (Caro et al., 2019). Although mainly the discussion so far has focused on defining these additional value elements, little effort has been put on demonstrating how these additional considerations can be implemented in modelling approaches to be used in the HTA context. To address this gap, the current study examined recent modelling approaches that included expanded or new definitions of value; two main analytical approaches were identified and advocated by most authors: a modification of the current CEA and the use of advanced decision-making techniques such as MCDA, both of which have merit. To date, however, no preferred method has been established for HTA adoption although several concerns have been raised regarding the implementation of MCDA as part of HTA decision-making (Marsh et al., 2018).

Despite these efforts, consensus on the most efficient and appropriate way to incorporate expanded definitions of value (and added benefit value frameworks) into current HTA in general and CEA methods in particular has remained elusive. One consensus finding from this review was that the current approach to CEA is lacking and that there are several elements–especially relating to the current narrow definition of value in CEA–that should be added to CEA methods going forward. These aspects represent missing pieces of the “Value Puzzle” (Figure 2) and illustrate the challenges assessors face in integrating these new factors into their decision-making. These factors have been identified by several groups, including the ISPOR Task Force, which summarized these concepts in the “Value Flower” (Lakdawalla et al., 2018; Willke et al., 2018; Garrison et al., 2019; Neumann et al., 2022). Generally, these missing aspects center around an expanded definition of value, one which includes more qualitative elements such as the ability of a treatment to provide hope to the patient (the value of hope), the value associated with extending life and opening possibilities for future treatments (real option value), the value of scientific discoveries and their wide applicability (scientific spillover), and more. These new elements reflect the prevailing opinion that the current approach, with its focus on cost-effectiveness and the use of metrics such as the ICER, is inadequate. However, these additional value elements may not transport at the same degree across disease areas and populations (Shafrin et al., 2021). Indeed, many are of the opinion that the central metric in these calculations–the QALY–is an inherently flawed metric built upon assumptions (Hall, 2020) which marginalizes the sickest in a population by presenting only an aggregate calculation of health (Caro et al., 2019). Indeed, many jurisdictions have begun to move away from the QALY, which has been outright rejected in Germany and Spain (Institute for Quality and Efficiency in Healthcare IQWiG, 2022) and remains largely unused in the United States (Neumann and Weinstein, 2010), France (Rumeau-Pichon and Harousseau, 2014), and some Latin American countries (Brixner et al., 2017). It is thus important to recognize the limitations of the QALY as a final, lone decision metric and that its use represents a first albeit limited step in the process of assessing value in pharmaceutical innovation. Clearly, the lack of enthusiasm for traditional CEA methods among HTA bodies and the feedback from patients necessitates a new approach to decision-making.

The current study identified two main themes recommended to address the shortcomings of the current system: adoption of either MCDA methods or modification of the current CEA approach. The former was advocated for in the majority of articles included in the review and has been widely discussed in the CEA space as a viable alternative for some time; however, it has failed to gain traction, at least in part due to its overly mechanistic nature (Kennedy, 2009; Baltussen et al., 2019) and tendency to ignore opportunity costs (Campillo-Artero et al., 2018; Marsh et al., 2018; Baltussen et al., 2019). Quite the reverse, the use of mCEA methods has been suggested as a viable avenue for change that simultaneously addresses concerns raised by Caro et al., 2019 (Caro et al., 2019), who suggested the current CEA methods continue to be utilized by HTA bodies mainly out of convenience, and due to the lack of a viable, proven alternative. Thus, one of the draws to modifying current methods is the fact that it does not stray far from the status quo. With infrastructure in place and decades of published decisions, a major change in methodology may not be palatable for key stakeholders. Some of the recommendations in the current review slightly revised the current approach but did not recommend major changes (Kristensen et al., 2017; Pearson et al., 2019; Diaby et al., 2021). As such, these recommendations perhaps do not do enough to address current concerns. Garrison et al., 2017 (Garrison et al., 2017) proposed the use of ECEA methods to incorporate the value of “knowing” into CEA, which broadly incorporates several elements identified as valuable in our review, including reducing uncertainty and incorporating insurance value, real option value and scientific spillover into CEA. Another approach that holds promise is the ACEA approach (Zamora et al., 2021) which, like ECEA, combines the known methodology while still incorporating robust definitions of value including the value of hope, real option value, and insurance value. Indeed, the summary of the ISPOR Special Task Force report (Garrison et al., 2020) advocated for the use of ACEA methods as a way to combine the known (and widely accepted) clinico-economic aspects of traditional CEA with a comprehensive list of qualitative elements reflecting the various definitions of value. That recommended method would allow for the consideration of additional value elements (insurance, disease severity, hope, and real option value) while also allowing concepts such as equity and the benefit of scientific spillover from new technologies to be incorporated into deliberations. These additional value elements can be part of the technologies scoping exercises and tailored to the patients’ preferences. While more research is needed to refine the methods, this approach shows promise and may best address the documented shortcomings of the current approach.

Beyond methodology, HTA agencies face many other challenges in their efforts to fairly evaluate new therapies. As environmental awareness and concern grows worldwide, HTA agencies will be required to include an evaluation of the impact of a health technology’s production, use, and disposal. Toolan et al. (Toolan et al., 2023) have recently summarized the challenges associated with this effort and identified several approaches that HTA agencies may adopt during their assessment, including republishing of data in the public domain, considering environmental data in parallel with health economic data, integrating environmental data into existing methodologies, or specific evaluation of technologies that may have minimal health benefits but claim environmental benefits with their use. From a more patient-centric perspective, patients’ perspectives and preferences have been suggested as important factors that warrant more attention in the HTA process. Several authors have referred to “The 5Ps” as important contextual considerations in HTA, namely, that the perspectives of many stakeholders–patients, practitioners, payers, producers and policymakers–must be part of any CEA (Phelps and Madhavan, 2017; Slejko et al., 2019; Hall, 2020; van Overbeeke et al., 2021). Incorporating patient preference and experience, and their perception of the quality of life amidst their illness, offsets the objective nature of the traditional CEA methods and theoretically allows for a more comprehensive assessment (Sarri et al., 2021). For example, factors important to the patient regarding the impact their diagnosis will have on those around them (Vrinzen et al., 2022) or life satisfaction should be considered in any assessment (Hall, 2020). Furthermore, a patient’s preference can be reflected in their willingness to pay for or undergo treatment based on whether that treatment can offer them hope for recovery (Peasgood et al., 2022). Several authors have noted that patients are more willing to pay for a “hopeful therapy”, with patients with cancer identified as those who prefer a therapy that has the possibility of a large therapeutic gain, even when the average response to that therapy may be similar to other options (Lakdawalla et al., 2012). As Hall comments: a patient who adapts to illness may live longer but may be less able to fight off future illness. Do the patient’s values change as they adapt to their disease? And how does the QALY account for this adaptation (Hall, 2020)? Administratively, the financial burden placed on healthcare systems will only continue to increase. Healthcare systems stretched thin by the recent COVID-19 pandemic face ongoing challenges in integrating costs for new therapies into an already strained system (Epstein et al., 2020; Information CIfH, 2022; Youn et al., 2022). However, recent trends such as the growing use of real-world evidence (RWE) in healthcare research in general and with it a concomitant uptake in the use of RWE in regulatory and HTA agency filings may provide the opportunity to unravel existing health inequalities that directly fit in the decision-making (Sarri, 2022). However, the potential of RWE to capture the direction and magnitude of impact a new health technology may have on health inequalities has not been fully explored (Goetghebeur and Cellier, 2018). Proposed checklists to guide HTA decision-makers include equity considerations in their assessment may help on this front (Benkhalti et al., 2021). The struggle for HTA staff to keep pace with evolving RWE methodological complexities adds to the challenges facing these agencies. This is especially true in cases of rare disease or where ethical concerns prevent the designing of placebo-controlled, two-armed studies (Thorlund et al., 2020; Ramagopalan et al., 2021; Popat et al., 2022). All told, the challenges facing HTA bodies are layered and complex. More case studies are needed to demonstrate how reliably these holistic value aspects can be integrated into HTA, although buy-in among assessors and researchers is also required, to facilitate the widespread use of new and expanded methodologies and the learnings from demonstration of case studies.

This study should be considered with the following limitations. The pragmatic nature of the search, while comprehensive, could have missed some relevant articles, although the broad nature of the search may mitigate this concern. Related is the decision to include only peer-reviewed articles in data/theme collection. Commentaries and/or editorials were excluded from this review, which may result in some valid recommendations regarding these modelling techniques being missed. However, any commentaries that were captured in the search were reviewed for relevant opinions and referenced in the discussion as appropriate. Finally, articles that focused on a specific disease were excluded, as the aim was to provide a broad overview of these modelling techniques. This may also result in missing some articles that may have offered valuable perspectives on this topic; however, the wider focus of the review may make the findings more broadly applicable and initiate some methodological discussion.

Conclusion

This research demonstrated that modelling methods are being expanded from the traditional CEA approach to incorporate value elements with a more holistic view of what matters most to patients and society. Although the methods differ, a consensus exists on the need for HTA agencies to redefine “value” with a wider lens that looks at more than just the clinical and economic benefits of a new technology. Societal factors and health equity scored highly as additional value elements. Future efforts are needed to increase the confidence of stakeholders in the importance of “testing” these expanded CEAs approaches in case studies.

Author contributions

JM drafted the manuscript and all authors reviewed, contributed to revisions, and approved the final version of the manuscript. All authors contributed to the article and approved the submitted version.

Funding

Financial support for this study was provided entirely by Takeda Pharmaceuticals, Inc. The funding agreement ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report.

Acknowledgments

The authors wish to acknowledge the editorial support by Colleen Dumont.

Conflict of interest

Authors JM, GS, AR, and AF were employed by Cytel, Inc. Author IO was employed by Takeda Pharmaceuticals, Inc.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Angelis, A., and Kanavos, P. (2017). Multiple criteria decision analysis (MCDA) for evaluating new medicines in health technology assessment and beyond: The advance value framework. Soc. Sci. Med. 188, 137–156. doi:10.1016/j.socscimed.2017.06.024

PubMed Abstract | CrossRef Full Text | Google Scholar

Asaria, M., Griffin, S., and Cookson, R. (2016). Distributional cost-effectiveness analysis: A tutorial. Med. Decis. Mak. 36 (1), 8–19. doi:10.1177/0272989X15583266

CrossRef Full Text | Google Scholar

Baltussen, R., Jansen, M. P. M., Bijlmakers, L., Grutters, J., Kluytmans, A., Reuzel, R. P., et al. (2017). Value assessment frameworks for HTA agencies: The organization of evidence-informed deliberative processes. Value Health 20 (2), 256–260. doi:10.1016/j.jval.2016.11.019

PubMed Abstract | CrossRef Full Text | Google Scholar

Baltussen, R., Marsh, K., Thokala, P., Diaby, V., Castro, H., Cleemput, I., et al. (2019). Multicriteria decision analysis to support health technology assessment agencies: Benefits, limitations, and the way forward. Value Health 22 (11), 1283–1288. doi:10.1016/j.jval.2019.06.014

PubMed Abstract | CrossRef Full Text | Google Scholar

Benkhalti, M., Espinoza, M., Cookson, R., Welch, V., Tugwell, P., and Dagenais, P. (2021). Development of a checklist to guide equity considerations in health technology assessment. Int. J. Technol. Assess. Health Care 37, e17. doi:10.1017/S0266462320002275

PubMed Abstract | CrossRef Full Text | Google Scholar

Breslau, R. M., Cohen, J. T., Diaz, J., Malcolm, B., and Neumann, P. J. (2023). A review of HTA guidelines on societal and novel value elements. Int. J. Technol. Assess. Health Care 39 (1), e31. doi:10.1017/S026646232300017X

PubMed Abstract | CrossRef Full Text | Google Scholar

Brixner, D., Maniadakis, N., Kaló, Z., Hu, S., Shen, J., and Wijaya, K. (2017). Applying multi-criteria decision analysis (MCDA) simple scoring as an evidence-based HTA methodology for evaluating off-patent pharmaceuticals (OPPs) in emerging markets. Value Health Reg. Issues 13, 1–6. doi:10.1016/j.vhri.2017.02.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Campillo-Artero, C., Puig-Junoy, J., and Culyer, A. J. (2018). Does MCDA trump CEA? Appl. Health Econ. Health Policy 16 (2), 147–151. doi:10.1007/s40258-018-0373-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Canadian Institute for Health Information (CIHI) (2022). COVID-19’s Impact on Health Care Systems (Accessed November 18, 2022).

Google Scholar

Caro, J. J., Brazier, J. E., Karnon, J., Kolominsky-Rabas, P., McGuire, A. J., Nord, E., et al. (2019). Determining value in health technology assessment: Stay the course or tack away? Pharmacoeconomics 37 (3), 293–299. doi:10.1007/s40273-018-0742-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Cookson, R., Mirelman, A. J., Griffin, S., Asaria, M., Dawkins, B., Norheim, O. F., et al. (2017). Using cost-effectiveness analysis to address health equity concerns. Value Health 20 (2), 206–212. doi:10.1016/j.jval.2016.11.027

PubMed Abstract | CrossRef Full Text | Google Scholar

Diaby, V., Ali, A., Babcock, A., Fuhr, J., and Braithwaite, D. (2021). Incorporating health equity into value assessment: Frameworks, promising alternatives, and future directions. J. Manag. Care Spec. Pharm. 27 (9-a Suppl. l), S22–S29. doi:10.18553/jmcp.2021.27.9-a.s22

PubMed Abstract | CrossRef Full Text | Google Scholar

Dionne, F., Mitton, C., Dempster, B., and Lynd, L. D. (2015). Developing a multi-criteria approach for drug reimbursement decision making: An initial step forward. J. Popul. Ther. Clin. Pharmacol. 22 (1), e68–e77.

PubMed Abstract | Google Scholar

DiStefano, M. J., and Levin, J. S. (2019). Does incorporating cost-effectiveness analysis into prescribing decisions promote drug access equity? AMA J. Ethics 21 (8), E679–E685. doi:10.1001/amajethics.2019.679

PubMed Abstract | CrossRef Full Text | Google Scholar

Epstein, R. H., Dexter, F., Smaka, T. J., and Candiotti, K. A. (2020). Policy implications for the COVID-19 pandemic in light of most patients (≥72%) spending at most one night at the hospital after elective, major therapeutic procedures. Cureus 12 (8), e9746. doi:10.7759/cureus.9746

PubMed Abstract | CrossRef Full Text | Google Scholar

Fornaro, G., Federici, C., Rognoni, C., and Ciani, O. (2021). Broadening the concept of value: A scoping review on the option value of medical technologies. Value Health 24 (7), 1045–1058. doi:10.1016/j.jval.2020.12.018

PubMed Abstract | CrossRef Full Text | Google Scholar

Garrison, L. P., Kamal-Bahl, S., and Towse, A. (2017). Toward a broader concept of value: Identifying and defining elements for an expanded cost-effectiveness analysis. Value Health 20 (2), 213–216. doi:10.1016/j.jval.2016.12.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Garrison, L. P., Neumann, P. J., and Willke, R. J. (2019). Reflections on the ISPOR special Task Force on U.S. Value frameworks: Implications of a health economics approach for managed care pharmacy. J. Manag. Care Spec. Pharm. 25 (11), 1185–1192. doi:10.18553/jmcp.2019.25.11.1185

PubMed Abstract | CrossRef Full Text | Google Scholar

Garrison, L. P., Zamora, B., Li, M., and Towse, A. (2020). Augmenting cost-effectiveness analysis for uncertainty: The implications for value assessment-rationale and empirical support. J. Manag. Care Spec. Pharm. 26 (4), 400–406. doi:10.18553/jmcp.2020.26.4.400

PubMed Abstract | CrossRef Full Text | Google Scholar

Goetghebeur, M. M., and Cellier, M. S. (2018). Can reflective multicriteria be the new paradigm for healthcare decision-making? The EVIDEM journey. Cost Eff. Resour. Allocation 16 (Suppl. 1), 54. doi:10.1186/s12962-018-0116-9

CrossRef Full Text | Google Scholar

Goetghebeur, M. M., Wagner, M., Khoury, H., Levitt, R. J., Erickson, L. J., and Rindress, D. (2008). Evidence and Value: Impact on DEcisionMaking-the EVIDEM framework and potential applications. BMC Health Serv. Res. 8, 270. doi:10.1186/1472-6963-8-270

PubMed Abstract | CrossRef Full Text | Google Scholar

Goldman, D., Lakdawalla, D., Philipson, T. J., and Yin, W. (2010). Valuing health technologies at NICE: Recommendations for improved incorporation of treatment value in HTA. Health Econ. 19 (10), 1109–1116. doi:10.1002/hec.1654

PubMed Abstract | CrossRef Full Text | Google Scholar

Guidelines for the economic (2017). Guidelines for the economic evaluation of health technologies.

Google Scholar

Hall, A. (2020). Quality of life and value assessment in health care. Health Care Anal. 28 (1), 45–61. doi:10.1007/s10728-019-00382-w

PubMed Abstract | CrossRef Full Text | Google Scholar

Hendriks, S., and Pearson, S. D. (2021). Assessing potential cures: Are there distinctive elements of value beyond health gain? J. Comp. Eff. Res. 10 (4), 255–265. doi:10.2217/cer-2020-0190

PubMed Abstract | CrossRef Full Text | Google Scholar

Information CIfH (2022). Overview: COVID-19’s impact on health care systems.

Google Scholar

Institute for Quality and Efficiency in Healthcare (IQWiG) (2022). Methods for assessment of the relation of benefits to costs in the German statutory healthcare system. Available at: http://www.iqwig.de/download/08-10-14_Methods_of_the_Relation_of_Benef​its_to_Costs_v_1_1.pdf.

Google Scholar

Kennedy, I. (2009). Appraising the value of innovation and other benefits, 22. New York: A short study for NICE.

Google Scholar

Koksalan, M., Wallenius, J., and Zionts, S. (2011). Multiple criteria decision making: From early history to the 21st century. Singapore: World Scientific.

Google Scholar

Krahn, M., Miller, F., Bayoumi, A., Brooker, A. S., Wagner, F., Winsor, S., et al. (2018). Development of the ontario decision framework: A values based framework for health technology assessment. Int. J. Technol. Assess. Health Care 34 (3), 290–299. doi:10.1017/S0266462318000235

PubMed Abstract | CrossRef Full Text | Google Scholar

Kristensen, F. B., Lampe, K., Wild, C., Cerbo, M., Goettsch, W., and Becla, L. (2017). The HTA core Model(®)-10 Years of developing an international framework to share multidimensional value assessment. Value Health 20 (2), 244–250. doi:10.1016/j.jval.2016.12.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Lakdawalla, D. N., Doshi, J. A., Garrison, L. P., Phelps, C. E., Basu, A., and Danzon, P. M. (2018). Defining elements of value in health care-A health economics approach: An ISPOR special Task Force report [3]. Value Health 21 (2), 131–139. doi:10.1016/j.jval.2017.12.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Lakdawalla, D. N., Romley, J. A., Sanchez, Y., Maclean, J. R., Penrod, J. R., and Philipson, T. (2012). How cancer patients value hope and the implications for cost-effectiveness assessments of high-cost cancer therapies. Health Aff. (Millwood) 31 (4), 676–682. doi:10.1377/hlthaff.2011.1300

PubMed Abstract | CrossRef Full Text | Google Scholar

Library UoC (2022). Literature reviews for the health sciences. Available at: https://canberra.libguides.com/c.php?g=940615&p=6808850.

Google Scholar

Marsh, K. D., Sculpher, M., Caro, J. J., and Tervonen, T. (2018). The use of MCDA in HTA: Great potential, but more effort needed. Value Health 21 (4), 394–397. doi:10.1016/j.jval.2017.10.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Neumann, P. J., Garrison, L. P., and Willke, R. J. (2022). The history and future of the "ISPOR value flower": Addressing limitations of conventional cost-effectiveness analysis. Value Health 25 (4), 558–565. doi:10.1016/j.jval.2022.01.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Neumann, P. J., and Weinstein, M. C. (2010). Legislating against use of cost-effectiveness information. N. Engl. J. Med. 363 (16), 1495–1497. doi:10.1056/NEJMp1007168

PubMed Abstract | CrossRef Full Text | Google Scholar

Ngorsuraches, S. (2021). Using latent class and quantum models to value equity in health care: A tale of 2 stories. J. Manag. Care Spec. Pharm. 27 (9-a Suppl. l), S12–S16. doi:10.18553/jmcp.2021.27.9-a.s12

PubMed Abstract | CrossRef Full Text | Google Scholar

Nord, E., Daniels, N., and Kamlet, M. (2009). QALYs: Some challenges. Value Health 12 (Suppl. 1), S10–S15. doi:10.1111/j.1524-4733.2009.00516.x

PubMed Abstract | CrossRef Full Text | Google Scholar

O'Rourke, B., Oortwijn, W., and Schuller, T. (2020a). Announcing the new definition of health technology assessment. Value Health 23 (6), 824–825. doi:10.1016/j.jval.2020.05.001

PubMed Abstract | CrossRef Full Text | Google Scholar

O'Rourke, B., Oortwijn, W., and Schuller, T. International Joint Task Group (2020b). The new definition of health technology assessment: A milestone in international collaboration. Int. J. Technol. Assess. Health Care 36 (3), 187–190. doi:10.1017/S0266462320000215

PubMed Abstract | CrossRef Full Text | Google Scholar

Organization WH (2022). Health equity. Available at: https://www.who.int/health-topics/health-equity#tab=tab_1.

Google Scholar

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., et al. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Bmj 372, 790–799. doi:10.1016/j.rec.2021.07.010

CrossRef Full Text | Google Scholar

Pearson, S. D., Ollendorf, D. A., and Chapman, R. H. (2019). New cost-effectiveness methods to determine value-based prices for potential cures: What are the options? Value Health 22 (6), 656–660. doi:10.1016/j.jval.2019.01.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Peasgood, T., Mukuria, C., Rowen, D., Tsuchiya, A., and Wailoo, A. (2022). Should we consider including a value for "hope" as an additional benefit within health technology assessment? Value Health 25 (9), 1619–1623. doi:10.1016/j.jval.2022.03.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Phelps, C. E., and Madhavan, G. (2017). Using multicriteria approaches to assess the value of health care. Value Health 20 (2), 251–255. doi:10.1016/j.jval.2016.11.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Popat, S., Liu, S. V., Scheuer, N., Hsu, G. G., Lockhart, A., Ramagopalan, S. V., et al. (2022). Addressing challenges with real-world synthetic control arms to demonstrate the comparative effectiveness of Pralsetinib in non-small cell lung cancer. Nat. Commun. 13 (1), 3500. doi:10.1038/s41467-022-30908-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Ramagopalan, S., Gupta, A., Arora, P., Thorlund, K., Ray, J., and Subbiah, V. (2021). Comparative effectiveness of atezolizumab, nivolumab, and docetaxel in patients with previously treated non-small cell lung cancer. JAMA Netw. Open 4 (11), e2134299. doi:10.1001/jamanetworkopen.2021.34299

PubMed Abstract | CrossRef Full Text | Google Scholar

Rand, L. Z., and Kesselheim, A. S. (2021). Controversy over using quality-adjusted life-years in cost-effectiveness analyses: A systematic literature review. Health Aff. (Millwood). 40 (9), 1402–1410. doi:10.1377/hlthaff.2021.00343

PubMed Abstract | CrossRef Full Text | Google Scholar

Reed, S. D., Dubois, R. W., Johnson, F. R., Caro, J. J., and Phelps, C. E. (2019). Novel approaches to value assessment beyond the cost-effectiveness framework. Value Health 22 (6s), S18–s23. doi:10.1016/j.jval.2019.04.1914

PubMed Abstract | CrossRef Full Text | Google Scholar

Rumeau-Pichon, C., and Harousseau, J. L. (2014). Analysis of cost-effectiveness assessments in France by the French national authority for health (has). Value Health 17 (7), A414. doi:10.1016/j.jval.2014.08.997

CrossRef Full Text | Google Scholar

Sanders, G. D., Neumann, P. J., Basu, A., Brock, D. W., Feeny, D., Krahn, M., et al. (2016). Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: Second Panel on cost-effectiveness in health and medicine. Jama 316 (10), 1093–1103. doi:10.1001/jama.2016.12195

PubMed Abstract | CrossRef Full Text | Google Scholar

Sarri, G. (2022). Can real-world evidence help restore decades of health inequalities by informing health care decision-making? Certainly, and here is how. Front. Pharmacol. 13, 905820. doi:10.3389/fphar.2022.905820

PubMed Abstract | CrossRef Full Text | Google Scholar

Sarri, G., Freitag, A., Szegvari, B., Mountian, I., Brixner, D., Bertelsen, N., et al. (2021). The role of patient experience in the value assessment of complex technologies - do HTA bodies need to reconsider how value is assessed? Health Policy 125 (5), 593–601. doi:10.1016/j.healthpol.2021.03.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Shafrin, J., Dennen, S., Pednekar, P., Birch, K., Bhor, M., Kanter, J., et al. (2021). For which diseases do broader value elements matter most? An evaluation across 20 ICER evidence reports. J. Manag. Care Spec. Pharm. 27 (5), 650–659. doi:10.18553/jmcp.2021.20471

PubMed Abstract | CrossRef Full Text | Google Scholar

Slejko, J. F., Mattingly, T. J., Mullins, C. D., Perfetto, E. M., and dosReis, S. (2019). Future of patients in healthcare evaluation: The patient-informed reference case. Value Health 22 (5), 545–548. doi:10.1016/j.jval.2019.02.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Thorlund, K., Dron, L., Park, J. J. H., and Mills, E. J. (2020). Synthetic and external controls in clinical trials - a primer for researchers. Clin. Epidemiol. 12, 457–467. doi:10.2147/CLEP.S242097

PubMed Abstract | CrossRef Full Text | Google Scholar

Toolan, M., Walpole, S., Shah, K., Kenny, J., Jónsson, P., Crabb, N., et al. (2023). Environmental impact assessment in health technology assessment: Principles, approaches, and challenges. Int. J. Technol. Assess. Health Care 39 (1), e13. doi:10.1017/S0266462323000041

PubMed Abstract | CrossRef Full Text | Google Scholar

Torbica, A., Tarricone, R., and Drummond, M. (2018). Does the approach to economic evaluation in health care depend on culture, values, and institutional context? Eur. J. Health Econ. 19 (6), 769–774. doi:10.1007/s10198-017-0943-1

PubMed Abstract | CrossRef Full Text | Google Scholar

van Overbeeke, E., Forrester, V., Simoens, S., and Huys, I. (2021). Use of patient preferences in health technology assessment: Perspectives of Canadian, Belgian and German HTA representatives. Patient 14 (1), 119–128. doi:10.1007/s40271-020-00449-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Vrinzen, C. E. J., Bloemendal, H. J., Stuart, E., Makady, A., van Agthoven, M., Koster, M., et al. (2022). Cancer treatments touch a wide range of values that count for patients and other stakeholders: What are the implications for decision-making? Cancer Med. 12, 6105–6116. doi:10.1002/cam4.5336

PubMed Abstract | CrossRef Full Text | Google Scholar

Willke, R. J., Neumann, P. J., Garrison, L. P., and Ramsey, S. D. (2018). Review of recent US value frameworks-A health economics approach: An ISPOR special Task Force report [6]. Value Health 21 (2), 155–160. doi:10.1016/j.jval.2017.12.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Youn, H. M., Quan, J., Mak, I. L., Yu, E. Y. T., Lau, C. S., Ip, M. S. M., et al. (2022). Long-term spill-over impact of COVID-19 on health and healthcare of people with non-communicable diseases: A study protocol for a population-based cohort and health economic study. BMJ Open 12 (8), e063150. doi:10.1136/bmjopen-2022-063150

PubMed Abstract | CrossRef Full Text | Google Scholar

Zamora, B., Garrison, L. P., Unuigbe, A., and Towse, A. (2021). Reconciling ACEA and MCDA: Is there a way forward for measuring cost-effectiveness in the U.S. Healthcare setting? Cost. Eff. Resour. Alloc. 19 (1), 13. doi:10.1186/s12962-021-00266-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhang, M., Bao, Y., Lang, Y., Fu, S., Kimber, M., Levine, M., et al. (2022). What is value in health and healthcare? A systematic literature review of value assessment frameworks. Value Health 25 (2), 302–317. doi:10.1016/j.jval.2021.07.005

PubMed Abstract | CrossRef Full Text | Google Scholar