Efficacy, Safety, and Economics of Innovative Medicines: The Role of Multi-Criteria Decision Analysis and Managed Entry Agreements in Practice and Policy

Through the years, solutions for accelerated access to innovative treatments are implemented in models of regulatory approvals, yet with limited data. Besides efficacy data, providing adequate safety data is key to transferring conditional marketing authorization to final marketing authorization. However, this remains a challenge because of the restricted availability and transferability of such data. Within this study, we set up a challenge to analyze the answers of two questions. First, from regulatory bodies' point of view, we bring the question of whether multi-criteria decision analysis (MCDA) is an adequate tool for further improvement of health technology assessment (HTA) of innovative medicines. Second, we ask if managed entry agreements (MEAs) pose solutions for facilitating the access to innovative medicines and further strengthening the evidence base concerning efficacy and effectiveness, as well as safety. Elaborating on such challenges brought us to conclude that increasing the attention to safety in MCDAs and MEAs will increase the trust of the authorities and improve the access for the manufacturers and the early availability of safe and effective medicines for the patients.

Introduction

Is there a perfect formula to balance pharmaceutical innovation and affordability? While the number of innovative medicines is increasing, health-care expenditure tends to go down. Currently, targeted oncological and orphan medications are the biggest challenges, testing the affordability and accessibility by different health systems. In the last decades, cost-effectiveness has become a core element health technology assessment (HTA). Ideally the cost-effectiveness should be leveled with budget impact, but obviously this is not always the case. Seen from regulatory bodies' point of view, we ask ourselves whether multi-criteria decision analysis (MCDA) is an adequate tool for further improvement of HTA of innovative medicines and if managed entry agreements (MEAs) pose solutions for facilitating the access to innovative medicines and further strengthening the evidence base concerning efficacy and effectiveness, as well as safety.

Regulatory Efforts for Early Medicine Access

Medicines' pathways consist of development, evaluation, authorization, access, reimbursement, and adequate safety monitoring. These processes involve manufacturers/pharma-industry; international and national regulatory bodies issuing marketing authorizations (MAs); and HTA bodies that are regulating access, pricing, and reimbursement of medicines, as well as health professionals and patients. All of them are representing stakeholders that can benefit from an optimized value-based health care. Yet all have their own challenges. Further in this section, we present the perspective of the regulatory bodies and their interaction with the HTA bodies, and we trigger the interest of the patients' and manufacturers' role in the regulatory processes.

In the past years, we can notice a trend of changing models of regulatory approvals with solutions for accelerated access to innovative treatments with yet limited evidence on efficiency and safety in particular. In this set of rules, the confirmatory clinical trials are not necessarily yet part of the pre-authorization within conditional MAs. Subsequently, they can turn into a full MA after confirming the efficiency and safety in real-live observations. To facilitate these processes, regulatory authorities made adequate steps such as centralized procedure (Europe), authorization under exceptional circumstances (Europe), shorter accelerated access evaluation period (Europe, the UK, and Japan) (1–4), fast-track process (the USA), complementary regulatory process (Canada) (5, 6), or priority reviews and rapid approvals (China) (4).

Once the MA is issued, cost-effectiveness and budget impact need to be shown to justify the reimbursement of the medicine. As of that point, HTA bodies take the leading role in addressing both the pricing and the reimbursement. To facilitate this process, the European Medicines Agency (EMA) and the European Network for Health Technology Assessment (EUnetHTA) have recently allowed simultaneous advising to the manufacturers so that integrated data sets can be used for the purposes of all these bodies. Patients' opinions are included in these processes, e.g., in the early dialog of medical development, scientific advice, protocol assistance, parallel advice with HTA bodies, the initial evaluation for authorization, the development of risk minimization measures, setting up additional requirements for follow-up studies on safety, and continued monitoring of its efficacy and safety (1, 7). In the subsequent safety monitoring, patients' experiences are obviously core (7). Nowadays, registries are often used for benefit–risk monitoring [as described in risk management plans (RMPs)] of innovative medicines. A recent study has shown that such monitoring is compulsory for 9% of the centrally registered medicines, 66% of the “exceptional circumstances' registrations,” and 12% of the products with a conditional MA in Europe (8). Safety is the main focus in the majority (71%) of the aforementioned registries. While the product-oriented registries are preferred by the industry, regulatory bodies find the use of disease registries more convenient (8). The type and number of registries among various countries are given in Table 1. Furthermore, the European Network of Centers for Pharmacoepidemiology and Pharmacovigilance (ENCePP) supports the maintenance and monitoring studies for the benefit–risk balance (25). It should be emphasized that patient monitoring obtained via registries is open-ended and serves for continuous observations, unlike the monitoring studies that are based on research goals and have limited observational time frame (25, 26). Each of these data sources brings valuable safety information for the observed medicine or disease. Safety has an important role throughout medicines' life cycle. Therefore, filling in the gap identified in the pre-submission phase by introducing better safety setups in the early dialogs for access to innovative medicines can be beneficial for both the industry and the authorities (8, 9, 27).

TABLE 1

Table 1. Some characteristics of the health-economic pathways among countries.

While regulatory bodies are making efforts to get the processes to work, manufacturers are more and more pushed to show excellence in reporting and fulfilling the payor-specific needs while demonstrating the real-world value of the newly introduced medicines. The challenge is to optimally integrate this with HTA processes, organization, and outcomes.

Health Technology Assessment in Practice and Policy

HTA uses a rather broad spectrum of elements to inform the decision makers. The information however is predominantly based on the economic analyses, in particular cost-effectiveness analyses (CEAs) and cost utility analyses if quality-adjusted life-year (QALY) is the measurement for consequence. CEAs for new treatments are used to leverage the (relative) effectiveness and safety over the standard/common treatments and costs, as well as to demonstrate authorities if the new medicine is investment worthy. Such measurement of allocative efficiency usually shows extra costs/expenses but also more effects than the standard treatment; nevertheless, ideally of course, it should show less expenses combined with higher effectiveness (28). This challenge becomes even bigger when discussing innovative medical solutions. At least two questions are arising: how can manufacturers maximize the value of their innovative medicines and do the criteria used in conventional HTA sufficiently capture all aspects of value (29–31)?

Utilization of conventional HTA is widely accepted; however, certain countries are recently deciding to opt in toward a more MCDA-attributed HTA. Conventional HTA traditionally has one or just a few aspects dominating. The incremental cost-effectiveness ratio (ICER), usually representing cost per QALY, is taken as a common measuring value. Yet it seems not always to be sufficient because it does not capture all aspects of value, notably with relevant aspects of safety often being missed (29, 30). In that context, MCDA may be better suited for evaluating the value of health-care interventions. Moreover, MCDA is already used to show the consistency benefit–risk balance in the benefit–risk assessments (32). In the pathway of converting conventional HTA to MCDA, several modifiers could be considered on the standing conventions: proportional shortfall, differential willingness to pay (WTP) by burden, severity, equity, rarity, end of life, fair innings, and evidence on safety (10, 11, 33).

The ICER could be integrated together with these modifiers to better reflect the value. For example, the proportional shortfall may pose a solution in regard to QALY equity rather than using the standard incremental approach, where QALYs are attributed with equal value for any disease burden (11, 34). Furthermore, the threshold that distinguishes between “passed” and “failed” in cost-effectiveness seems arbitrary and potentially not justifying its major sole role that it has in some countries. Moreover, thresholds are variable between countries, both numerically and structurally, as some countries use fixed and others differential thresholds. In the Netherlands and Norway, the cost-effectiveness threshold is proportionally dependent on the burden of disease, and differential thresholds apply. Severity is sometimes related to the differential thresholds, but not necessarily (11, 12). Incorporating equity in the ICER calculation would insinuate adding societal preferences to cost-effectiveness. Moreover, assigning equity weighting to the QALY will allow more perspicuous allocative decision making (11, 35, 36). Rarity plays an important role in defining the value of the orphan medicines (33, 37). End of life, as a criterion for value, is usually attributed to interventions accompanied with the highest threshold, like those applied in England (38). Evidence on safety within a MCDA provides parameter estimation but surely represents a challenge to reflect societal implications. Those can be addressed by new approaches such as safety by design into MCDA (39). As application of one modifier may not grasp all details of the QALYs, a multi-value approach is worth considering (40). Notably, full integration of safety aspects, inclusive mental as well as physical aspects, in the QALY remains a challenge.

Several frameworks and initiatives tried to structure the MCDA criteria. The Evidence and Value Impact on DEcisionMaking (EVIDEM) framework is often seen to be used as a MCDA analytical tool, accounting for disease severity, population, urgent needs, comparative effectiveness, comparative safety/tolerability, comparative health, comparative costs, benefit, quality of evidence, clinical practice guidelines, and value evidence (41, 42). The BEACON framework stands for burden/target population, environment, affordability/value, comparator, outcomes, and number of studies/quality of evidence (15). Disease-oriented frameworks can also be found. The European Society for Medical Oncology (ESMO) uses the ESMO-Magnitude of Clinical Benefit Scale (MCBS) tool to quantify value in cancer care (43). The same focus remains in The American Society of Clinical Oncology (ASCO) and the National Comprehensive Cancer Network (NCCN) (44, 45).

Some suggest a value-matrix frame to quantify the measurements with MCDA and serve as a quality assessment tool in combining HTA and MCDA (46, 47). Despite the effort for structuring MCDA, the methodologies applied remain heterogeneous. Some examples include discrete choice experiment (DCE), Potentially all Pairwise Ranking of all possible alternatives (PAPRIKA), analytic hierarchy process (AHP), Measuring Attractiveness by a Categorical Based Evaluation TecHnique (MACBETH), Simple Attribute Rating Technique (SMART), direct weighing, weighted benefit scores (WBSs), and Elimination and Choice Expressing Reality (ELECTRE) (10, 48, 49). While some suggest that the commonly used technique for value in MCDA quantification is weighted sum approach (50), others emphasize the AHP and the multi-attribute utility theory (MAUT) as the most common (51). New approaches combining MCDA and WTP have also been explored (51). In one approach, the WTP is closely related to safety outcomes of the new interventions (52). Yet the aforementioned frameworks do not consistently use all the value of safety while preforming MCDA.

In the case of one solution fits all, the health-care systems would remain homogenous. However, HTA bodies vary through the sets of national priorities. Every country has their own reimbursement and pricing policy (relevant characteristics of HTA per country are given in Table 1). An important challenge concerns the integration of national HTAs into multinational collaborations. Beside the existence of the EUnetHTA, which aims to unify the HTA methodologies, HTA is still used for different purposes. For example, the HTA in France, Italy, and Czech Republic is used as supplementary advice for reimbursement or price decisions of the health authorities; while in Norway, Sweden, and England, it is used as a tool for final reimbursement or rejection of funding (13). Other examples from Europe, e.g., Germany, show transparency in prices discounting and mainly focuses on clinical outcomes. Unlike Germany, in the UK, the focus is on CEA, with cost/QALY dominating, but potentially less transparent. In Sweden, pricing processes are transparent, clinical evidence is required, and CEA is considered (9). The Dutch system can be conceived as approaching MCDA. Even though not completely transparent, societal perspective is applied together with the care about the medical need, cost-effectiveness, quality of life, budget impact, efficacy, effectiveness, and safety as well as severity as a separate criterion (11). More details about the HTA focus and role in reimbursement, as well as some implementation elements and level of MCDA, are given in Table 1.

Furthermore, the budget impact analysis (BIA) can be complemented by horizon scanning used to prioritize the fund allocation. This tool is used in Italy, France, Sweden, Norway, England, the Netherlands, and Canada. Yet establishment and maintenance of such tool pose concerns regarding ethics and data exchange and is a time-consuming activity (14, 53). A recent study addressing the problem of the rare disease non-alcoholic steatohepatitis showed that MCDA is promising in supporting early HTA, illustrating high consistency in results across England, France, and Germany (54).

Managed Entry Agreements in Practice and Policy

While the World Health Organization (WHO) is actively working on providing equal access to essential medicines, innovative medicines remain in the hands of the manufacturers and patent holder, so it remains a challenge to arrange access at affordable prices and circumstances for society (55). Conceived high medicine prices are reported to remain the main barrier to be overcome in health-care systems (56). Furthermore, the health-care expenditure is related with gross domestic product (GDP) representing the wealth of one country. Consequently, developed countries are more likely to have higher health budget and therefore better chance for access to innovative medicines than the developing ones (57).

MEAs were introduced as a tool for overcoming these accessibility barriers but also to mitigate the uncertainty attributed to economic evaluations and BIAs accounting for the real-world evidence (58). Often, MEAs represent an integrated effort by the manufacturers on the one hand and the health authorities, Ministries of Health—potentially with other ministries, like the Ministry of Economics or Finance—on the other hand (27). MEAs are often differentiated into financial-oriented and outcome-based ones. Illustrative examples of both are a budget cap and no-cure/no-pay, respectively (59). While the benefits of MEA are obvious, there are remaining challenges to be addressed. The global trend of MEA reflects dominant utilization of the financial-based ones, like in England, Portugal, Lithuania, Belgium, and Cyprus. Outcome-based MEA are seen in the Netherlands, Sweden, and Czech Republic (58, 60). Further details over MEA country specifics are given in Figure 1 and Table 1. To be noted, MEA is part of a dynamic field, and countries' preferences change easily; for example, the Netherlands switched from outcome-based MEA to price negotiations (59).

FIGURE 1

Figure 1. Type of managed entry agreements across countries. MEA, managed entry agreement.

Nevertheless, a recent study brings to attention that the increased availability of innovative medicines is in line with the increased number of MEA, giving real-world evidence from around the globe (61).

In the regulatory pathway, outcome-based MEA was not commonly used for the products with conditional MAs, yet monitoring is more likely to accompany outcome-based MEAs than the financial-based ones (8, 27). Furthermore, as manufacturers provide additional data to support their conditional MA, authorities seem not always to have all the information needed. Divergence appears in the requirements of HTA and regulatory bodies when further proof of efficiency and safety needs to be provided. While HTA bodies in one adaptive pathway require observational study outcomes, the regulatory authorities stick to randomized controlled trial (RCT) evidence for the Phase III. Unlike the payors and HTA bodies, manufacturers are keen toward value-based MEA, posing a good solution for market positioning of their products.

Actionable Recommendations While Using Multi-Criteria Decision Analysis and Managed Entry Agreements

Among the many advantages complementing the HTA with MCDA, potential limitations may appear (15, 50, 62). To overcome such posed challenges, concrete actions are required. Therefore, we recommend:

• setting up criteria in which medicines are eligible for MCDA,

• specifying the selection and weighting criteria,

• choosing the “right model” for the investigated intervention,

• setting up guidelines and methods for unified outcome interpretation,

• presenting a clear definition of the value perspective and accounting safety in this context,

• budget consideration, and

• handling uncertainty.

Having this addressed will bring us closer to the answer if MCDA becomes the new standard for HTA appraisal.

When the access of innovative medicines in relation to MEAs is concerned, there is room for improvement in diverse national settings' functionalities and implementation. In this matter, many initiatives are undertaken to facilitate these processes, such as Shaping European Early Dialogues (SEED), aiming to support the mutual interests of HTA bodies and manufacturers through overcoming the national obstacles regarding data from the initial phases of clinical development (15). Furthermore, the International Coalition of Medicines Regulatory Authorities (ICMRA) already joined forces with all regulatory authorities toward facilitating accelerated procedures and equitable access to global clinical trials data regarding COVID-19. Moreover, they emphasize the importance of further monitoring for reflecting the real-life situation (63). This current setup might serve as an example for establishing better monitoring platforms for innovative medicines as subjects of MEAs. Bearing in mind the importance of real-life observations, effectiveness and safety reporting are pivotal data to be exchanged between the health authorities. Therefore, further development of activities such as EMA Registries Initiative is important and welcome (8).

MEA-related recommendations for better practices:

• establishing mutual/shared monitoring platforms for innovative medicines that would allow data exchange across the countries and save time and resources and

• support of collaborating initiatives for safety monitoring registries.

Conclusion

The access to innovative medicines can be facilitated through several go-to-market mechanisms. It might be beneficial to use adaptive pathways where RA and HTA provide early advice accounting safety. In addition, considering migration toward MCDA will highlight more aspects of value, account for safety, and give manufacturers “a joker” for a next step further in the access. More research is needed to prioritize the value elements, position the safety, and establish a common MCDA structure. Additionally, MEAs appear to pose a good solution for access of innovative medicines. However, further work should be done on solutions to overcome the obstacles around monitoring platforms in regard to patient safety in particular, sharing blinded data among countries to prevent double work and fasten submission processes. Increasing the attention to safety in MCDAs and MEAs will increase the trust of the authorities and improve the access for the manufacturers and the early availability safe and effective medicines for the patients.

Author Contributions

The study was initially conceptualized by TF and MJP. TF wrote the first draft and extracted the data used in the table, which further were reviewed, adjusted, and edited together with EPvP and MJP. All authors have read and agreed to the published version of the manuscript.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

1. EMA. Authorisation of Medicines. European Medicines Agency (2018). Available online at: https://www.ema.europa.eu/en/about-us/what-we-do/authorisation-medicines (accessed February 12, 2020).

Google Scholar

2. EMA. EMA. European Medicines Agency (2018). Available online at: https://www.ema.europa.eu/en/human-regulatory/marketing-authorisation/accelerated-assessment (accessed April 4, 2020).

3. GOVUK. Accelerated Access Review: Final Report. GOVUK. Available online at: https://www.gov.uk/government/publications/accelerated-access-review-final-report (accessed April 4, 2020).

4. Deloitte LLP. Patient Access to Innovative Medicines in Europe A Collaborative and Value Based Approach. (2019). Available online at: https://www2.deloitte.com/content/dam/Deloitte/cz/Documents/legal/deloitte-uk-patient-access-to-innovative-medicine-in-europe.pdf (accessed August 02, 2020).

5. FDA. Fast Track, Breakthrough Therapy, Accelerated Approval, Priority Review. FDA (2019). Available online at: https://www.fda.gov/patients/learn-about-drug-and-device-approvals/fast-track-breakthrough-therapy-accelerated-approval-priority-review (accessed April 4, 2020).

6. Baird LG, Banken R, Eichler H-G, Kristensen FB, Lee DK, Lim JCW, et al. Accelerated access to innovative medicines for patients in need. Clin Pharmacol Therapeut. (2014) 96:559–71. doi: 10.1038/clpt.2014.145

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Stakeholder Engagement Report 2018–2019. 24.

8. Bouvy JC, Blake K, Slattery J, Bruin MLD, Arlett P, and Kurz X. Registries in European post-marketing surveillance: a retrospective analysis of centrally approved products, 2005–2013. Pharmacoepidemiol Drug Safety. (2017) 26:1442–50. doi: 10.1002/pds.4196

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Panteli D, Arickx F, Cleemput I, Dedet G, Eckhardt H, Fogarty E, et al. Pharmaceutical regulation in 15 European countries: review. Health Syst Transit. (2016) 18:1–118.

PubMed Abstract | Google Scholar

10. Marsh K, Lanitis T, Neasham D, Orfanos P, and Caro J. Assessing the value of healthcare interventions using multi-criteria decision analysis: a review of the literature. Pharmacoeconomics. (2014) 32:345–65. doi: 10.1007/s40273-014-0135-0

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Reckers-Droog VT, van Exel NJA, and Brouwer WBF. Looking back and moving forward: on the application of proportional shortfall in healthcare priority setting in the Netherlands. Health Policy. (2018) 122:621–9. doi: 10.1016/j.healthpol.2018.04.001

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Versteegh MM, Ramos IC, Buyukkaramikli NC, Ansaripour A, Reckers-Droog VT, and Brouwer WBF. Severity-adjusted probability of being cost effective. PharmacoEconomics. (2019) 37:1155–63. doi: 10.1007/s40273-019-00810-8

CrossRef Full Text | Google Scholar

13. Vogler S, Paris V, Ferrario A, Wirtz VJ, de Joncheere K, Schneider P, et al. How can pricing and reimbursement policies improve affordable access to medicines? lessons learned from European Countries. Appl Health Econ Health Policy. (2017) 15:307–21. doi: 10.1007/s40258-016-0300-z

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Vogler S, Paris V, and Panteli D. Ensuring access to medicines: How to redesign pricing, reimbursement and procurement? In: Richardson E, Palm W, Mossialos E, editors. Copenhagen: European Observatory on Health Systems and Policies (2018). Available online at: http://www.ncbi.nlm.nih.gov/books/NBK526411/ (accessed March 13, 2020).

PubMed Abstract | Google Scholar

15. Dunlop WCN, Mullins CD, Pirk O, Goeree R, Postma MJ, Enstone A, et al. BEACON: a summary framework to overcome potential reimbursement hurdles. PharmacoEconomics. (2016) 34:1051–65. doi: 10.1007/s40273-016-0427-7

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Canadian Patient Safety Institute. Available online at: https://www.patientsafetyinstitute.ca/en/Pages/default.aspx (accessed July 29, 2020).

17. Ferrario A, Arāja D, Bochenek T, Catić T, Dankó D, Dimitrova M, et al. The implementation of managed entry agreements in Central and Eastern Europe: findings and implications. Pharmacoeconomics. (2017) 35:1271–85. doi: 10.1007/s40273-017-0559-4

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Gamba S, Pertile P, and Vogler S. The impact of managed entry agreements on pharmaceutical prices. Health Econ. (2020) 29:47–62. doi: 10.1002/hec.4112

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Ferrario A, and Kanavos P. Managed Entry Agreements for Pharmaceuticals: The European Experience. Available online at: https://scholar.google.com/scholar_lookup?title=Managed%20Entry%20Agreements%20for%20Pharmaceuticals%3A%20the%20European%20Experience&publication_year=2013&author=A.%20Ferrario&author=P.%20Kanavos (accessed October 27, 2020).

Google Scholar

20. Klemp M, Frønsdal KB, and Facey K. What principles should govern the use of managed entry agreements? Int J Technol Assess Health Care. (2011) 27:77–83. doi: 10.1017/S0266462310001297

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Granados A, Sampietro-Colom L, Asua J, Conde J, and Vazquez-Albertino R. Health Technology Assessment in Spain. Int J Technol Assess Health Care. (2000) 16:532–59. doi: 10.1017/S026646230010114X

CrossRef Full Text | Google Scholar

22. Murphy A, and Redmond S. To HTA or Not to HTA: identifying the factors influencing the rapid review outcome in Ireland. Value Health. (2019) 22:385–90. doi: 10.1016/j.jval.2018.10.011

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Csanádi M, Ozierański P, Löblová O, King L, Kaló Z, and Botz L. Shedding light on the HTA consultancy market: insights from Poland. Health Policy. (2019) 123:1237–43. doi: 10.1016/j.healthpol.2019.08.008

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Hajek P, Pecen L, Bulejova L, Cook M, Dolezal T, Dolezel J, et al. Multicriteria Decision Analysis (MCDA) in HTA—pilot study in the Czech Republic. Value Health. (2014) 17:A439. doi: 10.1016/j.jval.2014.08.1143

PubMed Abstract | CrossRef Full Text | Google Scholar

25. EMA. Patient Registries. European Medicines Agency (2018). Available online at: https://www.ema.europa.eu/en/human-regulatory/post-authorisation/patient-registries (accessed July 30, 2020).

Google Scholar

26. ENCePP. ENCePP Home Page. (2020). Available online at: http://www.encepp.eu/ (accessed July 31, 2020).

27. Bouvy JC, Sapede C, and Garner S. Managed entry agreements for pharmaceuticals in the context of adaptive pathways in Europe. Front Pharmacol. (2018) 9:280. doi: 10.3389/fphar.2018.00280

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Drummond MF, Sculpher MJ, Claxton K, Stoddart GL, and Torrance GW. Methods for the Economic Evaluation of Health Care Programmes. Oxford: Oxford University Press. (2015).

Google Scholar

29. Shafrin J, Lang K, and Maclean R. COVID-19 pandemic vindicates the ISPOR value flower. J Clin Pathways. (2020). Available online at: https://www.journalofclinicalpathways.com/article/covid-19-pandemic-vindicates-ispor-value-flower (accessed July 30, 2020).

Google Scholar

30. Lakdawalla DN, Doshi JA, Garrison LP, Phelps CE, Basu A, and Danzon PM. Defining elements of value in health care—a health economics approach: an ISPOR special task force report [3]. Value Health. (2018) 21:131–9. doi: 10.1016/j.jval.2017.12.007

CrossRef Full Text | Google Scholar

31. Toumi M, and Rémuzat C. Value added medicines: what value repurposed medicines might bring to society? J Mark Access Health Policy. (2016) 5:1264717. doi: 10.1080/20016689.2017.1264717

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Thokala P, Devlin N, Marsh K, Baltussen R, Boysen M, Kalo Z, et al. Multiple criteria decision analysis for health care decision making—an introduction: report 1 of the ISPOR MCDA Emerging Good Practices Task Force. Value Health. (2016) 19:1–13. doi: 10.1016/j.jval.2015.12.003

CrossRef Full Text | Google Scholar

33. Medic G, Korchagina D, Young KE, Toumi M, Postma MJ, Wille M, et al. Do payers value rarity? An analysis of the relationship between disease rarity and orphan drug prices in Europe. J Market Access Health Policy. (2017) 5:1299665. doi: 10.1080/20016689.2017.1299665

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Rawlins M. Reflections: NICE, health economics, and outcomes research. Value Health. (2012) 15:568–9. doi: 10.1016/j.jval.2012.03.002

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Baeten SA, Baltussen RMPM, Uyl-de Groot CA, Bridges J, and Niessen LW. Incorporating equity-efficiency interactions in cost-effectiveness analysis-three approaches applied to breast cancer control. Value Health. (2010) 13:573–9. doi: 10.1111/j.1524-4733.2010.00718.x

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Bleichrodt H, Doctor J, and Stolk E. A non-parametric elicitation of the equity-efficiency tradeoff in cost-utility analysis. J Health Econ. (2005) 24:655–78. doi: 10.1016/j.jhealeco.2004.10.001

CrossRef Full Text | Google Scholar

37. Kolasa K, Zwolinski KM, Zah V, Kaló Z, and Lewandowski T. Revealed preferences towards the appraisal of orphan drugs in Poland - multi criteria decision analysis. Orphanet J Rare Dis. (2018) 13:67. doi: 10.1186/s13023-018-0803-9

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Danzon PM, Drummond MF, Towse A, and Pauly MV. Objectives, budgets, thresholds, and opportunity costs—a health economics approach: an ISPOR special task force report [4]. Value Health. (2018) 21:140–5. doi: 10.1016/j.jval.2017.12.008

CrossRef Full Text | Google Scholar

39. Trump BD, Keisler JM, Galaitsi SE, Palma-Oliveira JM, and Linkov I. Safety-by-design as a governance problem. Nano Today. (2020) 35:100989. doi: 10.1016/j.nantod.2020.100989

CrossRef Full Text | Google Scholar

40. van de Wetering EJ, Stolk EA, van Exel NJA, and Brouwer WBF. Balancing equity and efficiency in the Dutch basic benefits package using the principle of proportional shortfall. Eur J Health Econ. (2013) 14:107–15. doi: 10.1007/s10198-011-0346-7

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Wagner M, Khoury H, Willet J, Rindress D, and Goetghebeur M. Can the EVIDEM framework tackle issues raised by evaluating treatments for rare diseases: analysis of issues and policies, and context-specific adaptation. PharmacoEconomics. (2016) 34:285–301. doi: 10.1007/s40273-015-0340-5

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Goetghebeur MM, and Cellier MS. Can reflective multicriteria be the new paradigm for healthcare decision-making? The EVIDEM journey. Cost Eff Resour Alloc. (2018) 16:54. doi: 10.1186/s12962-018-0116-9

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Cherny NI, Sullivan R, Dafni U, Kerst JM, Sobrero A, Zielinski C, et al. A standardised, generic, validated approach to stratify the magnitude of clinical benefit that can be anticipated from anti-cancer therapies: the European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS). Ann Oncol. (2015) 26:1547–73. doi: 10.1093/annonc/mdv249

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Schnipper LE, Davidson NE, Wollins DS, Blayney DW, Dicker AP, Ganz PA, et al. Updating the American Society of clinical oncology value framework: revisions and reflections in response to comments received. JCO. (2016) 34:2925–34. doi: 10.1200/JCO.2016.68.2518

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Willke RJ, Neumann PJ, Garrison LP, and Ramsey SD. Review of recent US value frameworks—a health economics approach: an ISPOR special task force report [6]. Value Health. (2018) 21:155–60. doi: 10.1016/j.jval.2017.12.011

CrossRef Full Text | Google Scholar

46. Goetghebeur MM, Wagner M, Khoury H, Levitt RJ, Erickson LJ, and Rindress D. Evidence and Value: impact on DEcisionMaking – the EVIDEM framework and potential applications. BMC Health Serv Res. (2008) 8:270. doi: 10.1186/1472-6963-8-270

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Goetghebeur MM, Wagner M, Khoury H, Rindress D, Grégoire J-P, and Deal C. Combining multicriteria decision analysis, ethics and health technology assessment: applying the EVIDEM decisionmaking framework to growth hormone for Turner syndrome patients. Cost Eff Resour Alloc. (2010) 8:4. doi: 10.1186/1478-7547-8-4

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Wahlster P, Goetghebeur M, Kriza C, Niederländer C, Kolominsky-Rabas P, and National Leading-Edge Cluster Medical Technologies “Medical Valley EMN.” Balancing costs and benefits at different stages of medical innovation: a systematic review of Multi-criteria decision analysis (MCDA). BMC Health Serv Res. (2015) 15:262. doi: 10.1186/s12913-015-0930-0

CrossRef Full Text | Google Scholar

49. Marsh K, IJzerman M, Thokala P, Baltussen R, Boysen M, Kaló Z, et al. Multiple criteria decision analysis for health care decision making—emerging good practices: report 2 of the ISPOR MCDA Emerging Good Practices Task Force. Value Health. (2016) 19:125–37. doi: 10.1016/j.jval.2015.12.016

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Thokala P, and Duenas A. Multiple criteria decision analysis for health technology assessment. Value Health. (2012) 15:1172–81. doi: 10.1016/j.jval.2012.06.015

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Phelps CE, Lakdawalla DN, Basu A, Drummond MF, Towse A, and Danzon PM. Approaches to aggregation and decision making—a health economics approach: an ISPOR Special Task Force Report [5]. Value Health. (2018) 21:146–54. doi: 10.1016/j.jval.2017.12.010

CrossRef Full Text | Google Scholar

52. Angelis A, and Kanavos P. Multiple Criteria Decision Analysis (MCDA) for evaluating new medicines in Health Technology Assessment and beyond: the advance value framework. Soc Sci Med. (2017) 188:137–56. doi: 10.1016/j.socscimed.2017.06.024

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Zimmermann N, Vogler S, and Bak Pedersen H. Policy options to deal with high-cost medicines – survey with European policy-makers. J Pharm Policy Pract. (2015) 8:P8, 2052-3211-8-S1-P8. doi: 10.1186/2052-3211-8-S1-P8

CrossRef Full Text | Google Scholar

54. Angelis A, Linch M, Montibeller G, Molina-Lopez T, Zawada A, Orzel K, et al. Multiple criteria decision analysis for HTA across four EU member states: piloting the advance value framework. Soc Sci Med. (2020) 246:112595. doi: 10.1016/j.socscimed.2019.112595

PubMed Abstract | CrossRef Full Text | Google Scholar

55. WHO. WHO Medicines Strategy: Framework for Action in Essential Drugs and Medicines Policy - 2000 - 2003. (2000). Available online at: https://apps.who.int/medicinedocs/en/d/Jwhozip16e/ (accessed February 12, 2020).

Google Scholar

56. Nicod E, and Kanavos P. Commonalities and differences in HTA outcomes: a comparative analysis of five countries and implications for coverage decisions. Health Policy. (2012) 108:167–77. doi: 10.1016/j.healthpol.2012.09.012

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Rawlins MD. Crossing the fourth hurdle. Br J Clin Pharmacol. (2012) 73:855–60. doi: 10.1111/j.1365-2125.2012.04263.x

CrossRef Full Text | Google Scholar

58. Kanavos P, Ferrario A, Tafuri G, and Siviero P. Managing risk and uncertainty in health technology introduction: the role of managed entry agreements. Glob Policy. (2017) 8:84–92. doi: 10.1111/1758-5899.12386

CrossRef Full Text | Google Scholar

59. Wenzl M, and Chapman S. Performance-based managed entry agreements for new medicines. In: OECD Countries and EU Member States: How They Work and Possible Improvements Going Forward. Paris: Organisation for Economic Co-operation and Development (2019). Available online at: https://www.oecd.org/health/health-systems/HWP-115-MEAs.pdf

Google Scholar

60. Ferrario A, and Kanavos P. Dealing with uncertainty and high prices of new medicines: a comparative analysis of the use of managed entry agreements in Belgium, England, the Netherlands and Sweden. Soc Sci Med. (2015) 124:39–47. doi: 10.1016/j.socscimed.2014.11.003

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Zampirolli Dias C, Godman B, Gargano LP, Azevedo PS, Garcia MM, Souza Cazarim M, et al. Integrative review of managed entry agreements: chances and limitations. PharmacoEconomics. (2020) 38:1165–85. doi: 10.1007/s40273-020-00943-1

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Marsh K, Goetghebeur M, Thokala P, and Baltussen R. Multi-Criteria Decision Analysis to Support Healthcare Decisions. Springer International Publishing (2017). doi: 10.1007/978-3-319-47540-0. Available online at: https://www.springer.com/gp/book/9783319475387

PubMed Abstract | CrossRef Full Text | Google Scholar

63. EMA ICMRA. International Coalition of Medicines Regulatory Authorities (ICMRA). European Medicines Agency (2018). Available online at: https://www.ema.europa.eu/en/partners-networks/international-activities/multilateral-organisations-initiatives/international-coalition-medicines-regulatory-authorities-icmra (accessed April 29, 2020).

Google Scholar