Maria Dimitrova1*†
Ivett Jakab2†
Zornitsa Mitkova1
Maria Kamusheva1
Konstantin Tachkov1
Bertalan Nemeth2
Antal Zemplenyi2,3
Dalia Dawoud4,5
Diana M. J. Delnoij6,7
François Houýez8
Zoltan Kalo2,9- 1Faculty of Pharmacy, Medical University of Sofia, Sofia, Bulgaria
- 2Syreon Research Institute, Budapest, Hungary
- 3Center for Health Technology Assessment and Pharmacoeconomics Research, Faculty of Pharmacy, University of Pécs, Pécs, Hungary
- 4National Institute for Health and Care Excellence, London, United Kingdom
- 5Faculty of Pharmacy, Cairo University, Cairo, Egypt
- 6Erasmus School of Health Policy and Management, Erasmus University, Rotterdam, Netherlands
- 7National Health Care Institute (Zorginstituut Nederland), Diemen, Netherlands
- 8EURORDIS: Eurordis, European Organization for Rare Diseases, Paris, France
- 9Center for Health Technology Assessment, Semmelweis University, Budapest, Hungary
Patients' perspectives are important to identify preferences, estimate values and appreciate unmet medical needs in the process of research and development and subsequent assessment of new health technologies. Patient and public involvement in health technology assessment (HTA) is essential in understanding and assessing wider implications of coverage and reimbursement decisions for patients, their relatives, caregivers, and the general population. There are two approaches to incorporating the patients' voice in HTA, preferably used in a mix. In the first one, patients, caregivers and/or their representatives directly participate at discussions in different stages of the HTA process, often at the same table with other stakeholders. Secondly, patient involvement activities can be supported by evidence on patient value and experience collected directly from patients, caregivers and/or their representatives often by patient groups Patient involvement practices, however, are limited in Central and Eastern European (CEE) countries without clear methodology or regulatory mechanisms to guide patient involvement in the HTA process. This poses the question of transferability of practices used in other countries, and might call for the development of new CEE-specific guidelines and methods. In this study we aim to map potential barriers of patient involvement in HTA in countries of the CEE region.
Introduction
Patients' perspectives are important to identify preferences, estimate values and appreciate unmet medical needs in the process of research and development and subsequent assessment of new health technologies (1). Health technology assessment (HTA) is a multidisciplinary process that uses explicit methods to determine the value of a health technology from different dimensions. Such health technology can be a medical test, device, medicine, vaccine, medical procedure, program, or even a health policy intervention (2). Patient and public involvement in HTA is essential in understanding and assessing wider implications of coverage and reimbursement decisions for patients, their relatives, caregivers, and the general population (1).
Patient involvement is intended to inform all the elements of an HTA from shaping research questions, early dialogues, informing cost-effectiveness models and/or the deliberation process (3, 4). There are two approaches to incorporating the patients' voice in HTA, preferably used in a mix. In the first one, patients, caregivers and/or their representatives directly participate at discussions in different stages of the HTA process, often at the same table with other stakeholders (5). Within this approach, several different methods can be used reflecting different aspects and levels of involvement [e.g., call for written comments, organizing a patient panel, inviting patient(s) to an Advisory Board or focus groups]. Secondly, patient involvement activities can be supported by evidence on patient value and experience collected directly from patients, caregivers and/or their representatives often by patient groups (6). Subsequently, the need for scientific justification of the point of view of patients and society is enhanced by the increasing number of studies to measure not only patients‘ preferences (i.e., time trade-off, standard gamble, etc.) but also patients‘ involvement in the collection of patient‘s reported outcomes (7, 8). Evidence generated through patient preference studies is becoming recognized by HTA organizations as a valuable addition to health technology submissions (9).
The general guide for patient involvement and mix of methods to use should be developed together with patients and fitting the local regulatory environment (2). The method and level of HTA is determined at the national and regional level, along with the method and level of patient involvement. There are European level initiatives such as the European network for Health Technology Assessment (EUnetHTA) jointly assessing selected new technologies and just recently there was a new harmonized regulation on HTA adopted in the European Union (EU), but no EU regulation of HTA applies until January 2025 (10). Thereby, to this date, the approach to patient involvement differs greatly by countries and regions (2, 11, 12). A survey by the European Patients' Forum in 2012 concluded that there are regional differences in the proportion of HTA agencies with and without patient involvement within Europe (13). These findings are in line with results of a survey performed by Health Technology Assessment International's Patient and Citizen Involvement Group (HTAi PCIG) in 2016 (14). The following countries reported patient involvement activities in one or more of their HTA organizations: Australia, Canada, Columbia, England, France, Germany, Italy, Netherlands, Sweden, Taiwan, Poland, Scotland and Wales. Remarkably, there was only one HTA organization from the Central Eastern European (CEE) region reporting on patient involvement activities—the Agency for Health Technology Assessment in Poland. Other CEE countries' HTA organizations did not respond to the survey, thereby there might have been some unreported patient involvement activities in CEE countries. However, the regional difference in response in itself warrants further investigation of patient involvement in HTA in CEE countries.
Compared other regions of Europe, CEE countries are in general at less advanced stages of implementing HTA in spite of the great need for evidence-based resource allocation decisions (15). Some exceptions exist, and a general positive trend can be observed, but the gap between the CEE region and the rest of Europe is still detectable. There are additional—albeit relating—cultural, historical, economic, organizational differences to be taken into consideration when applying good practices of HTA to the CEE context. This is particularly the case for patient involvement in HTA. Patient involvement practices are limited in CEE countries without clear methodology or regulatory mechanisms to guide patient involvement in the HTA process. This poses the question of transferability of practices used in other countries, and might call for the development of new CEE-specific guidelines and methods. However, this requires more insight into patient involvement HTA practices in CEE and the factors that promote or inhibit this. Therefore, in this study we aim to map potential barriers of patient involvement in HTA in countries of the CEE region.
Methodology
This research was conducted as part of the HTx project. HTx is a Horizon 2020 project supported by the European Union lasting for 5 years from January 2019 (www.htx-h2020.eu). The main aim of HTx is to create a framework for the Next Generation Health Technology Assessment to support patient-centered, societally oriented, real-time decision-making on access to and reimbursement for health technologies throughout Europe. Through Work Package 5, HTx aims to assess transferability aspects of novel HTA methodology from Western Europe (WE) countries to CEE countries and form recommendations. Patient-centered and socially-oriented HTA being in the focus of HTx, patient involvement in HTA was selected as a good practice to be included in such an assessment.
The study was conducted in three phases: (1) a scoping literature review to identify potential barriers of patient involvement in HTA, (2) a workshop with relevant stakeholders from CEE countries and experts from the HTx consortium to identify additional barriers, (3) an iterative process ran throughout these phases by CEE researchers from the HTx consortium deduplicating, merging and categorizing identified barriers.
Literature Review
The scoping literature review aimed to identify publications discussing potential barriers of patient involvement in HTA. The literature search was conducted through the PubMed database on the 30th of September in 2020, using the combination of the following keywords: patient; public; health technology assessment; HTA; involve; engage. The search was limited to English-language papers published in the past 10 years. Websites of relevant European Commission funded policy research projects (Innovative Medicines Initiative, Horizon 2020) were screened for project deliverables concerning patient involvement in HTA. Additionally we included experts of the field (patient involvement in HTA) that could propose additional peer-reviewed articles deem important but missed by the literature search.
Identified articles were deduplicated and screened first in the title and abstract screening phase, then those included were reviewed in full-text. The following exclusion criteria were used: (1) No abstract; (2) Not English language, (3) Not published in the past 10 years; (4) Not discussing HTA; (5) Not focusing on patient involvement; (6) Not mentioning any barriers of patient involvement in HTA. Identified barriers were extracted and served as a basis for the workshop and the iterative process.
Webinar
The second step of the study was a live webinar organized for HTx consortium members and CEE stakeholders, including payers, academics, healthcare professionals, industry and patient representatives. The aim of the webinar was to present study results and further identify barriers of patient involvement in HTA from the different perspectives, relevant for the CEE region. Because of the challenges related with the existing COVID-19 pandemic the webinar was held online 0.73 attendees from 12 CEE countries managed to joined the webinar on the 4th of December 2020. All invited attendees got pre-meeting materials for preparations and the interim results of the scoping literature review were presented as a basis for discussion. Based on their expertise and perspective the attendees provided written comments on the most important barriers identified through the literature search process.
Iterative Process
From September 2020 to January 2021, parallel to the other research phases, the main research team conducted the iterative process. The main research team consisted of eight researchers from the two CEE partners within the HTx consortium coming from different areas of health economics and patient-centered research. Final consensus for the identified barriers was reached after series of regular research team meetings for deduplication, merge, categorization and clarification of the identified barriers.
Results
Thirty-two (n = 32) published scientific articles and two gray literature sources meeting the predefined criteria were identified by the scoping literature review (Figure 1).
Figure 1. Flow-diagram of articles screened in the scoping literature review.
The list with the included papers (n = 34) with basic characteristics can be found in Table 1.
Table 1. List of papers included in the scoping literature review.
Included articles focused on (1) incentives and expectations of patient involvement in HTA (2) different methods of inclusion (3) methods of impact assessment (4) mapping current practices and actual impact. All included articles mentioned at least one potential barrier of patient involvement in HTA. The scoping review shows that, after 2015, there is a significant rise in the published articles looking into challenges and possible barriers of patient involvement in HTA.
The final set of potential barriers came along as a result of the scoping review, workshop and the parallel iterative process. Twenty-five (n = 25) potential barriers were identified and investigated. First, barriers were classified based on whose perspective they relate to from the two main stakeholder groups included in these patient involvement activities in HTA—payers/HTA organization representatives and patients. Fourteen (n = 14) barriers appeared from the side of payers/HTA bodies and eleven (n = 11) from the side of patients (Table 2). Then to ease understanding these barriers were grouped as follows: (1) payer/HTA side: Limited willingness to involve patients; Conflict of interest and confidentiality; Difficulties to finding the “right” patient; Lack of human resources at relevant public institutes; Not knowing how to involve patients. (2) Patient side: Lack of understanding the decision context; Lack of knowledge and guidance of evidence-based advocacy; Lack of resources to be spent on meaningful patient representation; Lack of ethical guidance for representativeness.
Table 2. Potential barriers of patient involvement in HTA in CEE countries.
Most of the barriers coming from the perspective of the payers/HTA bodies are related to the lack of defined rules how and when to include patients‘ representatives which increases the risk for lack of trust and fear to include patients in the HTA process.
From patients‘ perspective the most of the identified barriers were associated with lack of sufficiently explained methodology for the patient‘s role in the HTA process—lack of HTA and regulatory processes knowledge, medical language knowledge, etc. This could be attributed to the lack of organized programs in the health care systems at all for patient engagement in collecting patient reported outcomes and real world evidence.
Discussion
Patient engagement in HTA is considered as a transformative strategy which still needs to be adopted in most European countries (47). To a certain extent, the difficulties identified here are not specific for patient engagement in HTA nor for CEE countries, but emerge in patient involvement in healthcare decision-making in general and worldwide. For example, De Graaff et al. elaborate on questions such as whom to involve, how to involve patients and the public, and how to value their input in healthcare decision-making. They argue that more attention should be paid to the work needed for patient and public involvement to explain the gap between expected and current practice (48). Specific barriers such as time investment and budgetary constraints have been reported by Wiering et al. in their study of patient involvement in the development of Patient-Reported Outcome Measures (8). Peeters et al. describe patient organizations' involvement in quality improvement projects in The Netherlands as vulnerable, because of insecure funding and lack of negotiating power (49). Wiig et al. mapped patient involvement in regulatory practice in Norway, England, the Netherlands, and Australia. Their study pointed to several difficulties, such as how to incorporate patients' input, lack of willingness of patients to be involved, time and costs required, barriers related to organizational procedures, and dealing with emotions (50). Finally, in their analysis of decision-making in Dutch HTA practice, Moes et al. point to the risk of so-called epistemic injustice that occurs if patients are being “frustrated in their capacity to be heard and make themselves understood” (51).
Some studies also raise the questions of how we can build trust, partnership and collaborative working environment and what tools and knowledge could be effective to bring together patients/public and policy makers (52, 53).
However, despite the fact that several difficulties are of a more general nature, there are specific barriers for implementation in CEE countries, that call for specific guidelines of patient engagement (37). These specific barriers are, for instance, uncertainty in the role of civil societies, including patient organizations in political processes, or more pressing budgetary constraints. The development of CEE specific guidelines can build on general guidelines on patient involvement in HTA incorporating learnings from the identified barriers. Regarding patient engagement and patient preferences in HTA, many approaches were identified and listed as good practice documents, tracing possible mechanisms for inclusion of the patients in the decision-making process. These approaches also aim to establish guidance on different stakeholder involvement, identify emerging strategies and state of art methods do overcome challenges related to the patient engagement in the HTA process (25, 54–68). Notably, the European Patients' Academy on Therapeutic Innovation (EUPATI) published four guidance papers for patient involvement, one of which was about HTA. This guideline includes suggested working practices for both HTA agencies and patient organizations, as well as suggested patient involvement activities for general HTA processes and individual HTAs. Additionally, both EUnetHTA and Heath Technology Assessment International's Patient and Citizen Involvement Interest Group (HTAi PCIG) regularly publishes and updates guides and templates to aid patient involvement practices (69–71).
To our best knowledge, this is the first published study focusing on barriers of patient involvement in HTA specifically in CEE countries. However, a report with similar scope was identified through the gray literature review. The PARADIGM project was a public-private Innovative Medicines Initiative (IMI) partnership active between 2018 and 2020 (72). The project's mission was to provide a unique framework that enables structured, effective, meaningful, ethical, innovative, and sustainable patient engagement and demonstrates the “return on the engagement” for all players. Within the sustainability assessment of the patient engagement roadmap developed through the project, a workshop was held for stakeholders across the CEE region (37). Four end goals were identified for the European patient engagement landscape: (1) Establish an ethical, trust-based collaboration among all patient engagement stakeholders involved in medicines development; (2) Secure inclusive and diverse patient engagement; (3) Embed patient engagement in the mind-set and at every step and across organizations; (4) Ensure dedicated leadership and operational time, resources and funding. Forty-three (n = 43) barriers were identified challenging the reach to these four end goals. The concept was tested through a CEE workshop, however, the deliverable did not focus on CEE-specific barriers and the scope was research and development, not HTA.
A recent systematic literature review by Gagnon and colleagues (5) in 2021 aimed to summarize current evidence on patient and public involvement in HTA and to propose a framework to assess its impact. Thirty-one (n = 31) studies were included described in 36 publications. One study reporting on the a HTA case study of palliative care involved two CEE countries amongst others, Lithuania and Poland (73). All other studies had settings other than the CEE region; most of the included studies were conducted in Canada, followed by Italy, England, Germany and Finland, Austria, Ireland, Scotland, South Korea and Spain. Barriers published in the paper are in line with our findings and there are certain limitations of our research. Firstly, in terms of methodology, a non-systematic approach was taken when reviewing the literature and we limited our search to articles published in English. There is a chance some papers are missed from our review Secondly, CEE stakeholders included in the webinar had the chance to comment on the list of barriers and propose missing ones, however, the collated barriers were not ranked in terms of priority. We plan to continue our research with this step. Thirdly, our study did not cover potential action plans and recommendations to address the identified barriers. We deem this step to be crucial in advocating for a change of patient involvement practices, thereby our future steps will cover this aspect as well.
Next steps of the research include the ranking of identified barriers and proposing solutions by a broad CEE stakeholder group. The results of the latter studies are to be published in the future. All future studies are planned to be published to serve as a tool for meaningful patient involvement in HTA and pricing and reimbursement decisions in the CEE region.
Conclusion
In conclusion, there is a lack of published evidence on real-world extent of and barriers to patient involvement in HTA in CEE countries. On the other hand, there are available guidelines and best practices that could be adopted in these settings. Twenty-five potential barriers of patient involvement in HTA were identified relevant for the CEE context. These will need further investigation to assess their relative importance and develop potential solutions and recommendations for action to address them.
Data Availability Statement
The raw data supporting the conclusions of this article is available upon request from authors.
Author Contributions
MD performed the scoping review. MD and IJ drafted the manuscript. MK, ZM, KT, BN, AZ, DDa DDe, FH, and ZK reviewed the paper. All authors have provided valuable contributions to the manuscript, read, and approved the final manuscript.
Funding
The HTx project has received funding from the European Unions Horizon 2020 research and innovation program under grant agreement N° 825162. This dissemination reflects only the author's view and the Commission is not responsible for any use that may be made of the information it contains.
Conflict of Interest
Authors IJ, AZ, BN, and ZK are employed by Syreon Research Institute. At the time of the study IJ was the President of the European Patients' Forum Youth Group and a Board of Trustees member at the EUPATI Foundation.
The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1. Hunter A, Facey K, Thomas V, Haerry D, Warner K, Klingmann I, et al. EUPATI guidance for patient involvement in medicines research and development: health technology assessment. Front Med. (2018) 5:231. doi: 10.3389/fmed.2018.00231
2. O'Rourke B, Oortwijn W, Schuller T, International Joint Task Group. The new definition of health technology assessment: a milestone in international collaboration. Int J Technol Assess Health Care. (2020) 36:187–90. doi: 10.1017/S0266462320000215
3. Single AN, Facey KM, Livingstone H, Silva AS. Stories of patient involvement impact in health technology assessments: a discussion paper. Int J Technol Assess Health Care. (2019) 35:266–72. doi: 10.1017/S0266462319000552
4. SEED Project (Shaping European Early Dialogues: The Seed Project). Available online at: https://www.eunethta.eu/seed/ (accessed March 16, 2022).
5. Gagnon MP, Tantchou Dipankui M, Poder TG, Payne-Gagnon J, Mbemba G, Beretta V. Patient and public involvement in health technology assessment: update of a systematic review of international experiences. Int J Technol Assess Health Care. (2021) 37:e36:1–16. doi: 10.1017/S0266462321000064
6. Staniszewska S, Söderholm Werkö S. Mind the evidence gap: the use of patient-based evidence to create “complete HTA” in the twenty-first century. Int J Technol Assess Health Care. (2021) 37:e46. doi: 10.1017/S026646232100012X
7. Whitty JA. An international study of the public engagement practices of health technology assessment organizations. Value Health. (2013) 16:155–63. doi: 10.1016/j.jval.2012.09.011
8. Wiering B, De Boer D, Delnoij D. Patient involvement in the development of patient-reported outcomes: the developers‘ perspective. BMC Health Serv Res. (2017) 17:635. doi: 10.1186/s12913-017-2582-8
9. Bouvy JC, Cowie L, Lovett R, Morrison D, Livingstone H, Crabb N. Use of patient preference studies in HTA Decision making: a NICE perspective. Patient. (2020) 13:145–9. doi: 10.1007/s40271-019-00408-4
10. EUnetHTA. Available online at: https://www.eunethta.eu/. HTA Regulation https://ec.europa.eu/commission/presscorner/detail/en/IP_21_6771 (accessed February 8, 2022).
11. Guidelines for Stakeholder Engagement in Health Technology Assessment in Ireland Health Information and Quality Authority. Available online at: https://www.hiqa.ie/sites/default/files/2017-01/HTA-Guidelines-Stakeholder-Engagement.pdf (accessed February 20, 2022).
12. Serrano-Aguilar P, Asua-Batarrita J, Molina-Lopez M, Espallargues M, Pons-Rafols K, Garcia-Armesto S, et al. The spanish network of agencies for health technology assessment and services of the national health system (RedETS). Int J Tech Assess Health Care. (2019) 35:176–80. doi: 10.1017/S0266462319000205
13. European Patients Forum. Patient Involvement in Health Technology Assessment in Europe. Results of the EPF Survey. Available online at: https://www.eu-patient.eu/globalassets/projects/hta/hta-epf-final-report2013.pdf (accessed February 20, 2022).
14. Haley D, Werko S, Bakri R, Cameron A, Göhlen B, Myles S, et al. Involvement of consumers in health technology assessment activities by INAHTA agencies. Int J Technol Assess Health Care. (2013) 29:79–83. doi: 10.1017/S026646231200075X
15. Németh B, Goettsch W, Kristensen FB, Piniazhko O, Huić M, Tesar T, et al. The transferability of health technology assessment - the European perspective with focus on central and Eastern European countries, Exp Rev Pharmacoecon Outcomes Res. (2020) 20:321–30. doi: 10.1080/14737167.2020.1779061
16. Facey K, Boivin A, Gracia J, Hansen HP, Lo Scalzo A, Mossman J, et al. Patients' perspectives in health technology assessment: a route to robust evidence and fair deliberation. Int J Technol Assess Health Care. (2010) 26:334–40. doi: 10.1017/S0266462310000395
17. Frønsdal KB, Facey K, Klemp M, Norderhaug IN, Mørland B, Røttingen JA. Health technology assessment to optimize health technology utilization: using implementation initiatives and monitoring processes. Int J Technol Assess Health Care. (2010) 26:309–16. doi: 10.1017/S0266462310000309
18. Gagnon MP, Desmartis M, Lepage-Savary D, Gagnon J, St-Pierre M, Rhainds M, et al. Introducing patients' and the public's perspectives to health technology assessment: a systematic review of international experiences. Int J Technol Assess Health Care. (2011) 27:31–42. doi: 10.1017/S0266462310001315
19. Menon D, Stafinski T. Role of patient and public participation in health technology assessment and coverage decisions. Expert Rev Pharmacoecon Outcomes Res. (2011) 11:75–89. doi: 10.1586/erp.10.82
20. Danner M, Hummel JM, Volz F, van Manen JG, Wiegard B, Dintsios CM, et al. Integrating patients' views into health technology assessment: analytic hierarchy process (AHP) as a method to elicit patient preferences. J Int J Technol Assess Health Care. (2011) 27:369–75. doi: 10.1017/S0266462311000523
21. Gagnon MP, Gagnon J, St-Pierre M, Gauvin FP, Piron F, Rhainds M, et al. Involving patients in HTA activities at local level: a study protocol based on the collaboration between researchers and knowledge users. BMC Health Serv Res. (2012) 12:14. doi: 10.1186/1472-6963-12-14
22. Cavazza M, Jommi C. Stakeholders involvement by HTA organisations: why is so different? Health Policy. (2012) 105:236–45. doi: 10.1016/j.healthpol.2012.01.012
23. Drummond M, Tarricone R, Torbica A. Assessing the added value of health technologies: reconciling different perspectives. Value Health. (2013) 16(Suppl. 1):S7–13. doi: 10.1016/j.jval.2012.10.007
24. Gagnon MP, Candas B, Desmartis M. Involving patient in the early stages of health technology assessment (HTA): a study protocol. BMC Health Serv Res. (2014) 14:273. doi: 10.1186/1472-6963-14-273
25. Mühlbacher AC. Patient-centric HTA: different strokes for different folks. Expert Rev Pharmacoecon Outcomes Res. (2015) 15:591–7. doi: 10.1586/14737167.2015.1038245
26. Dipankui MT, Gagnon MP, Desmartis M, Légaré F, Piron F, Gagnon J, et al. Evaluation of patient involvement in a health technology assessment. Int J Technol Assess Health Care. (2015) 31:166–70. doi: 10.1017/S0266462315000240
27. Abelson J, Wagner F, DeJean D, Boesveld S, Gauvin FP, Bean S, et al. Public and patient involvement in health technology assessment: a framework for action. Int J Tecnol Assess Health Care. (2016) 32:256–64. doi: 10.1017/S0266462316000362
28. Husereau D, Henshall C, Sampietro-Colom L, Thomas S. Changing health technology assessment paradigms? Int J Technol Assess Health Care. (2016) 32:191–9. doi: 10.1017/S0266462316000386
29. Hämeen-Anttila K, Komulainen J, Enlund H, Mäkelä M, Mäkinen E, Rannanheimo P, et al. Incorporating patient perspectives in health technology assessments and clinical practice guidelines. Res Social Adm Pharm. (2016) 12:903–13. doi: 10.1016/j.sapharm.2015.12.005
30. Iskrov G, Stefanov R. Criteria for drug reimbursement decision-making: an emerging public health challenge in Bulgaria. Balkan Med J. (2016) 33:27–35. doi: 10.5152/balkanmedj.2015.15185
31. Weeks L, Polisena J, Scott AM, Holtorf AP, Staniszewska S, Facey K. Evaluation of patient and public involvement initiatives in health technology assessment: a survey of international agencies. Int J Technol Assess Health Care. (2017) 33:715–23. doi: 10.1017/S0266462317000976
32. Wortley S, Tong A, Howard K. Community views and perspectives on public engagement in health technology assessment decision making. Aust Health Rev. (2017) 41:68–74. doi: 10.1071/AH15221
33. Iskrov G, Miteva-Katrandzhieva T, Stefanov R. Health technology assessment and appraisal of therapies for rare diseases. Adv Exp Med Biol. (2017) 1031:221–31. doi: 10.1007/978-3-319-67144-4_13
34. Addario BJ, Fadich A, Fox J, Krebs L, Maskens D, Oliver K, et al. Patient value: perspectives from the advocacy community. Health Expect. (2018) 21:57–63. doi: 10.1111/hex.12628
35. Scott AM, Wale J. Patient advocate perspectives on involvement in HTA: an international snapshot. Res InvolvEngagem. (2017) 3:2. doi: 10.1186/s40900-016-0052-9
36. Wale J, Scott AM, Hofmann B, Garner S, Low E, Sansom L. Why patients should be involved in health technology assessment. Int J Tecnol Assess Health Care. (2017) 33:1–4. doi: 10.1017/S0266462317000241
37. Facey KM, Bedlington N, Berglas S, Bertelsen N, Single AN, Thomas V. Putting patients at the centre of healthcare: progress and challenges for health technology assessments. Patient. (2018) 11:581–9. doi: 10.1007/s40271-018-0325-5
38. Simpson S, Cook A, Miles K. Patient and public involvement in early awareness and alert activities: an example from The United Kingdom. Int J Technol Assess Health Care. (2018) 34:10–7. doi: 10.1017/S0266462317004421
39. Boudes M, Robinson P, Bertelsen N, Brooke N, Hoos A, Boutin M, et al. What do stakeholders expect from patient engagement: are these expectations being met? Health Expect. (2018) 21:1035–45. doi: 10.1111/hex.12797
40. Janssens R, van Overbeeke E, Verswijvel L, Meeusen L, Coenegrachts C, Pauwels K, et al. Patient involvement in the lifecycle of medicines according to Belgian stakeholders: the gap between theory and practice. Front Med. (2018) 5:285. doi: 10.3389/fmed.2018.00285
41. Paradigm Early Dialogue. Available online at: http://imi-paradigm.eu/Paradigm-documents/Report-on-PARADIGM-HTA-Early-Dialogues-Workshop-FINAL.pdf (accessed December 21, 2021).
42. Littlejohns P, Kieslich K, Weale A, Tumilty E, Richardson G, Stokes T, et al. Creating sustainable health care systems. J Health Organ Manag. (2019) 33:18–34. doi: 10.1108/JHOM-02-2018-0065
43. Wilking N, Bucsics A, Kandolf Sekulovic L, Kobelt G, Laslop A, et al. Achieving equal and timely access to innovative anticancer drugs in the European Union (EU): summary of a multidisciplinary CECOG-driven roundtable discussion with a focus on Eastern and South-Eastern EU countries. ESMO Open. (2019) 4:e000550. doi: 10.1136/esmoopen-2019-000550
44. Babac A, Damm K, Graf von der Schulenburg JM. Patient-reported data informing early benefit assessment of rare diseases in Germany: a systematic review. Health Econ Rev. (2019) 9:34. doi: 10.1186/s13561-019-0251-9
45. Wale JL, Sullivan M. Exploration of the visibility of patient input in final recommendation documentation for three health technology assessment bodies. Int J Technol Assess Health Care. (2020) 36:197–203. doi: 10.1017/S0266462320000240
46. Paradigm. Available online at: https://imi-paradigm.eu/wp-content/uploads/2020/09/PARADIGM-D6.5_PE-Sustainability-roadmap_FINAL.pdf (accessed December 21, 2021).
47. Perfetto EM, Harris J, Mullins CD, dosReis S. Emerging good practices for transforming value assessment: patients' voices, patients' values. Value Health. (2018) 21:386–93. doi: 10.1016/j.jval.2017.11.013
48. de Graaff B, Kleinhout-Vliek T, Van de Bovenkamp H. In the works: patient and public involvement and engagement in healthcare decision-making. Health Expect. (2021) 24:1903–4. doi: 10.1111/hex.13339
49. Peeters MG, Delnoij DM, Friele RD. Stronger, but not (yet) an equal. The use of quality improvement instruments and strategies by patient organisations in the Netherlands. Soc Sci Med. (2014) 115:56–63. doi: 10.1016/j.socscimed.2014.05.043
50. Wiig S, Rutz S, Boyd A, Churruca K, Kleefstra S, Haraldseid-Driftland C, et al. What methods are used to promote patient and family involvement in healthcare regulation? A multiple case study across four countries. BMC Health Serv Res. (2020) 20:616. doi: 10.1186/s12913-020-05471-4
51. Moes F, Houwaart E, Delnoij D, Horstman K. Questions regarding 'epistemic injustice' in knowledge-intensive policymaking: two examples from dutch health insurance policy. Soc Sci Med. (2020) 245:112674. doi: 10.1016/j.socscimed.2019.112674
52. Boothe K. Getting to the table: changing ideas about public and patient involvement in Canadian drug assessment. J Health Polit Policy Law. (2019) 44:631–63. doi: 10.1215/03616878-7530825
53. Boothe K. (Re)defining legitimacy in Canadian drug assessment policy? Comparing ideas over time. Health Econ Policy Law. (2021) 16:424–39. doi: 10.1017/S1744133121000013
54. Bridges JFP, Jones C. Patient-based health technology assessment: a vision of the future. Int J Technol Assess Health Care. (2007) 23:30e5. doi: 10.1017/S0266462307051549
55. Abelson J, Giacomini M, Lehoux P, Gauvin F-P. Bringing “the public” into health technology assessment and coverage policy decisions: from principles to practice. Health Policy. (2007) 82:37e50. doi: 10.1016/j.healthpol.2006.07.009
56. NICE. Patient and Public Involvement Policy. n.d. Available online at: https://www.nice.org.uk/about/nice-communities/public-involvement/patient-andpublic-involvement-policy (accessed December 21, 2021).
57. Perfetto EM, Oehrlein EM, Boutin M, Reid S, Gascho E. Value to whom? The patient voice in the value discussion. Value Health. (2017) 20:286e91. doi: 10.1016/j.jval.2016.11.014
58. Hansen HP, Draborg E, Kristensen FB. Exploring qualitative research synthesis: the role of patients' perspectives in health policy design and decision-making. Patient. (2011) 4:143e52. doi: 10.2165/11539880-000000000-00000
59. Frank L, Forsythe L, Ellis L, Schrandt S, Sheridan S, Gerson J, et al. Conceptual and practical foundations of patient engagement in research at the patient-centered outcomes research institute. Qual Life Res. (2015) 24:1033–41. doi: 10.1007/s11136-014-0893-3
60. Bridges JF, Hauber AB, Marshall D, Lloyd A, Prosser LA, Regier DA, et al. Conjoint analysis applications in healthda checklist: a report of the ISPOR good research practices for conjoint analysis task force. Value Health. (2011) 14:403–13. doi: 10.1016/j.jval.2010.11.013
61. Johnson FR, Lancsar E, Marshall D, Kilambi V, Mühlbacher A, Regier DA, et al. Constructing experimental designs for discrete-choice experiments: report of the ISPOR conjoint analysis experimental design good research practices task force. Value Health. (2013) 16:3e13 doi: 10.1016/j.jval.2012.08.2223
62. Hauber AB, González JM, Groothuis-Oudshoorn CG, Prior T, Marshall DA, et al. Statistical methods for the analysis of discrete choice experiments: a report of the ISPOR conjoint analysis good research practices task force. Value Health. (2016) 19:300–15. doi: 10.1016/j.jval.2016.04.004
63. Pinho-Gomes A, Stone J, Shaw T, Heath A, Cowl J, Norburn L, et al. Values, principles, strategies, and frameworks underlying patient and public involvement in health technology assessment and guideline development: A scoping review. Int J Technol Assess Health Care. 38:E46. doi: 10.1017/S0266462322000289
64. Piérart J, Léonard C, Chalon PX, Daue F, Mertens R. Stakeholder Involvement in KCE Working Processes. Brussels: Belgian Health Care Knowledge Centre (KCE) (2012). Report No.: Report 174C.
65. Mallery C, Canachari D, Smeeding L, Fernandez J, Lavallee D, Siegel J, et al. Innovative methods for stakeholder engagement: an environmental scan PHP5. Value Health. (2012) 15: PA14. doi: 10.1016/j.jval.2012.03.082
66. Deverka PA, Lavallee DC, Desai PJ, Esmail LC, Ramsey SD, Veenstra DL, et al. Stakeholder participation in comparative effectiveness research: defining a framework for effective engagement. J Comp Eff Res. (2012) 1:181–94. doi: 10.2217/cer.12.7
67. Guidance on information governance for health and social care services in Ireland. Health Information and Quality Authority. (2012). Available online at: https://www.hiqa.ie/sites/default/files/2017-01/HTA-Guidelines-Stakeholder-Engagement.pdf (accessed December 22, 2021).
68. Available online at: https://redets.sanidad.gob.es/productos/buscarProductos.do?metodo=descargarFichero&idProducto=1032 (accessed March 16, 2022).
69. HTA. Interest Group Patient and Citizen Involvment. Available online at: https://htai.org/interest-groups/pcig/ (accessed May 02, 2022).
70. EunetHTA Guidelines on Patient Input in REA. (2019). Available online at: https://www.eunethta.eu/wp-content/uploads/2019/06/Final_290519_Patient-Input-in-REAs.pdf (accessed February 18, 2022).
71. Patient Input in Relative Effectiveness Assessments. (2019). Available online at: https://www.eunethta.eu/wp-content/uploads/2019/06/Final_290519_Patient-Input-in-REAs.pdf (accessed February 18, 2022).
72. RARADIGM. Patient Engagement Toolbox. Available online at: https://imi-paradigm.eu/ (accessed February 10, 2022).
Keywords: patient engagement, health technology assessment (HTA), barrier, central and eastern EU countries, potential
Citation: Dimitrova M, Jakab I, Mitkova Z, Kamusheva M, Tachkov K, Nemeth B, Zemplenyi A, Dawoud D, Delnoij DMJ, Houýez F and Kalo Z (2022) Potential Barriers of Patient Involvement in Health Technology Assessment in Central and Eastern European Countries. Front. Public Health 10:922708. doi: 10.3389/fpubh.2022.922708
Received: 18 April 2022; Accepted: 13 June 2022;
Published: 28 July 2022.
Edited by:
Chiara de Waure, University of Perugia, ItalyReviewed by:
Janet L. Wale, HTAi Patient and Citizen Involvement in HTA Interest Group, CanadaKalina Andreevska, Sofia University, Bulgaria
Copyright © 2022 Dimitrova, Jakab, Mitkova, Kamusheva, Tachkov, Nemeth, Zemplenyi, Dawoud, Delnoij, Houýez and Kalo. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Maria Dimitrova, mdimitrova@pharmfac.mu-sofia.bg
†These authors have contributed equally to this work and share first authorship