The Value(s) of Vaccination: Building the Scientific Evidence According to a Value-Based Healthcare Approach

Objectives: To provide a new value-based immunization approach collating the available scientific evidence on the topic.

Methods: Four value pillars (personal, allocative, technical, and societal) applied to vaccination field were investigated. A systematic literature review was performed querying three database from December 24th, 2010 to May 27th, 2020. It included studies on vaccine-preventable diseases (VPDs) that mentioned the term value in any part and which were conducted in advanced economies. An in-depth analysis was performed on studies addressing value as key element.

Results: Overall, 107 studies were considered. Approximately half of the studies addressed value as a key element but in most of cases (83.3%) only a single pillar was assessed. Furthermore, the majority of papers addressed the technical value by looking only at classical methods for economic assessment of vaccinations whereas very few dealt with societal and allocative pillars.

Conclusions: Estimating the vaccinations value is very complex, even though their usefulness is certain. The assessment of the whole value of vaccines and vaccinations is still limited to some domains and should encompass the wider impact on economic growth and societies.

Introduction

Optimization of the use of limited available resources has been one of the main topics of discussion in healthcare so far. Over the past few years, different techniques have been developed to solve the problem of healthcare systems sustainability including evidence-based decision making (to ensure that only interventions with strong evidence of effectiveness and cost-effectiveness are used), quality improvement, and cost reduction (1). The value-based healthcare approach also emerged in order to address this issue with the aim to increase the value that is derived from the resources available for the population (2).

Michael Porter, in 2010, introduced the concept of value in healthcare describing it as the “health outcome achieved per dollar spent” (3). Given the characteristics of European health systems that aim to ensure equity, universality and solidarity to all, efficacy, and efficiency parameters highlighted by Porter's value paradigm are necessary but not sufficient (4). Allocation of resources should guarantee the highest level of quality and value in respect to the principles of equity in finance and distribution, hence a paradigm shift in the analysis of value was proposed by Sir Muir Gray (5), with the aim of integrating the programming of health services to the approaches of modern population medicine (5). The suggested triple value paradigm identified allocative, technical, and personal values as cornerstone of healthcare delivery (6). Subsequently, in 2019, the Expert Panel on Effective Ways of Investing in Health (EXPH) of the European Commission proposed a comprehensive concept built on four value-pillars to define “value(s)-based healthcare” for conveying the guiding principles of solidarity-based healthcare systems (1):

– The personal value in terms of appropriate care to achieve patients' personal goals;

– The allocative value, namely the equitable distribution of resources across all patient groups;

– The technical value meant as the achievement of the best possible outcomes with available resources;

– The societal value that is represented by the contribution of healthcare to social participation and connectedness.

Among health technologies, vaccines are one of the most successful of contemporary times. In the last 50 years, worldwide, the elimination and the control of many diseases was made possible by vaccines which drastically reduced mortality rate and related complications. According to the World Health Organization (WHO), vaccinations prevent 2–3 million deaths every year (7). Vaccinations also contribute to the sustainability of health systems thanks to the reduction in hospitalizations and required medical treatment, visits, and examinations and, therefore, direct medical costs (8). Moreover, through the avoidance of the disease occurrence and its consequences, vaccination promotes the overall economic growth of countries and counters poverty (9, 10). Despite these important benefits vaccinations are currently challenged by several concerns, such as unequal access, lack of resources, and vaccine hesitancy. An exploitation of the broad value of vaccines and vaccination could help contrasting such concerns and strengthening vaccination policies and strategies.

The aim of this study is to identify and systematically describe how the value of vaccine(s) and vaccination has been addressed in the academic literature considering the four EXPH value pillar framework. The description of the scientific evidence on the whole value of vaccination will be useful to foster a new value-based immunization approach.

Methods

A systematic review of the academic literature was carried out bearing in mind the four pillars of value (personal, allocative, technical, societal) as defined by the document by the EXPH of the European Commission (1). The relevant dimensions considered within each pillar were based on a comprehensive reading of EXPH report (1) and were as follows:

– Personal value: clinical and patients reported outcomes and experience measures, citizens' involvement and empowerment;

– Allocative value: access to vaccination, equity in provision, affordability, unwarranted variations, innovation;

– Technical value: health technology assessment (HTA) and models for assessing the economic evaluation of vaccines/vaccinations;

– Societal value: impact on population's wellbeing, productivity, and social cohesion; indirect and community protection; accountability of the decision-making process.

The systematic review was conducted and reported according to the Preferred Reporting Items for Systematic Reviews (PRISMA) (11).

Search Strategy

For each pillar, a systematic review was carried out. Hence, four systematic reviews were conducted. PubMed and Web of Science databases were queried. For the technical value pillar, the University of York's Center for Reviews and Dissemination (CRD) database was also used.

The search strings—reported as Supplementary Materials—were launched on May 27th 2020. The systematic review was performed from December 24th, 2010 onwards as Michael Porter' article “What is value in health care?” was published December 23, 2010 (3). As described in introduction, this article was the first scientific contribution on the value of vaccination according to the conceptual framework of EXPH.

The articles records were entered in a work sheet and screened according to the inclusion/exclusion criteria.

Inclusion/Exclusion Criteria

Studies were considered eligible if they dealt with vaccine-preventable diseases (VPDs), were conducted in advanced economies according to the list of the International Monetary Fund¹ and mentioned the term “value” at least once in the entire text. The choice of including only studies conducted in advanced economies was made assuming that the basic necessities are fulfilled and thus the meaning of value could be more comparable. Only studies written in English were included. Commentary, editorials, conference presentation, and references not provided with full text were excluded as well as studies conducted in animals or in vitro.

Selection Process and Data Extraction and Synthesis

Five researchers (E.C., A.T., I.G., E.B., M.M.) independently screened titles and abstracts first and full texts afterwards. Any disagreement was resolved by discussion or by the involvement of two other researchers (G.E.C., C.d.W.).

The following data were extracted from all the articles eventually included: author, year, country, type of study, study objective (single vaccine/vaccination strategy), study aim, target population, setting, vaccination strategy (mandatory/recommended/other), vaccination policy (free/co-payment/out of pocket), centrality of the value in the paper (main object of the study or not). Papers addressing the value as main object were further analyzed in order to collect information about the domains considered within each pillar. The additional information collected to disclose the different dimensions were as follows:

– Personal value: clinical outcomes (efficacy, effectiveness, safety etc.), patients' reported outcomes (PROMs, PREMs, satisfaction, quality of life), patients' reported experience measures (preference, attitudes, perceptions, awareness, knowledge, confidence, convenience, complacency, wellbeing, productivity, etc.), citizens' involvement, and empowerment in vaccination (participation, engagement, empowerment, commitment);

– Allocative value: accessibility (people possibility to obtain the services when needed)², equity (absence of unfair and avoidable or remediable differences in vaccination provision among population groups)³, affordability (payment for health−care services must be based on the principle of equity, ensuring affordable services for all), appropriateness (i.e., effectiveness of the service for a particular type of patient, from which the patient could find a benefit), unwarrantedvariations (overuse, underuse), innovation (in terms of organizational program or strategy setup to improve vaccination coverages);

– Technical value: economic model (cost of illness, cost effectiveness, cost benefit, cost utility, budget impact, fiscal impact, other), general drivers of costs (direct healthcare costs, direct non−healthcare costs, indirect costs, intangible costs, tax revenue), vaccination/vaccine−related drivers of costs (vaccine safety, vaccine efficacy, costs of vaccine administration, costs of vaccination program), and HTA;

– Societal value: population's wellbeing (productivity, social cohesion, etc.), indirect/community protection (herd immunity, social responsibility, etc.), shared decision-making process on (accountability, transparency, appraisal criteria, monitoring of the impact of the decision appraisal, etc.), and Health Impact Assessment (HIA).

Data were included in Tables 1–4 and summarized descriptively through frequencies.

TABLE 1

Table 1. Personal value as main object of the study (present in objectives/methods).

TABLE 2

Table 2. Allocative value as main object of the study (present in objectives/methods).

TABLE 3

Table 3. Technical value as main object of the study (present in objectives/methods).

TABLE 4

Table 4. Societal value as main object of the study (present in objectives/methods).

Results

The overall research in the three databases yielded a total of 9,714 articles across the four pillars. After duplicates removal, 7,122 articles were screened based on title and abstract. In total, 704 articles were assessed for eligibility and 107 articles were eventually included. Of these, 78 (72.90%) were primary researches, 16 (14.95%) reviews, and 13 (12.15%) systematic reviews. Sixty (56.1%) articles tackled more than one pillar of the value. In particular, 44 (44.1%) addressed two pillars, 13 (12.1%) three pillars, and 3 (2.8%) all four pillars.

Sixty studies (56.1%) analyzed one or more vaccines, while 47 (43.9%) analyzed a vaccination strategy. For those studies about vaccines, 15 focused on influenza (25.0%), nine on pneumococcal vaccines (either conjugate or polysaccharide) (15.0%), seven on Human papilloma virus (HPV) (11.7%), four on multiple vaccines (6.67%), three on Herpes zoster virus (HZV) (5.0%), three on rotavirus (5.0%), one on Meningococcus B (MenB), one on mumps, and one on Norovirus (1.7% each); for 16 studies the vaccine was not specified (26.7%). As for the 47 studies focused on a vaccination strategy, eight regarded influenza (17.0%), eight pneumococcal vaccines (either conjugate or polysaccharide) (17.0%), eight HPV (17.0%), six multiple vaccines (12.8%), three HZV (6.4%), one pertussis (2.1%), one rubella (2.1%), and one rotavirus (2.1%); 11 studies did not specify the vaccination strategy (23.4%). Most of the studies were conducted in the USA (n = 27, 25.27%); six (5.61%) in Japan; five (4.67%) in the UK; five (4.67%) in Australia and five (4.67%) in Italy; four (3.74%) in Germany and Canada each; three (2.80%) in Spain, three in Belgium (2.80%) and three (2.80%) in Norway; 20 (18.69%) studies had multiple countries as a setting, while for 17 (15.89%) the country setting was not specified; the remaining five (0.93%) studies were conducted in Denmark, France, Korea, New Zealand, Sweden.

Value was the main objective of the research in 54 studies (50.5%), nine among them addressing more than one value pillar. A description of the main findings for each pillar is reported hereafter.

Personal Pillar

Fifteen studies addressed personal value as key element (Table 1). Of these, eight were primary researches (75.9%), four reviews (13.8%), and three systematic reviews (10.3%).

Eight studies (53.3%) did not mention any specific vaccine (12, 14, 16, 19, 20, 23, 25, 26), three focused on HPV (20%) (13, 15, 17), two addressed childhood vaccination in general (13.3%) (18, 24), one MenB vaccination (21), and one seasonal influenza (6.7% each) (22).

Only two studies out of 15 (13.3%) took into consideration clinical outcomes (25, 26) in terms of impact on mortality and morbidity, whereas most of the articles (86.7%) reported information about patients' reported experience measures (12–24). Eventually, the issue of citizens' involvement and empowerment was assessed in three out of 15 articles (20.0%) (17, 22, 23). Patients' experience was analyzed mainly in terms of knowledge (13, 22, 23), perceptions/beliefs (12, 13, 16–22, 24), attitudes (12–14, 16, 18, 19, 23), and preferences (15, 18, 19, 21, 22). Citizens' involvement and empowerment in vaccination was investigated in terms of empowerment (22, 23) and engagement (12, 23).

Allocative Pillar

The search brought a total number of six articles addressing allocative value as main objective, among which three (50%) were primary studies (12, 28, 30) and three (50%) reviews (26, 27, 29). The studies were very heterogeneous in terms of vaccines taken into account and only few dealt with a specific vaccination, namely influenza in one case (16.7%) (28), and HPV in another one (16.7%) (27).

None of the included articles addressed the issue of unwarranted variations. Three (50%) papers addressed the topic of the affordability (26, 27, 29), whereas four (66.7%) articles brought the attention on innovative aspects in terms of organizational delivery (28), informational model (12, 30) or extension of the vaccination to previously excluded populations (27). Equity was addressed by two (33.33%) studies (26, 27), while only one (16.67%) tackled accessibility and appropriateness (27).

Technical Pillar

Thirty-six studies addressed the technical value as a key element. The majority (eight studies; 22.22%) of the studies concerned influenza vaccines (35, 38, 48–52, 58) and HPV (eight studies; 22.22%) (27, 33, 40–42, 46, 55, 61), five pneumococcal vaccines (13.9%) (34, 36, 44, 45, 62), three rotavirus (8.3%) (39, 47, 59), while six (16.7%) multiple vaccines (31, 32, 37, 43, 56, 60). Eventually, six (16.67%) did not address any specific vaccines in particular (25, 26, 29, 53, 54, 57).

With respect to specific dimensions considered within the pillar, a total of 19 (52.8%) papers focused on the cost-effectiveness, including also the cost-utility (25, 27, 32, 34, 35, 37, 38, 40, 43, 44, 46–48, 50, 53, 56, 59–61). Six studies (16.7%) addressed the cost of illness (48–50, 54, 58, 62), two (5.6%) the cost-benefit (33, 55), and one (2.8%) the budget impact (45). The study by Jiménez et al. (46) was a HTA report addressing also current use of the technology, technical characteristics, and the clinical effectiveness. Four papers (11.1%) considered alternative ways/models to assess the economic value of vaccination (31, 39, 41, 42). One article assessed the economic surplus produced by vaccination defined as the sum of all health and economic benefits of vaccination, minus the costs of vaccine development, production, and distribution (42). One study developed a novel approach (using the payoff method) to value real options in health care (41). Quality of Care (QoC)-Isocurve Model and budget optimization model was used by one study (39), while one developed a new value-based incentive payment model for immunizations delivered in community pharmacies (31). Eventually, four studies (11.1%) either described or tried to identify the broader potential economic impacts of vaccination (26, 29, 36, 57). In particular, Postma et al. (57) stressed the difficulty of assessing the full economic (and societal) value of the vaccination and suggested that wider economic and societal values should be integrated or at least considered in the economic values of vaccination programs. Lifelong benefits for individuals, the reduction of indirect costs, and improved quality of life should not be underestimated and should be captured as an integral part of the economic assessment of vaccination programs (57). Bloom et al. suggested an urgent need of concerted efforts to identify datasets that could estimate the full benefits of vaccination programs and fit the challenges. High-quality data are needed to support research to “enjoy gains in public health that are more transformational than incremental” (29).

Societal Pillar

Nine studies analyzed value as main objective in the societal pillar. Five (55.6%) were primary studies (10, 23, 30, 57, 63), three (33.3%) reviews (26, 29, 64), and one (11.1%) was a systematic review (36). Study vaccinations were influenza in one paper (11.1%) (63) pneumococcal in another one (11.1%) (36) and HPV (11.1%) (10). Eventually, in six studies (66.7%) the study vaccination was not reported (23, 26, 29, 30, 57, 64).

Concerning the specific dimensions considered within the pillar, the topics addressed were population's wellbeing (seven studies, 77.8%) (10, 23, 26, 29, 36, 57), indirect/community protection (five studies, 55.5%) (10, 26, 29, 36, 57, 63), and shared decision-making (three studies, 33.3%) (23, 30, 64). Health Impact Assessment was not performed by any of the selected studies.

Population's wellbeing was indirectly described as gain of productivity (10, 26, 29, 57), healthy aging (57), as well as through social cohesion in terms of awareness of the population of being part of a community and of contributing to it (23, 36, 57). Of note, Barnighausen spoke about “outcome-related productivity,” which considers direct health care costs or loss of productive time and “long term mental, physical, or cognitive impairments” and “behavior-related productivity gains” brought by vaccination (10). Also benefits from reduction in the development of drug resistance was mentioned (10, 36). Finally, shared decision-making was proposed by three articles in different forms: a set of core evaluation criteria to be applied to the vaccine evaluation procedure, with the ultimate aim to improve the accountability in vaccine decision-making (64); a community intervention to reduce vaccine hesitancy by providing pro-vaccine parents with tools to “engage in positive dialogue about immunization in their community,” so that pro-vaccine parents were empowered to be immunization advocates (23); and with the use of an interactive, internet-based decision tool that could help parents taking informed decisions about vaccination for their children (30).

Discussion

Although the real and tangible benefits of vaccines are recognized globally, in many countries a decline in coverages has been observed in recent years and is still ongoing. Concerns or fears about vaccines safety, lack of trust, social norms, myths undermining confidence in vaccines, failure by some healthcare to provide evidence-informed advice, barriers in the access to vaccines (e.g., poor availability, co-payments), and failure to understand the underlying mechanisms that decrease vaccination confidence are some of the barriers to vaccination (1). The understanding of the broad value of vaccination and the effective translation of this knowledge to the different stakeholders is therefore important to strengthen vaccination policies and strategies and counteracting disinformation and, misinformation.

This systematic review has shown that the value of vaccine(s) and vaccination(s) was addressed by a few number of papers in the last 10 years. Only 54 out of the initial 7,122 articles (0.8%) addressed the topic in depth. The most studied vaccines were seasonal influenza, pneumococcal, and HPV vaccines. Most of the papers included in the systematic review addressed the technical value and then the personal one, while the allocative and societal values were addressed in very few papers. These results are noteworthy as it calls the attention on the need for more research on these dimensions that are relevant for fully addressing the value of vaccination also from the societal perspectives.

Nonetheless, it should be pointed out that the assessment of the allocative and societal pillars poses several challenges. In fact, the allocative value encompasses many dimensions that cannot be assessed in a standardized way and are context-specific. The role played by equity, accessibility and appropriateness within the immunization agenda 2030 (65) and also in respect to COVID-19 vaccines emphasizes the importance of their evaluation (66).

As far as the societal value is concerned, several interesting suggestions for the development and the application of new methods and criteria emerged in respect to the impact of vaccination on social cohesion and Health Impact Assessment (AMR). In particular, with respect to AMR, vaccinations are still undervalued, even though they could reduce both the incidence of potential resistant infections and the inappropriate use of antimicrobials (67). Another important but still under-investigated aspect is the patients/citizens' empowerment and engagement in the light of a shared decision-making, that is a key component of patient-centered health systems. An important point should be made also in regard to the evaluation of the technical value of vaccines and vaccination as the most of economic evaluations still adopt a narrow perspective on vaccines and vaccination benefits considering only health care cost savings and health gains. Nevertheless, a broader perspective should be undertaken, considering also outcome-related and care-related productivity gains, behavior-related output gains, health-based community externalities, risk reduction gains, reduction of comorbidities and other infections, and improvement in social equity (68). Nevertheless, also for this purpose, new tools are needed (68).

This systematic review has actually showed that there is room for improvements in regard to the concept of the value-based healthcare applied to vaccination.

To the best of our knowledge, this is the first systematic review that collated together the evidence on this topic shedding light on research gaps and needs. Evidence on the personal, allocative, technical, and societal value of vaccine(s)/vaccination published in the last decade and systematized in our review was described as foundation of a following consultation with international experts of the field. The result was the issuing of recommendations for research, decision-making, and public engagement to drive a value-based decision-making on vaccination (69). Further research activities would be desirable to address the same issues in low- and middle-income countries where the broad value of vaccines could be evaluated to a different extent.

Several limitations should be considered when interpreting results. Only articles published in English until May 27^th, 2020 were included, which might have led to fail identifying all the available evidence on the topic up to now. Furthermore, a selection bias could not be completed ruled out even though the screening process was performed by five researchers independently and based on a lenient criterion (value mentioned in any part of the text). The search strings as well as the synthesis of evidence was informed by the content of the document published by the EXPH of the European Commission but this cannot exclude that some dimensions could have been missed. Furthermore, in order to keep the systematic review strictly focused on the concept of value, articles were not included if they did not explicitly mention that word. We are aware that a lot of primary studies addressing the different dimensions of value of vaccines and vaccination have been published up to now, but our goal was not to make a systematic review of studies focused on single dimensions. A quality assessment of the included articles was not performed; therefore, we could not assess the methodological correctness of the included studies. However, in our opinion, this does not impair our work as we wanted to provide an overview of pillars and dimensions assessed without addressing the robustness of methods used to do it. Eventually, the heterogeneity of evidence limits the possibility to further elaborate on information and data coming from the studies and issue definite findings. Nevertheless, in our opinion, this first depiction could help bringing forward the assessment and the appraisal of the value of vaccines and vaccination in the field of both academic research and supranational, national and local decision-making.

Conclusions

Even though vaccines have contributed substantially to the reduction of the burden from infectious diseases, estimating their value is very complex. Each year, three million lives are saved thanks to vaccination. In developed countries, routine vaccinations have led to complete elimination or control of many diseases, drastically reducing their incidence, mortality rate and related complications (70). This article constitutes a comprehensive source of information on the value of vaccines/vaccinations and depicts the current evidence on the assessment of their personal, allocative, technical, and societal values. Increasing awareness of the true value of vaccines and vaccinations is of great importance as long as hesitation and underuse of vaccines may still lead to serious outbreaks. In fact, in a context of increasing pressure on healthcare budgets, vaccinations can contribute to the sustainability of health systems through a reduced and more efficient use of healthcare resources (71). As presented, the assessment of the whole value of vaccines and vaccinations needs to consider not just the direct impact on health and healthcare but also the wider impact on economic growth and societies. These wider impacts, although difficult to measure and still under-investigated, should be taken into consideration to better depict the whole values of vaccines and vaccinations and counteract vaccine hesitancy and misuse.

Author Contributions

GEC, WR, and CdW conceptualized, designed, and supervised the study. AT, EC, MM, IG, and EB performed the data collection, screened the articles, and analyzed the data. AT and EC wrote the original draft of the manuscript that was revised and edited by GEC and CdW. All authors revised the manuscript and approved its submission for publication and certified that the work is original and their own.

Funding

This study was funded by VIHTALI (Value In Health Technology and Academy for Leadership & Innovation), Spin-Off of Università Cattolica del Sacro Cuore, Rome, Italy. This paper reports the results of a VIHTALI project entitled “The value(s) of vaccination: building the scientific evidence according to a value-based healthcare approach”. The project was also funded by MSD.

Conflict of Interest

GEC, EC, AT, IG, EB, MM, and CdW received a fee by VIHTALI (Value in Health Technology and Academy for Leadership & Innovation, Spin-Off of Università Cattolica del Sacro Cuore, Rome, Italy) for the scientific activities of the project entitled “The value (s) of vaccination: building the scientific evidence according to a Value-Based Healthcare approach.” WR was not paid for his time by him. The project was funded by MSD. The funders had no role in the design of the study, the collection, analysis and interpretation of data, or the writing of the manuscript.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We would like to thank the members of the steering committee of the project (France David Bloom, Americo Cicchetti, Siddhartha Datta, Giovanni Rezza, Luigi Siciliani, Mondher Toumi, York Zollner) for their input on the reporting of results of the systematic review.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2022.786662/full#supplementary-material

Footnotes

1. ^https://www.imf.org

2. ^https://www.who.int/gender-equity-rights/understanding/accessibility-definition/en/#:\sim:text=%E2%80%9Cis%20
understood%20as%20the%20availability,services%20when%20they%20need%20them%E2%80%9D

3. ^https://www.who.int/health-topics/social-determinants-of-health#tab=tab_3

References

1. Expert Panel on Effective Ways of Investing in Health (EXPH). Draft Opinion on Defining value in “value-based healthcare”. (2019). Available online at: https://ec.europa.eu/health/expert_panel/sites/expertpanel/files/024_valuebasedhealthcare_en.pdf (accessed August, 2021).

2. Gray M. Value based healthcare. BMJ. (2017). 356:j437. doi: 10.1136/bmj.j437

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Porter ME. What is value in health care? N Engl J Med. (2010) 363:2477–81. doi: 10.1056/NEJMp1011024

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Kaplan RS, Porter ME. How to solve the cost crisis in health care. Harv Bus Rev. (2011). 89:46–52, 54, 56–61 passim.

PubMed Abstract | Google Scholar

5. Gray JA. The shift to personalised and population medicine. Lancet. (2013) 382:200–1. doi: 10.1016/s0140-6736(13)61590-1

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Gray M. Designing healthcare for a different future. J R Soc Med. (2016) 109:453–8. doi: 10.1177/0141076816679781

PubMed Abstract | CrossRef Full Text | Google Scholar

7. WHO. What is Health Financing for Universal Coverage? Geneva: World Health Organization. 2020. Available online at: http://www.who.int/health_financing/universal_coverage_definition (accessed August, 2021).

8. WHO. Global Action Plan on Antimicrobial Resistance. Draft Resolution with Amendments Resulting From Informal Consultations. (2015). Available online at: http://apps.who.int/gb/ebwha/pdf_files/WHA68/A68_ACONF1Rev1-en.pdf (accessed August, 2021).

9. Bloom DE. The value of vaccination. Adv Exp Med Biol. (2011) 697:1–8. doi: 10.1007/978-1-4419-7185-2_1

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Bärnighausen T, Bloom DE, Cafiero-Fonseca ET, O'Brien JC. Valuing vaccination. Proc Natl Acad Sci USA. (2014) 111:12313–9. doi: 10.1073/pnas.1400475111

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. (2009) 6:e1000097. doi: 10.1371/journal.pmed.1000097

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Attwell K, Freeman M. I immunise: an evaluation of a values-based campaign to change attitudes and beliefs. Vaccine. (2015) 33:6235–40. doi: 10.1016/j.vaccine.2015.09.092

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Baumann A, Andersen B, Østergaard L, Larsen MB. Sense and sensibility: decision-making and sources of information in mothers who decline HPV vaccination of their adolescent daughters. Vaccine X. (2019). 2:100020. doi: 10.1016/j.jvacx.2019.100020

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Böhm R, Betsch C, Korn L. Selfish-rational non-vaccination: experimental evidence from an interactive vaccination game. J Econ Behav Organ. (2016). 131(Pt B):183–95. doi: 10.1016/j.jebo.2015.11.008

CrossRef Full Text | Google Scholar

15. Brown DS, Poulos C, Johnson FR, Chamiec-Case L, Messonnier ML. Adolescent girls' preferences for HPV vaccines: a discrete choice experiment. Adv Health Econ Health Serv Res. (2014) 24:93–121.

PubMed Abstract | Google Scholar

16. Cataldi JR, Sevick C, Pyrzanowski J, Wagner N, Brewer SE, Narwaney KJ, et al. Addressing personal parental values in decisions about childhood vaccination: measure development. Vaccine. (2019) 37:5688–97. doi: 10.1016/j.vaccine.2019.08.009

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Dunn AG, Leask J, Zhou X, Mandl KD, Coiera E. Associations between exposure to and expression of negative opinions about human papillomavirus vaccines on social media: an observational study. J Med Internet Res. (2015). 17:e144. doi: 10.2196/jmir.4343

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Gidengil C, Lieu TA, Payne K, Rusinak D, Messonnier M, Prosser LA. Parental and societal values for the risks and benefits of childhood combination vaccines. Vaccine. (2012) 30:3445–52. doi: 10.1016/j.vaccine.2012.03.022

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Lavail KH, Kennedy AM. The role of attitudes about vaccine safety, efficacy, and value in explaining parents' reported vaccination behavior. Health Educ Behav. (2013) 40:544–51. doi: 10.1177/1090198112463022

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Luyten J, Beutels P. The social value of vaccination programs: beyond cost-effectiveness. Health Aff (Millwood). (2016) 35:212–8. doi: 10.1377/hlthaff.2015.1088

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Marshall HS, Chen G, Clarke M, Ratcliffe J. Adolescent, parent and societal preferences and willingness to pay for meningococcal B vaccine: a discrete choice experiment. Vaccine. (2016) 34:671–7. doi: 10.1016/j.vaccine.2015.11.075

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Nowak GJ, Shen AK, Schwartz JL. Using campaigns to improve perceptions of the value of adult vaccination in the United States: health communication considerations and insights. Vaccine. (2017) 35:5543–50. doi: 10.1016/j.vaccine.2017.08.064

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Schoeppe J, Cheadle A, Melton M, Faubion T, Miller C, Matthys J, et al. The immunity community: a community engagement strategy for reducing vaccine hesitancy. Health Promot Pract. (2017) 18:654–61. doi: 10.1177/1524839917697303

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Steens A, Stefanoff P, Daae A, Vestrheim DF, Riise Bergsaker MA. High overall confidence in childhood vaccination in Norway, slightly lower among the unemployed and those with a lower level of education. Vaccine. (2020) 38:4536–41. doi: 10.1016/j.vaccine.2020.05.011

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Timmis JK, Rigat F, Rappuoli R. Core values for vaccine evaluation. Vaccine. (2017) 35(Suppl 1):A57–62. doi: 10.1016/j.vaccine.2016.11.034

PubMed Abstract | CrossRef Full Text | Google Scholar

26. van der Putten IM, Evers SM, Deogaonkar R, Jit M, Hutubessy RC. Stakeholders' perception on including broader economic impact of vaccines in economic evaluations in low and middle income countries: a mixed methods study. BMC Public Health. (2015). 15:356. doi: 10.1186/s12889-015-1638-0

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Audisio RA, Icardi G, Isidori AM, Liverani CA, Lombardi A, Mariani L, et al. Public health value of universal HPV vaccination. Crit Rev Oncol Hematol. (2016) 97:157–67. doi: 10.1016/j.critrevonc.2015.07.015

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Bell S, Chantler T, Paterson P, Mounier-Jack S. Is flu vaccination opt-out feasible? Evidence from vaccination programme implementers and managers in the English National Health Service. Vaccine. (2020) 38:4183–90. doi: 10.1016/j.vaccine.2020.04.026

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Bloom DE. Valuing vaccines: deficiencies and remedies. Vaccine. (2015) 33(Suppl 2):B29–33. doi: 10.1016/j.vaccine.2015.03.023

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Connolly T, Reb J. Toward interactive, Internet-based decision aid for vaccination decisions: better information alone is not enough. Vaccine. (2012) 30:3813–8. doi: 10.1016/j.vaccine.2011.12.094

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Bacci JL, Hansen R, Ree C, Reynolds MJ, Stergachis A, Odegard PS. The effects of vaccination forecasts and value-based payment on adult immunizations by community pharmacists. Vaccine. (2019) 37:152–9. doi: 10.1016/j.vaccine.2018.11.018

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Barbieri M, Capri S. Is vaccination good value for money? A review of cost-utility analyses of vaccination strategies in eight European countries. Epidemiol. Biostat. Publ Health. (2016). 13:e11853-1-30. doi: 10.2427/11853

CrossRef Full Text | Google Scholar

33. Bärnighausen T, Bloom DE, Cafiero ET, O'Brien JC. Economic evaluation of vaccination: capturing the full benefits, with an application to human papillomavirus. Clin Microbiol Infect. (2012) 18(Suppl 5):70–6. doi: 10.1111/j.1469-0691.2012.03977.x

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Boccalini S, Bechini A, Levi M, Tiscione E, Gasparini R, Bonanni P. Cost-effectiveness of new adult pneumococcal vaccination strategies in Italy. Hum Vaccin Immunother. (2013) 9:699–706. doi: 10.4161/hv.23268

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Brogan AJ, Talbird SE, Davis AE, Thommes EW, Meier G. Cost-effectiveness of seasonal quadrivalent versus trivalent influenza vaccination in the United States: a dynamic transmission modeling approach. Hum Vaccin Immunother. (2017) 13:533–42. doi: 10.1080/21645515.2016.1242541

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Cafiero-Fonseca ET, Stawasz A, Johnson ST, Sato R, Bloom DE. The full benefits of adult pneumococcal vaccination: a systematic review. PLoS ONE. (2017). 12:e0186903. doi: 10.1371/journal.pone.0186903

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Damm O, Ultsch B, Horn J, Mikolajczyk RT, Greiner W, Wichmann O. Systematic review of models assessing the economic value of routine varicella and herpes zoster vaccination in high-income countries. BMC Public Health. (2015). 15:533. doi: 10.1186/s12889-015-1861-8

PubMed Abstract | CrossRef Full Text | Google Scholar

38. de Waure C, Veneziano MA, Cadeddu C, Capizzi S, Specchia ML, Capri S, et al. Economic value of influenza vaccination. Hum Vaccin Immunother. (2012) 8:119–29. doi: 10.4161/hv.8.1.18420

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Dort T, Schecroun N, Standaert B. Improving the hospital quality of care during winter periods by optimizing budget allocation between rotavirus vaccination and bed expansion. Appl Health Econ Health Policy. (2018) 16:123–32. doi: 10.1007/s40258-017-0362-6

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Favato G, Baio G, Capone A, Marcellusi A, Costa S, Garganese G, et al. Novel health economic evaluation of a vaccination strategy to prevent HPV-related diseases: the BEST study. Med Care. (2012). 50:1076–85. doi: 10.1097/MLR.0b013e318269e06d [published correction appears in Med Care. (2013) 51:378].

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Favato G, Baio G, Capone A, Marcellusi A, Saverio Mennini F, A. novel method to value real options in health care: the case of a multicohort human papillomavirus vaccination strategy. Clin Ther. (2013) 35:904–14. doi: 10.1016/j.clinthera.2013.05.003

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Herlihy N, Hutubessy R, Jit M. Current global pricing for human papillomavirus vaccines brings the greatest economic benefits to rich countries. Health Aff (Millwood). (2016) 35:227–34. doi: 10.1377/hlthaff.2015.1411

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Hoshi SL, Seposo X, Okubo I, Kondo M. Cost-effectiveness analysis of pertussis vaccination during pregnancy in Japan. Vaccine. (2018) 36:5133–40. doi: 10.1016/j.vaccine.2018.07.026

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Hoshi SL, Kondo M, Okubo I. Economic evaluation of vaccination programme of 13-valent pneumococcal conjugate vaccine to the birth cohort in Japan. Vaccine. (2013) 31:2762–71. doi: 10.1016/j.vaccine.2013.03.052

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Jiang Y, Gervais F, Gauthier A, Baptiste C, Martinon P, Bresse X, et al. comparative public health and budget impact analysis of pneumococcal vaccines: the French case. Hum Vaccin Immunother. (2015) 11:2188–97. doi: 10.1080/21645515.2015.1011957

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Jiménez E, Torkilseng EB, Klemp M. Cost-Effectiveness of HPV-Vaccination of Boys Aged 12 in a Norwegian Setting. Oslo: Knowledge Centre for the Health Services at The Norwegian Institute of Public Health (NIPH). (2015).

PubMed Abstract | Google Scholar

47. Kotirum S, Vutipongsatorn N, Kongpakwattana K, Hutubessy R, Chaiyakunapruk N. Global economic evaluations of rotavirus vaccines: a systematic review. Vaccine. (2017) 35:3364–86. doi: 10.1016/j.vaccine.2017.04.051

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Lee BY, Bacon KM, Donohue JM, Wiringa AE, Bailey RR, Zimmerman RK. From the patient perspective: the economic value of seasonal and H1N1 influenza vaccination. Vaccine. (2011) 29:2149–58. doi: 10.1016/j.vaccine.2010.12.078

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Lee BY, Bartsch SM, Brown ST, Cooley P, Wheaton WD, Zimmerman RK. Quantifying the economic value and quality of life impact of earlier influenza vaccination. Med Care. (2015) 53:218–29. doi: 10.1097/MLR.0000000000000302

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Lee BY, Bartsch SM, Willig AM. The economic value of a quadrivalent versus trivalent influenza vaccine. Vaccine. (2012). 30:7443–6. doi: 10.1016/j.vaccine.2012.10.025 [published correction appears in Vaccine. (2013) 31:2477–9].

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Lee BY, Tai JH, McGlone SM, Bailey RR, Wateska AR, Zimmer SM, et al. The potential economic value of a 'universal' (multi-year) influenza vaccine. Influenza Other Respir Viruses. (2012) 6:167–75. doi: 10.1111/j.1750-2659.2011.00288.x

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Lee BY, Shah M. Prevention of influenza in healthy children. Expert Rev Anti Infect Ther. (2012) 10:1139–52. doi: 10.1586/eri.12.106

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Leidner AJ, Murthy N, Chesson HW, Biggerstaff M, Stoecker C, Harris AM, et al. Cost-effectiveness of adult vaccinations: a systematic review. Vaccine. (2019) 37:226–34. doi: 10.1016/j.vaccine.2018.11.056

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Ozawa S, Portnoy A, Getaneh H, Clark S, Knoll M, Bishai D, et al. Modeling the economic burden of adult vaccine-preventable diseases in the United States. Health Aff (Millwood). (2016) 35:2124–32. doi: 10.1377/hlthaff.2016.0462

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Park M, Jit M, Wu JT. Cost-benefit analysis of vaccination: a comparative analysis of eight approaches for valuing changes to mortality and morbidity risks. BMC Med. (2018). 16:139. doi: 10.1186/s12916-018-1130-7

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Philipson TJ, Thornton Snider J, Chit A, Green S, Hosbach P, Tinkham Schwartz T, et al. The social value of childhood vaccination in the United States. Am J Manag Care. (2017) 23:41–7.

PubMed Abstract | Google Scholar

57. Postma MJ, Carroll S, Brandão A. The societal role of lifelong vaccination. J Mark Access Health Policy. (2015). 3:10.3402/jmahp.v3.26962. doi: 10.3402/jmahp.v3.26962

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Preaud E, Durand L, Macabeo B, Farkas N, Sloesen B, Palache A, et al. Annual public health and economic benefits of seasonal influenza vaccination: a European estimate. BMC Public Health. (2014). 14:813. doi: 10.1186/1471-2458-14-813

PubMed Abstract | CrossRef Full Text

59. Reyes JF, Wood JG, Beutels P, Macartney K, McIntyre P, Menzies R, et al. Beyond expectations: post-implementation data shows rotavirus vaccination is likely cost-saving in Australia. Vaccine. (2017) 35:345–52. doi: 10.1016/j.vaccine.2016.11.056

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Rheingans R, Amaya M, Anderson JD, Chakraborty P, Atem J. Systematic review of the economic value of diarrheal vaccines. Hum Vaccin Immunother. (2014) 10:1582–94. doi: 10.4161/hv.29352

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Tully SP, Anonychuk AM, Sanchez DM, Galvani AP, Bauch CT. Time for change? An economic evaluation of integrated cervical screening and HPV immunization programs in Canada. Vaccine. (2012) 30:425–35. doi: 10.1016/j.vaccine.2011.10.067

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Waye A, Chuck AW. Value added by the Prevnar 13 childhood immunization program in Alberta, Canada (2010–2015). Drugs Real World Outcomes. (2015) 2:311–8. doi: 10.1007/s40801-015-0037-2

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Asgary A. Assessing households' willingness to pay for an immediate pandemic influenza vaccination programme. Scand J Public Health. (2012) 40:412–7. doi: 10.1177/1403494812453884

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Timmis JK, Black S, Rappuoli R. Improving accountability in vaccine decision-making. Expert Rev Vaccines. (2017) 16:1057–66. doi: 10.1080/14760584.2017.1382358

PubMed Abstract | CrossRef Full Text | Google Scholar

65. World Health Organization. Immunization Agenda 2030: A Global Strategy to Leave No One Behind. (2019). 1–29.

66. National National Academies of Sciences Engineering and Medicine. (2020). Framework for Equitable Allocation of COVID-19 Vaccine. Washington, DC: The National Academies Press. doi: 10.17226/25917

PubMed Abstract | CrossRef Full Text | Google Scholar

67. Bloom DE, Black S, Salisbury D, Rappuoli R. Antimicrobial resistance and the role of vaccines. Proc Natl Acad Sci USA. (2018) 115:12868–71. doi: 10.1073/pnas.1717157115

PubMed Abstract | CrossRef Full Text | Google Scholar

68. Bloom DE, Brenzel L, Cadarette D, Sullivan J. Moving beyond traditional valuation of vaccination: needs and opportunities. Vaccine. (2017) 35(Suppl 1):A29–35. doi: 10.1016/j.vaccine.2016.12.001

PubMed Abstract | CrossRef Full Text | Google Scholar

69. de Waure C, Calabrò GE, Ricciardi W, on behalf of the Value(s) of Vaccination Project Steering Committee. Recommendations to drive a value-based decision-making on vaccination. Expert Rev Vaccines. (2022) 2022:1–8. doi: 10.1080/14760584.2022.2021880

PubMed Abstract | CrossRef Full Text | Google Scholar

70. The economic value of vaccination: why prevention is wealth. J Mark Access Health Policy. (2015). 3:10.3402/jmahp.v3.29414. doi: 10.3402/jmahp.v3.29414

PubMed Abstract | CrossRef Full Text

71. Largeron N, Lévy P, Wasem J, Bresse X. Role of vaccination in the sustainability of healthcare systems. J Mark Access Health Policy. (2015). 3:10.3402/jmahp.v3.27043. doi: 10.3402/jmahp.v3.27043

PubMed Abstract | CrossRef Full Text | Google Scholar