Sharing lessons learned from COVID-19 vaccine introductions: a global community forum for countries

To optimize the efficient introduction and deployment of COVID-19 vaccines across the globe during the COVID-19 pandemic, in April 2021 WHO launched a new process and tools for countries to rapidly review the early phase of countries’ COVID-19 vaccine introduction. This methodology is called the COVID-19 vaccination intra-action review, also known as mini COVID-19 vaccine post-introduction evaluation (mini-cPIE). As of November 2022, 46 mini-cPIEs had been conducted. In collaboration with Project ECHO, WHO convened and facilitated real-time experience sharing and peer-learning among countries following their mini-cPIEs through a virtual global real-time learning forum. This five-session clinic series was attended by 736 participants from 129 countries. Based on post-session feedback surveys, when asked about the utility of the sessions, half of the participants said that sessions led them to review national guidelines and protocols or make other changes to their health systems. The post-series survey sent following the end of the clinic series showed that at least eight countries subsequently conducted a mini-cPIE after participating in the clinics, and participants from at least nine countries indicated the experience shared by peer countries on the clinic largely benefited their COVID-19 vaccine introduction and deployment. In this article, we highlight the benefits and importance of creating a global experience-sharing forum for countries to connect and share pertinent learnings in real-time during an international public health emergency. Moving forward, it is critical to foster a culture of individual and collective learning within and between countries during public health emergencies, with WHO playing an important convening role.

Introduction

The World Health Organization (WHO) recommends that countries conduct a post-introduction evaluation (PIE) 6–12 months following the introduction of a new vaccine (1) to help countries evaluate implementation and take corrective actions. A PIE aims to fully assess the progress of vaccine introduction and can be time and resource-intensive requiring in-person field visits at all levels of the immunization system (2, 3). The COVID-19 pandemic, however, necessitated the rapid vaccination of the entire global population (4). Countries had little time to conduct a full programme evaluation such as a PIE, especially during the early phases of COVID-19 vaccine introduction and deployment (5).

To address this need, WHO developed a new evaluation method by adapting the WHO COVID-19 intra-action review (IAR) for countries to conduct a simpler, shorter and more flexible version of the PIE process, coined as mini-cPIE (6–8). The mini-cPIE was developed in line with the National Deployment and Vaccination Plan for COVID-19 vaccines (NDVP) (9), COVID-19 vaccine post-introduction evaluation (cPIE) and COVID-19 Strategic Preparedness and Response Plan (SPRP) (10). The WHO guidance for conducting a COVID-19 IAR (11) was developed soon after the pandemic onset for key stakeholders to collectively identify practices and lessons learned to course-correct and improve the ongoing response (12). During the IAR, a facilitator guides group discussions, in-person or virtual, over a period of 2–3 days using trigger questions pre-selected to address country needs. Based on the findings, the stakeholders then collectively recommend short-and mid-to long-term follow-up activities.

As countries and partners began using the mini-cPIE, they were eager to learn about ongoing vaccination implementation challenges and emerging best practices to improve their own processes and coverage. With this goal in mind, WHO collaborated with Project ECHO (Extension for Community Healthcare Outcomes) to create a global virtual community of practice (vCoP) called the mini-cPIE clinic, consisting of virtual learning sessions for countries to share and learn from experiences in real-time with immunization peers. These were referred to as “clinics” henceforth. During clinics, countries that had recently conducted a mini-cPIE were invited to present their learnings and best practices to peers that were planning or considering conducting a mini-cPIE. This was followed by Q&A/discussion that allowed for direct connection between countries.

Five clinic sessions were conducted from July to December 2021 (13). Clinic design was informed both by a past WHO-ECHO vCOP series on broader COVID-19 immunization, and by Project ECHO’s 20 years of experience implementing the ECHO collaborative-learning model. This model supports low-dose, high-frequency, virtual case-based learning to promote timely peer-to-peer information sharing and collaborative problem solving. This learning approach has proved to be instrumental in improving health outcomes during COVID-19, in other public health emergency responses and as related to general continuing health professional education (14–19).

The objectives of the clinics were threefold: (1) to create a forum for countries having conducted a mini-cPIE to share their experiences and learnings with stakeholders and peer countries; (2) to cultivate a community of learning and collaborative problem solving for COVID-19 vaccine introduction; and (3) to provide opportunities for countries interested in conducting a mini-cPIE to receive practical tips and technical guidance. The intended audience of the clinics were country immunization representatives interested in conducting, planning to conduct, or who had conducted a mini-cPIE, as well as partners technically or financially supporting these activities.

In this article, we describe the clinic design and participants’ characteristics, challenges, lessons, and best practices shared by presenting countries, and the perceived value of the clinics among participants.

Methods

Clinic design, content, and promotion

Between July and December 2021, WHO hosted a series of monthly, 90-min clinics on the Zoom videoconferencing platform. Clinics began with two to three country presentations followed by facilitated discussion, presentations from experts on immunization topics of interest, questions and answers, and interactive polls for real-time feedback from participants. The clinics were designed to be interactive, with participating countries able to directly engage with and ask questions to presenting countries. All questions that were not addressed during the clinic due to time constraints were compiled and shared with all participants and posted on the TechNet-21 immunization community collaboration platform (13). To maximize accessibility and promote multilingualism, simultaneous interpretation was provided to and from English, French, Spanish, Russian, Arabic and Portuguese. To further encourage participant engagement, a Telegram instant messaging group was established to offer participants an additional opportunity to continue discussions and experience sharing asynchronously outside the clinics.

Over the course of the six-month series, the Ministries of Health of 10 countries from four WHO regions, the African (AFR), American (AMR), Eastern Mediterranean (EMR) and South-East Asia (SEAR) regions, accepted the invitation to virtually present on their experiences with recently conducted mini-cPIEs (Table 1). The first two clinics addressed the conduct of and general learnings from mini-cPIEs -- including successes, challenges, and lessons learned -- and covered all immunization programme areas defined in the COVID-19 National Deployment Vaccination Plan (NDVP). The format of the clinics evolved through the series based on participant feedback and immunization topics that arose during that period of the pandemic. For the final three clinics, the presenting countries were asked to focus on specific immunization programme themes pertinent to challenges observed during the COVID-19 vaccine introduction and deployment (Table 1).

Table 1

Table 1. Overview of the virtual mini-cPIE clinic series hosted by WHO and ECHO project, July–December 2021.

Advocacy and outreach efforts were made to ensure the intended target audiences were aware of scheduled clinic sessions. These included engaging with WHO regional and country focal points for both immunization and health emergencies so that they would share information about upcoming clinics with relevant country-level counterparts. Participants were also invited to join the sessions relevant to their interests through email announcements, social media, and Project ECHO and TechNet-21 platforms. Pre-registration was required for attendance and open to the public. After each session, Project ECHO distributed session materials (i.e., recordings, presentations, and Q&A responses) along with announcements for the subsequent session. Session materials were also posted on the TechNet-21 collaboration platform.

Data collection and analysis

Participant demographic data were collected upon registration via a unique Zoom link, including name, email, gender, area(s) of expertise, professional affiliation, geographic level (e.g., multi-country, national, sub-national), and country. Registration and attendance data from Zoom reports were linked by email address. The first session registration data did not include gender, area(s) of expertise, and region of work. That and a few incomplete responses explain discrepancies in the result’s profile data totals.

A link to an anonymous online post-session survey (Supplementary Appendix 1) was shared with all participants for the third, fourth and fifth clinics, with digital certificates of participation provided to participants who completed the survey. The survey asked participants about their knowledge of the session’s topic before and after; relevance of the session to their current work; balance of lecture and interactivity; and intention to use what was learned in their work. In addition, Project ECHO distributed a post-series survey via email in November 2022 (eleven months after the last session) to all program participants to solicit feedback about the entire clinic series, including qualitative examples of barriers to conducting mini-cPIEs and improvements to vaccine roll-out linked to series learnings (Supplementary Appendix 2). The University of New Mexico Health Sciences Institutional Review Board approved the evaluation (ID 20–469).

Chi-square and Fisher’s exact tests were used to compare results across dichotomized geography (multi-country versus single-country) and country income levels. The Wilcoxon signed-rank test was used to compare knowledge before and after sessions. Quantitative analyses were conducted in R version 2022.07.1. Open-ended text responses were coded systematically using NVivo 1.4.1.

Results

Characteristics of participants

From 28 July 2021 until 14 December 2021, 996 attendances were recorded over five clinics. Among these, 736 unique individuals from 129 countries across all six WHO regions participated in at least one clinic session (Table 2 and Supplementary Figure S1). The number of participants in each session ranged widely from 88 to 241. Most (n = 563; 76.5%) participants attended only one session and 23.5% (n = 173) attended two or more. More participants (n = 436; 59.6%) described themselves as working in a single country versus multiple countries (n = 296; 40.4%) (i.e., working at the regional or global level). More than half (59.2%) of participants were in low or lower-middle income countries. The most common areas of expertise among attendees were immunization (n = 244; 37.7%) followed by emergency preparedness (n = 149; 23.0%) and program management (n = 146; 22.5%) (Table 2). Participants attended an average of 1.4 clinics each. Participants representing a single country were less likely to attend multiple sessions than those working at the regional or global level (Table 2).

Table 2

Table 2. Demographics of participants who attended the virtual mini-cPIE clinic series hosted by WHO and ECHO project, July–December 2021.

Lessons learned and best practices shared by countries

Country case studies were presented by either government representatives or WHO immunization experts from Bhutan, The Plurinational State of Bolivia, Democratic Republic of Congo (DRC), Gambia, Ghana, Mozambique, Senegal, South Sudan, Somalia, and Uganda. Speakers shared the importance of strong political commitment to mobilize national resources, bilateral negotiations for donations to increase access to vaccines, and multi-sector engagement to mobilize human resources. Strong inter-sectoral collaboration contributed to successes in every thematic area. Leading by example, political and community leaders were able to build trust and improve vaccine uptake.

Immunization experts from countries highlighted their successes with key innovations. Bhutan described how the Prime Minister boosted public confidence by receiving the first and second doses of a heterologous regimen. Bhutan achieved 95% vaccine coverage for the first dose and more than 90% vaccine coverage for the second dose following the first national vaccination campaigns. The Gambia described using a “vaccine caravan” to improve vaccine uptake and engage remote communities. Ghana used an innovative yet cost-effective approach for authentication of vaccination status with metallic holograms on vaccination cards, as well as utilizing drones to deliver vaccines to hard-to-reach areas.

Challenges and proposed solutions were shared together, although sometimes these solutions highlighted additional difficulties. For example, Senegal successfully established a COVID-19 Adverse Event Following Immunization (AEFI) committee and investigated all serious AEFIs; however, with this success came the challenge of how the country was addressing the lack of free medical care for those experiencing serious AEFIs. Uganda addressed delays in deployment of funds to the operational level by emphasizing early and transparent communication with health workers to instill confidence and allow continued vaccination services during resolution of administrative problems. Somalia described challenges in the availability and distribution of doses due to short expiry dates and described forecasting tools used to improve vaccine management and distribution. Bolivia worked to address coverage disparities through review of uptake data disaggregated by geographic area, target population, age, and gender. In low uptake populations, Bolivia prioritized single-dose regimens to minimize the risk of drop-out and reduce vaccine access inequity.

Strong risk communication and social listening activities at subnational and local levels were described by several Ministries of Health as essential to successful COVID-19 vaccine deployment. South Sudan dispelled rumors by acting on behavior survey findings through media engagement, high-level advocacy meetings, radio programmes and jingles, especially when new variants of concern emerged and created a loss of public confidence in COVID-19 vaccines. In the session focusing on unique risk communication and community engagement approaches, the DRC described strategies to build confidence among high-risk populations through strong interpersonal communication of health workers, pre-registration and monitoring of vaccination status to allow health worker follow-up and engagement. Mozambique used multiple communication channels to create public demand for vaccines and regularly monitored and managed rumors through a technical working group and digital platform.

Poll responses

During the clinics, polls gathered real-time feedback from attendees. Examples of polling questions used were, “What topics/themes related to COVID-19 vaccine roll-out would you like to see discussed in future mini-cPIE clinics?” and “In a few words, what risk communication and community engagement strategies have you seen that worked well to increase COVID-19 vaccine uptake?.” The polls, whose answers were visible to all, were successful in enabling quick sharing of perspectives from multiple country or partner representatives from the audience and stimulated further discussion on new topics (Supplementary Figure S2).

Post-clinic feedback surveys

The response rate for post-session surveys was 11.0% (n = 117). In post-session surveys, respondents rated their post-session knowledge significantly higher than before the session (Supplementary Figure S3) (p < 0.001). Almost all survey respondents (n = 110; 94.0%) reported the “right” balance of didactic and interactive learning. Additionally, most (n = 110; 94.0%) would definitely or probably recommend the session to a colleague and 80.2% reported the session was extremely or very relevant to their work. Most respondents would “definitely” (n = 94; 80.3%) or “probably” (n = 17; 14.5%) use information from the session in their respective work. The most common way participants planned to use the information was to share with colleagues (n = 81; 69.2%) followed by general use (n = 73; 62.4%), looking up additional information (n = 63; 53.8%), and making guidelines, protocols, or other changes to health systems (n = 60; 51.3%). Almost one-third (n = 37; 31.6%) planned to change how they worked with patients or community members.

Qualitative comments from post-session surveys reinforced the quantitative findings (Table 3). Respondents found the knowledge and information shared in each session useful and noted plans to share with colleagues or stakeholders as well as using them to develop communication strategies for advocacy or community outreach. One respondent reported, “I liked the ideas of evidence-based communication, analysis of community feedback to craft communication products that respond to community inquiries, etc.” while another said, “The information shared in the session increased my understanding of challenges in other contexts which also apply to the African region.” Answers from respondents helped inform priority COVID-19 topics for future sessions, including vaccines, booster dose policies, vaccine misinformation and hesitancy, financing, and metrics. Respondents also appreciated peer-sharing: “It was a great cross learning opportunity. The challenges [encountered] in [fragile, conflict-affected, and vulnerable] settings are similar but the way these are addressed is the key learning.”

Table 3

Table 3. Feedback from participants to the open-ended questions of the post-session survey following the virtual mini-cPIE clinic sessions hosted by WHO and ECHO project, November 2022.

Post-series survey

The response rate for the post-series survey was 5.4% (n = 40) (Table 4). Most respondents primarily worked in a single country characterized as low or low-middle income (n = 19; 63.3%). Fifteen respondents (37.5%) from eight countries mentioned that their countries began planning or conducting a mini-cPIE due to their participation in the clinic series. For 66.7% of those respondents (n = 10), participation in the series significantly helped inform planning, implementation, and follow-up. Planning or implementation of a mini-cPIE was more likely to be reported by low income or lower-middle income country respondents compared to high income or upper-middle income countries (p < 0.01; Supplementary Figure S4).

Table 4

Table 4. Characteristics of post-series survey respondents and reported impact of the virtual mini-cPIE clinic series hosted by WHO and ECHO project, November 2022.

Nearly three quarters of respondents (72.5%, n = 29) affirmed that their participation in the clinic series had improved their countries’ COVID-19 vaccine introduction and deployment a lot or some amount (Table 4). The improvements shared by the respondents varied but referred to specific strategies learned during the clinics. For example, “More vaccination posts were created to support community access and decongest health facilities. In addition, vaccination campaigns to de-stigmatize vaccines were held across the country physically and through media.” Additionally, respondents mentioned adjustments to their national vaccination plans, new strategies for prioritization of high-risk populations, strengthening vaccination data management, and long-term integration of COVID-19 vaccination efforts within the national immunization and health system. Seven respondents shared barriers to using learnings from the series including immunization decision-making being outside their scope of work or scheduling conflicts to participate in more sessions.

Discussion

WHO, recognizing the importance and urgency of real-time peer-learning, leveraged its global coordinating role to rapidly convene countries and partners in the clinics described in this article. WHO, in collaboration with Project ECHO, created a global virtual learning forum for countries to highlight and share with peer countries their respective challenges, solutions, and innovations experienced during their COVID-19 vaccine introduction and deployment. While Project ECHO had prior experience establishing virtual communities of practice related to clinical medicine topics (16–18), this method for real-time peer-to-peer virtual learning was first used for the global immunization community during the pandemic. At the start of the COVID-19 pandemic prior to the mini-cPIE clinic series, WHO and Project ECHO collaborated on the ACT Accelerator COVID-19 initiatives (19), and a second series that convened partners and practitioners to build global capacity in preparation for COVID-19 vaccination (14). Several studies have shown that virtual training can help equip healthcare professionals with knowledge and skills for effective vaccination deployment during critical times of the COVID-19 pandemic (20, 21). The mini-cPIE clinic series was novel as it not only showcased the use of the intra-action review process recommended by the International Health Regulations Emergency Committee (22) early in the pandemic, but it also leveraged the virtual multi-country platform for case-based learning including country-level case presentations and participation from national public health practitioners.

Use of a virtual platform was essential given pandemic restrictions on in-person convening but also greatly reduced costs for convening large audiences. The establishment of the virtual platform also enabled WHO to continually advocate on behalf of Member States for additional technical and financial support from partners and donors. Participants who responded to post-session surveys found the clinics valuable. Survey responses show the value of clinics for participants in several ways, including a self-reported significant increase in knowledge, a high likelihood of endorsing the clinics, and plans to use the newly acquired knowledge. Evidence from the post-series survey reinforces this value, particularly for low and middle-income countries, by supporting the planning or conducting a mini-cPIE. Perceived value is also reflected in the suggestions for improvement, including requests for more sessions, more topics, and more time for discussion and interaction. The fact that most attendees attended only one and not multiple sessions suggests that either attendees were interested in a specific country presentation or a vaccination topic, or they were busy and could only select specific sessions to attend. Some participants may have taken advantage of the option to watch recordings of the sessions in addition to attending live.

Participant feedback was used to adapt clinic content to be most relevant and useful. For example, one of the clinics was dedicated to COVID-19 vaccine introduction and deployment in Fragile States/humanitarian contexts (13). The importance of strong risk communication strategies and community engagement was mentioned in multiple clinics, and given common challenges in this area, participants also expressed interest in having a focused session on the topic. This became the theme for session four (13), where presenters and participants shared regional and context-specific challenges and solutions. Global expert and audience contributions touched on key learnings that have since been documented in the literature (23, 24) including the importance of just-in-time risk-communication training and evidence-based community engagement approaches that rely on understanding the behavioral and social drivers to increase uptake and decrease hesitancy (25–28).

During the clinics, Q&A sessions generated considerable interest among participants, especially in the use of innovative approaches to increase vaccine coverage. Although it is encouraging to learn of the many emerging innovations borne of the COVID-19 crisis, best practice still requires that innovations be piloted and evaluated before scale-up, to assure that the innovative initiatives are beneficial, cost-effective, locally appropriate and sustainable. Equally, documentation of the impact of different approaches used by countries has been important to inform and optimize their use in the longer term for COVID-19 vaccine deployment, for other emergency vaccine responses or for routine vaccination programs (29).

In addition to innovation, countries described and discussed how best practice vaccination strategies from established childhood (e.g., polio, measles) and adult (e.g., influenza) immunization programmes were used to improve COVID-19 vaccination uptake. For example, strong outreach or mobile immunization sessions were described as crucial in many settings to reach priority populations. This finding aligns with literature documenting the use of outreach and mobile vaccination units to promote equity and reach underserved populations both prior to (30, 31) and during the pandemic (32–34). Moreover, enhanced engagement with private sector health providers expanded access to non-traditional or new vaccination sites. Strategic private sector engagement has been described elsewhere as an effective strategy for mass health emergency vaccination (35). During the pandemic, working collaboratively with different sectors in a whole-of-society approach was also highlighted in the global analysis of IARs conducted by WHO (36).

Established practice for one country might be innovation for another. Countries shared the mosaic of digital strategies reported subjectively to improve aspects of their vaccination programs: virtual training platforms; digital data management tools for tracking vaccine supply and distribution; electronic vaccination registries; electronic vaccination cards with sophisticated methods for authentication; digital platforms and social media for social mobilization, monitoring rumors, and infodemic management. Peer-to-peer learning on these practices allowed clinic participants the opportunity to seek support from country representatives or global partners to develop in-country capacity for the real-world implementation of evidenced-based strategies for digital applications across the immunization system from vaccine supply to delivery to monitoring to communications and community engagement. Evidence was beginning to accrue on use of these strategies prior to the pandemic (37, 38) but has expanded greatly with the pressures of the COVID-19 vaccine rollout (39, 40).

The development of this COVID-19 vaccination IAR (mini-cPIE) toolkit and the clinics initiated a new and collaborative process between the Health Security Preparedness Department and the Department of Immunization, Vaccines and Biologicals at WHO. The development process provided opportunities for relevant stakeholders and end-users at all organizational levels (global, regional, country) to provide feedback. WHO’s collaboration with Project ECHO’s academic team leveraged their global vCOP and interactive webinar production experience and added a new perspective for achieving the right balance of information sharing and peer-to-peer learning on global COVID-19 immunization. WHO often encourages countries to adopt a multi-sectoral, whole-of-government and whole-of-society approach to emergency preparedness and response. In this clinic series, WHO also followed this model from the design to the implementation.

Our analysis is limited by the fact that the response rates to the post-session and the post-series surveys were relatively low and cannot be generalized to all participants. For the post-series survey, this is likely related to the delay of almost a year after the completion of the clinics before the survey was conducted. It is possible that participants most likely to respond were those who experienced the greatest benefit. Regardless, the feedback provided by respondents indicates that the series may have prompted at least eight countries to conduct mini-cPIEs and many more felt the material shared led to positive change in practice.

Conclusion and future directions

The creation of this virtual global real-time learning forum allowed countries to connect and share their COVID-19 vaccine introduction and deployment challenges and successes during an acute global emergency. A similar approach can be used to share pertinent topics beyond vaccination in real-time in future public health emergencies. The mini-cPIE clinics have also served as a springboard for a subsequent community of learning addressing lessons learned on COVID-19 immunization practices beyond the mini-cPIE (41). Moving forward, it is critical to foster a culture of individual and collective learning within and between countries during a public health emergency to help accelerate the sharing of new knowledge, promising practices, and real-world implementation of evidenced based practices, with global health agencies such as WHO playing an important facilitating role.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author contributions

JW: Conceptualization, Data curation, Investigation, Methodology, Supervision, Writing – original draft, Writing – review & editing, Resources. CC: Conceptualization, Data curation, Investigation, Methodology, Writing – original draft, Writing – review & editing. AB: Data curation, Formal analysis, Methodology, Visualization, Writing – original draft, Writing – review & editing. BS: Conceptualization, Methodology, Project administration, Resources, Supervision, Writing – review & editing. AG: Data curation, Writing – original draft, Writing – review & editing. LB: Methodology, Project administration, Writing – review & editing. LM: Conceptualization, Funding acquisition, Project administration, Resources, Writing – review & editing. DC: Conceptualization, Resources, Supervision, Writing – review & editing. LV: Conceptualization, Funding acquisition, Project administration, Resources, Supervision, Writing – review & editing.

Funding

The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.

Acknowledgments

We would like to thank all of the session presenters who took the time to contribute during an extremely busy pandemic response. We also thank Shoshanna Goldin for her guidance for establishing the series, Carole Tevi Benissan, Armand Ngassa Mbanya, Denis Charles, Candice Vente and Hilary Kagume Njenge for their support and contributions in the planning and coordination of the Mini-cPIE clinics, Alexandre Pascutto for his assistance in making resources available on Technet-21 platform and in setting up the Telegram SMS group, Sonya Bedge and Iiris Rinne for providing IT support during the clinics, and Stella Chungong, Ann Lindstrand, and Ann Moen, for their leadership, vision and support to make this mini-cPIE clinic series possible.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2024.1376113/full#supplementary-material

References

1. World Health Organization. New vaccine post-introduction evaluation (PIE) tool. (2010). Available at: https://www.who.int/immunization/monitoring_surveillance/resources/PIE_tool/en/ (Accessed December 12, 2022).

Google Scholar

2. Wang, SA, Hyde, TB, Mounier-Jack, S, Brenzel, L, Favin, M, Gordon, WS, et al. New vaccine introductions: assessing the impact and the opportunities for immunization and health systems strengthening. Vaccine. (2013) 31:B122–8. doi: 10.1016/j.vaccine.2012.10.116

Crossref Full Text | Google Scholar

3. World Health Organization. COVID-19 vaccine post-introduction evaluation (cPIE) guide, revised 10 October 2023. (2023). Available from: https://www.who.int/publications/i/item/WHO-2019-nCoV-cPIE-2023.1 (Accessed October 10, 2023).

Google Scholar

4. World Health Organization. WHO SAGE values framework for the allocation and prioritization of COVID-19 vaccination. (2020). Available at: https://www.who.int/publications/i/item/who-sage-values-framework-for-the-allocation-and-prioritization-of-covid-19-vaccination

Google Scholar

5. Masresha, B, Ruiz, MAS, Atuhebwe, P, and Mihigo, R. The first year of COVID-19 vaccine roll-out in Africa: challenges and lessons learned. Pan Afr Med J. (2022) 41:2. doi: 10.11604/pamj.supp.2022.41.2.33686

PubMed Abstract | Crossref Full Text | Google Scholar

6. Copper, FA, Chiu de Vazquez, C, Bell, A, Mayigane, LN, Vedrasco, L, and Chungong, S. Preparing for COVID-19 vaccine roll-out through simulation exercises. The lancet. Glob Health. (2021) 9:e742–3. doi: 10.1016/S2214-109X(21)00051-6

Crossref Full Text | Google Scholar

7. World Health Organization. Guidance for conducting a country COVID-19 intra-action review (IAR): addendum 1 - 28 April 2021. (2021). Available at: https://www.who.int/publications/i/item/WHO-2019-nCoV-Country-IAR-add.1-2021.1 (Accessed December 12, 2022).

Google Scholar

8. World Health Organization. Evaluation of COVID-19 vaccine introduction – Mini cPIE. (2021). Available at: https://www.who.int/publications/i/item/new-vaccine-post-introduction-evaluation-tool (Accessed December 12, 2022).

Google Scholar

9. World Health Organization. Guidance on developing a national deployment and vaccination plan for COVID-19 vaccines. (2020). Available at: https://www.who.int/publications/i/item/WHO-2019-nCoV-Vaccine_deployment-2020.1 (Accessed December 12, 2022).

Google Scholar

10. World Health Organization. COVID-19 strategic preparedness and response plan (SPRP 2021). (2021). Available at: https://www.who.int/publications/i/item/WHO-WHE-2021.02 (Accessed December 12, 2022).

Google Scholar

11. World Health Organization. Guidance for conducting a country COVID-19 intra-action review (IAR). (2020). Available at: https://www.who.int/publications/i/item/WHO-2019-nCoV-Country_IAR-2020.1 (Accessed December 12, 2022).

Google Scholar

12. Mayigane, LN, Chiu de Vazquez, C, Vente, C, Charles, D, Copper, FA, Bell, A, et al. The necessity for intra-action reviews during the COVID-19 pandemic. Lancet Glob Health. (2020) 8:e1451–2. doi: 10.1016/S2214-109X(20)30414-9

PubMed Abstract | Crossref Full Text | Google Scholar

13. Walldorf, J. Mini-cPIE (COVID-19 vaccination IAR) clinics: experience sharing and lessons learned. (2021). Available at: https://www.technet-21.org/en/hot-topics-items/15268-mini-cpie-covid-19-vaccination-iar-clinics-experience-sharing-and-lessons-learned (Accessed December 12, 2022).

Google Scholar

14. Goldin, S, Hood, N, Pascutto, A, Bennett, C, Barbosa de Lima, AC, Devereaux, N, et al. Building global capacity for COVID-19 vaccination through interactive virtual learning. Hum Resour Health. (2022) 20:16. doi: 10.1186/s12960-022-00710-7

Crossref Full Text | Google Scholar

15. Gleason, LJ, Beiting, KJ, Walker, J, Shervani, S, Graupner, J, Mittal, K, et al. Using Telementoring to share best practices on COVID-19 in post-acute and long-term care facilities. J Am Geriatr Soc. (2020) 68:E58–60. doi: 10.1111/jgs.16840

PubMed Abstract | Crossref Full Text | Google Scholar

16. De Witt, JB, Brazil, K, Passmore, P, Buchanan, H, Maxwell, D, McIlfatrick, SJ, et al. Evaluation of the impact of telementoring using ECHO(c) technology on healthcare professionals’ knowledge and self-efficacy in assessing and managing pain for people with advanced dementia nearing the end of life. BMC Health Serv Res. (2018) 18:228. doi: 10.1186/s12913-018-3032-y

Crossref Full Text | Google Scholar

17. Arora, S, Thornton, K, Murata, G, Deming, P, Kalishman, S, Dion, D, et al. Outcomes of treatment for hepatitis C virus infection by primary care providers. N Engl J Med. (2011) 364:2199–207. doi: 10.1056/NEJMoa1009370

PubMed Abstract | Crossref Full Text | Google Scholar

18. Struminger, B, Arora, S, Zalud-Cerrato, S, Lowrance, D, and Ellerbrock, T. Building virtual communities of practice for health. Lancet. (2017) 390:632–4. doi: 10.1016/S0140-6736(17)31666-5

PubMed Abstract | Crossref Full Text | Google Scholar

19. Wilson, K, Dennison, C, Struminger, B, Armistad, A, Osuka, H, Montoya, E, et al. Building a virtual global knowledge network during the coronavirus disease 2019 pandemic: the infection prevention and control global webinar series. Clin Infect Dis. (2021) 73:S98–S105. doi: 10.1093/cid/ciab320

Crossref Full Text | Google Scholar

20. Goldin, S, Kong, S, Tokar, A, Utunen, H, Ndiaye, N, Bahl, J, et al. Learning from a massive open online COVID-19 vaccination training experience: survey study. JMIR Public Health Surveill. (2021) 7:e33455. doi: 10.2196/33455

PubMed Abstract | Crossref Full Text | Google Scholar

21. Shirahmadi, S, Hazavehei, SMM, Abbasi, H, Otogara, M, Etesamifard, T, Roshanaei, G, et al. Effectiveness of online practical education on vaccination training in the students of bachelor programs during the COVID-19 pandemic. PLoS One. (2023) 18:e0280312. doi: 10.1371/journal.pone.0280312

PubMed Abstract | Crossref Full Text | Google Scholar

22. World Health Organization. Statement on the fourth meeting of the international health regulations (2005) emergency committee regarding the outbreak of coronavirus disease (COVID-19). (2020). Available at: https://www.who.int/news/item/01-08-2020-statement-on-the-fourth-meeting-of-the-international-health-regulations-(2005)-emergency-committee-regarding-the-outbreak-of-coronavirus-disease-(covid-19) (Accessed December 12, 2022).

Google Scholar

23. Kamruzzaman, M, Rahman, A, Reidpath, DD, and Akhter, S. Risk communication and community engagement in the context of COVID-19 response in Bangladesh: a qualitative study. Front Public Health. (2024) 11:1267446. doi: 10.3389/fpubh.2023.1267446

Crossref Full Text | Google Scholar

24. Ditekemena, JD, Nkamba, DM, Mutwadi, A, Mavoko, HM, Siewe Fodjo, JN, Luhata, C, et al. COVID-19 vaccine acceptance in the Democratic Republic of Congo: a cross-sectional survey. Vaccines. (2021) 9:153. doi: 10.3390/vaccines9020153

Crossref Full Text | Google Scholar

25. Attwell, K, Betsch, C, Dubé, E, Sivelä, J, Gagneur, A, Suggs, LS, et al. Increasing vaccine acceptance using evidence-based approaches and policies: insights from research on behavioural and social determinants presented at the 7th annual vaccine acceptance meeting. Int J Infect Dis. (2021) 105:188–93. doi: 10.1016/j.ijid.2021.02.007

PubMed Abstract | Crossref Full Text | Google Scholar

26. Bergen, N, Kirkby, K, Fuertes, CV, Schlotheuber, A, Menning, L, Mac Feely, S, et al. Global state of education-related inequality in COVID-19 vaccine coverage, structural barriers, vaccine hesitancy, and vaccine refusal: findings from the global COVID-19 trends and impact survey. Lancet Glob Health. (2023) 11:e207–17. doi: 10.1016/S2214-109X(22)00520-4

PubMed Abstract | Crossref Full Text | Google Scholar

27. Finney Rutten, LJ, Zhu, X, Leppin, AL, Ridgeway, JL, Swift, MD, Griffin, JM, et al. Evidence-based strategies for clinical organizations to address COVID-19 vaccine hesitancy. Mayo Clin Proc. (2021) 96:699–707. doi: 10.1016/j.mayocp.2020.12.024

PubMed Abstract | Crossref Full Text | Google Scholar

28. Ottewell, A, Martin, K, and Plescia, M. Evidenced-based approaches states and territories can implement to advance adult immunization uptake during the COVID-19 pandemic and beyond. J Public Health Manag Pract. (2022) 28:99–100. doi: 10.1097/PHH.0000000000001473

Crossref Full Text | Google Scholar

29. Nabia, S, Wonodi, CB, Vilajeliu, A, Sussman, S, Olson, K, Cooke, R, et al. Experiences, enablers, and challenges in service delivery and integration of COVID-19 vaccines: a rapid systematic review. Vaccines. (2023) 11:974. doi: 10.3390/vaccines11050974

PubMed Abstract | Crossref Full Text | Google Scholar

30. Dadari, I, Higgins-Steele, A, Sharkey, A, Charlet, D, Shahabuddin, A, Nandy, R, et al. Pro-equity immunization and health systems strengthening strategies in select Gavi-supported countries. Vaccine. (2021) 39:2434–44. doi: 10.1016/j.vaccine.2021.03.044

Crossref Full Text | Google Scholar

31. Ducharme, J, Correa, GC, Reynolds, HW, Sharkey, AB, Fonner, VA, and Johri, M. Mapping of pro-equity interventions proposed by immunisation programs in Gavi health systems strengthening Grants. Vaccine. (2023) 11:341. doi: 10.3390/vaccines11020341

PubMed Abstract | Crossref Full Text | Google Scholar

32. Leibowitz, A, Livaditis, L, Daftary, G, Pelton-Cairns, L, Regis, C, and Taveras, E. Using mobile clinics to deliver care to difficult-to-reach populations: a COVID-19 practice we should keep. Prev Med Rep. (2021) 24:101551. doi: 10.1016/j.pmedr.2021.101551

PubMed Abstract | Crossref Full Text | Google Scholar

33. Zhang, X, Tulloch, JSP, Knott, S, Allison, R, Parvulescu, P, Buchan, IE, et al. Evaluating the impact of using mobile vaccination units to increase COVID-19 vaccination uptake in Cheshire and Merseyside, UK: a synthetic control analysis. BMJ Open. (2023) 13:e071852. doi: 10.1136/bmjopen-2023-071852

PubMed Abstract | Crossref Full Text | Google Scholar

34. Hopkins, KL, Underwood, T, Iddrisu, I, Woldemeskel, H, Bon, HB, Brouwers, S, et al. Community-based approaches to increase COVID-19 vaccine uptake and demand: lessons learned from four UNICEF-supported interventions. Vaccines. (2023) 11:1180. doi: 10.3390/vaccines11071180

PubMed Abstract | Crossref Full Text | Google Scholar

35. Goldberg, SA, Callaway, D, Resnick-Ault, D, Mandavia, S, Martinez, R, Bass, M, et al. Critical concepts for COVID-19 mass vaccination site operations. Disaster Med Public Health Prep. (2023) 17:e60. doi: 10.1017/dmp.2021.319

PubMed Abstract | Crossref Full Text | Google Scholar

36. World Health Organization. A global analysis of COVID-19 intra-action reviews: reflecting on, adjusting and improving country emergency preparedness and response during a pandemic. (2022). Available at: https://apps.who.int/iris/handle/10665/365488

Google Scholar

37. Odone, A, Ferrari, A, Spagnoli, F, Visciarelli, S, Shefer, A, Pasquarella, C, et al. Effectiveness of interventions that apply new media to improve vaccine uptake and vaccine coverage. Hum Vaccin Immunother. (2015) 11:72–82. doi: 10.4161/hv.34313

PubMed Abstract | Crossref Full Text | Google Scholar

38. Bozzola, E, Spina, G, Tozzi, AE, and Villani, A. Global measles epidemic risk: current perspectives on the growing need for implementing digital communication strategies. Risk Manag Healthcare Policy. (2020) 13:2819–26. doi: 10.2147/RMHP.S201279

PubMed Abstract | Crossref Full Text | Google Scholar

39. Fagherazzi, G, Goetzinger, C, Rashid, M, and Aguayo, G. Huiart L digital health strategies to fight COVID-19 worldwide: challenges, Recommendations, and a Call for Papers. J Med Internet Res. (2020) 22:e19284. doi: 10.2196/19284

PubMed Abstract | Crossref Full Text | Google Scholar

40. Odone, A, Gianfredi, V, Sorbello, S, Capraro, M, Frascella, B, Vigezzi, GP, et al. The use of digital technologies to support vaccination Programmes in Europe: state of the art and best practices from experts’ interviews. Vaccine. (2021) 9:1126. doi: 10.3390/vaccines9101126

PubMed Abstract | Crossref Full Text | Google Scholar

41. Gurung, S. Learning collective Initiative’ in Covid-19 vaccine delivery partnership (2022). Available at: https://www.technet-21.org/en/hot-topics-items/15384-learning-collective-initiative-in-covid-19-vaccine-delivery-partnership (Accessed December 12, 2022).

Google Scholar