Evaluation of a massive open online course for just-in-time training of healthcare workers

Introduction: COVID-19 created a global need for healthcare worker (HCW) training. Initially, mass trainings focused on public health workers and physicians working in intensive care units. However, in resource-constrained settings, nurses and general practitioners provide most patient care, typically lacking the training and equipment to manage critically ill patients. We developed a massive open online course (MOOC) for HCWs in resource-constrained settings aimed at training bedside providers caring for COVID-19 patients. We describe the development, implementation and analysis of this MOOC.

Methods: From May through June 2020, the course was developed by a multi-disciplinary team and launched on two online platforms in July. The 4-hour course comprises 6 video-based modules. Student knowledge was assessed using pre- and post-module quizzes and final exam, while demographics and user experience were evaluated by pre- and post-course surveys and learning platform data.

Results: From July 17th to September 24th, 30,859 students enrolled, 18,818 started, and 7,101 completed the course. Most participants worked in healthcare (78%) and resided in lower middle- (38%) or upper middle- (20%) income countries. Learners from upper middle-income and lower middle-income countries had higher completion rates. Knowledge gains were observed from pre-module to post-module quizzes and a final exam. Afterward, participants reported increased self-efficacy regarding course objectives, a 0.63 mean increase on a 4-point scale (95% CI [0.60,0.66]). Most participants (93%) would recommend the course to others.

Conclusion: This article demonstrates the potential of MOOCs to rapidly provide access to emerging medical knowledge during a public health crisis, particularly for HCWs in high- and middle-income countries.

Introduction

Since the first patients with COVID-19 were reported in Wuhan, China, at the end of 2019, cases have been documented on every continent in the world with current estimates in excess of 766 million cases worldwide (1). While countries spanning the entire income spectrum have been impacted, low- and middle-income countries (LMICs) as well as remote and rural healthcare systems in high-income countries (HICs) have been particularly vulnerable — they are at increased risk of being completely overwhelmed, potentially leading to enormous, yet preventable, loss of life (2). Though many health system elements are required to respond to this health crisis, equipping the world’s 60 million healthcare workers with the appropriate knowledge and skills to care for patients with COVID-19 infections is one critical component. Evidence suggests that the early case fatality rate for infected patients quickly dropped 20%, in part, through increased provider experience and the resulting improvements in their routine care (3). This pandemic has highlighted the need for timely, widespread, and effective healthcare worker training focused on the bedside care of patients during global health crises, both now and into the future.

In early 2020, during the initial stages of the pandemic, a number of educational initiatives were launched at global, national, and local levels. The first open-access COVID-19 educational programs focused on caring for critically ill patients requiring invasive mechanical ventilation or public health interventions (4, 5). These programs mirrored synchronous efforts to strengthen infrastructure, such as increased intensive care unit bed capacity and ventilator availability. While the vast majority of patients suffering from COVID-19 did not require invasive mechanical ventilation, the mortality of those who did require intubation remained stubbornly high (6, 7).

In most settings worldwide, and particularly in resource-limited settings, patient care is primarily provided by nurses and other non-physician providers in conjunction with generalist physicians. These providers typically have minimal training in performing advanced airway interventions or managing complex, critically ill patients (8–10). Consequently, the opportunity for improving patient outcomes in most regions of the world was in strengthening the skills necessary for the identification, evaluation and treatment of COVID-19 patients with mild to moderate illness — where appropriate early intervention could obviate the need for invasive mechanical ventilation (11–13). To date, however, there is a paucity of evidence that exigent inservice training targeting providers in LMICs and rural and remote areas (i.e., resource-limited), particularly in the midst of a global health crisis, can be effective for knowledge uptake or influencing clinical practice (14, 15).

Massive open online courses (MOOCs) may be a valuable method of disseminating optimal patient care recommendations to bedside healthcare workers during healthcare crises. Thus far, the vast majority of MOOCs have been designed for, and utilized by, students in North America and Europe (16, 17). Very little published literature has reported on MOOCs designed for inservice training of healthcare workers in LMICs, despite their significant potential impact in this space (18–21).

In addition to identifying effective methodologies for reaching and training healthcare workers in lower resource settings, presenting credible and evidence-based training materials is also critical. An infodemic — where vast amounts of information and misinformation on a topic are readily available leading to confusion and fallacy — has been a consistent challenge during the COVID-19 pandemic, documented across over 30 countries (22, 23). To combat this, a WHO technical consultation on the COVID-19 infodemic has called for strategic partnerships across all sectors, including social media/technology, academia, and civil society, to serve as trusted information sources (24).

Using a self-directed learning theory, we developed and deployed a free-of-charge MOOC entitled “COVID-19: Training for Healthcare Workers” (16). Self-directed learning theory allows individuals to guide their learning, establish their learning objectives, and manage their time based on their needs while still benefiting from access to carefully curated content (25). Our aim was to evaluate the viability of MOOCs to provide rapid access to emerging medical knowledge in the early stages of the COVID-19 pandemic. We hypothesized that participants would show improvement in knowledge gain and self-efficacy toward relevant course topics centered around diagnostic assessment and treatment of COVID-19.

Materials and methods

Educational design and delivery

In May through July of 2020, we developed a massive open online course using an international team of 22 emergency medicine physician educators, 3 medical illustrators, 3 video editors, 2 education technology staff, and 1 project manager. The majority of the development team had prior experience designing and building online educational programs. The framework used to construct the learning objectives was the backward design process: (1) Identify desired results; (2) Determine acceptable evidence; and, (3) Plan learning experiences and instruction (26). Learning objectives were defined by a core physician leadership team. The academic faculty that developed the COVID-19 course learning objectives were all medical school faculty from U.S. academic institutions—many with extensive experience designing medical school courses and developing online curricula for medical education. These objectives focused on the essential knowledge and skills deemed necessary for bedside healthcare workers to recognize and care for suspected and confirmed COVID-19 patients in the early and middle stages of their disease course. The course was broken down into 6 modules comprising 15 topic-focused lecture videos (Box 1).

BOX 1 COVID-19 course modules.

Key features and PPE

• Recognizing key features

• PPE and scene safety

Clinical assessment

• Approach to the sick patient

• Shock evaluation at the bedside

• Assessing the dyspneic patient - clinical

Diagnostic assessment

• Assessing the dyspneic patient - diagnostic

• Ultrasound in COVID-19

Early treatment

• Treating the mildly dyspneic patient

• Treating the moderately dyspneic patient - part 1

• Treating the moderately dyspneic patient - part 2

Advanced treatment

• Treating the severely dyspneic patient - part 1

• Treating the severely dyspneic patient - part 2

• Treating the severely dyspneic patient - part 3

Invasive mechanical ventilation

• Ventilator management - part 1

• Ventilator management - part 2

We anticipated that the majority of our target learners — self-directed, currently practicing healthcare workers in LMICs and remote and rural settings — would engage in course content via mobile devices. To optimize uptake by these learners, we aimed to produce brief (≤10 min) videos with minimal amounts of simplified text and universal imagery. We followed design principles developed through prior work aimed at using visual styles that resonate across diverse global audiences. The 15 video-based lectures totaled approximately 3-h in length and were accompanied by lecture handouts for review and reference. The course was subsequently launched on two online platforms, Coursera and EdX, on July 17th, 2020. It was also offered as a course on the free mobile application, Digital Medic.

Course analysis

From July 17th to September 24th, 2020, we recruited all learners who enrolled in the course on Coursera and EdX to participate in our course evaluation. Written informed consent was obtained digitally through a request on the course platforms. Learners who chose not to participate in the evaluation could still access the same course materials. Consenting participants completed questionnaires before and after the course and their course quiz and exam scores were de-identified and included in the analysis. Ethical approval was obtained by the Institutional Review Board at Stanford University (Protocol 57831).

In the baseline survey, participants reported their age, gender, profession, context of employment, education level, and ethnicity/race. Participants’ country of occupancy was obtained from Coursera and EdX directly. Participant knowledge was assessed using pre- and post-module quizzes and a final exam. Knowledge questions were drafted by the faculty content experts who designed the module materials and reviewed by at least two additional faculty and education and instructional design team members. The percentage of correct answers during a participant’s first attempt was used to determine their score. Students were asked to rate their confidence before and after the course in various domains using a 4-point Likert scale. A 4-point scale was chosen in order to force a specific opinion and eliminate a “neutral” option. Each item was rated as follows: 1 = Strongly disagree, 2 = Somewhat disagree, 3 = Somewhat agree, 4 = Strongly agree.

Data from the pre-course demographic survey were summarized using descriptive statistics. To evaluate predictors of course completion, univariate and multivariable logistic regression was used. Paired t-tests were used to detect differences in participant knowledge and self-efficacy scores before and after the course. To assess changes in knowledge, we compared pre-module versus post-module scores, as well as pre-module versus final exam scores. To examine whether there were differences between learners from different backgrounds, knowledge and self-efficacy scores were stratified by occupation and country income level for additional analysis. Data from the participants’ course satisfaction ratings were summarized using descriptive statistics. All statistical analyses were performed using STATA SE version 16.

Results

Course engagement

Between July 17th and September 24th, 2020, 30,859 learners enrolled in the course with 18,818 starting the course. Of those who started the course, 10,714 participated in the pre-course survey, 7,101 completed the course, and 5,184 completed the post-course survey (Figure 1). Female participants accounted for 55% of enrollees that started the course (Table 1). The majority of participants (69%) were less than 40 years of age. The median time to course completion was 4.2 days (IQR 1.0–11.9 days).

Figure 1

Figure 1. Participant enrollment and completion.

Table 1

Table 1. Characteristics of participants who started the course.

Participants by provider type

MOOC participants were predominantly healthcare workers and held higher education degrees. Healthcare workers that were neither physicians nor nurses accounted for the highest percentage of course participants who started the course (33%) while physicians accounted for one-fifth of all learners (Table 1). Students made up one quarter of participants. This included both those who identified as working at a clinical site and those that reported not practicing clinically. Non-healthcare workers accounted for 9%.

Geographic distribution of participants

A plurality of participants (40%) came from high-income countries (HICs) followed closely by lower-middle (38%) and upper-middle-income (20%) countries (Table 1). Participation by geographic region was highest in North America (25%) but closely followed by South Asia (24%) and Latin America and the Caribbean (17%). The smallest percentage of participants came from Sub-Saharan Africa (6%).

Knowledge gain

Overall course participants demonstrated significant improvement in knowledge upon course completion. While the mean score of pre-module quizzes was 52%, participants averaged 74% on post-module quizzes and 78% on the final exam (mean difference pre vs. post 23% [p < 0.001] and pre vs. final 26% [p < 0.001]). Non-physician healthcare workers and students demonstrated knowledge gain on par with physician participants (Figure 2). Similar improvements in knowledge were obtained by participants from HICs compared to LMICs, with participants in LMICs demonstrating slightly greater improvements in knowledge (mean difference, pre vs. post: 23% LMICs vs. 22% HICs (p < 0.001) and pre vs. final: 27% LMICs vs. 25% HICs [p < 0.001]) (Figure 2).

Figure 2

Figure 2. Knowledge gain by provider type and country income level.

Predictors of course completion

On multivariable analysis, participants from low-middle and high-middle-income countries were more likely to complete the course compared to learners from HICs (Table 2). Physicians, nurses and students had similar course completion rates with lower completion rates noted in other healthcare workers and non-healthcare workers. Additionally, participants reporting female gender and from 40 to 59 years old, compared with those <39 years of age, were less likely to complete the course.

Table 2

Table 2. Course completion rates by characteristic^a.

Survey results

Following course completion, there were substantial improvements in learner confidence in caring for COVID-19 patients and in their self-assessment of both the adequacy of their training and access to information regarding COVID-19 (Figure 3). Further, healthcare workers who completed the course strongly agreed that the course was relevant and provided them with new knowledge about COVID-19. Most learners (92.5%) stated they were likely to recommend this course to their colleagues.

Figure 3

Figure 3. Learner self-assessment of training and access to information.

Discussion

Principle findings

This article demonstrates that MOOCs can effectively reach practicing healthcare workers in both high-income and middle-income countries and provide timely clinical training during a healthcare crisis. Providers from a multitude of backgrounds sought out self-directed clinical training on the recognition and care of COVID-19 patients, with a high percentage of learners completing the course. Survey findings revealed that knowledge scores improved regardless of provider background and geography. Supplementing these gains, providers reported increased confidence in their clinical skills to care for COVID-19 patients as well as the availability of both relevant and accurate information regarding the pandemic.

Findings in context

Just-in-time learning during a pandemic or other health crisis is imperative to improving patient outcomes and protecting the healthcare workforce. While healthcare resources have been strained in many environments during the pandemic, those in LMICs and rural and remote areas are scarce at baseline, making their response to the pandemic even more challenging (27–32). Furthermore, HCWs in LMICs may experience lower access to pandemic-related training, potentially impacting preparedness and quality of care (33). The immediate necessity of HCW training during the pandemic adds substantially to the ongoing burden of delivering quality care to patients. However, with the rapid rise in access to smartphones over the past decade, mobile online training has become feasible and accepted in many settings (34). This approach allows for the development of rapidly scalable training programs with the potential for broad distribution, without taxing the already-limited local resources.

Evans and colleagues reported on a MOOC deployed during the Ebola outbreak in 2014–2015 aimed at educating the general population on the virus. One-third of their learners came from developing economies, however, only a small fraction of their learners came from nations with active Ebola outbreaks and they did not report specifically on HCW participation or knowledge gain (35). Sneddon et al. (20) conducted a MOOC aimed at improving HCW antimicrobial stewardship. While 70% of their learners were HCWs, the course was administered as live participation over 6 weeks and assessment of knowledge gain and course efficacy by participant clinical job, demographic factors, or country income level were not evaluated.

During the COVID-19 pandemic in addition to our course, other international health organizations, universities, and nations have launched online training programs that have enrolled large numbers of users (5, 36, 37). Utenen et al. launched a course on the OpenWHO platform that covered multiple topics related to COVID-19, from emerging respiratory viruses to community engagement (5). This MOOC experienced tremendous enrollment in both the Spanish and English versions of the course. Interestingly, the Spanish version of the course had a dramatically higher completion rate (36%) than the English version (3%). Sixty percent of users were listed as “Other” or “Student” and they did not specifically report the percent of HCWs practicing clinically nor include knowledge assessments. The distribution of users accessing the OpenWHO COVID-19 course, our offering, and other similar MOOCs has repeatedly skewed toward middle- and high-income-countries. While governments in low-income countries (LICs) have implemented pandemic-related information dissemination and contact tracing through mobile applications, pandemic-related online education for HCW in LICs remains relatively underexplored (38). While MOOC’s can clearly reach healthcare workers in these nations, it remains unknown whether open-access online training solutions are equally viable options in low-income countries (LICs), given limited data due to low enrollment of providers from LICs.

Encouragingly, course completion rates, knowledge scores, and survey responses all suggested — despite the course materials being developed in English and mostly by providers based in the United States — that the course was effective across diverse geographic locals. In contrast to the typical <10% completion rates among MOOCs, our course had a completion rate of 38% (39). Additionally, knowledge scores improved significantly regardless of provider geography. The improvement in knowledge was also reflected in learner sentiment, with over 90% of learners reporting the receipt of adequate training and improved confidence in their ability to care for patients with COVID-19. These positive findings were consistent for non-physician healthcare workers too, as this cohort reported strong improvements in knowledge and self-efficacy.

Non-physician providers make up the majority of all healthcare workers, and account for even higher percentages of the workforce in LMICs (40). A plurality (31%) of learners in our program were healthcare workers representing neither physicians (21%) nor nurses (12%), such as pharmacists (3%) and emergency medical technicians (2%). The diverse training backgrounds of our enrolled providers, including non-physician healthcare workers, attests to their substantial interest in clinical training during a global health crisis; and, a free-of-charge MOOC effectively contributed to meeting this widespread demand.

Several factors may have contributed to the course’s relatively high completion rate, knowledge gain, and feedback scores documented in this report. First, our education and design teams have extensive experience in delivering both in-person and online content to healthcare providers from across the globe. This experience enabled course creators to adopt a versatile style that incorporated limited text, simple language, references to trusted materials, and highly focused content with direct relevance to clinical practice. Second, unlike traditional MOOCs, which often take months to complete, our MOOC was developed with an eye toward practicing healthcare workers whose time for continuing education was extremely limited and who required information on COVID-19 immediately. In line with self-directed learning theory, the entire course was designed to be completed within 5 hours, and its duration was segmented into brief modules requiring <15 min of continuous engagement. This time-sensitive format was intended to boost engagement from providers in LMICs, who might have restricted availability (due to job or home commitments) or unreliable internet access. Third, we were able to offer the course and certification free of charge (Coursera) or at an extremely reduced cost (EdX). Many non-physician healthcare workers in LMICs are often living near the poverty line themselves and cost may heavily influence their ability to access training programs.

Challenges and next steps

Our evaluation found the course was successful on many fronts, however, a number of limitations exist. First, the COVID-19 pandemic has brought with it innumerable challenges, one of which is the deluge of information, and misinformation, available to individuals and communities (22–24). The rapidly evolving evidence used to guide best clinical practices along with the disparate recommendations from local, national, and international health organizations, particularly early on during the pandemic, made it challenging to produce educational materials that could gain broad acceptance yet contain clinically actionable information. Furthermore, as information evolves, keeping materials and guidelines up to date poses a significant challenge, particularly at an international level (41). Even when course materials are kept current, providers that have completed the course may not continue to engage with new or revised materials, leading to knowledge decay (42).

Second, while our findings of short-term knowledge gain and improvements in participant self-assessment are encouraging, they may not be clinically meaningful. Future research is required to evaluate if inservice training for healthcare workers via a MOOC during a healthcare crisis improves long term knowledge, clinical decision making and patient-oriented outcomes.

Third, while thousands of healthcare workers were trained, millions more healthcare workers are practicing on the frontlines and require clinically relevant information during the pandemic. From our results, it is unclear if MOOCs can effectively reach providers in low-income countries and in rural and remote environments, where access to the internet may be tenuous or nonexistent. Alternative course delivery options and additional languages are being launched as prior evidence demonstrates that offering MOOCs in multiple languages can dramatically increase enrollment (5).

Conclusion

Future pandemics and outbreaks will require a rapid, consolidated global response to minimize patient morbidity and mortality. Several strategies have been identified to prevent further escalation to the level of a global health crisis, notably HCW education on infection prevention and control strategies (43). Given the country-to-country variability regarding infection prevention guidelines, MOOCS can address the need for rapid, standardized access to emerging medical knowledge during public health crises (41).

This study demonstrates that MOOCs can effectively reach practicing HCWs in low-income countries and provide inservice training during a global health crisis. Physician and non-physician providers from a multitude of geographies and backgrounds sought out self-directed clinical training on recognizing and caring for COVID-19 patients with a high percentage of learners completing the course. Knowledge improved across all participant groups regardless of demographic and other characteristics. Future research is required to understand the impact on patient-oriented outcomes and how to better reach HCWs in low-income countries.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by Institutional Review Board at Stanford University (Protocol 57831). The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

MS: Conceptualization, Data curation, Formal analysis, Methodology, Project administration, Validation, Writing – original draft, Writing – review & editing. JJ: Conceptualization, Data curation, Formal analysis, Methodology, Validation, Writing – original draft, Writing – review & editing. KA: Conceptualization, Writing – original draft, Writing – review & editing, Data curation, Formal analysis, Methodology, Validation. CP: Conceptualization, Writing – original draft, Writing – review & editing. PA: Conceptualization, Writing – original draft, Writing – review & editing. AP: Conceptualization, Writing – original draft, Writing – review & editing. AM: Conceptualization, Writing – original draft, Writing – review & editing. SM: Conceptualization, Data curation, Formal analysis, Methodology, Project administration, Validation, Writing – original draft, Writing – review & editing.

Funding

The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.

Acknowledgments

The authors would like to acknowledge the many individuals that contributed to training course development and review. We would also like to thank the front-line healthcare workers who are struggling daily to ameliorate the impact of the COVID-19 pandemic.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2024.1395931/full#supplementary-material

Abbreviations

HCW, healthcare worker; HICs, high-income countries; ICUs, intensive care units; LMICs, low- and middle-income-countries; MOOC, massive open online course; PPE, personal protective equipment; SD, standard deviation

References

1. World Health Organization. WHO coronavirus Disease (COVID-19) dashboard. (2020). Available at:https://covid19.who.int

Google Scholar

2. Stewart, R, El-Harakeh, A, and Cherian, SA. Evidence synthesis communities in low-income and middle-income countries and the COVID-19 response. Lancet. (2020) 396:1539–41. doi: 10.1016/S0140-6736(20)32141-3

Crossref Full Text | Google Scholar

3. Ledford, H. Why do COVID death rates seem to be falling? Nature. (2020) 587:190–2. doi: 10.1038/d41586-020-03132-4

PubMed Abstract | Crossref Full Text | Google Scholar

4. edX. (2020). Mechanical ventilation for COVID-19. Available at:https://www.edx.org/course/mechanical-ventilation-for-covid-19

Google Scholar

5. Utunen, H, Ndiaye, N, Piroux, C, George, R, Attias, M, and Gamhewage, G. Global reach of an online COVID-19 course in multiple languages on OpenWHO in the first quarter of 2020: analysis of platform use data. J Med Internet Res. (2020) 22:e19076. doi: 10.2196/19076

PubMed Abstract | Crossref Full Text | Google Scholar

6. Balsari, S, Sange, M, and Udwadia, Z. COVID-19 care in India: the course to self-reliance. Lancet Glob Health. (2020) 8:e1359–60. doi: 10.1016/S2214-109X(20)30384-3

PubMed Abstract | Crossref Full Text | Google Scholar

7. Botta, M. Ventilation management and clinical outcomes in invasively ventilated patients with COVID-19 (PRoVENT-COVID): a national, multicentre, observational cohort study. Lancet Respir Med. (2021) 9:139–48. doi: 10.1016/S2213-2600(20)30459-8

Crossref Full Text | Google Scholar

8. Baker, T. Essential care of critical illness must not be forgotten in the COVID-19 pandemic. Lancet. (2020) 395:1253–4. doi: 10.1016/S0140-6736(20)30793-5

Crossref Full Text | Google Scholar

9. Obermeyer, Z, Abujaber, S, Makar, M, Stoll, S, Kayden, SR, Wallis, LA, et al. Emergency care in 59 low- and middle-income countries: a systematic review. Bull World Health Organ. (2015) 93:577–586G. doi: 10.2471/BLT.14.148338

PubMed Abstract | Crossref Full Text | Google Scholar

10. Ventura, C, Gibson, C, and Collier, GD. Emergency medical services resource capacity and competency amid COVID-19 in the United States: preliminary findings from a national survey. Heliyon. (2020) 6:e03900. doi: 10.1016/j.heliyon.2020.e03900

PubMed Abstract | Crossref Full Text | Google Scholar

11. Çalışkan, F, Mıdık, Ö, Baykan, Z, Şenol, Y, Tanrıverdi, EÇ, Tengiz, Fİ, et al. The knowledge level and perceptions toward COVID-19 among Turkish final year medical students. Postgrad Med. (2020) 132:764–72. doi: 10.1080/00325481.2020.1795486

Crossref Full Text | Google Scholar

12. Echoru, I, Kasozi, KI, Usman, IM, Mutuku, IM, Ssebuufu, R, Ajambo, PD, et al. University lecturers and students could help in community education about SARS-CoV-2 infection in Uganda. Health Serv Insights. (2020) 13:117863292094416. doi: 10.1177/1178632920944167

Crossref Full Text | Google Scholar

13. Taghrir, MH, Borazjani, R, and Shiraly, R. COVID-19 and Iranian medical students; a survey on their related-knowledge, preventive behaviors and risk perception. Arch Iran Med. (2020) 23:249–54. doi: 10.34172/aim.2020.06

Crossref Full Text | Google Scholar

14. Rowe, AK, Rowe, SY, Peters, DH, Holloway, KA, Chalker, J, and Ross-Degnan, D. Effectiveness of strategies to improve health-care provider practices in low-income and middle-income countries: a systematic review. Lancet Glob Health. (2018) 6:e1163–75. doi: 10.1016/S2214-109X(18)30398-X

PubMed Abstract | Crossref Full Text | Google Scholar

15. Seymour-Walsh, A, Bell, A, Webber, A, and Smith, T. Adapting to a new reality: COVID-19 coronavirus and online education in the health professions. Rural Remote Health. (2020) 20:6000. doi: 10.22605/RRH6000

PubMed Abstract | Crossref Full Text | Google Scholar

16. Alturkistani, A, Lam, C, Foley, K, Stenfors, T, Blum, ER, Van Velthoven, MH, et al. Massive open online course evaluation methods: systematic review. J Med Internet Res. (2020) 22:e13851. doi: 10.2196/13851

PubMed Abstract | Crossref Full Text | Google Scholar

17. Bozkurt, A, Akgün-Özbek, E, and Zawacki-Richter, O. Trends and patterns in massive open online courses: review and content analysis of research on MOOCs (2008-2015). Int Rev Res Open Distrib Learn. (2017) 18, 118–147. doi: 10.19173/irrodl.v18i5.3080

Crossref Full Text | Google Scholar

18. Veletsianos, G, and Shepherdson, P. A systematic analysis and synthesis of the empirical MOOC literature published in 2013–2015. Int Rev Res Open Distrib Learn. (2016) 17:198–221. doi: 10.19173/irrodl.v17i2.2448

Crossref Full Text | Google Scholar

19. Zhu, M, Sari, A, and Lee, MM. A systematic review of research methods and topics of the empirical MOOC literature (2014–2016). Internet High Educ. (2018) 37:31–9. doi: 10.1016/j.iheduc.2018.01.002

Crossref Full Text | Google Scholar

20. Sneddon, J, Barlow, G, Bradley, S, Brink, A, Chandy, SJ, and Nathwani, D. Development and impact of a massive open online course (MOOC) for antimicrobial stewardship. J Antimicrob Chemother. (2018) 73:1091–7. doi: 10.1093/jac/dkx493

PubMed Abstract | Crossref Full Text | Google Scholar

21. Bettiol, S, Psereckis, R, and MacIntyre, K. A perspective of massive open online courses (MOOCs) and public health. Front Public Health. (2022) 10:10. doi: 10.3389/fpubh.2022.1058383

PubMed Abstract | Crossref Full Text | Google Scholar

22. Zarocostas, J. How to fight an infodemic. Lancet. (2020) 395:676. doi: 10.1016/S0140-6736(20)30461-X

PubMed Abstract | Crossref Full Text | Google Scholar

23. Cha, M, Cha, C, Singh, K, Lima, G, Ahn, YY, Kulshrestha, J, et al. Prevalence of misinformation and Factchecks on the COVID-19 pandemic in 35 countries: observational Infodemiology study. JMIR Hum Factors. (2021) 8:e23279. doi: 10.2196/23279

PubMed Abstract | Crossref Full Text | Google Scholar

24. Tangcharoensathien, V, Calleja, N, Nguyen, T, Purnat, T, D’Agostino, M, Garcia-Saiso, S, et al. {citation}framework for managing the COVID-19 Infodemic: methods and results of an online, crowdsourced WHO technical consultation. J Med Internet Res. (2020) 22:e19659. doi: 10.2196/19659

PubMed Abstract | Crossref Full Text | Google Scholar

25. Maya-Jariego, I, Holgado, D, Gonzalez-Tinoco, E, Castano-Munoz, J, and PUnie T,. Typology of motivation and learning intentions of users in MOOCs: the MOOCKNOWLEDGE study. Educ Tech Research Dev. (2020) 68:203–24. doi: 10.1007/s11423-019-09682-3

Crossref Full Text | Google Scholar

26. Wiggins, GM, and Jay,. Understanding by Design. 2nd ed. Alexandria, VA, USA: Hawker Brownlew Education. (2005).

Google Scholar

27. Wahlster, S, Sharma, M, Lewis, AK, Patel, PV, Hartog, CS, Jannotta, G, et al. Pandemic’s effect on critical care resources and providers. Chest. (2019) 159:619–33. doi: 10.1016/j.chest.2020.09.070

Crossref Full Text | Google Scholar

28. Starr, N, Rebollo, D, Asemu, YM, Akalu, L, Mohammed, HA, Menchamo, MW, et al. Pulse oximetry in low-resource settings during the COVID-19 pandemic. Lancet Glob Health. (2020) 8:e1121–2. doi: 10.1016/S2214-109X(20)30287-4

PubMed Abstract | Crossref Full Text | Google Scholar

29. Keeley, C, Jimenez, J, Jackson, H, Boudourakis, L, Salway, RJ, Cineas, N, et al. Staffing up for the surge: expanding the new York City public hospital workforce during the COVID-19 pandemic: article describes how new York City’s public health care system rapidly expanded capacity across 11 acute-care hospitals and three new field hospitals to meet the challenges of the COVID-19 pandemic. Health Aff. (2020) 39:1426–30. doi: 10.1377/hlthaff.2020.00904

PubMed Abstract | Crossref Full Text | Google Scholar

30. Cheng, KJG, Sun, Y, and Monnat, SM. COVID-19 death rates are higher in rural counties with larger shares of blacks and Hispanics. J Rural Health. (2020) 36:602–8. doi: 10.1111/jrh.12511

PubMed Abstract | Crossref Full Text | Google Scholar

31. Lobo, SM, Creutzfeldt, CJ, Maia, IS, Town, JA, Amorim, E, Kross, EK, et al. Perceptions of critical care shortages, resource use, and provider well-being during the COVID-19 pandemic: a survey of 1,985 health care providers in Brazil. Chest. (2022) 161:1526–42. doi: 10.1016/j.chest.2022.01.057

PubMed Abstract | Crossref Full Text | Google Scholar

32. Boro, E, and Stoll, B. Barriers to COVID-19 health products in low-and middle-income countries during the COVID-19 pandemic: a rapid systematic review and evidence synthesis. Front Public Health. (2022) 10:928065. doi: 10.3389/fpubh.2022.928065

PubMed Abstract | Crossref Full Text | Google Scholar

33. Alhadi, B, Khder, MM, Ali, SR, Taha, K, and Manzour, AF. Health care workers’ perceptions of their hospitals’ preparedness during the COVID-19 virus pandemic in three different world regions. Clin Epidemiol Glob Health. (2023) 21:101278. doi: 10.1016/j.cegh.2023.101278

PubMed Abstract | Crossref Full Text | Google Scholar

34. Ozdalga, E, Ozdalga, A, and Ahuja, N. The smartphone in medicine: a review of current and potential use among physicians and students. J Med Internet Res. (2012) 14:e128. doi: 10.2196/jmir.1994

PubMed Abstract | Crossref Full Text | Google Scholar

35. Evans, DP, Luffy, SM, Parisi, S, and del Rio, C. The development of a massive open online course during the 2014–15 Ebola virus disease epidemic. Ann Epidemiol. (2017) 27:611–5. doi: 10.1016/j.annepidem.2017.07.137

PubMed Abstract | Crossref Full Text | Google Scholar

36. Johns Hopkins coronavirus resource center. Contact Tracing Tools, Information, and Resources - Johns Hopkins. Available at:https://coronavirus.jhu.edu/contact-tracing (Accessed November 30, 2020).

Google Scholar

37. Government of India launches online training platform- iGOT in. (2020). Available at:https://www.investindia.gov.in/team-india-blogs/government-india-launches-online-training-platform-igot-fight-against-covid-19

Google Scholar

38. Winkie, MJ, and Nambudiri, VE. A tale of two applications: lessons learned from national LMIC COVID applications. J Am Med Inform Assoc JAMIA. (2022) 30:781–6. doi: 10.1093/jamia/ocac146

Crossref Full Text | Google Scholar

39. Khalil, H, and Ebner, M. MOOCs completion rates and possible methods to improve retention - a literature review. World Conference on Educational Multimedia, Hypermedia and Telecommunications 2014 (ED-Media). (2014);10.

Google Scholar

40. World Health Organization ed. Monitoring the building blocks of health systems: A handbook of indicators and their measurement strategies. Geneva: World Health Organization (2010). 92 p.

Google Scholar

41. Qureshi, MO, Chughtai, AA, and Seale, H. Recommendations related to occupational infection prevention and control training to protect healthcare workers from infectious diseases: a scoping review of infection prevention and control guidelines. BMC Health Serv Res. (2022) 22:272. doi: 10.1186/s12913-022-07673-4

PubMed Abstract | Crossref Full Text | Google Scholar

42. Arsenault, C, Rowe, SY, Ross-Degnan, D, Peters, DH, Roder-DeWan, S, Kruk, ME, et al. How does the effectiveness of strategies to improve healthcare provider practices in low-income and middle-income countries change after implementation? Secondary analysis of a systematic review. BMJ Qual Saf. (2022) 31:123–33. doi: 10.1136/bmjqs-2020-011717

PubMed Abstract | Crossref Full Text | Google Scholar

43. Tusabe, F, Tahir, IM, Akpa, CI, Mtaki, V, Baryamujura, J, Kamau, B, et al. Lessons learned from the Ebola virus Disease and COVID-19 preparedness to respond to the human Monkeypox virus outbreak in low- and middle-income countries. Infect Drug Resist. (2022) 15:6279–86. doi: 10.2147/IDR.S384348

PubMed Abstract | Crossref Full Text | Google Scholar