WhatsApp-Delivered Intervention for Continued Learning for Nurses in Pakistan During the COVID-19 Pandemic: Results of a Randomized-Controlled Trial

The COVID-19 pandemic has necessitated support for continued learning in frontline practitioners through online digital mediums that are convenient and fast to maintain physical distancing. Nurses are already neglected professionals for support in training for infection control, leadership, and communication in Pakistan and other developing countries. For that reason, we aimed to deliver a WhatsApp-based intervention for continued learning in nurses who are currently working in both private and public sector. A 12-week intervention was delivered to 208 nurses (102 in the control group and 106 in the intervention group) who had been employed in the clinical setting during data collection. The analysis reveals that nurses in the intervention group show significantly better results for learning in “infection prevention and control” and “leadership and communication.” Results of a content analysis based on participant's feedback also confirm that the WhatsApp-based intervention is a valuable tool for education. This study highlights the effectiveness of online-based digital interventions as a convenient training tool for awareness and management of infectious diseases, leadership, and communication during COVID-19 and beyond. Furthermore, this study emphasizes that group interventions with other healthcare practitioners and the role of on-going longer WhatsApp-based interventions can become integral tools to support continued learning and patient safety practices.

Introduction

Nurses across the globe have been struggling with multiple and compounding challenges since the onslaught of the COVID-19 pandemic (1). A range of challenges have been reported in the literature, including shortages of Personal Protective Equipment (PPE) (2), deficiencies in staffing and compensation (3), inability to maintain physical distancing due to nature of job (4), and fear of acquiring the infection and passing it to family members (5). However, less attention has been given to the predicament of continued learning during the pandemic (6). Continued learning in nurses is empirically associated with lower mortality and morbidity among patients (7). Empowering nurses with skill development and on-the-job training also has benefits for nurse's self-esteem and their job satisfaction (8). In addition, continued learning is known to provide nurses with better opportunities for professional development and career advancement (9, 10).

Nurses require support for continued learning in three keys areas during the pandemic. Firstly, learning for infection prevention and control is needed to prevent exposure to infection and patient placement and isolation precautions (11). Despite the fact that nurses are frontline practitioners and more frequently exposed to infectious disease in the clinical setting (12), there has been inadequate support for infection control training since the outbreak of the COVID-19 pandemic (13). The training that has been delivered has been criticized for being either too long and too difficult to follow, or too short with lack of opportunity for providing questions and receiving adequate answers (14).

Different learning elements related to infection prevention and control are associated with improved care delivery practices in nurses and recovery rates for patients. These learning elements include better understanding of infection prevention and control, improved planning and monitoring, and the ability to evaluate oneself and colleagues (15). With both the Ebola and COVID-19 pandemic, the lack of nurse preparation for infection prevention has contributed to a delayed response and increased rate of mortality and morbidity (16, 17). Coping capacities of nurses are linked to the combination of pre-service and on-the-job learning support which provide positive incentives for functional efficiency and patient safety culture.

Secondly, nurses are in need for continued learning about COVID-19 (18). Some literature confirms that less than half of healthcare practitioners, including nurses, have knowledge about COVID-19 and patient management (19). Other research suggests that the majority of nurses and other healthcare practitioners are dependent on social media for updates about disease management (20) and need regular and intensive learning support (21). Ongoing and updated knowledge of COVID-19 is integral for nurses, and other frontlines workers, during the pandemic as it shapes patient care protocols and successful discharge rates (22). Knowledge and continued learning of the management of COVID-19 patients is also important in emergency situations and during urgent care coordination between healthcare providers.

Better knowledge of disease epidemiology in nurses is positively associated with reduced adverse patient outcomes including misdiagnosis and medication errors (23). As highlighted by International Labor Organization and World Bank, the novelty of the disease and guidelines for the management protocols of mild to severe patients of COVID-19 is evolving and, thus, there is need for continued training for nurses and other healthcare practitioners (24). Effective learning support for disease knowledge and management can improve the overall performance capacity of healthcare service delivery, especially in resource short developing regions. Furthermore, it is a vital factor that improves job satisfaction and emotional health of nurses during critical times (25).

Thirdly, nurses need support in continued learning for leadership and communication skills (26). This is especially relevant during pandemics when nurses are required to play a more autonomous and self-reliant role in emergency situations (27). As frontline providers, each nurse, regardless of designation, must be supported for continued education to assume leadership roles. The ability of nurses to have skills for leadership, to speak up, and come up with innovative ideas in emergency situations is important (28). Nurses that are empowered with continued learning for leadership and management during the pandemic are in a better position to take responsibility for efficient care delivery to patients.

Healthcare providers, including nurses, are particularly vulnerable to communication problems in the current pandemic, given its unique and uncertain nature (29). Evidence highlights that the pandemic has left nurses with more burnout and stress due to the increase in communication problems and conflict at the workplace (30). This is primarily due to the difficult circumstances, but also because nurses bear the brunt of negative reactions in hospital settings and public zones due to lower professional status (31). It is important that nurses are able to manage conflict at the workplace for their emotional health, but also to secure quality services for patients and family attendants (32). Similarly, improved communication between nurses and colleagues is integral for preventing psychological distress in nurses and improving patient outcomes (33). Ultimately, nurse initiatives for leadership and effective communication during the pandemic will also determine their professional growth and status in the future (34).

Situation in Pakistan

On-the-job training and continued learning for clinical nurses is overall a neglected topic in Pakistan (35). At the outbreak of the COVID-19 pandemic, the government of Pakistan initiated briefing sessions for infection control and awareness among healthcare practitioners. But not all the sessions have been comprehensive, ongoing or supportive for reinforcement, or with provision for questions and answers (36). Some local research has identified that only half of nurses received any formal training of infection prevention and control or how to manage COVID-19 cases (37). Further research highlighted that healthcare practitioners did not receive any support for learning or training. And even if they received some support, it only consisted of a briefing in a single session by a senior doctor or a placement of notes across wards by an infection control nurse (37). Furthermore, many practitioners have been provided written guidelines without instruction or specific guidelines.

In Pakistan, there is a critical need for nurses and other allied healthcare staff to receive continued learning support (38). Healthcare professionals have reported the need for clearer communication related to infection prevention and control (39). There is also a need for a more systematic and integrated infection control team at each hospital, with a trained infection control focal person responsible for ensuring prevention and standard practices meeting updated international guidelines (40). Training to nurses for leadership and communication also needs prioritization in Pakistan (41). Nurses have mostly been excluded as leaders in the clinical setting, with leadership training and leadership roles being allocated to physicians and hospital administration (42). In addition, most nurses report communication problems with their leaders and coworkers (43). Continued learning and training for nurses in leadership and communication must be initiated in order to secure high-quality healthcare services. The development of effective treatment plans during the pandemic and beyond is a necessity (44).

Role of Social Media in Continued Learning

The constantly changing nature of providing and receiving information as well as uncertainty and inadequacy of training processes has made it imperative for nurses and other healthcare practitioners to have supplementary support for learning and knowledge building. A salient question during the COVID-19 pandemic is to ascertain how to support frontline practitioners like nurses for continued learning and pandemic preparedness (45). Improving training support requires a multi-pronged approach, with support coming in not just from hospital administration and the government, but also public health experts and the private sector. In fact, multiple platforms or methods of communication are preferred for receiving training and accessing information related to infection prevention and disease management (46). The long and erratic work hours during the coronavirus pandemic has necessitated that nurses be supported with different modes of training which suits their schedule (45). Single didactic training sessions with large number of participants have received less appreciation from participants. Prior research supports the effectiveness of employing a distributed practice approach for enhancing knowledge retention (47).

WhatsApp-based interventions to support learning can be beneficial as they allow participants to download and review material when they are free and relaxed (48). Some of the advantages of WhatsApp-based learning include (i) free choice of time during the day to absorb material and interact with questions, (ii) supporting a combination of training tools (text, audio, image, and video), and (iii) maintaining physical distancing during the pandemic. However, a recent health intervention has emphasized that WhatsApp moderators must be present to facilitate learning and communication between trainers and participants (49).

Aims of Study

There is immense reliance on nurses with regard to patient safety. However, very little effort is made to invest in interventions for their continual learning support and leadership roles in developing countries, including Pakistan (12, 50). Empowering nurses through training during the COVID-19 pandemic is an essential step to facilitate nurse leadership and safety culture in hospitals (19, 51).

The overall scope of this research is to improve nurse leadership in maintaining minimum service delivery standards for hospital infection control with respect to COVID-19, but also other infectious disease burdens beyond the pandemic (52–54). We believe that research about potential benefits for on-going training for nurses and digital literacy options will develop broader options for training support for nurses in Pakistan and other developing regions. In lieu of this, the aim of the study is to deliver a digital intervention for continued learning to supplement and reinforce information for currently working nurses in the age of coronavirus with respect to three areas: (i) infection prevention and control, (ii) COVID-19-related knowledge, and (iii) leadership and communication. Improvement in learning and efficacy of the intervention will be assessed by making a comparative study of participant's knowledge base before and after the intervention and by comparison with a control group who has not received the intervention.

In this way, this study aims to investigate three research questions:

1. Is there a significant difference between the pre-post-test results of the intervention and control group after the intervention, in comparison to their baseline scores and in comparison to each other?

2. Do the socio-demographic characteristics of the nurse participants in the intervention group show significant associations with post-test results?

3. Is there any feedback about the value of the WhatsApp-based intervention from participants in the intervention group in order to plan future interventions?

Methods

Sample

We included nurses in our study who were employed in public or private hospitals of Lahore, Pakistan, during the COVID-19 pandemic. Nurses who were not employed at time of data collection were excluded from study participation. There were 81,816 nurses registered nurses in Punjab (55), with no confirmed or published data about the number of nurses in Lahore. We used the approximate population percentage of Lahore city, which is 10% of Punjab's population, to estimate a nurse population of 8,181 for Lahore. We then used the Taro Yamane formulae to estimate a sample size of 381 nurses. Lahore is the capital of the largest province of Pakistan, Punjab province, with an estimated population of above 11 million people. There are an approximate 15 private and 27 public hospitals in the city. All the sampled hospitals in this study belong to the Category 1 classification of teaching hospitals that have more than 500 beds and have been facilitating coronavirus patients in ICUs or separate wards. The focus in Category 1 hospitals secures the homogeneity of the sample and workplace environment. Four hospitals in total were sampled purposively, two each from the private and public sector. We were able to gain access to contact information (mobile number and email addresses) of between 111 and 203 nurses from each sampled hospital.

Data Collection

We sent both phone and WhatsApp text messages to a total of 400 nurses, 100 randomly selected from each hospital, informing them about the research objectives and study design. A total of 372 nurses responded to the message and 344 agreed to participate in the study. We divided consenting participants by random alternate selection into the intervention group and control group with 172 participants each. By the end of the intervention, a total of 106 nurse participants remained part of the intervention group and 102 nurse participants from the control group returned the complete post-test survey.

Both the pre and post-test survey was administered through Google Survey Form, which is a free and safe method of collecting confidential data (56). Data were stored on a secure “cloud” database, where it was automatically sorted and scored. Each participant was allocated a code and no confidential data or names were shared. Data were held securely with the lead author (SRJ). The survey for this study was implemented on Google by one of the authors (SRJ) with a password protected account. The first set of questions on the survey included demographic data including the complete names of the nurses, their respective WhatsApp number and email address, name of lead intervention facilitator/WhatsApp moderator, WhatsApp group code, and hospital affiliation. This data was used to link the pre- and post-test data for each participant.

The flow of nurse participants in the study is summarized in Figure 1. Reasons for participant drop-out were: (i) internet connectivity problems and cost of downloading videos on mobile data (n = 74; 54.4%), (ii) exit from group without response or explanation (n = 22; 16.2%), (iii) lack of time to be part of the intervention group and to read materials or watch videos (n = 15; 11.0%), (iv) self or family member infected with COVID-19 (n = 13; 9.6%), and (v) revoked permission from family members/husband to remain part of a WhatsApp group because of work-family conflict (n = 12; 8.8%).

FIGURE 1

Figure 1. Flow of participants in the study.

Intervention Design

The research design included administration of a pre-post-test survey and communication of a literacy booklet in English to both the control and intervention group. The intervention group received a more detailed intervention with learning information communicated via WhatsApp. The entire intervention lasted 12 weeks and included the following seven steps, which are also summarized in Table 1.

TABLE 1

Table 1. Literacy modules instruction plan for intervention group.

Step 1: Pre-test Survey

The pre-test survey was administered to both the control and intervention group through Google survey form. Each participant was sent a reminder through WhatsApp to complete the survey.

Step 2: Continued Learning Booklet

The booklet was sent through both email and WhatsApp individually to each participant in the control and intervention group. The continued learning booklet was distributed in English and Urdu language. It was developed based on a combination of relevant international standard guidelines. The booklet contained three modules with information related to the three study areas (Supplementary Data Sheets 1–3): (i) infection prevention and control (57, 58), (ii) COVID-19 knowledge (59, 60), and (iii) leadership and communication (61–63).

Step 3: Preliminary Zoom Sessions for Introduction (Intervention Group)

The principal investigators (SRJ, AJ, RRD, and SKB) of the study held an introductory Zoom session with the participants in the intervention group. The participants in the intervention group were divided in groups of 16–20 for the Zoom sessions, and the same groups were used to form the WhatsApp groups in the next step of the intervention. We encountered problems in coordinating a suitable time for group members. For that reason, for each group an attempt was made to hold at least three Zoom sessions to accommodate all participants and their requests for different time slots. During the Zoom introductory session with each group the following topics were discussed: (i) the objectives of the intervention in more detail, (ii) review knowledge of netiquette and observance of ethics in dealing with fellow participants during the intervention, (iii) elements of the content of the three modules, and (iv) introducing intervention deliverers/specialists in the field who would be sending video tutorials over the 12-week intervention period. The Zoom sessions were aimed to be a team building exercise to support participants to read the material and engage in feedback and Q&A sessions and also build a rapport with the WhatsApp facilitators. This Zoom session was also recorded and sent to each WhatsApp group.

Steps 4–6: WhatsApp Group Learning (Intervention Group)

As mentioned above, 20 WhatsApp groups with 16–20 participants each were formed for the intervention group for the duration of the intervention. Each group was managed by one WhatsApp moderator and a team of health experts (three authors of this study-SH, FA, and AS), who were responsible for administration and support of participants through different hours of the day (Figure 2). The WhatsApp group was used to send text messages, PDF files, images, audio notes, and video tutorials related to the three learning areas of (i) infection prevention and control, (ii) COVID-19 knowledge, and (iii) leadership and communication. A total of 21 tutorials by five experts and clinicians, with 5–10 min duration each, were sent to the intervention group participants every 2 days. Including case-study discussion and time for question and answers, each of the three learning areas was covered over a period of 18 days each.

FIGURE 2

Figure 2. Screenshot of a WhatsApp communication to the intervention group including a video tutorial by subject expert and introductory text.

Daily messages were sent to provide support in answering questions of participants and coordinating feedback and comments about the training. Though nursing school education is conducted in English in Pakistan, Urdu is the national language of the country and the medium of instruction at primary and secondary level includes Urdu, English and the provincial language (64). It was thus deemed valuable to provide reinforcement by translating the literacy booklet material in Urdu to help in building memory blocks and retention of information.

Step 7: Post-test Survey

The post-test survey was administered to both the control and intervention group through Google survey form. Again, participants were individually sent a reminder through WhatsApp to complete the survey.

Facilitators of the Intervention

A team of 20 nurses was recruited to support the project for (i) support to and coordination of the nurse participants for the study, (ii) creation and moderation of the WhatsApp groups, and (iii) referral of questions and answers between nurse participants and trainers for continued learning. All WhatsApp nurse moderators were registered and licensed nurses from Pakistan Nursing Council with at least 1 year of experience in the clinical setting. Two authors of this study (NR and SRJ) were responsible for a one-week training of the WhatsApp moderators for the objective of the study and how to facilitate the intervention.

The continued learning written material and video tutorials have been developed by six experts (RZ, SH, AM, FA, AS, and SKB) who had one or more of the following qualifications: (i) medical doctor currently working in the COVID-19 ward or isolation centers; (ii) trainer for infection prevention and control and coronavirus management; and/or (iii) trainer for public health safety and leadership and communication at the workplace.

Pre- and Post-test-survey

The pre-post-test survey has been developed using items from the intervention material. A total of 42 questions were part of the survey, with 10 questions for socio-demographic characteristics, nine questions related to the workplace, and eight questions each for the three study intervention areas (“infection prevention and control,” “COVID-19 knowledge,” and “leadership and communication”). A five-point Likert scale was used with responses from “strongly disagree,” “disagree,” “neutral,” “agree,” and “strongly agree.” Coding ranged from “strongly disagree” being allocated a score of 1 to “strongly agree” being allocated a score of 5. Cronbach's alpha was calculated in reliability analyses for the three domain scales. All three scales showed good reliability (infection prevention and control: α = 0.770; COVID-19 knowledge: α = 0.746; leadership and communication: α = 0.759).

Data Analysis

Quantitative: Statistical Analysis

The hypotheses for this study are:

H1: Nurses in the intervention group, compared to the control group, report an increase in knowledge in the three study intervention areas: (i) infection prevention and control, (ii) COVID-19 knowledge, and (iii) leadership and communication.

H2: Nurses in the intervention group report an increase in knowledge compared to their baseline scores, in the three study intervention areas: (i) infection prevention and control, (ii) COVID-19 knowledge, and (iii) leadership and communication.

H3: Socio-demographic characteristics of the intervention group are associated with improved learning areas in the post-intervention scores.

Data analysis was performed to determine the main effect of the intervention using SPSS 25.0. Descriptive statistics were used to describe the characteristics of the control and intervention group. One-way ANOVA was performed to assess the relationship between socio-demographic characteristics with each of the three study intervention areas (“infection prevention and control,” “COVID-19 knowledge,” and “leadership and communication”) at baseline. Paired sample t-tests were run to compare the post intervention mean differences between the control and intervention group for each item used to measure the three study domains.

The three study domains were compounded for further analysis with five points on each item; with the highest score for satisfactory learning and knowledge. Respective compounded scores for each of the three domains are considered as dependent variables for the analysis. Independent sample t-tests were used to compare the mean difference between the control and intervention group for the three study intervention areas. Finally, ANCOVA was performed to assess mean differences post intervention holding confounding variables (baseline results, age, gender, income, years of work, degree type, and type of contract) constant. All p < 0.05 were considered significant for this study.

Qualitative: Content Analysis

The WhatsApp conversations were transferred to Microsoft Word for coding of themes. A total of 1,368 messages were received from nurse participants through the 12-week intervention. Independent manual coding (65) was performed by two of the authors (RD and SJ). At the first step, all emoticons were excluded (examples of these emoticons include thumbs up, clapping hands, raising hands, folded hands, and smiling face). Next, common thematic categories were grouped together based on frequency. There were some messages typed in Urdu, which were translated into English. Translations were doubled-checked by a third author (SKB) through the forward backward method (66). Authors held meetings to confirm and corroborate final themes. Final themes were also discussed with two the WhatsApp moderators and 10 nurse participants for further verification (67).

Ethics

Ethics approval for this study was taken from the Institutional Review Board, Forman Christian College (A Chartered University). All ethics were observed diligently with respect to informed consent and confidentiality of nurse participants. No names of nurses and hospitals have been reported. Participants were not provided any incentive to participate in the study and they were informed about the freedom to exit the intervention if and when they chose.

We ensured that all participants who were grouped together in one WhatsApp group (not more than 20 participants) were from the same hospital and had a common employer. In this way, we were able to guarantee both consent of participants and safety with regard to data security and privacy protection (68).

Results

Socio-Demographic Characteristics

The socio-demographic characteristics of the sample are presented in Table 2. The majority of respondents are aged below 39 years and female. ANOVA results show homogeneity of the control and intervention group at baseline (Supplementary Material 1).

TABLE 2

Table 2. Descriptive statistics of the sample (n = 208).

Pre- and Post-intervention Mean Results for the Intervention Areas

Table 3 presents the pre- and post-intervention mean results for the intervention and control group with respect to the three study intervention areas. At post-intervention, the mean values for the intervention group, compared to the control group, are higher for all intervention areas: (i) “infection prevention and control” (Mean difference: 1.11); (ii) “COVID-19 knowledge” (Mean difference: 1.84); and (iii) “leadership and communication” (Mean difference: 1.45). Additionally, all three study domains show significant results after the intervention.

TABLE 3

Table 3. Mean results for the three intervention areas for the control and intervention group at pre- and post-intervention (n = 208).

Post-intervention Results for Items of the Intervention Areas

Table 4 presents the post-intervention results for each item [means (M) and standard deviations (SD)] of the three intervention areas (“infection prevention and control,” “COVID-19 knowledge,” and “leadership and communication”). For “infection prevention and control,” the following items show greater mean values and significant results in the intervention group: (i) Knowledge of what is part of infection prevention and control (p = 0.001); (ii) Confidence in knowledge about controlling infection at the workplace (p = 0.040); (iii) Confidence in collaboration with other healthcare workers for infection prevention and control (p = 0.044); (iv) Confidence in contributing to written records for infection prevention and control (p = 0.019).

TABLE 4

Table 4. Mean results for individual items measuring the intervention area for the control and intervention group at post-intervention (n = 208).

For “COVID-19 knowledge,” the following items show greater mean values in the intervention group with significant results: (i) Knowledge regarding COVID-19 infection and spread (p = 0.016); (ii) Knowledge regarding COVID-19 diagnosis and vaccination development (p = 0.048); (iii) Ability to advise others about COVID-19 prevention and protection (p = 0.011); (iv) Ability to manage people with suspected COVID-19 infection (p = 0.022); and (v) Ability to manage people with mild COVID-19 infection (p = 0.050).

For “leadership and communication,” the following items show greater mean values and significant results in the intervention group: (i) Knowledge in defining skilled nurse leadership; (p < 0.001); (ii) Knowledge regarding conflict management at workplace (p = 0.001); (iii) Knowledge of the different components and channels for communication at the workplace (p = 0.011); (iv) Confidence in ability to communicate more effectively at work (p = 0.041); and (v) Ability to assume leadership (p = 0.026).

Stratification of Post-intervention Results for the Intervention Group

Table 5 presents the mean results at post-intervention for the intervention group stratified by socio-demographic characteristics. There are no significant results associated with socio-demographic characteristics of participants in the intervention group, except for one area: Under the intervention area of “leadership and communication,” we find a significantly higher mean in improved learning for nurses with BSc Degrees compared to nurses who are Diploma holders (Mean difference:1.28).

TABLE 5

Table 5. Comparative means by socio-demographic characteristics of the intervention group at post-intervention (n = 106).

ANCOVA

The ANCOVA results for the intervention and control group at post-intervention stage holding confounding variables (baseline results, age, gender, income, years of work, degree type, and type of contract) constant, is presented in Table 6. A statistically significant result reveals a higher mean score for the intervention group (34.36 ± 2.68) compared to the control group (33.25 ± 3.14) for the intervention area of “infection prevention and control.” The intervention area of “leadership and communication” also shows a higher mean score for the intervention group (33.54 ± 2.98) compared to the control group (32.09 ± 3.10). Though the mean results are higher for the intervention group compared to the control group for learning in “COVID-19 knowledge,” the results are not significant.

TABLE 6

Table 6. ANCOVA results for the three study domains for the control and intervention group after controlling for confounding variables (n = 208).

Content Analysis

Intervention participant responses counted from the WhatsApp group included a total of 1,368. Four main thematic responses of the intervention group were identified: (i) Appreciation for the intervention (772 messages, 56%); (ii) Specific questions for instructors (284 messages, 21%); (iii) Request for more information or WhatsApp learning interventions (232 messages, 17%); and (iv) Feedback to improve the intervention (80 messages, 6%). Table 7 summarizes these responses and also includes direct quotes of participants.

TABLE 7

Table 7. Themes from the content analysis describing intervention group participant interaction.

Discussion

To the best of our knowledge, this is the first study to attempt delivering a digital literacy intervention for continued learning through WhatsApp to practicing nurses during the COVID-19 pandemic in Pakistan. The results suggest that online digital learning, and WhatsApp specifically, provides convenient and effective support for continued learning in nurses as an additional tool to advance literacy, communication, and skill development.

Adherence to the International Council of Nurses International Code of Ethics implies that all nurses, regardless of designation, and demographic differences, are committed to maintaining competency through continual learning (69). With respect to the first hypothesis of our study, the results confirm that nurses in the intervention group, compared to the control group, do report an increase in knowledge in the three areas of intervention delivery (“infection prevention and control,” “COVID-19 knowledge,” and “leadership and communication”). Previous research confirms the effectiveness of online-based digital interventions for nurses and also argues for blended training combining digital and in-person support (70). Our findings also corroborate further research which suggests that digital interventions can strengthen the workforce for infection control practices and consequently improve health system performance at macro-level (71).

Findings also identify the specific areas where the intervention group, compared to the control group, report improved knowledge about COVID-19. Our results suggest that online-based digital literacy interventions for COVID-19 knowledge and management have an effect on the confidence and perceived ability of practitioners. Other research suggests that nurse literacy interventions about epidemiology of disease and patient management have an impact on improved clinical practices and participation in improved care planning for patients (72). Furthermore, recent research confirms that online-based digital interventions have the benefit of rapid deployment of educational material and awareness in frontline healthcare providers (73).

Results also ascertained specific areas of improvement in the intervention group, compared to the control group, under the intervention area of “leadership and communication.” Findings reveal that participants from the intervention group have improved knowledge in defining skilled nurse leadership and differentiating components and channels for communication at the workplace. The intervention group was also able to identify better practices for conflict management at the workplace and communicate more effectively at work. Overall, our results suggest that learning for leadership and communication through online-based digitals platforms are valuable and can be delivered in parallel to other online interventions being currently planned for the psychological wellbeing of nurses (74).

With regard to the second hypothesis of the study, univariate results confirm that the intervention group does report significant increases in knowledge compared to their baseline scores for the three study intervention areas. However, holding confounding variables constant, ANCOVA results show that the intervention group shows significantly improved results for only two study intervention areas, which are “infection prevention and control,” and “leadership and communication.” Results confirm that online WhatsApp interventions to improve nurse management are of value during the pandemic and in times of emergency. It is possible that the “COVID-19 knowledge” intervention area is not significant, because healthcare workers need additional, long-run, and continuous learning support for this area, especially in developing regions where government response and information sharing is weak (75). Even frontline practitioners from the developed world are suffering from low confidence in managing COVID-19 patients due to the uncertainty of the disease and evolving knowledge about variants (76).

Results for the third hypothesis of the study suggest that there is no significant association with socio-demographic characteristics of intervention participants and intervention areas. In this way, regardless of designation and social background, intervention participants show homogenous learning outcomes. The International Code of Ethics, outlined by the International Council of Nurses, also suggests that all nurses, regardless of designation and demographic differences, are committed to maintaining competency through continual learning (69). However, under the intervention area of “leadership and communication,” we found one significant demographic characteristic for improved learning in BSc Degree graduates compared to the Diploma holders. This finding implies that BSc degree holders absorb or value literacy and knowledge related to leadership and communication more. It may also be that BSc holders feel they have a better chance in practicing leadership and having opportunities for leadership positions in the future, thus the improved learning. Other scholars have argued that the Pakistan Nursing Council and Higher Education Commission must ensure additional training, capacity building, and minimum standards for the 3-year Nursing Diploma program (35). There is also concern that unless nurses are supporting in both training and career enhancement it can contribute to nurse stress and poor service delivery (77).

Finally, content analysis of the responses of the intervention participants during the intervention revealed that the majority appreciated the training and opportunity for continued learning through WhatsApp. Approximately one-fifth requested for similar interventions to be repeated and some requested specifically for more training under the area of communication and conflict management. A significant number also remained involved during the intervention in asking questions related to the virus, vaccination, and how to support vulnerable populations. The main feedback for change in the intervention was to ask for group trainings with other healthcare professionals, specifically supervisors. There was also request to include hospital administration in training for infection control in order to mobilize key reform in hospital environment and supply of resources.

Limitations

There are some limitations to this study. The sample size was small and restricted to nurse participants from one city. However, findings may be generalizable for teaching hospitals of category 1 across the country as the health regulatory departments at provincial level follow the same guidelines. Similarly, we hope that training and continued learning reforms in major cities like Lahore will be replicated in other cities. There was a high dropout during the intervention period, mainly due to lack of time and also internet speed issues. Mean results for “COVID-19 knowledge” may not be representative for long-term perceptions of literacy as the epidemiology of the disease evolves. There is scope to repeat this study with a bigger sample, longer intervention period, and to include other healthcare practitioners.

Conclusions

WhatsApp education and learning interventions for frontline healthcare practitioners show promise due to the convenience of at-hand, self-paced, and interactive sessions that uses multiple mediums. Participants in the intervention group of our study reported significant improvement in learning after the intervention for “infection prevention and control,” “COVID-19 knowledge,” as well as “leadership and communication.” Findings add to existing research that nurses require longer periods of training and support for continued learning and not just training that is restricted to a few hours or 2–3 days. It is also important that even during pandemics and physical distancing, we must continue online-based digital learning for practicing frontline workers to support a learning mindset without breaks in skill development and knowledge building. Since nurses with BSc backgrounds show improved learning in “leadership and communication,” we recommend skill development and certificate equivalency for Diploma holding nurses in Pakistan to support their learning ability and overall professional development and care delivery practices.

Our results provide impetus for health educators and practitioners who are planning training and interventions to use WhatsApp and other digital literacy interventions in times of pandemics and other health crises for ongoing and immediate communication. We recommend stakeholders like the health ministry, employers and hospital administration to conduct or outsource training through digital platforms as a more cost effective and time-saving means of continued learning. We also recommend more interdisciplinary and interprofessional online-based training between administration, allied health staff, and senior healthcare practitioners to support continued learning, enhanced communication, and improved patient safety practices. This study also makes a case for bilingual training for nurses. It is important for nurses to receive training and be comfortable in delivering care and talking to their patients in their local language with respect to treatment plan and infection management. Finally, we recommend that WhatsApp health education trainings can become more popular if (i) there are improved options for privacy protection of participants, with group administrators being able to conceal contact information of participants, and (ii) on the completion employers and health institutes can provide collaborative support for participant certificates that are recognized.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The study was reviewed and approved by the Institutional Review Board, Forman Christian College. The participants provided their written informed consent to participate in this study.

Author Contributions

SJ: conceptualization, methodology, and writing—original draft preparation. SJ, AJ, RD, and QM: formal analysis. SJ, NR, SB, RZ, SH, AM, FA, and AS: investigation. RZ, KJW, QM, and FF: writing—review and editing. RZ and FF: supervision. All authors have read and agreed to the published version of the manuscript.

Funding

This project has been funded by Shahid Hussain Foundation. The grant (PKR 400,000/ USD 2,672.26) has been used to pay the nurse intervention facilitators for assistance in sampling and moderation of WhatsApp groups. Twenty nurse WhatsApp moderators/intervention facilitators have been paid PKR 20,000/ USD 131.36 each for this project directly through Shalamar Hospital Finance Office. Principal Investigators of the project did not receive any remuneration for this project. We acknowledge support for the publication costs by the Open Access Publication Fund of Bielefeld University.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We would like to thank all the nurse participants in this study for their generous time and commitment to remaining in the intervention. We would also like to thank the WhatsApp group moderators and other facilitators for this project: Sharoon Rufan, Ali Hamza, Shahrukh Khan, Sulemain Ghouri, Sidratul Muntaha, Sehrish Kiran, Mehtabel Munice, Warda Mahin, Ali Hasnain Bhutta, Sidra Komal, Nabeel Ashraf, Saheefa Johnson, Saba Parveen, Amir Sultan, Sunaina George, Ghani Ullah, Maqooda Sattar, Umar Hayat, Kainat Anwer, Ayesha Ayub, Aberra Naz, and Fatima Majeed. We are also grateful to the nurse practitioners who supported us during the sampling process and discussions for finalization of qualitative findings, including: Maqsooda Athar, Saima Illahi, Parveen Hussain, Talib Shamim, Safdar Ali, Ansab Khalid, Aqsa Vali, Arifa Batool, Faiz Manzoor, Ghaus Munawar, Shaukat Ayaz, and Bushra Sheikh.

We would like to thank Ali Abbas Naqvi for support in proofing video content and editing videos. Finally, we would like to thank our student researchers Amna Maryam and Zara Shah for assistance in data entry and coding.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2022.739761/full#supplementary-material

References

1. Jackson D, Bradbury-Jones C, Baptiste D, Gelling L, Morin K, Neville S, et al. Life in the pandemic: Some reflections on nursing in the context of COVID-19. J Clin Nurs. (2020) 29:2041–3. doi: 10.1111/jocn.15257

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Park C-Y, Kim K, Roth S. Global Shortage of Personal Protective Equipment Amid COVID-19: Supply Chains, Bottlenecks, and Policy Implications. Manila: Asian Development Bank (2020). doi: 10.22617/BRF200128-2

CrossRef Full Text | Google Scholar

3. Haas S, Swan BA, Jessie AT. The impact of the coronavirus pandemic on the global nursing workforce. Nurs Econ. (2020) 38:231–7.

PubMed Abstract | Google Scholar

4. Galehdar N, Toulabi T, Kamran A, Heydari H. Exploring nurses' perception of taking care of patients with coronavirus disease (COVID-19): A qualitative study. J Nursing Open. (2021) 8:171–9. doi: 10.1002/nop2.616

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Ardebili ME, Naserbakht M, Bernstein C, Alazmani-Noodeh F, Hakimi H, Ranjbar H. Healthcare providers experience of working during the COVID-19 pandemic: a qualitative study. Am J Infect Control. (2021) 49:547–54. doi: 10.1016/j.ajic.2020.10.001

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Al Thobaity A, Alshammari F. Nurses on the frontline against the COVID-19 pandemic: an integrative review. Dubai Med J. (2020) 3:87–92. doi: 10.1159/000509361

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Wong CA, Cummings GG, Ducharme L. The relationship between nursing leadership and patient outcomes: a systematic review update. J Nurs Manag. (2013) 21:709–24. doi: 10.1111/jonm.12116

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Schmidt SW. The relationship between satisfaction with workplace training and overall job satisfaction. Hum Res Dev Q. (2007) 18:481–98. doi: 10.1002/hrdq.1216

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Meretoja R, Koponen L. A systematic model to compare nurses' optimal and actual competencies in the clinical setting. J Adv Nurs. (2012) 68:414–22. doi: 10.1111/j.1365-2648.2011.05754.x

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Price S, Reichert C. The importance of continuing professional development to career satisfaction and patient care: meeting the needs of novice to mid-to late-career nurses throughout their career span. Administr Sci. (2017) 7:17. doi: 10.3390/admsci7020017

CrossRef Full Text | Google Scholar

11. Dewart G, Corcoran L, Thirsk L, Petrovic K. Nursing education in a pandemic: academic challenges in response to COVID-19. Nurse Educ Today. (2020) 92:104471. doi: 10.1016/j.nedt.2020.104471

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Huang L, Lin G, Tang L, Yu L, Zhou Z. Special attention to nurses' protection during the COVID-19 epidemic. Crit Care. (2020) 24:120. doi: 10.1186/s13054-020-2841-7

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Wang J, Zhou M, Liu F. Reasons for healthcare workers becoming infected with novel coronavirus disease 2019 (COVID-19) in China. J Hosp Infect. (2020) 105:100–1. doi: 10.1016/j.jhin.2020.03.002

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Gamage B, Moore D, Copes R, Yassi A, Bryce E. Protecting health care workers from SARS and other respiratory pathogens: a review of the infection control literature. Am J Infect Control. (2005) 33:114–21. doi: 10.1016/j.ajic.2004.12.002

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Houghton C, Meskell P, Delaney H, Smalle M, Glenton C, Booth A, et al. Barriers and facilitators to healthcare workers' adherence with infection prevention and control (IPC) guidelines for respiratory infectious diseases: a rapid qualitative evidence synthesis. Cochrane Database Syst Rev. (2020) 4:CD013582. doi: 10.1002/14651858.CD013582

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Liu Y, Wang H, Chen J, Zhang X, Yue X, Ke J, et al. Emergency management of nursing human resources and supplies to respond to coronavirus disease 2019 epidemic. Int J Nurs Sci. (2020) 7:135–8. doi: 10.1016/j.ijnss.2020.03.011

PubMed Abstract | CrossRef Full Text | Google Scholar

17. McNiel PL, Elertson KM. Reality check: preparing nursing students to respond to Ebola and other infectious diseases. Nurs Educ Perspect. (2017) 38:42–3. doi: 10.1097/01.NEP.0000000000000076

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Desroches ML, Ailey S, Fisher K, Stych J. Impact of COVID-19: Nursing challenges to meeting the care needs of people with developmental disabilities. Disabil Health J. (2021) 14:101015. doi: 10.1016/j.dhjo.2020.101015

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Elhadi M, Msherghi A, Alkeelani M, Zorgani A, Zaid A, Alsuyihili A, et al. Assessment of healthcare workers' levels of preparedness and awareness regarding COVID-19 infection in low-resource settings. Am J Trop Med Hyg. (2020) 103:828–33. doi: 10.4269/ajtmh.20-0330

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Chan AK, Nickson CP, Rudolph JW, Lee A, Joynt GM. Social media for rapid knowledge dissemination: early experience from the COVID-19 pandemic. Anaesthesia. (2020) 75:1579–82. doi: 10.1111/anae.15057

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Liu Q, Luo D, Haase JE, Guo Q, Wang XQ, Liu S, et al. The experiences of health-care providers during the COVID-19 crisis in China: a qualitative study. Lancet Glob Health. (2020) 8:e790–8. doi: 10.1016/S2214-109X(20)30204-7

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Nyashanu M, Pfende F, Ekpenyong M. Exploring the challenges faced by frontline workers in health and social care amid the COVID-19 pandemic: experiences of frontline workers in the English Midlands region, UK. J Interprof Care. (2020) 34:655–61. doi: 10.1080/13561820.2020.1792425

PubMed Abstract | CrossRef Full Text | Google Scholar

23. World Health Organization. Nurses and Midwives Critical for Infection Prevention and Control. Geneva: World Health Organization (2020).

Google Scholar

24. Chun HK, Comyn P, Moreno da Fonseca P. Skills Development in the Time of COVID-19: Taking Stock of the Initial Responses in Technical and Vocational Education and Training. Geneva: International Labour Office (2021).

Google Scholar

25. del Carmen Giménez-Espert M, Prado-Gascó V, Soto-Rubio A. Psychosocial risks, work engagement, and job satisfaction of nurses during COVID-19 pandemic. Front Public Health. (2020) 8:566896. doi: 10.3389/fpubh.2020.566896

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Yaghobian M, Farhan R, Navipour H, Vanaki Z. Competencies of charge nurses: a systematic review and thematic synthesis. J Pakistan Med Assoc. (2020) 70:1225–31. doi: 10.5455/JPMA.38686

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Catton H. Nursing in the COVID-19 pandemic and beyond: protecting, saving, supporting and honouring nurses. Int Nurs Rev. (2020) 67:157–9. doi: 10.1111/inr.12593

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Morrison SM, Symes L. An integrative review of expert nursing practice. J Nurs Scholarsh. (2011) 43:163–70. doi: 10.1111/j.1547-5069.2011.01398.x

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Beneria A, Arnedo M, Contreras S, Pérez-Carrasco M, Garcia-Ruiz I, Rodríguez-Carballeira M, et al. Impact of simulation-based teamwork training on COVID-19 distress in healthcare professionals. BMC Med Educ. (2020) 20:515. doi: 10.1186/s12909-020-02427-4

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Hu D, Kong Y, Li W, Han Q, Zhang X, Zhu LX, et al. Frontline nurses' burnout, anxiety, depression, and fear statuses and their associated factors during the COVID-19 outbreak in Wuhan, China: a large-scale cross-sectional study. EClinicalMedicine. (2020) 24:100424. doi: 10.1016/j.eclinm.2020.100424

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Bagcchi S. Stigma during the COVID-19 pandemic. Lancet Infectious Diseases. (2020) 20:782. doi: 10.1016/S1473-3099(20)30498-9

PubMed Abstract | CrossRef Full Text | Google Scholar

32. González-Gil MT, González-Blázquez C, Parro-Moreno AI, Pedraz-Marcos A, Palmar-Santos A, Otero-García L et al. Nurses' perceptions and demands regarding COVID-19 care delivery in critical care units and hospital emergency services. Intensive Crit Care Nurs. (2021) 62:102966. doi: 10.1016/j.iccn.2020.102966

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Darban F, Balouchi A, Narouipour A, Safarzaei E, Shahdadi H. Effect of communication skills training on the burnout of nurses: a cross-sectional study. J Clin Diagn Res. (2016) 10:IC01–4. doi: 10.7860/JCDR/2016/19312.7667

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Gong Z, Van Swol LM Li F, Gilal FG. Relationship between nurse's voice and self-leadership: a time-lagged study. Nurs Open. (2021) 8:1038–47. doi: 10.1002/nop2.711

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Huda SU, Alisbinati ASA. Nursing education in Pakistan: challenges and trends in degree program. Int J Nurs Educ. (2015) 7:59–62. doi: 10.5958/0974-9357.2015.00194.4

CrossRef Full Text | Google Scholar

36. Ahmad J, Anwar S, Latif A, Haq NU, Sharif M, Nauman AA. The association of PPE availability, training and practices with COVID-19 sero-prevalence in nurses and paramedics in tertiary care hospitals of Peshawar, Pakistan. Disaster Med Public Health Prep. (2020) 438:1–5. doi: 10.1017/dmp.2020.438

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Khan S, Khan M, Maqsood K, Hussain T, Zeeshan M. Is Pakistan prepared for the COVID-19 epidemic? A questionnaire-based survey. J Med Virol. (2020) 92:824–32. doi: 10.1002/jmv.25814

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Savul S, Lalani FK, Ikram A, Khan MA, Khan MA, Ansari J. Infection prevention and control situation in public hospitals of Islamabad. J Infect Dev Ctries. (2020) 14:1040–6. doi: 10.3855/jidc.12779

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Saqlain M, Munir MM, Rehman SU, Gulzar A, Naz S, Ahmed Z, et al. Knowledge, attitude, practice and perceived barriers among healthcare workers regarding COVID-19: a cross-sectional survey from Pakistan. J Hosp Infect. (2020) 105:419–23. doi: 10.1016/j.jhin.2020.05.007

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Waris A, Khan AU, Ali M, Ali A, Baset A. COVID-19 outbreak: current scenario of Pakistan. New Microbes New Infect. (2020) 35:100681. doi: 10.1016/j.nmni.2020.100681

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Asif M, Jameel A, Hussain A, Hwang J, Sahito N. Linking transformational leadership with nurse-assessed adverse patient outcomes and the quality of care: assessing the role of job satisfaction and structural empowerment. Int J Environ Res Pub Health. (2019) 16:2381. doi: 10.3390/ijerph16132381

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Jafree SR. Women, Healthcare, and Violence in Pakistan. Oxford: Oxford University Press. (2018).

Google Scholar

43. Jafree SR, Zakar R, Zakar MZ, Fischer F. Nurse perceptions of organizational culture and its association with the culture of error reporting: a case of public sector hospitals in Pakistan. BMC Health Serv Res. (2015) 16:3. doi: 10.1186/s12913-015-1252-y

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Jia Y, Chen O, Xiao Z, Xiao J, Bian J, Jia H. Nurses' ethical challenges caring for people with COVID-19: a qualitative study. Nurs Ethics. (2020) 28:33–45. doi: 10.1177/0969733020944453

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Lim SH, Bouchoucha SL, Aloweni F. Evaluation of infection prevention and control preparedness in acute care nurses: factors influencing adherence to standard precautions. Infect Dis Health. (2021) 26:132–8. doi: 10.1016/j.idh.2020.11.005

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Mayer RE. The promise of multimedia learning: using the same instructional design methods across different media. Learn Instr. (2003) 13:125–39. doi: 10.1016/S0959-4752(02)00016-6

CrossRef Full Text | Google Scholar

47. Marks S, Edwards S, Jerge EH. Rapid deployment of critical care nurse education during the COVID-19 pandemic. Nurse Lead. (2020) 19:165–9. doi: 10.1016/j.mnl.2020.07.008

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Coleman E, O'Connor E. The role of WhatsApp® in medical education; a scoping review and instructional design model. BMC Med Educ. (2019) 19:279. doi: 10.1186/s12909-019-1706-8

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Pereira AAC, Destro JR, Bernuci MP, Garcia LF, Lucena TFR. Effects of a whatsapp-delivered education intervention to enhance breast cancer knowledge in women: mixed-methods study. JMIR Mhealth Uhealth. (2020) 8:e17430. doi: 10.2196/17430

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Huang L, Lei W, Xu F, Liu H, Yu L. Emotional responses and coping strategies in nurses and nursing students during Covid-19 outbreak: a comparative study. PLoS ONE. (2020) 15:e0237303. doi: 10.1371/journal.pone.0237303

PubMed Abstract | CrossRef Full Text | Google Scholar

51. He X-h, Feng Y-r, Li G-m, Pang X-j, Chen T, Zhou Y-l, et al. The impact of goggle-associated harms to health and working status of nurses during management of COVID-19. medRxiv [Preprint]. (2020). doi: 10.1101/2020.05.11.20094854

CrossRef Full Text | Google Scholar

52. Atique S, Bautista JR, Block LJ, Lee JJ, Lozada-Perezmitre E, Nibber R, et al. nursing informatics response to COVID-19: perspectives from five regions of the world. J Adv Nurs. (2020) 76:2462–8. doi: 10.1111/jan.14417

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Bianchi M, Bagnasco A, Bressan V, Barisone M, Timmins F, Rossi S, et al. review of the role of nurse leadership in promoting and sustaining evidence-based practice. J Nurs Manag. (2018) 26:918–32. doi: 10.1111/jonm.12638

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Morsiani G, Bagnasco A, Sasso L. How staff nurses perceive the impact of nurse managers' leadership style in terms of job satisfaction: a mixed method study. J Nurs Manag. (2017) 25:119–28. doi: 10.1111/jonm.12448

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Government of the Punjab. Punjab Health Statistics 2020. Report prepared by Bureau of Statistics Planning and Development Board. Lahore: Government of the Punjab. Available online at: http://bos.gop.pk/system/files/Punjab%20Health%20Statistics%202019-2020.pdf (accessed July 9, 2021).

Google Scholar

56. Rayhan RU, Zheng Y, Uddin E, Timbol C, Adewuyi O, Baraniuk JN. Administer and collect medical questionnaires with Google documents: a simple, safe, and free system. App Med Inform. (2013) 33:12–21.

PubMed Abstract | Google Scholar

57. Dramowski A. Infection Prevention and Control: A Guide for Healthcare Workers in Low-Resource Settings. Cape Town: Bettercare. (2014).

PubMed Abstract | Google Scholar

58. Queensland Health. Interim Infection Prevention and Control Guidelines for the Management of COVID-19 in Healthcare Settings. Brisbane: Queensland Government (2020).

Google Scholar

59. Centers for Disease Control Prevention. Training for Healthcare Professionals. (2019). Available online at: https://www.cdc.gov/coronavirus/2019-ncov/hcp/training.html (accessed July 9, 2021).

Google Scholar

60. World Health Organization. Country and Technical Guidance – Coronavirus Disease (COVID-19). (2019). Available online at: https://www.who.int/emergencies/diseases/novel-coronavirus-2019/technical-guidance (accessed July 9, 2021).

Google Scholar

61. Bass BM, Bass R. Handbook of Leadership: Theory, Research, and Application. New York, NY: Free Press (2008).

Google Scholar

62. Johansen ML. Keeping the peace: conflict management strategies for nurse managers. Nurs Manag. (2012) 43:50–4. doi: 10.1097/01.NUMA.0000410920.90831.96

CrossRef Full Text | Google Scholar

63. Weiss SA, Tappen RM, Grimley K. Essentials of Nursing Leadership and Management. Philadelphia, PA: F A Davis Company. (2010).

Google Scholar

64. Ministry of Education Government of Pakistan. National Education Policy. (2015). Available online at: http://itacec.org/document/2015/7/National_Education_Policy_2009.pdf (accessed July 9, 2021).

Google Scholar

65. Linderman A. Computer content analysis and manual coding techniques: A comparative analysis. In: West MD, editor. Theory, Method, and Practice in Computer Content Analysis. Westport: Ablex Publishing Corp. (2001). p. 97–109.

Google Scholar

66. Degroot AM, Dannenburg L, Vanhell JG. Forward and backward word translation by bilinguals. J Mem Lang. (1994) 33:600–29. doi: 10.1006/jmla.1994.1029

CrossRef Full Text | Google Scholar

67. Krippendorff K. Reliability in content analysis: Some common misconceptions and recommendations. Hum Commun Res. (2004) 30:411–33. doi: 10.1111/j.1468-2958.2004.tb00738.x

CrossRef Full Text | Google Scholar

68. De Benedictis A, Lettieri E, Masella C, Gastaldi L, Macchini G, Santu C, et al. WhatsApp in hospital? An empirical investigation of individual and organizational determinants to use. PLoS ONE. (2019) 14:e0209873. doi: 10.1371/journal.pone.0209873

PubMed Abstract | CrossRef Full Text | Google Scholar

69. International Council of Nurses. The ICN and Code of Ethics for Nurses. Available online at: https://www.icn.ch/sites/default/files/inline-files/2012_ICN_Codeofethicsfornurses_%20eng.pdf (accessed July 9, 2021).

Google Scholar

70. McCutcheon K, Lohan M, Traynor M, Martin D. A systematic review evaluating the impact of online or blended learning vs. face-to-face learning of clinical skills in undergraduate nurse education. J Adv Nurs. (2015) 71:255–70. doi: 10.1111/jan.12509

PubMed Abstract | CrossRef Full Text | Google Scholar

71. Mahmood S, Hasan K, Carras MC, Labrique A. Global preparedness against COVID-19: we must leverage the power of digital health. JMIR Public Health Surveil. (2020) 6:e18980. doi: 10.2196/18980

PubMed Abstract | CrossRef Full Text | Google Scholar

72. Whitehead D. Is there a place for epidemiology in nursing? Nurs Stand. (2000) 14:35. doi: 10.7748/ns2000.07.14.42.35.c2871

PubMed Abstract | CrossRef Full Text | Google Scholar

73. Feldman M, Lacey Krylova V, Farrow P, Donovan L, Zandamela E, Rebelo J, et al. Community health worker knowledge, attitudes and practices towards COVID-19: learnings from an online cross-sectional survey using a digital health platform, UpSCALE, in Mozambique. PLoS ONE. (2021) 16:e0244924. doi: 10.1371/journal.pone.0244924

PubMed Abstract | CrossRef Full Text | Google Scholar

74. Blake H, Bermingham F, Johnson G, Tabner A. Mitigating the psychological impact of COVID-19 on healthcare workers: a digital learning package. Int J Environ Res Pub Health. (2020) 17:2997. doi: 10.3390/ijerph17092997

PubMed Abstract | CrossRef Full Text | Google Scholar

75. Upadhyaya DP, Paudel R, Bromberg DJ, Acharya D, Khoshnood K, Lee K et al. Frontline healthcare workers' knowledge and perception of COVID-19, and willingness to work during the pandemic in Nepal. Healthcare. (2020) 8:554. doi: 10.3390/healthcare8040554

PubMed Abstract | CrossRef Full Text | Google Scholar

76. Prescott K, Baxter E, Lynch C, Jassal S, Bashir A, Gray J. COVID-19: how prepared are front-line healthcare workers in England? J Hosp Infect. (2020) 105:142–5. doi: 10.1016/j.jhin.2020.04.031

PubMed Abstract | CrossRef Full Text | Google Scholar

77. Dos Santos LM. Stress, burnout, and low self-efficacy of nursing professionals: a qualitative inquiry. Healthcare. (2020) 8:424. doi: 10.3390/healthcare8040424

PubMed Abstract | CrossRef Full Text | Google Scholar