A Nationwide Virtual Research Education Program for Medical Students in Pakistan: Methodological Framework, Feasibility Testing, and Outcomes

Introduction: Equipping young medical trainees with fundamental research skills can be a promising strategy to address the need for professionals who can understand and responsibly communicate evolving scientific evidence during a pandemic. Despite an ardent interest to partake in research, most educational institutions in Pakistan and other low-middle income countries have not yet adopted a comprehensive strategy for research skills education. The authors aimed to design and assess the feasibility of implementing the first nation-wide virtual research workshop for medical students in Pakistan.

Methods: The course “Beginners Guide to Research,” designed as a nation-wide virtual research workshop series, was conducted for medical students across Pakistan in June 2020. Four interactive live workshops took place online on alternate days from June 22nd, 2020, to June 27th, 2020, each lasting 1–2 h. Outcomes included: (i) reach, (ii) efficacy as indexed by pre-post change in score pertaining to knowledge and application of research and (iii) self-rated perceptions about understanding of research on a Likert scale.

Results: 3,862 participants enrolled from 41 cities and 123 institutions. Enrolled participants belonged to the following provinces: Sindh (n = 1,852, 48.0%), Punjab (n = 1,767, 45.8%), Khyber Pakhtunkhwa (n = 109, 2.8%), Azad Jammu and Kashmir (n = 84, 2.2%) Balochistan (n = 42, 1.1%). We also saw a few registrations from international students (n = 8, 0.2%). Mean (SD) age of enrolled medical students was 21.1 (2.1) years, 2,453 (63.5%) participants were female and 2,394 (62.0%) were from private-sector medical colleges. Two thousand ninety-three participants participants filled out all four pre-test and post-test forms. The total median knowledge score improved from 39.7 to 60.3% with the highest improvements in concepts of research bioethics and literature search (p < 0.001) with greater change for females compared to males (+20.6 vs. +16.2%, p < 0.001) and private institutions compared to public ones (+16.2 vs. +22.1%, p < 0.001).

Conclusion: The overwhelming enrollment and significant improvement in learning outcomes (>50% of baseline) indicate feasibility of a medical student-led research course during a pandemic, highlighting its role in catering to the research needs in the LMICs.

Introduction

Responsible and effective scientific communication has been one of the key lessons during the pandemic (1). Such communication requires a sound understanding of science, epidemiology, and research in medicine (2). This communication is led by experts and is informed by incoming scientific evidence, which is crucial to disseminate appropriately to the public in a time of crisis like the pandemic and during interactions with patients otherwise. However, over the last three decades, the percentage of physician-scientists has plummeted with numbers going down from 4.7% in the 1980s to 1.5% in the 2010s in the US alone (3, 4). While evidence is lacking for a low-middle income country such as Pakistan, there are only a handful of physician-scientists in the country, which can be attributed to an lack of skills, knowledge, mentorship, and funding support leading to poor understanding of research causing misinformation (5, 6).

To address this challenge and reverse this trend, it is necessary to develop and encourage a research potential early on in budding physician-scientists. This is of pronounced importance in settings like Pakistan, which are plagued with a lack of adequate public health training institutions, lower rates of literacy in the general population and an overall mistrust in the profession of medicine. It is critical to develop and encourage an understanding of scientific research potentially early on in emerging physician-scientists during their medical training (7). This skills development lies at the forefront of the information age, where wide access to evidence-based practice and bench-to-bedside translational research has already led the paradigm shift in teaching and learning (8). Thereby, developing equitable mechanisms for resource-limited settings to equip trainees with research skills is imperative. If left unattended, this deficit will only widen the divide and disparity in science and medicine.

While the importance of developing these research skills within medical students has been well-outlined, many universities in Pakistan have not yet adopted a comprehensive curricular strategy which could have come in very handy during the pandemic (9). Aligning with recent calls for medical students to be involved in the pandemic response (10, 11), the Student Research Forum, an autonomous research body at a private medical university decided to implement an intervention for medical students. The intervention was conceptualized to aid bridging the gap of evidence-based communication. In this paper, we first describe the process our team underwent to design and implement the intervention. We also evaluate the feasibility of this intervention.

Methods

Study Setting

The Student Research Forum (SRF) is a student-run and faculty-supervised research organization registered at the Aga Khan University. The study setting included SRF, the university administrative departments and the SRF Student Ambassador Network. Medical colleges in developing countries such as Pakistan have only recently begun to dedicate their resources toward promoting research within the student community (12). South Asia contributes to only 1.2% of research articles despite being one-fifth of the world's population (13). A cross sectional study conducted in Pakistan showed that a majority of the medical students from either public or private sector universities were keenly interested in medical research and 91% expressed an interest toward having medical research a part of basic medical education (9).

Study Design

A nation-wide virtual research workshop series conducted for medical students across Pakistan was designed as a feasibility study (14, 15). The main objectives of feasibility included (i) the assessment of recruitment capability and resulting sample characteristics, data collection procedures and outcome measures, (ii) acceptability of the intervention and study procedures, resources, and ability to manage and implement the study and intervention, and (iii) preliminary evaluation of participant responses to the intervention (16). Objective outcomes included change in scores (pre-test to post-test) pertaining to knowledge about key research topics and application of research methods. Subjective outcomes involved changes in perceptions toward research on a standard Likert scale.

Sampling Strategy and Study Enrollment

A triphasic enrollment strategy was deployed, which involved marketing and communications on three concurrent fronts: social media (Facebook, Twitter, and Instagram) of SRF. Registrations were done using a Google Form link for 2 weeks prior to the workshop series. The online workshop series was freely accessible to all registered participants, who were provided de-identification codes for the assessment study. Submission of both pre and post questionnaires contributed toward the eligibility criteria for inclusion in analysis. This study was approved by the institutional review board (AKU ERC 2020-5131-11751). Digital informed consent was obtained from all students at the time of enrollment.

Intervention Design and Implementation

“Beginners Guide to Research” was a nation-wide virtual research workshop series conducted for medical students across Pakistan. The strategic planning cycle (Figure 1) focused on course development and refinement, identification of subject experts in the university and development of the logistical and information technology infrastructure. Four interactive workshops were planned, each with specific learning outcomes that were predesigned and scrutinized by experts in research methodology (Table 1).

FIGURE 1

Figure 1. Snapshot of the multifaceted design, implementation process and an overview of the students' research journey.

TABLE 1

Table 1. Summary of the key objectives and format of instruction and assessment employed in the research skills development courses.

The workshop series commenced on the 22nd of June 2020 and continued until the 27th of June 2020. The workshop series consisted of four sessions conducted online using Zoom application Version: 5.0.4 (25694.0524) and Live-Streaming Webinar (Video IBM Channel). Figure 1 comprehensively details the research journey that each individual student underwent throughout the series, with a succinct description of the pertinent learning outcomes and sub-themes. An orientation program ensued to give participants an overview of the course framework.

The duration of each workshop was between 1 and 2 h, which was the duration pre-selected to find the balance between providing speakers enough time to explain basic concepts and the audience adequate and digestible content that did not exceed their online attention span. Concepts outlined in the workshop were kept to a beginner's level of understanding.

Interaction and Communication

To make the workshops interactive and to enable brisk communication, the entire cohort of participants was divided into 16 groups and each group was appointed a team-lead from the organizing team, along with a few other members for troubleshooting and communication purposes. Two platforms were maintained and monitored for incoming questions and correspondence: “Backchannel Chat” and a dedicated email address. Both these platforms were operational in real-time and for the duration of the workshop. This “many-to-some-to-one” strategy enabled participants to reach out to the central team and, by extension, the presenter.

Additionally, some workshops also employed techniques for the session presenters to interact with the audience using the “one-to-many” format by conducting live practice-based demonstrations. For example, the session on “Literature Search” included a live demonstration on PubMed, whereby the participants could both observe the presenter conducting the search in real-time, and simultaneously perform the same on their end also.

Data Collection Tools

The study was evaluated for two domains of feasibility: reach and preliminary efficacy (17). Outreach was determined using the total number of registrations, institutions, and cities across Pakistan.

The questionnaires to assess efficacy had two components:

1. Test Section: This included both scenario-based application of basic research concepts and knowledge-based assessment of fundamental terms and principles. The testing approach was Best Choice Questions (BCQs). Pre-test and post-test questionnaires evaluated participants on their knowledge of research and workshop session objectives, as well as application of concepts. The questions were designed as multiple-choice questions including conventional multiple choice and scenario-based questions.

2. Participants' perceptive rating section: This aimed to evaluate the participants' own perception of their understanding of research sub-topics corresponding to each session assessed on a self-rating scale. A content-relevant Likert's scale was developed to evaluate this, whereby “5” was equivalent to strong agreement with the prompt and “1” was equivalent to strong disagreement. Similar to the test section, participants' perception rating was also collected before and after the workshop.

To ensure reliability of the questionnaire, a pilot run of each test was also conducted. Medical students who were not part of the workshop organizing team and were not registered participants of the workshop were selected for a trial run of the questionnaires from a pre-test perspective. The pilot run was successful and did not require any significant amendments to the questionnaires.

Statistical Analysis Plan

Analysis was done using StataCorp. 2019 (Stata Statistical Software: Release 16. College Station, TX: StataCorp LLC). After the initial registrations were procured, each participant was allocated a unique De-Identification code (D-ID). This step was important to maintain participants' confidentiality and to reduce the risk of researcher bias. To ensure validity and accuracy, only those who filled both pre-test and post-test of a particular workshop were included in the final analysis. Pre-test and post-test scores were calculated overall, for each workshop and for eight distinct categories. Since the data were skewed, median and interquartile range (IQR) were used as a measure of central tendency, and change in medians were calculated for each of the above. The significance in change of scores was calculated using Wilcoxon signed-rank test, and p-values <0.05 were statistically significant.

Results

Reach

The series saw enrollments from a total of 3,862 participants from 41 cities and 123 institutions (Table 2). Enrolled participants belonged to the following provinces: Sindh (n = 1,852, 48.0%), Punjab (n = 1,767, 45.8%), Khyber Pakhtunkhwa (n = 109, 2.8%), Azad Jammu and Kashmir (n = 84, 2.2%), Balochistan (n = 42, 1.1%). We also saw a few registrations from international students (n = 8, 0.2%). In terms of intra-city representation, 1,701 (44.0%) of the participants came from Karachi. Two thousand six hundred ninety-one (69.7%) participants reported having no academic publishing experience prior to registering in the workshop series.

TABLE 2

Table 2. Enrollments, demographics and participant characteristics (n = 3,862).

The mean (SD) age of the registered participants was 21.1 (2.1) years. Two thousand four hundred fifty-three (63.5%) participants were females, 903 (23.4%) were in their first year of medical school training and 2,394 (62.0%) hailed from the private-sector medical institutions of Pakistan.

Preliminary Efficacy

Pre-post Knowledge Scores

The assessment part of the workshop was voluntary. In order of the respective workshops (1 through 4), a total of 2,718, 2,690, 2,525, and 2,480 respondents filled out both pre-tests and post-tests for respective workshops indicating slight decline per session. Overall, a total of 2,093 (54% of total registrations) participants filled out pre-test and post-test questionnaires for all 4 sessions.

Full series' analysis (N = 2,093) showed that the overall median (IQR) score increased from 27 (22–32) to 41 (31–48) (p < 0.001) and the percentage median score, as a proportion of the maximum possible score, improved from 39.7 to 60.3% (Figure 2; Table 3). Similarly, individual workshop scores showed a positive increase in participant scores after attending the workshop (p < 0.001; Table 3). The workshops on literature search along with those on abstract and manuscript writing showed the highest change in scores (Δ Median = +4 for both, Δ% score = +26.7 and +20.0, respectively). The workshops on referencing and introduction to research also showed positive deflections in the participant scores (Δ Median = +3 and +2, Δ % score = +15.8 and +14.3, respectively).

FIGURE 2

Figure 2. Objective assessment: box-and-whisker plots showcasing changes in total scores from baseline pre-workshop aggregates to post-workshop aggregates.

TABLE 3

Table 3. Comparison and univariate analysis of pre-test and post-test knowledge using Multiple Choice Questions (MCQ) format questionnaires.

Analysis by categories showed that the highest improvements in scores were observed in bioethics of research and concepts of literature search (Δ Median = +4, Δ % score = 33.3 for both). Concepts and application of referencing showed the least improvement in scores (Δ Median = +1 and +2, Δ % score = +12.5 and +18.0, respectively).

We found that both females and males started at the same baseline median scores, but females showed a significantly higher improvement in scores (Δ Median = +14 vs. +11, Δ % score = +20.6 vs. +16.2, p < 0.001). Similarly, students at both public and private institutions started at the same baseline scores but the latter performed significantly better in terms of score improvement (Δ Median = +11 vs. +15, Δ % score = +16.2 vs. +22.1, p < 0.001). Although the year of education did not show significant differences in improvement between the categories, an increasing trend in both baseline pre-test and post-test scores was noted as the year of education increased.

Pre-post Self-Rating Scores

On the self-rating scale, the greatest overall red-to-green shift from disagreement (dark red and red) to agreement (dark green and green) was observed for the second (literature search) and third (abstract and manuscript writing) workshops (Figure 3). Similarly, within the sub-themes, the largest red-to-green shift was appreciated in perceptions about Medical Subject Headings (MESH terms) and Boolean operators (both part of the second workshop), whereas the smallest corresponding shift was seen in “research question and hypothesis” and “methodological approach to research,” (both part of the first workshop). Each item showed a positive trend from strong disagreement (dark red) and disagreement (red) to agreement (green) and strong agreement (dark-green).

FIGURE 3

Figure 3. Subjective assessment: categorical assortment of participants' self-perceptions.

Discussion

Our main findings primarily indicate the feasibility of a large-scale national virtual research workshop designed for medical students. The workshop attracted 3,862 registrations across 123 different institutions. Assessment of 2,093 participants indicated an appreciable improvement in overall participant knowledge scores from 39.7 to 60.3% (increase was >50% of baseline) over the course of the series. To our knowledge, this is the first paper to introduce a virtual nation-wide research workshop for medical students in the region in response to the pandemic.

Participants attending our workshop had a baseline assessment score of 39.7%. Many other interventions have implemented and evaluated research skills training at local and regional levels (18–22). A cross-sectional study conducted in India showed the impact of a “Mentored Student Project” in fostering a positive attitude toward student research (23, 24). Within Pakistan, a single-center study in 2006 reported the baseline mean knowledge and attitude scores of medical students regarding research to be 49.0 and 53.7%, respectively (25). Similar to our findings, this study also reported a higher baseline knowledge score on items pertaining to literature search on MEDLINE (60.9%), components of manuscripts (60.4%) and writing of introduction (57.8%), compared to lower scores on themes pertaining to scientific hypothesis (29.4%) and characteristics of research science (23.9%) (25). Another cross-sectional study conducted on 184 medical students in Jeddah, Saudi Arabia reported the cumulative scores for students' knowledge and attitudes regarding participation in scientific research, to be 57 and 76%, respectively (26). The baseline knowledge shown in these studies and our findings collectively demonstrate how critical the need is for a strategically planned research curriculum in the region. However, understanding the baseline status-quo should not be left unattended, but rather be the foundation for a rigorous research program implementation and analysis of post-intervention outcomes.

Due to limitations imposed by the COVID-19 pandemic, our research team leveraged the concurrent ongoing switch to digital technology. Within Pakistan and beyond, there has been a quick uptake of online educational media such as Zoom, Google Hangouts and Skype (27–31). While digital learning and virtual platforms did exist before the pandemic, their use has been small-scale, unstructured, sparse, and unsustainable. Given the rapid uptake of these platforms with the onset of the COVID-19 pandemic, it was possible to reach out to and connect with a large number of participants in real time. It also ensured that costs of transportation, accommodation and logistics to another city were no longer an impediment. This ensured that the workshop remained accessible and free-of-cost to the participants, which further increased the turnout.

Online delivery of educational content brings forth a multitude of advantages, predominantly cost-effectiveness, scalability, replicability and increased accessibility (32–35). Multiple reports, including those from Pakistan, have demonstrated strong support for online learning (36–38), while others have suggested that traditional modalities of learning may be more effective in enhancing knowledge outcomes (39, 40). While it does not compare the two, our study does resonate with the support expressed for online teaching-learning mechanisms. This is seen particularly by the high response rate, significant improvement in key learning metrics and the feedback received throughout the series. Further downstream, post-pandemic, it may be useful to compare the outcomes of such an intervention between virtual and traditional platforms.

An interesting finding from our study was that a large majority of participants in the workshop were females (n = 2,453, 63.5%) and scored significantly higher than their male counterparts (Δ % score = +20.6 vs. +16.2, p < 0.001). This is different from baseline snapshots in Pakistan (25) and Saudi Arabia (41), which have reported male medical students to have a higher attitude score toward research. In contrast, a recent assessment of medical research perceptions amongst medical students in Beirut, Lebanon reports female medical students to have a more positive perception (42). The higher enrollment and better percentage improvement in female students further indicates the need to sustain this drive and to work on developing a greater interest toward research in their male counterparts.

Furthermore, a large majority (n = 1,852, 48.0%) of medical students belonged to the province of Sindh, particularly from Karachi. This can partly be explained by the fact the course was led by a medical school within the city. Also, the city, Karachi has a higher literacy rate (~75%), which is one of Pakistan's largest metropolitan cities (43). Medical students from Punjab, the most populous province of Pakistan, also participated in high numbers (n = 1,767, 45.8%). Data also show that the highest number of registered medical colleges (53%) are from Punjab (44). However, we observed that our enrollments did not significantly reach the less advantaged provinces in other parts of Pakistan like Balochistan where literacy rate is below 10% (43). This serves as a call-to-action, accentuating that a lot more needs to be done to increase access and ensure equity in such programs.

Although our workshop was the first intervention of its kind during the pandemic, the curriculum content and teaching-learning pedagogy were carefully developed and curated by experts in academic medicine. In order to attract a wider target audience from varying diasporas across the country and to transcend geographical barriers, the workshop series was deployed on a virtual platform and was kept free-of-cost. This ensured that a student's socio-economic status did not impact their motivation to partake in this learning opportunity. Planning and strategy development were very much emphasized to ensure that the learning and feedback tools were easily comprehensible to a diverse participant population. In the setting of the COVID-19 pandemic, this novel intervention enabled students, experiencing varying levels of detachment from their institutional educational curriculum, to connect and learn together in the pursuit of acquiring new skill sets. It also ensured that students could productively use their free time, while away from formal, in-person, educational programming. Since research education already faces deficits in medical education curricula, this opportunity guaranteed that students were not disadvantaged due to the pandemic, by further reductions in the training of these important skills.

A major strength of the study was led by medical students themselves at a private medical school in the country. We believe this is an example of student leadership during the pandemic apart from their skill set to be utilized in healthcare to designing courses for peers to further research understanding. The study is encouraging in the sense that medical students in LMIC can also be engaged as a workforce similar to high-income countries (45, 46).

There were similar limitations of the study. A major limitation to our study was that despite a large number of workshop participants (3,862), only 2,093 ultimately chose to be part of the assessment study and filled out all eight pre-tests and post-tests. Since the tests were only open at specific intervals before and after the workshop, this can be partly explained by logistical or technical issues that some participants may have encountered. Electricity outages and problems with internet connectivity (47) might have affected participants in certain parts of the country, as Pakistan still has only 85 million 3G/4G subscribers and 40% internet coverage for a population of 0.2 billion people (48). Due to this, while not everyone was able to participate in the time-limited assessments, we wanted to make sure that no student got left behind in the learning component (our primary objective with the workshops). To tackle this head-on, our team posted summary handouts detailing major points discussed in the workshop. This was done only after the post-test submission was closed, to also ensure that the assessment results were reliable. Moreover, our sessions did not have one-to-one direct interaction between the participants and the presenters. Since brisk communication is a central to any teaching-learning activity (49), we anticipated that this may have interrupted rapport building and concomitantly affected the learning outcomes of a few participants. To counter this, we ensured that 20 organizers were available at every time to answer individual queries and troubleshooting concerns.

Future Implications

As a long-term plan, our team has taken the initiative of establishing a nationwide undergraduate research network. Over 1,000 students from all over the country have expressed interest and sent applications to be research ambassadors at their home institutions. We hope that this workshop series was the steppingstone to establishing the first country wide undergraduate student research network.

Conclusions

This workshop series establishes the first step in the development and feasibility testing of a curated virtual learning platform that has connected the undergraduate community in a low middle-income country. The overwhelming response seen with registrations (N = 3,862), particularly from students logging in from remote and rural areas of Pakistan, highlights the need to develop similar virtual workshops for medical students. It is imperative that this new-found momentum does not erode with time, and similar interventions are systematically launched to make undergraduate research understanding more accessible.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, upon reasonable request.

Ethics Statement

The study was reviewed and approved by Ethics Review Committee, Aga Khan University, Karachi. Digital informed consent was obtained from all students at the time of enrollment.

Author Contributions

AN and MI have co-led the entire workshop series planning and execution at every stage since (and including) conception. They have both equally contributed in the writing of the manuscript and final revisions up until the last internal review. HS, HR, FA, FK, SA, AS, MM, and AM have actively participated in the conception, strategic planning, pilot, execution, data analysis and reporting, and manuscript preparation and final revisions. BH, MR, FJ, MT, and AH acted as expert supervisors and participated in the following: providing regular input during the planning and conception stages, assisting with the execution of the workshop series, reviewing and interpreting data, and preparing the manuscript up till the final revisions. AH and MT have led this series as principal investigator and co-investigator, respectively. All authors have read the final draft and approved it before submission.

Funding

This study was supported by Medical College, Aga Khan University, Karachi, Pakistan.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We would like to acknowledge Dr. Kulsoom Ghias Ph.D., Dr. Saadia Fatima Ph.D., Dr. Naveerah Khan MBBS, Dr. Kaleem Ahmed MBBS, and Mr. Ibrahim Sajid for curating and teaching the course.

References

1. Pollett S, Rivers C. Social media and the new world of scientific communication during the COVID-19 pandemic. Clin Infect Dis. (2020) 71:2184–6. doi: 10.1093/cid/ciaa553

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Zarocostas J. How to fight an infodemic. Lancet. (2020) 395:676. doi: 10.1016/S0140-6736(20)30461-X

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Garrison HH, Deschamps AM. NIH research funding and early career physician scientists: continuing challenges in the 21st century. FASEB J. (2014) 28:1049–58. doi: 10.1096/fj.13-241687

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Feldman AM. the national institutes of health physician-scientist workforce working group report: a roadmap for preserving the physician-scientist. Clin Transl Sci. (2014) 7:289–90. doi: 10.1111/cts.12209

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Khan H, Taqui AM, Khawaja MR, Fatmi Z. Problem-based versus conventional curricula: influence on knowledge and attitudes of medical students towards health research. PLoS ONE. (2007) 2:e632. doi: 10.1371/journal.pone.0000632

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Griffin MF, Hindocha S. Publication practices of medical students at British medical schools: experience, attitudes and barriers to publish. Med Teach. (2011) 33:e1–8. doi: 10.3109/0142159X.2011.530320

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Ashcroft J, Byrne MH V, Brennan PA, Davies RJ. Preparing medical students for a pandemic: a systematic review of student disaster training programmes. Postgrad Med J. (2021) 97:368–79. doi: 10.1136/postgradmedj-2020-137906

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Stone C, Dogbey GY, Klenzak S, Van Fossen K, Tan B, Brannan GD. Contemporary global perspectives of medical students on research during undergraduate medical education: a systematic literature review. Med Educ Online. (2018) 23:1537430. doi: 10.1080/10872981.2018.1537430

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Ejaz K, Shamim MS, Shamim MS, Hussain SA. Involvement of medical students and fresh medical graduates of Karachi, Pakistan in research. J Pak Med Assoc. (2011) 61:115–20.

PubMed Abstract | Google Scholar

10. Iglésias MM. Can students make a difference in science during a pandemic? Porto Biomed J. (2020) 5:e099. doi: 10.1097/j.pbj.0000000000000099

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Bauchner H, Sharfstein J. A bold response to the COVID-19 pandemic: medical students, national service, and public health. JAMA. (2020) 323:1790–1. doi: 10.1001/jama.2020.6166

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Latif MZ, Wajid G. Reforming medical education in Pakistan through strengthening departments of medical education. Pakistan J Med Sci. (2018) 34:1439. doi: 10.12669/pjms.346.15942

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Mushtaq A, Fatima R, Rehman A. Medical students and medical research in Pakistan. J Pak Med Assoc. (2012) 62:527–8.

Google Scholar

14. Eldridge SM, Chan CL, Campbell MJ, Bond CM, Hopewell S, Thabane L, et al. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. BMJ. (2016) 2016:355. doi: 10.1136/bmj.i5239

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Lancaster GA, Thabane L. Guidelines for reporting non-randomised pilot and feasibility studies. Pilot Feasibility Stud. (2019) 5:114. doi: 10.1186/s40814-019-0499-1

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Orsmond GI, Cohn ES. The distinctive features of a feasibility study: objectives and guiding questions. OTJR. (2015) 35:169–177. doi: 10.1177/1539449215578649

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Bowen DJ, Kreuter M, Spring B, Cofta-Woerpel L, Linnan L, Weiner D, et al. How we design feasibility studies. Am J Prev Med. (2009) 36:452–7. doi: 10.1016/j.amepre.2009.02.002

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Cruser des A, Brown SK, Ingram JR, Podawiltz AL, Dubin BD, Colston JS, et al. Learning outcomes from a biomedical research course for second year osteopathic medical students. Osteopath Med Prim Care. (2010) 4:4. doi: 10.1186/1750-4732-4-4

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Hunskaar S, Breivik J, Siebke M, Tømmerås K, Figenschau K, Hansen J-B. Evaluation of the medical student research programme in Norwegian medical schools. A survey of students and supervisors. BMC Med Educ. (2009) 9:43. doi: 10.1186/1472-6920-9-43

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Abdulghani HM, Shaik SA, Khamis N, Al-Drees AA, Irshad M, Khalil MS, et al. Research methodology workshops evaluation using the Kirkpatrick's model: translating theory into practice. Med Teach. (2014) 36 (Suppl. 1):S24–9. doi: 10.3109/0142159X.2014.886012

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Bates I, Akoto AYO, Ansong D, Karikari P, Bedu-Addo G, Critchley J, et al. Evaluating health research capacity building: an evidence-based tool. PLoS Med. (2006) 3:e299. doi: 10.1371/journal.pmed.0030299

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Bates I, Ansong D, Bedu-Addo G, Agbenyega T, Akoto AYO, Nsiah-Asare A, et al. Evaluation of a learner-designed course for teaching health research skills in Ghana. BMC Med Educ. (2007) 7:18. doi: 10.1186/1472-6920-7-18

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Devi V, Ramnarayan K, Abraham R, Pallath V, Kamath A, Kodidela S. Short-term outcomes of a program developed to inculcate research essentials in undergraduate medical students. J Postgrad Med. (2015) 61:163. doi: 10.4103/0022-3859.159315

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Devi V, Abraham RR, Adiga A, Ramnarayan K, Kamath A. Fostering research skills in undergraduate medical students through mentored students projects: example from an Indian medical school. Kathmandu Univ Med J. (2010) 8:294–8. doi: 10.3126/kumj.v8i3.6215

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Khan H, Khawaja MR, Waheed A, Rauf MA, Fatmi Z. Knowledge and attitudes about health research amongst a group of Pakistani medical students. BMC Med Educ. (2006) 6:54. doi: 10.1186/1472-6920-6-54

PubMed Abstract | CrossRef Full Text | Google Scholar

26. AlSayegh AS, Enayah SK, Khoja WN, Enayah RK, Sendi NS. Assessment of the current knowledge and willingness to conduct medical research work of future healthcare providers: a cross-sectional study in Jeddah, Saudi Arabia. J Fam Med Prim care. (2020) 9:1522–7. doi: 10.4103/jfmpc.jfmpc_1047_19

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Dedeilia A, Sotiropoulos MG, Hanrahan JG, Janga D, Dedeilias P, Sideris M. Medical and surgical education challenges and innovations in the COVID-19 era: a systematic review. In vivo. (2020) 34:1603–11. doi: 10.21873/invivo.11950

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Abbasi MS, Ahmed N, Sajjad B, Alshahrani A, Saeed S, Sarfaraz S, et al. E-Learning perception and satisfaction among health sciences students amid the COVID-19 pandemic. Work. (2020) 67:549–56. doi: 10.3233/WOR-203308

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Sandhu P, de Wolf M. The impact of COVID-19 on the undergraduate medical curriculum. Med Educ Online. (2020) 25:1764740. doi: 10.1080/10872981.2020.1764740

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Neupane HC, Sharma K, Joshi A. Readiness for the online classes during COVID-19 pandemic among students of Chitwan Medical College. J Nepal Health Res Counc. (2020) 18:316–319. doi: 10.33314/jnhrc.v18i2.2725

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Nepal S, Atreya A, Menezes RG, Joshi RR. Students' perspective on online medical education amidst the COVID-19 pandemic in Nepal. J Nepal Health Res Counc. (2020) 18:551–5. doi: 10.33314/jnhrc.v18i3.2851

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Alsoufi A, Alsuyihili A, Msherghi A, Elhadi A, Atiyah H, Ashini A, et al. Impact of the COVID-19 pandemic on medical education: medical students' knowledge, attitudes, and practices regarding electronic learning. PLoS ONE. (2020) 15:e0242905. doi: 10.1371/journal.pone.0242905

PubMed Abstract | CrossRef Full Text | Google Scholar

33. George PP, Papachristou N, Belisario JM, Wang W, Wark PA, Cotic Z, et al. Online eLearning for undergraduates in health professions: a systematic review of the impact on knowledge, skills, attitudes and satisfaction. J Glob Health. (2014) 4:010406. doi: 10.7189/jogh.04.010406

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Camargo CP, Tempski PZ, Busnardo FF, Martins M de A, Gemperli R. Online learning and COVID-19: a meta-synthesis analysis. Clinics. (2020) 75:e2286. doi: 10.6061/clinics/2020/e2286

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Sawarynski KE, Baxa DM. Utilization of an online module bank for a research training curriculum: development, implementation, evolution, evaluation, and lessons learned. Med Educ Online. (2019) 24:1611297. doi: 10.1080/10872981.2019.1611297

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Jafri L, Siddiqui I, Khan AH, Tariq M, Effendi MUN, Naseem A, et al. Fostering teaching-learning through workplace based assessment in postgraduate chemical pathology residency program using virtual learning environment. BMC Med Educ. (2020) 20:383. doi: 10.1186/s12909-020-02299-8

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Anwar A, Mansoor H, Faisal D, Khan HS. E-Learning amid the COVID-19 lockdown: standpoint of medical and dental undergraduates. Pakistan J Med Sci. (2021) 37:217–222. doi: 10.12669/pjms.37.1.3124

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Mumtaz N, Saqulain G, Mumtaz N. Online academics in Pakistan: COVID-19 and beyond. Pakistan J Med Sci. (2021) 37:283–7. doi: 10.12669/pjms.37.1.2894

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Kemp N, Grieve R. Face-to-face or face-to-screen? Undergraduates' opinions and test performance in classroom vs. online learning. Front Psychol. (2014) 5:1278. doi: 10.3389/fpsyg.2014.01278

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Hale LS, Mirakian EA, Day DB. Online vs classroom instruction: student satisfaction and learning outcomes in an undergraduate allied health pharmacology course. J Allied Health. (2009) 38:e36–42.

PubMed Abstract | Google Scholar

41. Noorelahi MM, Soubhanneyaz AA, Kasim KA. Perceptions, barriers, and practices of medical research among students at Taibah College of Medicine, Madinah, Saudi Arabia. Adv Med Educ Pract. (2015) 6:479–85. doi: 10.2147/AMEP.S83978

PubMed Abstract | CrossRef Full Text | Google Scholar

42. El Achi D, Al Hakim L, Makki M, Mokaddem M, Khalil PA, Kaafarani BR, et al. Perception, attitude, practice and barriers towards medical research among undergraduate students. BMC Med Educ. (2020) 20:195. doi: 10.1186/s12909-020-02104-6

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Hunter R. Education in Pakistan, Education System Profiles (Asia Pacific) World Education News + Reviews (2020). Available online at: https://wenr.wes.org/2020/02/education-in-pakistan

Google Scholar

44. Medical Colleges Doctors in Pakistan - Too Many or Too Few? Available online at: https://nation.com.pk/21-Sep-2019/medical-colleges-and-doctors-in-pakistan-too-many-or-too-few (accessed October 8, 2020).

Google Scholar

45. Guragai M, Achanta A, Gopez AY, Niyotwambaza J, Cardoso LG, Estavillo NL, et al. Medical students' response to the COVID-19 pandemic: experience and recommendations from five countries. Perspect Biol Med. (2020) 63:623–631. doi: 10.1353/pbm.2020.0051

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Kochis M, Goessling W. Learning during and from a crisis: the student-led development of a COVID-19 curriculum. Acad Med. (2021) 96:399–401. doi: 10.1097/ACM.0000000000003755

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Farooq F, Rathore FA, Mansoor SN. Challenges of online medical education in Pakistan during COVID-19 pandemic. J Coll Physicians Surg Pak. (2020) 30:67–9. doi: 10.29271/jcpsp.2020.Supp1.S67

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Pakistan Telecommunication Authority - Telecom Indicators. Available online at: https://www.pta.gov.pk/en/telecom-indicators (accessed January 19, 2021).

Google Scholar

49. Marshall AL, Wolanskyj-Spinner A. COVID-19: challenges and opportunities for educators and generation z learners. Mayo Clin Proc. (2020) 95:1135–7. doi: 10.1016/j.mayocp.2020.04.015

PubMed Abstract | CrossRef Full Text | Google Scholar