Maria Helena Machado
Sílvia Paredes1,2
Laura Ribeiro- 1Department of Public Health and Forensic Sciences and Medical Education, Faculty of Medicine, University of Porto, Porto, Portugal
- 2Department of Endocrinology, Centro Hospitalar Universitário do Porto, Porto, Portugal
- 3I3S-Instituto de Investigação e Inovação em Saúde, University of Porto, Porto, Portugal
The COVID-19 pandemic had a substantial social, economic, political, and educational impact worldwide. Due to the social contact restrictions, areas such as medical education were highly affected. Assessment in medical education, was already a sensitive topic, and it proved to be even more challenging as different teaching and learning contexts required huge adaptations in a short period. This systematic review provides an overview of the impact of COVID-19 on the assessment of medical students and can serve as a reference to improve this area. We conducted the review based on the PRISMA tool and searched in PubMed, EBSCO, and ScienceDirect. Studies describing the assessment methodologies used during the pandemic were included. Of the 501 initial articles, 18 were included in this review. Collected data was based on the regime, subject, teaching/assessment methodologies, platforms used, grades, students’ and teachers’ perceptions, and measures to prevent academic dishonesty. The results suggest that technology played a central role during the pandemic, and universities were concerned about the transition to online learning regarding teaching and assessment, but students and teachers should be prepared for it. Formative assessment methodologies and feedback were emphasized, and summative tools were adapted to prevent fraud. Students and teachers were generally satisfied with online learning and assessment, which had no significant difference in the examination scores, but they preferred conventional teaching. The COVID-19 pandemic brought an opportunity to analyze and rethink the medical curriculum. Thus, further investigations are needed on combining traditional and online teaching strategies and emphasis on the assessment.
1 Introduction
Due to the rapid spreading of coronavirus (COVID-19) around the world, the World Health Organization (WHO) declared it a global pandemic (WHO, n.d.). This pandemic has had a significant impact on all countries. Governments worldwide trying to control the spread of the infection by implementing lockdowns, social distancing, travel restrictions, and the prohibition of public gatherings (Mittal et al., 2020). This lead to a global shutdown (Feeley et al., 2022), which also included closing schools and moving to online teaching (Hanafy et al., 2021).
Due to these restrictions, areas such as education, namely medical education, were highly affected. Universities needed to make quick decisions and adopt effective measures to allow the continuity of medical education, keeping their standards and adjusting to the environmental and social limitations of the pandemic (Chaudhuri et al., 2022).
Facing the pandemic, medical schools had no alternative but to implement new teaching methodologies, which also constituted an opportunity to improve the progress of medical education (Chaudhuri et al., 2022).
While implementing these changes, major concerns were: to keep students, patients, and healthcare workers safe, to adequately prepare students for professional practice and maintain academic integrity (Adeleke et al., 2020).
So universities quickly adapted to virtual learning and teaching (Feeley et al., 2022) by utilizing online telecommunication platforms (Sani et al., 2020), also called Emergency Remote Teaching (Rahim, 2020).
The use of online learning has been increasing even before the pandemic with the advantage that students can learn anytime and anywhere (Suwannaphisit et al., 2021). Also flexibility, learner-centred, self-directed learning, and cost-effectiveness were crucial online teaching advantages while facing the pandemic (Pathak and Athavale, 2021).
On the other hand, social isolation and technical problems are disadvantages of online education (Pathak and Athavale, 2021).
Most college students in developed countries have access to at least one device that can support their online learning like a desktop computer, laptop, tablet, or smartphone (Fuller et al., 2020).
New teaching methodologies included: online lectures (recorded or live), live meetings, video interpretation (Suwannaphisit et al., 2021), simulation (Mileder et al., 2021; Palés-Argullós and Gomar-Sancho, 2021), Mixed reality (MR) (Minty et al., 2022) and Virtual Classroom Training (VCT) (Nathan et al., 2022).
E-learning (electronic learning) shifted how students learn and medical education is delivered and has become a part of most curricula (Harden and Laidlaw, 2017).
But due to the solid human component and interaction with patients, colleagues, and the team inherent in medicine, it was not easy to continue education. Particularly challenging areas were clinical teaching and the assessment of students near graduation.
Continuous learning and assessments are crucial for the entire educational process of medical education (Mittal et al., 2020).
The trend in medical education, competency-based medical education, emphasizes that the medical curriculum should be developed based on what we want students to demonstrate (Harden and Laidlaw, 2017). An appropriate assessment methodology is at the base of competency-based medical education (Steinemann et al., 2021). This means that assessment methodologies should be based on the learning outcomes/competencies (Harden and Laidlaw, 2017). The pandemic and the need to adapt strategies for enhancing learning through appropriate assessment methodologies constituted an opportunity for shifting the assessment focus toward attaining competencies (Chaudhuri et al., 2022).
Reliability in assessment ensures the consistency and stability of assessment results while validity in assessment ensures the accuracy and truthfulness of the results in measuring the intended construct or outcome. Assessment methodologies should be reliable, consistent, valid, and feasible, and have a positive impact on the student’s learning (Harden and Laidlaw, 2017).
New reliable assessment methodologies that focus on “mastery in clinical reasoning, problem-solving, and decision-making skills” needed to be implemented (Chaudhuri et al., 2022, p. 2).
Assessment is crucial to teaching and learning as it determines if learning objectives have been achieved (Aaraj et al., 2022). But assessment is also “one of the most difficult areas in which to get agreement” (Harden and Laidlaw, 2017, p. 219).
Historically medical education assessment consisted of a written exam sitting in a room at the university with invigilators to prevent academic malpractice (Hope et al., 2021). The COVID-19 pandemic has made traditional medical education examination formats impossible and online assessment was a potential replacement (Sani et al., 2020).
In the article “twelve tips for teaching medical students online under COVID-19,” formative assessment methods like discussion forums, real-time online chat, and communication applications should be emphasized during the lockdown period (Jiang et al., 2021). Different assessment methods can be used to assess the cognitive, skills, and affective domains (Chaudhuri et al., 2022). Knowledge, as a cognitive domain was usually assessed in theory papers (Mittal et al., 2020), while the clinical practice assessment focuses more on skills and goes beyond the cognitive domain to the psychomotor, affective domain, and communication skills (Mittal et al., 2020).
But the switch to online assessment must be carefully implemented based on the available data (Sani et al., 2020) since assessment methodologies should be reliable, consistent, valid, and feasible and positively impact the student’s learning (Harden and Laidlaw, 2017).
The use of online assessment methodologies also brought new concerns and fears about dishonest assessment behaviors and academic integrity (Fuller et al., 2020; Sani et al., 2020), poor feedback, indiscriminate high notes (Steinemann et al., 2021), fairness, and questions of validity and reliability (Aaraj et al., 2022). One of the major challenges of online assessment, in particular, remote electronic exams (E-exams) is academic dishonesty (Elsalem et al., 2021).
Assessment of medical students is a complex theme, and there is still a gap in the literature on this subject. For Hope et al. (2021), research on the student experience on online assessment is an important priority in medical education.
Although some authors refer to the negative effect of the pandemic on medical education (Shaiba et al., 2021), this could also be seen as an opportunity to reflect and learn valuable lessons for the future.
The specific objectives of this review were:
• Explore the assessment methodology used and changes in the assessment methodology in medical schools owing to the pandemic
• Identify the most used pedagogical assessment tools
• Understand students’ and teachers’ perceptions and satisfaction
• Investigate the strategies used to keep fairness and prevent fraud
• Investigate the consequences for the future of medical education
2 Methods
2.1 Search strategy
This systematic literature research and consequent data assessment was conducted following the recommendations of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement (Page et al., 2020), and the protocol for this systematic review was registered on Open Science Framework (OSF).
2.2 Information sources
To achieve appropriate results, three online databases were chosen: PubMed (National for Biotechnology Information, NCIB) [PubMed (nih.gov)], ScienceDirect (Elsevier) [ScienceDirect.com | Science, health and medical journals, full text articles and books.], and EBSCO [CINAHL Database | EBSCO] because they contain the journals that publish on health and medical sciences, including medical education journals and are the most commonly used in systematic reviews on this area.
2.3 Study selection and data collection/extraction process
These three databases were checked for studies published from January 2020 until August 2022. Due to the rapid expansion of literature related to COVID-19, the databases were last accessed on 30.08.2022 to include the latest evidence available on this subject, and each Database was investigated with an appropriate search strategy.
After finishing the first phase of the search strategy, which consisted of the database screening, 582 articles were identified (291 in PubMed, 199 in ScienceDirect, and 92 in EBSCO).
All references and abstracts were collected in a single file (Mendeley software), and duplicates were removed.
After removing duplicates, two independent reviewers screened all the remaining 501 titles and abstracts according to our criteria.
2.4 Eligibility criteria
The inclusion and exclusion criteria were defined using the PICOS tool:
P (population) – undergraduate medical students
I (intervention) – studies that described assessment during COVID-19
C (comparison- comparative interventions) – studies with or without a comparison group
O (Outcomes of interest) – assessment procedure changes, students’ and teachers’ perceptions, feedback, scores, and fraud
(S) (study design) – qualitative, quantitative, and mixed studies
Studies were excluded based on: population (not medical students – n = 170), time (studies with a duration longer than 3 years – n = 8), type of document (n = 19), and being outside of scope (n = 261) – just referring to the COVID-19 pandemic, not covering undergraduate medical students and not fostering the assessment.
The remaining 43 full-text articles were assessed by two reviewers independently for eligibility. 25 papers were excluded for failing to meet the inclusion criteria [out of scope (n = 15) and poor description of the assessment methodology (n = 10)].
Finally, the remaining 18 articles were included in this systematic review.
2.5 Data coding and extraction
Data coding and extraction were conducted independently and documented on a pre-made data extraction form.
In addition to basic information about the study and population, articles were analyzed based on the medical year of students, subject, and regime (online or in-person teaching). In the case of online teaching and assessment, the platforms used were also identified.
It was also essential to identify the teaching methodology adopted by the different universities in the context of COVID-19 and if it also changed and how.
Regarding the assessment, information was collected based on the assessment methodology and tools adopted and if there was an adjustment due to the coronavirus contact restrictions and how these occurred.
Studies that compared student grades before and after COVID-19 also analyzed and documented if grade adjustment occurred.
Summative and formative assessment strategies were identified, as well as the assessed domains (when described).
In such a quick transition, where students’ lives were affected in many aspects, it was also important to know students’ and teachers’ perceptions.
Another question of this review was if the universities took measures to prevent academic malpractice in terms of invigilated/non-invigilated examination, the adaption of the assessment methodology, and the establishment or not of one assessment committee.
The evaluation regarding grades or feedback practices was also an exciting analysis point.
3 Results
The Flow diagram below (Figure 1) illustrates the literature search, selection strategy, and articles included on this review.
Figure 1. PRISMA flow diagram illustrating the literature search, selection strategy, and articles included on this review (Page et al., 2020).
3.1 Selected studies characteristics
The final 18 articles included were published between 2020 and 2022 and the highest publishing rate was in 2021 (n = 10).
Most studies were conducted in the United Kingdom (n = 4) and Saudi Arabia (n = 4), followed by the USA (n = 3).
Regarding the study design, the majority were cross-sectional studies (n = 9).
Regarding Quartile (2021), from the 18 articles included, n = 4 were published in Q1 journals, n = 7 in Q2 and the remaining 7 articles were published in Q3 and Q4 journals.
3.2 Population
The population sample in all the studies was undergraduate medical students in different stages of medical school.
Three of the included studies also analyzed teachers’ perceptions of online teaching and assessment (Elzainy et al., 2020; Hanafy et al., 2021; Shaiba et al., 2021).
The studies included had different sample sizes from n = 9 (Blythe et al., 2021) to n = 8,094 (Hope et al., 2021).
Age and student gender were not always mentioned in the studies. From the 6 studies where age was mentioned, the mean age of the participants was 23 years old (Co et al., 2021; Hanafy et al., 2021; Kronenfeld et al., 2021; Suwannaphisit et al., 2021; Co and Chu, 2022; Nathan et al., 2022).
Concerning gender, it was only mentioned in 5 of the 18 studies (Co et al., 2021; Hanafy et al., 2021; Suwannaphisit et al., 2021; Co and Chu, 2022) (Nathan et al., 2022). In three, most participants were male, while in the other two, the majority were female.
Of the 18 studies, four did not report the student’s academic year (Alkhowailed et al., 2020; Prigoff et al., 2021; Yuda Handaya et al., 2021; Minty et al., 2022). Students were also in different phases of the medical curriculum. Most studies are based on final-year medical students (n = 5), followed by first-year medical students (n = 4). No investigation was conducted explicitly with 2nd and 4th-year medical students. One study included medical students from all year groups (Nathan et al., 2022), and another had students from the 1st and 3rd academic years (Elzainy et al., 2020).
3.3 Regime
Eleven studies were conducted in an online learning environment (Alkhowailed et al., 2020; Elzainy et al., 2020; Blythe et al., 2021; Co et al., 2021; Hernandez et al., 2021; Kronenfeld et al., 2021; Prigoff et al., 2021; Shaiba et al., 2021; Yuda Handaya et al., 2021; Chaudhuri et al., 2022; Minty et al., 2022), while the other 7 had a mix of online learning and face to face encounters (especially with students on clinical practice academical years) (Adeleke et al., 2020; Hanafy et al., 2021; Hope et al., 2021; Suwannaphisit et al., 2021; Aaraj et al., 2022; Co and Chu, 2022; Nathan et al., 2022). As expected, because of the restrictions imposed by the pandemic, no study had a just face-to-face traditional teaching methodology.
3.4 Teaching methodologies
In all of the 18 studies, universities decided to change the regime to online, and in particular, in cases where they could not precinct of clinical practice, students ended up doing it but in a shorter period (Adeleke et al., 2020; Suwannaphisit et al., 2021; Aaraj et al., 2022).
The teaching methodologies adopted mentioned in the articles were: online classes (Elzainy et al., 2020; Shaiba et al., 2021; Suwannaphisit et al., 2021; Yuda Handaya et al., 2021; Aaraj et al., 2022), flipped classroom (Chaudhuri et al., 2022), interactive sessions (Hanafy et al., 2021; Prigoff et al., 2021; Co and Chu, 2022), tutorial video (Prigoff et al., 2021; Yuda Handaya et al., 2021), seminars (Hanafy et al., 2021), online PBL (Alkhowailed et al., 2020; Elzainy et al., 2020; Kronenfeld et al., 2021) and laboratory sessions (Hanafy et al., 2021).
3.5 Subject
All the subject contents mentioned in the studies were part of the medical curriculum.
Five studies did not mention a subject content (Alkhowailed et al., 2020; Elzainy et al., 2020; Blythe et al., 2021; Hope et al., 2021; Minty et al., 2022).
39% of the studies focused mainly on the Surgical area (Co et al., 2021; Kronenfeld et al., 2021; Prigoff et al., 2021; Yuda Handaya et al., 2021; Co and Chu, 2022; Nathan et al., 2022), Medicine, Obstetric and Paediatrics (Shaiba et al., 2021; Aaraj et al., 2022), Orthopedics (Suwannaphisit et al., 2021), Family Medicine (Adeleke et al., 2020), Anatomy (Hernandez et al., 2021), Physiology (Chaudhuri et al., 2022) and Human Body Course (Hanafy et al., 2021).
3.6 Length of the intervention
The length of the intervention was quite variable in the 18 studies.
In 5 studies, the duration of the intervention was not mentioned (Adeleke et al., 2020; Blythe et al., 2021; Co et al., 2021; Prigoff et al., 2021; Minty et al., 2022).
The shortest interventions were just single sessions (Hope et al., 2021; Shaiba et al., 2021; Co and Chu, 2022), and the most extended intervention took 23 months (Suwannaphisit et al., 2021).
3.7 Comparative interventions
55% of the articles compared two types of interventions.
16% had an Intervention Group (IG) and Control Group (CG) occurring at the same time, usually to compare the effectiveness of online and face-to-face teaching methodologies (Elzainy et al., 2020; Yuda Handaya et al., 2021; Co and Chu, 2022). The other 39% compared the data during the COVID-19 pandemic with a different time (usually before Covid) in terms of, for example, student grades (Hernandez et al., 2021; Kronenfeld et al., 2021; Prigoff et al., 2021; Aaraj et al., 2022) or during the lockdown and after lockdown to analyzed students perception and satisfaction (Co et al., 2021; Hanafy et al., 2021; Hope et al., 2021).
3.8 Assessment tools
The 18 articles in this systematic review identified a considerable diversity of assessment tools (summative and formative). Figure 2 provides an overview of the assessment tools mentioned in the studies.
Figure 2. Assessment tools mentioned in the studies. (Adeleke et al., 2020; Alkhowailed et al., 2020; Elzainy et al., 2020; Hanafy et al., 2021; Blythe et al., 2021; Co et al., 2021; Hernandez et al., 2021; Hope et al., 2021; Kronenfeld et al., 2021; Prigoff et al., 2021; Shaiba et al., 2021; Suwannaphisit et al., 2021; Yuda Handaya et al., 2021; Aaraj et al., 2022; Chaudhuri et al., 2022; Minty et al., 2022; Nathan et al., 2022).
3.9 Assessed domains
Only 3 studies explicitly mentioned the domains assessed: cognitive (Co and Chu, 2022) and skills (Co et al., 2021; Yuda Handaya et al., 2021; Co and Chu, 2022).
3.10 Platforms used
Zoom was the most used of the online learning and assessment platforms mentioned in the articles, with 50%. Other platforms mentioned were Blackboard, Youtube, Whatsapp, Moodle, Google Classroom, and BARCO.
3.11 Feedback
Five studies mentioned oral feedback given after the examination (Adeleke et al., 2020; Alkhowailed et al., 2020; Blythe et al., 2021; Hanafy et al., 2021; Chaudhuri et al., 2022).
3.12 Students and teachers’ perception
Figure 3 summarizes students’ and teachers’ perception of online teaching and assessment.
Figure 3. Students’ and teachers’ perceptions of online teaching and assessment.
3.13 Measures to prevent dishonesty
Seven of the included studies (39%) mentioned a preoccupation to prevent cheating and fraud due to different strategies adopted by medical schools to prevent dishonest behaviors:
• Students were asked to keep their cameras on throughout the exam (Shaiba et al., 2021);
• Problem-based MCQ with a time limit to answer and only one attempt. Time was recorded, and the link for the exam was only open for 30 min (Chaudhuri et al., 2022)
• Short answer type questions were problem-based, and 15 questions should be sent in 1 h (Chaudhuri et al., 2022)
• The oral examination had a stipulated time (Chaudhuri et al., 2022)
• Case scenarios were frequently changed (Shaiba et al., 2021);
• Assessment based on the student’s performance during PBL sessions rather than MCQ (associated with higher chances of cheating) (Elzainy et al., 2020)
• Open book exam (Prigoff et al., 2021)
• Oral portfolio (Adeleke et al., 2020)
• Plagiarism check of the submitted documents (Adeleke et al., 2020)
• Each student completed an honour code attestation (Hernandez et al., 2021)
• Students turned their cameras downwards to allow the teacher to see their hands (Nathan et al., 2022)
3.14 Invigilation
Three of the included studies mentioned a webcam invigilated format (Hope et al., 2021; Shaiba et al., 2021; Nathan et al., 2022).
3.15 Electronic assessment committee
To implement online assessment three of the universities established one electronic assessment committee (Alkhowailed et al., 2020; Elzainy et al., 2020; Shaiba et al., 2021) to: revise online tests, help to solve any technical obstacle during the online exam, and analyze the results of online assessments (Elzainy et al., 2020).
A digitalization committee was created to digitalize the teaching activities and construct digital evaluation and feedback sheets (Alkhowailed et al., 2020).
3.16 Grading adjustment
Only one study referred grading adjustment due to the assessment methodology change, and students were informed about this procedure before the exam (Prigoff et al., 2021).
3.17 Pass grades, honours, high-pass
Just one study referred that pass grades, honours, and high-pass remained the same (Prigoff et al., 2021).
3.18 Study quality and risk of bias assessment
The quality of the publications was assessed based on the Quartile of the journal found on the journal’s website or InCites Journal Citation Reports (JCR) [Journal Citation Reports – Home (clarivate.com)].
4 Discussion
The restrictions imposed by governments to control the pandemic left medical schools with no other option than adapt and rearrange teaching and examination strategies.
The results of this systematic review suggest that universities were concerned about the transition from remote to online teaching in terms of teaching and assessment.
The speed with which solutions needed to be taken was challenging for universities, teachers, and students. Studies conducted at the beginning of the pandemic reflect more challenges in the transition to online learning, which is understandable, considering the short time schools had to prepare.
Initially, one of the first measures imposed by the government was the transition to online learning, and students also left clinical placements. According to the different pandemic phases, universities rearranged the teaching methodologies, and subjects who demanded patient contact were either delayed, replaced by simulation, or conducted in a different environment with fewer students and with the implementation of measures to prevent the spread of infection.
Technology played a central role in teaching, and platforms like Zoom, Blackboard, Whatsapp, Youtube, Moodle, and Google Classroom were used more than ever. Already in 2008, Ellaway and Masters (2008) emphasized the important role of technology not only in supporting teaching and learning but also in formative and summative assessment.
Remtulla (2020) also emphasized the importance of Technology on medical education not only during COVID-19 but the possibility of an upcoming revolution on medical education due to the increasing use of technology tools (Remtulla, 2021).
Regarding the evaluation, formative assessment methodologies (attending classes, more case-based questions, PBL) and feedback were emphasized. Feedback plays an even more important role in online education because it ensures student development keeping them informed of their progress (Masters et al., 2022).
Summative assessment tools like Objective Structured Examination and MCQ continued to be predominantly used but with some arrangements.
The OSCEs took an online format with simulated patients. The impossibility of face-to-face teaching also underlined the importance of technology, for example, video to assess students’ practice skills. The possibility to record and create an online repository of students’ performance was seen as an advantage for students’ progress.
And the questions in general (multiple-choice, short answer type, essay questions) were clinical scenario based to avoid fraud. There was a growing use of oral examination, for example, with oral portfolio discussion. Also, the open-book examination was put into perspective, and new means to make learning more attractive and interactive, like weekly quizzes were implemented.
Another concern of the studies was understanding the impact of the pandemic and the changes in medical education on students’ and teachers’ satisfaction and well-being.
In terms of comparing online and face-to-face learning and assessment, online methodologies were generally well accepted by both students and teachers, recognizing that it was the best way to keep medical education ongoing. In general, students were satisfied and positively responded to online learning and assessment.
In one study, students perceived stress decreased and immediate feedback after the online examination and regular feedback contributed to coping better with online teaching (Chaudhuri et al., 2022). A study including students from 20 different universities in Spain also showed a better perception of online learning directly associated to the professor-student interactions (Arco et al., 2021). Also in medical universities in Jordan, students had a significant higher satisfaction with online learning when instructors actively participated on the online sessions (Al-Balas et al., 2020).
In terms of reliability and validity, studies showed no significant difference between conventional and online assessment methodologies used during COVID-19 pandemic.
In terms of pass grades, honours, high-pass, and examination scores, almost all of the studies showed either no difference compared to the previous academic year scores with traditional learning or an increase in the mean scores with online assessment.
Also, the two studies with virtual OSCE students performed comparably to real patient scenarios (Shaiba et al., 2021; Minty et al., 2022).
Hope et al. (2021) found no gap between students’ performance undertaking traditional examinations and online examinations. Hernandez et al. (2021) also concluded that there was no degradation in students’ performance due to the transition from face-to-face to online learning. Sometimes was detected an improvement in students’ grades (Hernandez et al., 2021) and one possible explanation for this performance improvement could be that the students had more time to invest in their studies (Hernandez et al., 2021).
In just one of the studies, a grading adjustment was mentioned. The Columbia University College of Physicians and Surgeons planned a grading adjustment to avoid higher scores due to the assessment methodology change to an open-book exam where they used the historic grades and deducted the difference in median scores between the previous and current students (Prigoff et al., 2021). Students were informed about this procedure before and after the exam (Prigoff et al., 2021).
One study reflected a lower satisfaction among older students with the online assessment, which could be explained by not being comfortable with the use of technology (Shaiba et al., 2021). A study developed in Spain at the University of Cordova and the University of Lleida with the objective to identify barriers that students experienced during online learning also concluded that some students who did not have their own technology devices or had to share them with family members experienced more difficulties following the courses (Pla and Arco, 2023).
Studies also recognized that both staff and students should be prepared for online learning and assessment. The fact that technology is part of our days does not mean that every student is familiar with it (Ribeiro et al., 2016; Masters et al., 2022).
A study conducted with lectures from Catalan universities aimed to analyze the Training Programs and Perception of Impact on Teaching Practices before and after the pandemic and showed an increase importance of e-learning training for university professors which includes training with digital tools, online assessment and design of online courses (Ramos-pla and Arco, 2021).
Both students and teachers showed agreement in favor of online examinations allowing immediate feedback, but they also agreed with the higher risk of dishonesty (Hanafy et al., 2021).
Accurate and valid assessment has also been a concern of medical schools, especially when it happens remotely (Masters et al., 2022).
Studies that mentioned a preoccupation to prevent fraud described that medical schools adopted different strategies like: the foundation of an electronic assessment committee, changes in the assessment methodology/assessment tools, new rules for electronic examination, and invigilation (Alkhowailed et al., 2020; Elzainy et al., 2020; Hope et al., 2021; Shaiba et al., 2021; Chaudhuri et al., 2022; Nathan et al., 2022).
One of the articles explored open-book examinations to prevent dishonesty, but they did not have high acceptance among students. This corroborates the literature that for an open-book examination to be successful, there are many issues to be considered, like adapting the type of questions (Masters et al., 2022).
Generally, conventional teaching and assessment were perceived as more effective, accessible, with fewer technical difficulties and less fraud and cheating.
Although students generally well accepted online teaching during the pandemic, survey results suggest they would not recommend keeping electronic assessments in the future.
4.1 Strengths and limitations
Despite the interesting findings, the results can only be interpreted considering some limitations. Due to the emergency, medical schools were concerned about finding solutions to keep medical education, and the main focus was not the investigation and publication. This, associated with the complexity of a subject like assessment in medical education, could underlie the existing gap in information on this subject.
In some cases, it was not possible to directly compare studies because study design, length, and focus varied immensely.
The sample size also differed a lot between studies, and most of the included studies had a very small sample size.
The studies included in this systematic review were conducted in diverse cultures and COVID-19 pandemic stages with different restrictions. Some studies compared grades before and after the pandemic. Still, the circumstances were different since, in many cases, students had already started with one assessment type and quickly transitioned to another.
Articles written in 2020 were based on a description of the measures taken by the universities in response to the pandemic, predominantly based on online methodologies, while future studies varied on the regime, often a mix of online and in-person methods.
Despite all of that, this systematic review also has important strengths.
After searching the literature, a gap in this field was identified, and it confirmed that is no other systematic review or ongoing study with a similar research question.
This study focuses on the assessment of undergraduate medical students. This has always been an exciting and controversial issue in higher education. Through the analysis of medical education during the COVID-19 pandemic, important lessons can be taken in the future.
4.2 Recommendations/suggestions for future research
Further investigations could research the effectiveness of combining online and conventional learning and assessment methods in medical education (Hanafy et al., 2021).
In some studies, like in the College of Medicine King Saud University, students and teachers were satisfied with the alternative found to keep medical school examinations like e-OSCE during the circumstances of the pandemic (Shaiba et al., 2021).
It would be interesting to know if there was the possibility of traditional OSCE and e-OSCE which one they would prefer.
Students’ and teachers’ perceptions of the assessment methodologies used during the pandemic were analyzed in a situation with no alternative. It would be interesting to know in other circumstances than the pandemic if the perception (especially in cases where it was positive) would remain the same.
The tendency for dishonest behaviors related to online assessment methodologies could also be explored, as well as the measures to prevent it.
5 Conclusion
After conducting this systematic review is possible to conclude that the COVID-19 pandemic and, consequently, the measures imposed by the governments to avoid the spread of the virus severely affected medical education worldwide since traditional medicine teaching happens not just in the classroom but also in the bedside.
The complexity of the assessment and the deficit of information on this area was once again confirmed by this review. However, some general conclusions could be taken.
The pandemic significantly impacted the development and improvement of diversified assessment methodologies and thus constituted an opportunity for reflection, learning, and evolution in medical education assessment.
The transition to online learning required the adaptation of the teaching and assessment methodologies and adopting measures to prevent dishonesty behaviors. The importance of medical students’ assessment was deepened, and the concern of the universities with this theme was evident, for example, through the establishment of evaluation committees. Formative assessment methodologies’ advantages were also highlighted, with feedback assuming a crucial role.
These solutions were generally well accepted by both students and teachers since there was no alternative but some advantages of these methodologies that could be seen as a positive change for the future of medical education, complementing the best of conventional and online teaching strategies.
In the global world that we are living in, technology has a significant influence on health sciences and medical care. Still, the utilization of these technologies in medical teaching has the potential to transform medical education. The investment in technology-supported assessment today may be an opportunity to improve medical teaching assessment and be prepared in case of future worldwide disruptions. However, for online teaching to work, universities, teachers and students must be prepared for it.
In conclusion, despite the challenges for medical education under severe contact restrictions, COVID-19 also brought an opportunity to analyze and rethink the medical curriculum.
Author contributions
MM: Conceptualization, Investigation, Writing – original draft, Data curation, Methodology. SP: Conceptualization, Investigation, Methodology, Writing – review & editing. LR: Conceptualization, Investigation, Methodology, Writing – review & editing, Supervision.
Funding
The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.
Conflict of interest
This systematic review was developed in the context of a master’s course on Academic and Clinical Education at the Faculty of Medicine of the University of Porto.
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Aaraj, S., Farooqi, F., Saeed, N., and Khan, S. (2022). Impact of COVID pandemic and hybrid teaching on final year MBBS students’ end of clerkship exam performance [internet]. Pak. J. Med. Sci. 38, 113–117. doi: 10.12669/pjms.38.1.4645
Adeleke, O. A., Cawe, B., and Yogeswaran, P. (2020). Opportunity for change: undergraduate training in family medicine. South African Fam Pract. 62, 1–3. doi: 10.4102/safp.v62i1.5225
Al-Balas, M., Ibrahim Al-Balas, H., Jaber, H. M., Obeidat, K., Al-Balas, H., Aborajooh, E. A., et al. (2020). Distance learning in clinical medical education amid COVID-19 pandemic in Jordan: current situation, challenges, and perspectives. BMC Med. Educ. 20:341. doi: 10.1186/s12909-020-02257-4
Alkhowailed, M. S., Rasheed, Z., Shariq, A., Elzainy, A., El Sadik, A., Alkhamiss, A., et al. (2020). Digitalization plan in medical education during COVID-19 lockdown. Inform. Med. Unlocked 20:100432. doi: 10.1016/j.imu.2020.100432
Arco, I., Flores, Ò., and Ramos-pla, A. (2021). Structural model to determine the factors that affect the quality of emergency teaching, according to the perception of the student of the first university courses. Sustainability 13:2945. doi: 10.3390/su13052945
Blythe, J., Patel, N. S. A., Spiring, W., Easton, G., Evans, D., Meskevicius-Sadler, E., et al. (2021). Undertaking a high stakes virtual OSCE (“VOSCE”) during COVID-19. BMC Med. Educ. 21:221. doi: 10.1186/s12909-021-02660-5
Chaudhuri, A., Mondal, T., Paul, S., and Goswami, A. (2022). A study of performance of first MBBS students in internal assessment conducted online during the COVID-19 pandemic. Med. J. Dr DY Patil Vidyapeeth 15:555. doi: 10.4103/mjdrdypu.mjdrdypu_506_20
Co, M., and Chu, K. M. (2022). A prospective case-control study on online teaching of ultrasonography skills to medical students during COVID-19 pandemic. Heliyon 8:e08744. doi: 10.1016/j.heliyon.2022.e08744
Co, M., Chung, P. H. Y., and Chu, K. M. (2021). Online teaching of basic surgical skills to medical students during the COVID-19 pandemic: a case–control study. Surg. Today 51, 1404–1409. doi: 10.1007/s00595-021-02229-1
Ellaway, R., and Masters, K. (2008). AMEE guide 32: E-learning in medical education part 1: learning, teaching and assessment. Med. Teach. 30, 455–473. doi: 10.1080/01421590802108331
Elsalem, L., Al-Azzam, N., Jum’ah, A. A., and Obeidat, N. (2021). Remote E-exams during COVID-19 pandemic: a cross-sectional study of students’ preferences and academic dishonesty in faculties of medical sciences. Ann. Med. Surg. 62, 326–333. doi: 10.1016/j.amsu.2021.01.054
Elzainy, A., El Sadik, A., and Al, A. W. (2020). Experience of e-learning and online assessment during the COVID-19 pandemic at the College of Medicine, Qassim University. J. Taibah Univ. Med. Sci. 15, 456–462. doi: 10.1016/j.jtumed.2020.09.005
Feeley, A., Feeley, I., Healy, E., Sheehan, E., and Hehir, D. J. (2022). Acquisition of surgical skills in medical students via telementoring: a randomized control trial. J. Surg. Res. 270, 471–476. doi: 10.1016/j.jss.2021.10.007
Fuller, R., Joynes, V., Cooper, J., Boursicot, K., and Roberts, T. (2020). Could COVID-19 be our ‘there is no alternative’ (TINA) opportunity to enhance assessment? Med. Teach. 42, 781–786. doi: 10.1080/0142159X.2020.1779206
Hanafy, S. M., Jumaa, M. I., and Arafa, M. A. (2021). A comparative study of online learning in response to the coronavirus disease 2019 pandemic versus conventional learning. Saudi Med. J. 42, 324–331. doi: 10.15537/smj.2021.42.3.20200741
Harden, RM, and Laidlaw, JM. Essential skills for a medical teacher – an introduction to teaching and learning in medicine. 2 China: Elsevier (2017).
Hernandez, T., Fallar, R., and Polydorides, A. D. (2021). Outcomes of remote pathology instruction in student performance and course evaluation. Acad. Pathol. 8:23742895211061822. doi: 10.1177/23742895211061822
Hope, D., Davids, V., Bollington, L., and Maxwell, S. (2021). Candidates undertaking (invigilated) assessment online show no differences in performance compared to those undertaking assessment offline [internet]. Med. Teach. 43, 646–650. doi: 10.1080/0142159X.2021.1887467
Jiang, Z., Wu, H., Cheng, H., Wang, W., Xie, A., and Fitzgerald, S. R. (2021). Twelve tips for teaching medical students online under COVID-19. Med. Educ. Online 26:1854066. doi: 10.1080/10872981.2020.1854066
Kronenfeld, J. P., Ryon, E. L., Kronenfeld, D. S., Hui, V. W., Rodgers, S. E., Thorson, C. M., et al. (2021). Medical student education during COVID-19: electronic education does not decrease examination scores. Am. Surg. 87, 1946–1952. doi: 10.1177/0003134820983194
Masters, K., Taylor, D., Loda, T., and Herrmann-Werner, A. (2022). AMEE guide to ethical teaching in online medical education: AMEE guide no. 146. Med. Teach. 44, 1194–1208. doi: 10.1080/0142159X.2022.2057286
Mileder, L. P., Bereiter, M., and Wegscheider, T. (2021). Telesimulation as a modality for neonatal resuscitation training. Med Educ. Online 26:1892017. doi: 10.1080/10872981.2021.1892017
Minty, I., Lawson, J., Guha, P., Malik, R., Cerneviciute, R., Kinross, J., et al. (2022). The use of mixed reality technology for the objective assessment of clinical skills: a validation study [Internet]. BMC Med. Educ. 22:639. doi: 10.1186/s12909-022-03701-3
Mittal, SK, Kumari, R, Kathrotia, R, Bisht, M, Mittal, S, Patil, PM, et al. Changing paradigm of practical assessment modalities in medical education in the era of COVID-19: a cross-sectional study [Internet]. J Med Evid. (2020). Available at: https://essentials.ebsco.com/search/eds/details/changing-paradigm-of-practical-assessment-modalities-in-medical-education-in-the-era-of-covid-19-a?query=AND assessment AllFields AND online AllFields AND medicine AllFields AND education AllFields AND covid
Nathan, A., Fricker, M., Georgi, M., Patel, S., Hang, M. K., Asif, A., et al. (2022). Virtual interactive surgical skills classroom: a parallel-group, non-inferiority, adjudicator-blinded, randomised controlled trial (VIRTUAL). J. Surg. Educ. 79, 791–801. doi: 10.1016/j.jsurg.2021.11.004
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., et al. (2020). Statement: an updated guideline for reporting systematic reviews. BMJ 372:372. doi: 10.1136/bmj.n71
Palés-Argullós, J., and Gomar-Sancho, C. (2021). La educación médica en el Grado en Medicina en época de pandemia y de post-pandemia COVID-19. Rev. Med. Cine. 16, 13–23. doi: 10.14201/rmc202016e1323
Pathak, A. S., and Athavale, S. A. (2021). Learning amidst the COVID-19 pandemic- a medical student’s perspective. Int. J. Anat. Radiol. Surg. 2019, 2019–2021.
Pla, A. R., and Arco, I. (2023). Training, personal and environmental barriers of online education. Educar 59, 457–471.
Prigoff, J., Hunter, M., and Nowygrod, R. (2021). Medical student assessment in the time of COVID-19. J. Surg. Educ. 78, 370–374. doi: 10.1016/j.jsurg.2020.07.040
Rahim, A. F. A. (2020). Guidelines for online assessment in emergency remote teaching during the COVID-19 pandemic [Internet]. Educat. Med. J. 12, 59–68. doi: 10.21315/eimj2020.12.2.6
Ramos-pla, A., and Arco, I. (2021). Education sciences university professor training in times of COVID-19: analysis of training programs and perception of impact on teaching practices. Educ. Sci 11:684. doi: 10.3390/educsci11110684
Remtulla, R. (2021). The present and future applications of technology in adapting medical education amidst the COVID-19 pandemic. JMIR Med. Educ. 6, 6–9. doi: 10.2196/20190
Ribeiro, L., Severo, M., and Ferreira, M. A. (2016). Performance of a core of transversal skills: self-perceptions of undergraduate medical students. BMC Med Educ 16, 1–7. doi: 10.1186/s12909-016-0527-2
Sani, I., Hamza, Y., Chedid, Y., Amalendran, J., and Hamza, N. (2020). Understanding the consequence of COVID-19 on undergraduate medical education: medical students’ perspective. Ann Med Surg. 58, 117–119. doi: 10.1016/j.amsu.2020.08.045
Shaiba, L. A., Alnamnakani, M. A., Temsah, M.-H., Alamro, N., Alsohime, F., Alrabiaah, A., et al. (2021). Medical Faculty’s and students’ perceptions toward pediatric electronic OSCE during the COVID-19 pandemic in Saudi Arabia. Healthcare 9:950. doi: 10.3390/healthcare9080950
Steinemann, S., Korndorffer, J., Dent, D., Rucinski, J., Newman, R. W., Blair, P., et al. (2021). Defining the need for faculty development in assessment. Am. J. Surg. 222, 679–684. doi: 10.1016/j.amjsurg.2021.06.010
Suwannaphisit, S., Anusitviwat, C., Tuntarattanapong, P., and Chuaychoosakoon, C. (2021). Comparing the effectiveness of blended learning and traditional learning in an orthopedics course. Ann. Med. Surg. 72:103037. doi: 10.1016/j.amsu.2021.103037
WHO (n.d.). Coronavirus disease (COVID-19) pandemic [Internet]. Available at: https://www.who.int/europe/emergencies/situations/covid-19
Keywords: medical education, medical students, online assessment, online learning, COVID-19
Citation: Machado MH, Paredes S and Ribeiro L (2023) COVID-19 impact on the assessment methodology of undergraduate medical students: a systematic review of the lessons learned. Front. Educ. 8:1304596. doi: 10.3389/feduc.2023.1304596
Edited by:
Anca Maria Cimpean, Victor Babes University of Medicine and Pharmacy, RomaniaReviewed by:
Anabel Ramos-Pla, University of Lleida, SpainErkan Hasan Atalmış, Manisa Celal Bayar University, Türkiye
Copyright © 2023 Machado, Paredes and Ribeiro. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Laura Ribeiro, lribeiro@med.up.pt