E-Learning Is Not Inferior to On-Site Teaching in a Psychiatric Examination Course

Background: Implementing e-learning into medical education is a growing field of research. Researchers have had positive experiences so far, and evidence suggests it to be no less effective than offline teaching. However, there are a few findings concerning psychiatric education and the use of simulated patients online.

Methods: We developed an online workshop for medical students at our psychiatric clinic, including group work exercises, lectures, and interviews with simulated patients. To compare the learning outcome, a cohort of students learning online was compared with a previous cohort that learned on-site. The same objective structured clinical examination (OSCE) was used in both cases. Evaluation questionnaires were gathered from students and lecturers and were compared with the former semesters along with the exam results.

Results: The exam grades did not significantly differ between on-site and online teaching, even though students rated their own communication skills better with online teaching. We also found that the connection experienced between students and teachers was impaired without on-site contact.

Discussion: We conclude that an online course may be an effective alternative to on-site teaching but requires further improvement to maintain a dependable student–teacher relationship.

Introduction

For many years, e-learning has been a growing field. In medical education, patient interaction and the special skill set of physicians call for specific teaching methods. There are several reports showing that e-learning is as effective as conventional lectures and that even practical courses can successfully be held online. However, evidence is still lacking (1–3).

Specifically, there are a few findings regarding the teaching of psychiatric examination and communication skills online. When considering the constructive alignment of the learning process, it becomes clear that communication can only be learned by communicating (4). While the use of simulated patients to teach and test practical and communicative skills is well-established (5–7), it is unclear whether it can be transferred to an online learning environment with video conferences.

When discussing digital teaching alternatives, the students' as well as the lecturers' preferences should be considered (8, 9).

We transferred the contents of our practical course program, including the exam, to video conferences and our university's digital platform. Students were asked to answer a questionnaire about the course afterward.

Purpose of the present study:

In this study,

- we explored whether it is possible to create an online course that teaches psychiatric history taking, examination, and communication to fourth year medical students.

- we compared whether the exam results of students who were taught exclusively online differ from those taught in our established on-site course.

- we investigated whether the teaching modalities had an impact on student satisfaction.

- we assessed how the online learning environment is accepted by both lecturers and students.

Methods

Participants

Fourth-year medical students complete an obligatory 1-week practical course at our psychiatric clinic. In the winter semester of 2019/20, when the course was held on-site, we examined 156 students in an objective structured clinical examination (OSCE) of which 150 (return rate: 96% female: 87, male: 54, unknown: 9) participated in the questionnaire. For the online teaching summer semester of 2020, 175 students were examined and 156 (return rate: 89% female: 93, male: 65, unknown: 2) participated in the questionnaire.

Additionally, we questioned all 16 (return rate: 75%) lecturers teaching in the summer semester about their experiences. The lecturers were the same in the on-site and online semesters, with the exception of four lecturers due to clinical rotations.

The participating medical students and lecturers voluntarily agreed to participate in the survey. The students were explicitly informed that participation would not impact their exam grades and that the questionnaire answers and grades would be processed anonymously. This study was submitted to the ethical committee of the Friedrich-Alexander University Erlangen-Nuremberg and received a designation of exempt.

Course Concept

The practical education consists of 5 course days and the following OSCE. Each day includes a 45-min theoretical seminar about common psychiatric syndromes presented by one of our senior physicians. In previous semesters, two to three real and/or simulated patients were then interviewed by the students in groups of seven on the wards. During the online semester, we used conference calls, in which only simulated patients were available due to privacy protection concerns. The actors, lecturers, and students gave feedback based on their perception of the interview. We expanded the course in the digital setting by offering a 20-min group exercise every day. Additionally, we offered a quiz for self-testing and attached resources based on our online lecture content such as podcasts and reading material (5). After 1 week, one of four standardized patient cases was given to the students as an OSCE. In the summer semester of 2020, this OSCE was also carried out via video conference.

Simulated Patients

The pool of simulated patients includes 6 professional actors and 10 amateurs, with a mean of seven semesters of experience as a simulated patient (5 males, 11 females) and around 16 assignments in each semester. Roles played include dementia, delirium, schizophrenia, major depression, bipolar disorder, adaptive disorder, obsessive–compulsive disorder, anxiety disorder, sexual arousal disorder, and sexual dysfunction. They are regularly trained and supervised in their roles, as well as in giving feedback to the students.

Objective Structured Clinical Examination Testing

Simulated patients will perform one out of four clinical cases (dementia, schizophrenia, bipolar disorder (current depressive episode), obsessive–compulsive disorder). The examiners are not the same as the lecturers. To provide a fair exam, we use a standardized questionnaire, rating performance of communication models, medical thoroughness, and the following case presentation (Supplementary Figure 1). The time frame of the interview was 15 min plus 90-s patient presentation. We taught and examined two communication models: the WRMS model (Wait-Repeat-Mirror-Summarize), used to support active listening (10), and the more advanced NURSE model (Naming-Understanding-Respecting-Supporting-Exploring Emotions), aiding the use of emotions in communication (11). One of the four cases (major depression/bipolar disorder) included taking the sexual history of patients. The scores we compared can be seen in Table 1. The same OSCE has been performed in the last four semesters, showing no improvement of the overall results over time. This indicates that previous knowledge of the exam structure is not advantageous to the students' performance; rather, their actual subject knowledge is examined.

TABLE 1

Table 1. OSCE performance scores: composition of the compared scores with exemplary tested items and achievable number of points.

Questionnaire

The students answered the questionnaire following completion of the course and prior to the exam. We assessed 15 items, including interest, motivation, self-assessment, in-course feedback and overall enjoyment. The rating was based on German school grades [1 (totally agree) to 6 (totally disagree)]. The students' gender and individual study duration were collected as demographic data (Supplementary Figure 2).

Furthermore, students were asked to compare the online course with their previous on-site learning experience (better–same–worse). Lecturers were asked to compare the online course to the previous semesters (better–same–worse).

Digital Environment

We used Zoom (Zoom Version 5.0.2., Zoom Video Communications Inc., California, USA) as a platform for conducting video conferences.

Ilias StudOn (ILIAS Version 5.4.17, ILIAS open-source e-Learning e.V., Cologne, Germany) is the online e-learning system of the University of Erlangen-Nuremberg.

Data Analysis

Statistical analysis was performed using SPSS (SPSS 21, IBM, Armonk USA). We used hierarchic linear modeling (HLM 7, Scientific Software International, Inc., Skokie, USA). Level-1 variables describe data collected from a single student (gender, duration of study), while level-2 variables consider data collected in the same week (lecturer, simulated patients, examiner, OSCE case, online/on-site teaching) (12). Results of the hierarchic linear modeling are reported as the difference between slopes (b) of the regression curves and its significance (p).

Results

The Test Results Did Not Differ Between Online and On-Site Learning

We compared the total number of points achieved in the OSCE as well as various sub-scores in the last on-site semester (winter semester 2019/20, n = 150) with those achieved in the e-learning semester (summer semester 2020, n = 156). The result of the power calculation indicated that with a total n of 306, small-to-medium effect sizes could be detected (ρ = 0.185). The intercept-only model revealed an ICC of 0.385. Thus, 38.5% of the variance in total exam score is between groups (including variables such as the semester, the lecturer, and the examiner) and 61.5% of the variance in the total exam score is between students (including variables such as age and duration of study) within a given group.

There was no significant difference in the total score (b = −1.47, p = 0.127), the communication scores (WRMS: b = −0.12, p = 0.425; NURSE: b < −0.01, p = 0.991), the psychopathological symptoms score (b = −0.28, p = 0.561), or the clinical presentation (b = −0.39, p = 0.306). However, there was a significant difference in the taking of the patient's sexual history (b = −0.91, p = 0.011), meaning that the students of the on-site semester performed better at this task than the students of the e-learning semester (Figure 1).

FIGURE 1

Figure 1. Tukey plot of the OSCE results with sub scores (WRMS, Wait-Repeat-Mirror-Summarize-Model; NURSE, Naming-Understanding-Respecting-Supporting-Exploring-Emotions-Model; PPA, Psychopathological Assessment; Sexual history; Patient presentation). *p < 0.05.

Gender Differences in the Objective Structured Clinical Examination Score

The students' gender significantly affected the exam results when including data from both semesters (b = 1.57, p = 0.003). In the subsequent analysis of the individual semesters, we discovered no significant relationship between gender and total score in the on-site semester (b = 0.73, p = 0.290), while there was a significant relationship between gender and total score in the online semester (b = 2.43, p = 0.002). Male students scored significantly fewer points in the online semester. We analyzed the relationship between gender and total score with consideration of the teaching method (online vs. on-site) as a potential moderator/mediator. In the analysis, we observed a difference between the estimation of fixed effects and the estimation of fixed effects with robust standard errors. Therefore, we decided to use the more conservative approach and report the results of the fixed effects with robust standard errors. There was a significant relationship between the method (online vs. on-site) and the total score (b = −4.42, p = 0.023), as well as a significant cross-level interaction between method and gender (b = 1.90, p = 0.042), whereas the relationship between gender and total score was no longer significant (b = 0.49, p = 0.495). This suggests that the method (online vs. on-site) is a mediator in the relationship between gender and total score: there is no stand-alone effect of gender on the total score; the effect is completely mediated by the fact that male students had worse total scores only in the online course environment.

On-Site Learning Receives Better Rating in Student–Teacher Relationship Items and Structure

The return rate of the student questionnaire was 98% for the on-site winter semester (n = 153) and 89% for the online summer semester (n = 156).

During the on-site semester, the students rated the following items better: the subjective support they received from their lecturer (b = 0.37, p = 0.017, Figure 2D), how much their interest in psychiatry was increased by the lecturer (b = 0.39, p = 0.030, Figure 2B), how much they enjoyed the course (b = 0.28, p = 0.031, Figure 2C), as well as how they would grade the course overall (b = 0.20, p = 0.034, Figure 2A). However, as previously reported, these differences did not impact the exam grade.

FIGURE 2

Figure 2. Distribution of questionnaire ratings sorted by item (A–H), comparing between the semesters (winter semester 2019/20, n = 150; summer semester 2020, n = 156) using hierarchic linear modeling (*significant differences between the semesters; WRMS, Wait-Repeat-Mirror-Summarize-Model; NURSE, Naming-Understanding-Respecting-Supporting-Exploring-Emotions-Model).

With E-Learning, Students' Self-Assessment in Communication Is Higher

Contrarily, students' self-assessment in the online semester improved for three communication heavy skills: the use of the WRMS strategy (b = −0.43, p = 0.002, Figure 2F), the use of the NURSE strategy (b = −0.61, p < 0.001, Figure 2G), and their ability to take the sexual history of a patient (b = −0.40, p = 0.035, Figure 2H).

Acceptance and Preference

The questionnaire item “I think the structure of the course is suited to teaching psychiatric expertise” was rated better for the on-site semester (b = 0.32, p = 0.003, Figure 2E).

When asked to compare the online course with their previous on-site learning experience, 50.0% of the students of the summer semester preferred the on-site semester, 24.2% preferred online teaching, and 25.8% rated them equally (n = 120).

Lecturer feedback had a return rate of 75%. Seventy-five percent of the lecturers preferred the on-site semester, 8% preferred the online semester, and 17% rated them equally (n = 12).

Discussion

Overall, we did not observe a difference in the exam scores between the on-site and the online semesters, except for the taking of the sexual history. The questionnaire showed that while students rated their own communication skills better in the online semester, they felt less connected to their lecturer.

It Is Possible to Create an Exclusively Digital Course

The existing digital structures were flexible enough in executing our plans to combine video conferences with online group work, as well as providing online materials for self-study. Zoom proved viable in imitating a telemedical setting. The online learning tool of the university, based on ILIAS, provided options to implement resources and assignments in a variety of media formats (weblinks, pictures, videos, podcasts, and texts), allowing for a more diverse learning experience. Students, lecturers, and actors had no issues using the platforms after an introduction. Technical problems did not occur on a relevant scale. Thus, the realization of the project did not meet many obstacles. As a result, the exam and questionnaire results were interesting to observe.

The Only Difference in the Test Results Was the Taking of the Sexual History When Comparing On-Site With Online Learning

The students' overall performance and the sub-scores were not relevantly impaired in the exclusive online teaching environment.

The only score that significantly changed was the taking of the sexual history. It is worth noting that this score is tested only in one-fourth of the exams (major depression/bipolar disorder). Wide fluctuations of this score have therefore also been observed in previous semesters. However, since sexual history often is a difficult topic to address (13), one could argue that the training lacked in thoroughness, the lecturers expressed more personal reservations in the online setting, or the online setting was indeed unsuitable in forming a dependable student–teacher relationship. Regarding the latter, Palmer et al. (14) created an online course to train students to interview patients about adverse sexual effects of SSRIs and observed positive communicative developments. This could indicate that the online setting itself is not at fault, rather, the degree to which our lecturers adapted the teaching of the sexual history taking to an online environment. We hypothesize that reservations (e.g., shame) regarding talking about sexuality may be harder to overcome in an online environment, and this impacted the quality of the teaching.

Overall, the results did not indicate a difference in the general student performance, thus, suggesting that the transfer of our course to an online setting did not have a considerable impact on the learning effectiveness.

Several authors studying the quality and practicability of online medical teaching reported similar findings. There is a wide range of methods described by the term “web-based learning,” complicating the navigation of literature (15). Our study included multiple such approaches, which were mostly tested as a stand-alone intervention. There are several findings that video lectures and live lectures are equally effective in teaching new content (1, 16, 17). Providing digital content, in addition to lectures and simulated patient interviews, is a useful tool to promote deep learning (18, 19). Topic-focused interventions can improve specific skills, such as taking the patient's sexual history (20, 21). The core of our course is the training of communication skills. Kaltman et al. (22) found that students trained with interactive online videos demonstrated more of the taught patient-centered interview skills in a face-to-face. The usefulness of simulations in medical education has been long since established (23). Parisi et al. used an approach using Zoom in small groups with simulated patients with diabetes, similarly to our study. They found it to be effective as well, but emphasized the lack of non-verbal communication training, which is especially important in intercultural contexts (24). Mohos et al. found the transfer of their course to an online environment to be a success, with consistent outcomes compared with previous semesters. Nevertheless, they raised concerns considering the missing personal contact (25), which we came across as well.

The Subjective Learning Experience Seems to Differ Between Online and On-Site Teaching

The students evaluated the on-site semester better in student–teacher-related items (subjective support from the teacher, increased interest in psychiatry, and overall enjoyment), as well as the course as a whole. This may indicate that the establishment of face-to-face contact with the lecturer plays a significant role in the perception of the experience as pleasant and helpful (26). As the participants' satisfaction influences the success of a didactic method (on-site/online), it may be considered that while the exam results stayed the same, the course did not entirely meet its standards (27). Thus, it should be aimed to include elements in online courses in which space for such interaction is provided (28). For example, an introductory round at the start of the week, as well as daily flash feedback rounds could be included to promote group synergy. More time for discussion of the topic could further improve this. With respect to the structure of the course, we repeatedly received verbal feedback emphasizing that it is preferable if the lecturer does not switch during the week.

In the online semester, the students self-assessed their communication skills (WRMS, NURSE, and SEX) better. It is notable that only the communication strategies but no other learning content, such as the examination of symptoms or the clinical presentation, were rated differently. This may have various reasons. The simulated patients may have been primed to react to these communication models in a more obvious way. It is also possible that without proper non-verbal communication, the students had to rely on the learned strategies or could better focus on the patient's reaction. Perhaps the lowered social anxiety allowed for bolder self-assessment. Special consideration should be given to the self-assessment of summer students, who rated their sexual history taking skills higher than the previous semester but achieved lower scores in their exams. This may indicate that the self-assessment of students, at least in the form tested in this study, is not reliable when compared with the results of an OSCE (29).

Differences Regarding Gender

The method (online vs. on-site) is a mediator in the relationship between gender and total score. We found that female students reached a significantly higher average total score in the online semester. We cannot conclude whether this is because female students performed better during the online semester or if male students struggled more, neither whether it was directly caused by our e-learning method or surrounding circumstances. González-Gómez et al. (30) found that female students were more accepting of e-learning, which might be the reason for their better performance in the OSCE during the online semester. Thus, gender differences should be considered when investigating e-learning in educational research, as findings similar to ours have been reported before (31, 32).

Students' and Lecturers' Reservations Toward E-Learning Are to Be Considered

We also observed that the majority of students (50%) and lecturers (75%) of the summer semester preferred on-site teaching. Students also thought that the structure of the on-site course is more suited to teach psychiatric expertise.

Mullins et al. (33) also found that students preferred on-site psychiatric lectures, even though they liked the flexibility and availability of online lectures. While there is still a preference for traditional teaching methods, the acceptance of online teaching methods is increasing, and participants adapt well when confronted with them (34, 35). Nevertheless, the lecturers' willingness and trust in the teaching method has a relevant role in its proper implementation (9).

Limitations and Further Research

The power analysis showed that we had a sufficient number of participants, but it should be considered that the summer semester of 2020, at the time of our intervention, was also severely influenced by the COVID-19 pandemic, which may have had a number of unforeseen effects on our results (36–39). Also, our groups where not randomized or controlled (40).

The OSCE testing we performed has been validated throughout the previous four semesters and has proved reliable. However, it cannot be excluded that conducting the exam online has an effect on the examiner that mingles with the influences of our intervention.

A general problem is that the variance of the lecturer, student, and examiner has a significant impact on the outcome of learning and exam results. The influence of the teaching method we investigated may be overshadowed by these variables.

Conclusion

Overall, we can conclude that a practical course in psychiatry can be conducted in an online setting with video conferences. The exam grades did not differ in comparison to on-site teaching. Items illustrating the student–teacher relationship were rated better when teaching in person, indicating that this is an issue to be considered in future studies.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The participating medical students and lecturers voluntarily agreed to participate in the survey. The participation did not impact the exam grades, and the questionnaire answers and grade were processed anonymously. This study was submitted to the ethical committee of the Friedrich Alexander Universität Erlangen-Nuremberg and received a designation of exempt.

Author Contributions

CR, JU, MR, JK, and PS designed the study. CR, JU, and PS collected the data. Data were analyzed and evaluated by CR, JU, MR, JK, and PS. The statistics were carried out by CR and MR. CR, JU, MR, and PS drafted the manuscript. All authors critically reviewed the manuscript and provided constructive comments to improve the quality of the manuscript. All authors have read and approved the final manuscript. All authors agree to be accountable for all aspects of the work.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The present work was performed in partial fulfillment of the requirements for obtaining the degree Dr. med. of CR. We gratefully thank Martina Manna for the language editing. We acknowledge support from Deutsche Forschungsgemeinschaft (DFG) and Friedrich-Alexander-University Erlangen-Nuremberg (FAU) within the funding program Open Access Publishing.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2021.624005/full#supplementary-material

References

1. Pei L, Wu H. Does online learning work better than offline learning in undergraduate medical education? A systematic review and meta-analysis. Med Educ Online. (2019) 24:1666538. doi: 10.1080/10872981.2019.1666538

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Richmond H, Copsey B, Hall AM, Davies D, Lamb SE. A systematic review and meta-analysis of online versus alternative methods for training licensed health care professionals to deliver clinical interventions. BMC Med Educ. (2017) 17:227. doi: 10.1186/s12909-017-1047-4

PubMed Abstract | CrossRef Full Text | Google Scholar

3. McCutcheon K, Lohan M, Traynor M, Martin D. A systematic review evaluating the impact of online or blended learning vs. face-to-face learning of clinical skills in undergraduate nurse education. J Adv Nurs. (2015) 71:255–70. doi: 10.1111/jan.12509

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Duffy FD, Gordon GH, Whelan G, Cole-Kelly K, Frankel R, Buffone N, et al. Assessing competence in communication and interpersonal skills: the Kalamazoo II report. Acad Med. (2004) 79:495–507. doi: 10.1097/00001888-200406000-00002

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Reed S, Shell R, Kassis K, Tartaglia K, Wallihan R, Smith K, et al. Applying adult learning practices in medical education. Curr Probl Pediatr Adolesc Health Care. (2014) 44:170–81. doi: 10.1016/j.cppeds.2014.01.008

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Wündrich M, Peters J, Philipsen A, Kopasz M, Berger M, Voderholzer U. Einsatz von Simulationspatienten in den Lehrfächern Psychiatrie und Psychotherapie. Eine kontrollierte Pilotstudie. Nervenarzt. (2008) 79:1273–82. doi: 10.1007/s00115-008-2493-3

CrossRef Full Text | Google Scholar

7. Park KH, Park SG. The effect of communication training using standardized patients on nonverbal behaviors in medical students. Korean J Med Educ. (2018) 30:153–9. doi: 10.3946/kjme.2018.90

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Link TM, Marz R. Computer literacy and attitudes towards e-learning among first year medical students. BMC Med Educ. (2006) 6:34. doi: 10.1186/1472-6920-6-34

PubMed Abstract | CrossRef Full Text | Google Scholar

9. O'Doherty D, Dromey M, Lougheed J, Hannigan A, Last J, McGrath D. Barriers and solutions to online learning in medical education - an integrative review. BMC Med Educ. (2018) 18:130. doi: 10.1186/s12909-018-1240-0

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Langewitz W, Laederach K, Buddeberg C. Ärztliche Gesprächsführung. In: Buddeberg C, editor. Psychosoziale Medizin. Berlin: Springer (2004). p. 373–407.

Google Scholar

11. Back AL, Arnold RM, Baile WF, Fryer-Edwards KA, Alexander SC, Barley GE, et al. Efficacy of communication skills training for giving bad news and discussing transitions to palliative care. Arch Intern Med. (2007) 167:453–60. doi: 10.1001/archinte.167.5.453

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Woltman H, Feldstain A, MacKay JC, Rocchi M. An introduction to hierarchical linear modeling. Tutor Quant Methods. (2012) 8:52–69. doi: 10.20982/tqmp.08.1.p052

CrossRef Full Text | Google Scholar

13. Forsyth S, Rogstad K. Sexual health issues in adolescents and young adults. Clin Med (Lond). (2015) 15:447–51. doi: 10.7861/clinmedicine.15-5-447

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Palmer BA, Lee JH, Somers KJ, Swintak CC, Rullo J, Bright RP, et al. Effect of in-class vs. online education on sexual health communication skills in first-year medical students: a pilot study. Acad Psychiatry. (2019) 43:175–9. doi: 10.1007/s40596-018-0949-8

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Cook DA, Garside S, Levinson AJ, Dupras DM, Montori VM. What do we mean by web-based learning? A systematic review of the variability of interventions. Med Educ. (2010) 44:765–74. doi: 10.1111/j.1365-2923.2010.03723.x

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Brockfeld T, Müller B, de Laffolie J. Video versus live lecture courses: a comparative evaluation of lecture types and results. Med Educ Online. (2018) 23:1555434. doi: 10.1080/10872981.2018.1555434

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Solomon DJ, Ferenchick GS, Laird-Fick HS, Kavanaugh K. A randomized trial comparing digital and live lecture formats ISRCTN40455708. BMC Med Educ. (2004) 4:27. doi: 10.1186/1472-6920-4-27

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Warren N, Parker S, Khoo T, Cabral S, Turner J. Challenges and solutions when developing online interactive psychiatric education. Australas Psychiatry. (2020) 28:359–62. doi: 10.1177/1039856220901477

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Ryznar E, Hamaoka D, Lloyd RB. Pilot study of an online self-directed learning module for medical decision-making capacity. Acad Psychiatry. (2020) 44:408–12. doi: 10.1007/s40596-020-01215-y

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Wasser TD, Hu J, Danzig A, Yarnell-MacGrory S, Guzman JR, Michaelsen K. Teaching forensic concepts to residents using interactive online modules. J Am Acad Psychiatry Law. (2020) 48:77–83. doi: 10.29158/JAAPL.003890-20

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Tessler Lindau S, Goodrich KG, Leitsch SA, Cook S. Sex in the curriculum: the effect of a multi-modal sexual history-taking module on medical student skills. Sex Educ. (2008) 8:1–9. doi: 10.1080/14681810701811753

CrossRef Full Text | Google Scholar

22. Kaltman S, Talisman N, Pennestri S, Syverson E, Arthur P, Vovides Y. Using Technology to enhance teaching of patient-centered interviewing for early medical students. Simul Healthc. (2018) 13:188–94. doi: 10.1097/SIH.0000000000000304

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Issenberg SB, McGaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach. (2005) 27:10–28. doi: 10.1080/01421590500046924

PubMed Abstract | CrossRef Full Text

24. Parisi MC, Frutuoso L, Benevides SS, Barreira NH, Silva JL, Pereira MC, et al. The challenges and benefits of online teaching about diabetes during the COVID-19 pandemic. Diabetes Metab Syndr. (2020) 14:575–6. doi: 10.1016/j.dsx.2020.04.043

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Mohos A, Mester L, Barabás K, Nagyvári P, Kelemen O. Orvos-beteg kommunikációs gyakorlat szimulált pácienssel a koronavírus-járvány idején. (A COVID-19-pandémia orvosszakmai kérdései). Orv Hetil. (2020) 161:1355–62. doi: 10.1556/650.2020.31930

CrossRef Full Text | Google Scholar

26. Alnabelsi T, Al-Hussaini A, Owens D. Comparison of traditional face-to-face teaching with synchronous e-learning in otolaryngology emergencies teaching to medical undergraduates: a randomised controlled trial. Eur Arch Otorhinolaryngol. (2015) 272:759–63. doi: 10.1007/s00405-014-3326-6

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Haidet P, Stein HF. The role of the student-teacher relationship in the formation of physicians. The hidden curriculum as process. J Gen Intern Med. (2006) 21(Suppl 1):S16–20. doi: 10.1111/j.1525-1497.2006.00304.x

PubMed Abstract | CrossRef Full Text | Google Scholar

28. KerŽič D, TomaŽevič N, Aristovnik A, Umek L. Exploring critical factors of the perceived usefulness of blended learning for higher education students. PLoS One. (2019) 14:e0223767. doi: 10.1371/journal.pone.0223767

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Pawluk SA, Zolezzi M, Rainkie D. Comparing student self-assessments of global communication with trained faculty and standardized patient assessments. Curr Pharm Teach Learn. (2018) 10:779–84. doi: 10.1016/j.cptl.2018.03.012

PubMed Abstract | CrossRef Full Text | Google Scholar

30. González-Gómez F, Guardiola J, Martín Rodríguez Ó, Montero Alonso MÁ. Gender differences in e-learning satisfaction. Comput Educ. (2012) 58:283–90. doi: 10.1016/j.compedu.2011.08.017

CrossRef Full Text | Google Scholar

31. Ramírez-Correa PE, Arenas-Gaitán J, Rondán-Cataluña FJ. Gender and acceptance of E-learning: a multi-group analysis based on a structural equation model among college students in Chile and Spain. PLoS One. (2015) 10:e0140460. doi: 10.1371/journal.pone.0140460

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Wijaya TT, Ying Z, Suan L. Gender and self regulated learning during COVID-19 pandemic in Indonesia. Basicedu. (2020) 4:725–32. doi: 10.31004/basicedu.v4i3.422

CrossRef Full Text | Google Scholar

33. Mullins D, Jabbar F, Fenlon N, Murphy KC. The digital age: is this the future of medical education? A cross-sectional study to assess medical students' opinions about e-learning in psychiatry undergraduate medical education. Ir J Psychol Med. (2014) 31:89–96. doi: 10.1017/ipm.2014.15

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Gormley GJ, Collins K, Boohan M, Bickle IC, Stevenson M. Is there a place for e-learning in clinical skills? A survey of undergraduate medical students' experiences and attitudes. Med Teach. (2009) 31:e6–12. doi: 10.1080/01421590802334317

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Wynter L, Burgess A, Kalman E, Heron JE, Bleasel J. Medical students: what educational resources are they using? BMC Med Educ. (2019) 19:36. doi: 10.1186/s12909-019-1462-9

CrossRef Full Text | Google Scholar

36. Looi JC, Bonner D, Maguire P, Finlay A, Keightley P, Parige R, et al. Flattening the curve of COVID-19 for medical education in psychiatry and addiction medicine. Australas Psychiatry. (2020) 29:31–4. doi: 10.1177/1039856220946647

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Hodges C, Moore S, Lockee B, Trust T, Bond A. The Difference Between Emergency Remote Teaching and Online Learning. (2020). Available online at: https://er.educause.edu/articles/2020/3/the-difference-between-emergency-remote-teaching-and-online-learning (cited October 02, 2020).

Google Scholar

38. Horesh D, Kapel Lev-Ari R, Hasson-Ohayon I. Risk factors for psychological distress during the COVID-19 pandemic in Israel: loneliness, age, gender, and health status play an important role. Br J Health Psychol. (2020) 25:925–33. doi: 10.1111/bjhp.12455

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Conversano C, Di Giuseppe M, Miccoli M, Ciacchini R, Gemignani A, Orrù G. Mindfulness, age and gender as protective factors against psychological distress during COVID-19 pandemic. Front Psychol. (2020) 11:1900. doi: 10.3389/fpsyg.2020.01900

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Sinclair P, Kable A, Levett-Jones T. The effectiveness of internet-based e-learning on clinician behavior and patient outcomes: a systematic review protocol. JBI Database Syst Rev Implement Rep. (2015) 13:52–64. doi: 10.11124/jbisrir-2015-1919

CrossRef Full Text | Google Scholar