The Impact of Manuscript Learning vs. Video Learning on a Surgeon's Confidence in Performing a Difficult Procedure

Introduction: Operative surgical videos are a popular educational resource, not commonly a part of a peer-reviewed article. We wanted to evaluate the impact of either reading a peer-reviewed manuscript or watching an operative video on a surgeon's confidence in performing a complex case.

Methods: Pediatric surgeons and fellows were asked to complete an initial questionnaire to assess their confidence (formulated as a score) in the diagnosis and operative repair of anal stenosis and rectal atresia.

Results: Of 101 pediatric surgeons and fellows, 52 (51%) were randomized into a “manuscript” group and 49 (49%) into a “video” group. The mean confidence before the intervention was the same in the two groups (6.4 vs. 6.6). Attending surgeons started with more confidence than trainees (7.1 vs. 5.3, p < 0.001). In the manuscript group, the average confidence increased to 7.7 (p = 0.005), and in the video group the average confidence increased to 7.9 (p = 0.001) globally. Trainees in the video group significantly improved their confidence to a score of 6.6 (p = 0.035), as did attending surgeons to 8.5 (p = 0.01). In the manuscript group, only attendings significantly improved their confidence by 1.5–8.3 (p < 0.001), whereas trainees did not with a difference of 1.3 (p = 0.194). When considering experience level, physicians who reported never having performed this surgery improved only by reading the manuscript (3.9–6.2) (p = 0.004), not by watching the video (5.4–6.6) (p = 0.106). Surgeons with experience doing this operation (>5 times) did not improve their confidence by reading the manuscript (p = 0.10), nor by watching the video (p = 0.112).

Conclusion: Reviewing either a detailed manuscript or operative video on the surgical management of rectal atresia and anal stenosis demonstrated a significant increase in self-reported confidence. Trainees benefitted the most from operative videos, whereas experienced surgeons did not improve their confidence by reading the manuscript nor watching the video.

Introduction

Traditionally, surgeons and residents have learned new operative techniques by reading about surgical techniques and trying to improve their skills in the operating room. Recently, newly emerged media, such as video, are already being used in educating medical students and seem to have value as an addition to traditional educational measures (1, 2). In recent years, several services have surfaced which provide videos of surgeries as a way of teaching the audience a specific surgical technique. There is, however, little evidence that the use of these videos is an improvement over the use of a manuscript. Therefore, we set out to compare if watching a peer-reviewed video improves the confidence of surgeons in performing an operation more than the reading of a peer-reviewed technical manuscript.

Methods

Inclusion/Exclusion Criteria

Any pediatric surgeon or pediatric surgery resident could participate in this study. The study was approved and excempted from a full review by the Institutional Review Board of Nationwide Childrens Hospital due to the voluntary, consented and anonymous participation of our study population. We contacted surgeons and residents by email and asked them to anonymously complete a questionnaire assessing their confidence in diagnosing and operating on patients with anal stenosis or rectal atresia. The participants were then randomized into two groups: one group watched the video, and the other group read the manuscript. Thereafter, they completed the same questionnaire as before the intervention, to assess any difference in confidence.

Instruments Used

To assess the confidence of physicians in performing the surgery, we developed an 11-item questionnaire (see addendum A). Participants were asked about their experience in these operations and whether they were a trainee or an attending surgeon, enabling us to perform a subgroup analysis on these groups. For the overall analysis, we looked at the individual questions, but averaged the confidences of the questions into a total score as well.

The manuscript describing the operative technique was published earlier in a peer-reviewed journal (3). Furthermore, the technique video on the same topic was also peer-reviewed and published by the same team on the C-Surgeries video site.

Statistics

Mann-Whitney U-tests were performed to compare the total pre- and post-confidence between the video and the manuscript groups. The same test was used to see if there was any improvement in either group for the subgroup analysis and to look at the individual questions. For the overall comparison, a p < 0.05 was considered significant. When comparing the individual questions, after applying a Bonferroni correction, a p < 0.00833 was considered significant.

Results

Demographics

One hundred one surgeons completed the pre- and post-intervention questionnaires. Of those, 52 (51%) were randomized to the manuscript group and 49 (49%) to the video group. Of the surgeons completing both surveys, 66 (65%) were attending pediatric surgeons, 19 (65%) were pediatric surgery residents (19%), and 16 (16%) were pediatric surgery fellows. Thirty-two (32%) participants reported never having performed either surgery, whereas 46 (46%) participants had performed it between one to five times, and 23 (23%) surgeons had performed the surgery more than five times.

Pre-intervention Confidence

The baseline overall confidence was 6.6 in the video group and 6.4 in the manuscript group (p = 0.962). Attending surgeons reported an overall confidence of 7.4 in the video group and 6.8 in the manuscript group, whereas these numbers were 5.2 and 5.5 for trainees. Physicians who had not previously performed the surgery noted a pre-intervention confidence of 5.4 in the video group and 3.9 in the manuscript group, whereas these were 6.4 and 6.7 for surgeons having performed this surgery one to five times. Finally, surgeons that had performed this particular technique at least five times reported a confidence of 8.7 in the video group and 8.5 in the manuscript group before receiving the intervention.

Post-intervention Confidence

After either reading the manuscript or watching the video, the confidence rose to 7.7 (p = 0.005) in the video group and to 7.9 (p = 0.001) in the manuscript group. When comparing the two groups, there was no significant difference between the total scores (p = 0.790) or any of the separate questions. However, when looking at the individual questions in the video and manuscript groups separately, the manuscript group improved significantly on five of six questions, whereas the video group improved only on two questions (see Table 1).

TABLE 1

Table 1. All questions an the average confidence score per group.

Subgroup Analysis (Trainee/Attending, Times Performed)

We also analyzed the influence of experience on the increase of confidence (see Table 2). When shown the video, trainees improved their confidence from 5.2 (n = 26) to 6.6 (n = 20, p = 0.035). The confidence of attending surgeons who watched the educational video increased from 7.4 (n = 43) to 8.5 (n = 29, p = 0.01).

TABLE 2

Table 2. Experience and the influence on confidence.

Attending surgeons who read the manuscript significantly increased their scores, as well (from 6.8 to 8.3, p = 0.001). This was not the case for trainees: as a result of reading the article, there was no statistically significant increase in confidence (5.5–6.8, p = 0.194).

We also took into account the number of times a surgeon had done this particular operation. Surgeons having never performed this operation significantly improved their confidence in both groups: from 5.4 to 6.6 (p = 0.048) in the video group and from 3.9 to 6.2 (p < 0.001) in the manuscript group. The same conclusion can be drawn when looking at surgeons who had performed this surgery between one and five times: when watching the video, their confidence rose from 6.4 to 7.8 (p = 0.006), whereas it increased from 6.7 to 8.2 (p = 0.001) when reading the article.

Experienced surgeons (>5 surgeries performed) significantly increased their confidence only by watching the video (8.7–9.5, p = 0.042), but did not improve by reading the manuscript (8.5–9.1, p = 0.0918).

Discussion

Earlier Evidence on Manuscript vs. Video Learning

In the last few years, there have been quite a few studies looking at the use of videos as a novel way of teaching surgeries, with overall favorable results. The results have reported less need for staff takeover and fewer errors by surgery residents (4, 5), and better overall understanding and more interest in surgical procedures in medical students (6). One study reported that video was the educational method of choice for residents (7). However, no study reports on the efficiency or efficacy of video-based education compared with learning by reading a manuscript in which the surgical technique is explained.

Interpretation of Pre-intervention Confidence

The pre-intervention confidence in both groups was comparable (6.6 in the video group, 6.4 in the manuscript group). These finding are in the range of what one might expect of the confidence of a group of pediatric surgeons in a specific surgical technique.

Interpretation of Post-intervention Confidence

As expected, both groups significantly improved their confidence in performing this particular operation. When looking at the overall confidence, the video group and the manuscript group improved similarly: from 6.6 to 7.7 and 6.4 to 7.9, respectively. However, when looking at the six questions separately, the surgeons who read the manuscript improved their confidence significantly on five of six questions, whereas surgeons who watched the video improved only on two questions. One could therefore argue that the manuscript seems better at improving the broader confidence of every aspect of diagnosing and treating rectal atresia and stenosis.

Interpretation of Subgroup Analysis

Attending surgeons benefited from both the video and the manuscript, whereas trainees benefitted only by watching the video. It is conceivable that with more general surgery knowledge (anatomy, techniques), it is easier to learn from an article. On the other hand, a trainee with fewer skills benefits from the visual instruction that a video offers. An earlier study found that faculty can name more laparoscopic facts when shown a video than can residents shown the same video, but the study did not measure actual proficiency in surgery or confidence to perform the operation (8).

Interestingly, when categorizing participants according to the number of times they have performed the operation, surgeons with the most experience do not significantly improve their scores when reading the manuscript vs. watching the video. Participants with no experience in this particular surgery at all are better off reading the manuscript.

Conclusion

Watching a video and reading a manuscript about a specific surgical technique similarly improve a surgeon's confidence in performing the operative technique. However, reading a manuscript improves almost every aspect of the surgery, whereas watching a video does not. Surgeons with no experience seem to win greater confidence in performing a procedure after watching a video, but an actual performance on the procedure itself, and its outcomes can't be concluded from this data.

Author Contributions

CR and AD had the idea, planned and performed the study. VL edited the video and manuscript used for the study. JC did the statistical analysis on the paper. RW and ML edited and corrected the manuscript.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

1. Woodham LA, Ellaway RH, Round J, Vaughan S, Poulton T, Zary N, et al. Medical student and tutor perceptions of video versus text in an interactive online virtual patient for problem-based learning: a pilot study. J Med Internet Res. (2015) 17:e151. doi: 10.2196/jmir.3922

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Shevell AH, Thomas A, Fuks A. Teaching professionalism to first year medical students using video clips. Med Teach. (2015) 37:935–42. doi: 10.3109/0142159X.2014.970620

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Lane VA, Wood RJ, Reck C, Skerritt C, Levitt MA. Rectal atresia and anal stenosis: the difference in the operative technique for these two distinct congenital anorectal malformations. Tech Coloproctol. (2016) 20:249–54. doi: 10.1007/s10151-016-1435-5

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Hamour AF, Mendez AI, Harris JR, Biron VL, Seikaly H, Côté DWJ, et al. A high-definition video teaching module for thyroidectomy surgery. J Surg Educ. (2017) 75:481–88. doi: 10.1016/j.jsurg.2017.07.019

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Mendez A, Seikaly H, Ansari K, Murphy R, Cote D. High definition video teaching module for learning neck dissection. J Otolaryngol Head Neck Surg. (2014) 43:7. doi: 10.1186/1916-0216-43-7

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Hotokezaka M, Chijiiwa K, Kondo K, Kai M, Eto TA, Hidaka H, et al. Video monitoring and slide and video presentations as tools for surgical education. Hepatogastroenterology (2008) 55:1519–22.

PubMed Abstract | Google Scholar

7. Hayden EL, Seagull FJ, Reddy RM. Developing an educational video on lung lobectomy for the general surgery resident. J Surg Res. (2015) 196:216–20. doi: 10.1016/j.jss.2015.02.020

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Abdelsattar JM, Pandian TK, Finnesgard EJ, El Khatib MM, Rowse PG, Buckarma EH, et al. Do you see what i see? how we use video as an adjunct to general surgery resident education. J Surg Educ. (2015) 72:e145–50. doi: 10.1016/j.jsurg.2015.07.012

PubMed Abstract | CrossRef Full Text | Google Scholar