Novel workflow analysis of robot-assisted hysterectomy through objective performance indicators: a pilot study

Introduction: The curriculum for a da Vinci surgeon in gynecology requires special training before a surgeon performs their first independent case, but standardized, objective assessments of a trainee’s workflow or skills learned during clinical cases are lacking. This pilot study presents a methodology to evaluate intraoperative surgeon behavior in hysterectomy cases through standardized surgical step segmentation paired with objective performance indicators (OPIs) calculated directly from robotic data streams. This method can provide individual case analysis in a truly objective capacity.

Materials and methods: Surgical data from six robot-assisted total laparoscopic hysterectomies (rTLH) performed by two experienced surgeons was collected prospectively using an Intuitive Data Recorder. Each rTLH video was annotated and segmented into specific, functional surgical steps based on the recorded video. Once annotated, OPIs were compared through workflow analysis and across surgeons during two critical surgical steps: colpotomy and vaginal cuff closure.

Results: Through visualization of the individual steps over time, we observe workflow consistencies and variabilities across individual surgeons of a similar experience level at the same hospital, creating unique surgeon behavior signatures across each surgical case. OPI differences across surgeons were observed for both the colpotomy and vaginal cuff closure steps, specifically reflecting camera movement, energy usage and clutching behaviors. Comparing colpotomy and vaginal cuff closure time needed for the step and the events of energy use were significantly different (p < 0.001). For the comparison between the two surgeons only the event count for camera movement during colpotomy showed significant differences (p = 0.03).

Conclusion: This pilot study presents a novel methodology to analyze and compare individual rTLH procedures with truly objective measurements. Through collection of robotic data streams and standardized segmentation, OPI measurements for specific rTLH surgery steps can be reliably calculated and compared to those of other surgeons. This provides opportunity for critical standardization to the gynecology field, which can be integrated into individualized training plans in the future. However, more studies are needed to establish context surrounding these metrics in gynecology.

Introduction

Since the introduction of the da Vinci Surgical System in the 1990s, its use has gained importance in gynecology and other surgical fields (1–3). Since then, Intuitive has established curricula to be completed before the surgeon uses the da Vinci Surgical System in the operating room, as robotic surgery performance has been shown to be dependent on the expertise of the surgeon (4). However, there is still limited understanding of a true surgical learning curve (5). Most studies define the learning curve of robotic surgery via reduced surgical time, docking time, console time, rates of complications, blood drop, and time of hospital stay (5–7). Moreover, many assessments are time-and resource-consuming, subjective, not reproducible, and poorly comparable across surgeons.

One of many ways to look at surgical behavior and the improvement of robotic surgical skills may be the analysis of workflows using individual application of camera movement, energy use and clutching to reposition the hands within the console during specific surgical steps. These robotic data streams can be captured directly from the da Vinci Surgical System using an Intuitive Data Recorder (IDR, Intuitive Surgical Ltd., Sunnyvale, California, United States) (8). Recently, the field of surgical data science has emerged, with a growing interest in objective performance indicators (OPIs), metrics calculated directly from the robotic system’s data streams, that provide truly objective measurements and behaviors within individual surgeries (9, 10). OPIs have been utilized in other surgical specialties to correlate with surgical skill, workflow, and outcomes (11, 12), but no studies have been performed for gynecology procedures.

This feasibility study, which is the first in the gynecology space, introduces a methodology specific robot-assisted total laparoscopic hysterectomy (rTLH) and utilizes it to evaluate surgical workflow and intraoperative behaviors using OPIs. With more research in this space, such objective parameters will enable surgeon proficiency identification and tailored training to the learning surgeon.

Materials and methods

Raw data collection and annotation

Six rTLH were performed at the Department of Women’s Health at the University Hospital of Tübingen using the da Vinci Si surgical system (Intuitive Surgical Ltd., Sunnyvale, California, United States). Synchronized video and accompanying robotic data streams were captured using an IDR on loan from Intuitive Surgical for the purpose of this study. All data were encrypted (AES-256) and stored on the Intuitive Data Recorder. Data were copied to an external hard disk (with encrypted access) and sent to Intuitive Surgical for professional annotation. No sound was recorded and neither patient nor surgeon were identifiable from the recordings. Date and time markers were part of the data captured from the da Vinci Surgical System, data allowing identification of the patient was not forwarded to Intuitive. Video annotation started with the insertion of the camera to the abdominal cavity. Video scrubbing algorithm was used to block out endoscope events outside of the body to guarantee patient and OR staff anonymity.

Each video was segmented into functional surgical steps specific to rTLH by a professional annotator (LP), a data scientist trained in gynecologic anatomy and all surgical steps of rTHL. The annotator indicated the start and stop times of each step according to a standardized annotation card (Table 1), which provides a detailed start and stop action for the specific surgical step (Figure 1). Each step may occur multiple times, identifying when a surgeon alternates between steps. As annotations were limited to functional steps, gaps between steps could also be identified. These include cleaning of the camera, change of instruments, surgeon idle time, etc. When a surgeon switched from one surgical step to the next within 2 seconds, no gap would be inserted by the annotator. Metric data that can be extracted via the IDR from the robot are: frequency of camera movement, energy use and clutch use for each side. Within each surgical step these parameters were used to calculate surgical activity within the time parameters of each step. Hereby OPIs can now be analyzed for each individual surgical step. These provide a truly objective measure of surgical behavior that can be attributed to specific rTLH steps. Since during colpotomy and closure of the vaginal cuff camera movement, use of energy and clutch use are frequently applied in every hysterectomy and therefore a large amount of data was available, these two surgical steps were examined in detail.

Table 1

Table 1. Functional surgical steps utilized for annotation during rTLH.

Figure 1

Figure 1. Images depicting start and stop moments for segmentation of hysterectomy. Vaginal cuff closure is presented as an example here. The surgical step begins with the clamping of the needle in the needle holder and ends with the cutting of the barbed thread after complete closure of the colpotomy.

Ethics statement

Ethical approval was obtained from the Ethics Committee of Tübingen University Medical Faculty and University Hospital (621/2018BO1). Three operations, each performed by two experienced da Vinci surgeons, were selected to evaluate the feasibility of the method in this pilot study.

Statistical analysis

Statistical analysis was performed with Microsoft excel and R version 4.3 and RStudio version 2023.06.1 + 524 using the tidyverse (2.0.0) packages. Statistical comparisons were carried out with the student’s t test. The data are expressed as the mean ± standard deviation. p values of <0.05 indicate statistical significance.

Results

Visualization of standardized surgical steps across two experienced surgeons enable rTLH workflow comparison

Individual steps across six rTLH cases from two experienced surgeons, who performed more than 30 TLHs using the da Vinci surgical system, were segmented, plotted, and compared (Figure 2). Case numbers 1–3 were performed by surgeon A, while cases 4–6 were performed by surgeon B. Patient characteristics are displayed in Table 2. By displaying each individual step over time, it is possible to observe the sequence of surgery exactly. Roughly similar sequence patterns were observed across the individual surgeries and across the two surgeons, with dissection of the fallopian tube followed by dissection of the round ligament, dissection of the broad ligament, bladder flap creation, division of the uterine vessels, colpotomy with removal of the uterus, and vaginal cuff closure using a barbed thread (Figure 1). An obvious difference observed was that surgeon A began with the hysterectomy on the left side, while surgeon B began on the right side. Despite the side differences on which surgery was stared, hereafter the surgical steps of all 6 surgeries follow the standard surgical steps described in the annotation card (Table 1—steps 5 to 11).

Figure 2

Figure 2. Visualization of surgical workflows across six unique rTLH cases enables objective workflow comparisons. Each color bar represents a surgical step with the length of the bar corresponding to its duration. Gaps between steps indicate idle time or surgical activity not aligned to the standardized hysterectomy steps. The existence of multiple bars of the same step name represent step switching by the surgeon or pause from the surgeon. All cases exhibited step switching and lapses between surgical steps, with each case exhibiting an individualized signature, enabling insights into surgical technique and workflow.

Table 2

Table 2. Patient characteristics.

Additionally, all cases display both blank time periods with no surgical steps indicated and surgical step switching. Blank periods occur when the change between surgical steps last longer than two seconds, e.g., due to change of instruments, need to obtain additional equipment such as retrieval bags or morcellators, or idle time. Gaps were observed in all six surgeries. Surgical step switching in all cases indicates that although a general workflow can be observed, the surgeon would perform a step for a certain duration, then return to that same step, displaying unique workflow signatures.

Individual differences across cases can be observed, as well. In surgery no. 3, extensive adhesiolysis was performed at the beginning of the procedure (Figure 2, broad orange bar). This may not be required in all cases, as variable presence of adhesions may eliminate the need for this step. Additionally, surgery no. 4 exhibits a longer duration for removal of the uterus, which follows the colpotomy. This case represents a patient with a large uterus myomatosus, which could not be retrieved through the vagina, so a laparoscopic morcellation was performed. This is similar to surgery no. 6, although the removal of the uterus is in reverse order to surgery no. 4. In this particular surgery, a vaginal morcellation was performed for large uterus myomatosus followed by laparoscopic suturing of the colpotomy. Together, this visualization uncovers both similarities and differences in individual hysterectomies and reflects unique components to aide in analysis of surgical workflow. Patient characteristics are displayed in Table 2.

Step transition probabilities elucidate consolidated workflows of rTLH

Figure 3 shows the probability of the sequence of the surgical steps of all six rTLH surgeries. Lighter squares indicate a higher probability and black indicates that the transition never occurred. All surgeries start with the default “start” step and then the next step with highest probability is “mobilize colon/removal of adhesions (optional),” but “dissection of fallopian tube (r side) (optional)” and “division of the broad ligament (r side)” also occur with smaller probability. As most surgeries end with the vaginal cuff closure this square is the lightest in the last line. In one case the uterus was morcellated due to the size after the vaginal cuff closure, so there is also a square indicating the probability of the removal of the uterus which is darker than the one for vaginal cuff closure.

Figure 3

Figure 3. Step transition probability from six procedures from two surgeons combined. The ordering is right/left for paired steps.

Surgical step order was determined from the calculated median fractional step order, but if a step had both a left and a right variant (e.g., Division of the Broad Ligament), then right was ordered before left. This fixed ordering provided consistency, as the right and left variants of a step had similar fractional step orders. Dissection of the infundibulopelvic ligament was excluded in this figure, as this step occurred only twice, once left and once right in case 5. A linear progression of the individual surgical steps from the upper left to the lower right corner of the diagram is apparent. Deviations from the direct diagonal indicate deviation from the standardized sequence of the individual surgical steps.

Note that the center diagonal, consisting of a surgical step followed by the same surgical step, is almost black with the exception of “vaginal cuff closure” indicating, that there was an alternation of steps. The parallel diagonal lighter lines beside the “black” line indicate a tendency of surgeons to keep working on the same side.

OPI comparisons for colpotomy and vaginal cuff closure

OPIs have been shown to provide insight into intraoperative surgeon behavior (13, 14). In addition to comparison of surgical step duration and workflow, data captured directly from the robotic data streams, such as events of energy use, clutching behavior, and camera movements were calculated into OPIs and observed. For detailed analysis of OPIs we choose the two most complex and standardized surgical steps of TLH: colpotomy and vaginal cuff closure. When comparing colpotomy to vaginal cuff closure for all six surgeries time (seconds) needed for the step (281.1 ± 83.7 vs. 677.5 ± 81.0) and the events of energy use (38.2 ± 11.3 vs. 6.3 ± 3.1) were significantly different (p < 0.001) (see Figure 4). For the events of camera movement and clutch use no differences were observed.

Figure 4

Figure 4. Whisker box plot. Comparison of event counts (clutch use, camera movement and energy use) between colpotomy and vaginal cuff closure for all six surgeries.

Importantly, OPI differences between the two surgeons were observed (see Figure 5). In comparable time of the respective steps, the event count for camera movement varies significantly between the two surgeons during the colpotomy step (surgeon A: 35.3 ± 7.6, surgeon B: 18.3 ± 5.3; p = 0.03) but not for vaginal cuff closure (surgeon A: 21.7 ± 15.1, surgeon B: 14.7 ± 5.3; p = 0.28). Clutching event count (addition of event clutch left, right and both) showed no significant differences between the two surgeons performing colpotomy and vaginal cuff closure (colpotomy: surgeon A: 7.3 ± 3.1, surgeon B: 2.7 ± 0.5; p = 0.051 and vaginal cuff closure: surgeon A: 9.0 ± 0.8, surgeon B: 6.3 ± 4.9; p = 0.25). Use of energy during colpotomy and vaginal cuff closer showed no significant difference between the two surgeons (colpotomy: surgeon A: 35.3 ± 9.3, surgeon B: 41.0 ± 12.3; p = 0.09 and vaginal cuff closure: surgeon A: 4.3 ± 2.4, surgeon B: 8.3 ± 2.5; p = 0.32), despite energy use was, as typical for this step, more frequently during colpotomy. Outliers in the frequency of current application indicate difficulty in hemostasis at the vaginal edges. Although the difference in clutching during colpotomy is not significant, there is a trend towards more actions for surgeon A, which provides preliminary feasibility for the need for future investigation.

Figure 5

Figure 5. Event counts for clutch use, camera movements, and energy use can be compared across surgeons for specific steps. Box and whisker plots comparing event counts of clutching, camera movements, and energy use during colpotomy (top) and vaginal cuff closure (bottom) for Surgeon A and B during rTLH. “x” represents mean.

Discussion

This study provides the gynecology field a novel methodology to investigate OPIs in rTLH. We utilize this method to compare surgical workflows and surgeon behaviors across cases and surgeons. We show that annotation and visualization of independent surgical steps enables workflow comparisons across individual surgeries, as well as identification of OPIs differences and similarities in surgeon behavior. This could be a tool to monitor and adjust learning plans for gynecologic robotic surgery trainees. Together, this work lays the foundation for future gynecology studies using case segmentation and OPIs.

Hysterectomy is a highly standardized operation with fixed sequential surgical steps (15). As such, it was an ideal model to visualize surgical workflow changes for a small data set across two surgeons. In the step transition probability analysis (Figure 3), a clear workflow can be seen reliably from step to predicted step, supporting the high standardization of this procedure. We do observe a slight divergent dimming in the middle of the heat map, illuminating two distinct surgical approaches across surgeons; Surgeon A began hysterectomies on the left side and Surgeon B on the right side. Together, this work provides feasibility for unique surgical workflow signatures, which could be used for training, identification of complex cases, surgical techniques, and more. Metchik demonstrated that using a forward and backward entropy, similar to our model, behavioral patterns in the change between individual surgical steps can be shown in order to improve learning curves and workflows (16).

In this study, we analyzed the two most standard surgical steps in rTLH in detail: colpotomy and closure of the vaginal cuff. Since the circumference of the vagina must be viewed during the colpotomy, this surgical step exhibits high counts of camera movements. The use of energy is also high during this step as the vagina is opened by using monopolar energy. Despite both surgeons being experienced in rTLH, remarkably, clear OPI differences were observed. Thus, with the help of OPIs, not only can surgical workflow be elucidated, differences in surgical techniques and preferences can be identified and compared. In the future, it may be possible to distinguish between different surgeons and different surgeries based on the analysis of OPI signatures alone.

Our proposed methodology may enable tailored gynecology learning plans and opportunities to track learning progress. Although this study compared surgeries performed by experienced surgeons, it is likely that there will also be differences between experts and trainees, as has been shown in previous OPI studies in other specialties (10, 13). Other studies have shown the potential for similar metric data to track learning progression, such as Turner’s et al work in simulator studies (17) and Ma’s et al work in tissue models (18). These and other investigations of OPI utility have been increasing for the past 5 years, showing promise for surgical workflow analysis, training, skill, and correlation to patient outcomes (18, 19). This work is most prevalent in urology (13, 14, 18, 19), with limited published work emerging in the thoracic (9, 10) and general surgery specialties (8, 11, 20). However, no studies to date have utilized such technologies and methodologies in gynecology. As such, much work is needed in clinical gynecology cases to validate this potential.

International societies such as the European Society for Gynaecological Endoscopy (ESGE) have recognized the importance of developing structured training in robotics and have drawn up their own curriculum (Gynaecological Endoscopic Surgical Education and Assessment—GESEA programme) (21). In the future, analyses of segmented videos and OPIs could become part of such a structured learning concept in order to objectively quantify learning progress and possibly compare it with a large group of robotic surgeons.

The addition of visualization of surgical workflows, as shown in Figure 2, to the analysis of OPIs provide the additional advantage of identifying difficult situations during the surgery. This can be helpful in monitoring learning progress. Unusual situations can be reviewed retrospectively using video to identify deviations from the standard to fine tune training.

Although we present significant advantages to OPI evaluations, this study has a number of limitations. It must be noted that this work is a feasibility study on the use of a recorder for video and metric data for the use of the da Vinci Surgical System, which initially covers only six surgeries. Further studies are needed to support the data presented here and pave the way for routine use of OPI measurements. Additionally, this study does not utilize any trainees or surgeons that are in the initial stages of their learning curve, which will be critical for future studies. Although we evaluate surgeon behavior use as it pertains to camera manipulation, energy use, and clutching, we did not investigate any kinematic indicators of performance, which will be needed to elucidate surgeon behaviors in the future. Since a barbed thread was used for the vaginal flap closure, the knotting, which requires special fine motor skills, was not part of the study.

Together, the foundation laid in this work opens the door to countless and critical future investigations for truly objective characterizations and inquiry of intraoperative surgical behaviors, which can be used to train exceptional surgeons objectively and efficiently, leading to better patient outcomes.

Conclusion

This pilot study presents a novel methodology to analyze and compare individual hysterectomy procedures across surgeons with truly objective measurements. Through collection of robotic data streams and standardized segmentation of hysterectomy cases, OPI measurements for specific rTLH surgery steps can be reliably calculated and compared to those of other surgeons. Utilization of this methodology provides opportunity for critical standardization to the gynecology field, which could be integrated into individualized training plans in the future. However, more data is needed to establish context surrounding these metrics as they pertain to gynecology.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Ethics statement

The studies involving humans were approved by the Ethics Committee of Tübingen University Medical Faculty and University Hospital. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

FN: Conceptualization, Data curation, Formal analysis, Project administration, Writing – original draft. SB: Conceptualization, Writing – review & editing. AB: Writing – review & editing. MS: Writing – original draft, Writing – review & editing. LP: Data curation, Formal analysis, Writing – review & editing. XL: Data curation, Formal analysis, Writing – review & editing. ME: Data curation, Formal analysis, Writing – review & editing. CW: Writing – review & editing. TD: Data curation, Formal analysis, Visualization, Writing – original draft. CR: Writing – review & editing. BK: Conceptualization, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This study was funded by the Department of Obstetrics and Gynecology, University Hospital Tübingen with the support of Intuitive Surgical SÀRL. Intuitive Surgical SÀRL provided financial support for study-related administration and the use of an Intuitive Data Recorder (IDR) was granted during the course of the study. The funder was not involved in the study design, collection, analysis, interpretation of data, the writing of this article, or the decision to submit it for publication.

Acknowledgments

Part of this study was presented at the DGGG (German Society of Gynecology and Obstetrics) Congress in Munich, 14.10.2022 as oral presentation, at the ESGE (European Society of Gynecological Endoscopy) Congress in Lisbon, 03.-05.10.2022 as ePoster and at the ESGE Congress in Brussels, 03.10.2023, as oral presentation. We acknowledge support from the Open Access Publication Fund of the University of Tuebingen.

Conflict of interest

MS, LP, XL, and ME were employed by Intuitive Surgical Inc.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Abbreviations

OPI, objective performance indicator; rTLH, robot-assisted total laparoscopic hysterectomy.

References

1. Nobbenhuis, MAE, Gul, N, Barton-Smith, P, O’Sullivan, O, Moss, E, and Ind, TEJ. Robotic surgery in gynaecology. BJOG Int J Obstet Gynaecol. (2022) 130:17242. doi: 10.1111/1471-0528.17242

Crossref Full Text | Google Scholar

2. Muaddi, H, Hafid, ME, Choi, WJ, Lillie, E, de Mestral, C, Nathens, A, et al. Clinical outcomes of robotic surgery compared to conventional surgical approaches (laparoscopic or open): a systematic overview of reviews. Ann Surg. (2021) 273:467–73. doi: 10.1097/SLA.0000000000003915

Crossref Full Text | Google Scholar

3. Leal Ghezzi, T, and Campos Corleta, O. 30 years of robotic surgery. World J Surg. (2016) 40:2550–7. doi: 10.1007/s00268-016-3543-9

Crossref Full Text | Google Scholar

4. Meier, M, Horton, K, and John, H. Da Vinci© skills simulator™: is an early selection of talented console surgeons possible? J Robot Surg. (2016) 10:289–96. doi: 10.1007/s11701-016-0616-6

Crossref Full Text | Google Scholar

5. Soomro, NA, Hashimoto, DA, Porteous, AJ, Ridley, CJA, Marsh, WJ, Ditto, R, et al. Systematic review of learning curves in robot-assisted surgery. BJS Open. (2020) 4:27–44. doi: 10.1002/bjs5.50235

PubMed Abstract | Crossref Full Text | Google Scholar

6. Eddib, A, Jain, N, Aalto, M, Hughes, S, Eswar, A, Erk, M, et al. An analysis of the impact of previous laparoscopic hysterectomy experience on the learning curve for robotic hysterectomy. J Robot Surg. (2013) 7:295–9. doi: 10.1007/s11701-012-0388-6

Crossref Full Text | Google Scholar

7. Nezhat, C, and Lakhi, N. Learning experiences in robotic-assisted laparoscopic surgery. Best Pract Res Clin Obstet Gynaecol. (2016) 35:20–9. doi: 10.1016/j.bpobgyn.2015.11.009

Crossref Full Text | Google Scholar

8. Kaoukabani, G, Gokcal, F, Fanta, A, Liu, X, Shields, M, Stricklin, C, et al. A multifactorial evaluation of objective performance indicators and video analysis in the context of case complexity and clinical outcomes in robotic-assisted cholecystectomy. Surg Endosc. (2023) 37:8540–51. doi: 10.1007/s00464-023-10432-z

PubMed Abstract | Crossref Full Text | Google Scholar

9. Brown, KC, Bhattacharyya, KD, Kulason, S, Zia, A, and Jarc, A. How to bring surgery to the next level: interpretable skills assessment in robotic-assisted surgery. Visc Med. (2020) 36:463–70. doi: 10.1159/000512437

PubMed Abstract | Crossref Full Text | Google Scholar

10. Lazar, JF, Brown, K, Yousaf, S, Jarc, A, Metchik, A, Henderson, H, et al. Objective performance indicators of cardiothoracic residents are associated with vascular injury during robotic-assisted lobectomy on porcine models. J Robot Surg. (2022) 17:669–76. doi: 10.1007/s11701-022-01476-9

PubMed Abstract | Crossref Full Text | Google Scholar

11. Devin, CL, Gillani, M, Shields, MC, Eldredge, K, Kucera, W, Rupji, M, et al. Ratio of economy of motion: a new objective performance Indicator to assign consoles during dual-console robotic Proctectomy. Am Surg. (2023) 89:3416–22. doi: 10.1177/00031348231161767

Crossref Full Text | Google Scholar

12. Lazar, JF, Jarc, A, and Oh, D. Task-based objective performance indicators in robotic lobectomy offer a novel avenue for case assessments. JAMA Surg. (2023) 158:1103. doi: 10.1001/jamasurg.2023.0363

Crossref Full Text | Google Scholar

13. Hung, AJ, Chen, J, Jarc, A, Hatcher, D, Djaladat, H, and Gill, IS. Development and validation of objective performance metrics for robot-assisted radical prostatectomy: a pilot study. J Urol. (2018) 199:296–304. doi: 10.1016/j.juro.2017.07.081

PubMed Abstract | Crossref Full Text | Google Scholar

14. Hung, AJ, Oh, PJ, Chen, J, Ghodoussipour, S, Lane, C, Jarc, A, et al. Experts vs super-experts: differences in automated performance metrics and clinical outcomes for robot-assisted radical prostatectomy. BJU Int. (2019) 123:861–8. doi: 10.1111/bju.14599

PubMed Abstract | Crossref Full Text | Google Scholar

15. Pados, G, Becker, S, Rovira Negre, R, Rabischong, B, Ferreira, H, Rossitto, C, et al.. Working group of ESGE: surgical steps of total laparoscopic hysterectomy: part 1: benign disease by the European Society for Gynaecological Endoscopy (ESGE). Facts Views Vis Obgyn. (2019) 11:103–10.

Google Scholar

16. Metchik, A, Bhattacharyya, K, Yousaf, S, Jarc, A, Oh, D, and Lazar, JF. A novel approach to quantifying surgical workflow in robotic-assisted lobectomy. Int J Med Robot. (2023) 20:e2546. doi: 10.1002/rcs.2546

Crossref Full Text | Google Scholar

17. Turner, TB, and Kim, KH. Mapping the robotic hysterectomy learning curve and re-establishing surgical training metrics. J Gynecol Oncol. (2021) 32:e58. doi: 10.3802/jgo.2021.32.e58

PubMed Abstract | Crossref Full Text | Google Scholar

18. Ma, R, Lee, RS, Nguyen, JH, Cowan, A, Haque, TF, You, J, et al. Tailored feedback based on clinically relevant performance metrics expedites the acquisition of robotic suturing skills-an unblinded pilot randomized controlled trial. J Urol. (2022) 208:414–24. doi: 10.1097/JU.0000000000002691

PubMed Abstract | Crossref Full Text | Google Scholar

19. Hung, AJ, Chen, J, Che, Z, Nilanon, T, Jarc, A, Titus, M, et al. Utilizing machine learning and automated performance metrics to evaluate robot-assisted radical prostatectomy performance and predict outcomes. J Endourol. (2018) 32:438–44. doi: 10.1089/end.2018.0035

PubMed Abstract | Crossref Full Text | Google Scholar

20. Meara, M, Pieper, H, Shields, M, Woelfel, I, Wang, T, Renton, D, et al. What influences general surgery residents’ prospective entrustment and operative time in robotic inguinal hernia repairs. Surg Endosc. (2023) 37:7908–13. doi: 10.1007/s00464-023-10242-3

PubMed Abstract | Crossref Full Text | Google Scholar

21. ESGE , Pathways GESEA Robotics Educational Programme. Available at: https://gesea.eu/pathways/robotics/, last access July 04th 2024, (2024)

Google Scholar