High fidelity simulation of the endoscopic transsphenoidal approach: Validation of the UpSurgeOn TNS Box

Objective: Endoscopic endonasal transsphenoidal surgery is an established technique for the resection of sellar and suprasellar lesions. The approach is technically challenging and has a steep learning curve. Simulation is a growing training tool, allowing the acquisition of technical skills pre-clinically and potentially resulting in a shorter clinical learning curve. We sought validation of the UpSurgeOn Transsphenoidal (TNS) Box for the endoscopic endonasal transsphenoidal approach to the pituitary fossa.

Methods: Novice, intermediate and expert neurosurgeons were recruited from multiple centres. Participants were asked to perform a sphenoidotomy using the TNS model. Face and content validity were evaluated using a post-task questionnaire. Construct validity was assessed through post-hoc blinded scoring of operative videos using a Modified Objective Structured Assessment of Technical Skills (mOSAT) and a Task-Specific Technical Skill scoring system.

Results: Fifteen participants were recruited of which n = 10 (66.6%) were novices and n = 5 (33.3%) were intermediate and expert neurosurgeons. Three intermediate and experts (60%) agreed that the model was realistic. All intermediate and experts (n = 5) strongly agreed or agreed that the TNS model was useful for teaching the endonasal transsphenoidal approach to the pituitary fossa. The consensus-derived mOSAT score was 16/30 (IQR 14–16.75) for novices and 29/30 (IQR 27–29) for intermediate and experts (p < 0.001, Mann–Whitney U). The median Task-Specific Technical Skill score was 10/20 (IQR 8.25–13) for novices and 18/20 (IQR 17.75–19) for intermediate and experts (p < 0.001, Mann-Whitney U). Interrater reliability was 0.949 (CI 0.983–0.853) for OSATS and 0.945 (CI 0.981–0.842) for Task-Specific Technical Skills. Suggested improvements for the model included the addition of neuro-vascular anatomy and arachnoid mater to simulate bleeding vessels and CSF leak, respectively, as well as improvement in materials to reproduce the consistency closer to that of human tissue and bone.

Conclusion: The TNS Box simulation model has demonstrated face, content, and construct validity as a simulator for the endoscopic endonasal transsphenoidal approach. With the steep learning curve associated with endoscopic approaches, this simulation model has the potential as a valuable training tool in neurosurgery with further improvements including advancing simulation materials, dynamic models (e.g., with blood flow) and synergy with complementary technologies (e.g., artificial intelligence and augmented reality).

Introduction

Simulation-based training has gained widespread acceptance within neurosurgery over the last decade. This has primarily been driven by restrictions on trainee working hours, increased sub-specialisation and the growing demands to improve patient safety and quality of care (1). These restrictions have meant it has become increasingly challenging to gain wide exposure to neurosurgical procedures, develop surgical skills and achieve the core clinical competencies. These challenges have further been exacerbated by the recent COVID-19 pandemic, which led to a reduction in elective and non-elective cases and subsequently impacted neurosurgical training (2, 3).

Simulation has been proposed as a potential solution to these challenges by allowing for the acquisition and development of technical and non-technical skills in a time-effective manner (4). It provides a safe, realistic, controlled environment where technically demanding tasks can be performed to completion and errors can be made. It permits trainees to enhance their understanding of complex anatomy and reduces the learning curve of surgical procedures (5, 6).

High-fidelity simulation is emerging as a powerful, cost-effective resource within surgical education and has gained growth in sub-speciality areas in neurosurgery, such as aneurysmal surgery (7–9) and neuro-oncology (10). With minimally invasive approaches becoming a standard in the neurosurgical armamentarium, the use of high-fidelity simulation has become an increasingly important method to overcome the steep learning curve associated with these techniques. This is particularly relevant in endoscopic endonasal transsphenoidal surgery due to the complex anatomy, high risk of complications and difficult ergonomics (11, 12). Additionally, there is heterogeneity in the way these procedures are performed which may lead to variation in outcomes (13). As such, dedicated fellowships in endoscopic endonasal surgery are often required to achieve competencies. However, gaining proficiency in endoscopic endonasal techniques requires regular practice and cannot be achieved by observation alone.

There are several simulators available for neuroendoscopic approaches ranging from low fidelity bell pepper trainers to augmented reality (AR)/virtual reality (VR) platforms (14). Many of these have demonstrated validity and utility in improving operative performance (15–18). VR/AR platforms have several advantages within neurosurgery including their low cost, ability to simulate a wide variety of cases and unlimited repetition. However, a drawback of VR/AR simulators is they often lack realistic haptic feedback. More recently, a high-fidelity hybrid (physical and virtual) simulator (UpSurgeOn, TNS Box) has been developed as means to provide a realistic, hands-on experience and improve technical skills in endoscopic endonasal transsphenoidal surgery.

Before implementation as a training tool, however, simulation models require validation to assess their effectiveness. Steps of validation include assessment for realism (face validity), usefulness as a training tool as viewed by experts (content validity) and the ability to differentiate levels of surgical experience (construct validity) (19, 20). At present, there are no validated studies for the UpSurgeOn TNS model. Herein, we aim to determine if the UpSurgeOn TNS Box is a valid education tool for endoscopic endonasal transsphenoidal surgery by assessing its face, content and construct validity.

Methods

Model

The Transsphenoidal Box (TNS) is a high-fidelity simulator for the transsphenoidal endonasal approach to the pituitary fossa (Figures 1, 2). The TNS box comprises of a nasal cavity with a 3D face overlay and is manufactured using silicones and resins through 3D printing. The model enables users to explore the nasal cavity, identify anatomical landmarks and resect pituitary tumours endoscopically through careful dissection and drilling techniques.

FIGURE 1

Figure 1. Upsurgeon TNS Box simulator.

FIGURE 2

Figure 2. Workstation set up with the UpSurgeOn TNS Box and endoscope.

Participants

Fifteen participants were enrolled in a prospective, single-centre study. Participants were recruited from multiple neurosurgical centres within the United Kingdom. Subjects were categorised as novice- those in neurosurgical training, or intermediate/experts- senior neurosurgical residents and fellows with prior dedicated pituitary training, and consultants. The novice group had no experience in performing endoscopic transsphenoidal approaches whilst the intermediate and expert participants had varying experience. Participants completed a pre-study questionnaire on demographics and self-report of surgical experience (Appendix 1). Demographic data collected included age, handedness, speciality, stage of training, number of operations observed, assisted and performed, and number of transsphenoidal cases observed, assisted and performed (Tables 1, 2).

TABLE 1

Table 1. Participant demographics.

TABLE 2

Table 2. Participant experience.

Surgical task

Participants were asked to perform a sphenoidotomy using the TNS Box. The task was performed endoscopically using the Olympus S200 visera elite endoscope and participants were supplied with an instrument set. The task focused on the core steps and specific technical skills required during the nasal phase of the eTSA derived from an expert Delphi consensus (13). The four key steps examined included:

(1) identification of the choana, septum and nasal turbinates

(2) lateral displacement of the superior and middle nasal turbinate

(3) ability to identify and assess limits of the sphenoid ostium and

(4) performing an anterior sphenoidotomy appropriately whilst protecting nasoseptal pedicle.

Step (1) on the TNS Box is demonstrated in Figure 3 and anterior sphenoidotomy in Figure 4.

FIGURE 3

Figure 3. Participant identifying choana and nasal turbinate's on the TNS Box.

FIGURE 4

Figure 4. Anterior sphenoidotomy performed on the TNS Box.

Face and content validity

Intermediate and expert participants completed a post-task questionnaire to assess simulator realism (face validity) and its usefulness as a training tool (content validity) based on a 5-point Likert scale (1 = Strongly Disagree, 2 = Disagree, 3 = Neither Agree nor Disagree, 4 = Agree, 5 = Strongly Agree) (Appendix 2) (28).

Construct validity

Construct validity was assessed through a post-hoc blinded review of recorded videos by two expert authors. Both expert authors had assisted and performed >150 eTSA procedures and have trainer roles in this sub-speciality. Operative performance was assessed using the modified Objective Structured Assessment of Technical Skills (OSATS) criteria (Appendix 3) (29). Six metrics were assessed, and each was ranked from 1 to 5. A Task-Specific Technical Skill scoring system was generated using published workflow analyses on endoscopic transsphenoidal pituitary adenoma resection (13). Each of the four task steps were ranked from 1 denoting completely inadequate to 5 denoting excellent (Appendix 3).

Statistical analysis

Median values for OSATS and Task-Specific Technical skills were analysed for statistical differences between groups using Mann-Whitney U test. Analysis was performed using GraphPad (GraphPad Software Inc, California, USA) and SPSS Version 28.0 (IBM, UK). Results with a p-value of <0.05 were considered a statistically significant difference.

Results

Participants

Fifteen participants (10 novices, 1 intermediate and 4 experts) completed the surgical task. The novice group consisted of 10 neurosurgical trainees ranging from year one of training to year six and had minimal endoscopic pituitary experience. The intermediate and expert group were combined and consisted of one consultant neurosurgeon, three senior clinical pituitary fellows and one senior resident all of whom had dedicated sub-speciality experience. The median age of the novice group was 30 years (range: 26–33 years), whereas the median age of the intermediate and expert group was 39 years (range: 33–41 years). The group consisted of 8 females (53.3%) and 7 (46.7%) males. Fourteen participants (93.3%) were right-handed and one participant (6.7%) was left-handed (Table 1).

Face and content validity

All intermediate and expert participants (n = 5, 100%) completed the post-task questionnaire. Sixty percent (n = 3) of intermediate and expert participants agreed that the TNS Box was realistic, whilst forty percent (n = 2) were neutral. Intermediate and expert participants strongly agreed (n = 3, 60%) or agreed (n = 2, 40%) that the TNS Box was useful as a training tool.

Construct validity

The total OSATS and Task-Specific Technical Skills scores were calculated for each participant. Median total OSATS score was 16/30 (IQR 14–16.75) for novices and 29/30 (IQR 27–29) for intermediate and experts (p < 0.001, Mann-Whitney U) (Figure 5). Median Task-Specific Technical Skills score was 10/20 (IQR 8.25–13) for novices and 18/20 (IQR 17.75–19) for intermediate and experts (p < 0.001, Mann- Whitney U) (Figure 6). Six of ten novices did not complete sphenoidotomy whilst all intermediate and experts completed the task.

FIGURE 5

Figure 5. Total OSAT score for novice and intermediate and expert participants. The horizontal solid line in the middle of the box is the median value of the scores and the lower and upper boundaries indicate the 25th and 75th percentiles, respectively. The largest and smallest observed values are shown; lines (whiskers) are drawn from the ends of the box to those values.

FIGURE 6

Figure 6. Total task specific technical skills score for novice and intermediate and expert participants. The horizontal solid line in the middle of the box is the median value of the scores and the lower and upper boundaries indicate the 25th and 75th percentiles, respectively. The largest and smallest observed values are shown; lines (whiskers) are drawn from the ends of the box to those values.

Interrater reliability

Interrater reliability was used to assess the extent to which the two raters agreed. Interrater reliability was 0.949 (CI 0.983–0.853) for OSATS and 0.945 (CI 0.981–0.842) for Task-Specific Technical Skills.

Qualitative feedback

Intermediate and experts were asked to provide feedback on the TNS model. Responses focused on suggested improvements including making the models dynamic by introducing CSF and blood as well as pulsatile vessels. Other responses focused on the materials used suggesting changes to bone to make it less thick as well as changes to soft tissue including the mucosa and tumour. The mucosa was described as too elastic whilst the tumour was too firm and not in keeping with that of human tissue.

Discussion

Principal findings

In this study, we sought validation of the TNS Box for the endoscopic endonasal transsphenoidal approach. The TNS Box was validated according to 3 components: face, content and construct validity. Face validity is a subjective measurement used to determine if the model provides a realistic demonstration of the task. In the context of surgical simulation validation, expert opinions are sought to provide a reliable measure of face validity. This is, in part, due to the increased level of familiarity with the operative condition. Therefore, face validity of the TNS Box in our study was determined by assessing the responses of the intermediate and expert neurosurgeons. Responses from intermediate and experts were variable, with three agreeing the model was realistic and two remaining neutral. On this basis, the TNS Box showed moderate face validity.

A goal of simulation is to closely reproduce the advantages offered by cadaveric training. However, formulating materials to mimic real tissue and reproducing the haptics of the procedure can often be challenging. Although realism of the simulator may be useful for simulation-based training in surgery, several studies have shown the degree of fidelity appears to be independent of educational effectiveness, providing the model meets the functional requirements of the clinical task (6, 30, 31). Other studies have also shown the effect of simulator fidelity depends upon the experience of the user and goals of the learner (32), where low-fidelity simulation is more suitable for initial learning and performance improvement, and high-fidelity simulation more suitable for the transfer of skills and assessment (33). In our study, the TNS Box showed high content validity with all intermediate and experts confirming its utility as a training tool for the eTSA approach. Furthermore, nine of ten novices ranked the TNS Box as a useful learning tool.

Finally, an important consideration when validating a simulation model is its ability to discriminate between different levels of surgical proficiency. For a simulator to have construct validity, experts must outperform novices during standardised simulated tasks. In our study, this was based on performance scores which considered several parameters including technical skills and ability to perform sphenoidotomy. Six of ten novices were unable to perform opening of the sphenoid ostium, whilst all intermediate and experts performed this task. The TNS model showed a high construct validity with statistical differences in both the OSATS and Task-Specific Technical Skills score between the novice and the intermediate and expert groups. The construct validity findings were further supported by high interrater reliability.

Whilst the TNS Box has demonstrated face, content and construct validity, it does however have several limitations. At present, it lacks important neurovascular anatomy including the carotid arteries and sphenopalatine arteries. These are important structures to be aware of whilst undertaking this approach. Another limitation is the type of material used. Compared to human tissue, the mucosa is more elastic and the sella is thicker requiring a drill to access the tumour. As well as that, the tumour is too firm. Due to these limitations, phases beyond the nasal phase of the procedure were not evaluated. These aforementioned features could be addressed in order to reproduce the consistency close to that of human soft tissue and bone and thus enabling evaluation of the sphenoid, sellar and closure phases. Additional features to make the simulator more sophisticated could be the addition of arachnoid mater in order to simulate CSF leak.

Findings in context of existing literature

Endoscopic endonasal transsphenoidal surgery is a technically demanding operation with a steep learning curve. The procedures tend to be low volume but require a high number to achieve surgical proficiency. As the use of this approach to treat sellar and suprasellar lesions continues to expand, there becomes a greater need for the use of training tools to enhance the understanding of anterior skull base anatomy and advance neuroendoscopy skills.

There are several challenges inherent to the endoscopic endonasal transsphenoidal approach which make the use of simulation models an ideal platform to develop surgical techniques. Firstly, the endoscope can be technically challenging to use due to its limited angulation and manoeuvrability. Secondly, endoscopic transsphenoidal approach (eTSA) to the anterior skull base often requires extensive bone work adjacent to critical neurovascular structures. Thirdly, access to the area is through a narrow surgical corridor. Limited space, due to the presence of the endoscope and surgical instruments in the field, creates ergonomic challenges. Simulation models enable the trainee to develop these technical skills and gain familiarity with the instruments used. They enable performance feedback in a safe environment and have the potential to create and develop skills transferrable to the operating theatre.

A variety of simulators for training in endoscopic skull base surgery have been produced (Table 3). Cadaveric models have been used to simulate injury to the internal carotid artery (ICA), a rare but life-threatening complication of the eTSA. Donoho et al. perfused cadavers with artificial blood and demonstrated face and construct validity, as well as utility as an educational tool (23, 34). Other cadaveric studies have looked at simulating CSF leak repair and demonstrated face, content and construct validity (24).

TABLE 3

Table 3. Examples of current simulation models for the endoscopic endonasal approach.

Advances in 3D printing have allowed for simulation models to be created for complex surgical pathology with adequate tactile feedback. Studies have demonstrated their use in anatomic education (35) and pre-operative planning of endoscopic endonasal transsphenoidal surgery. Huang et al. showed the potential of 3D printing to demonstrate adjacent anatomical relationships and aid planning techniques for tumour resection (21). Other studies have used 3D models to improve eTSA technical skills including drilling techniques, curetting, aspiration and navigation (36, 37). Although these models have shown potential application, they have not undergone validation. Other efforts at the simulation of endoscopic endonasal skull base surgery have included VR and AR simulation models (17, 26). Although VR/AR has the potential as an effective educational tool, limitations in haptic feedback remain a major concern.

In the future, surgical simulations will become more sophisticated through the integration of high-fidelity haptic feedback. This will offer several advantages including improved tissue manipulation, better spatial awareness and a reduction in surgical complications (38). The integration of AR into high-fidelity simulators has been a promising new development. AR overlays have the potential to simulate variations in surgical anatomy, illustrate various surgical approaches and develop complex surgical skills (39). Furthermore, this provides a good solution for high-fidelity neurosurgical training in low-middle income countries (40). The application of AI to surgical simulation may prove to be another useful addition by providing objective feedback on performance and differentiating between skill levels (41). Studies have shown its potential for improving operative efficiency and patient outcomes through its ability to recognise stages of the procedure, provide guidance on instruments required and identify important anatomic relations (42).

Strengths and limitations

There are several strengths to our study. Construct validity was assessed using the OSATS score which is a well-established tool used to assess technical skills. This allowed for a valid and reliable assessment of the participant's surgical skills. The task assessed was derived from an expert Delphi consensus on the phases and core steps for endoscopic transsphenoidal pituitary adenoma resection (13). This provided a framework to assess knowledge of the approach and surgical ability. Operative videos were assessed blindly by two expert neurosurgeons. The inter-rater reliability was high suggesting a high degree of agreement between the two examiners.

This study however has several limitations. Firstly, we did not assess predictive validity. This would be a useful measure to determine improvement in skills and performance in the operative theatre. However, this concept is difficult to evaluate due to patient safety and ethical concerns. Secondly, the AR overlay function for the TNS Box was not used in our study in order to simplify the task. However, this is a useful addition to the model which can be used to simulate variations in anatomy and demonstrate various surgical approaches. Thirdly, the number of participants involved in the study was rather low, however this is in keeping with existing validation studies (43). In addition, we combined intermediate and expert surgeons within our study. This meant there was some variation in surgical procedures observed, assisted and performed. However, several studies within the literature support this method for validation studies (43). Lastly, the model had a few limitations including the types of material used making it largely inconsistent with that of human tissue, the lack of bleeding vessels and important neurovascular structures, and the lack of arachnoid mater to simulate CSF leak. As a result of the unrealistic consistency of the tumour and the sphenoid sinus anatomy, only the nasal phase of the procedure was examined. With the iterative improvement of the TNS Box, the sphenoid, sellar and closure phases can be evaluated.

Conclusions

This present study demonstrates face, content and construct validity for the TNS Box and highlights the model as a potentially useful surgical skills training tool for the eTSA approach. Further improvements include advancing simulation materials, dynamic models (e.g., with blood flow), synergy with complimentary technologies (e.g., AI and AR), and integration as a supplement to modern surgical curricula.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author contributions

NN– data curation; formal analysis; writing—original draft preparation; writing—reviewing and editing. DZK – conceptualisation, data curation; methodology, writing—reviewing and editing. JGH –conceptualisation, data curation; methodology, writing—reviewing and editing. JB – writing—reviewing and editing. AB – writing—reviewing and editing. JD – writing—review and editing. SS – writing—reviewing and editing. FN – writing—reviewing and editing. ND – conceptualisation; methodology; writing—reviewing and editing; supervision. HJM – conceptualisation; methodology; writing—reviewing and editing; supervision. All authors contributed to the article and approved the submitted version.

Funding

D.Z.K. is funded by an NIHR Academic Clinical Fellowship. SS is funded by the Francis Crick Institute.

Acknowledgments

We would like to thank Wellcome / EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London Hospitals and University College London BRC Neuroscience for supporting the research. We would also like to thank Health Education England London School of Surgery COVID recovery programme for supporting the course.

Conflict of interest

FN is the founder and CEO of UpSurgeOn. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Bina RW, Lemole GM, Dumont TM. On resident duty hour restrictions and neurosurgical training: review of the literature. J Neurosurg. (2016) 124(3):842–8. doi: 10.3171/2015.3.JNS142796

CrossRef Full Text | Google Scholar

2. Robertson FC, Lippa L, Broekman MLD. Task shifting and task sharing for neurosurgeons amidst the COVID-19 pandemic. J Neurosurg. (2020) 133(1):5–7. doi: 10.3171/2020.4.JNS201056

CrossRef Full Text | Google Scholar

3. Tsermoulas G, Zisakis A, Flint G, Belli A. Challenges to neurosurgery during the coronavirus disease 2019 (COVID-19) pandemic. World Neurosurg. (2020) 139:519–25. doi: 10.1016/j.wneu.2020.05.108

CrossRef Full Text | Google Scholar

4. Akhtar KSN, Chen A, Standfield NJ, Gupte CM. The role of simulation in developing surgical skills. Curr Rev Musculoskelet Med. (2014) 7(2):155–60. doi: 10.1007/s12178-014-9209-z

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Harrop J, Lobel DA, Bendok B, Sharan A, Rezai AR. Developing a neurosurgical simulation-based educational curriculum: an overview. Neurosurgery. (2013) 1;73(suppl_1):S25-9. doi: 10.1227/NEU.0000000000000101

CrossRef Full Text | Google Scholar

6. Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch-Jensen P. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg. (2004) 91(2):146–50. doi: 10.1002/bjs.4407

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Wang JL, Yuan ZG, Qian GL, Bao WQ, Jin GL. 3D Printing of intracranial aneurysm based on intracranial digital subtraction angiography and its clinical application. Med (United States). (2018) 97(24):1–6. doi: 10.1097/MD.0000000000011103

CrossRef Full Text | Google Scholar

8. Mashiko T, Kaneko N, Konno T, Otani K, Nagayama R, Watanabe E. Training in cerebral aneurysm clipping using self-made 3-dimensional models. J Surg Educ. (2017) 74(4):681–9. doi: 10.1016/j.jsurg.2016.12.010

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Belykh E, Miller EJ, Lei T, Chapple K, Byvaltsev VA, Spetzler RF, et al. Face, content, and construct validity of an aneurysm clipping model using human placenta. World Neurosurg. (2017) 105:952-60.e2. doi: 10.1016/j.wneu.2017.06.045

CrossRef Full Text | Google Scholar

10. Gélinas-Phaneuf N, Choudhury N, Al-Habib AR, Cabral A, Nadeau E, Mora V, et al. Assessing performance in brain tumor resection using a novel virtual reality simulator. Int J Comput Assist Radiol Surg. (2014) 9(1):1–9. doi: 10.1007/s11548-013-0905-8

CrossRef Full Text | Google Scholar

11. Younus I, Gerges MM, Uribe-Cardenas R, Morgenstern PF, Eljalby M, Tabaee A, et al. How long is the tail end of the learning curve? Results from 1000 consecutive endoscopic endonasal skull base cases following the initial 200 cases. J Neurosurg. (2021) 134(3):750–760. doi: 10.3171/2019.12.JNS192600

CrossRef Full Text | Google Scholar

12. Qureshi T, Chaus F, Fogg L, Dasgupta M, Straus D, Byrne RW. Learning curve for the transsphenoidal endoscopic endonasal approach to pituitary tumors. Br J Neurosurg. (2016) 30(6):637–42. doi: 10.1080/02688697.2016.1199786

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Marcus HJ, Khan DZ, Borg A, Buchfelder M, Cetas JS, Collins JW, et al. Pituitary society expert delphi consensus: operative workflow in endoscopic transsphenoidal pituitary adenoma resection. Pituitary. (2021) 24(6):839–53. doi: 10.1007/s11102-021-01162-3

CrossRef Full Text | Google Scholar

14. Gomar-Alba M, Parrón-Carreño T, Narro-Donate JM, Vargas-López AJ, Castelló-Ruiz MJ, García-Pérez F, et al. Neuroendoscopic training in neurosurgery: a simple and feasible model for neurosurgical education. Child’s Nerv Syst. (2021) 37(8):2619–24. doi: 10.1007/s00381-021-05190-z

CrossRef Full Text | Google Scholar

15. Shinomiya A, Shindo A, Kawanishi M, Miyake K, Nakamura T, Matsubara S, et al. Usefulness of the 3D virtual visualization surgical planning simulation and 3D model for endoscopic endonasal transsphenoidal surgery of pituitary adenoma: technical report and review of literature. Interdiscip Neurosurg Adv Tech Case Manag. (2018) 13:13–9. doi: 10.1016/j.inat.2018.02.002

CrossRef Full Text | Google Scholar

16. Thawani JP, Ramayya AG, Abdullah KG, Hudgins E, Vaughan K, Piazza M, et al. Resident simulation training in endoscopic endonasal surgery utilizing haptic feedback technology. J Clin Neurosci. (2016) 34:112–6. doi: 10.1016/j.jocn.2016.05.036

CrossRef Full Text | Google Scholar

17. Rosseau G, Bailes J, del Maestro R, Cabral A, Choudhury N, Comas O, et al. The development of a virtual simulator for training neurosurgeons to perform and perfect endoscopic endonasal transsphenoidal surgery. Neurosurgery. (2013) 73 Suppl 1:85–93. doi: 10.1227/NEU.0000000000000112

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Tai BL, Wang AC, Joseph JR, Wang PI, Sullivan SE, McKean EL, et al. A physical simulator for endoscopic endonasal drilling techniques: technical note. J Neurosurg. (2016) 124(3):811–6. doi: 10.3171/2015.3.JNS1552

PubMed Abstract | CrossRef Full Text | Google Scholar

19. McDougall EM. Validation of surgical simulators. J Endourol. (2007) 21(3):244–7. doi: 10.1089/end.2007.9985

CrossRef Full Text | Google Scholar

20. Van Nortwick SS, Lendvay TS, Jensen AR, Wright AS, Horvath KD, Kim S. Methodologies for establishing validity in surgical simulation studies. Surgery. (2010) 147(5):622–30. doi: 10.1016/j.surg.2009.10.068

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Huang X, Fan N, Wang HJ, Zhou Y, Li X, Jiang XB. Application of 3D printed model for planning the endoscopic endonasal transsphenoidal surgery. Sci Rep. (2021) 11(1):5333. doi: 10.1038/s41598-021-84779-5

CrossRef Full Text | Google Scholar

22. Okuda T, Yamashita J, Fujita M, Yoshioka H, Tasaki T, Kato A. The chicken egg and skull model of endoscopic endonasal transsphenoidal surgery improves trainee drilling skills. Acta Neurochir (Wien). (2014) 156(7):1403–7. doi: doi: 10.1007/s00701-014-2035-7

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Donoho DA, Johnson CE, Hur KT, Buchanan IA, Fredrickson VL, Minneti M, et al. Costs and training results of an objectively validated cadaveric perfusion-based internal carotid artery injury simulation during endoscopic skull base surgery. Int Forum Allergy Rhinol. (2019) 9(7):787–94. doi: 10.1002/alr.22319

PubMed Abstract | CrossRef Full Text | Google Scholar

24. AlQahtani A, Albathi A, Castelnuovo P, Alfawwaz F. Cerebrospinal fluid leak repair simulation model: face, content, and construct validation. Am J Rhinol Allergy. (2021) 35(2):264–71. doi: 10.1177/1945892420952262

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Gagliardi F, Chau AM, Mortini P, Caputy AJ, Gragnaniello C. Skull base neuroendoscopic training model using a fibrous injectable tumor polymer and the nico myriad. J Craniofac Surg. (2018) 29(1):e25–e8. doi: 10.1097/SCS.0000000000004042

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Kim DH, Kim HM, Park JS, Kim SW. Virtual reality haptic simulator for endoscopic Sinus and skull base surgeries. J Craniofac Surg. (2020) 31(6):1811–4. doi: 10.1097/SCS.0000000000006395

CrossRef Full Text | Google Scholar

27. De Notaris M, Palma K, Serra L, Enseñat J, Alobid I, Poblete J, et al. A three-dimensional computer-based perspective of the skull base. World Neurosurg. (2014) 82(6 Suppl):S41–8. doi: 10.1016/j.wneu.2014.07.024

CrossRef Full Text | Google Scholar

28. Joshi A, Kale S, Chandel S, Pal D. Likert Scale: explored and explained. Br J Appl Sci Technol. (2015) 7(4):396. doi: 10.9734/BJAST/2015/14975

CrossRef Full Text | Google Scholar

29. Martin JA, Regehr G, Reznick R, Macrae H, Murnaghan J, Hutchison C, et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg. (1997) 84(2):273–8. doi: 10.1046/j.1365-2168.1997.02502.x

CrossRef Full Text | Google Scholar

30. Grober ED, Hamstra SJ, Wanzel KR, Reznick RK, Matsumoto ED, Sidhu RS, et al. The educational impact of bench model fidelity on the acquisition of technical skill: the use of clinically relevant outcome measures. Ann Surg. (2004) 240(2):374–81. doi: 10.1097/01.sla.0000133346.07434.30

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Chandrasekera SK, Donohue JF, Orley D, Barber NJ, Shah N, Bishai PM, et al. Basic laparoscopic surgical training: examination of a low-cost alternative. Eur Urol. (2006) 50(6):1285–90; 1290–1. doi: 10.1016/j.eururo.2006.05.052

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Brydges R, Carnahan H, Rose D, Rose L, Dubrowski A. Coordinating progressive levels of simulation fidelity to maximize educational benefit. Acad Med. (2010) 85(5):806–12. doi: 10.1097/ACM.0b013e3181d7aabd

CrossRef Full Text | Google Scholar

33. Aggarwal R, Mytton OT, Derbrew M, Hananel D, Heydenburg M, Issenberg B, et al. Training and simulation for patient safety. Qual Saf Health Care. (2010) 19 Suppl 2:i34-43. doi: 10.1136/qshc.2009.038562

CrossRef Full Text | Google Scholar

34. Donoho DA, Pangal DJ, Kugener G, Rutkowski M, Micko A, Shahrestani S, et al. Improved surgeon performance following cadaveric simulation of internal carotid artery injury during endoscopic endonasal surgery: training outcomes of a nationwide prospective educational intervention. J Neurosurg. (2021) 135(5):1347–55. doi: 10.3171/2020.9.JNS202672

CrossRef Full Text | Google Scholar

35. Zheng JP, Li CZ, Chen GQ, Song GD, Zhang YZ. Three-Dimensional printed skull base simulation for transnasal endoscopic surgical training. World Neurosurg. (2018) 111:e773-e82. doi: 10.1016/j.wneu.2017.12.169

CrossRef Full Text | Google Scholar

36. Wen G, Cong ZX, Liu KD, Tang C, Zhong C, Li L, et al. A practical 3D printed simulator for endoscopic endonasal transsphenoidal surgery to improve basic operational skills. Child’s Nerv Syst. (2016) 32(6):1109–16. doi: 10.1007/s00381-016-3051-0

CrossRef Full Text | Google Scholar

37. Narayanan V, Narayanan P, Rajagopalan R, Karuppiah R, Rahman ZAA, Wormald PJ, et al. Endoscopic skull base training using 3D printed models with pre-existing pathology. Eur Arch Oto-Rhino-Laryngology. (2015) 272(3):753-7. doi: 10.1007/s00405-014-3300-3

CrossRef Full Text | Google Scholar

38. Favier V, Subsol G, Duraes M, Captier G, Gallet P. Haptic fidelity: the game changer in surgical simulators for the next decade? Front Oncol. (2021) 11:713343. doi: 10.3389/fonc.2021.713343

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Herur-Raman A, Almeida ND, Greenleaf W, Williams D, Karshenas A, Sherman JH. Next-Generation simulation—integrating extended reality technology into medical education. Front Virtual Real. (2021) 115. doi: 10.3389/frvir.2021.693399

CrossRef Full Text | Google Scholar

40. Nicolosi F, Rossini Z, Zaed I, Kolias AG, Fornari M, Servadei F. Neurosurgical digital teaching in low-middle income countries: beyond the frontiers of traditional education. Neurosurg Focus. (2018) 45(4):E17. doi: 10.3171/2018.7.FOCUS18288

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Mirchi N, Bissonnette V, Yilmaz R, Ledwos N, Winkler-Schwartz A, Del Maestro RF. The virtual operative assistant: an explainable artificial intelligence tool for simulation-based training in surgery and medicine. PLoS One. (2020) 15(2):e0229596. doi: 10.1371/journal.pone.0229596

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Khan DZ, Luengo I, Barbarisi S, Addis C, Culshaw L, Dorward NL, et al. Automated operative workflow analysis of endoscopic pituitary surgery using machine learning: development and preclinical evaluation (IDEAL stage 0). J Neurosurg. (2021) 1(aop) 1–8. doi: 10.3171/2021.6.JNS21923

CrossRef Full Text | Google Scholar

43. Marcus HJ, Hughes-Hallett A, Pratt P, Yang G-Z, Darzi A, Nandi D. Validation of martyn to simulate the keyhole supraorbital subfrontal approach. Bull R Coll Surg Engl. (2014) 96(4):120–1. doi: 10.1308/rcsbull.2014.96.4.120

CrossRef Full Text | Google Scholar

Appendix 1 Pre-study questionnaire on the demographics of each participantName:Age:Handedness:Speciality:Stage of training:Number of neurosurgical cases (of any sort)

(a) Observed

(b) Assisted

(c) Performed

Number of transsphenoidal cases (of any sort)

(a) Observed

(b) Assisted

(c) Performed

Appendix 2 Post-study questionnaire examining the face and content validity of the TNS BoxFace ValidityThe model was realistic?

□ Strongly Agree

□ Agree

□ Neutral

□ Disagree

□ Strong Disagree

Content ValidityThe model was useful?

□ Strongly Agree

□ Agree

□ Neutral

□ Disagree

□ Strongly Disagree

Appendix 3 Score sheet used to assess construct validity of the TNS Box

.