Organ curvature sensing using pneumatically attachable flexible rails in robotic-assisted laparoscopic surgery

In robotic-assisted partial nephrectomy, surgeons remove a part of a kidney often due to the presence of a mass. A drop-in ultrasound probe paired to a surgical robot is deployed to execute multiple swipes over the kidney surface to localise the mass and define the margins of resection. This sub-task is challenging and must be performed by a highly-skilled surgeon. Automating this sub-task may reduce cognitive load for the surgeon and improve patient outcomes. The eventual goal of this work is to autonomously move the ultrasound probe on the surface of the kidney taking advantage of the use of the Pneumatically Attachable Flexible (PAF) rail system, a soft robotic device used for organ scanning and repositioning. First, we integrate a shape-sensing optical fibre into the PAF rail system to evaluate the curvature of target organs in robotic-assisted laparoscopic surgery. Then, we investigate the impact of the PAF rail’s material stiffness on the curvature sensing accuracy, considering that soft targets are present in the surgical field. We found overall curvature sensing accuracy to be between 1.44% and 7.27% over the range of curvatures present in adult kidneys. Finally, we use shape sensing to plan the trajectory of the da Vinci surgical robot paired with a drop-in ultrasound probe and autonomously generate an Ultrasound scan of a kidney phantom.

1 Introduction

Partial nephrectomy is a laparoscopic surgical procedure in which a portion of the kidney is removed. This operation maximises the patient’s postoperative kidney function compared to total nephrectomy (Kaul et al., 2007) as it preserves renal function. Robotic-Assisted Partial Nephrectomy (RAPN) employs robotics during this complex procedure and improves patient outcomes, as detailed in Kaul et al. (2007) and Bhayani (2008). Namely, it shortens hospital stays, reduces post-operative pain, and minimizes recovery time. Robotic assistance for high precision surgical tasks can also reduce surgeon fatigue (Van Der Schatte Olivier et al., 2009; Stefanidis et al., 2010), thus improving accuracy. The RAPN surgical procedure is described in detail in Bhayani (2008). Tumour margin identification can be done in preoperative imaging modalities, Computer Tomography (Su et al., 2009) and Magnetic Resonance Imaging (Shingleton and Sewell, 2001), and this information can be visualised intraoperatively through 3D model visualisation and image-guided navigation Ferguson et al. (2018). Practically however, most RAPN surgeons utilize intraoperative ultrasound (US) to evaluate tumour margins during the procedure (Hekman et al., 2018). In Kaczmarek et al. (2013a), the authors suggest that robotic and laparoscopic approaches have comparable perioperative outcomes when they help surgeons to guide drop-in US probes during the kidney scan, with the former having the advantage of increased surgeon dexterity and potential autonomy.

Despite the additional degrees of freedom (DoF) provided by robotic laparoscopic tools over hand-held laparoscopic tools, RAPN is still a challenging procedure that requires a highly skilled surgeon with years of speciality training (Larcher et al., 2019). US scanning of the kidney is also a demanding sub-task due to slippage of the US probe on the organ surface, and needing to maintain contact between the tissue and the probe while also doing image interpretation with respect to tool actions (Yakoubi et al., 2012; Kaczmarek et al., 2013b; Autorino et al., 2014).

A potential aid for this sub-task is the deployment of soft robotic systems in the surgical workspace. Recently, the use of soft robots for minimally invasive surgery has gathered attention due to their inherent flexibility and compliance with their environment (Runciman et al., 2019). In previous work (Stilli et al., 2019), we presented a Pneumatically Attachable Flexible rail (PAF rail) to enable stable, track-guided US scanning of the kidney during RAPN. The PAF rail is attached to the kidney surface using a continuous suction cup. Stable track-guided US scanning is achieved by connecting the drop-in US probe to the perimeter of the rail. In Ettorre et al. (2019) the authors have investigated the autonomous deployment of the PAF rail on the surface of the organ and its use in intraoperative organ manipulation. In Wang et al. (2020), Wang et al. studied the 3D reconstruction of a mass embedded in a kidney phantom when the PAF rail guides the US probe. Accurate shape sensing of the PAF rail within the surgical field could further improve the deployment of this system on the targeted organ surface while autonomously controlling the probes’ trajectory.

The complete deployment process of the PAF rail and its use with US probes is detailed in Stilli et al. (2019). However, deploying such a device during surgery presents many control challenges. Embedding sensors in the soft robot can give the information needed to meet said control challenges, but these sensors must have the ability to bend, twist, and contort in tandem with the soft robot. In surgical soft robotics, the most common sensing methods are external vision-based systems using intra-operative imaging, Wang et al. (2020); Luo et al. (2015) and electromagnetic tracking, Luo et al. (2015), Lun et al. (2019), particularly in needle-based (Hakime et al., 2012) and catheter-based interventions (Lugez et al., 2017; Schwein et al., 2018). But both these techniques present some drawbacks. Vision-based shape tracking is met with visual occlusion due to the constrained surgical workspace, while electromagnetic tracking is prone to extensive errors due to local field distortions. Consequently, researchers have investigated the fibre-optics sensors in this context (Silvestri and Sche, 2011; Sareh et al., 2015). These fibre-optic sensors are flexible, biocompatible, immune to electromagnetic interference and have small radial dimensions, making them ideal for surgical applications (Mishra et al., 2011). In particular, Fibre Bragg Grating (FBG)-based sensors allow direct multi-point strain measurements along the axial direction of the fibre and can contribute to real-time shape reconstruction (Polygerinos et al., 2010).

In this paper, we assess the performance of the real-time curvature and shape sensing of the PAF rail system using embedded FBG-based shape sensors. We further demonstrate how the PAF rail local shape can help plan a trajectory and autonomously guide an intraoperative US probe, thus having the potential to reduce surgeons’ cognitive load while improving tumour margins identification. This can also improve the 3D reconstruction accuracy of malignant masses while working towards being able to overlay intraoperatively the 3D reconstructed images in the field of view of the surgeon.

The paper is structured as follows; in Section 2 the FBG-based curvature and shape sensing theory, and mechanical design and fabrication of the sensorised PAF rail is introduced. In Section 3, we show experimental studies evaluating the curvature sensing with the PAF rail applied to phantoms of different curvatures and materials. Different rail materials of various stiffness are also evaluated. We also demonstrate the ability to use the FBG-sensed shape to perform an autonomous US scan of a kidney phantom using the da Vinci Surgical System. Discussion of our findings is given in Section 4, and finally, Section 5 contains conclusion and future work.

2 Materials and methods

2.1 FBG-based shape sensing

FBG sensors are optical sensors that utilise Bragg reflection to measure strain and temperature. FBG sensor fabrication requires altering an optical fibre with a laser and an interferometer to draw a periodic change in the refractive index (Bronnikov et al., 2019), known as a grating. Each grating is wavelength-specific and, only a subset of the light spectra, the Bragg wavelength, is reflected, while the rest is transmitted. We can express this relationship as:

${\lambda }_B=2n_{\mathit{eff}}\mathrm{\Lambda }(1)$

where λ_B is the central wavelength of the reflected spectrum, n_eff is the effective refractive index of the fibre core and Λ is the grating pitch (Zhuang et al., 2018).

The shape-sensor we use is a custom-made multi-core fibre (MCF) (CP-FBG DTG^® (Draw Tower Grating), FBGS International, Jena, Germany), with a central core surrounded by seven equally spaced outer cores. Each core contains 25 FBGs spaced at 10 mm intervals along the optical axis, giving an overall sensing length of 240 mm. Bending of the shape-sensing fibre causes strains in the FBGs, in turn causing shifts in the Bragg wavelengths of the gratings, which are monitored by an optical interrogator. The raw wavelength data are converted to strains for each grating, and the strains of the four gratings at each position along the fibre are then used to compute to a local curvature at that position. The inital data acquisition and processing is performed by proprietary software (IllumiSense v3.1. x, FBGS International, Jena, Germany) and a proprietary LabVIEW VI (National Instruments, Austin, TX, United States), and the curvature data is recorded and visualised through a custom Python application.

2.2 Design and fabrication of the sensorized PAF rail

The design of the PAF rail suction line used in this work is based on the optimisation study firstly presented in Stilli et al. (2019). The structure of the PAF rail-tool interface is an improved version of the design proposed in Wang et al. (2020), where the customised grasping slot is replaced by a continuous segmented fin, enabling grasping of the PAF rail at any point along it. We added a channel of 1 mm diameter along the internal perimeter of the rail to embed the shape sensing fibre, as shown in Figure 1.

FIGURE 1

FIGURE 1. Schematic diagram of the PAF rail with integrated shape sensing fibre. The image on the left illustrates how the system could be attached to the kidney during RAPN. The images on the right illustrate the system design and the integration of the fibre sensing capability within the rail.

The mould for this prototype was designed in SolidWorks (Dassault Systèmes, Vélizy-Villacoublay, France) and 3D printed with an Objet260 Connex (Stratasys, Eden Prairie, MN, United States) in VeroClear resin material. Liquid silicone was degassed in a vacuum chamber for 10 minutes before being injected into the mould and left to cure at room temperature for the required period.

We aimed to identify the silicone material that provided the most accurate shape sensing of the surface on which PAF rail system is deployed while ensuring enough structural rigidity to mechanically pair with the modified US probe presented in Wang et al. (2020). To ensure robust pairing between the probe and the rail profile, a certain level of stiffness is required. We also hypothesized that PAF rails made of silicone significantly stiffer than the tissue targeted would locally deform it, while systems significantly softer would conform better but provide less accurate shape sensing. The goal of this part of the study was to identify the best trade-off between shape sensing accuracy, mechanical pairing and navigation of the US probe. To validate our hypotheses, we selected five silicones from the supplier Smooth-On Inc. (Macungie, PA, US), and we fabricated five PAF rails with different shore hardness: DragonSkin™ 10 NV (Shore hardness 10 A), DragonSkin™ 20 (Shore hardness 20 A), DragonSkin™ 30 (Shore hardness 30 A), Smooth-Sil™ 940 (Shore hardness 40 A) and Smooth-Sil 950™ (Shore hardness 50 A).

The shape-sensing optical fibre is inserted in the rail through fine bore low-density polyethylene (LDPE) tubing (Ø0.86 mm ID, Ø1.52 mm OD). The tubing is fixed at the distal end of the PAF rail by pushing it through the silicone. Then, we secure it with Sil-Poxy™ silicone adhesive (Smooth-On Inc., Macungie, PA, US) at the proximal and distal ends, as depicted in Figure 2.

FIGURE 2

FIGURE 2. PAF rail with integrated Fine Bore LDPE tubing to house the FBGS DTG fibre.

Since the sensing length of the shape-sensing fibre is longer than the rail, only eight groups of four gratings lie within the rail profile, thus only these gratings are used to compute the PAF rail shape. The first grating group lies 3 mm from the rail’s proximal end to guarantee that the curvature sensed by those gratings represents its current shape accurately. The fibre is secured to the tubing with Sil-Poxy™ silicone adhesive at the proximal end.

2.3 Phantom design and fabrication

To test the curvature sensing performance of the sensorized PAF rail, we fabricated several curvature phantoms of different curvature radii R and materials.

For the ground truth experiment, seven concentric circular grooves were laser cut into a sheet of acrylic plastic. The grooves each had a constant radius R, ranging from 30 mm to 110 mm in 20 mm increments. These measurements were chosen to correspond to the range of curvatures of an adult human kidney, as obtained from analysis of the KiTs19 dataset (Heller et al., 2019; Heller et al., 2021). This dataset contained abdominal CT images of 210 adult patients who underwent partial or radical nephrectomy. This will be referred to as the calibration plate for the remainder of this paper. The experimental setup is shown in Figure 3.

FIGURE 3

FIGURE 3. Schematic presenting an overview of the hardware and software elements of the proposed shape sensing system integration in the PAF rail system.

For experiments involving the PAF rail, we 3D printed a rigid curvature block comprised of seven different constant curvature surfaces. The curvatures range from 30 mm to 110 mm with 20 mm increments and, each step has an elevation of 15 mm to accommodate for the width of the rail (Figure 4E).

FIGURE 4

FIGURE 4. Soft curvature phantoms (material, curvature radius): (A) EcoFlex™ 00–20, 110 mm (B) EcoFlex™ 00–20, 30 mm (C) DragonSkin™, 110 mm (D) DragonSkin™30, 30 mm. Rigid curvature phantoms: (E) 110 mm, 90 mm, 70 mm, 50 mm, 30 mm (ascending curvature).

In addition, we fabricated four soft curvature phantoms with R of 30 mm and 110 mm, each in DragonSkin™ 30 (Shore hardness 30 A) and Ecoflex™ 00–20 (Shore hardness 00–20), Smooth-On Inc. (Macungie, PA, US) (Figures 4A–D). These materials are suggested in Adams et al. (2017) and Cheung et al. (2014) for the most realistic development of soft kidney phantoms. The phantom specifications are summarised in Table 1.

TABLE 1

TABLE 1. Phantom specifications.

For the case study, we designed and fabricated an anatomically accurate kidney phantom with realistic mechanical properties, such as shape, volume and stiffness. Based on the literature (Adams et al., 2017; Filippou and Tsoumpas 2018; Maneas et al., 2018; Fohely et al., 2022), we assessed that 3-D printing a negative mould and moulding tissue mimicking material within it would be the preferred method for fabricating a phantom with our desired mechanical and imaging properties. Using 3D Slicer (Pieper et al., 2004), we reconstructed a 3D volume of an adult kidney using the CT images and semantic segmentation labels available from the KiTs19 dataset and made a negative mould in Clear Resin. Tissue-mimicking material (polyvinyl alcohol—PVA) was poured and cast within it. A spherical structure was fabricated and embedded within the phantom to replicate a malignant mass. We followed the same method used to make the kidney phantom but added a freeze-thaw cycle during the process (Mackle et al., 2019). The additional freeze-thaw cycle increased the stiffness of the mass and the vasculature, thereby providing more realistic mechanical properties.

2.4 Ground truth curvature sensing

To assess the accuracy of the shape sensing fibre in combination with the optical system and IllumiSense software used for subsequent experiments, we evaluated the performance of the curvature sensing on a range of curvatures characteristic of a human kidney, as obtained from analysis of the KiTs19 dataset. We set the reference wavelengths by holding the fibre straight and flat against a parallel surface. Then, we positioned the sensing portion in each of the grooves of the calibration plate. The grooves held the fibre in place such that there was no need for external fixation. Wavelength shift and curvature were recorded at each grating index and for each curvature for 30 iterations. Then, the recorded curvatures were averaged over 30 data collected at a 100 Hz rate for noise filtering. We repeated eight times the batch curvature measurements resetting the reference wavelength for each repetition of data collection. Since we experienced random noise during four recordings, we decided to label the defective recordings as outliers and removed them from the presented data.

2.5 Curvature sensing

The fibre was embedded in each of the PAF rail samples as described in Section II.B. We used a 3 CFM single-stage vacuum pump (Bacoeng, Hawthorne, CA) to vacuumize a 12-L vacuum chamber (Bacoeng, Hawthorne, CA) and monitored the pressure thanks to an embedded manometer. The vacuum pressure used for all the tests was P_abs = 7.325kPa, as discussed in Stilli et al. (2019). The chamber was connected with a pressure line to the tested PAF rail sample. Each of the five PAF rails was tested individually and, each test was repeated five times.

The PAF rail sample was suctioned to each of the curved surfaces of the rigid curvature block and soft curvature phantoms. The soft curvature phantoms were clamped in a vice to ensure that the stiffness of each material remained constant for different radii. Curvature data from the eight DTGs present within the rail was recorded for 30 iterations. The experiments were repeated five times.

2.6 Stiffness tests

We conducted stiffness tests on porcine kidney tissue samples, silicone samples from the supplier Smooth-On Inc. and a PVA sample to select the material closest to the stiffness of an adult human kidney and to quantify the relative stiffness differences between the rail and the target organ. Using a surgical scalpel and a template, we excised cylindrical kidney tissue samples from the thickest part of each kidney. We fabricated silicone samples of the same geometry using a mould 3D printed in Clear Resin. The PVA sample was cut out of a PVA block to mitigate shrinkage when moulding. The geometrical properties of each kidney and silicone sample are summarised in Table 4. We conducted the stiffness tests using a UR3e robot arm (Universal Robots, Odense, Denmark), to which we attached a Mini 40 Force Sensor (ATI Industrial Automation, NC, United States) with a custom mount. The mount also included a custom indenter. Then, we planned trajectories and recorded position and force data with ROS.

2.7 Case study

In a robotic-assisted partial nephrectomy, the surgeon must identify the tumour margins before the excision. To do so, the surgeon scans the kidney with a drop-in intraoperative US probe paired with either the EndoWrist^® Prograsp™ or the Large Needle Driver (LDN) of the da Vinci Surgical System (Intuitive Surgical Inc., Sunnyvale, CA, US). It is a challenging task that can only be achieved by a highly-skilled surgeon.

As a proof-of-concept of the fibre integration in the PAF rail for path planning, we automated this US scan. The robotic system employed to conduct this demonstration is the first generation da Vinci robot alongside the da Vinci Research Kit (dVRK) platform (Kazanzides et al., 2014; D’Ettorre et al., 2021). We equipped the first da Vinci Patient Side Manipulator (PSM1) with the EndoWrist^® Prograsp™ Forceps paired with the drop-in US probe BK X12C4 (BK-Medical Holding Inc., Peabody, Massachusetts), as shown in Figure 5.

FIGURE 5

FIGURE 5. Case study experimental setup. (A) FBGS Interrogator and Fan-Out Box: the fan-out is used to connect the multicore shape-sensing fibre to the optical interrogator. (B) FBGS DTG fibre. (C) PVA kidney phantom. (D) DS30 PAF Rail. (E) BK X12C4 Drop-In Ultrasound Probe. (F) EndoWrist Prograsp Forceps, (G) da Vinci surgical robot.

Among the PAF rail systems manufactured in different materials, we selected the one made of DragonSkin™30 for this specific testing because it provides the best trade-off in terms of adhesion performance on hard surfaces and ability to withstand high vacuum pressure without collapsing. Further findings are discussed in section III.B. The case study we are presenting in this section can be summarised with the following steps (See also Figure 5):

1. The fibre is embedded in the DragonSkin™30 rail.

2. Suction is achieved by applying vacuum pressure on the line (as described in III.C) to attach the PAF rail to the kidney phantom.

3. The drop-in US probe is manually paired with the EndoWrist^® Prograsp™ forceps gripper of the da Vinci Surgical System itself mounted on one of the dVRK arms.

4. The PSM1 is manually positioned so that the connector at the tip of the drop-in US probe is paired with the rail profile perpendicularly to the fibre in correspondence of the first grating.

5. We take an instantaneous reading of the 2-D shape of the optical fibre generated by the FBGS proprietary software. This shape is computed out of the spectra of the fibre recorded through the hardware connection with the FBGS Fan Out Box and Interrogator.

6. The 2-D shape is converted into a 3-D point trajectory, assuming no displacement along the z-direction since we paired the probe perpendicular to the PAF rails. And publish to the dVRK computer through ROS.

7. Then, the 3-D trajectory is converted to a PosedStamped message in ROS. Here, we control the frame called “PSM1_psm_base_link”, which corresponds to the tooltip coordinates. First, we extract the current frame. For each position along the trajectory, we compute the novel frame by adding the 3-D coordinates to the current frame.

8. Finally, the poses are published to the dVRK. The US probe moves along the PAF-rail following said trajectory and generate US images of the phantom.

3 Results and discussion

3.1 Ground truth curvature sensing

Figure 6 shows the sensed curvature against geometric curvature for each of the curvature grooves in the calibration plate at each grating. Sensing accuracy increases as the curvature is reduced while variation along the fibre (for each grating) also decreases, giving an overall better curvature measurement at larger radii.

FIGURE 6

FIGURE 6. Ground Truth Sensed Curvature at each grating. Mean sensed curvature (solid line) against groove curvature (dashed line) at each grating index along the fibre. The shaded region represents the standard deviation over five iterations of recordings.

The absolute and % errors to geometric curvature are summarised in Table 2 for the entire length of the fibre (gratings 1 to 25 inclusive) and in Table 3 for a subset of the gratings (gratings 11 to 19 inclusive).

TABLE 2

TABLE 2. Bare fibre sensed curvature errors (gratings 1–25).

TABLE 3

TABLE 3. Bare fibre sensed curvature errors (gratings 11–19).

We further evaluated the curvature sensing accuracy of each grating by calculating the ratio of sensed curvature to geometric curvature, averaged over the curvature range. We expect a value of one for perfect sensing accuracy (shown in Figure 7).

FIGURE 7

FIGURE 7. Curvature sensing accuracy of each grating along the fibre. Accuracy is defined as the ratio of sensed curvature to geometric curvature, for perfect sensing we would expect a value of 1 a each grating.

Overall, the average sensing error (average of all relative errors at all grating indices at all radii) is 1.02 ± 0.03 × 10^–2 mm (2.9%). This is comparable to curvature accuracy reported in Zhuang et al. (2018).

3.2 Curvature sensing

Figure 8 shows the average sensed curvature of each rail material for each radius compared with the ground-truth sensed curvature and geometric curvature when tested with the rigid phantom. The rail sensed curvature accuracy deteriorates as radius is decreased, agreeing with the bare fibre sensing behaviour. At R = 110 mm the best rail is DS30 while the worst is SS950. At R = 90 mm the best rail is SS940 while the worst is SS950. At R = 70 mm the best rail is SS950 while the worst is DS20. At R = 50 mm and R = 30 mm overall sensing accuracy is significantly worse, however SS950 and SS940 are the best performers respectively, while DS20 is the worst at both radii. Furthermore, each datapoint has a large uncertainty arising from the large variances in the raw data.

FIGURE 8

FIGURE 8. Rail-sensed curvature at each radius (decreasing L-R) on the rigid phantom. The materials on the x-axis are in order of increasing material stiffness. Green corresponds to the rail with best sensing accuracy, red to the worst.

In this experiments, we studied the influence of the rail material over curvature sensing performance. The idea behind these experiments was to select a suitable material for the task that provided the best curvature sensing performance.

Thus, to improve the readability of the results presented in Figures 8, 9, we coloured in green the rail expressing the lowest absolute error between measured and ground truth curvature. On the contrary, we highlighted the worst-performing rail in red.

FIGURE 9

FIGURE 9. Rail-sensed curvature at 110 mm (top) and 30 mm (bottom) on the soft phantoms: DragonSkin™(L) and EcoFLex™(R). The materials on the x-axis are in order of increasing material stiffness. Green corresponds to the rail with best sensing accuracy, red to the worst.

Figure 10 shows that there is a trend linking the curvature sensing accuracy to tested curvature. At smaller curvatures, the rails prove to be more accurate than at larger curvatures. It contradicts the results obtained during the ground truth curvature sensing presented in Figure 6. One hypothesis to explain this trend is the effect of the stiffness of the optical fibre exceeding that of the rail. This mismatch in stiffness properties may cause the fibre to bend tangentially to the rail at smaller radii (50 mm and 30 mm), as opposed to concentric to it. Further experiments could ascertain the stiffness of the fibre to compare with the rail, however this is out of the scope of this study.

FIGURE 10

FIGURE 10. Absolute curvature sensing error for each rail material over the radii tested on the rigid phantom. Ground truth curvature sensing error is added as a baseline.

In Figure 8, DS20 and SS950 present the worst sensing accuracy for two or more curvatures and SS940 had the best curvature sensing performance on the rigid phantom. DS30 had the second best performance.

Subsequently, we evaluated the curvature sensing performance on soft phantoms that have similar material properties to that of kidney tissue. Results for the soft curvature phantoms for each rail material are shown in Figure 9. For this experiment, only the extremities of the range of radii were tested. Again, all figures show the error to ground-truth sensed curvature and geometric curvature. Figure 9 shows the results for phantoms made in DragonSkin™ 30 silicone (Left), and Figure 9R shows that for EcoFlex™ 00–20 silicone.

In agreement with both the bare fibre data and the rigid sensing data, we see that sensing accuracy is better R = 110 mm than at R = 30 mm. At R = 110 mm DS30 is the worst rail material for both DragonSkin™ and EcoFlex™ phantoms. In the former phantom material, SS940 is the best rail material while in the latter phantom material, SS950 is the best rail material. At R = 30 mm all rail materials exhibit similar poor performance ($>25$% error to geometric curvature). There is no data for the SS950 rail on the EcoFlex™ phantom as the rail could not stick to this radius for the required period. The softer rails tend to collapse under vacuum pressure. As a result, they tend to express which can explain lower curvature sensing accuracy. Moreover, stiffer rails were harder to stick to the phantom surface. Especially with SS950, we could not maintain the contact at R = 30 with the phantoms for the experiment duration.

Current experiments suggests that SS940 proved better curvature sensing over all the tested curvatures and phantoms as depicted in Figure 10. However, it was hard to stick it to the surface of the rigid curvature block and this material was not selected for the experiments presented in the next section. Thus, since the soft phantoms did not highlighted any significant trend and that DS30 was the second best material on the rigid curvature, we selected it for the next experiment.

3.3 Material stiffness experiments

The experimental stress-strain curves were obtained for each of the tested materials. The Young’s Modulus of the materials considered was obtained by applying a polynomial best-fit line to the data between 7.5% and 15% of maximum compression. We summarised these values in Table 4 alongside the documented Shore Hardness of the silicone materials.

TABLE 4

TABLE 4. Experimental material stiffness properties.

3.4 Case study

We slid the US probe along the rail for 20 mm before it detached from it. However, the contact ensured a stable imaging of the phantom, including the embedded mass simulating a tumour embedded into it, as shown in Figure 11A). Then, the probe disconnected from the phantom and the US was lost. Multiple reasons can explain this trajectory error.

FIGURE 11

FIGURE 11. Case study snapshot. (A) Stream snapshot recorded with the drop-in US probe. (B) Camera top view. (C) Camera lateral view. (D) Computed drop-in US trajectory.

Since the current fibre integration design prevents the probe from sliding along the rail on one side, we attached the probe on the opposite side from the sensing fibre. In this experiment, we assumed that the shape of the kidney was uniform along the width of the rail. However, in reality, this is not the case. Additionally, we planned the trajectory of the US probe in a plane (as shown in Figure 11D). In the 3D space, this can be sufficient control only if we ensure that the longitudinal plane of the rail is parallel to the trajectory plane. However, we manually positioned the probe perpendicular to the rail (as shown in Figures 11B, C), which might induce the trajectory error. With further development, we could better align these planes. For instance, we could use ArUco markers (Romero-Ramirez et al., 2018) to compute the transformation matrix between them and update the probe’s trajectory from the results.

Finally, the rail attachment only helps to slide the probe along the rail (See Figures 11B, C). The actual US element is located at 3 cm from this contact point. In our case, it helps us to obtain an US stream even if the probe attachment detaches from the rail. But, in further studies, we would need to integrate that information into the path planning of the probe.

4 Conclusion

This paper presents an integrated system for curvature sensing of the PAF rails achieved by integrating a multi-core FBG optical fibre into the body of the device. The system uses local strain measurements to sense curvature along the axis of the rails, therefore providing information on the local curvature of the rail without the need for additional sensors. The accuracy of the curvature sensing has been evaluated by comparing the sensed curvature to a range of known curvatures in the range of a human kidney. The system has shown promising results; the bare FBGS fibre can sense these curvatures with mean error ranging between 1.44% and 2.72% over the range of radii (when considering the central subsection of the sensing portion of the fibre).

The accuracy of the curvature sensing when the fibre is embedded in the PAF rail of different materials has also been evaluated. The DS30 rail was identified as the optimal rail material based on sensing accuracy, reliability and material properties. When this material is used, the FBG sensing system can sense curvatures between 30 mm and 110 mm with a mean error ranging between 1.35% at 110 mm and 18.9% at 30 mm. Significant improvements can be made by further evaluating the systematic errors affecting the system when the rail is vacuumised and increasing the iterations of experiments from 5 to 10 in order to reduce the variance.

Furthermore, we have demonstrated the ability to use the sensed curvature to control the trajectory of the da Vinci surgical robot in real-time. Using the da Vinci Research Kit, we autonomously planned and executed the trajectory of the US probe paired with the PAF rail during a kidney phantom scan (single swipe). We inferred the trajectory from the sensed curvatures of the FBGS system. Qualitatively the resultant images compared well with those achieved by manual execution. We propose this system for autonomous or semi-autonomous guidance of the US probe by the da Vinci surgical robot to achieve a more stable scan and improved ultrasound image than manual swiping.

Future work will focus on quantitatively evaluating the effectiveness of the FBGS-sensed curvature to plan the path of the drop-in ultrasound probe on the surface of the kidney, understanding the systematic and random errors present in the curvature sensing performance of the sensorised PAF rails, and their impact in the trajectory executed by the da Vinci surgical robot. We will also look to involve clinicians in quantitative evaluation of the autonomously acquired US images compared with manual acquired ones.

Data availability statement

The raw data supporting the conclusion of this article will be made available by the authors, without undue reservation.

Author contributions

AS and ED conceived the presented idea. AM developed the prototypes and experimental hardware. SD developed the software. AM and SD designed and carried out the experiments, and performed the analysis. AM, SD, and JC contributed to interpretation of results. AM wrote the manuscript with input from SD, JC, ED, LL, AS, and DS. AS and DS supervised the project.

Funding

This research was funded in whole, or in part, by the Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS) [203145/Z/16/Z]; the Engineering and Physical Sciences Research Council (EPSRC) [EP/P012841/1]; and the Royal Academy of Engineering Chair in Emerging Technologies Scheme [CiET 1819/2/36]. For the purpose of open access, the authors have applied a CC BY public copyright licence to any author accepted manuscript version arising from this submission.

Acknowledgments

The authors wish to acknowledge Efthymios Maneas and Prof. Adrien Desjardins for their help and guidance with the fabrication of the kidney phantom introduced in Section 2.3.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Adams, F., Qiu, T., Mark, A., Fritz, B., Kramer, L., Schlager, D., et al. (2017). Soft 3D-printed phantom of the human kidney with collecting system. Ann. Biomed. Eng. 45, 963–972. doi:10.1007/s10439-016-1757-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Autorino, R., Khalifeh, A., Laydner, H., Samarasekera, D., Rizkala, E., Eyraud, R., et al. (2014). Robot-assisted partial nephrectomy (RAPN) for completely endophytic renal masses: A single institution experience. BJU Int. 113, 762–768. doi:10.1111/BJU.12455

PubMed Abstract | CrossRef Full Text | Google Scholar

Bhayani, S. B. (2008). Da Vinci robotic partial nephrectomy for renal cell carcinoma: An atlas of the four-arm technique. J. Robotic Surg. 1, 279–285. doi:10.1007/s11701-007-0055-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Bronnikov, K., Wolf, A., Yakushin, S., Dostovalov, A., Egorova, O., Zhuravlev, S., et al. (2019). Durable shape sensor based on FBG array inscribed in polyimide-coated multicore optical fiber. Opt. Express 27, 38421. doi:10.1364/OE.380816

PubMed Abstract | CrossRef Full Text | Google Scholar

Cheung, C. L., Looi, T., Lendvay, T. S., Drake, J. M., and Farhat, W. A. (2014). Use of 3-dimensional printing technology and silicone modeling in surgical simulation: Development and face validation in pediatric laparoscopic pyeloplasty. J. Surg. Educ. 71, 762–767. doi:10.1016/j.jsurg.2014.03.001

PubMed Abstract | CrossRef Full Text | Google Scholar

D’Ettorre, C., Mariani, A., Stilli, A., Rodriguez Y Baena, F., Valdastri, P., Deguet, A., et al. (2021). Accelerating Surgical Robotics Research: A Review of 10 Years with the da Vinci Research Kit. IEEE Robotics Automation Mag. 28, 56–78. doi:10.1109/MRA.2021.3101646

CrossRef Full Text | Google Scholar

Ettorre, C. D., Stilli, A., Dwyer, G., Neves, J. B., Tran, M., and Stoyanov, D. (2019). “Semi-autonomous interventional manipulation using pneumatically attachable flexible rails,” in IEEE International Conference on Intelligent Robots and Systems, Macau, China, 03-08 November 2019. doi:10.1109/IROS40897.2019.8967789

CrossRef Full Text | Google Scholar

Ferguson, J. M., Cai, L. Y., Reed, A., Siebold, M., De, S., Herrell, S. D., et al. (2018). “Toward image-guided partial nephrectomy with the da Vinci robot: Exploring surface acquisition methods for intraoperative re-registration,” in Conference: Image-Guided Procedures, Robotic Interventions, and Modeling, San Diego, California, United States, 19 - 23 February 2023. doi:10.1117/12.2296464

CrossRef Full Text | Google Scholar

Filippou, V., and Tsoumpas, C. (2018). Recent advances on the development of phantoms using 3D printing for imaging with CT, MRI, PET, SPECT, and ultrasound. Med. Phys. 45, e740–e760. doi:10.1002/MP.13058

PubMed Abstract | CrossRef Full Text | Google Scholar

Fohely, F., Oglat, A., Sabarna, K., Shweiki, Z., Hamoudeh, B., and Shalaan, R. (2022). Fabrication of low-cost realistic three-dimensional static kidney phantom for ultrasound-guided biopsy applications. J. Med. Ultrasound 30, 36. doi:10.4103/JMU.JMU_181_20

PubMed Abstract | CrossRef Full Text | Google Scholar

Hakime, A., Deschamps, F., De Carvalho, E. G. M., Barah, A., Auperin, A., and De Baere, T. (2012). Electromagnetic-tracked biopsy under ultrasound guidance: Preliminary results. Cardiovasc. Interventional Radiology 35, 898–905. doi:10.1007/s00270-011-0278-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Hekman, M. C., Rijpkema, M., Langenhuijsen, J. F., Boerman, O. C., Oosterwijk, E., and Mulders, P. F. (2018). Intraoperative imaging techniques to support complete tumor resection in partial nephrectomy. Eur. Urol. Focus 4, 960–968. doi:10.1016/j.euf.2017.04.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Heller, N., Isensee, F., Maier-Hein, K. H., Hou, X., Xie, C., Li, F., et al. (2021). The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 challenge. Med. Image Anal. 67, 101821. doi:10.1016/j.media.2020.101821

PubMed Abstract | CrossRef Full Text | Google Scholar

Heller, N., Sathianathen, N., Kalapara, A., Walczak, E., Moore, K., Kaluzniak, H., et al. (2019). The KiTS19 challenge data: 300 kidney tumor cases with clinical context, CT semantic segmentations, and surgical outcomes. Available at: https://arxiv.org/abs/1904.00445v2.

Google Scholar

Kaczmarek, B. F., Sukumar, S., Kumar, R. K., Desa, N., Jost, K., Diaz, M., et al. (2013a). Comparison of robotic and laparoscopic ultrasound probes for robotic partial nephrectomy. J. Endourology 27, 1137–1140. doi:10.1089/end.2012.0528

PubMed Abstract | CrossRef Full Text | Google Scholar

Kaczmarek, B. F., Sukumar, S., Petros, F., Trinh, Q. D., Mander, N., Chen, R., et al. (2013b). Robotic ultrasound probe for tumor identification in robotic partial nephrectomy: Initial series and outcomes. Int. J. Urology 20, 172–176. doi:10.1111/J.1442-2042.2012.03127.X

PubMed Abstract | CrossRef Full Text | Google Scholar

Kaul, S., Laungani, R., Sarle, R., Stricker, H., Peabody, J., Littleton, R., et al. (2007). Da vinci-assisted robotic partial nephrectomy: Technique and results at a mean of 15 Months of follow-up. Eur. Urol. 51, 186–192. doi:10.1016/j.eururo.2006.06.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Kazanzides, P., Chen, Z., Deguet, A., Fischer, G. S., Taylor, R. H., and Dimaio, S. P. (2014). An open-source research kit for the da Vinci® Surgical System. Proc. - IEEE Int. Conf. Robotics Automation 6434–6439. doi:10.1109/ICRA.2014.6907809

CrossRef Full Text | Google Scholar

Larcher, A., Muttin, F., Peyronnet, B., De Naeyer, G., Khene, Z. E., Dell’Oglio, P., et al. (2019). The learning curve for robot-assisted partial nephrectomy: Impact of surgical experience on perioperative outcomes. Eur. Urol. 75, 253–256. doi:10.1016/J.EURURO.2018.08.042

PubMed Abstract | CrossRef Full Text | Google Scholar

Lugez, E., Sadjadi, H., Joshi, C. P., Akl, S. G., and Fichtinger, G. (2017). Improved electromagnetic tracking for catheter path reconstruction with application in high-dose-rate brachytherapy. Int. J. Comput. Assisted Radiology Surg. 12, 681–689. doi:10.1007/s11548-017-1534-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Lun, T. L. T., Wang, K., Ho, J. D., Lee, K. H., Sze, K. Y., and Kwok, K. W. (2019). Real-time surface shape sensing for soft and flexible structures using fiber Bragg gratings. IEEE Robotics Automation Lett. 4, 1454–1461. doi:10.1109/LRA.2019.2893036

CrossRef Full Text | Google Scholar

Luo, X., Wan, Y., and He, X. (2015). Robust electromagnetically guided endoscopic procedure using enhanced particle swarm optimization for multimodal information fusion. Med. Phys. 42, 1808–1817. doi:10.1118/1.4915285

PubMed Abstract | CrossRef Full Text | Google Scholar

Mackle, E. C., Maneas, E., Little, C., Carr, E., Xia, W., Nikitichev, D., et al. (2019). “Wall-less vascular poly(vinyl) alcohol gel ultrasound imaging phantoms using 3D printed vessels,” in Progress in biomedical optics and imaging - proceedings of SPIE. Editors R. Liang, J. Hwang, T. J. Pfefer, and G. Vargas (San Francisco, CA: SPIE). doi:10.1117/12.2510033

CrossRef Full Text | Google Scholar

Maneas, E., Xia, W., Nikitichev, D. I., Daher, B., Manimaran, M., Wong, R. Y. J., et al. (2018). Anatomically realistic ultrasound phantoms using gel wax with 3D printed moulds. Phys. Med. Biol. 63, 015033. doi:10.1088/1361-6560/AA9E2C

PubMed Abstract | CrossRef Full Text | Google Scholar

Mishra, V., Singh, N., Tiwari, U., and Kapur, P. (2011). Fiber grating sensors in medicine: Current and emerging applications. Sensors Actuators, A Phys. 167, 279–290. doi:10.1016/j.sna.2011.02.045

CrossRef Full Text | Google Scholar

Pieper, S., Halle, M., and Kikinis, R. (2004). 3D slicer; 3D slicer. 2004 2nd IEEE Int. Symposium Biomed. Imaging Macro Nano (IEEE Cat No. 04EX821) 2, 632–635. doi:10.1109/ISBI.2004.1398617

CrossRef Full Text | Google Scholar

Polygerinos, P., Zbyszewski, D., Schaeffter, T., Razavi, R., Seneviratne, L. D., and Althoefer, K. (2010). MRI-compatible fiber-optic force sensors for catheterization procedures. IEEE Sensors J. 10, 1598–1608. doi:10.1109/JSEN.2010.2043732

CrossRef Full Text | Google Scholar

Romero-Ramirez, F. J., Muñoz-Salinas, R., and Medina-Carnicer, R. (2018). Speeded up detection of squared fiducial markers. Image Vis. Comput. 76, 38–47. doi:10.1016/J.IMAVIS.2018.05.004

CrossRef Full Text | Google Scholar

Runciman, M., Darzi, A., and Mylonas, G. P. (2019). Soft robotics in minimally invasive surgery. Soft Robot. 6, 423–443. doi:10.1089/soro.2018.0136

PubMed Abstract | CrossRef Full Text | Google Scholar

Sareh, S., Noh, Y., Li, M., Ranzani, T., Liu, H., and Althoefer, K. (2015). Macrobend optical sensing for pose measurement in soft robot arms. Smart Mater. Struct. 24, 125024. doi:10.1088/0964-1726/24/12/125024

CrossRef Full Text | Google Scholar

Schwein, A., Kramer, B., Chinnadurai, P., Virmani, N., Walker, S., O’Malley, M., et al. (2018). Electromagnetic tracking of flexible robotic catheters enables “assisted navigation” and brings automation to endovascular navigation in an in vitro study. J. Vasc. Surg. 67, 1274–1281. doi:10.1016/j.jvs.2017.01.072

PubMed Abstract | CrossRef Full Text | Google Scholar

Shingleton, W. B., and Sewell, P. E. (2001). Percutaneous renal tumor cryoablation with magnetic resonance imaging guidance. J. Urology 165, 773–776. doi:10.1016/S0022-5347(05)66523-8

CrossRef Full Text | Google Scholar

Silvestri, S., and Sche, E. (2011). “Optical-Fiber measurement systems for medical applications,” in Optoelectronics - devices and applications (Calicut, India: National Institute of Technology, Calicut). doi:10.5772/18845

CrossRef Full Text | Google Scholar

Stefanidis, D., Wang, F., Korndorffer, J. R., Dunne, J. B., and Scott, D. J. (2010). Robotic assistance improves intracorporeal suturing performance and safety in the operating room while decreasing operator workload. Surg. Endosc. 24, 377–382. doi:10.1007/s00464-009-0578-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Stilli, A., Dimitrakakis, E., D’ettorre, C., Tran, M., and Stoyanov, D. (2019). Pneumatically attachable flexible rails for track-guided ultrasound scanning in robotic-assisted partial nephrectomy - a preliminary design study. IEEE Robotics Automation Lett. 4, 1208–1215. doi:10.1109/LRA.2019.2894499

CrossRef Full Text | Google Scholar

Su, L. M., Vagvolgyi, B. P., Agarwal, R., Reiley, C. E., Taylor, R. H., and Hager, G. D. (2009). Augmented reality during robot-assisted laparoscopic partial nephrectomy: Toward real-time 3D-CT to stereoscopic video registration. Urology 73, 896–900. doi:10.1016/j.urology.2008.11.040

PubMed Abstract | CrossRef Full Text | Google Scholar

Van Der Schatte Olivier, R. H., Van’t Hullenaar, C. D., Ruurda, J. P., and Broeders, I. A. (2009). Ergonomics, user comfort, and performance in standard and robot-assisted laparoscopic surgery. Surg. Endosc. 23, 1365–1371. doi:10.1007/s00464-008-0184-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, C., Komninos, C., Andersen, S., D’Ettorre, C., Dwyer, G., Maneas, E., et al. (2020). Ultrasound 3D reconstruction of malignant masses in robotic-assisted partial nephrectomy using the PAF rail system: A comparison study. Int. J. Comput. Assisted Radiology Surg. 15, 1147–1155. doi:10.1007/s11548-020-02149-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Yakoubi, R., Autorino, R., Laydner, H., Guillotreau, J., White, M. A., Hillyer, S., et al. (2012). Initial laboratory experience with a novel ultrasound probe for standard and single-port robotic kidney surgery: Increasing console surgeon autonomy and minimizing instrument clashing. Int. J. Med. Robotics Comput. Assisted Surg. 8, 201–205. doi:10.1002/RCS.452

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhuang, W., Sun, G., Li, H., Lou, X., Dong, M., and Zhu, L. (2018). FBG based shape sensing of a silicone octopus tentacle model for soft robotics. Optik 165, 7–15. doi:10.1016/j.ijleo.2018.03.087

CrossRef Full Text | Google Scholar