A novel 3D deep learning model to automatically demonstrate renal artery segmentation and its validation in nephron-sparing surgery

Abstract

Purpose:

Nephron-sparing surgery (NSS) is a mainstream treatment for localized renal tumors. Segmental renal artery clamping (SRAC) is commonly used in NSS. Automatic and precise segmentations of renal artery trees are required to improve the workflow of SRAC in NSS. In this study, we developed a tridimensional kidney perfusion (TKP) model based on deep learning technique to automatically demonstrate renal artery segmentation, and verified the precision and feasibility during laparoscopic partial nephrectomy (PN).

Methods:

The TKP model was established based on convolutional neural network (CNN), and the precision was validated in porcine models. From April 2018 to January 2020, TKP model was applied in laparoscopic PN in 131 patients with T1a tumors. Demographics, perioperative variables, and data from the TKP models were assessed. Indocyanine green (ICG) with near-infrared fluorescence (NIRF) imaging was applied after clamping and dice coefficient was used to evaluate the precision of the model.

Results:

The precision of the TKP model was validated in porcine models with the mean dice coefficient of 0.82. Laparoscopic PN was successfully performed in all cases with segmental renal artery clamping (SRAC) under TKP model’s guidance. The mean operation time was 100.8 min; the median estimated blood loss was 110 ml. The ischemic regions recorded in NIRF imaging were highly consistent with the perfusion regions in the TKP models (mean dice coefficient = 0.81). Multivariate analysis revealed that the feeding lobar artery number was strongly correlated with tumor size and contact surface area; the supplying segmental arteries number correlated with tumor size.

Conclusions:

Using the CNN technique, the TKP model is developed to automatically present the renal artery trees and precisely delineate the perfusion regions of different segmental arteries. The guidance of the TKP model is feasible and effective in nephron-sparing surgery.

Introduction

As a minimally invasive nephron-sparing surgery, laparoscopic partial nephrectomy (laparoscopic PN) is a mainstream treatment for cT1a renal tumors (1). In laparoscopic PN, renal artery clamping induces warm ischemic injury (WII) (2), which could be minimized by segmental renal artery clamping (SRAC) technique, converting global parenchymal ischemia to regional ischemia (3–6). To implement the SRAC technique, dual-source computed tomography (DSCT) angiography was applied to reveal a high-quality three-dimensional vasculature model of the renal hilum, and identify the target segmental arteries needed to be clamped if their branches enter or abut the tumor (6, 7). However, DSCT angiography is unable to provide the details of the perfusion regions of different segmental arteries, and the determination of the target arteries is inaccurate occasionally, which could lead to insufficient clamping and result in arterial bleeding (5). Therefore, a more precise clamping strategy is required.

Based on the contrast CT scan, organ segmentation with different kinds of statistical models were reported in several abdominal organs over the years (8–10). Previously, in our center, to meet the requirement of a more precise SRAC technique, a novel functional three-dimensional perfusion model was established to determine the target arteries by depicting the parenchymal perfusion regions of different segmental arteries using the semi-automatic segmentation of the kidney and renal arteries (11). Recently, with the development of medical image processing technology, convolutional neural network (CNN) as a kind of deep learning technique has gradually begun to be applied in the segmentation of organs and vasculature (12). Based on the CNN, we previously created a series of novel methods, which could provide a fully automatic segmentation of kidney, tumor, and renal artery trees (13, 14). In present study, integrating CNN technique and the distance transformation algorithm, a novel three-dimensional perfusion model was established, which was called the tridimensional kidney perfusion (TKP) model.

Materials and methods

The establishment procedure of the TKP model was presented. The accuracy of the TKP model was verified in porcine models, and the feasibility and efficacy of this model were assessed in patients undergoing laparoscopic PN with SRAC.

Establishment of the TKP model

Automatic segmentation of the kidneys and tumors

Our homemade three-dimensional fully-supervised convolutional neural (FCN) network with a pyramid-pooling module (PPM) (3D_FCN_PPM) was proposed for segmentation of kidneys and tumors previously (13) (Figure 1). During the establishment of the 3D_FCN_PPM network, the abdominal CT images of 140 patients were recruited from the department of radiology after the informed consent was obtained. And the images were obtained and analyzed in Dicom format. Ninety images were used for the training set, and the remaining 50 images were used for testing. The 3D_FCN_PPM network was demonstrated to be efficient and precise in segmentation with the dice coefficient equal to 0.931 for kidney and 0.802 for renal tumors.

Figure 1

Automatic segmentation of the renal artery trees

Integrating the technologies of deep-priori anatomy (DPA), dense-biased network (DenseBiasNet), and hard-region adaptation loss (HRA loss), we proposed a fine three-dimensional renal artery segmentation framework, called DPA-DenseBiasNet framework (14). The DPA-DenseBiasNet framework was based on a two-stage CNN, including (1) autoencoder (AE) network pre-training and (2) DPA features embedding and DenseBiasNet training. AE is an unsupervised neural network, which can extract anatomical features (15). In this framework, AE is applied to acquire the representation ability of anatomical features (DPA features) through a big unlabeled dataset. In stage 2, extracted DPA features from AE are embedded in the DenseBiasNet system, forming the priori anatomy information, which can adapt anatomical variations. Finally, modified by HRA loss function, a precise tridimensional renal artery segmentation is achieved (Figure 2). During the establishment of the DPA-DenseBiasNet framework, a total of 196 patients with 392 kidney images were recruited. Fifty-two labeled images and 236 unlabeled images were used for training, and 104 labeled images were used for testing. The DPA-DenseBiasNet was demonstrated to have high predictive accuracy in renal artery segmentation with a mean dice coefficient of 0.884.

Figure 2

Estimation of the arterial perfusion regions on the renal parenchyma

After automatic segmentation, the estimation procedure based on the two-step algorithm in C++ programming: (1) set the lobar arteries and their branches to the same category and marked with the same color if they branch out from the same segmental artery (Figure 3A, the arteries with the same color are the same segmental artery subtree); (2) the distance transformation algorithm is used to find the closest lobar arteries or their branches for every voxel point in the renal parenchyma as its blood supply vessel, and the color of this point is marked. All voxel points in the renal parenchyma are categorized according to their colors, and the perfusion regions of different segmental arteries are then depicted (Figure 3B).

Figure 3

By the automatic segmentation and the perfusion region estimation algorithm, the TKP model is finally established.

Validation in animal models

Subjects preparation

The validation procedure in swine was approved by the Animal Use and Management Ethics Committee of Nanjing Medical University. Six domestic female pigs with 11 kidneys were recruited and the median weight was 39.5 kg (Table 1). All swine were intramuscularly injected with xylazine (1.5 mg/kg), atropine (0.02 mg/kg) and diazepam (10 mg) for initial anesthesia and intravenously injected with propofol (25 μg/kg/min) for maintaining anesthesia. Tracheal intubation was conducted and the right femoral artery was punctured.

Validation procedure

All subjects underwent contrast CT scan to establish the TKP models (Figures 4A–D). One candidate segmental artery in each kidney was selected and ligated with a double-strand 1/0 suture during open surgery. The ischemia region was revealed and recorded (Figures 4E, F). To evaluate the accuracy of the TKP model, a second contrast CT scan was performed after open surgery to present the actual ischemia region (Figure 4H). Using the method of dice coefficient, the actual ischemia region from the second contrast CT scan was compared with the perfusion region predicted by the TKP model (Figure 4G).

Figure 4

Clinical application in laparoscopic PN

Patients preparation

Laparoscopic PN with SRAC under the TKP model’s guidance was performed in 131 patients from April 2018 to January 2020. All the recruited patients signed a written informed consent form approved by the institutional review board of Nanjing Medical University. Inclusion criteria were: 1) a single localized mass ≤4 cm (clinical T1a); 2) normal renal function (defined as creatinine clearance rate (CCR) ≥90 ml/min). All patients underwent a contrast CT scan to establish the TKP model before operation. The R.E.N.A.L scores were applied to estimate the complexity of tumors, including Radius (maximum tumor diameter), Exophytic/Endophytic, Nearness of the tumor to the collecting system, Anterior/Posterior and Location relative to the polar lines (16, 17). The contact surface area (CSA) of tumors, as another index predicting the tumor complexity (18), could be calculated by area element algorithm according to the model. Additionally, the numbers of target segmental arteries and feeding lobar arteries of tumors could be provided from the model.

Precise determination of the target segmental arteries

In the TKP models, the tumors, segmental renal arteries and their corresponding perfusion regions were automatically presented. The target segmental arteries supplying tumors were determined by the perfusion regions wherein the renal tumors were confined (Figures 5A, G).

Figure 5

Surgical procedure and follow-up

All surgical procedure were conducted by the same surgeon (Pengfei Shao). During laparoscopic PN, target segmental arteries determined by TKP models were clamped by bulldog clamps. Immediately after clamping, 5 mg indocyanine green (ICG) was intravenously injected and the absence of perfusion on renal parenchyma was presented in near-infrared fluorescence (NIRF) imaging (Figures 5D, J). The similarity was evaluated between the absence of perfusion in NIRF imaging and the predicted perfusion region in TKP model by the method of dice coefficient. Tumor resection was conducted and the parenchymal defect was closed. Finally, the mass was retrieved to receive a pathological examination.

The follow-up period was defined as the duration from the date of operation to the date of the most recent examination. For follow-up and surveillance, abdominal plain CT scan was performed at 3 and 6 months and every 6 months. Chest CT scan and abdominal contrast CT scans were performed every 6 and 12 months, respectively.

Statistical analysis

Categorical variables were presented as frequencies and percentages. Continuous variables were reported as mean ± SD (normal distribution) or medians and ranges (abnormal distribution). Complications were analyzed according to the Clavien–Dindo system (19). Logistic regression analysis was used to test the correlation of tumor characteristics and the number of arteries supplying the tumor. All statistical analyses were conducted using IBM SPSS v.22 (SPSS Inc., Chicago, IL, USA), and two-sided p < 0.05 was considered to be statistically significant. Dice coefficient was applied to evaluate the similarity of regions or volumes in two images, and high similarity was defined as dice coefficient > 0.7.

Results

Of 11 porcine kidneys, the median number of segmental arteries was 3. After clamping, the ischemic regions were located on the upper, middle, and lower polar in three, one, and seven kidneys, respectively (Table 1). As shown in Figure 4, the boundaries of the perfusion regions in the TKP models were consistent with the ischemic lines recorded intraoperatively. It was demonstrated to have high similarity between the actual ischemia region from the post-operative CT scan and the perfusion region from the TKP model (dice coefficient = 0.82) (Table 1).

In clinical procedure, basic characteristics are shown in Table 2. There were 97 males and 34 females, aged 56.3 ± 11.4 years, with a mean body mass index of 25.1 kg/m². The mean tumor size (radius of the tumor as maximal diameter) was 2.5 cm, and the R.E.N.A.L score was 6.4 ± 1.4. According to the TKP model, the CSA was 13.5 ± 11.3 cm². There were 79, 49, and 3 patients with tumors supplied by one, two, and three target segmental arteries, respectively. Furthermore, subclassified by the numbers of feeding lobar arteries, there were 21, 56, 43, 10, and 1 patients with tumors supplied by one, two, three, four and five lobar arteries, respectively.

Laparoscopic PN with precise SRAC was successfully performed under the TKP models’ guidance in all patients. The mean operation time was 100.8 min, with a mean warm ischemic time (WIT) of 27.0 min. The median estimated blood loss (EBL) was 110 ml (40 - 400 ml). There were no patients converting to main renal artery clamping, radical nephrectomy, or open surgery. No arterial bleeding or uncontrolled hemorrhage from the tumor bed occurred during tumor resection (Table 3). The ischemic regions recorded by ICG with NIRF imaging were highly consistent with the perfusion regions in the TKP models, with the dice coefficient of 0.81 (0.72–0.94) (Figure 5 and Table 3). The median length of stay after the operation was 7 days. Postoperative complications occurred in eight (6.1%) patients, including five patients with grade 1 complication (hematuria not requiring intervention), two patients with grade 2 complication (hematuria requiring blood transfusion), and one patient with grade 3a complication (bleeding requiring embolization intervention under local anesthesia). Pathology and follow-up results were also revealed in Table 3. At a median follow-up of 21 months, no patient had tumor recurrence or metastasis.

According to the number of tumor supplying arteries, subjects were divided into subgroups with one and two or more supplying segmental arteries or subgroups with 1–2 and 3–5 supplying lobar arteries. The comparisons of tumor characteristics between these subgroups are conducted in Table 4. Furthermore, logistic regression analysis is applied and presented in Table 5. On multivariate analysis, the numbers of both supplying segmental and lobar arteries strongly correlated with tumor size (OR = 5.92, p = 0.000 for segmental arteries and OR = 4.84, p = 0.002 for lobar arteries). The larger the tumor size, the more the number of supplying segmental and lobar arteries. The CSA of the tumor had correlation with the number of supplying lobar arteries (OR = 1.11, p = 0.014), instead of segmental arteries (p = 0.815). None of the correlations was found between the other tumor characteristics and the number of supplying arteries.

Discussion

Traditionally, DSCT angiography was utilized to reveal the three-dimensional hilar vasculature during SRAC procedure (5). In DSCT angiography, the target arteries were determined manually, leading to the underestimation of artery branches feeding both tumor and the surrounding normal tissue (7). So measurement bias, insufficient clamping, arterial bleeding or even converting to the main artery clamping during resection might occur. For a more efficient and precise SRAC technique, the TKP model was established using a homemade CNN technology, becoming an automatic tool in the surgical strategy-making of the SRAC during nephron-sparing surgery (14). ICG with NIRF imaging, as an empirical technique of fluorescence guidance (20), was introduced in this study to delineate the real ischemic area after clamping. Our results confirmed that the perfusion regions predicted in the TKP model were highly consistent with the real ischemic area in NIRF imaging during operation. In this study, under the guidance of the TKP models, all surgical procedures were performed successfully, and there was no occurrence of uncontrolled bleeding during tumor resection.

Recently, by constructing various models, researchers have been exploring the navigation technique in PN with SRAC. Ukimura et al. established a 3D model that could present opaque tumors and renal arterial trees by making the renal parenchyma semitransparent (21). By manually segmenting kidney shape, vasculature, collecting system, and tumor, Porpiglia proposed a hyperaccuracy 3D model (22), which used an augmented reality (AR) technique to guide surgeons during operation (23). Additionally, the goal of purely automatic segmentation of different organs and even renal artery trees using CNNs was achieved (24–26). We had developed a series of novel CNNs, including 3D_FCN_PPM and DPA-DenseBiasNet, providing a precise segmentation of kidney, tumors, renal arteries and their branches (distal to interlobar arteries) (13, 14). These homemade CNNs, along with the distance transformation algorithm, made the establishment of the TKP model fully automated. And the automatic procedure could reduce the manual workload of delineation in the radiological process and significantly improve the efficiency of the preoperative plan of laparoscopic PN with a precise SRAC technique.

Using CNN techniques, automatic segmentation could be accurate to distal interlobar arteries, facilitating a more precise estimation of the arterial perfusion regions. In the future, in combination with the AR technique, it is expected that the TKP model could be implanted into the surgery console and become visual and synchronous. It is beneficial for improving the accuracy and efficacy of SRAC technique and tumor resection.

Leslie and his colleagues presented the CSA as a novel parameter to predict the complexity of renal tumors (18). In our study, tumor size and CSA strongly correlated with the number of feeding arteries. Larger CSA was accompanied by more feeding lobar arteries, instead of the target segmental arteries. In the future, the number of lobar arteries feeding the tumor is expected to predict renal tumor complexity and become an indicator in the scoring system to evaluate the difficulty of PN.

This study is not devoid of limitations. Firstly, we still lack a multi-center research. Secondly, we lack a randomized controlled study to compare the TKP model and the traditional DSCT angiography model since the former is a newly developed technology.

Notwithstanding these limitations, the TKP model was more than satisfactory because of the hyperaccuracy verified during operation. It is expected to become a comprehensive tool with multiple functions, such as preoperative assessment of tumor complexity, automatic planning of surgical strategy and real-time navigation of selective clamping and tumor resection.

Conclusions

Using the CNN technique, the TKP model is developed to automatically present the renal artery trees and precisely delineate the perfusion regions of different segmental arteries. The guidance of the TKP model is feasible and effective in nephron-sparing surgery.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

• 1

  Van PoppelHBeckerFCadedduJAGillISJanetschekGJewettMAet al. Treatment of localised renal cell carcinoma. Eur Urol (2011) 60(4):662–72. doi: 10.1016/j.eururo.2011.06.040

  • CrossRef
  • Google Scholar

• 2

  PorpigliaFFioriCBertoloRMorraIRussoRPiccoliGet al. Long-term functional evaluation of the treated kidney in a prospective series of patients who underwent laparoscopic partial nephrectomy for small renal tumors. Eur Urol (2012) 62(1):130–5. doi: 10.1016/j.eururo.2012.02.001

  • CrossRef
  • Google Scholar

• 3

  LiPQinCCaoQLiJLvQMengXet al. A retrospective analysis of laparoscopic partial nephrectomy with segmental renal artery clamping and factors that predict postoperative renal function. BJU Int (2016) 118(4):610–7. doi: 10.1111/bju.13541

  • CrossRef
  • Google Scholar

• 4

  ShaoPQinCYinCMengXJuXLiJet al. Laparoscopic partial nephrectomy with segmental renal artery clamping: technique and clinical outcomes. Eur Urol (2011) 59(5):849–55. doi: 10.1016/j.eururo.2010.11.037

  • CrossRef
  • Google Scholar

• 5

  ShaoPTangLLiPXuYQinCCaoQet al. Precise segmental renal artery clamping under the guidance of dual-source computed tomography angiography during laparoscopic partial nephrectomy. Eur Urol (2012) 62(6):1001–8. doi: 10.1016/j.eururo.2012.05.056

  • CrossRef
  • Google Scholar

• 6

  ShaoPTangLLiPXuYQinCCaoQet al. Application of a vasculature model and standardization of the renal hilar approach in laparoscopic partial nephrectomy for precise segmental artery clamping. Eur Urol (2013) 63(6):1072–81. doi: 10.1016/j.eururo.2012.10.017

  • CrossRef
  • Google Scholar

• 7

  XuYShaoPZhuXLvQLiuWXuHet al. Three-dimensional renal CT angiography for guiding segmental renal artery clamping during laparoscopic partial nephrectomy. Clin Radiol (2013) 68(11):e609–16. doi: 10.1016/j.crad.2013.06.002

  • CrossRef
  • Google Scholar

• 8

  OkadaTLinguraruMGHoriMSummersRMTomiyamaNSatoY. Abdominal multi-organ segmentation from CT images using conditional shape-location and unsupervised intensity priors. Med Image Anal (2015) 26(1):1–18. doi: 10.1016/j.media.2015.06.009

  • CrossRef
  • Google Scholar

• 9

  FreimanMKronmanAEssesSJJoskowiczLSosnaJ. Non-parametric iterative model constraint graph min-cut for automatic kidney segmentation. Med Image Comput Comput Assist Interv (2010) 13(Pt 3):73–80. doi: 10.1007/978-3-642-15711-0_10

  • CrossRef
  • Google Scholar

• 10

  HellerNIsenseeFMaier-HeinKHHouXXieCLiFet al. The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 challenge. Med Image Anal (2021) 67:101821. doi: 10.1016/j.media.2020.101821

  • CrossRef
  • Google Scholar

• 11

  ZhangSYangGTangLLvQLiJXuYet al. Application of a Functional3-dimensional perfusion model in laparoscopic partial nephrectomy with precise segmental renal artery clamping. Urology (2019) 125:98–103. doi: 10.1016/j.urology.2018.12.023

  • CrossRef
  • Google Scholar

• 12

  LiskowskiPKrawiecK. Segmenting retinal blood vessels with deep neural networks. IEEE Trans Med Imaging (2016) 35(11):2369–80. doi: 10.1109/TMI.2016.2546227

  • CrossRef
  • Google Scholar

• 13

  YangGLiGPanTKongYWuJShuHet al. (2018). Automatic segmentation of kidney and renal tumor in CT images based on 3D fully convolutional neural network with pyramid pooling module, in: 2018 24th International Conference on Pattern Recognition (ICPR).

  • Google Scholar

• 14

  HeYYangGYangJChenYKongYWuJet al. Dense biased networks with deep priori anatomy and hard region adaptation: Semi-supervised learning for fine renal artery segmentation. Med Image Anal (2020) 63:101722. doi: 10.1016/j.media.2020.101722

  • CrossRef
  • Google Scholar

• 15

  Jonathan MasciUMCire¸sanDSchmidhuberJ¨urgen. (2011). Stacked convolutional auto-encoders for hierarchical feature extraction, in: International Conference on Artificial Neural Networks, . pp. 52–9, Part I(LNCS 6791).

  • Google Scholar

• 16

  KutikovAUzzoRG. The R.E.N.A.L. nephrometry score: a comprehensive standardized system for quantitating renal tumor size, location and depth. J Urol (2009) 182(3):844–53. doi: 10.1016/j.juro.2009.05.035

  • CrossRef
  • Google Scholar

• 17

  BorgmannHReissAKKuroschMFilmannNFreesSMagerRet al. R.E.N.A.L. score outperforms PADUA score, c-index and DAP score for outcome prediction of nephron sparing surgery in a selected cohort. J Urol (2016) 196(3):664–71. doi: 10.1016/j.juro.2016.03.176

  • CrossRef
  • Google Scholar

• 18

  LeslieSGillISde Castro AbreuALRahmanuddinSGillKSNguyenMet al. Renal tumor contact surface area: a novel parameter for predicting complexity and outcomes of partial nephrectomy. Eur Urol (2014) 66(5):884–93. doi: 10.1016/j.eururo.2014.03.010

  • CrossRef
  • Google Scholar

• 19

  DindoDDemartinesNClavienPA. Classification of surgical complications: a new proposal with evaluation in a cohort of 6336 patients and results of a survey. Ann Surg (2004) 240(2):205–13. doi: 10.1097/01.sla.0000133083.54934.ae

  • CrossRef
  • Google Scholar

• 20

  KraneLSHemalAK. Emerging technologies to improve techniques and outcomes of robotic partial nephrectomy: Striving toward the pentafecta. Urol Clin North Am (2014) 41(4):511–9. doi: 10.1016/j.ucl.2014.07.006

  • CrossRef
  • Google Scholar

• 21

  UkimuraOGillIS. Imaging-assisted endoscopic surgery: Cleveland clinic experience. J Endourol (2008) 22(4):803–10. doi: 10.1089/end.2007.9823

  • CrossRef
  • Google Scholar

• 22

  PorpigliaFFioriCCheccucciEAmparoreDBertoloR. Hyperaccuracy three-dimensional reconstruction is able to maximize the efficacy of selective clamping during robot-assisted partial nephrectomy for complex renal masses. Eur Urol (2018) 74(5):651–60. doi: 10.1016/j.eururo.2017.12.027

  • CrossRef
  • Google Scholar

• 23

  PorpigliaFCheccucciEAmparoreDPiramideFVolpiGGranatoSet al. Three-dimensional augmented reality robot-assisted partial nephrectomy in case of complex tumours (PADUA >/=10): A new intraoperative tool overcoming the ultrasound guidance. Eur Urol (2020) 78(2):229–38. doi: 10.1016/j.eururo.2019.11.024

  • CrossRef
  • Google Scholar

• 24

  WangCRothHRKitasakaTOdaMHayashiYYoshinoYet al. Precise estimation of renal vascular dominant regions using spatially aware fully convolutional networks, tensor-cut and voronoi diagrams. Comput Med Imaging Graph (2019) 77:101642. doi: 10.1016/j.compmedimag.2019.101642

  • CrossRef
  • Google Scholar

• 25

  DouQYuLChenHJinYYangXQinJet al. 3D deeply supervised network for automated segmentation of volumetric medical images. Med Image Anal (2017) 41:40–54. doi: 10.1016/j.media.2017.05.001

  • CrossRef
  • Google Scholar

• 26

  KamnitsasKLedigCNewcombeVFJSimpsonJPKaneADMenonDKet al. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal (2017) 36:61–78. doi: 10.1016/j.media.2016.10.004

  • CrossRef
  • Google Scholar