Exploring pathological signatures for predicting the recurrence of early-stage hepatocellular carcinoma based on deep learning

Background: Postoperative recurrence impedes the curability of early-stage hepatocellular carcinoma (E-HCC). We aimed to establish a novel recurrence-related pathological prognosticator with artificial intelligence, and investigate the relationship between pathological features and the local immunological microenvironment.

Methods: A total of 576 whole-slide images (WSIs) were collected from 547 patients with E-HCC in the Zhongshan cohort, which was randomly divided into a training cohort and a validation cohort. The external validation cohort comprised 147 Tumor Node Metastasis (TNM) stage I patients from The Cancer Genome Atlas (TCGA) database. Six types of HCC tissues were identified by a weakly supervised convolutional neural network. A recurrence-related histological score (HS) was constructed and validated. The correlation between immune microenvironment and HS was evaluated through extensive immunohistochemical data.

Results: The overall classification accuracy of HCC tissues was 94.17%. The C-indexes of HS in the training, validation and TCGA cohorts were 0.804, 0.739 and 0.708, respectively. Multivariate analysis showed that the HS (HR= 4.05, 95% CI: 3.40-4.84) was an independent predictor for recurrence-free survival. Patients in HS high-risk group had elevated preoperative alpha-fetoprotein levels, poorer tumor differentiation and a higher proportion of microvascular invasion. The immunohistochemistry data linked the HS to local immune cell infiltration. HS was positively correlated with the expression level of peritumoral CD14⁺ cells (p= 0.013), and negatively with the intratumoral CD8⁺ cells (p< 0.001).

Conclusions: The study established a novel histological score that predicted short-term and long-term recurrence for E-HCCs using deep learning, which could facilitate clinical decision making in recurrence prediction and management.

Introduction

Hepatocellular carcinoma (HCC) is the sixth most common malignancy and the fourth leading cause of cancer related deaths worldwide (1). Several treatments including surgery, locoregional therapies, and immunotherapy have been adopted as standards of care according to different tumor stages (2–4). Curative resection offers a chance of improved survival for HCC patients, especially those with early-stage tumors that are defined as Barcelona Clinic Liver Cancer (BCLC) stages 0 and A (5). Although the 5-year survival rate can reach 70% in early-stage HCC patients (6), half of patients suffer recurrence after liver resection (7) due to the lack of approved adjuvant therapies. Therefore, the precise prediction of postoperative recurrence is urgently needed.

Common prognostic factors for predicting early-stage HCC recurrence include pathological features, clinical biomarkers and genetic signatures (8). Recently, Yuan et al. developed a CpG methylation signature to elucidate the recurrence patterns in early-stage HCC with concordance indexes (C-indexes) of approximately 0.7 in three datasets (9). Compared to traditional staging systems, radiomics models also showed favorable efficacy in the recurrence prediction of HCC patients within the Milan criteria (10). In terms of histological features, specific structures such as microvascular invasion (MVI) and tumor-infiltrating lymphocytes were associated with recurrence risk (11, 12). However, due to the high heterogeneity in HCC, valuable information based on whole-slide images (WSIs) has not been thoroughly detected. Furthermore, the correlation between pathological texture and recurrence remains unknown.

In the past decade, breakthroughs in artificial intelligence (AI) have made remarkable progress in cancer research (13, 14). With the increasingly high capacity of deep learning, a large amount of work involved in the histopathological fields has been carried out, including tumor diagnosis, subtyping, grading, staging, and prognostic prediction (15–17), as well as the identification of pathological features, biomarkers, and genetic changes (18, 19). The advent of digital WSIs of tissue has not only economized the great amount of time or manual labor needed but also potentiated mining of subvisual morphometric phenotypes and ultimately improved patient management or therapeutic decision-making. Previous studies have originally proposed survival indicators based on digital WSIs via deep learning (20, 21). Nevertheless, the existing computational methods can identify only basic HCC structures. Complex and rich information contained in architectural features such as the portal area and lymphocytes is still hampered to be expounded. Herein, we explored more distinctive histological features to further describe the recurrence patterns and clinicopathological information of early-stage HCC.

The present study successfully developed a convolutional neural network (CNN) based on six classes of HCC tissues (namely, tumor region, normal liver tissue, portal area, fibrosis, hemorrhage/necrotic area, and lymphocyte area) and constructed a histological score (HS) via least absolute shrinkage and selection operator (LASSO) Cox regression to assess patients’ recurrence risk after hepatectomy. The novel model was validated in the three independent cohorts. By stratifying patients into different risk subgroups, their prognosis could be precisely appraised and multiomics characteristics were investigated.

Materials and methods

Patient cohort and study design

A total of 416 WSIs and 387 corresponding early-stage HCC patients who underwent radical resection at Zhongshan Hospital from January 2006 to December 2011 were retrospectively enrolled as the first dataset (Figure 1). The inclusion criteria were as follows: 1) pathologically proven HCC; 2) no neoadjuvant antitumor therapy; 3) Child–Pugh class A or B before surgery; and 4) BCLC stage 0-A. The exclusion criteria were as follows: 1) presence of other pathological types, such as intrahepatic cholangiocarcinoma (ICC) or combined hepatocellular cholangiocarcinoma (CHC); 2) previous antitumor treatment; 3) missing clinical information; and 4) death or disease recurrence within 1 month after resection. Data on tumor stages were collected according to the BCLC staging system (22) and China Liver Cancer (CNLC) (23) staging guidelines.

FIGURE 1

Figure 1 Workflow and general methodology of the study. (A) The recurrence-related scores were first developed and internally validated in a series of patients with BCLC stage 0-A treated by curative resection at Zhongshan Hospital. The scores were then externally validated by TNM-I stage patients in the TCGA cohort. (B) We first developed the neural network using 116 whole-slide images (WSIs) as the category-based training data. The network was then used to analyze the remaining WSIs and generate the classification maps. Pathological image features were extracted from typical tiles. Next, we constructed two recurrence prediction scores via LASSO-Cox. CS was composed of clinical and histological characteristics. HS was developed based on pure histological features. Finally, we analyzed the model discrimination ability, patient prognosis and local immune cell infiltration. TCGA, The Cancer Genome Atlas; PR, precision-recall; CS, combined score; HS, histological score; TPR, true positive rate; FPR, false positive rate.

Following the same inclusion and exclusion criteria, another 160 patients who underwent curative partial hepatectomy at Zhongshan Hospital from October 2014 to December 2014 and 160 WSIs were enrolled as the second dataset. We combined the two datasets and randomized cases into the training and validation cohorts at a ratio of 7:3.

We enrolled 154 WSIs and 147 patients from The Cancer Genome Atlas (TCGA) database as the external validation cohort (https://portal.gdc.cancer.gov/). Due to the lack of information on BCLC staging, all the enrolled cases had tumor node metastasis (TNM) stage I disease, which fulfilled the criterion of BCLC stage 0-A.

The follow-up was censored in December 2019. Recurrence was the primary endpoint in the present study. HCC recurrence was defined as the appearance of a newly detected HCC tumor confirmed on two radiologic images, with or without an elevation in serum tumor markers. Time to recurrence (TTR) was defined as the time between surgery and recurrence or metastasis. Recurrence-free survival (RFS) and overall survival (OS) were the secondary endpoints. RFS was defined as the time from the date of hepatectomy to the date of recurrence, metastasis, death, or the last follow-up. OS was defined as the time between resection and death. The study obtained ethical approval from the Institutional Review Board of Zhongshan Hospital and complied with the standards of the Declaration of Helsinki. Informed consent was received from each patient before surgery.

Preparation of H&E staining and immunohistochemistry for immune markers

Hematoxylin and eosin (H&E) staining was performed on paraffin-embedded tissues that were at 4 µm thickness. Tissue microarray (TMA) construction and immunohistochemistry for 28 immune markers were conducted as previously described (11, 24). Some of the previous data were used directly as a complement to the results (11).

Image annotation and processing

We randomly selected 116 WSIs for annotation in the Zhongshan cohort. Using ASAP 1.8, two pathologists manually annotated and fully examined the slides in six categories: tumor region, normal liver tissue, portal area, fibrosis, hemorrhage/necrotic area, and lymphocyte area. The annotated WSIs were divided into training, validation, and testing datasets at a ratio of 8:1:1. The annotated tissue areas were extracted based on the binary mask obtained by OTSU (25) and then divided into small squares, 299 pixels×299 pixels in size, called “tiles”. After image incision, data enhancement methods were used to balance the number of tiles for the six categories (details are provided in the Supplementary Methods).

Standardization of TCGA diagnostic slides

Due to the disparity in the staining and scanning process between the Zhongshan and TCGA cohorts, the trained neural model could not be directly applied to the TCGA WSIs. We modified the traditional Reinhard algorithm (26) to standardize stainingin both the Zhongshan and TCGA cohorts.

Classification network

We mainly proposed training a classification network to discriminate six types of HCC tissues. Inception V3 (27) was used as the basic model. Classification maps were derived after image recognition. Morphological processing was utilized to optimize the original classification maps. We used the t-distributed stochastic neighbor embedding (t-SNE) algorithm to visualize the segmental results. The pathological signatures were then extracted from the ten tiles with the highest prediction probability for each type (details are provided in the Supplementary Methods).

Establishment of the histological score and combined score

LASSO Cox regression (9) was applied to obtain high-dimensional prognostic features in histology. With recurrence status and TTR as labels, a combined score (CS) was constructed by integrally analyzing the histological signatures and clinical markers. The clinical markers included sex, age, serum levels of alpha-fetoprotein (AFP) and alanine transaminase (ALT), and morphological information such as tumor number and size. In the same way, we constructed a histological score (HS) through pure histological features (details are provided in the online Supplementary Materials). The optimal cutoff value for survival time was obtained through the “survminer” package. The patients were then divided into high-risk and low-risk subgroups for further survival comparisons.

Correlation between HS and immune infiltration

To evaluate the correlation between the HS and immune infiltration conditions, we applied hierarchical clustering analysis for specific immune markers in the TMA. In the TCGA datasets, the CIBERSORT algorithm (28) was used to explore the quantity of tumor-infiltrating immune cells based on the transcriptome signature.

Statistical analysis

Continuous variables are expressed as the median (IQR) and were compared with using Mann–Whitney U test. Categorical variables are expressed as numbers and percentages, and were compared with the χ² test or Fisher’s exact test. Kaplan–Meier curves with the log-rank test were used to compare survival. The LASSO Cox method was used to select independent factors associated with recurrence. Hazard ratios (HRs) and 95% confidence intervals (CIs) were also estimated by means of univariable and multivariable Cox analyses. A two-tailed p value> 0.05 was considered statistically significant. Model discrimination was assessed by the overall C-index, receiver operating characteristic (ROC) curve and net reclassification improvement (NRI) (29). Statistical analysis was performed using R-software 3.6.3 (R Foundation, Vienna, Austria) and SPSS ^® 22.0 (IBM, Armonk, New York, USA).

Results

Patient demographics and clinical information

Supplementary Table S1 describes the demographic, clinical, and tumor characteristics of patients in the training and validation cohorts. The median ages in the training and validation cohorts were 53 years and 55 years, respectively. Most patients in both cohorts were male. More than 80% of patients were infected with hepatitis B virus (HBV) and diagnosed with liver cirrhosis. Over half of the patients presented with elevated serum AFP levels in both cohorts (55.6% and 59.8%). Microvascular invasion was detected in 23.2% and 26.8% of patients in the training and validation cohorts, respectively. A majority of patients had a single lesion and were diagnosed as CNLC stage Ia. More patients were diagnosed with BCLC stage A than stage 0 in both cohorts. No significant differences were observed for the demographic information between the two cohorts.

After a median follow-up of 54.2 months (range, 3.0 to 64.6) for the Zhongshan population, 44.4% of patients (243/547) suffered tumor recurrence and 25.8% (141/547) died. The 1-, 3-, and 5-year recurrence rates were 17.1%, 33.1%, and 48.1%, respectively, and the 1-, 3-, and 5-year RFS rates were 82.2%, 66.4%, and 51.5%, respectively.

In terms of the TCGA cohort, the median follow-up time was 27.0 months (range, 1.1 to 115.9). A total of 38.8% of patients (57/147) suffered tumor recurrence and 21.1% (31/147) died. The 1-, 3-, and 5-year recurrence rates were 19.2%, 45.4%, and 62.0%, respectively, and the 1-, 3-, and 5-year RFS rates were 76.7%, 49.5%, and 34.5%, respectively.

Visualization of category-based HCC tissue

We first built the category-based model by training a classification network according to HCC tissue. The distribution of HCC tissues in a slide is displayed in Figure 2A. Corresponding histological images of the six categories before and after staining standardization are shown in Supplementary Figures S1A, B. Representative examples of the raw outputs of the classification network and their postprocessing results by morphology are revealed in Figure 2B, with different colors representing distinct tissue components. A visualization of a typical classification map in the TCGA cohort is shown in Supplementary Figure S1C.

FIGURE 2

Figure 2 Visualization of WSI classification for HCC tissue in the Zhongshan cohort. (A) Six classic categories of HCC tissue, including normal liver tissue (NLT), portal area (PA), fibrosis (FI), lymphocyte area (LA), tumor region (TR) and hemorrhage/necrotic area (H/NA). (B) Two representative outputs of the classification network. Red represents TR, green represents NLT, light blue represents FI, dark blue represents PA, purple represents LCA, and yellow represents H/NA. (C) Precision-recall curve of category-based sampling. (D) Normalized confusion matrix for the classification network. (E) t-SNE analysis for visualization of six tissue categories. (F) CAM results for visualization of HCC tissues. HCC, hepatocellular carcinoma; CAM, class activation mapping; t-SNE, t-distributed stochastic neighbor embedding.

After comprehensive training, we used the testing dataset to verify the performance of the computation network, which revealed an overall accuracy of 94.17%. Specifically, the accuracy values for normal liver tissue and tumor regions, shown in the confusion matrix (Figure 2C), were 0.984 and 0.950, respectively. A precision-recall (PR) curve rather than an ROC curve was applied to evaluate the model to minimize the imbalance. The area under the curve (AUC) values of each tissue type, except for the portal area, in our classification model exceeded 0.920 with the highest in tumor region classification (AUC= 0.997) (Figure 2D). Figure 2E shows the t-SNE visualization of the classification results. One thousand tiles were randomly selected for each tissue category.

We then used class activation mapping (CAM) (30) of the last convolution layer to visualize the outputs of each tissue category, where redder and bluer heatmaps indicated regions with higher or lower interest, respectively (Figure 2F).

Signature extraction and construction of prognostic scores

Next, we analyzed all 416 WSIs in the training cohort using the classification network and extracted pathological signatures from tiles of each type of HCC tissue. A total of 133 signatures and their coefficients were derived from LASSO Cox analysis (Supplementary Figure S2). Univariable Cox analysis of signatures that had a significant impact on RFS is shown in Supplementary Table S2.

HS and CS were then obtained by the sum of products by indexes and coefficients (details are provided in the Supplementary Methods). A calibration curve was constructed to evaluate the prediction accuracy. HS showed great concordance between the predicted and observed recurrence probabilities in the training, validation and TCGA cohorts (Figures 3A–C). Similarly, CS performed well in the training and validation cohorts (Supplementary Figure S3). To further compare the prediction reliability between the new scores and traditional biomarkers and stages, we carried out ROC curve analysis based on the cases in the training and validation cohorts. In terms of 1-year RFS, the AUC values of HS and CS were 0.837 and 0.857, respectively, much higher than those of clinical indicators such as AFP or liver cirrhosis, and the CNLC or BCLC staging systems (Figure 3D). The AUC values of HS and CS reached 0.857 and 0.852 for 3-year RFS prediction and 0.826 and 0.845 for 5-year RFS prediction, respectively (Figures 3E, F).

FIGURE 3

Figure 3 Comparison of predictive performance between two novel models. Upper: The calibration curves for TTR of HS in the training (A), validation (B), and TCGA cohort (C). Down: The ROC curves for 1-year (D), 3-year (E) and 5-year (F) RFS based on different clinicopathological features and stages. TCGA, The Cancer Genome Atlas; RFS, recurrence-free survival; HS, histological score; CS, combined score; AFP, alpha-fetoprotein; MVI, microvascular invasion; BCLC, Barcelona Clinic Liver Cancer; CNLC, China Liver Cancer; AUC, area under the curve.

Comparison of HS and CS in predictive accuracy

The C-indexes of HS, CS and clinical signatures were calculated to estimate the possibility of using histological textures as a substitute for clinical indicators. Compared to the clinical signatures, HS and CS presented higher C-indexes in the Zhongshan datasets (Table 1). Specifically, the C-indexes of HS were 0.804, 0.739 and 0.708 in the training, validation and TCGA cohorts, respectively. Both HS and CS performed better than clinical signatures in RFS prediction (Supplementary Figure S4). Ultimately, ROC curves in the two Zhongshan datasets showed no significant differences between HS and CS for 1-year, 3-year, or 5-year RFS prediction (Supplementary Figure S5, Supplementary Table S3). The NRI elucidated the quantitative difference between HS and CS in TTR prediction (Supplementary Figure S6). Subsequently, the CS model was superior to HS in all three periods with a subtle advantage, but statistical significance was not reached (p> 0.1).

TABLE 1

Table 1 C-indexes of the novel scores.

Survival prediction of novel pathological predictors

Optimal cutoff values for HS and CS were determined using the “survminer” package (31). All the patients were then divided into a high-risk group (HS> -0.1605954, CS> 3.417892) and a low-risk group (HS≤ -0.1605954, CS≤ 3.417892). Figure 4 and Supplementary Figure S7 depict the survival curves of HS and CS, respectively. Generally, patients in the high-risk group from the three cohorts faced notable susceptibility to postoperative recurrence and poorer survival.

FIGURE 4

Figure 4 Kaplan-Meier curves for recurrence rate, OS, and RFS in the training (A–C), validation (D–F) and TCGA cohorts (G–I) based on HS. HS, histological score; OS, overall survival; RFS, recurrence free survival.

Prognostic predictors of RFS in early-stage HCC

We performed Cox proportional hazards regression analysis to explore the independent predictors for RFS in the training and validation cohorts (Table 2). Eleven candidates were proven to be significant in the univariable analysis and were then evaluated with multivariable Cox regression. The multivariable analysis revealed that MVI (HR= 1.459, 95% CI: 1.094-1.948) and HS (HR= 4.054, 95% CI: 3.397-4.838) were significant indicators.

TABLE 2

Table 2 Cox proportional hazards regression model showing the association of variables with RFS.

The correlation between HS and clinicopathological characteristics

As shown in Figure 5, the clinical characteristics and prognostic value of HS were compared in the different risk subgroups, namely, the low-risk group (n= 332) and the high-risk group (n= 215). Compared to patients in the low-risk group, more patients in the high-risk group had elevated AFP levels (65.6% vs 51.2%). Moreover, HCCs in the high-risk group were characterized by poorer tumor differentiation, and a higher proportion of MVI. Seventy-eight (23.5%) patients in the low-risk group experienced recurrence. Conversely, 76.7% of patients in the high-risk group suffered recurrence during the follow-up. In particular, HS was proven to be a prognostic factor in all the subgroups in terms of patients’ clinicopathological characteristics.

FIGURE 5

Figure 5 Forest plot of recurrence-free survival based on HS in the Zhongshan cohort. HS, histological score; HR, hazard ratio. ; CI, confidence interval; AFP, α-fetoprotein; ALT, alanine aminotransferase; ALB, albumin; GGT, γ-glutamyl transpeptidase; MVI, micro vascular invasion; CNLC, China Liver Cancer Staging; BCLC, Barcelona Clinic Liver Cancer.

The histological markers including proliferation indexes, therapeutic targets, and specific diagnostic markers have been reported to be an efficient indicator for HCC diagnosis and prognosis after resection (32–34). The immunohistochemistry (IHC) and quantitative analysis for six markers was performed in 160 patients (from October 2014 to December 2014). Compared to HS low-risk group, patients in the high-risk group had a higher expression of heat shock protein 70 (HSP70, p= 0.01). A slightly higher proportion of Ki-67⁺ cells was observed in the high-risk group although significance was not reached (Supplementary Figure S8).

The correlation between pathological subtype and histologic features was also assessed. The results revealed that more patients (12%) in the HS high-risk subgroup were diagnosed with macrotrabecular-massive HCC (MTM-HCC) compared to 5% in the low-risk subgroup (Supplementary Figure S9).

The correlation between HS and the immune microenvironment

Using TMA data, we examined the expression patterns and distributions of 14 immune markers in 175 HCC patients with early-stage HCC (35). To investigate the relationship between HS and local immune status, we carried out clustering and correlation analysis (Figure 6A). We found that HS was positively correlated with the expression of peritumoral CD14 (p= 0.013, R= 0.187), but negatively correlated with the infiltration of intratumoral CD8 (p< 0.001, R= 0.275). Typical immunohistochemical images of the two markers are shown in Figure 6B. To investigate the interrelation between immune markers, we performed correlation analysis in different risk subgroups of HS (Figures 6C, D). Compared to the low-risk group, the predominant immune cells in the high-risk group were characterized by CD66-, CD68-, CD103- and CXCR5-positive cells infiltrating both tumoral and peritumoral tissues, which implied that macrophages or neutrophils may play a potential role in the progression of recurrence.

FIGURE 6

Figure 6 The relationship between HS and immune infiltration condition. (A) Heatmap and cluster analysis of the Zhongshan TMA. (B) Typical immunohistochemical pictures of CD8_T AND CD14_P. (C) The corrplots of immune markers in the HS low-risk group. (D) The interaction analysis of immune markers in the HS high-risk group. (E) Boxplot of immune cells in the TCGA cohort. Data are compared using Wilcoxon test. *p < 0.05;. (F) Correlation analysis between HS and M2macrophages in the TCGA cohort. HS, histological score; TCGA, The Cancer Genome Atlas.

The immune microenvironment of cases in the TCGA datasets was investigated by CIBERSORT algorithm (28). The boxplot revealed significantly higher expression of monocytes and lower expression of M2 macrophages in the HS low-risk subgroup (Figure 6E). HS was positively correlated with the infiltration of M2 macrophages (p= 0.011, R=0.275, Figure 6F). Similar findings suggested an increased interaction among macrophages, dendritic cells and other immune cells based on the corrplots in the TCGA database (Supplementary Figure S10).

Discussion

The emergence of AI has reformed multiple aspects of cancer management. The combination of deep learning and digital WSIs has alleviated the labor in detection and revealed the decent accuracy and efficacy for prognostic models in different solid tumors. The recurrence of early-stage HCC after resection remains a major obstacle in curative treatment. Although multiomics analysis highlighted potential recurrence-related determinants and therapeutic targets (8), its high cost and complexity have hindered its prevalent utilization. Herein, we integrated a neural network with the massive WSIs of HCC patients in BCLC stage 0-A and successfully developed an efficient recurrence prediction index that was prognostic of OS, TTR, and RFS. The novel index was validated in three independent cohorts, demonstrating the generalizability of our approach. By analyzing substantial TMA data, we found that immune infiltration status could potentially provide prognostically valuable information on histological texture.

Recent studies on AI reported novel prognostic models for HCC patients based on pathological images. Saillard et al. established two independent scores using an unsupervised neural network algorithm and attention mechanism according to tumoral or nontumoral annotated tiles (20). Both models showed high accuracy in survival prediction and strong correlations between clinical characteristics. Gao et al. innovatively divided HCC slides into four categories,tumor tissue, normal liver tissue, stroma, and necrosis, and then discriminated the features with high AUC values (21). The study developed and validated an efficient survival prediction model based on a large number of WSIs.

In these two previous studies, pathological signatures were directly generated and extracted by traditional CNNs. CNNs are usually regarded as a “black box”, in which the data are processed through complex computing layers; thus, it is difficult to concretize and interpret the relevant features of samples. In the present research, we manually input abundant signatures of the most relevant tiles, automatically screened the signatures with the recurrence time as the labels via LASSO Cox regression, and visually presented the weight of each signature. This procedure bypassed the need for the manual recognition of numerous postprocess tiles. In addition, the image signatures that came from empirical utilization ensured the feasibility (36, 37).

Compared to reported studies, our study has advantages in terms of multicategory training, heatmap visualization, a more accurate prediction of recurrence and multifaceted analysis. HCC is highly heterogeneous with regard to not only genetics or epigenetics but also histology. Despite the classic histological categories mentioned above, the portal area and lymphocyte area are also important structures. Recently, tertiary lymphoid structures were proven to be a favorable factor for prognosis and recurrence status (38), and immune infiltration conditions are becoming more determinant in precision therapy (39). The portal area is a connective tissue among hepatic lobules consisting of branches of the hepatic artery, portal vein and hepatic ducts, in which lymphatic vessels and nerve fibers exist. It was reported that the number of portal areas and inflammation or iron deposition around the portal area were associated with the pathogenesis of HCC (40, 41). Herein, these two structures were originally annotated and trained in the study. The AUC value for the portal area was 0.757, lower than that of the other five tissues, which could be caused by poor structural purity and inadequate amounts. Technically, modified image standardization balanced the color differences among tiles, which raised the overall recognition accuracy for HCC structures up to 94.17%. We also applied the CAM method to visualize the importance of the local structure. As shown in Figure 2, cells attracted more attention for recurrence than cell-free areas such as fibrosis and necrosis areas, which made it easier for us to intuitively understand the microscopic information.

Subsequently, we established two recurrence prediction scores via LASSO Cox analysis and derived each score through the overall WSIs to maximally preserve the pathological signatures of all sections. Both scores showed great congruence with the recurrence probability and survival. No significant difference was found between HS and CS under NRI analysis, which consequently suggested the feasibility of the potential replacement of important clinical characteristics with pure histological features. In Gao et al’s research, the newly constructed score aimed at OS and its median C-indexes reached 0.731 and 0.713 in two cohorts. Therefore, we specifically targeted recurrence conditions after resection and complemented the recurrence-related information in the TCGA database, which was not included in Gao’s study. In our study, the C-indexes of HS for TTR prediction reached 0.804, 0.739 and 0.708 in the training, validation and TCGA cohorts, respectively.

We compared clinicopathological characteristics between the two risk subgroups. Notably, a larger proportion of MTM-HCC was found in the HS high-risk group, indicating that invisible information processed by deep learning could be explained by the specific texture of tumor cells. In contrast to Saillard et al’s dissected tiles of MTM structures, we analyzed the pathological subtype by a whole slide, which was closer to the definition of MTM-HCC (42). As a rare and highly malignant tumor subtype of HCC, MTM-HCC was proven to be correlated with an increased recurrence risk and poor survival (42, 43). Our study supported this finding. Staining markers for pathological evaluation have gained increasing attention in postoperative management. HSP70 is identified as an upregulated marker in HCC components and performs its role at several points of apoptotic signaling (33). A significant difference was observed in the expression of HSP70 between two risk subgroups, which was consistent with the previous study (44).

The immune microenvironment plays a crucial role in tumor progression and recurrence. Using TMA data, we fully explored the relationship between HS and immune infiltration. CD14-P and CD8-T were shown to be of significance. Our results were supported by a previous study showing that high densities of both CD3(+) and CD8(+) T cells in both the interior and margin were significantly associated with a low rate of recurrence (p= 0.007) and prolonged RFS (p= 0.002) (45). A recent study reported that a high density of marginal CCR1⁺ CD14⁺ monocytes positively correlated with CCL15 expression and was an independent index for dismal survival (46). Moreover, peritumoral monocytes were found to promote HCC progression by inducing cancer cell autophagy (47), which probably led to the dense expression of CD14-P in patients with high recurrence risk.

From TCGA immune data, we observed that more M2 macrophages were aggregated in the tumor tissues of patients with high HS. This finding was consistent with the prognostic value and tumor biochemical modulation of M2 macrophages (48).

The correlation analysis in both the Zhongshan and TCGA cohorts implied different interaction patterns of immune markers. The stronger interactive effect of macrophages and dendritic cells in the HS high-risk subgroup highlighted the important status of antigen presentation during tumor progression. The result may offer new prospects for further fundamental research.

There are several limitations in our study. First, our training data came from a single institution. There may be image inconsistency in model validation; thus, standardization is an essential step for processing. Second, the patients in both cohorts were predominantly infected with HBV, which reduced the representativity of an extensive HCC population. A more rigorous external validation dataset needs to be validated before routine clinical use. Third, our research carried out deep learning at the histologic level of WSIs. Further study could focus on single-cell discrimination, such as lymphocytes and cancer-associated fibroblasts. Fourth, AI-based study of multiomics sequencing information, more staining markers or images of multicomplex immunofluorescence could be further combined into the prediction model.

Conclusion

In conclusion, the study proposed an efficient recurrence prediction score for patients with early-stage HCC based on deep learning. The prognostic pathological features identified in digital WSIs composed a computable index to discriminate patients in terms of their relapse risk. The new model derived by weakly supervised training facilitated the classification process of typical HCC tissues, depicted the immune infiltration condition in intratumoral and peritumoral structures, and highlighted the clinical characteristics that were significant to prognosis. Further AI research may pay attention to interpretation at the cellular level and the integration of therapeutic decisions or multiomics sequencing.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The study obtained ethical approval from the Institutional Review Board of Zhongshan Hospital (B2021-611) and complied with the standards of the Declaration of Helsinki. Informed consent was received from each patient before the research.

Author contributions

(I) Conception and design: W-FQ, M-XT, Y-HS; (II) Administrative support: Y-HS, Y-YH, HZ; (III) Provision of study materials or patients: Y-HS, Y-YH, HZ, JZ, JF; (IV) Collection and assembly of data: W-FQ, M-XT, J-TQ, Y-CG, W-RL, ZT, C-YT, W-AH, X-YL, Z-XW, KQ; (V) Data analysis and interpretation: W-FQ, M-XT, J-TQ; (VI) Manuscript writing: W-FQ, M-XT, J-TQ; (VII) Final approval of manuscript: All authors.

Funding

This work was supported by grants from the National Natural Science Foundation of China (No.81773067, 81902963, 882073217, 82073218, and 82003084), Shanghai Sailing Program (19YF1407800), Intelligent Chronic Disease Management System Based on Edge and Cloud Computing Cooperation (2020-002), Shanghai Municipal Science and Technology Major Project (Grant No. 2018SHZDZX05). Shanghai Municipal Key Clinical Specialty. CAMS Innovation Fund for Medical Sciences (CIFMS) (2019-I2M-5-058). National Key R&D Program of China (2020YFE0202200, 2018YFF0301102 and 2018YFF0301105).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fonc.2022.968202/full#supplementary-material

References

1. Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin (2018) 68(6):394–424. doi: 10.3322/caac.21492

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Llovet JM, De Baere T, Kulik L, Haber PK, Greten TF, Meyer T, et al. Locoregional therapies in the era of molecular and immune treatments for hepatocellular carcinoma. Nat Rev Gastroenterol Hepatol (2021) 18(5):293–313. doi: 10.1038/s41575-020-00395-0

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Rizzo A, Ricci AD, Gadaleta-Caldarola G, Brandi G. First-line immune checkpoint inhibitor-based combinations in unresectable hepatocellular carcinoma: current management and future challenges. Expert Rev Gastroenterol Hepatol (2021) 15(11):1245–51. doi: 10.1080/17474124.2021.1973431

PubMed Abstract | CrossRef Full Text | Google Scholar

4. De Lorenzo S, Tovoli F, Barbera MA, Garuti F, Palloni A, Frega G, et al. Metronomic capecitabine vs. best supportive care in child-pugh b hepatocellular carcinoma: a proof of concept. Sci Rep (2018) 8(1):9997. doi: 10.1038/s41598-018-28337-6

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Marrero JA, Kulik LM, Sirlin CB, Zhu AX, Finn RS, Abecassis MM, et al. Diagnosis, staging, and management of hepatocellular carcinoma: 2018 practice guidance by the american association for the study of liver diseases. Hepatology (2018) 68(2):723–50. doi: 10.1002/hep.29913

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Tsilimigras DI, Bagante F, Sahara K, Moris D, Hyer JM, Wu L, et al. Prognosis after resection of barcelona clinic liver cancer (BCLC) stage 0, a, and b hepatocellular carcinoma: A comprehensive assessment of the current BCLC classification. Ann Surg Oncol (2019) 26(11):3693–700. doi: 10.1245/s10434-019-07580-9

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Tabrizian P, Jibara G, Shrager B, Schwartz M, Roayaie S. Recurrence of hepatocellular cancer after resection: patterns, treatments, and prognosis. Ann Surg (2015) 261(5):947–55. doi: 10.1097/SLA.0000000000000710

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Jiang Y, Sun A, Zhao Y, Ying W, Sun H, Yang X, et al. Proteomics identifies new therapeutic targets of early-stage hepatocellular carcinoma. Nature (2019) 567(7747):257–61. doi: 10.1038/s41586-019-0987-8

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Qiu J, Peng B, Tang Y, Qian Y, Guo P, Li M, et al. CpG methylation signature predicts recurrence in early-stage hepatocellular carcinoma: Results from a multicenter study. J Clin Oncol (2017) 35(7):734–42. doi: 10.1200/JCO.2016.68.2153

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Ji GW, Zhu FP, Xu Q, Wang K, Wu MY, Tang WW, et al. Radiomic features at contrast-enhanced CT predict recurrence in early stage hepatocellular carcinoma: A multi-institutional study. Radiology (2020) 294(3):568–79. doi: 10.1148/radiol.2020191470

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Liu WR, Tian MX, Tang Z, Fang Y, Zhou YF, Song SS, et al. Nine-factor-based immunohistochemistry classifier predicts recurrence for early-stage hepatocellular carcinoma after curative resection. Br J Cancer (2020) 123(1):92–100. doi: 10.1038/s41416-020-0864-0

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Lim KC, Chow PK, Allen JC, Chia GS, Lim M, Cheow PC, et al. Microvascular invasion is a better predictor of tumor recurrence and overall survival following surgical resection for hepatocellular carcinoma compared to the milan criteria. Ann Surg (2011) 254(1):108–13. doi: 10.1097/SLA.0b013e31821ad884

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts H. Artificial intelligence in radiology. Nat Rev Cancer (2018) 18(8):500–10. doi: 10.1038/s41568-018-0016-5

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Bhinder B, Gilvary C, Madhukar NS, Elemento O. Artificial intelligence in cancer research and precision medicine. Cancer Discov (2021) 11(4):900–15. doi: 10.1158/2159-8290.CD-21-0090

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Bera K, Schalper KA, Rimm DL, Velcheti V, Madabhushi A. Artificial intelligence in digital pathology - new tools for diagnosis and precision oncology. Nat Rev Clin Oncol (2019) 16(11):703–15. doi: 10.1038/s41571-019-0252-y

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Ahn JC, Connell A, Simonetto DA, Hughes C, Shah VH. Application of artificial intelligence for the diagnosis and treatment of liver diseases. Hepatology (2021) 73(6):2546–63. doi: 10.1002/hep.31603

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Campanella G, Hanna MG, Geneslaw L, Miraflor A, Werneck Krauss Silva V, Busam KJ, et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat Med (2019) 25(8):1301–9. doi: 10.1038/s41591-019-0508-1

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Coudray N, Ocampo PS, Sakellaropoulos T, Narula N, Snuderl M, Fenyo D, et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat Med (2018) 24(10):1559–67. doi: 10.1038/s41591-018-0177-5

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Liao H, Long Y, Han R, Wang W, Xu L, Liao M, et al. Deep learning-based classification and mutation prediction from histopathological images of hepatocellular carcinoma. Clin Transl Med (2020) 10(2):e102. doi: 10.1002/ctm2.102

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Saillard C, Schmauch B, Laifa O, Moarii M, Toldo S, Zaslavskiy M, et al. Predicting survival after hepatocellular carcinoma resection using deep learning on histological slides. Hepatology (2020) 72(6):2000–13. doi: 10.1002/hep.31207

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Shi JY, Wang X, Ding GY, Dong Z, Han J, Guan Z, et al. Exploring prognostic indicators in the pathological images of hepatocellular carcinoma based on deep learning. Gut (2021) 70(5):951–61. doi: 10.1136/gutjnl-2020-320930

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Llovet JM, Bru C, Bruix J. Prognosis of hepatocellular carcinoma: the BCLC staging classification. Semin Liver Dis (1999) 19(3):329–38. doi: 10.1055/s-2007-1007122

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Xie DY, Ren ZG, Zhou J, Fan J, Gao Q. 2019 Chinese clinical guidelines for the management of hepatocellular carcinoma: Updates and insights. Hepatobiliary Surg Nutr (2020) 9(4):452–63. doi: 10.21037/hbsn-20-480

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Gao Q, Qiu SJ, Fan J, Zhou J, Wang XY, Xiao YS, et al. Intratumoral balance of regulatory and cytotoxic t cells is associated with prognosis of hepatocellular carcinoma after resection. J Clin Oncol (2007) 25(18):2586–93. doi: 10.1200/JCO.2006.09.4565

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Otsu N. A threshold selection method from gray-level histograms. IEEE Trans syst. man cybern. (1979) 9(1):62–6. doi: 10.1109/TSMC.1979.4310076

CrossRef Full Text | Google Scholar

26. Reinhard E, Adhikhmin M, Gooch B, Shirley P. Color transfer between images. IEEE Comput Graphics Appl (2001) 21(5):34–41. doi: 10.1109/38.946629

CrossRef Full Text | Google Scholar

27. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z eds. Rethinking the inception architecture for computer vision. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016; Las Vegas, NV, USA: IEEE

Google Scholar

28. Newman AM, Liu CL, Green MR, Gentles AJ, Feng W, Xu Y, et al. Robust enumeration of cell subsets from tissue expression profiles. Nat Methods (2015) 12(5):453–7. doi: 10.1038/nmeth.3337

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Leening MJ, Vedder MM, Witteman JC, Pencina MJ, Steyerberg EW. Net reclassification improvement: computation, interpretation, and controversies: a literature review and clinician's guide. Ann Intern Med (2014) 160(2):122–31. doi: 10.7326/M13-1522

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A editors. Learning Deep Features for Discriminative Localization. 016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). CVPR (2016) Las Vegas, NV, USA: IEEE.

Google Scholar

31. Xi J, Yin J, Liang J, Zhan C, Jiang W, Lin Z, et al. Prognostic impact of radiological consolidation tumor ratio in clinical stage IA pulmonary ground glass opacities. Front Oncol (2021) 11:616149. doi: 10.3389/fonc.2021.616149

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Cao Y, Ke R, Wang S, Zhu X, Chen J, Huang C, et al. DNA topoisomerase IIalpha and Ki67 are prognostic factors in patients with hepatocellular carcinoma. Oncol Lett (2017) 13(6):4109–16. doi: 10.3892/ol.2017.5999

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Wang C, Zhang Y, Guo K, Wang N, Jin H, Liu Y, et al. Heat shock proteins in hepatocellular carcinoma: Molecular mechanism and therapeutic potential. Int J Cancer (2016) 138(8):1824–34. doi: 10.1002/ijc.29723

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Zhou F, Shang W, Yu X, Tian J. Glypican-3: A promising biomarker for hepatocellular carcinoma diagnosis and treatment. Med Res Rev (2018) 38(2):741–67. doi: 10.1002/med.21455

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Tian MX, Liu WR, Wang H, Zhou YF, Jin L, Jiang XF, et al. Tissue-infiltrating lymphocytes signature predicts survival in patients with early/intermediate stage hepatocellular carcinoma. BMC Med (2019) 17(1):106. doi: 10.1186/s12916-019-1341-6

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Klimov S, Miligy IM, Gertych A, Jiang Y, Toss MS, Rida P, et al. A whole slide image-based machine learning approach to predict ductal carcinoma in situ (DCIS) recurrence risk. Breast Cancer Res (2019) 21(1):83. doi: 10.1186/s13058-019-1165-5

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Yu KH, Zhang C, Berry GJ, Altman RB, Re C, Rubin DL, et al. Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features. Nat Commun (2016) 7:12474. doi: 10.1038/ncomms12474

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Calderaro J, Petitprez F, Becht E, Laurent A, Hirsch TZ, Rousseau B, et al. Intra-tumoral tertiary lymphoid structures are associated with a low risk of early recurrence of hepatocellular carcinoma. J Hepatol (2019) 70(1):58–65. doi: 10.1016/j.jhep.2018.09.003

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Sia D, Jiao Y, Martinez-Quetglas I, Kuchuk O, Villacorta-Martin C, Castro de Moura M, et al. Identification of an immune-specific class of hepatocellular carcinoma, based on molecular features. Gastroenterology (2017) 153(3):812–26. doi: 10.1053/j.gastro.2017.06.007

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Lukacs-Kornek V. The role of lymphatic endothelial cells in liver injury and tumor development. Front Immunol (2016) 7:548. doi: 10.3389/fimmu.2016.00548

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Chung JW, Shin E, Kim H, Han HS, Cho JY, Choi YR, et al. Hepatic iron overload in the portal tract predicts poor survival in hepatocellular carcinoma after curative resection. Liver Int (2018) 38(5):903–14. doi: 10.1111/liv.13619

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Ziol M, Pote N, Amaddeo G, Laurent A, Nault JC, Oberti F, et al. Macrotrabecular-massive hepatocellular carcinoma: A distinctive histological subtype with clinical relevance. Hepatology (2018) 68(1):103–12. doi: 10.1002/hep.29762

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Mule S, Galletto Pregliasco A, Tenenhaus A, Kharrat R, Amaddeo G, Baranes L, et al. Multiphase liver MRI for identifying the macrotrabecular-massive subtype of hepatocellular carcinoma. Radiology (2020) 295(3):562–71. doi: 10.1148/radiol.2020192230

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Bauer K, Nitsche U, Slotta-Huspenina J, Drecoll E, von Weyhern CH, Rosenberg R, et al. High HSP27 and HSP70 expression levels are independent adverse prognostic factors in primary resected colon cancer. Cell Oncol (Dordr) (2012) 35(3):197–205. doi: 10.1007/s13402-012-0079-3

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Gabrielson A, Wu Y, Wang H, Jiang J, Kallakury B, Gatalica Z, et al. Intratumoral CD3 and CD8 t-cell densities associated with relapse-free survival in HCC. Cancer Immunol Res (2016) 4(5):419–30. doi: 10.1158/2326-6066.CIR-15-0110

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Liu LZ, Zhang Z, Zheng BH, Shi Y, Duan M, Ma LJ, et al. CCL15 recruits suppressive monocytes to facilitate immune escape and disease progression in hepatocellular carcinoma. Hepatology (2019) 69(1):143–59. doi: 10.1002/hep.30134

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Chen DP, Ning WR, Li XF, Wei Y, Lao XM, Wang JC, et al. Peritumoral monocytes induce cancer cell autophagy to facilitate the progression of human hepatocellular carcinoma. Autophagy (2018) 14(8):1335–46. doi: 10.1080/15548627.2018.1474994

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Yeung OW, Lo CM, Ling CC, Qi X, Geng W, Li CX, et al. Alternatively activated (M2) macrophages promote tumour growth and invasiveness in hepatocellular carcinoma. J Hepatol (2015) 62(3):607–16. doi: 10.1016/j.jhep.2014.10.029

PubMed Abstract | CrossRef Full Text | Google Scholar