A Support Vector Machine Based on Liquid Immune Profiling Predicts Major Pathological Response to Chemotherapy Plus Anti-PD-1/PD-L1 as a Neoadjuvant Treatment for Patients With Resectable Non-Small Cell Lung Cancer

The biomarkers for the pathological response of neoadjuvant chemotherapy plus anti-programmed cell death protein-1/programmed cell death-ligand 1 (PD-1/PD-L1) (CAPD) are unclear in non-small cell lung cancer (NSCLC). Two hundred and eleven patients with stage Ib-IIIa NSCLC undergoing CAPD prior to surgical resection were enrolled, and 11 immune cell subsets in peripheral blood were prospectively analyzed using multicolor flow cytometry. Immune cell subtypes were selected by recursive feature elimination and least absolute shrinkage and selection operator methods. The support vector machine (SVM) was used to build a model. Multivariate analysis for major pathological response (MPR) was also performed. Finally, five immune cell subtypes were identified and an SVM based on liquid immune profiling (LIP-SVM) was developed. The LIP-SVM model achieved high accuracies in discovery and validation sets (AUC = 0.886, 95% CI: 0.823–0.949, P < 0.001; AUC = 0.874, 95% CI: 0.791–0.958, P < 0.001, respectively). Multivariate analysis revealed that age, radiological response, and LIP-SVM were independent factors for MPR in the two sets (each P < 0.05). The integration of LIP-SVM, clinical factors, and radiological response showed significantly high accuracies for predicting MPR in discovery and validation sets (AUC = 0.951, 95% CI: 0.916–0.986, P < 0.001; AUC = 0.943, 95% CI: 0.912–0.993, P < 0.001, respectively). Based on immune cell profiling of peripheral blood, our study developed a predictive model for the MPR of patients with NSCLC undergoing CAPD treatment that can potentially guide clinical therapy.

Introduction

The combination of anti-programmed death receptor 1 (PD-1), or its ligand (PD-L1), and chemotherapy has recently gained attention in patients with advanced non-small cell lung cancer (NSCLC). In the KEYNOTE-407, KEYNOTE-189, and IMPOWER-130 studies (1–3), platinum-based double-chemotherapy plus anti-PD-1/PD-L1 (such as atezolizumab and pembrolizumab) showed significantly higher objective response, longer progression-free survival (PFS), and overall survival (OS) than chemotherapy alone in patients with metastatic NSCLC. The latest CHECKMATE-816 study showed that neoadjuvant nivolumab plus chemotherapy significantly improves the major pathological response (MPR), as reported in the American Society of Clinical Oncology 2021 (abstract number: 8503). The tumor mutation burden and PD-L1 expression were not found to be related to MPR in patients with NSCLC treated with neoadjuvant chemotherapy plus anti-PD-1/PD-L1 (CAPD). A recent study (SAKK 16/14) reported that patients who achieved an MPR showed longer event-free survival and OS than patients who did not among patients who underwent a combination of perioperative durvalumab and neoadjuvant chemotherapy (4). However, only a subset of patients with NSCLC acquired an MPR when undergoing CAPD as a neoadjuvant treatment. Therefore, identifying novel and useful biomarkers to predict the patients most likely to acquire MPR from CAPD before surgery is critical.

Tumor-infiltrating lymphocytes and peripheral blood immune cells were found to be related to the response of solid tumors to therapy (5–9). Circulating exhausted-phenotype CD8⁺ T cells are associated with poor immunological response to pembrolizumab in patients with stage IV melanoma (10). Furthermore, circulating PD-1⁺CD8⁺ T cells, memory T cells, and elevated monocyte levels are strong predictors of response to immunotherapy (11–13). Therefore, we speculated that the immune cell subsets in peripheral blood may be associated with the MPR to CAPD as a neoadjuvant treatment of patients with NSCLC; if confirmed, an immune cell model can be built.

Machine learning has been gaining attention with respect to medical image recognition tasks and building prognostic prediction models from high-dimensional gene expression profiling data (14–18). A CHECKMATE-025 study developed a Bayesian network model for predicting immunotherapy prognosis in patients with metastatic renal cell carcinoma (19). Using radiomics biomarkers, radiology text reports, or somatic mutations, machine learning models could estimate the response and prognosis in patients with NSCLC treated with immunotherapy (20–22). To the best of our knowledge, the use of a support vector machine (SVM) based on immune cells to predict the MPR to CAPD treatment has never been reported.

Here, immune cell profiling was performed before the initial CAPD neoadjuvant therapy of patients with NSCLC before surgery. Using machine learning, a predictive immunological model was constructed and validated that can help identify patients who would most likely acquire MPR while undergoing CAPD.

Methods

Patients

Each patient provided detailed informed consent to the investigator. PD-L1 expression and EGFR/ALK mutation status were not necessary conditions for enrollment. The key eligibility criteria were as follows: (i) patients were at least 18 years old and willing to provide routine peripheral blood (2 mL) for immune cell analysis; (ii) Eastern Cooperative Oncology Group performance status was 0–1; (iii) a tissue biopsy of the tumor was confirmed to be lung adenocarcinoma (LUAD) or lung squamous cell carcinoma (LUSC) before any treatment; and (iv) patients with resectable stage Ib-IIIa NSCLC were examined using whole-body computed tomography (CT) or positron emission tomography-CT. The exclusion criteria were: (i) patients who could not tolerate treatment, such as those allergic to albumin-bound paclitaxel; (ii) patients undergoing CAPD but whose NSCLC was progressing rapidly or showed organ metastasis and were not suitable for resection treatment. Between September 2019 and June 2021, 211 patients who were receiving neoadjuvant treatment before surgery and met the criteria were recruited at the Cancer Hospital of the University of Chinese Academy of Sciences (CHUCAS) (Supplementary Figure 1). The patients were randomly divided in a 3:2 ratio into a discovery set (n = 127) and validation set (n = 84). The institutional review board of the Second Affiliated Hospital of Guizhou Medical University and CHUCAS approved our clinical research design. We have been conducted in accordance with the World Medical Association’s Declaration of Helsinki.

Study Design

A flowchart of the study design is shown in Figure 1. Before treatment, whole blood samples were collected from patients and rapidly tested by multicolor flow cytometry before initial treatment. The detailed results of peripheral immune cells and clinical characteristics were recorded. Two hundred and eleven patients with NSCLC received 2–4 cycles of CAPD. According to the MPR status, the doctors then performed radical surgery of lung cancer. The pathological response of tumor tissues was estimated by a senior pathologist. We then analyzed the association between clinical factors and MPR using univariate analysis of both cohorts (discovery and validation sets). The immune cells in peripheral blood were chosen by recursive feature elimination (RFE) and least absolute shrinkage and selection operator (LASSO) methods. After integrating these two selection methods, the final immune cell subtypes were confirmed. An SVM model based on liquid immune profiling (LIP-SVM) was developed and validated in the two cohorts. Multivariate analysis for MPR was performed for patients with NSCLC using logistic regression. The integration of LIP-SVM, clinical factors, or radiological response was evaluated in terms of predictive accuracy for MPR. Contrast-enhanced CT images were examined to clearly identify the primary tumor. The radiological response, including complete response (CR), partial response (PR), stable disease (SD), and progressive disease (PD), was estimated before radical surgery by a senior thoracic radiologist using Response Evaluation Criteria in Solid Tumors (RECIST) version 1.1.

FIGURE 1

Figure 1 The study flowchart from patient enrollment to machine learning. Whole blood samples from patients with NSCLC on an empty stomach were collected and analyzed by multicolor flow cytometry before treatment. Patients with NSCLC were subjected to neoadjuvant CAPD treatment followed by radical surgery of lung cancer and MPR evaluation of tumor tissues. The association between clinical factors, radiological response, and MPR was analyzed by univariate analysis. The immune cells were selected by RFE and LASSO methods. The LIP-SVM signature was built and tested in the discovery and validation sets. Then, multivariate analysis for MPR was performed by logistic regression. Finally, three models integrating LIP-SVM, clinical factors, or radiological response were evaluated for predicting MPR. NSCLC, non-small cell lung cancer; CAPD, chemotherapy plus anti-PD-1/PD-L1; MPR, major pathological response; RFE, recursive feature elimination; LASSO, least absolute shrinkage and selection operator; LIP-SVM, support vector machine model based on liquid immune profiling.

Assessment of MPR for Patients Receiving CAPD

MPR is defined as the reduction of viable tumors to clinically defined significant margins, depending on the particular histological type of lung cancer and the specific treatment type. All histological types of lung cancer had an MPR with a histological definition of less than or equal to 10% of the viable tumor. The MPR was calculated by dividing the size of the viable tumor by the size of the tumor bed. Here, this was used to establish the threshold of the number of clinical trials. The pathology report recorded the total number of masses in the tumor bed, including some uninvolved lungs, even if these masses were not entirely composed of the tumor bed. The MPR can also be classified as a primary pulmonary tumor, where little or no viable metastatic carcinomas were found in the lymph nodes (ypT0, N1, 2, 3).

Neoadjuvant Treatment Strategy of CAPD

Patients with LUAD preoperatively received a folic acid, vitamin B12, and glucocorticoid pretreatment that is prescribed according to the local guidelines for pemetrexed. All patients received intravenous injections of cisplatin (75 mg/m², d1) or carboplatin (under the concentration-time curve, 5 mg/mL/min, d1) and pemetrexed (500 mg/m², d1) for 2–4 cycles. Patients with LUSC preoperatively received intravenous injections of cisplatin (75 mg/m², d1) or carboplatin (under the concentration-time curve, 5 mg/mL/min, d1) and nab-paclitaxel (135 mg/m², d1, d8) for 2–4 cycles. The anti-PD-1 regimen included camrelizumab (3 mg/kg, Q2W), pemetrexed (500 mg/m², Q3W), nivolumab (3 mg/kg, Q2W), toripalimab (3 mg/kg, Q2W), tislelizumab (200 mg/m², Q3W), or sintilimab (200 mg, Q3W), whereas the anti-PD-L1 regimen included durvalumab (10 mg/kg, Q2W); the patients received both injections following each chemotherapy cycle. Chemotherapy and anti-PD-1/PD-L1 regimens were administered every 3 weeks to patients with LUAD and LUSC.

Defining and Profiling the Immune Cell Subtypes in Peripheral Blood

We evaluated four types of circulating immune cells: B, T, natural killer (NK), and natural killer T (NKT) cells. B and T cells were defined by CD19 expression (CD19⁺ B cells) and CD3 expression (CD3⁺ T cells), respectively. The presence of CD8 and CD4 was used to identify T-lymphocyte subsets (CD3⁺CD8⁺ T cells and CD3⁺CD4⁺ T cells). Memory (CD4⁺CD45RO⁺) T cells and CD4⁺ naïve (CD4⁺CD45RA⁺) T cells were identified by CD45RA and CD45RO expression. A combination of CD56 and CD3 was used to identify NKT (CD3⁺CD56⁺) and NK (CD3^-CD56⁺) lymphocyte subsets. Activated CD8⁺ T cells (CD8⁺CD38⁺ T cells) were recognized by CD38 expression. BD Biosciences (San Jose, CA, USA) provided the following antibodies in Supplementary Table 1: CD8-FITC (#555366), CD4-FITC (#550628), CD3-FITC (#555332), CD56-FITC (#55664), CD19-FITC (#555412), CD45RO-APC (#559865), CD38-PE (#555460), CD45RA-PE (#555489), and FITC/APC/PE controls (#55749; #555748; #5555776). For lymphocyte staining, 4 mL of peripheral blood was collected into a blood tube with ethylenediaminetetraacetic acid and an anticoagulant. Next, 20 μL of CD3-FITC/CD56-PE, CD19-FITC, CD4-FITC/CD45RO-APC/CD45RA-PE, CD8-FITC/CD38-PE, and FITC/PE/APC isotype controls was separately added to five flow cytometry tubes and 100 μL of every blood sample was added to every test tube. The tubes were sufficiently mixed with corresponding antibodies in the dark and incubated for 30 min at room temperature (20°C). A hemolytic agent (up to 2 mL; #70-LSB3; BD Biosciences) was then added to every tube. The supernatant was then removed by centrifugation (6 min), washed twice with phosphate-buffered saline (#SH300256; Hyclone, Logan, UT, USA), and resuspended in paraformaldehyde. Flow cytometry (FACSVia; BD Biosciences) was used to examine the cells. More than 2,000 cells were detected at the lymphocyte gate in every sample. CellQuest Pro software (BD Biosciences) was used to analyze the percentages of positively labeled lymphocytes. The staining procedure was completed and analyzed within 24 h after blood collection.

Feature Selection of RFE and LASSO Algorithms

Two feature selection methods (RFE and LASSO) were used in this study. RFE recursively reduces the size of the examined feature set to select features. The prediction model based on the original features is trained and a weight is assigned to each feature. Features with a minimum absolute weight are recursively removed until the desired number is reached. Random forest function and 5-fold cross-validation (CV) sampling were used for RFE. In addition, we used LASSO to select the most important immune cells from the discovery set. A log partial likelihood subject based on the LASSO method is minimized to add the absolute values of the parameters. Here, the standardized constraint parameter was set to -1.434. RFE and LASSO were performed using the “caret” package in R version 3.5.1.

SVM Building Model

The SVM is a classical model of machine learning with important value in tumor classification, prognosis, and treatment response predictions (23). Radial basis function, the most popular kernel function of SVM for nonlinear classification, can significantly improve the classification ability of the SVM by mapping the original input space to the feature space. The original nonlinear input space is transformed into the linear separability space and classified linearly in the feature space. The equation we used was as follows:

$$K(x,z)=\exp (-\gamma {\left|x-z\right|}^2)$$

where γ is greater than 0 and the parameters need to be adjusted. The tuning parameters were set as sigma = 0.035, C = 100, and cross = 10. The “kernlab” library was implemented using R software.

Statistical Analysis

R software and GraphPad Prism were used to perform statistical analysis. A correlation heatmap was generated using the “pheatmap” package to depict the relationships between immune cells in the discovery and validation sets. Correlation scatter plots were used to indicate associations between immune cells. The scatter dot and box plots were used to represent the median and 95% confidence intervals (CI). The differences between groups were analyzed using the Mann–Whitney U test. The receiver operating characteristic curves (ROC) were plotted using the “pROC” package. The frequencies of two groups were compared using the chi-square test. Univariate and multivariate logistic analyses for MPR were performed in the two sets. Statistical significance was defined as P-value < 0.05.

Results

Clinical Characteristics and MPR

The baseline characteristics of 211 patients from two independent cohorts are presented in Table 1. Most patients (92.42%) were male, old (> 60 years; 60.67%), smokers (78.67%), and with squamous cancer (82.46%). Stage IIIa was assigned to 68.72% of patients, 89.57% received anti-PD-1 immunotherapy, 70.61% underwent two treatment cycles, and 45.50% of cases experienced radiological PR (Table 1). The pathological response was MPR or no-MPR in the discovery (46.46% or 36.90%) and validation (53.54% or 63.10%) sets, respectively. No variables significantly differed between the two cohorts (P > 0.05).

TABLE 1

Table 1 Characteristics of patients in the discovery and validation sets.

By analyzing the association between clinical data and MPR in the discovery and validation sets, we found that old age (> 60 y) was significantly correlated with MPR (P = 0.023) (Supplementary Figure 2A), patients with squamous cancer presented with a higher MPR than patients with adenocarcinoma (P = 0.025) (Supplementary Figure 2B), and patients who underwent an anti-PD-L1 regimen showed a higher MPR than those who underwent an anti-PD-1 regimen (P = 0.023) (Supplementary Figure 2C). Moreover, patients with radiological CR and PR showed a higher MPR than patients with radiological SD or PD (both P < 0.001) (Supplementary Figure 2D). The detailed results of the two cohorts are presented in Supplementary Table 2.

Most Immune Cells Are Similar Between the Discovery and Validation Sets

The results of the discovery set were similar with those of the validation set (Figures 2A, B). The relative abundance of CD4⁺CD45RA^- and CD3⁺CD4⁺ T cells showed a positive correlation (r = 0.621, P < 0.001; r = 0.534, P < 0.001), whereas that of CD3^-CD19⁺ B cells and CD3^-CD56⁺ NK cells showed a negative correlation (r = -0.373, P < 0.001; r = -0.390, P < 0.001) (Figures 2C, D).

FIGURE 2

Figure 2 Correlation between immune cell subtypes in the discovery and validation sets. Correlation heatmap depicts the relationship between each immune cell subtype in the discovery (A) and validation sets (B), respectively; Relationship between CD4⁺CD45RA^- T cells and CD3⁺CD4⁺ T cells/CD4⁺CD45RA^- T cells and CD3⁺CD4⁺ T cells in the discovery (C) and validation (D) cohorts.

Five Immune Cells Were Selected and Significantly Associated With MPR

To find suitable predictors for MPR, the immune cells before neoadjuvant therapy were identified, and then RFE and LASSO were used to perform feature selection. Based on 5-fold CV analysis, root mean square error (RMSE) showed that a combination of six variables had the smallest errors, and thus these immune cells were selected (Figure 3A). Then, LASSO coefficient analysis of 12 features of immune cells was performed, after which eight coefficients were chosen based on a 5-fold CV analysis of minimum criteria (Figure 3B). To find robust features, we selected five immune cell types with overlapping features for the SVM model (Figure 3C). CD3⁺CD56⁺ NKT cells were found significantly associated with MPR, and can thus be a predictor for MPR (P < 0.001, Figure 3D). In the discovery set, the percentage of CD3^-CD19⁺ B cells was higher in the no-MPR group (P = 0.011, Figure 3D), whereas that of CD3^-CD56⁺ NK cells was higher in the MPR group (P = 0.032). Moreover, patients in the MPR group had a higher percentage of CD4⁺CD45RA^- T cells than those in the no-MPR group (P = 0.017). The percentage of CD4⁺CD45RA⁺ T cells was higher in the no-MPR group than in the MPR group.

FIGURE 3

Figure 3 Immune cell selection and correlation with MPR. RFE (A) and LASSO (B) algorithms were used to perform optimal feature selection; Overlapping features were selected using two algorithms (C); Percentages of immune cells were compared between the MPR and no-MPR groups in patients with NSCLC from the discovery and validation sets treated with CAPD (D). MPR, major pathological response; RFE, recursive feature elimination; LASSO, least absolute shrinkage and selection operator; NSCLC, non-small cell lung cancer; CAPD, chemotherapy plus anti-PD-1/PD-L1.

LIP-SVM Was Built and Validated in Another Independent Cohort

To develop the SVM model, fine-tuning was performed during the training process. LIP-SVM was calculated as the output score of machine learning in the discovery and validation sets. We found that patients in the MPR group showed significantly higher LIP-SVM scores than patients in the no-MPR group in both cohorts (both P < 0.001; Figures 4A, B). To compare the accuracy of different models or predictors in our study, we first used three meaningful clinical features, including age (≤ 60 vs. > 60 years), histology (squamous vs. adenocarcinoma), and immunotherapy (anti-PD-1 vs. anti-PD-L1), to build a clinical model using logistic regression [area under the curve (AUC) = 0.663, 95% CI: 0.568–0.756, P = 0.001; AUC = 0.680, 95% CI: 0.565–0.796, P = 0.005, respectively; Figures 4C, D]. Moreover, the radiological response (CR, PR, SD, and PD) was also a strong indicator for predicting MPR of the two sets (AUC = 0.840, 95% CI: 0.772–0.908, P < 0.001; AUC = 0.820, 95% CI: 0.727–0.913, P < 0.001, respectively; Figures 4C, D). Comparing the above two predictors, the LIP-SVM signature showed the higher accuracy in the discovery and validation sets (AUC = 0.886, 95% CI: 0.823–0.949, P < 0.001; AUC = 0.874, 95% CI: 0.791–0.958, P < 0.001, respectively; Figures 4C, D). To better understand the utility of LIP-SVM, pathological images from four patients in validation cohorts are presented. After 2–4 cycles of neoadjuvant CAPD before resection of lung cancer, patients 1 and 2 with MPR had lower LIP-SVM scores than patients 3 and 4 with no-MPR (0.379 and -0.049 vs. 0.936 and 1.049; Figure 4E). The model is now available for free online testing (https://pengjie.shinyapps.io/lipsvm/).

FIGURE 4

Figure 4 LIP-SVM score was compared, and signatures were predicted for MPR using the two sets. LIP-SVM score comparisons between MPR and no-MPR groups of patients with NSCLC in the discovery (A) and validation sets (B). The area under the ROC curves of LIP-SVM signature, clinical model, and radiological response were plotted and compared for the discovery (C) and validation (D) sets; LIP-SVM score for each patient with NSCLC treated with CAPD (E); The clinical model included age, histology, and immunotherapy. The area under the ROC curves of the LIP-SVMRC model, LIP-SVM signature combined with the clinical model, and LIP-SVM signature combined with radiological response were plotted for the discovery (F) and validation (G) sets. LIP-SVM, support vector machine model based on immune cell profiling; MPR, major pathological response. LIP-SVM, support vector machine model based on immune cell profiling; ROC, receiver operating characteristic; NSCLC, non-small cell lung cancer; CAPD, chemotherapy plus anti-PD-1/PD-L1; LIP-SVMRC, LIP-SVM signature plus radiological response plus clinical model.

LIP-SVM Was an Independent Indicator of MPR in Patients With NSCLC

Based on the multivariate logistic regression analysis in the discovery set, age, radiological response, and LIP-SVM signature were independent risk factors for MPR [P < 0.016, P < 0.001, and P < 0.001; OR = 0.169 (0.034–0.649), 7.318 (3.147–20.588), and 29.788 (1.572–56.574), respectively; Table 2]. Age, radiological response, and LIP-SVM signature were also independent risk factors for MPR in the validation set [P = 0.046, P = 0.002, and P < 0.001; OR = 0.162 (0.019–0.990), 15.352 (3.487–27.805), and 28.186 (1.096–58.460), respectively]. To further improve the predictive accuracy, we integrated a combination of clinical factors or radiological response for multivariate analysis; we found that the LIP-SVMRC (LIP-SVM signature + radiological response + clinical model) model had the highest accuracy in the discovery and validation sets (AUC = 0.951, 95% CI: 0.916–0.986, P < 0.001; AUC = 0.943, 95% CI: 0.912–0.993, P < 0.001, respectively; Figures 4F, G). Two models (LIP-SVM signature + radiological response and LIP-SVM signature + clinical model) also exhibited high predictive accuracy for MPR in the discovery (AUC = 0.935, 95% CI: 0.895–0.975, P < 0.001; AUC = 0.907, 95% CI: 0.855–0.959, P < 0.001, respectively) and validation (AUC = 0.931, 95% CI: 0.899–0.994, P < 0.001; AUC = 0.905, 95% CI: 0.837–0.934, P < 0.001, respectively; Figures 4F, G) sets.

TABLE 2

Table 2 Multivariate analysis for MPR in the discovery and validation sets.

Discussion

In this prospective study, we selected five immune cell subtypes in peripheral blood and found that patients with three favorable immune cells (CD3⁺CD56⁺ NKT, CD3^-CD56⁺ NK, and CD4⁺CD45RA^- T cells) had a high MPR. Based on the SVM algorithm, the LIP-SVM signature was developed and can be used to predict the MPR of CAPD treatment. Multivariate analysis for MPR in the discovery and validation sets revealed that old age, radiological response, and LIP-SVM signature were positive independent factors of MPR. Combined with clinical factors, radiological response, and LIP-SVM, the LIP-SVMRC model exhibited the highest accuracy for predicting MPR compared with the other two models. These findings indicate that LIP-SVMRC can be used as a novel tool for the effective identification of patients that may acquire MPR from CAPD.

Several studies have reported that the combination of chemotherapy and anti-PD-1/PD-L1 treatment for metastatic NSCLC as a first-line therapy significantly improves the treatment response, OS, and PFS of patients (24–27). However, some studies have reported that neoadjuvant combination immunotherapy, especially CAPD (4, 28), makes it challenging to identify significant biomarkers for predicting MPR in patients with NSCLC undergoing CAPD. In our study, we found that old (> 60 years) patients with squamous cancer had higher positive CAPD response, indicating that the combination of platinum-based double-chemotherapy (nab-paclitaxel/pemetrexed) and immunotherapy was more suitable for these patients. Although the sample size of patients on anti-PD-L1 was small (13 and 9 patients in the discovery and validation sets, respectively), the combination anti-PD-L1 regimen showed a higher MPR than the combination anti-PD-1 as a neoadjuvant treatment for patients with NSCLC. This may be because anti-PD-L1 treatment affects both the tumor microenvironment [e.g., T and B cells, dendritic cells (DCs), and macrophages] and the tumor itself, which frequently express PD-L1 (29, 30). In the evaluation of treatment response, we found that there was a partial discrepancy between radiological and pathological methods; nevertheless, radiological evaluation of neoadjuvant treatment response can aid preoperative prediction of MPR.

In addition to clinical factors and radiological response, detailed knowledge of the patient’s immune status of peripheral blood is needed to evaluate the efficacy of combination anti-PD-1/PD-L1 treatment, and tumor immunogenicity score is evaluated as a predictor of immune checkpoint inhibitor response (31, 32). Previous studies have reported that specific PD-1⁺CD56⁺ and CD8⁺ T cells frequently indicate a good prognosis in patients with melanoma treated with immunotherapy (33, 34). However, the immunological biomarkers for predicting MPR with CAPD as a first-line neoadjuvant remain unclear. The main reason for this is the paucity of reported data on neoadjuvant immunotherapy. In our prospective study, we revealed five immune cell subtypes based on liquid immune profiling for predicting the MPR of patients with stage Ib-IIIa NSCLC treated with CAPD. NKT cells have been reported to play a critical role in inducing cross-talk of plasmacytoid DCs with conventional DCs, which is associated with the generation of memory CD8⁺ T cells (35). A recent study reported that elevated peripheral NK cell numbers in patients with NSCLC are associated with responses (CR/PR) to immunotherapy (36). Our study revealed the positive correlations between CD3⁺CD56⁺ NKT or CD3^-CD56⁺ NK cells and the MPR to CAPD. NKT/NK cells may play important roles in antitumor immunity, such as reactivation of fatigued immune cells derived from the tumor microenvironment.

Although immunotherapy can unleash CD8⁺ T cells and specific mutation-associated neoantigens, some tumor microenvironment factors (such as hypoxia and toxic metabolites) inhibit T cell activation (37, 38). A recent study showed that PD-1⁺CD8⁺ T cell-positive tumors are significantly associated with poor response to anti-PD-1 therapy (39). An increase in circulating PD-1⁺CD8⁺ T cell numbers in the early stage of immunotherapy is also an indicator of poor prognosis in advanced cancers (5). In our study, we found that active CD8⁺CD38⁺ T cells of the peripheral blood were not associated with MPR in patients undergoing neoadjuvant CAPD, indicating that a subtype of CD8⁺ T cells was suppressed or exhausted. In a recent study, the change of abundance in CD4⁺CD45RA⁺ T cells was a predictive biomarker for PFS after chemoradiotherapy (40). Interestingly, CD4⁺CD45RA^- T cells were positively and CD4⁺CD45RA⁺ T cells were negatively correlated with MPR in our study. These results suggest that a subtype of CD4⁺ T cells plays a crucial role in determining immunotherapy response in the initial stage of CAPD treatment. Moreover, circulating CD3^-CD19⁺ B cell counts are increased in patients with oral squamous cell carcinoma after radical operation or chemotherapy (41), but their value in predicting treatment response and prognosis is unclear. In our study, we found that patients with neoadjuvant CAPD and acquired MPR exhibited a lower percentage of CD3^-CD19⁺ B cells than patients without MPR. This is the first report of an association between CD3^-CD19⁺ B cells and CAPD, revealing their potentially negative role in cancer treatment response or prognosis.

According to RECIST V.1.1, most patients (68.25%) acquired an objective response (CR/PR) to combination treatment, but this radiological method had a high specificity and low sensitivity in the two sets (specificity = 96.67 and 93.55; sensitivity = 58.21 and 45.28, respectively), which is not accurate enough to preoperatively predict pathological response. To precisely screen the patients for MPR, a machine learning method (integrating RFE, LASSO, and SVM algorithms) based on immune cell profiling was performed in this study. After fine-tuning the parameters, the LIP-SVM model exhibited higher predictive accuracy than the clinical model and evaluation of radiological response. Moreover, the LIP-SVM model showed an earlier prediction of MPR before initial CAPD than radiological estimation. Multivariate analysis for MPR also revealed that the LIP-SVM signature was an independent factor in both cohorts. Several studies have integrated clinical factors and series models to improve the accuracy and robustness capability (42–44). In our study, the AUC of the LIP-SVMRC model, which integrates three factors (immune cells, radiological evaluation, and clinical variables), was high in all models, indicating a greatly improved predictive accuracy. This preoperative prediction model of MPR may be helpful for guiding radical surgery and personalizing a treatment regimen for each patient.

Our study had two limitations. First, because of the high cost of analysis, targeted next-generation sequencing or whole-exome sequencing results were not analyzed in all samples from patients with NSCLC, which led to a small percentage of patients with test results that could not be analyzed. Second, our sample size was not large and multi-center prospective cohorts are required.

In conclusion, our study revealed the significant association between clinical variables (old age, squamous cancer, and anti-PD-L1 treatment), radiological response, and immune cells from peripheral blood and MPR in patients with NSCLC receiving 2–4 cycles of neoadjuvant CAPD. The classifications of the SVM model based on immune cell profiling and integration models provide a novel and noninvasive predictive method for identifying patients who may achieve MPR after CAPD.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by the Second Affiliated Hospital of Guizhou Medical University and the Cancer Hospital of the University of Chinese Academy of Sciences. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

Conception and design: JP. Administrative support: JP. Provision of study materials or patients: JP and DZ. Collection and assembly of data: JP. Data analysis and interpretation: JP. Manuscript writing: All authors. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by the Qian Dong Nan Science and Technology Program [qdnkhJz2020-013] and the Science and Technology Foundation of Guizhou Province [Grant No. Qian ke he ji chu-ZK 2021, yi ban 454], the Science and Technology Fund Project of Guizhou Provincial Health Commission [gzwjkj2019-1-077].

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We thank the Cancer Hospital of the University of Chinese Academy of Sciences (CHUCAS) and the Second Affiliated Hospital of Guizhou Medical University (SAHGMU) for their support.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fimmu.2021.778276/full#supplementary-material

References

1. West H, McCleod M, Hussein M, Morabito A, Rittmeyer A, Conter HJ, et al. Atezolizumab in Combination With Carboplatin Plus Nab-Paclitaxel Chemotherapy Compared With Chemotherapy Alone as First-Line Treatment for Metastatic Non-Squamous Non-Small-Cell Lung Cancer (Impower130): A Multicentre, Randomised, Open-Label, Phase 3 Trial. Lancet Oncol (2019) 20(7):924–37. doi: 10.1016/S1470-2045(19)30167-6

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Rodríguez-Abreu D, Powell SF, Hochmair MJ, Gadgeel S, Esteban E, Felip E, et al. Pemetrexed Plus Platinum With or Without Pembrolizumab in Patients With Previously Untreated Metastatic Nonsquamous NSCLC: Protocol-Specified Final Analysis From KEYNOTE-189. Ann Oncol (2021) 32(7):881–95. doi: 10.1016/j.annonc.2021.04.008

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Paz-Ares L, Vicente D, Tafreshi A, Robinson A, Soto Parra H, Mazières J, et al. A Randomized, Placebo-Controlled Trial of Pembrolizumab Plus Chemotherapy in Patients With Metastatic Squamous NSCLC: Protocol-Specified Final Analysis of KEYNOTE-407. J Thorac Oncol (2020) 15(10):1657–69. doi: 10.1016/j.jtho.2020.06.015

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Rothschild SA-O, Zippelius A, Eboulet EI, Savic Prince S, Betticher DA-O, Bettini A, et al. SAKK 16/14: Durvalumab in Addition to Neoadjuvant Chemotherapy in Patients With Stage IIIA(N2) Non-Small-Cell Lung Cancer-A Multicenter Single-Arm Phase II Trial. J Clin Oncol (2021) 39(26):2872–80. doi: 10.1200/JCO.21.00276

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Zhou JA-O, Donaubauer AJ, Frey BA-O, Becker I, Rutzner S, Eckstein MA-O, et al. Prospective Development and Validation of a Liquid Immune Profile-Based Signature (LIPS) to Predict Response of Patients With Recurrent/Metastatic Cancer to Immune Checkpoint Inhibitors. J Immunother Cancer (2021) 9(2):e001845. doi: 10.1136/jitc-2020-001845

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Ott PA, Bang YJ, Piha-Paul SA, Razak ARA, Bennouna J, Soria JC, et al. T-Cell-Inflamed Gene-Expression Profile, Programmed Death Ligand 1 Expression, and Tumor Mutational Burden Predict Efficacy in Patients Treated With Pembrolizumab Across 20 Cancers: KEYNOTE-028. J Clin Oncol (2019) 37(4):318–27. doi: 10.1200/JCO.2018.78.2276

PubMed Abstract | CrossRef Full Text | Google Scholar

7. House IG, Savas P, Lai J, Chen AXY, Oliver AJ, Teo ZL, et al. Macrophage-Derived CXCL9 and CXCL10 Are Required for Antitumor Immune Responses Following Immune Checkpoint Blockade. Clin Cancer Res (2020) 26(2):487–504. doi: 10.1158/1078-0432.CCR-19-1868

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Gide TN, Quek C, Menzies AM, Tasker AT, Shang P, Holst J, et al. Distinct Immune Cell Populations Define Response to Anti-PD-1 Monotherapy and Anti-PD-1/Anti-CTLA-4 Combined Therapy. Cancer Cell (2019) 35(2):238–55. doi: 10.1016/j.ccell.2019.01.003

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Ayers M, Lunceford J, Nebozhyn M, Murphy E, Loboda A, Kaufman DR, et al. IFN-Gamma-Related Mrna Profile Predicts Clinical Response to PD-1 Blockade. J Clin Invest (2017) 127(8):2930–40. doi: 10.1172/JCI91190

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Huang AC, Postow MA, Orlowski RJ, Mick R, Bengsch B, Manne S, et al. T-Cell Invigoration to Tumour Burden Ratio Associated With Anti-PD-1 Response. Nature (2017) 545(7652):60–5. doi: 10.1038/nature22079

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Kim M, Min YK, Jang J, Park H, Lee S, Lee CA-O. Single-Cell RNA Sequencing Reveals Distinct Cellular Factors for Response to Immunotherapy Targeting CD73 and PD-1 in Colorectal Cancer. J Immunother Cancer (2021) 9(7):e002503. doi: 10.1136/jitc-2021-002503

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Kim KH, Cho J, Ku BM, Koh J, Sun JM, Lee SH, et al. The First-Week Proliferative Response of Peripheral Blood PD-1(+)CD8(+) T Cells Predicts the Response to Anti-PD-1 Therapy in Solid Tumors. Clin Cancer Res (2019) 25(7):2144–54. doi: 10.1158/1078-0432.CCR-18-1449

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Krieg CA-O, Nowicka M, Guglietta S, Schindler S, Hartmann FA-O, Weber LA-O, et al. High-Dimensional Single-Cell Analysis Predicts Response to Anti-PD-1 Immunotherapy. Nat Med (2018) 24(2):144–53. doi: 10.1038/nm.4466

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Peng J, Kang S, Ning Z, Deng H, Shen J, Xu Y, et al. Residual Convolutional Neural Network for Predicting Response of Transarterial Chemoembolization in Hepatocellular Carcinoma From CT Imaging. Eur Radiol (2020) 30(1):413–24. doi: 10.1007/s00330-019-06318-1

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Hess JA-O, Unger K, Maihoefer CA-O, Schüttrumpf L, Wintergerst L, Heider T, et al. A Five-Microrna Signature Predicts Survival and Disease Control of Patients With Head and Neck Cancer Negative for HPV Infection. Clin Cancer Res (2019) 25(5):1505–16. doi: 10.1158/1078-0432.CCR-18-0776

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, et al. Dermatologist-Level Classification of Skin Cancer With Deep Neural Networks. Nature (2017) 542(7639):115–8. doi: 10.1038/nature21056

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Chaudhary K, Poirion OB, Lu L, Garmire LX. Deep Learning-Based Multi-Omics Integration Robustly Predicts Survival in Liver Cancer. Clin Cancer Res (2018) 24(6):1248–59. doi: 10.1158/1078-0432.CCR-17-0853

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Johannet PA-O, Coudray NA-O, Donnelly DM, Jour G, Illa-Bochaca I, Xia Y, et al. Using Machine Learning Algorithms to Predict Immunotherapy Response in Patients With Advanced Melanoma. Clin Cancer Res (2021) 27(1):131–40. doi: 10.1158/1078-0432.CCR-20-2415

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Gupta AA-O, Arora PA-O, Brenner DA-OX, Vanderpuye-Orgle J, Boyne DA-O, Edmondson-Jones M, et al. Risk Prediction Using Bayesian Networks: An Immunotherapy Case Study in Patients With Metastatic Renal Cell Carcinoma. JCO Clin Cancer Inform (2021) 5:326–37. doi: 10.1200/CCI.20.00107

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Sun R, Limkin EJ, Vakalopoulou M, Dercle L, Champiat S, Han SR, et al. A Radiomics Approach to Assess Tumour-Infiltrating CD8 Cells and Response to Anti-PD-1 or Anti-PD-L1 Immunotherapy: An Imaging Biomarker, Retrospective Multicohort Study. Lancet Oncol (2018) 19(9):1180–91. doi: 10.1016/S1470-2045(18)30413-3

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Arbour KA-O, Luu AT, Luo JA-OX, Rizvi H, Plodkowski AA-OX, Sakhi MA-O, et al. Deep Learning to Estimate RECIST in Patients With NSCLC Treated With PD-1 Blockade. Cancer Discov (2021) 11(1):59–67. doi: 10.1158/2159-8290.CD-20-0419

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Peng J, Zou D, Gong W, Kang S, Han L. Deep Neural Network Classification Based on Somatic Mutations Potentially Predicts Clinical Benefit of Immune Checkpoint Blockade in Lung Adenocarcinoma. Oncoimmunology (2020) 9(1):1734156. doi: 10.1080/2162402X.2020.1734156

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Jiang Y, Xie J, Han Z, Liu W, Xi S, Huang L, et al. Immunomarker Support Vector Machine Classifier for Prediction of Gastric Cancer Survival and Adjuvant Chemotherapeutic Benefit. Clin Cancer Res (2018) 24(22):5574–84. doi: 10.1158/1078-0432.CCR-18-0848

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Yang Y, Wang Z, Fang J, Yu Q, Han B, Cang S, et al. Efficacy and Safety of Sintilimab Plus Pemetrexed and Platinum as First-Line Treatment for Locally Advanced or Metastatic Nonsquamous NSCLC: A Randomized, Double-Blind, Phase 3 Study (Oncology Program by Innovent Anti-PD-1-11). J Thorac Oncol (2020) 15(10):1636–46. doi: 10.1016/j.jtho.2020.07.014

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Wang Z, Zhao J, Ma Z, Cui J, Shu Y, Liu Z, et al. A Phase 2 Study of Tislelizumab in Combination With Platinum-Based Chemotherapy as First-Line Treatment for Advanced Lung Cancer in Chinese Patients. Lung Cancer (2020) 147:259–68. doi: 10.1016/j.lungcan.2020.06.007

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Powell SF, Rodríguez-Abreu D, Langer CJ, Tafreshi A, Paz-Ares L, Kopp HG, et al. Outcomes With Pembrolizumab Plus Platinum-Based Chemotherapy for Patients With Non-Small-Cell Lung Cancer and Stable Brain Metastases: Pooled Analysis of KEYNOTE-021, 189, and 407. J Thorac Oncol (2021) 16(11):1883–92. doi: 10.1016/j.jtho.2021.06.020

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Awad MM, Gadgeel SM, Borghaei H, Patnaik A, Yang JC, Powell SF, et al. Long-Term Overall Survival From KEYNOTE-021 Cohort G: Pemetrexed and Carboplatin With or Without Pembrolizumab as First-Line Therapy for Advanced Nonsquamous NSCLC. J Thorac Oncol (2021) 16(1):162–8. doi: 10.1016/j.jtho.2020.09.015

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Provencio M, Nadal E, Insa A, García-Campelo MR, Casal-Rubio J, Dómine M, et al. Neoadjuvant Chemotherapy and Nivolumab in Resectable Non-Small-Cell Lung Cancer (NADIM): An Open-Label, Multicentre, Single-Arm, Phase 2 Trial. Lancet Oncol (2020) 21(11):1413–22. doi: 10.1016/S1470-2045(20)30453-8

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Yoh K, Matsumoto S, Furuya N, Nishino K, Miyamoto S, Oizumi S, et al. Comprehensive Assessment of PD-L1 Expression, Tumor Mutational Burden and Oncogenic Driver Alterations in Non-Small Cell Lung Cancer Patients Treated With Immune Checkpoint Inhibitors. Lung Cancer (2021) 159:128–34. doi: 10.1016/j.lungcan.2021.07.015

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Švajger U, Tešić N, Rožman P. Programmed Death Ligand 1 (PD-L1) Plays a Vital Part in DC Tolerogenicity Induced by IFN-γ. Int Immunopharmacol (2021) 99:107978. doi: 10.1016/j.intimp.2021.107978

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Zeng DA-O, Li M, Zhou R, Zhang J, Sun H, Shi M, et al. Tumor Microenvironment Characterization in Gastric Cancer Identifies Prognostic and Immunotherapeutically Relevant Gene Signatures. Cancer Immunol Res (2019) 7(5):737–50. doi: 10.1158/2326-6066.CIR-18-0436

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Charoentong P, Finotello F, Angelova M, Mayer C, Efremova M, Rieder D, et al. Pan-Cancer Immunogenomic Analyses Reveal Genotype-Immunophenotype Relationships and Predictors of Response to Checkpoint Blockade. Cell Rep (2017) 18(1):248–62. doi: 10.1016/j.celrep.2016.12.019

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Martens A, Wistuba-Hamprecht K, Yuan J, Postow MA, Wong P, Capone M, et al. Increases in Absolute Lymphocytes and Circulating CD4+ and CD8+ T Cells Are Associated With Positive Clinical Outcome of Melanoma Patients Treated With Ipilimumab. Clin Cancer Res (2016) 22(19):4848–58. doi: 10.1158/1078-0432.CCR-16-0249

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Bochem J, Zelba H, Amaral TA-O, Spreuer J, Soffel D, Eigentler T, et al. Peripheral PD-1+CD56+ T-Cell Frequencies Correlate With Outcome in Stage IV Melanoma Under PD-1 Blockade. PloS One (2019) 14(8):e0221301. doi: 10.1371/journal.pone.0221301

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Shimizu K, Asakura M Fau - Shinga J, Shinga J Fau - Sato Y, Sato Y Fau - Kitahara S, Kitahara S Fau - Hoshino K, Hoshino K Fau - Kaisho T, et al. Invariant NKT Cells Induce Plasmacytoid Dendritic Cell (DC) Cross-Talk With Conventional Dcs for Efficient Memory CD8+ T Cell Induction. J Immunol (2013) 190(11):5609–19. doi: 10.4049/jimmunol.1300033

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Juliá EP, Mandó P, Rizzo MM, Cueto GR, Tsou F, Luca R, et al. Peripheral Changes in Immune Cell Populations and Soluble Mediators After Anti-PD-1 Therapy in Non-Small Cell Lung Cancer and Renal Cell Carcinoma Patients. Cancer Immunol Immunother (2019) 68(10):1585–96. doi: 10.1007/s00262-019-02391-z

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Leone RD, Powell JA-O. Metabolism of Immune Cells in Cancer. Nat Rev Cancer (2020) 20(9):516–31. doi: 10.1038/s41568-020-0273-y

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Caushi JX, Zhang J, Ji Z, Vaghasia A, Zhang B, Hsiue EH, et al. Transcriptional Programs of Neoantigen-Specific TIL in Anti-PD-1-Treated Lung Cancers. Nature (2021) 596(7870):126–32. doi: 10.1038/s41586-021-03752-4

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Thommen DA-O, Koelzer VH, Herzig P, Roller A, Trefny MA-O, Dimeloe S, et al. A Transcriptionally and Functionally Distinct PD-1(+) CD8(+) T Cell Pool With Predictive Potential in Non-Small-Cell Lung Cancer Treated With PD-1 Blockade. Nat Med (2018) 24(7):994–1004. doi: 10.1038/s41591-018-0057-z

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Chen Y, Jin Y, Hu X, Chen MA-O. Effect of Chemoradiotherapy on the Proportion of Circulating Lymphocyte Subsets in Patients With Limited-Stage Small Cell Lung Cancer. Cancer Immunol Immunother (2021) 70(10):2867–76. doi: 10.1007/s00262-021-02902-x

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Yu T, Guo P, Wu Y, Zhao J, Wu L, Li C, et al. The Role of Chemotherapy and Operation on Lymphocytes Accumulation in Peripheral Blood Obtained From Patients With Oral Squamous Cell Carcinoma. Springerplus (2015) 4:698. doi: 10.1186/s40064-015-1485-6

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Kandimalla RA-O, Tomihara HA-O, Banwait JK, Yamamura K, Singh G, Baba H, et al. A 15-Gene Immune, Stromal, and Proliferation Gene Signature That Significantly Associates With Poor Survival in Patients With Pancreatic Ductal Adenocarcinoma. Clin Cancer Res (2020) 26(14):3641–8. doi: 10.1158/1078-0432.CCR-19-4044

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Kandimalla R, Shimura T, Mallik S, Sonohara F, Tsai S, Evans DB, et al. Identification of Serum MiRNA Signature and Establishment of a Nomogram for Risk Stratification in Patients With Pancreatic Ductal Adenocarcinoma. Ann Surg (2020). doi: 10.1097/SLA.0000000000003945

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Huang YQ, Liang CH, He L, Tian J, Liang CS, Chen X, et al. Development and Validation of a Radiomics Nomogram for Preoperative Prediction of Lymph Node Metastasis in Colorectal Cancer. J Clin Oncol (2016) 34(18):2157–64. doi: 10.1200/JCO.2015.65.9128

PubMed Abstract | CrossRef Full Text | Google Scholar