Development and Validation of a Radiomics Nomogram Using Computed Tomography for Differentiating Immune Checkpoint Inhibitor-Related Pneumonitis From Radiation Pneumonitis for Patients With Non-Small Cell Lung Cancer

Background: The combination of immunotherapy and chemoradiotherapy has become the standard therapeutic strategy for patients with unresected locally advance-stage non-small cell lung cancer (NSCLC) and induced treatment-related adverse effects, particularly immune checkpoint inhibitor-related pneumonitis (CIP) and radiation pneumonitis (RP). The aim of this study is to differentiate between CIP and RP by pretreatment CT radiomics and clinical or radiological parameters.

Methods: A total of 126 advance-stage NSCLC patients with pneumonitis were enrolled in this retrospective study and divided into the training dataset (n =88) and the validation dataset (n = 38). A total of 837 radiomics features were extracted from regions of interest based on the lung parenchyma window of CT images. A radiomics signature was constructed on the basis of the predictive features by the least absolute shrinkage and selection operator. A logistic regression was applied to develop a radiomics nomogram. Receiver operating characteristics curve and area under the curve (AUC) were applied to evaluate the performance of pneumonitis etiology identification.

Results: There was no significant difference between the training and the validation datasets for any clinicopathological parameters in this study. The radiomics signature, named Rad-score, consisting of 11 selected radiomics features, has potential ability to differentiate between CIP and RP with the empirical and α-binormal-based AUCs of 0.891 and 0.896. These results were verified in the validation dataset with AUC = 0.901 and 0.874, respectively. The clinical and radiological parameters of bilateral changes (p < 0.001) and sharp border (p = 0.001) were associated with the identification of CIP and RP. The nomogram model showed good performance on discrimination in the training dataset (AUC = 0.953 and 0.950) and in the validation dataset (AUC = 0.947 and 0.936).

Conclusions: CT-based radiomics features have potential values for differentiating between patients with CIP and patients with RP. The addition of bilateral changes and sharp border produced superior model performance on classifying, which could be a useful method to improve related clinical decision-making.

Introduction

Immune checkpoint inhibitors (ICIs) have established a new paradigm for cancer therapeutic and created many breakthroughs in clinical practice. Consolidation immunotherapy of immune checkpoint inhibitors following concurrent chemoradiotherapy (CCRT) is the current standard of care for patients with unresectable locally advance-stage non-small cell lung cancer (NSCLC) (1, 2). While the combination of ICIs and radiotherapy (RT) has shown promising prospects, treatment-related pneumonitis, including checkpoint inhibitor-related pneumonitis (CIP) and radiation pneumonitis (RP) (3, 4), one of the most frequent and clinically challenging adverse events in the combination setting, should raise concerns.

CIP is a rare but seriously adverse event with incidence of approximately 5% in any grade and <3% in grade 3 or higher level for NSCLC patients who received ICIs (5). Meanwhile, approximately 60% of patients with thoracic tumors receive RT at some point during the course of the disease (6). RT contributed to lung injury and induced pneumonitis. The incidence of radiation pneumonitis ranges from 4 to 10% in grade 3 or higher in lung cancer patients after radiotherapy within 6 months (7). Differentiating a diagnosis between CIP and RP is a difficult challenge in clinical practice. The clinical symptoms of CIP seem to be non-specific as far as clinical manifestations are being concerned. Clinical symptoms, such as fever, cough, and shortness of breath, are found in both CIP and RP (8). Additionally, the thoracic CT radiological findings of CIP are similar to the RP model, especially in the early stage of the disease course. Although some studies have reported the typical radiological findings of CIP and RP, these manifestations are only suggestive due to pneumonitis having a wide range of radiological appearance (9). Furthermore, these patients are at risk for both ICI- and RT-induced pneumonitis, and a differentiating diagnosis can have an important effect on clinical management, such as the decision to continue or restart the ICI treatment.

Radiomics is inspired by the combination of artificial intelligence and medical imaging. High-throughput and quantitative images of features reflect the underlying pathophysiology and reveal information on pathogenesis and etiology (10). CT radiomics analysis, as an interdisciplinary technique, has been widely used in distinguishing between benign and malignant tumors (11), predicting the prognosis of patients with a tumor (12), monitoring therapeutic responses (13), and gene expression (14), yet scanty attention has been paid to investigating how to distinguish between CIP and RP using radiomics features for patients with NSCLC. Given the previous conclusions, whether CT radiomics can be used for the identification of CIP and RP becomes crucial and worth exploring. Theoretically, the characteristics of pneumonitis provided by CT hide the potential correlation with pneumonitis etiology and can be quantitatively analyzed. It is expected that the high-throughput and high-dimensional CT radiomics features play a vital role in distinguishing between CIP and RP. We hypothesized that constructing a model and developing a quantitative tool could improve the diagnostic efficiency for CIP and RP via analysis of CT radiomics features and clinical or radiological parameters.

Materials and Methods

Study Design and Workflow

Firstly, NSCLC patients who were treated with ICIs or RT and developed pneumonitis were enrolled in this study. Chest CT images were collected for subsequent radiomics analysis. After that, radiomics features were extracted from regions of interest (ROIs). Then, radiomics features were selected based on their effectiveness in differentiating the subtype of pneumonitis. Next, Rad-scores and nomogram were constructed and evaluated in the training and validation datasets.

Patient Selection

In this study, a total of 126 NSCLC patients who received radiotherapy or immune checkpoint inhibitors and developed pneumonitis between April 2018 and August 2021 at Shandong Provincial Hospital Affiliated to Shandong First Medical University were recruited. The inclusion criteria were as follows: (1) pathological diagnosis of NSCLC by biopsy or bronchofiberoscopy, (2) standard chest CT scans, and (3) collection of CT images and clinical or radiological parameters of patients who developed pneumonitis. The exclusion criteria were as follows: (1) insufficient image quality, such as artifacts, (2) with treatment history of thoracic surgery, and (3) with the history of anti-tumor or anti-inflammatory therapy. In this study, pneumonitis was defined as immune checkpoint inhibitor-related and radiation pneumonitis which did not occur owing to other confirmed reasons such as bacterial and virus infections. The subtype of pneumonitis was determined by the following procedures. Patients with CIP were identified by a history of using immune checkpoint inhibitors, nonproductive cough, fever, and other clinical symptoms. Varied radiographic findings, such as from chest CT imaging, should be considered, such as cryptogenic organizing pneumonia, pneumonitis presenting as acute interstitial pneumonia, acute respiratory distress syndrome, etc. The RP datasets that consisted of 69 patients were randomly enrolled from NSCLC patients who were treated with thoracic radiotherapy and developed RP within 6 months after RT. The symptoms include shortness of breath, nonproductive cough, fever, and other clinical symptoms. The definitions of CIP and RP were consistent with the previous guidelines (15, 16). Clinical factors, including gender, age, smoking history, histology, and radiological findings—such as number of lobes, volume of lung, bilateral changes, and sharp border—were recorded.

CT Image Acquisition

CT images were collected from all enrolled patients. All CT scans were acquired from a 128-row CT scanner (Philips iCT 128, Philips Medical System, The Netherlands). The CT scans were acquired with the following protocols: tube voltage, 120 kV; tube current, ranging from 300 to 400 mA; slice thickness, 3 mm; matrix size, 512 × 512; in-plane resolution, 0.8142 × 0.8142 mm²; and helical scanning mode.

Region of Interest

The ROIs, defined as the lung injury region visualized on the lung parenchyma window of CT images, were delineated by two experienced radiologist and oncologist in all CT scans using AccuContour software (version 3.0, Manteia Medical Technologies Co. Ltd., Xiamen, China). Given that a larger variability existed in the border of the pneumonitis region, cylindrical ROIs of diameter 20 mm and height 15 mm (consecutive five slices) were contoured to ensure that the features are valuable and correct. Then, the contoured ROIs were checked and modified slice by slice by another experienced radiologist. Examples of contoured ROIs on the lung parenchyma window of CT images are depicted in Figure 1.

FIGURE 1

Figure 1 Examples of regions of interest for two subtypes of pneumonitis. (A) Checkpoint inhibitor-related pneumonitis on routine CT and (B) radiation pneumonitis on routine CT.

Feature Extraction

Radiomics features were extracted using embedded radiomics computational module-based PyRadiomics packages that enable feature calculation in the 3D slicer (version 4.11.2, www.slicer.org) software. In this study, 93 radiomics features were extracted from original CT images, including (1) 18 first-order intensity histogram (IH)-based features and statistical matrix (SM)-based features divided into (2) 24 gray-level co-occurrence matrix-based features, (3) 16 gray-level run-length matrix-based features, (4) 16 gray-level size zone matrix-based features, (5) 5 neighboring gray-tone difference matrices, and (6) 14 gray-level dependence matrix features. Moreover, 744 wavelet-based features (including IH and SM features) were extracted from eight wavelet decompositions.

Radiomics Signature and Nomogram Construction

Before radiomics signature building, feature selection was implemented to keep the signature more robust and effective. In this study, least absolute shrinkage and selection operator (LASSO) (with a binary regression model, a five-fold cross-validation method, an “auc” loss measurement, and using non-normalized data) was performed to determine the most predictive features. After feature selection, a radiomics signature, also termed Rad-score, was established from a linear combination of selected features and corresponding coefficients derived from LASSO.

Moreover, to explore whether clinical or radiological parameters will add more benefit for differentiating subtypes of pneumonitis, nomograms were constructed by incorporating Rad-score and clinical factors compared to Rad-score alone. Notably, the clinical factors used for nomogram establishment were tested via univariate analysis.

Validation of Radiomics Signature and Nomogram

The correlation between the Rad-score and subtype of pneumonitis was evaluated using the receiver operating characteristic curve (ROC) and area under the curve (AUC). Due to the limited sample size of patients, inevitable bias exists in the uneven appearance of the empirical ROC, which will result in a lower or higher estimated performance. Therefore, ROC curves may perform poorly for evaluation despite a superior AUC when the positive and negative data used for building a prediction model are imbalanced. Therefore, a precision–recall curve (PRC) that plots the positive prediction value against the true positive rate across all thresholds was recommendable for such case. It represents a more accurate method to assess established classification models, and the area under PRC is defined as average precision (AP). Consequently, the α-binomial model-based ROC curve and PRC proposed by Brodersen et al. to plot smooth curves were used to address the above-mentioned issues in this study (17). The discrimination of nomogram was evaluated by α-binomial model-based ROC curve and PRC. Meanwhile, the agreement between the actual CIP probability and predicted CIP probability was assessed by a calibration curve, and Hosmer–Lemeshow test was utilized to determine the agreement; a p-value >0.05 indicates good agreement. Finally, the Rad-score and the nomogram were compared using decision curve analysis with regards to clinical utility.

Statistical Analysis

Statistical analyses were performed in R software (version 3.3.1). Comparisons, calibration curve, decision curve analysis, and univariate analyses were implemented in R with the “stats” and “rms” packages. Chi-square test or Fisher’s test was used to analyze the categorical variables. Mann–Whitney U-tests were employed to compare the patients’ continuous characteristics where appropriate. In this study, univariate analysis was performed by Spearman’s correlation test, and a coefficient higher than 0.85 indicates that the clinical factors are correlated to the subtype of pneumonitis. LASSO was performed in R with the glmnet package. The reported statistical significance levels were all two-sided. The statistical significance level was set to 0.05.

Results

Patient Characteristics

In total, 126 consecutive patients were enrolled in this study and were divided into training and validation datasets with a ratio of 7:3, including 88 and 38 patients, respectively. The clinical and radiological factors are summarized in Table 1. There were no significant differences between these factors in the two sets, including gender, age, and radiological findings.

TABLE 1

Table 1 Clinical and radiological parameters of patients with pneumonitis in the training and validation datasets.

Radiomics Signature Construction and Validation

After feature selection using LASSO binary regression model, 11 radiomics features remained with non-zero coefficients, and the results are illustrated in Figure 2. A Rad-score was established using a linear combination of selected predictive features and corresponding coefficients. The formula for calculating the Rad score is as follows:

$$\begin{array}{l}\text{Rad}-\text{score}=\text{wavelet}.\text{LLH}.\text{ngtdm}.\text{Coarseness}\times 4.901552\\ +\text{wavelet}.\text{LHL}.\text{glcm}.\text{ClusterShade}\times 0.001658\\ +\text{wavelet}.\text{LHL}.\text{glrlm}.\text{LRHGLE}\times 0.00095\\ +\text{wavelet}.\text{LHL}.\text{glszm}.\text{ZoneVariance}\times 1.87\text{E}-06\\ +\text{wavelet}.\text{LHH}.\text{firstorder}.\text{Median}\times 0.104804\\ -\text{wavelet}.\text{HLL}.\text{firstorder}.\text{Kurtosis}\times 0.03584\\ +\text{wavelet}.\text{HLL}.\text{firstorder}.\text{Skewness}\times 0.53657\\ -\text{wavelet}.\text{HHL}.\text{firstorder}.\text{Mean}\times 1.27905\\ -\text{wavelet}.\text{HHL}.\text{firstorder}.\text{Skewness}\times 0.73588\\ -\text{wavelet}.\text{HHH}.\text{gldm}.\text{DependenceEntropy}\times 0.20179\\ +\text{wavelet}.\text{LLL}.\text{firstorder}.\text{Minimum}\times 1.03\text{E}-05\end{array}$$

FIGURE 2

Figure 2 Feature selection using the least absolute shrinkage and selection operator (LASSO) with a binary regression model. (A) The LASSO coefficient profile was plotted using coefficients against log(λ). (B) Tuning parameter against parameter log(λ). The areas under the curve were depicted with corresponding λ. The vertical lines are maximum and 1-standard error (1-se) criteria, respectively. As a result, 12 radiomics features with nonzero coefficients were selected using 1-se criteria.

We plotted box plots to show the statistical distribution of Rad-scores in the training and validation datasets, as shown in Figure 3. There were statistically significant differences between the training and validation datasets (all p-values <0.0001). The empirical and α-binormal-based ROC curves and PRCs of the established Rad-score are displayed in Figure 4. The empirical and α-binormal AUCs and APs of Rad-score are summarized in Table 2. The Rad-score achieved a good performance for making a distinction between CIP and RP in NSCLC patients with AUC_αbin = 0.891 (95%CI, 0.876–0.906) and AUC_emp = 0.896 (95%CI, 0.879–0.913), respectively. Similar results were committed in the validation dataset; the above-mentioned Rad-score showed a favorable assessment efficacy with AUC_αbin = 0.901 (95%CI, 0.855–0.947) and AUC_emp = 0.874 (95%CI, 0.843–0.905) for discriminating between patients with CIP and RP. The results of Rad-score had achieved a satisfactory performance.

FIGURE 3

Figure 3 Boxplots for subtypes of lung injury in the (A) training and (B) validation datasets, respectively. ****p-value <0.0001.

FIGURE 4

Figure 4 The performance of the developed Rad-score. (A) Receiver operating characteristic (ROC) curves. (B) Precision–recall curve (PRC). The subscripts emp and αbin mean empirical and α-binormal-based ROC or PRC, respectively.

TABLE 2

Table 2 Comparison of the performances of Rad-score and nomogram.

Nomogram Construction and Validation

Before nomogram construction, univariate analyses were performed for clinical factors and radiological features using chi-square test or Mann–Whitney U-tests. In the training dataset, the differences of clinicopathological characteristics and radiological findings between the patients with CIP and RP are shown in Table 3. The median age of patients with CIP and RP were 56.8 and 57.4, and there was no discrepancy in the subtype of pneumonitis by gender (p = 0.313) and smoking history (p = 0.200). Moreover, there was a trend towards more lung lobes (p = 0.184) and volumes (p = 0.101) infected in patients with radiation pneumonitis compared with patients with CIP. We did not figure out the values of histology (p = 0.067), radiological elements (p = 0.910), and other factors for classification between CIP and RP. The variables of bilateral changes (p < 0.001) and sharp border (p = 0.001) were considered independent biomarkers and had a statistically significant difference between CIP and RP.

TABLE 3

Table 3 Clinical and radiological parameters of patients with checkpoint inhibitor-related pneumonitis (CIP) and radiation pneumonitis (RP) in the training dataset.

To determine the benefits for prediction performances of radiomics features, the Rad-score, bilateral changes, and sharp border were incorporated into the radiomics nomogram, as shown in Figure 5. In Figure 6, the nomogram model displayed the highest discrepancy between CIP and RP with the empirical and α-binormal-based AUC of 0.953 and 0.950 in the training set. In the validation samples, the nomogram yielded the greatest AUC (0.947, 95% CI: 0.912–0.982; 0.936, 95%CI: 0.905–0.967), which confirmed that the nomogram model achieved better differential capacity than the Rad-score. As illustrated by the results, an obvious separation between CIP and RP was detected in the validation dataset with AP of 0.943 and 0.887 (as shown in Table 2).

FIGURE 5

Figure 5 Nomograms constructed in this study using the training dataset.

FIGURE 6

Figure 6 The performance of the developed nomogram. (A) Receiver operating characteristic (ROC) curves. (B) Precision–recall curve (PRC). The subscripts emp and αbin mean empirical and α-binormal-based ROC or PRC, respectively.

The calibration curve manifested sufficient consistency between estimated CIP probability using the nomogram and the actual observed outcome in Figure 7. The closer the calibration curve was to the diagonal, the better the predictive ability of the nomogram. The P-value of Hosmer–Lemeshow test for subtype of pneumonitis was 0.511. To determine the clinical utility, decision curves were plotted for Rad-score and radiomics nomogram (shown in Figure 8). It showed that the radiomics nomogram produced a greater net benefit than the Rad-score.

FIGURE 7

Figure 7 Calibration curve of the radiomics nomogram presented as a solid red line. The diagonal dashed line indicates perfect agreement.

FIGURE 8

Figure 8 The decision curves of the Rad-score, radiomics nomogram, and two extreme curves were plotted based on the validation dataset. It showed that the use of the radiomics nomogram to predict CIP probability in patients with non-small cell lung cancer has a greater benefit than the use of Rad-score.

Discussion

With the evolution of immunotherapy, the combination of immunotherapy and radiotherapy became increasingly important in guiding NSCLC treatment according to the latest NCCN guidelines (18). However, the widespread utilization of ICIs has contributed in off-target immune-related adverse events, especially CIP. In this retrospective study, we developed and validated a classification model based on CT radiomics for distinguishing between CIP and RP. Radiomics nomograms were extracted and showed good performance on the training and validation datasets with AUC = 0.953 and 0.947, which proved that the CT-based radiomics nomogram was feasible. In the radiomics nomogram, Rad-score has the largest weight and is more important for distinguishing subtypes of pneumonitis. All features used to construct the Rad-score were wavelets; the reasonable hypothesis was that the similar pixel intensity of different pneumonitis derived from original images provide limited differentiation power in the modeling process. However, multiple frequency decomposition of the original CT image can decode the hidden difference of phenotype between CIP and RP (19). As far as we are concerned, this is the first study that uses comprehensive information by including both radiomics, clinical, and radiological factors in the classification of CIP and RP.

Previous studies demonstrated the safety of thoracic RT in patients receiving ICIs (20). Nevertheless, current evidence has suggested that radiotherapy had immune stimulating effects, which could potentially enhance the effectiveness of ICIs and increase the risk of relevant immune-related adverse events (21). Preliminary studies mentioned the radiologic appearances and clinical symptoms of CIP, which were similar with RP characteristics (22). The clinical symptoms of pneumonitis after immune checkpoint inhibitors or radiotherapy included cough, dyspnea, fever, and shortness of breath. Moreover, the most frequent CT image findings are ground-glass opacities or consolidative opacities in subpleural regions among patients with CIP (23). Generally, distinguishing between CIP and RP has a crucial impact on the treatment to pneumonitis and decision-making on therapy in the near feature.

Since the occurrence of CIP, CT morphological features have been treated as important clues for diagnosing CIP. Our present study revealed that these features were valuable but not the only strong clue for diagnosing CIP or RP. We noted that bilateral changes and a sharp border showed significant difference for identifying CIP and RP. ICI pneumonitis trended toward a higher likelihood of bilateral CT changes (p < 0.001) than pneumonitis from RT alone. Patients with CIP were less likely to have a sharp border (p = 0.001) in comparison with RP. This conclusion was consistent with previous studies (24). On the other hand, the Rad-score and nomogram were evaluated by α-binominal model-based ROC and PRCs and compared to empirical curves. In this study, the results showed that no significant difference was observed between the two types of curve. Besides this, performance measures based on PRCs were helpful to supplement and validate the ROC curves. It can effectively address the practical limitations derived from empirical curves (17). Furthermore, the decision curves demonstrated that radiomics nomogram added more benefits for differentiating CIP from RP than Rad-score. Generally, the nomogram was robust and accurate for distinguishing the subtype of pneumonitis.

Our result verified the values of quantitative radiomics features, instead of qualitative radiological factors, to discriminate pneumonitis etiology. Recently, a study performed a radiomics analysis and extracted 1,860 features in a total of 290 patients who were treated with immunotherapy (25). Radiomics features were identified and predicted subsequent immunotherapy-related pneumonitis. In accordance with RP, some researchers stated that the multi-region CT radiomics features can help to predict grade ≥2 RP for NSCLC patients (26). As Du et al. proposed, a radiomics model based on thoracic cone-beam computed tomography had a potential value of RP prediction (27). The radiomics features had an ability to assess the heterogeneity of image intensities and highlight small differences that cannot be recognized by the naked eyes. Yang et al. utilized CT-based radiomics to differentiate COVID-19 from other pneumonias in patients with accuracy of 89.83% and AUC of 0.940 (28). Tabatabaei et al. inferred that radiomics features, conjoined with random forest and neural network, appeared to yield promising results in distinguishing between COVID-19 and H1N1 influenza by CT scanning, with AUC = 0.970 (29).

As far as we know, few studies focused on this topic. Chen et al. extracted radiomics features from CT images and only trained 29 patients with RT and 23 patients with ICIs (30). The classifier has shown a performance on the training dataset with AUC =0.79 and the validation dataset with AUC = 0.840. The results were in line with ours, but the predictive efficacy of this model was much lower. Cheng et al. collected the CT images of 73 NSCLC patients with ICI or RT and extracted only 3 types of radiomics features (31). They figured out that the kind of bag-of-word features achieved an AUC of 0.937. Despite the encouraging results, there were some limitations in the above-mentioned study. Firstly, the small sample size and limited radiomics features that resulted in this conclusion can be hardly generalized. Secondly, no classification model, such as radiomics signatures or nomogram, has been built for discriminating between CIP and RP. Our results indicated that an integrated model combined with radiomics features and clinical factors may help to differ CIP from RP in NSCLC. Thirdly, there were no clinical and laboratory parameters involved, which have been proved to reveal the mechanism of two kinds of pneumonitis to some extent (32, 33).

This difference in the pathogenesis of pneumonitis remains to be discussed. To our best knowledge, Chen et al. mentioned that checkpoint inhibitor pneumonitis was more likely to involve more lobes of the lung (24). Based on our wide clinical experience and practice, RP was commonly limited in the radiation field of the lung, while CIP was always irregularly distributed in random fields of the lung. Moreover, lymphocytes were predominant to the pathogenesis of pneumonitis. The relationship between neutrophil-to-lymphocyte ratio and CIP may provide a reasonable explanation (34). The value of inflammatory cytokine in the diagnosis of CIP has not yet been completely evaluated. IL-10, an anti-inflammatory cytokine, was maintained at a lower level in RP patients at baseline (35). In contrast, lung cancer patients with CIP, shown to have increased IL-6, were associated with the occurrence of disease (36).

It is necessary to identify the mechanism of pneumonitis arising from immunotherapy and radiotherapy. Radiotherapy could directly damage DNA and reactive oxygen species generation, then mediate intracellular signaling, and result in the release of molecules and cytokines (37). The occurrence of acute pneumonitis and chronic pulmonary fibrosis is mainly through the TGF-β/Smad (38), HMGB1/TLR4 (39), and Nrf2/ARE signaling pathways (40). Glucocorticoid drugs are the mainstay of RILI treatment in clinical practices. Regarding IRLI, generalized immune activation owing to checkpoint neutralization, preexisting autoantibodies, and off-target effects of T cell-mediated immunity will lead to immune-related lung injury (41). Immunosuppressive drugs such as glucocorticoids, mycophenolate mofetil, and cytokine inhibitors were mainly used for IRLI treatment (42). Zhang et al. reported the crosstalk among signaling pathways in RILI and IRLI, which noted that the TGF-β signaling pathway played an important role in the crosstalk between IRIL and RILI (41); more exploration and discovery are required to pinpoint the mechanisms of RILI and IRLI.

There are some limitations in this study. First, it is a retrospective study with 126 patients. A larger sample size and prospective study are necessary to explore and validate in the near future. Second, training and validation datasets were acquired from a single institution, which resulted in a conclusion hardly generalizable to other study centers. Further investigation will concentrate on samples from various institutions as external validation dataset. Third, no other laboratory parameters were included in the analysis. We trust that the subsequent research should be to develop a robust model.

Conclusions

In summary, there are many similarities of clinical symptoms and radiological findings between immune checkpoint inhibitor-related pneumonitis and radiation pneumonitis for patients with NSCLC, which propose great management challenges in clinical practice. Our results have successfully suggested that CT radiomics features are capable of differentiating between CIP and RP. Additions of bilateral changes and sharp border produce a superior nomogram model performance. Our findings suggest that a radiomics nomogram could be used for clinical decision-making and providing an accurate diagnostic tool.

Data Availability Statement

The original contributions presented in the study are included in the article/supplementary material. Further inquiries can be directed to the corresponding author.

Ethics Statement

Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author Contributions

QQ designed the study and wrote the manuscript. QW participated in the study designing and data collection. YW and AF provided the analysis of data and ROI segmentation. QQ and QW participated in data collection and offered guidance. LX carried out the study design and interpretation of data and drafted the manuscript. All authors contributed to the article and approved the submitted version.

Funding

This study was supported by the National Natural Science Foundation of China (grant number 82001902), Shandong Provincial Natural Science Foundation (grant numbers ZR2020QH198 and ZR2020QH200), Radiation Oncology Translational Medicine Foundation for Scientific Research of Bethune (grant number flzh202123), and the Special Tumor Foundation for Scientific Research of Saifu (grant number fszl202106). The funding sources had no role in the study design, data collection, analysis of interpretation, or writing of this manuscript.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Antonia SJ, Villegas A, Daniel D, Vicente D, Murakami S, Hui R, et al. Durvalumab After Chemoradiotherapy in Stage III Non-Small-Cell Lung Cancer. N Engl J Med (2017) 377(20):1919–29. doi: 10.1056/NEJMoa1709937

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Antonia SJ, Villegas A, Daniel D, Investigators P. Overall Survival With Durvalumab After Chemoradiotherapy in Stage III NSCLC. N Engl J Med (2018) 379(24):2342–50. doi: 10.1056/NEJMoa1809697

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Ko E, Raben D, Formenti S. The Integration of Radiotherapy With Immunotherapy for the Treatment of Non-Small Cell Lung Cancer. Clin Cancer Res (2018) 24(23):5792–806. doi: 10.1158/1078-0432.Ccr-17-3620

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Li M, Gan L, Song A, Xue J, Lu Y. Rethinking Pulmonary Toxicity in Advanced non-Small Cell Lung Cancer in the Era of Combining Anti-PD-1/PD-L1 Therapy With Thoracic Radiotherapy. Biochim Biophys Acta Rev Canc (2019) 1871(2):323–30. doi: 10.1016/j.bbcan.2019.02.004

CrossRef Full Text | Google Scholar

5. Nishino M, Giobbie-Hurder A, Hatabu H, Ramaiya N, Hodi F. Incidence of Programmed Cell Death 1 Inhibitor-Related Pneumonitis in Patients With Advanced Cancer: A Systematic Review and Meta-Analysis. JAMA Oncol (2016) 2(12):1607–16. doi: 10.1001/jamaoncol.2016.2453

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Kong F, Hayman J, Griffith K, Kalemkerian GP, Arenberg D, Lyons S, et al. Final Toxicity Results of a Radiation-Dose Escalation Study in Patients With Non-Small-Cell Lung Cancer (NSCLC): Predictors for Radiation Pneumonitis and Fibrosis. Int J Radiat Oncol Biol Phys (2006) 65(4):1075–86. doi: 10.1016/j.ijrobp.2006.01.051

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Bradley P, Paulus BR, Komaki PR, Masters G, Choy PH. Standard-Dose Versus High-Dose Conformal Radiotherapy With Concurrent and Consolidation Carboplatin Plus Paclitaxel With or Without Cetuximab for Patients With Stage IIIA or IIIB Non-Small-Cell Lung Cancer (RTOG 0617): A Randomised, Two-by-Two Factorial Phase 3 Study. Lancet Oncol (2015) 16(2):187–99. doi: 10.1016/S1470-2045(14)71207-0

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Naidoo J, Wang X, Woo K, Iyriboz T, Halpenny D, Cunningham J, et al. Pneumonitis in Patients Treated With Anti-Programmed Death-1/Programmed Death Ligand 1 Therapy. J Clin Oncol (2017) 35(7):709–17. doi: 10.1200/jco.2016.68.2005

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Schoenfeld J, Nishino M, Severgnini M, Manos M, Mak R, Hodi F. Pneumonitis Resulting From Radiation and Immune Checkpoint Blockade Illustrates Characteristic Clinical, Radiologic and Circulating Biomarker Features. J Immuno Canc (2019) 7(1):112. doi: 10.1186/s40425-019-0583-3

CrossRef Full Text | Google Scholar

10. Gillies R, Kinahan P, Hricak H. Radiomics: Images Are More Than Pictures, They Are Data. Radiology. (2016) 278(2):563–77. doi: 10.1148/radiol.2015151169

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Refaee T, Wu G, Ibrahim A, Halilaj I, Woodruff H. The Emerging Role of Radiomics in COPD and Lung Cancer. Resp Int Rev Thorac Disease (2020) 99(2):99–107. doi: 10.1159/000505429

CrossRef Full Text | Google Scholar

12. Yang L, Yang J, Zhou X, Huang L, Zhao W, Wang T, et al. Development of a Radiomics Nomogram Based on the 2D and 3D CT Features to Predict the Survival of non-Small Cell Lung Cancer Patients. Eur Radiol (2019) 29(5):2196–206. doi: 10.1007/s00330-018-5770-y

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Barabino E, Rossi G, Pamparino S, Fiannacca M, Caprioli S, Fedeli A, et al. Exploring Response to Immunotherapy in Non-Small Cell Lung Cancer Using Delta-Radiomics. Cancers (2022) 14(2):350. doi: 10.3390/cancers14020350

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Wen Q, Yang Z, Dai H, Feng A, Li Q. Radiomics Study for Predicting the Expression of PD-L1 and Tumor Mutation Burden in Non-Small Cell Lung Cancer Based on CT Images and Clinicopathological Features. Front Oncol (2021) 11:620246. doi: 10.3389/fonc.2021.620246

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Brahmer J, Lacchetti C, Schneider B, Atkins MB, Brassil KJ, Caterino JM, et al. Management of Immune-Related Adverse Events in Patients Treated With Immune Checkpoint Inhibitor Therapy: American Society of Clinical Oncology Clinical Practice Guideline. J Clin Oncol (2018) 36(17):1714–68. doi: 10.1200/jco.2017.77.6385

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Bledsoe T, Nath S, Decker R. Radiation Pneumonitis. Clin Chest Med (2017) 38(2):201–8. doi: 10.1016/j.ccm.2016.12.004

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Brodersen KH, Cheng SO, Stephan KE, Buhmann JM. The Binormal Assumption on Precision-Recall Curves. Int Conf Pattern Recog IEEE Comput Soc (2010). doi: 10.1109/ICPR.2010.1036

CrossRef Full Text | Google Scholar

18. Ettinger D, Wood D, Aisner D, Akerley W, Bauman JR, Bharat A, et al. NCCN Guidelines Insights: Non-Small Cell Lung Cancer, Version 22021. J Natl Compr Cancer Net JNCCN (2021) 19(3):254–66. doi: 10.6004/jnccn.2021.0013

CrossRef Full Text | Google Scholar

19. Cong P, Qiu Q, Li X, Sun Q, Yu X, Yin Y. Development and Validation a Radiomics Nomogram for Diagnosing Occult Brain Metastases in Patients With Stage IV Lung Adenocarcinoma. Trans Cancer Res (2021) 10(10):4375–86. doi: 10.21037/tcr-21-702

CrossRef Full Text | Google Scholar

20. Voong K, Hazell S, Fu W, Hu C, Lin CT, Ding K, et al. Relationship Between Prior Radiotherapy and Checkpoint-Inhibitor Pneumonitis in Patients With Advanced Non-Small-Cell Lung Cancer. Clin Lung Canc (2019) 20(4):e470–9. doi: 10.1016/j.cllc.2019.02.018

CrossRef Full Text | Google Scholar

21. Khunger M, Rakshit S, Pasupuleti V, Hernandez AV, Mazzone P, Stevenson J, et al. Incidence of Pneumonitis With Use of Programmed Death 1 and Programmed Death-Ligand 1 Inhibitors in Non-Small Cell Lung Cancer: A Systematic Review and Meta-Analysis of Trials. Chest. (2017) 152(2):271–81. doi: 10.1016/j.chest.2017.04.177

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Michot J, Bigenwald C, Champiat S, Collins M, Carbonnel F, Postel-Vinay S, et al. Immune-Related Adverse Events With Immune Checkpoint Blockade: A Comprehensive Review. Eur J Cancer (Oxf Engl 1990) (2016) 54:139–48. doi: 10.1016/j.ejca.2015.11.016

CrossRef Full Text | Google Scholar

23. Naidoo J, Page D, Li B, Connell LC, Schindler K, Lacouture ME, et al. Toxicities of the Anti-PD-1 and Anti-PD-L1 Immune Checkpoint Antibodies. Ann Oncol (2015) 26(12):2375–91. doi: 10.1093/annonc/mdv383

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Chen X, Sheikh K, Nakajima E, Lin CT, Lee J, Hu C, et al. Radiation Versus Immune Checkpoint Inhibitor Associated Pneumonitis: Distinct Radiologic Morphologies. Oncol (2021) 26(10):e1822–32. doi: 10.1002/onco.13900

CrossRef Full Text | Google Scholar

25. Colen R, Fujii T, Bilen M, Kotrotsou A, Abrol S, Hess KR, et al. Radiomics to Predict Immunotherapy-Induced Pneumonitis: Proof of Concept. Investig New Drugs (2018) 36(4):601–7. doi: 10.1007/s10637-017-0524-2

CrossRef Full Text | Google Scholar

26. Kawahara D, Imano N, Nishioka R, Ogawa K, Kimura T, Nakashima T, et al. Prediction of Radiation Pneumonitis After Definitive Radiotherapy for Locally Advanced Non-Small Cell Lung Cancer Using Multi-Region Radiomics Analysis. Sci Rep (2021) 11(1):16232. doi: 10.1038/s41598-021-95643-x

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Du F, Tang N, Cui Y, Wang W, Zhang Y, Li Z, et al. A Novel Nomogram Model Based on Cone-Beam CT Radiomics Analysis Technology for Predicting Radiation Pneumonitis in Esophageal Cancer Patients Undergoing Radiotherapy. Front Oncol (2020) 10:596013. doi: 10.3389/fonc.2020.596013

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Yang N, Liu F, Li C, Xiao W, Xie S, Yuan S, et al. Diagnostic Classification of Coronavirus Disease 2019 (COVID-19) and Other Pneumonias Using Radiomics Features in CT Chest Images. Sci Rep (2021) 11(1):17885. doi: 10.1038/s41598-021-97497-9

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Tabatabaei M, Tasorian B, Goyal M, Moini A, Sotoudeh H. Feasibility of Radiomics to Differentiate Coronavirus Disease 2019 (COVID-19) From H1N1 Influenza Pneumonia on Chest Computed Tomography: A Proof of Concept. Iranian J Med Sci (2021) 46(6):420–7. doi: 10.30476/ijms.2021.88036.1858

CrossRef Full Text | Google Scholar

30. Chen X, Sheikh K, Lin CT, Lee J, Voong KR. CT Radiomics and Machine Learning for Distinguishing Radiotherapy vs. Immune Checkpoint Inhibitor-Induced Pneumonitis in Non-Small Cell Lung Cancer Patients. Int J Radiat Oncol Biol Phys (2020) 108(3):S163. doi: 10.1016/j.ijrobp.2020.07.929

CrossRef Full Text | Google Scholar

31. Cheng J, Pan Y, Huang W, Huang K, Cui Y, Hong W, et al. Differentiation Between Immune Checkpoint Inhibitor-Related and Radiation Pneumonitis in Lung Cancer by CT Radiomics and Machine Learning. Med Phys (2022) 49(3):1547–58. doi: 10.1002/mp.15451

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Liu X, Shao C, Fu J. Promising Biomarkers of Radiation-Induced Lung Injury: A Review. Biomedicines (2021) 9(9):1181. doi: 10.3390/biomedicines9091181

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Johnson D, Balko J. Biomarkers for Immunotherapy Toxicity: Are Cytokines the Answer? Clin Cancer Res (2019) 25(5):1452–4. doi: 10.1158/1078-0432.Ccr-18-3858

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Matsukane R, Watanabe H, Minami H, Hata K, Suetsugu K, Tsuji T, et al. Continuous Monitoring of Neutrophils to Lymphocytes Ratio for Estimating the Onset, Severity, and Subsequent Prognosis of Immune Related Adverse Events. Sci Rep (2021) 11(1):1324. doi: 10.1038/s41598-020-79397-6

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Arpin D, Perol D, Blay J, Falchero L, Claude L, Vuillermoz-Blas S, et al. Early Variations of Circulating Interleukin-6 and Interleukin-10 Levels During Thoracic Radiotherapy are Predictive for Radiation Pneumonitis. J Clin Oncol (2005) 23(34):8748–56. doi: 10.1200/jco.2005.01.7145

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Lin X, Deng H, Yang Y, Wu J, Qiu G, Li S, et al. Peripheral Blood Biomarkers for Early Diagnosis, Severity, and Prognosis of Checkpoint Inhibitor-Related Pneumonitis in Patients With Lung Cancer. Front Oncol (2021) 11:698832. doi: 10.3389/fonc.2021.698832

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Azzam E, Jay-Gerin J, Pain D. Ionizing Radiation-Induced Metabolic Oxidative Stress and Prolonged Cell Injury. Cancer letter (2012) 327:48–60. doi: 10.1016/j.canlet.2011.12.012

CrossRef Full Text | Google Scholar

38. Saito A, Horie M, Nagase T. TGF-β Signaling in Lung Health and Disease. Int J Mol Sci (2018) 19(8):2460. doi: 10.3390/ijms19082460

CrossRef Full Text | Google Scholar

39. Liu L, Song C, Li J, Wang Q, Zhu M, Hu Y, et al. Fibroblast Growth Factor 10 Alleviates Particulate Matter-Induced Lung Injury by Inhibiting the HMGB1-TLR4 Pathway. Aging (2020) 12(2):1186–200. doi: 10.18632/aging.102676

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Cameron B, Sekhar K, Ofori M, Freeman M. The Role of Nrf2 in the Response to Normal Tissue Radiation Injury. Radiat Res (2018) 190(2):99–106. doi: 10.1667/rr15059.1

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Zhang Z, Zhou J, Verma V, Liu X, Wu M, Yu J, et al. Crossed Pathways for Radiation-Induced and Immunotherapy-Related Lung Injury. Front Immunol (2021) 12:774807. doi: 10.3389/fimmu.2021.774807

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Andruska N, Mahapatra L, Hebbard C, Patel P, Paul V. Severe Pneumonitis Refractory to Steroids Following Anti-PD-1 Immunotherapy. BMJ Case Rep (2018) bcr2018225937. doi: 10.1136/bcr-2018-225937

CrossRef Full Text | Google Scholar