Detection and evaluation of signals for immune-related adverse events: a nationwide, population-based study

Background: Immune checkpoint inhibitors (ICIs) are one of the main pillars of cancer therapy. Since other studies such as clinical trial and retrospective study have limitations for detecting the immune-related adverse events (irAEs) characterized by unpredictable onset, nonspecific symptoms and wide clinical spectrum, we aimed to identify the incidence of irAEs and to detect and evaluate the signals using real-world data.

Methods: Cancer patients treated with anticancer medications were analyzed using the nationwide health insurance claims database of South Korea from 2017 to 2019, and Clinical Data Warehouse (CDW) database of Asan Medical Center (AMC), a tertiary referral hospital, from 2012 to 2019. AEs of ICI users were compared with those of non-ICI anticancer medication users. PD-1 inhibitors (nivolumab and pembrolizumab) and PD-L1 inhibitors (atezolizumab) were evaluated. We defined an AE as a newly added diagnosis after the ICI prescription using an ICD-10 diagnostic code. A signal was defined as an AE that was detected by any one of the four indices of data mining: hazard ratio (HR), proportional claims ratio (PCR), claims odds ratio (COR), or information component (IC). All detected signals were reviewed and classified into well-known or potential irAEs. Signal verification was performed for targeted AEs using CDW of AMC using diagnostic codes and text mining.

Results: We identified 118 significant signals related to ICI use. We detected 31 well-known irAEs, most of which were endocrine diseases and skin diseases. We also detected 33 potential irAEs related to disorders in the nervous system, eye, circulatory system, digestive system, skin and subcutaneous tissues, and bones. Especially, portal vein thrombosis and bone disorders such as osteoporosis with pathological fracture and fracture of shoulder, upper arm, femur, and lower leg showed high HR in ICI users than in non-ICI users. The signals from hospital database were verified using diagnostic codes and text mining.

Conclusion: This real-world data analysis demonstrated an efficient approach for signal detection and evaluation of ICI use. An effective real-world pharmacovigilance system of the nationwide claims database and the EMR could complement each other in detecting significant AE signals.

1 Introduction

Immune checkpoint inhibitors (ICIs), which activates the immune system to fight against cancer, have evolutionally changed the recent trend of cancer therapy. Monoclonal antibodies targeting T-cell immune checkpoint molecules, including programmed cell death protein 1 (PD-1), programmed cell death ligand-1 (PD-L1), and cytotoxic T lymphocyte associated protein-4 (CTLA-4), are the most effective examples of ICIs. Randomized clinical trials involving ICIs have demonstrated the durable clinical benefits of these medications across a diverse range of cancers (1–3).

However, patients treated with ICI may experience immune-related adverse events (irAEs), which are autoimmune disease-like toxicities caused by immune reaction associated with this treatment. These events exhibit distinct characteristics compared to those observed with conventional cytotoxic chemotherapy or targeted agents. They can affect any part of the body with diverse severity and onset times (4, 5). The unique profile of irAEs, characterized by unpredictable onset, even after the completion of treatment, nonspecific symptoms, a broad clinical spectrum, and in some cases, irreversibility and fatality poses a significant challenge in the pharmacovigilance of ICIs. Clinical trials have reported various adverse events (AEs), but they have limitations since they collect data from a limited number of selected populations for a restricted duration of time. In particular, ICIs have often received accelerated approval from regulatory agencies based on results from earlier phase studies with small samples sizes and short study durations (6, 7). This may increase the risk of the AEs not being fully characterized in pre-marketing clinical trials. Therefore, post-marketing pharmacovigilance of ICIs is particularly crucial to capture real-world experiences. Several efforts have been undertaken to identify AE signals using post-market pharmacovigilance databases (8–13). However, these databases primarily consist of voluntary AE reports, making them susceptible to various reporting biases and rendering them inadequate for calculating AE incidence. Large-scale observational studies using real-world data, such as electronic health records, medical claims and billing data, and registries, have the potential to overcome the inherent limitations of clinical trials and spontaneous reporting databases. Given that the National Health Insurance (NHI) system in South Korea provides healthcare coverage to the entire population residing the country and that all reimbursing claims with diagnostic codes are evaluated by the Health Insurance Review and Assessment Service (HIRA), the claim data of HIRA can be considered an appropriate data source for pharmacovigilance research. Several attempts have been made to utilize health insurance claims data as a pharmacovigilance tool for conducting post-marketing surveillance, employing multiple data mining methods (14, 15).

In the present study, our objective was to monitor known irAEs in a real-world setting and identify potential new AE signals in patients undergoing treatment with ICIs. To achieve this, we utilized a nationwide, population-based claim database as a primary data source and further validated these signals using a clinical data warehouse.

2 Methods

2.1 Data source for signal detection

We defined the potential risk associated with ICI usage as a “signal”, a term frequently used in pharmacovigilance. This term denotes “Information originating from one or several sources, indicating a possible new causal relationship or a novel aspect of a known association between an intervention and an event or a group of related events, whether adverse or beneficial” (16, 17). Signal detection in this study was conducted using the HIRA database. The HIRA database includes all information on approximately 50 million people, the whole Korean population covered by the NHI program since 2000. Every resident in South Korea is provided with a unique civil registration number. The NHI program provides coverage for all residents in a form of compulsory social insurance, which ensured the complete follow-up of study participants. In South Korea, the NHI system started to reimburse ICI treatment for non-small cell lung cancer (NSCLC) and bladder cancer from August 2017, and for melanoma from February 2018 (18). We obtained the claims data of patients with NSCLC, melanoma, and bladder cancer that had been submitted by healthcare providers between January 1, 2014, and January 31, 2019, with anonymized identifiers provided by the HIRA to protect their privacy, in accordance with the Act on the Protection of Personal Information Maintained by Public Agencies. The database contains information on demographic records, diagnosis, procedures performed, and prescriptions. Demographic information included age and sex. All diagnoses were coded using the Korean Standard Classification of Disease, seventh edition (KCD-7), a modified version of the International Classification of Disease and Related Health Problems, 10th revision (ICD-10). The prescription data included generic name of the drug according to the HIRA drug formulary code, prescription date, and duration. Access to the HIRA database was limited to researchers who completed an application process and were granted approval for research access.

2.2 Ethics

This study was approved by the Institutional Review Board of Asan Medical Center (IRB No. 2019-1300, 2019-1247), and informed consent was waived.

2.3 Study subjects for signal detection

In this study, we retrospectively identified patients with NSCLC, bladder cancer, and melanoma by using the KCD-7 codes for primary and secondary diagnoses (C34, C67, and C43, respectively), as well as the corresponding cancer-specific deductible insurance codes (V027 for NSCLC, V193 for bladder cancer, and V194 for melanoma). These patients were then categorized into two groups: the ICI group and the non-ICI group. Utilizing the 5-year HIRA claims database (from January 1, 2014, to January 31, 2019), we selected patients who received anticancer medication between August 2017 and January 2019, based on the initiation date (August 2017) of the NHI coverage for ICIs. The ICI group consisted of patients who received at least one cycle of treatment with PD-1 inhibitors (nivolumab and pembrolizumab) or PD-L1 inhibitor (atezolizumab) inhibitor between August 2017 and January 2019. The non-ICI group, serving as the comparator, included patients who received non-ICI systemic anticancer agents such as cytotoxic chemotherapy and molecular targeted therapy (Supplementary Table S1) for NSCLC, bladder cancer, and melanoma during the same period as the ICI group. The index date for each patient was defined as the first prescription date of the corresponding drug, and all study patients were followed up for a maximum of 18 months from the index date. AEs in the ICI group were compared with those in the non-ICI anticancer medication group.

2.4 Definition of adverse events

In this study, AEs were identified from the claims study database by considering newly added diagnosis codes during treatment. AEs were defined as diagnoses that did not exist prior to exposure to the drug of interest but were added after exposure. The KCD-7 coding system, which consists of 21 chapters with various code blocks, was used for classification. Physicians in South Korea are required to report the most specific diagnosis code available according to the KCD-7 coding system, ranging from three to seven digits, in the medical care cost statements submitted to the NHI system.

Initially, a low-level classification was performed using three-digit KCD codes, resulting in a group of 1,348 codes. Subsequently, a mid-level classification was conducted using groups of three-digit KCD codes to categorize single conditions within each block, resulting in a total of 222 codes. However, certain disease categories were excluded from the study. The neoplasm category (KCD-7 codes, C00-D48) was excluded due to the difficulty in distinguishing it from existing cancers, which are already indications for ICIs. Additionally, pregnancy-related diagnosis including ‘‘Pregnancy, childbirth, and the puerperium’’ (O00-O99), ‘‘Certain conditions originating in the perinatal period’’ (P00-P96), and ‘‘Congenital malformations, deformations, and chromosomal abnormalities’’ (Q00-Q99) were excluded to focus on non-pregnant cancer patients.

2.5 Signal detection criteria

The hazard ratio (HR) was computed by dividing the incidence of ICI-specific AE pairs (A) by the total observed duration of claims (T₁), and then dividing the results by the incidence of non-ICI-AE pairs (C) divided by the total observed duration of claims (T₀). If the lower limit of the 95% confidence interval (CI) of an AE’s HR was greater than 1, it was considered a potential signal possibly associated with ICIs.

Disproportionality measures were also employed to detect AE signals, using proportional claims ratios (PCRs), claims odds ratios (CORs), and Bayesian confidence propagation neural networks (BCPNNs) of information components (ICs). These measures were specifically adapted for use with the claims database, as opposed to spontaneous reporting databases. Two-by-two tables were constructed, combining the frequency of drug–AE pairs and total claims, along with the estimated indices for each AE (Table 1).

Table 1

Table 1 Study framework of the disproportionality analysis of the health insurance claims database.

PCR is defined as the ratio between the incidence of a specific AE associated with a particular ICI and the incidence of AEs related to non-ICI drugs listed in the database (19). COR represents the odds ratio of developing a specific AE compared to all other events for the ICI group compared with the odds of developing AEs when using non-ICI drugs listed in the database (18, 19). A signal was deemed significant if the lower limit of the 95% CIs for PCR or COR was greater than 1, or if the number of ICI-specific AE pairs (A) was equal to or greater than 3. For the BCPNN analysis, a signal was considered statistically significant if the lower limit of 95% credible interval was above 0 (20) (Table 2).

Table 2

Table 2 Definition and signal detection criterion for data mining indices.

In this study, any AE that was detected by at least one of the four indices (HR, PCR, COR, and IC) was considered a signal. The statistical analyses were conducted using SAS Enterprise Guide (version 7.1; SAS institute, NC, USA) and R version 3.6.2 for Windows (R Foundation for Statistical Computing, Vienna, Austria). Significant signals only in the ICI group compared to the non-ICI group were presented.

2.6 Signal categorization

The detected AEs from the HIRA database were reviewed independently by two medical oncologists. They assessed the expectedness and frequency of the AEs compared to previous literature (21, 22). The detected AEs were categorized based on the following conditions: 1) Well-known irAEs that were established as related to immunotherapy based on previous literature, and/or are associated with the pathogenesis of the disease entity involving autoimmune activation; 2) potential irAEs that were not previously recognized as irAEs, but were considered clinically relevant and important.

2.7 Signal verification

Signal verification was conducted using the Clinical Data Warehouse database of Asan Medical Center (AMC) to validate targeted AEs. This validation involved utilizing diagnostic codes and text mining techniques. Recognizing the limitation of the HIRA database, which relies on health insurance claims data, the electronic medical record (EMR) from the tertiary care hospital, AMC, was employed to verify targeted AEs based on symptoms or laboratory abnormalities. The research team, consisting of clinical experts, reviewed and discussed the detected signals to choose the targeted AEs. Access to the CDW database is restricted to in-hospital researchers who have received approval from both the Institutional Review Board and the Data Subcommittee.

Among 21,205 cancer patients diagnosed with NSCLC, melanoma, or bladder cancer between July 1, 2012 and January 31, 2019 from the hospital EMR database, 9236 patients who received systemic anticancer therapy were identified using diagnostic and prescription codes (Supplementary Figure S1). These patients were classified into two groups: ICI users (n=836) and non-ICI users (n=8,400). The observation period for all participants commenced from their respective first prescription date, and they were followed up until the end of the study period.

In addition, we identified the other validation group of 1,706 ICI users from the hospital EMR database, comprising patients diagnosed with any type of cancer, for clinical text analysis (text mining). This validation group included patients who were not reimbursed for ICI treatment. Then, these 1,706 ICI users were matched in a 1:2 ratio to 3,412 non-ICI users based on age, sex, and cancer subtype to compare the detailed information documented in the narrative chart.

Among the detected AE signals, targeted AEs that could be easily detected using diagnostic codes or diagnostic codes required on prescriptions were searched using the diagnostic code information (‘diagnostic code from the hospital EMR data’, n=9236). On the other hand, text mining techniques were applied to search for targeted AE signals that could be easily detected in narrative EMR records such as nursing records and progression notes (‘text mining applied to the hospital EMR data’, n=5118; 1,706 ICI users and 3,412 non-ICI users). Thus, there were not only signals that were verified by both methods, but also other signals excluded from each analysis.

To explore the unstructured data obtained from the EMR, including the clinical notes written by medical staff, we utilized Standard Query Language with the LIKE function in the R package (SQLDF) to identify meaningful words. The operational definition of the verified signals, including the disease category and detailed search terms, can be found in Supplementary Table S2.

3 Results

3.1 Characteristics of the study subjects

A total of 314,423 patients were diagnosed with NSCLC, melanoma, or bladder cancer between July 2014 and January 2019. Among these patients, 25,817 were prescribed with anticancer medications from August 2017 to January 2019. Among them, 4,516 patients received ICIs, while 21,301 patients received non-ICI anticancer medications (Supplementary Figure S2).

Baseline characteristics of the study patients are shown in Supplementary Table S3. While there were statistically significant differences between the ICI group and the non-ICI group in all variables, the differences were not clinically significant. Overall, 74.1% of the patients were male, with a mean (± standard deviation; SD) age of 66.4 (± 10.0) years and a mean (± SD) Charlson comorbidity index (CCI) of 7.5 (± 3.6). The majority of patients had NSCLC (88.5%), while bladder cancer and melanoma accounted for 9.0% and 2.6% of the cases, respectively. In the ICI group, 75.6% of the patients were male, with a mean (± SD) age of 64.9 (± 10.0) years and a mean (± SD) CCI of 8.6 (± 3.6). The majority of patients in this group had NSCLC (83.1%), while bladder cancer and melanoma accounted for 11.4% and 5.6% of the cases, respectively.

3.2 Number of detected signals from the National Health Insurance claims database

Figure 1 illustrated the number of detected signals and the logical relations in which the detection criterion was satisfied by each index. Out of the 1,570 screened AEs using the KCD code, a total of 118 distinct AEs were identified as signals associated with ICIs using the HR index (n=115), PCR index (n=67), COR index (n=49), and IC index (n=43). The HR index demonstrated the highest number of detections, followed by PCR, COR, and IC, in descending order.

Figure 1

Figure 1 Number of detected signals related to immune checkpoint inhibitor use by data-mining indices in the nationwide claims database. HR, hazard ratio; PCR, proportional claims ratio; COR, claims odds ratio; IC, information component; KCD, Korean Standard Classification of Diseases. * means number of detected signals in mid-level.

3.3 Detected AE signals

3.3.1 Incidence of detected AE signals

When detected AEs were categorized by the organ system, the ICI group exhibited the highest incidence of AEs in the endocrine, respiratory, dermatologic, and musculoskeletal systems, which included disorders in the thyroid gland (E00–E07, 20.8%), other disorders of the thyroid gland (E07, 16.0%), influenza and pneumonia (J09–J18, 21.1%), pneumonia, (J18, 18.6%), pruritus, (L29, 11.4%), and arthropathies (M00–25, 17.5%) (Table 3; Supplementary Table S4). Mental and behavioral disorders (F00–99) as well as infectious and parasitic diseases (A00–B39) were also common AEs observed in the ICI group, ranging from 5% to 10% in incidence. Hepatic organ-related AEs (K71) were present in approximately 4% of the ICI group. Other AEs, such as neurologic (G00–99), circulatory (I00–99), and digestive diseases (K00–93), were relatively infrequent, with an incidence of less than <5%. The Table 3 displays 43 AE signals that were found to be significant in all four criteria (HR, PCR, COR, IC) among the 118 detected signals of AEs associated with the use of ICIs in the HIRA database. These signals met the threshold for significance across all four indices, indicating a strong association with ICI treatment.

Table 3

Table 3 Adverse event signals significant in all four criteria (HR, PCR, COR, IC) in the HIRA database.

3.3.2 Well-known immune-related AEs

Well-known irAEs were identified in endocrine and skin diseases in association with the use of ICIs (Figure 2A). The incidence of endocrine diseases was significantly higher in patients receiving ICIs, including endocrine gland disorders (HR: 5.5, 95% CI: 3.2–9.7), other types of hypothyroidism (HR: 4.4, 95% CI: 3.7–5.2), disorders of the thyroid gland (HR: 4.0, 95% CI: 3.6–4.4), and other disorders of the thyroid (HR: 3.8, 95% CI: 3.4–4.2). Furthermore, the use of ICIs was strongly associated with skin disease, such as vitiligo (HR: 17.7, 95% CI: 5.7–54.8), lichen planus (HR: 11.5, 95% CI: 2.9–46.0), and erythema multiforme (HR: 5.6, 95% CI: 2.3–13.4). In addition, as is well known, connected tissue disorders, including systemic lupus erythematosus (HR: 4.9, 95% CI: 1.7-4.9) and other systemic involvement of connective tissue (HR: 2.8, 95% CI: 1.5-2.8), as well as and nervous disease such as nerve, nerve root, and plexus disorders (HR: 1.3, 95% CI: 1.1–1.6), were detected with a significant association with ICI use.

Figure 2

Figure 2 Categorization of detected signals related to immune checkpoint inhibitor use from the nationwide claims database. (A) Well-known immune-related adverse events. (B) Potential immune-related adverse events. AE, adverse events; HR, Hazard ratio; CI, Confidence interval; PCR, Proportional Claims Ratio; COR, Claims Odds Ratio; IC, Information Component. * detected statistically significant results.

3.3.3 Potential ICI-related AE

Detected AEs that were not previously well-known but judged as clinically relevant and important were defined as potential ICI-related AEs. The following associations were identified: i) Infection-related AEs-other types of sepsis, herpes zoster, dermatophytosis, and pneumocystosis; ii) Endocrine-related AEs- type 2 diabetes mellitus, Cushing’s syndrome and disorders of mineral metabolism; iii) Mental and behavioral disorders-related AEs-delirium and mood disorders; iv) Eye and adnexa-related AEs-cataract, chorioretinal inflammation, and otitis media; v) Circulatory system-related AEs-cardiac arrest, intracerebral hemorrhage, and portal vein thrombosis; vi) Respiratory system-related AEs-status asthmaticus and hypersensitivity pneumonitis due to organic dust; vii) Digestive system-related AEs-acute appendicitis; viii) Skin and subcutaneous tissue-related AEs-eccrine sweat disorders and decubitus ulcer; and viii) Bone disorders-related AEs-osteoporosis with pathological fracture, disorders in the continuity of the bone, fracture of the shoulder and upper arm, fracture of the femur, and fracture of the lower leg, including the ankle (Figure 2B).

3.4 Signal verification with hospital EMR database

Among the detected AE signals, signal verification for 17 targeted AEs was performed using additional hospital EMR database to address the limitation of the HIRA claims data (Figure 3; Supplementary Table S2). The incidence of detected AE signals was compared between population-level and hospital-level database. The signal verification was performed in two ways, using the diagnostic code and text mining.

Figure 3

Figure 3 Signal verification from the nationwide claims database and hospital electronic medical record database. AE, adverse events; HR, Hazard ratio; CI, Confidence interval,.

From diagnostic codes in hospital-level EMR data (n=9,236; 836 ICI users and 8,400 non-ICI users), other types of sepsis (HR 2.3, 95% CI 1.5–3.6), nerve root and plexus disorders (HR 3.0, 95% CI 1.7–5.3), decubitus ulcer (HR 3.5, 95% CI 2.1–6.0), and convulsion (HR 2.8, 95% CI 1.4–5.7) were identified (Figure 3).

After defining targeted AEs by search term through text mining aside from using diagnostic codes (Supplementary Table S2), AE signals were verified from the hospital EMR database (Figure 3). In patients available for text mining (n=5118; 1,706 ICI users and 3,412 non-ICI users), the detected AEs were i) related to infections: other types of sepsis, herpes zoster, and pneumocystis; ii) related to metabolic disease: disorders in mineral metabolism; iii) related to mental disorders: delirium; iv) related to nervous disease: epilepsy; v) related to circulatory system: cardiac arrest: vi) related to skin and subcutaneous tissue: eccrine sweat disorders; and vii) related to symptom: somnolence and convulsion. These results indicate that AE signals detected from the NHI database were verified from the hospital EMR database, using both diagnostic codes and text mining.

4 Discussion

In the present study, we utilized both nationwide claims databases and hospital EMR to comprehensively monitor and detect signals of irAEs possibly associated with ICIs. The hospital EMR data, which includes structured and unstructured data such as clinical notes, was utilized to verify the detected irAE signals in the claim data. By integrating these two different real-world databases, we were able to identify irAE signals that were not well-established in the literature and assess their association with the use of ICIs. This integrated approach highlights the importance of utilizing both nationwide claims databases and hospital EMR data in real-world pharmacovigilance system. This comprehensive approach provides valuable insights into the safety profile of ICIs and contributes to the ongoing monitoring and management of irAEs in clinical practice.

Our analysis of claim data revealed the spectrum and incidences of well-known irAEs at the population level. The most common well-known irAEs observed were thyroid disorders, such as hypothyroidism, subclinical hypothyroidism, hyperthyroidism, and thyroiditis (20.8% in the ICI group vs. 7.6% in the non-ICI group; HR = 4.0, 95% CI 3.6–4.4). The incidence of thyroid irAEs in our study appeared to be higher than reported in phase III clinical trials, where the mean incidence was around 10.8% (23), but consistent with previous real-world studies, which have reported incidence rates ranging from 13% to 30% (24–26). In addition, our claim data analysis revealed other well-known irAEs across various organ systems, such as dermatologic AEs such as pruritus, dermatitis, and atopic dermatitis, musculoskeletal AEs such as systemic lupus erythematosus and other systemic involvement of connective tissue, gastrointestinal AEs such as hepatitis, respiratory AEs such as pneumonitis, ocular AEs such as iridocyclitis, and neurologic AEs such as meningitis (Figure 2). The incidence of these well-known typical irAEs was similar to previous studies involving PD-1/PD-L1 inhibitors (5, 27–30). Of note, the incidence of irAEs in respiratory system we detected was higher than that of previously reported (5, 31). These findings demonstrate the utility of nationwide claims databases in real-world AE monitoring and risk measurement.

Furthermore, our analysis unveiled new AE signals occurring more frequently in patients receiving ICIs compared to those receiving non-ICI agents. In the endocrine system, we identified type 2 diabetes mellitus (DM) (9.0% in the ICI group vs. 8.7% in the non-ICI group; HR = 1.2, 95% CI 1.1–1.4) and Cushing’s syndrome (0.1% in the ICI group vs. 0.0% in the non-ICI group; HR = 5.7, 95% CI 1.7–19.8) as a new signal associated with ICI use. While ICIs are known to rarely cause type 1 DM (32, 33), their association with new-onset type 2 DM has not been well established. However, recent studies have indicated that ICIs may exacerbate pre-existing type 2 DM (33, 34) and increase the risk of developing incident diabetes (35), which aligns with our findings. Regarding Cushing’s syndrome, it has been described in case reports of patients receiving ICIs (36, 37), but the specific details and relationship between Cushing’s syndrome and ICIs have not been elucidated due to its rarity. These findings underscore the value of utilizing large-scale population-based claim data to uncover rare AE profiles. Nevertheless, caution is warranted in interpretation, as there is potential for increased steroid administration in ICI-treated patients due to irAEs. This may partially contribute to bias, leading to a higher prevalence of diabetes or Cushing ‘s disease in the ICI group compared to the non-ICI group.

In the nervous system, we detected an association between ICIs and epilepsy (3.8% in the ICI group vs. 3.0% in the non-ICI group; HR = 1.6, 95% CI 1.3–1.9), as well as somnolence/stupor/coma (0.6% in the ICI group vs. 0.5% in the non-ICI group; HR = 1.7, 95% CI 1.1–2.5). These findings were verified using hospital EMR data through text mining (for epilepsy, 0.8% in the ICI group vs. 0.3% in the non-ICI group, HR = 2.5, [95% CI: 1.8-3.5]; for somnolence/stupor/coma, 20.6% vs. 8.1%, HR = 2.6 [95% CI: 2.2-3.1)]. While these AE signals might be secondary to other well-known neurologic irAE such as encephalitis, they may also be associated with unrevealed irAE. In the circulatory system, we found new signals of cardiac arrest (4.4% in the ICI group vs. 2.7% in the non-ICI group; HR = 2.0, 95% CI 1.7–2.3), intracerebral hemorrhage (0.4% in the ICI group vs. 0.3% in the non-ICI group; HR = 2.0, 95% CI 1.2–3.3), and portal vein thrombosis (0.1% in the ICI group vs. 0.0% in the non-ICI group; HR: 5.6, 95% CI: 1.8–17.3) associated with ICI use. Thrombosis have been increasingly reported as an AE associated with ICIs, although data on its association with ICIs compared to traditional chemotherapy are conflicting (38–40). Mechanisms involving activated T cells, proinflammatory cytokines, and endothelial and platelet activation have been proposed (41–46).

Another notable finding was the association between ICIs and bone disorders, including osteoporosis with pathological fracture and fractures of various sites. After first case series describing skeletal irAEs associated with ICI in 2018 (46), another large-scale pharmacovigilance analysis using the Food and Drug Association (FDA) Adverse Event Reporting System recently suggested a possible cause-effect relationship between serious vertebral fractures and ICI use in patients without apparent preexisting risk factors (47). Emerging evidence suggests that systemic activation of T cells leads to an osteoprotegerin ligand-mediated increase in osteoclastogenesis and bone loss. ICIs can enhance bone resorption by activating T cells (48), causing bone fragility and increasing the risk of fractures (49–51).

The verification of AE signals using the hospital EMR database demonstrated the complementarity of diagnostic codes and text mining. Text mining analysis has the potential of capturing AEs that clinicians have not yet assigned a diagnosis code to, or for which diagnostic codes could not assigned. The integration of these two approaches can enhance the detection of significant AE signals.

To the best of our knowledge, this population-based study is the first to investigate the nationwide incidence of AEs and compare the incidence of AEs between users of ICIs and conventional chemotherapy. Previous large-scale studies utilizing the FDA Adverse Event Reporting System or U.S. commercial insurance database have only reported the incidence of irAEs among ICI users in observational settings (48, 52). Moreover, our study included a large number of cancer patients treated with ICIs, surpassing the sample size of most randomized controlled trials (53). In addition, we applied multiple data mining indices and conducted a comprehensive review of the identified signals. If an AE signal was detected by multiple data mining indices, it was considered more robust compared to signals detected by a single index. The majority of signals reported in our study were detected using the HR, which reflects the time until the occurrence of an event. Consistent with our findings, another study that utilized a national health insurance claims database reported that the HR exhibited the highest sensitivity among the data mining indices (15). Therefore, it is recommended to apply multiple indices and compare the results, as it is challenging to determine the superior method of data mining (15, 54). Finally, we validated the signals identified from the hospital EMR database using two approaches: diagnostic codes and text mining.

Our study has several limitations. Firstly, due to the observation nature of this study, there is a possibility of misclassifying AEs. Since AEs were defined based on diagnostic codes, symptoms or non-diagnosed conditions may have gone undetected. Similarly, even if keywords were found in the hospital records, we cannot be certain if AEs actually occurred. Secondly, we were unable to assess the severity of AEs and detailed patient information such as cancer stage, performance status, and treatment history. Thirdly, it is conceivable that AEs associated with other anticancer medications may have been identified after transition to different anticancer medications within 18 months of initiating either ICI or non-ICI. Despite the potential for this bias, our intention was to capture signals for a sufficient duration after beginning study drug administration. The decision was motivated by the fact that various and significant irAEs such as endocrine AEs, colitis, nephritis or hepatitis can manifest at any time, even after the discontinuation of ICI therapy, and may wax and wane over time (55, 56). Fourthly, we only included PD1/PD-L1 inhibitors such as nivolumab, pembrolizumab, and atezolizumab. Hence, the safety profiles of other types of ICIs, such as CTLA-4 inhibitors, should be evaluated in the future. Finally, beyond signal detection and validation, which involved comparing the frequency of AEs between the ICI and non-ICI groups, we recognize the absence of in-depth analysis in this study. We are planning further research with more comprehensive investigations, including retrospective analyses using large-scale, long-term hospital databases, and analyses in prospective trials. These will aim to explore potential irAEs identified in our study, such as portal vein thrombosis and bone disorders.

5 Conclusion

This real-world data analysis utilizing both nationwide claims databases and hospital EMR, has demonstrated an effective approach for signal detection and evaluation of the irAEs. The development of an effective real-world pharmacovigilance system utilizing nationwide claims database can complement existing EMR systems and contribute to the early diagnosis and prompt treatment of rare irAEs. Further research and development of near real-time or earlier access to pharmacovigilance systems are warranted to enhance patient safety and improve outcomes.

Data availability statement

The data analyzed in this study is subject to the following licenses/restrictions: To utilize the Health Insurance Review and Assessment Service (HIRA) database, a formal application and approval process on the official website 'https://opendata.hira.or.kr/home.do.' was needed. Subsequent to the data export procedure, access to the dataset was granted on a limited and temporary basis, contingent upon a thorough review by the data-holding institution, with external network access restricted. For any inquiries or requests pertaining to dataset access, please direct them to 'https://opendata.hira.or.kr/home.do.

Ethics statement

The studies involving humans were approved by Institutional Review Board of Asan Medical Center (IRB No. 2019-1300, 2019-1247). The studies were conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation was not required from the participants or the participants’ legal guardians/next of kin in accordance with the national legislation and institutional requirements.

Author contributions

EK: Supervision, Writing – original draft, Writing – review & editing. Y-JK: Formal analysis, Methodology, Supervision, Visualization, Writing – original draft, Writing – review & editing. JH: Conceptualization, Supervision, Writing – original draft. MK: Formal analysis, Writing – original draft. SL: Conceptualization, Methodology, Writing – review & editing. SS: Conceptualization, Supervision, Writing – review & editing. JM: Data curation, Formal analysis, Writing – original draft. JO: Conceptualization, Funding acquisition, Methodology, Supervision, Writing – review & editing. SP: Conceptualization, Funding acquisition, Methodology, Project administration, Supervision, Writing – original draft, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This research was supported by the Bio & Medical Technology Development Program of the National Research Foundation (NRF) & funded by the Korean government (NRF-2019M3E5D4064636). This research was also supported by a grant of the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (grant number: HR21C0198). This research was also supported by The Technology Innovation Program (20004927, Upgrade of CDM based Distributed Biohealth Data Platform and Development of Verification Technology) funded by the Ministry of Trade, Industry & Energy (MOTIE, Korea). This research was also supported by a grant of the Korean Cancer Survivors Healthcare R&D Project through the National Cancer Center, funded by the Ministry of Health & Welfare, Republic of Korea (grant number: HA23C0460).

Acknowledgments

This study used the research data (M20190916964) provided by the Health Insurance Review and Assessment Service (HIRA).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Author disclaimer

The views expressed are those of the author(s) and not necessarily those of the HIRA and the Ministry of Health and Welfare.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fonc.2023.1295923/full#supplementary-material

References

1. Pardoll DM. The blockade of immune checkpoints in cancer immunotherapy. Nat Rev Cancer. (2012) 12(4):252–64. doi: 10.1038/nrc3239

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Herbst RS, Soria JC, Kowanetz M, Fine GD, Hamid O, Gordon MS, et al. Predictive correlates of response to the anti-PD-L1 antibody MPDL3280A in cancer patients. Nature (2014) 515(7528):563–7. doi: 10.1038/nature14011

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Topalian SL, Sznol M, McDermott DF, Kluger HM, Carvajal RD, Sharfman WH, et al. Survival, durable tumor remission, and long-term safety in patients with advanced melanoma receiving nivolumab. J Clin Oncol (2014) 32(10):1020–30. doi: 10.1200/JCO.2013.53.0105

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Postow MA, Sidlow R, Hellmann MD. Immune-related adverse events associated with immune checkpoint blockade. N Engl J Med (2018) 378(2):158–68. doi: 10.1056/NEJMra1703481

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Martins F, Sofiya L, Sykiotis GP, Lamine F, Maillard M, Fraga M, et al. Adverse effects of immune-checkpoint inhibitors: epidemiology, management and surveillance. Nat Rev Clin Oncol (2019) 16(9):563–80. doi: 10.1038/s41571-019-0218-0

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Gill J, Prasad V. A reality check of the accelerated approval of immune-checkpoint inhibitors. Nat Rev Clin Oncol (2019) 16(11):656–8. doi: 10.1038/s41571-019-0260-y

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Jardim DL, de Melo Gagliato D, Giles FJ, Kurzrock R. Analysis of drug development paradigms for immune checkpoint inhibitors. Clin Cancer Res (2018) 24(8):1785–94. doi: 10.1158/1078-0432.CCR-17-1970

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Salem JE, Manouchehri A, Moey M, Lebrun-Vignes B, Bastarache L, Pariente A, et al. Cardiovascular toxicities associated with immune checkpoint inhibitors: an observational, retrospective, pharmacovigilance study. Lancet Oncol (2018) 19(12):1579–89. doi: 10.1016/S1470-2045(18)30608-9

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Ma Z, Pei J, Sun X, Liu L, Lu W, Guo Q, et al. Pericardial toxicities associated with immune checkpoint inhibitors: A pharmacovigilance analysis of the FDA adverse event reporting system (FAERS) database. Front Pharmacol (2021) 12:663088. doi: 10.3389/fphar.2021.663088

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Li H, Sun X, Sun D, Zhao J, Xu Z, Zhao P, et al. Thromboembolic events associated with immune checkpoint inhibitors: A real-world study of data from the food and drug administration adverse event reporting system (FAERS) database. Int Immunopharmacol (2021) 98:107818. doi: 10.1016/j.intimp.2021.107818

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Chen C, Wu B, Zhang C, Xu T. Immune-related adverse events associated with immune checkpoint inhibitors: An updated comprehensive disproportionality analysis of the FDA adverse event reporting system. Int Immunopharmacol. (2021) 95:107498. doi: 10.1016/j.intimp.2021.107498

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Castinetti F. Thyroiditis and immune check point inhibitors: the post-marketing experience using the French National Pharmacovigilance database. Fundam Clin Pharmacol (2019) 33(2):239–40. doi: 10.1111/fcp.12453

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Garon-Czmil J, Petitpain N, Rouby F, Sassier M, Babai S, Yelehe-Okouma M, et al. Immune check point inhibitors-induced hypophysitis: a retrospective analysis of the French Pharmacovigilance database. Sci Rep (2019) 9(1):19419. doi: 10.1038/s41598-019-56026-5

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Choi NK, Chang Y, Kim JY, Choi YK, Park BJ. Comparison and validation of data-mining indices for signal detection: using the Korean national health insurance claims database. Pharmacoepidemiol Drug Saf (2011) 20(12):1278–86. doi: 10.1002/pds.2237

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Pacaci A, Gonul S, Sinaci AA, Yuksel M, Laleci Erturkmen GB. A semantic transformation methodology for the secondary use of observational healthcare data in postmarketing safety studies. Front Pharmacol (2018) 9:435. doi: 10.3389/fphar.2018.00435

PubMed Abstract | CrossRef Full Text | Google Scholar

16. What is a signal? (2022). Available at: https://who-umc.org/signal-work/what-is-a-signal/.

Google Scholar

17. Council for International Organizations of Medical Sciences (CIOMS). CIOMS Cumulative Pharmacovigilance Glossary. Geneva, Switzerland: CIOMS (2021).

Google Scholar

18. Service HIRA. Hira Notification: List of Reimbursable Drugs for Cancer Treatments (2021). Available at: https://www.hira.or.kr/bbsDummy.do?pgmid=HIRAA030023010000&brdScnBltNo=4&brdBltNo=45602&pageIndex=1.

Google Scholar

19. Evans SJ, Waller PC, Davis S. Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiol Drug Saf (2001) 10(6):483–6. doi: 10.1002/pds.677

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Bate A, Lindquist M, Edwards IR, Olsson S, Orre R, Lansner A, et al. A Bayesian neural network method for adverse drug reaction signal generation. Eur J Clin Pharmacol (1998) 54(4):315–21. doi: 10.1007/s002280050466

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Haanen J, Carbonnel F, Robert C, Kerr KM, Peters S, Larkin J, et al. Management of toxicities from immunotherapy: ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up. Ann Oncol (2017) 28(suppl_4):iv119–iv42. doi: 10.1093/annonc/mdx225

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Brahmer JR, Lacchetti C, Schneider BJ, Atkins MB, Brassil KJ, Caterino JM, et al. Management of immune-related adverse events in patients treated with immune checkpoint inhibitor therapy: American society of clinical oncology clinical practice guideline. J Clin Oncol (2018) 36(17):1714–68. doi: 10.1200/JCO.2017.77.6385

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Muir CA, Menzies AM, Clifton-Bligh R, Tsang VHM. Thyroid toxicity following immune checkpoint inhibitor treatment in advanced cancer. Thyroid (2020) 30(10):1458–69. doi: 10.1089/thy.2020.0032

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Luo J, Martucci VL, Quandt Z, Groha S, Murray MH, Lovly CM, et al. Immunotherapy-mediated thyroid dysfunction: genetic risk and impact on outcomes with PD-1 blockade in non-small cell lung cancer. Clin Cancer Res (2021) 27(18):5131–40. doi: 10.1158/1078-0432.CCR-21-0921

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Street S, Chute D, Strohbehn I, Zhao S, Rengarajan M, Faje A, et al. The positive effect of immune checkpoint inhibitor-induced thyroiditis on overall survival accounting for immortal time bias: a retrospective cohort study of 6596 patients. Ann Oncol (2021) 32(8):1050–1. doi: 10.1016/j.annonc.2021.05.357

PubMed Abstract | CrossRef Full Text | Google Scholar

26. von Itzstein MS, Gonugunta AS, Wang Y, Sheffield T, Lu R, Ali S, et al. Divergent prognostic effects of pre-existing and treatment-emergent thyroid dysfunction in patients treated with immune checkpoint inhibitors. Cancer Immunol Immunother (2022) 71(9):2169–81. doi: 10.1007/s00262-022-03151-2

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Mooradian MJ, Nasrallah M, Gainor JF, Reynolds KL, Cohen JV, Lawrence DP, et al. Musculoskeletal rheumatic complications of immune checkpoint inhibitor therapy: A single center experience. Semin Arthritis Rheumatol (2019) 48(6):1127–32. doi: 10.1016/j.semarthrit.2018.10.012

CrossRef Full Text | Google Scholar

28. Schneider BJ, Naidoo J, Santomasso BD, Lacchetti C, Adkins S, Anadkat M, et al. Management of immune-related adverse events in patients treated with immune checkpoint inhibitor therapy: ASCO guideline update. J Clin Oncol (2021) 39(36):4073–126. doi: 10.1200/JCO.21.01440

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Farooq MZ, Aqeel SB, Lingamaneni P, Pichardo RC, Jawed A, Khalid S, et al. Association of immune checkpoint inhibitors with neurologic adverse events: A systematic review and meta-analysis. JAMA Netw Open (2022) 5(4):e227722. doi: 10.1001/jamanetworkopen.2022.7722

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Xing P, Zhang F, Wang G, Xu Y, Li C, Wang S, et al. Incidence rates of immune-related adverse events and their correlation with response in advanced solid tumours treated with NIVO or NIVO+IPI: a systematic review and meta-analysis. J Immunother Cancer (2019) 7(1):341. doi: 10.1186/s40425-019-0779-6

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Su Q, Zhu EC, Wu JB, Li T, Hou YL, Wang DY, et al. Risk of pneumonitis and pneumonia associated with immune checkpoint inhibitors for solid tumors: A systematic review and meta-analysis. Front Immunol (2019) 10:108. doi: 10.3389/fimmu.2019.00108

PubMed Abstract | CrossRef Full Text | Google Scholar

32. de Filette JMK, Pen JJ, Decoster L, Vissers T, Bravenboer B, van der Auwera BJ, et al. Immune checkpoint inhibitors and type 1 diabetes mellitus: a case report and systematic review. Eur J Endocrinol (2019) 181(3):363–74. doi: 10.1530/EJE-19-0291

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Kotwal A, Haddox C, Block M, Kudva YC. Immune checkpoint inhibitors: an emerging cause of insulin-dependent diabetes. BMJ Open Diabetes Res Care (2019) 7(1):e000591. doi: 10.1136/bmjdrc-2018-000591

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Wright LA, Ramon RV, Batacchi Z, Hirsch IB. Progression to insulin dependence post-treatment with immune checkpoint inhibitors in pre-existing type 2 diabetes. AACE Clin Case Rep (2017) 3(2):e153–e157. doi: 10.4158/EP161303.CR

CrossRef Full Text | Google Scholar

35. Lee M, Jeong K, Park YR, Rhee Y. Increased risk of incident diabetes after therapy with immune checkpoint inhibitor compared with conventional chemotherapy: A longitudinal trajectory analysis using a tertiary care hospital database. Metabolism (2023) 138:155311. doi: 10.1016/j.metabol.2022.155311

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Lupu J, Pages C, Laly P, Delyon J, Laloi M, Petit A, et al. Transient pituitary ACTH-dependent Cushing syndrome caused by an immune checkpoint inhibitor combination. Melanoma Res (2017) 27(6):649–52. doi: 10.1097/CMR.0000000000000405

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Paepegaey AC, Dot JM, Beauvy J, Juttet P, Le Berre JP. Pembrolizumab-induced cyclic ACTH-dependent Cushing’s syndrome treated by a block-and-replace approach with osilodrostat. Ann Endocrinol (Paris) (2022) 83(1):73–5. doi: 10.1016/j.ando.2021.11.007

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Wang TF, Khorana AA, Carrier M. Thrombotic complications associated with immune checkpoint inhibitors. Cancers (Basel) (2021) 13(18):4606. doi: 10.3390/cancers13184606

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Moik F, Chan W-SE, Wiedemann S, Hoeller C, Tuchmann F, Aretin M-B, et al. Incidence, risk factors, and outcomes of venous and arterial thromboembolism in immune checkpoint inhibitor therapy. Blood (2021) 137(12):1669–78. doi: 10.1182/blood.2020007878

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Sussman TA, Li H, Hobbs B, Funchain P, McCrae KR, Khorana AA. Incidence of thromboembolism in patients with melanoma on immune checkpoint inhibitor therapy and its adverse association with survival. J Immunother Cancer (2021) 9(1):e001719. doi: 10.1136/jitc-2020-001719

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Cochain C, Chaudhari SM, Koch M, Wiendl H, Eckstein HH, Zernecke A. Programmed cell death-1 deficiency exacerbates T cell activation and atherogenesis despite expansion of regulatory T cells in atherosclerosis-prone mice. PloS One (2014) 9(4):e93280. doi: 10.1371/journal.pone.0093280

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Bu DX, Tarrio M, Maganto-Garcia E, Stavrakis G, Tajima G, Lederer J, et al. Impairment of the programmed cell death-1 pathway increases atherosclerotic lesion development and inflammation. Arterioscler Thromb Vasc Biol (2011) 31(5):1100–7. doi: 10.1161/ATVBAHA.111.224709

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Del Prete G, De Carli M, Lammel RM, D’Elios MM, Daniel KC, Giusti B, et al. Th1 and Th2 T-helper cells exert opposite regulatory effects on procoagulant activity and tissue factor production by human monocytes. Blood (1995) 86(1):250–7. doi: 10.1182/blood.V86.1.250.bloodjournal861250

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Horio Y, Takamatsu K, Tamanoi D, Sato R, Saruwatari K, Ikeda T, et al. Trousseau’s syndrome triggered by an immune checkpoint blockade in a non-small cell lung cancer patient. Eur J Immunol (2018) 48(10):1764–7. doi: 10.1002/eji.201847645

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Roopkumar J, Swaidani S, Kim AS, Thapa B, Gervaso L, Hobbs BP, et al. Increased incidence of venous thromboembolism with cancer immunotherapy. Med (N Y) (2021) 2(4):423–34. doi: 10.1016/j.medj.2021.02.002

CrossRef Full Text | Google Scholar

46. Engelmann B, Massberg S. Thrombosis as an intravascular effector of innate immunity. Nat Rev Immunol (2013) 13(1):34–45. doi: 10.1038/nri3345

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Moseley KF, Naidoo J, Bingham CO, Carducci MA, Forde PM, Gibney GT, et al. Immune-related adverse events with immune checkpoint inhibitors affecting the skeleton: a seminal case series. J Immunother Cancer (2018) 6(1):104. doi: 10.1186/s40425-018-0417-8

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Filippini DM, Gatti M, Di Martino V, Cavalieri S, Fusaroli M, Ardizzoni A, et al. Bone fracture as a novel immune-related adverse event with immune checkpoint inhibitors: Case series and large-scale pharmacovigilance analysis. Int J Cancer (2021) 149(3):675–83. doi: 10.1002/ijc.33592

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Liu S, Mi J, Liu W, Xiao S, Gao C. Blocking of checkpoint receptor PD-L1 aggravates osteoarthritis in macrophage-dependent manner in the mice model. Int J Immunopathol Pharmacol (2019) 33:2058738418820760. doi: 10.1177/2058738418820760

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Merlijn T, Swart KMA, van der Horst HE, Netelenbos JC, Elders PJM. Fracture prevention by screening for high fracture risk: a systematic review and meta-analysis. Osteoporos Int (2020) 31(2):251–7. doi: 10.1007/s00198-019-05226-w

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Kanis JA. Diagnosis of osteoporosis and assessment of fracture risk. Lancet (2002) 359(9321):1929–36. doi: 10.1016/S0140-6736(02)08761-5

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Cathcart-Rake EJ, Sangaralingham LR, Henk HJ, Shah ND, Riaz IB, Mansfield AS. A population-based study of immunotherapy-related toxicities in lung cancer. Clin Lung Cancer (2020) 21(5):421–7 e2. doi: 10.1016/j.cllc.2020.04.003

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Dudnik E, Moskovitz M, Rottenberg Y, Lobachov A, Mandelboim R, Shochat T, et al. Pembrolizumab as a monotherapy or in combination with platinum-based chemotherapy in advanced non-small cell lung cancer with PD-L1 tumor proportion score (TPS) >/=50%: real-world data. Oncoimmunology (2021) 10(1):1865653. doi: 10.1080/2162402X.2020.1865653

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Park G, Jung H, Heo SJ, Jung I. Comparison of data mining methods for the signal detection of adverse drug events with a hierarchical structure in postmarketing surveillance. Life (Basel) (2020) 10(8):138. doi: 10.3390/life10080138

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Larkin J, Chiarion-Sileni V, Gonzalez R, Grob JJ, Cowey CL, Lao CD, et al. Combined nivolumab and ipilimumab or monotherapy in untreated melanoma. N Engl J Med (2015) 373(1):23–34. doi: 10.1056/NEJMoa1504030

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Weber JS, Kahler KC, Hauschild A. Management of immune-related adverse events and kinetics of response with ipilimumab. J Clin Oncol (2012) 30(21):2691–7. doi: 10.1200/JCO.2012.41.6750

PubMed Abstract | CrossRef Full Text | Google Scholar