Skip to main content

ORIGINAL RESEARCH article

Front. Oncol., 05 March 2021
Sec. Women's Cancer

Immune Modeling Analysis Reveals Immunologic Signatures Associated With Improved Outcomes in High Grade Serous Ovarian Cancer

\nNicole E. JamesNicole E. James1Katherine Miller,Katherine Miller1,2Natalie LaFranzoNatalie LaFranzo3Erin Lips,Erin Lips1,2Morgan WoodmanMorgan Woodman1Joyce Ou,Joyce Ou2,4Jennifer R. Ribeiro,
Jennifer R. Ribeiro1,2*
  • 1Program in Women's Oncology, Department of Obstetrics and Gynecology, Women and Infants Hospital of Rhode Island, Providence, RI, United States
  • 2Department of Obstetrics and Gynecology, Warren-Alpert Medical School of Brown University, Providence, RI, United States
  • 3Cofactor Genomics, San Francisco, CA, United States
  • 4Department of Pathology, Women and Infants Hospital of Rhode Island, Providence, RI, United States

Epithelial ovarian cancer (EOC) is the most lethal gynecologic malignancy worldwide, as patients are typically diagnosed at a late stage and eventually develop chemoresistant disease following front-line platinum-taxane based therapy. Only modest results have been achieved with PD-1 based immunotherapy in ovarian cancer patients, despite the fact that immunological responses are observed in EOC patients. Therefore, the goal of this present study was to identify novel immune response genes and cell subsets significantly associated with improved high grade serous ovarian cancer (HGSOC) patient prognosis. A transcriptomic-based immune modeling analysis was employed to determine levels of 8 immune cell subsets, 10 immune escape genes, and 22 co-inhibitory/co-stimulatory molecules in 26 HGSOC tumors. Multidimensional immune profiling analysis revealed CTLA-4, LAG-3, and Tregs as predictive for improved progression-free survival (PFS). Furthermore, the co-stimulatory receptor ICOS was also found to be significantly increased in patients with a longer PFS and positively correlated with levels of CTLA-4, PD-1, and infiltration of immune cell subsets. Both ICOS and LAG-3 were found to be significantly associated with improved overall survival in The Cancer Genome Atlas (TCGA) ovarian cancer cohort. Finally, PVRL2 was identified as the most highly expressed transcript in our analysis, with immunohistochemistry results confirming its overexpression in HGSOC samples compared to normal/benign. Results were corroborated by parallel analyses of TCGA data. Overall, this multidimensional immune modeling analysis uncovers important prognostic immune factors that improve our understanding of the unique immune microenvironment of ovarian cancer.

Introduction

Epithelial Ovarian Cancer (EOC), is the most lethal of all gynecological malignancies, with approximately 21,750 women diagnosed, and 13,940 deaths from the disease in the United States in 2020 (1). The primary treatment regimen for advanced stage EOC consists of debulking surgery and platinum-taxane-based chemotherapy, with around 75% of patients achieving remission. However, a majority of patients experience recurrence within months to a few years, at which point the treatment response to traditional chemotherapy is low (2). To combat this chemoresistance, there has been a heavy focus on targeted therapies such as anti-angiogenic and poly (ADP-ribose) polymerase (PARP) inhibitors, which have shown promising results when used for maintenance or recurrent disease therapy. However, significant challenges still exist in producing long-term outcomes, with prognosis for advanced stage patients remaining poor (37).

Currently, several EOC clinical trials center upon immunotherapy (3). While melanoma and non-small cell lung cancer (NSCLC) patients have seen a great benefit from targeting immune checkpoint inhibitors such cytotoxic T lymphocyte-associated protein-4 (CTLA-4) or programmed cell death protein 1 (PD-1), EOC patients have only exhibited response rates of 10–15% (3). Despite these poor response rates, evidence suggests that EOC is an immunogenic cancer. It has been well-established that patients with a higher number of intratumoral T cells achieve a longer progression-free survival (PFS) and overall survival (OS) as compared to those with lower numbers (8). In addition, antigen-specific antibodies and tumor-reactive T cells have been detected in ovarian cancer (9). Thus, while evidence suggests that EOC patients could benefit from some form of immunotherapy, further research is needed in order to improve response rates.

While immune signatures may not be highly useful as predictive markers for immunotherapy in EOC, given the overall low response rates to currently studied PD-1 therapies, immune-related gene expression profiles may serve as prognostic indicators of response to frontline chemotherapy. Moreover, gaining an understanding of which immunological features in a tumor lead to improved outcomes could reveal opportunities for novel immunotherapeutic approaches. In ovarian cancer, studies have begun to explore the relationship between immune composition, immune-related genes, and clinical outcomes. Several studies have used computational analysis of tumors with publicly available gene expression data to report relationships between immune cell subsets or gene expression and prognosis (1013). Yet other studies have employed immunohistochemical approaches to establish a relationship between intra-epithelial T cells and other immune cell subsets and survival (14, 15). However, a complete understanding of how the immune microenvironment responds to chemotherapy and how these changes relate to clinical outcomes is lacking.

In this present study, we performed a targeted transcriptomic analysis of immune cell content and coinhibitory and costimulatory receptor genes and their respective ligands in a cohort of 26 high grade serous ovarian cancer (HGSOC) tumors. This multidimensional analysis identified a novel combination of immune response genes and subsets that are associated with patient outcomes, and revealed unique correlations between specific immune genes in HGSOC. Specific genes were further analyzed using The Cancer Genome Atlas (TCGA) ovarian cancer cohort. These results highlight the potential utility of novel immune markers as prognostic factors or to inform future therapeutic strategies.

Materials and Methods

Ovarian Cancer Tissue

Stage IIIC, grade 3 serous ovarian cancer patients were selected based on duration of PFS (time from completing chemotherapy to first recurrence). One patient in the long PFS group had Stage IV disease that was only designated as Stage IV due to pelvic lymph node spread, which was easily resected. Eleven patients had a PFS ≥ 54 months (range 54–99+); eight patients had a PFS ≤ 9 months (range 0–9); and seven had an intermediate PFS (range 16–47). The median age of the patients in the long PFS group was 55 (range 42–91) and the median age was 67.5 (range 45–79) for the short PFS group (p = 0.1226). The median CA125 value for patients in the long PFS group was 155, and 1,808 for the short PFS group (p = 0.0346). Median HE4 levels were not calculated because only ten patients out of the 19 had known HE4 levels. Some patients were treated with maintenance bevacizumab, as noted (Table 1). Formalin-fixed, paraffin embedded (FFPE) tissue sections were prepared from each patient's residual tissue block from their primary debulking surgery. All tumors were naïve to chemotherapy. Before submission, the tissue was analyzed by a pathologist at Women & Infants Hospital to determine that they met minimum guidelines for cellularity (≥80%) and viability (≥20%). All tissue was obtained and data managed under The Women and Infants Hospital Institutional Review Board approval of protocol #1326537.

TABLE 1
www.frontiersin.org

Table 1. Patient information.

Cofactor ImmunoPrism® Assay

All stages of the assay were performed in a CAP-accredited, CLIA-licensed clinical laboratory. Schillebeeckx et al. (16) describes the machine learning approach used to generate “health expression models” for accurately identifying percentages of immune cell populations in tumor tissue. When developing the immune Health Expression Models used in the ImmunoPrism assay, all models were built using purified immune cell populations, and validated using flow cytometry. For example, the surface markers used to identify Tregs (CD4+/CD25+/CD127low/–/CCR4+) were based on those used by the Human Immunophenotyping Consortium (17). The CD4+/CD25+/CD127low/– subpopulation of cells has been shown to be significantly enriched for Foxp3 expression compared to the overall CD4+/CD25+ population, and additionally display higher suppressive capability (18). We therefore expect the CD4+/CD25+/CD127low/–/CCR4+ cells isolated to be significantly enriched for Tregs. For macrophages, a serum-free in vitro differentiation model was used to generate M1 and M2 macrophages from peripheral blood monocytes. M1 macrophages were derived by activation with IFN-gamma and LPS, and M2 macrophages were derived by activation with IL-4, as described in The Biology of Cancer by Robert A. Weinberg (second edition) (19). Populations were confirmed by flow cytometry using the following markers: M1 macrophages are CD80+/CCR7+/CD206 low/CD209 low and M2 macrophages are CD206+/CD209+/CD80 low/CCR7 low.

LaFranzo et al. describes in detail the Cofactor Genomics predictive immune modeling workflow (16, 20), which is summarized as follows:

RNA Extraction: Unstained, unmounted FFPE sections from the same FFPE block were processed for RNA extraction using the Cofactor Prism Extraction Kit, following the manufacturer's suggested protocol.

Total RNA Quality Control: Unstained, unmounted FFPE sections from the same FFPE block were processed for RNA extraction using the Prism Extraction Kit (Cofactor Genomics, San Francisco, CA). Total RNA was evaluated for quality and quantity using the Bioanalyzer or TapeStation assay (Agilent, Santa Clara, CA), and the Qubit RNA HS or BR Assay (ThermoFisher, Waltham, MA). RNA concentration and quantity (in ng/μL and total ng) and quality (DV200, % of fragments above 200 nt) was evaluated to determine library input amount.

ImmunoPrism Library Preparation and Sequencing: Total RNA was processed for library construction by Cofactor Genomics according to the ImmunoPrism assay standard protocol for FFPE materials; 20–100 ng of RNA was used as input depending on sample quality. Libraries were sequenced as single-end 75 base pair reads on a NextSeq500 (Illumina, San Diego, CA) following the manufacturer's protocols.

ImmunoPrism Analysis: An individual ImmunoPrism report including expression characterization and immune cell quantification was provided for each sample processed. Samples were grouped according to clinical meta and outcomes data to generate ImmunoPrism Biomarker Reports, as appropriate.

Single analyte biomarkers were created using a linear threshold approach. All possible thresholds were considered and the threshold that jointly optimizes the leave-one-out specificity and sensitivity was chosen for each respective single analyte. This approach converges quickly to the population accuracy, even in small sample sizes, and is robust to class imbalance.

The multidimensional biomarker was created using machine learning to simultaneously consider multiple readouts from the ImmunoPrism assay. The process involves two steps: first, it chooses the highest performing combination of analytes, and then second, it chooses the highest performing machine learning hyperparameters to train the final model.

In the research-use-only version of the software, the top five analytes are considered, sorted by accuracy as an individual predictive readout and then by p-value. Next, all power set combinations of these five analytes were used to train Random Forest models with default hyperparameters. The analytes corresponding to the model with the lowest leave-one-out error were considered for the next step. In cases of ties, the model with more analytes was chosen. Using this final set of analytes, a grid search optimization over all hyperparameter values is performed. The model with the lowest leave-one-out error was used as the final model. The model's predictions of the left out samples were used to evaluate the performance of the final model.

Immunohistochemistry

FFPE human ovarian tissue slides were baked at 2 h at 65°C. Slides were then washed in xylene, 100, 95, 70% ethanol, deoxygenated water, and FTA Hemagglutination Buffer. Antigen retrieval was performed using Antigen Retrieval Solution (1X) (Vector Laboratories, H-3300) and heated to 95°C for 20 min. Slides were then blocked with 5% horse serum in FTA Hemagglutination buffer and incubated overnight in PVRL2/Nectin-2 antibody (Cell Signaling, 95333S, 1:100) at 4°C. Anti-rabbit IgG Dylight 488 secondary antibody (Vector Laboratories, DI-488, 1:1000) was then applied to slides following incubation in the dark at room temperature for 1 h. Slides were washed between each step using FTA Hemagglutination buffer and cover-slipped with DAPI containing mounting medium (Vector Laboratories, H-1200).

Image Analysis

Three randomly selected fields per case were selected based on DAPI staining and acquired with a spinning disk confocal Nikon Eclipse Ti microscope at 10x objective. Image analysis was performed on grayscale 8-bit images in ImageJ. Images were thresholded for specific staining and mean and maximum intensity was calculated. Representative images were taken using confocal microscopy at 20x objective.

The Cancer Genome Atlas

The ovarian cancer dataset (TCGA-OV) with complete RNA-sequencing results (n = 378) from The Cancer Genome Atlas was obtained using GenomicDataCommons (version 1.12.0) and RStudio (R version 4.0.0) (21, 22). Fragments Per Kilobase of transcript per Million mapped reads (FPKM) values, vital status, and days to death/follow-up were obtained for correlation and survival analyses.

Timer

TIMER2.0 (https://timer.cistrome.org) (2325) was used to determine percentages of immune cells in TCGA-OV samples. CIBERSORT was used as the immune deconvolution method.

cBioPortal

Mutation count data and RNA seq (V2 RSEM) data for TCGA-OV (Firehose Legacy) was downloaded from cBioPortal (https://cbioportal.org) (26, 27). Data was available for 185 samples.

Statistics

The statistics reported for the ImmunoPrism assay were produced via leave-one-out cross validation. For a dataset limited in size, leave-one-out cross validation gives the best approximation to how an estimator will generalize to future, independent samples. The process works by iterating n times (where there are n datapoints), each time learning a threshold considering n-1 points and testing the prediction of the nth, left out, point. Then, all n predictions are considered to calculate prediction statistics.

Wilcoxon Rank Sum Test was used to determine differences in median values for age and CA125 levels. Cuzick trend test was used to evaluate differences in medians between “long”, “intermediate”, and “short” PFS groups. Kaplan-Meier curves were generated for TCGA survival data, and log-rank p-values along with hazard ratios and 95% confidence intervals were determined. Spearman rank correlation was used to determine correlations between genes and cell types. Two-tailed unpaired t-test was used to determine differences in gene expression levels between mutation count groups. Statistical analyses were performed in GraphPad Prism and R version 4.0.3. p < 0.05 was considered significant.

Results

Prognostic Biomarker Status in Epithelial Ovarian Cancer Tissue

Twenty-six formalin fixed paraffin-embedded (FFPE) samples underwent ImmunoPrism immune profiling analysis (Cofactor Genomics, San Francisco, CA). Clinical data can be seen in Table 1. Nineteen samples that were stratified by PFS were included in the biomarker analysis to determine differences in a panel of immune genes and immune cell subtypes between the two groups (short vs. long PFS). Multidimensional immune profiling analysis revealed the combination of CTLA-4, LAG-3, and Tregs was significantly higher in patients with improved patient prognosis (Figure 1). Furthermore, individual analyte assessment revealed that LAG-3, CTLA-4, ICOS, and TNFRSF18 transcripts were significantly more abundant in the long PFS group compared to the short (p < 0.05). PD-1 also demonstrated a strong trend toward higher transcripts per million (TPM) in the long PFS group than in the short (p = 0.0575). The relationships between individual immune cell types and PFS did not reach the level of significance. Thresholds were determined for each cell type or gene, along with accuracy, positive predictive value (PPV), negative predictive value (NPV), sensitivity, and specificity (Table 2). RNA seq data is available through ArrayExpress (https://www.ebi.ac.uk/arrayexpress/) under accession number E-MTAB-9743.

FIGURE 1
www.frontiersin.org

Figure 1. ImmunoPrism® Multidimensional Biomarker Assessment of high grade serous ovarian cancer. (A) The most predictive marker from immune cell types (“immune health expression models”) and each type of immune gene were less accurate than a multidimensional biomarker comprising LAG-3, CTLA-4, and Tregs at predicting length of progression free survival. (B) The distributions of individual samples from both cohorts were evaluated for future performance using the multidimensional biomarker as the classifier. Multidimensional values are plotted along the x-axis, and the frequency of sample values are visualized in the vertical space. The sample distribution visualization is approximate. The cohorts are distinguished by color (green and blue) and shape (circle and square), and estimations are distinguished by outline color (white: correct estimation, yellow: incorrect estimation.) The dashed line at value = 0 indicates the prediction threshold that separates the cohorts. Samples that were incorrectly estimated are highlighted in yellow, and listed in Supplementary Table 2. (C) A Receiver Operating Characteristic (ROC) Curve was generated for the multidimensional biomarker. For various thresholds of the biomarker, the True Positive Rate (y-axis) is plotted against the False Positive Rate (x-axis). The area under the curve (AUC) is included in the lower right corner. The random predictor is shown as a dashed red line. (D) A confusion matrix was generated for the multidimensional biomarker. This matrix was used to calculate positive predictive value (PPV), negative predictive value (NPV), specificity, and sensitivity of the multidimensional biomarker by comparing how the samples are classified based on the assay (rows) vs. their known label based on clinical data (28). The incorrectly estimated samples are the same as those noted and labeled in the Multidimensional Biomarker Assessment.

TABLE 2
www.frontiersin.org

Table 2. Analyte table displaying cell type percentages and TPM values for all genes analyzed in the ImmunoPrism assay.

Relationship of Immune Genes and Cell Types With Progression-Free Survival

Next, we went on to analyze the entire cohort of samples, including those with short, intermediate, and long PFS. There were significant trends observed in the analysis of LAG-3 (p = 0.0016), ICOS (p = 0.0381), TNFRSF18 (p = 0.0354), PD-1 (p = 0.0474), and CTLA-4 (p = 0.0410) (Figure 2A). No significant trends were observed with any of the cell types and PFS (Supplementary Table 1). Interestingly, there was no correlation between “total immune” and PFS (Figure 2B), suggesting that patients with longer PFS don't necessarily have a larger total immune infiltrate.

FIGURE 2
www.frontiersin.org

Figure 2. Relationship of immune genes and cell types with progression-free survival. The entire 26 patient cohort, including patients with short, intermediate, and long PFS were analyzed. (A) Values were plotted for each group (horizontal bar indicates median). Cuzick trend test was used to determine significant trends in the three groups. The horizontal variation in each group is a random jitter and not the actual PFS values. (B) “Total immune” percentages were correlated with PFS by Spearman rank correlation.

Relationship of Top Differentially Expressed Genes With Overall Survival

We analyzed The Cancer Genome Atlas ovarian cancer cohort (TCGA-OV), and generated Kaplan-Meier curves for overall survival for each of the top differentially expressed genes (Figure 3A). In this larger, somewhat less homogeneous cohort, patients with higher levels of LAG-3 and ICOS had improved OS, with an HR of 0.6820 [0.4724–0.9846] (p = 0.0424) for LAG-3 and an HR of 0.5832 [0.4026–0.8448] (p = 0.0195) for ICOS. No significant differences were determined for the other candidate genes. Small differences in mean LAG-3 (p = 0.0371) and ICOS (p = 0.0799) mRNA transcript levels were observed in patients with low vs. high mutation counts, although this difference was not significant for ICOS (Figure 3B).

FIGURE 3
www.frontiersin.org

Figure 3. LAG-3 and ICOS are associated with overall survival and mutation counts in The Cancer Genome Atlas ovarian cancer cohort. (A) Kaplan-Meier curves for LAG-3, ICOS, CTLA-4, PD-1, PD-L1, and TNFRSF18 were generated using TCGA-OV dataset. The top and bottom quartiles of expression (n = 95/group) were used to define “high” vs. “low” expressing groups. Log-rank hazard ratios (HR) and p-values are reported, with 95% confidence intervals in parentheses. (B) TCGA-OV dataset was separated into low (8–43) and high (43–158) mutation count groups by median mutation count. mRNA expression for each gene (RNA seq V2 RSEM) was determined for each group, with the black bar indicating median mRNA level for each group. P-value was determined by two-tailed, unpaired student t-test.

Correlation of Top Differentially Expressed Genes Between Each Other and With Immune Cell Subsets

The top differentially expressed genes were correlated amongst each other and with immune cell subsets. No significant correlations emerged for LAG-3 with the rest of the top differentially expressed immune genes. Conversely, there were strong and significant correlations between ICOS, TNFRSF18, PD-1, PD-L1, and CTLA-4 amongst each other (r > 0.5, p < 0.001). Likewise, similar correlations between immune genes were observed in TCGA data, except LAG-3 also emerged as significantly correlated, although the strength of LAG-3 correlations was weaker than for ICOS (p < 0.0001 for all correlations) (Figures 4A,C–F). Because of reported relationships between PD-1 and LAG-3, as well as between ICOS and CTLA-4, we further examined these relationships in our cohort vs. TCGA. We observed a similar “fan” pattern in the correlation of LAG-3 and PD-1 in our cohort vs. TCGA, although the relationship between these two genes in TCGA dataset emerged as statistically significant (r = 0.5971, p < 0.0001), likely due to the large number of samples in the TCGA dataset (Figures 4C,D). We also observed a strong correlation between ICOS and CTLA-4 in both our cohort (r = 0.8913, p < 0.0001) and the TCGA ovarian cancer cohort (r = 0.9093, p < 0.0001) (Figures 4E,F).

FIGURE 4
www.frontiersin.org

Figure 4. Correlation analysis of immune response genes and individual immune cell subsets. (A) Spearman rank correlation analysis was used to determine the relationship between the top differentially expressed genes among each other in all 26 patient samples in our cohort (WIHRI), as well as in TCGA-OV cohort. Heat maps illustrating the strength of the correlations (r values) were generated. (B) Spearman rank correlation analysis was used to determine the relationship between immune co-receptors and immune cell subsets in the WIHRI cohort (left) and TCGA-OV cohort, using CIBERSORT data (right). (C,D) Spearman rank correlation analysis of PD-1 and LAG-3 in the WIHRI and TCGA cohort. (E,F) Spearman rank correlation analysis of CTLA-4 and ICOS in the WIHRI cohort and TCGA cohort. Spearman r and p-values are indicated on each graph.

We then performed correlations between the top differentially expressed genes and each immune cell subset in our cohort. There were no significant correlations between LAG-3 and any of the immune cell subtypes, while ICOS, TNFRSF18, PD-1, PD-L1, and CTLA-4 all significantly correlated (p < 0.05) to CD8, CD4, and Tregs, with the exception of TNFRSF18 and Tregs. There was a trend toward inverse correlation with all genes to CD14+ monocytes, but only PD-L1 significantly inversely correlated (p = 0.0230). No immune genes were significantly correlated to CD56+ NK cells or M2 macrophages. ICOS, TNFRSF18, PD-1, PD-L1, and CTLA-4 all significantly positively correlated to M1 macrophages (p < 0.05). ICOS and CTLA-4 were both significantly associated with CD19+ B cells (p < 0.05). The pattern of correlations between immune genes and cell types was strikingly similar when analyzing TCGA data, using TIMER (CIBERSORT) to generate correlations between immune cell composition and specific immune genes, although the strength of the correlations was overall weaker (Figure 4B).

PVRL2 Is the Most Highly Expressed Transcript in Ovarian Cancer Tissue and Is Upregulated in Ovarian Cancer Compared to Normal/Benign

PVRL2 was found to be the most abundantly expressed transcript out of all immune response genes and ligands within the modeling analysis. Interestingly, when we compared mean expression of PVRL2 to PD-L1, the most commonly studied immune ligand in EOC, we observed over 44-fold higher levels of PVRL2. This was also observed in TCGA-OV dataset, with 82-fold higher levels of PVRL2 compared to PD-L1 (Figures 5A,B). Next, we examined the correlation of PVRL2 to immune cell subsets and found that PVRL2 expression significantly inversely correlated to CD4+ T cells, CD19+ B cells, and M2 macrophages (p < 0.05) (Figure 5C). Examining PVRL2's expression with the top regulated immune response genes revealed a significant inverse relationship with PD-1, ICOS and CTLA-4, and a positive correlation with LAG-3 (p < 0.05 for all). The trend toward negative correlations between PVRL2 and immune genes and cell types was also observed for TCGA data, although only the correlations of PVRL2 with CD8+ T cells and M2 macrophages were significant (p < 0.05). These data sharply contrast with data for PD-L1, which was strongly and significantly correlated with all top immune response genes except LAG-3, as shown in Figure 4A. Next, we performed PVRL2 fluorescent immunohistochemistry on benign/normal (n = 5) and HGSOC (n = 15) samples. Mean and maximum intensity was 1.6-fold and 2.4-fold greater, respectively, in HGSOC compared to normal/benign tissue (p = 0.0021, p = <0.00001) (Figures 5D–F), which is in agreement with published studies (29, 30).

FIGURE 5
www.frontiersin.org

Figure 5. PVRL2 overexpression in high grade serous ovarian cancer. (A,B) Transcript levels of PVRL2 vs. PD-L1 in our cohort and TCGA cohort. p-values were determined by two-tailed paired student t-test. (C) Heat map of Spearman rank correlations between PVRL2 and individual immune cell subsets and top differentially expressed immune response genes in WIHRI and TCGA cohorts. Spearman r values are indicated by color. (D) Representative immunohistochemistry staining of PVRL2 in HGSOC and normal/benign ovaries. Negative control is secondary antibody alone. (E,F) Mean and maximum PVRL2 intensity (pixels) in serous ovarian cancer vs. normal/benign tissue.

Discussion

This multidimensional immune modeling analysis revealed that a signature of CTLA-4, LAG-3, and Tregs was significantly higher in patients with improved clinical outcomes in a subset of patients with HGSOC. While further studies are needed to make any firm conclusion about this multidimensional signature, this study serves as proof-of-principle that immunological signatures can be prognostically informative, and also serves to highlight potential unique immunological responses in ovarian cancer patients.

In addition to PD-1, CTLA-4 is one of the most clinically studied immune checkpoint receptors in EOC, with six open Phase I/II clinical trials evaluating the combination of anti-PD-1/PD-L1 and CTLA-4 therapy (31). While the clinical significance of targeting CTLA-4 has been well-studied, its prognostic value has not previously been investigated in HGSOC. Likewise, LAG-3, whose transcript level we also found to be higher in patients with longer PFS, is a co-inhibitory receptor that has received more attention recently in EOC. Preclinical studies have demonstrated the efficacy of targeting LAG-3 in combination with immune checkpoint receptors in EOC, and Phase I/II development of a LAG-3 monoclonal antibody is currently being investigated in pancreatic, breast, melanoma, and other solid tumors, including ovarian cancer (32). While studies have reported no association between patient outcomes and LAG-3 expression in ascites or residual patient tumors after neoadjuvant chemotherapy (32, 33), the relationship between LAG-3 expression in naïve tumors and clinical outcomes has not been previously studied.

Tregs are known to play a particularly important role in ovarian cancer pathogenesis. Intriguingly, it has been reported that response to immunotherapy is exceptionally challenging due to the fact that EOC tumors exhibit elevated levels of highly activated Tregs (31), with a distinct phenotype exhibiting higher levels of FOXP3, PD-1, 4-1BB, and ICOS compared to melanoma tumoral Tregs, while also displaying increased suppressive capabilities against cytotoxic T cell proliferation (34). Despite Tregs contributing to challenges with immunotherapy response in EOC, our study identified Tregs as part of the multidimensional biomarker for the long PFS group of patients. Contradictory reports exist on the prognostic significance of Tregs in ovarian cancer, with one study reporting a positive association between levels of Tregs in primary and metastatic lesions and patient outcomes, and another determining a negative association between Tregs and survival in all stages of disease (35, 36). While further studies will be required to understand these discrepant results, the outcomes from our study suggest that Tregs may be more abundant in a subset of patients with a longer PFS, although further studies in larger cohorts will be needed to fully evaluate this hypothesis. This observation is in agreement with the fact that Tregs tend to infiltrate proportionally with effector T cells, which are known to be prognostically favorable in ovarian cancer (8, 14, 15). One major caveat to consider in making conclusions regarding the prognostic ability of Tregs and effector T cells in ovarian cancer is their localization. Various reports on the effect of intra-tumoral vs. stromal T cell localization exist, with some reporting that intra-tumoral localization improves prognostication, while others determined that averaging effector T cells through the tumor and stroma improves prognostication (14, 15).

While we are the first to report a combined prognostic significance of CTLA-4, LAG-3, and Tregs, a major limitation of our study is the small sample size. Furthermore, all patients in our cohort did not receive the exact same therapy, as some patients were treated with bevacizumab in the maintenance setting. While the effect of the bevacizumab treatment should not be significant enough to change the patients' PFS grouping, in future studies this analysis should be repeated in a larger cohort to allow for separate analysis of patients that have received additional regimens apart from platinum-taxane chemotherapy.

Perhaps one of the most interesting findings from this study is the identification of ICOS's association with improved patient survival in EOC. ICOS appears to be a particularly relevant immune gene in ovarian cancer, as ICOS and LAG-3 were the only genes determined from our multidimensional analysis that were validated independently in TCGA cohort. ICOS is a member of the B7 CD28/CTLA-4 costimulatory family of receptors that play a vital role in T cell immunity (37). As an immune checkpoint agonist, ICOS acts as a counterpart to CD28, strongly enhancing T cell proliferation, survival, and differentiation. ICOS is an interesting co-stimulatory receptor, since its function depends on the T cell subset in which it is predominantly expressed. On Tregs, it may dampen the immune response, while on cytotoxic T cells, it may promote an immune response (38). While ICOS has been studied in other cancers, its role in EOC has yet to be well-elucidated. To date, there have only been two studies that have examined ICOS in the context of EOC, both of which have described its high expression on Tregs. Conrad et al. (39) reported that the majority of FOXP3+ Tregs in the EOC tumor microenvironment expressed ICOS, were responsible for the stimulation of immunosuppressive plasmacytoid dendritic cells, and were associated with disease progression. As stated earlier, Toker et al. (34) also reported that EOC Tregs exhibit a distinct phenotype that have higher levels of ICOS compared to melanoma Tregs, which may explain why melanoma patients have responded well to ICOS agonist interventions as these might primarily act on cytotoxic T cells in melanoma rather than Tregs. There is a need for larger studies to fully elucidate the expression profile of ICOS in EOC, in order to determine if it can be efficaciously targeted.

It is interesting to note that patient PFS did not correlate with total immune infiltrate and was not statistically associated with any particular immune cell infiltrate. However, since ICOS correlated well with T cell infiltrate, it is still possible that ICOS is prognostically favorable because of preferential increases in T cell infiltrates in patients with improved survival. However, LAG-3 did not correlate well with any immune cell populations in our cohort. These findings suggest that increased immune infiltrate alone is probably not responsible for the observed prognostic ability of these T cell co-receptors. Moreover, other T cell co-receptors examined, such as TIM-3 and BTLA, showed no association with survival, suggesting a unique role for ICOS and LAG-3 in HGSOC.

Finally, results from our immune modeling analysis revealed PVRL2 as the most abundantly expressed immune factor. PVRL2 has been previously shown to be overexpressed in ovarian cancer (29, 30, 40), which we confirmed in a small independent cohort. It is a member of the nectin and nectin-like family that contains receptors such as DNAM-1 (CD226), CD96, T cell immunoreceptor with Ig and ITIM domains (TIGIT), and poliovirus receptor related immunoglobin (PVRIG). PVRL2 binds with a higher affinity to PVRIG rather than TIGIT, which favors the ligand poliovirus receptor (PVR) and only weakly binds to PVRL2 (29). Furthermore, it was found that out of all cancer types studied, ovarian cancer exhibited the highest percentage of PVR-PVRL2+ cells (29), emphasizing a specific overexpression of PVRL2 unique to ovarian cancer.

While PVRL2 was universally highly expressed and no significant relationship existed between PVRL2 levels and PFS, results of the above described studies suggest that anti-PVRIG/PVRL2 based therapy has the potential to greatly impact EOC patients whose tumors lack PD-L1 expression. PD-1/PD-L1 targeting has been well-studied in EOC, despite the fact that only 33% of high grade ovarian cancer patients are considered PD-L1 positive and no significant correlation exists between PD-L1 levels and survival (41). Overall, while the PD-1/PD-L1 axis remains the most frequently investigated clinical immunotherapy target in EOC, it is still uncertain which subset of EOC patients will most benefit from this therapy since response rates remain low and expression analysis studies have shown conflicting data regarding the relationship of PD-1/PD-L1 and clinical outcomes (42). Our data reports an over 44-fold higher expression of PVRL2 compared to PD-L1 in our advanced stage cohort, highlighting the importance of further investigation of this pathway in EOC. Overall, the PVRIG/PVRL2 axis represents a potential therapeutic target that may aide in improving EOC patient response to immunotherapy.

Conclusions

Our immune modeling analysis revealed that CTLA-4, LAG-3, and Tregs are more abundant in HGSOC patients with longer PFS. ICOS was also significantly more highly expressed in patients with longer PFS, and strongly correlated with CTLA-4, PD-1, and specific subsets of immune cell infiltration. High ICOS and LAG-3 levels were significantly associated with longer survival in the ovarian cancer TCGA cohort. PVRL2 was the most abundantly expressed transcript in the ImmunoPrism analysis, and was overexpressed in a cohort of HGSOC tissue compared to benign/normal samples. Our findings highlight the idea that immunological signatures are related to patient outcomes in HGSOC.

Data Availability Statement

The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found in the article/Supplementary Material.

Ethics Statement

The studies involving human participants were reviewed and approved by Women & Infants Hospital Institutional Review Board. A waiver of written informed consent was obtained for this study in accordance with the national legislation and the institutional requirements.

Author Contributions

JR, JO, and NJ designed the study. KM, MW, and NJ executed experiments. KM and EL performed chart reviews. JR and NJ performed statistical analyses. JO obtained all samples and assured they met minimum quality standards. NL oversaw the ImmunoPrism assay and analysis and provided figures, tables, and methods pertaining to ImmunoPrism results. JR oversaw all aspects of the study. JR, NJ, and KM prepared the manuscript, which was edited by NL. All authors reviewed and approved the final version.

Funding

Research reported in this publication was supported by the Constance A. Howe's Women's Health Innovation Fund, the Rhode Island Hospital Center for Cancer Research Development COBRE Pilot Award supported by an Institutional Development Award (IDeA) from the National Institute of General Medical Sciences of the National Institutes of Health under Award Number 5P30GM110759; the Division of Gynecologic Oncology, Program in Women's Oncology at Women & Infants Hospital; Swim Across America; the Kilguss Research Core of Women & Infants Hospital supported by an Institutional Development Award (IDeA) from the National Institute of General Medical Sciences of the National Institutes of Health under grant number P30GM114750; and the COBRE Center for Computational Biology of Human Disease (CBHD) at Brown University supported by an Institutional Development Award (IDeA) from the National Institute of General Medical Sciences of the National Institutes of Health under grant number P20GM109035.

Conflict of Interest

NL is an employee of Cofactor Genomics.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The results shown here are in whole or part based upon data generated by the TCGA Research Network: https://www.cancer.gov/tcga. We would like to thank Dr. Joselynn Wallace in the COBRE Center for CBHD at Brown University for assistance in obtaining TCGA data. We thank Dr. Christina Raker in the Women and Infants Hospital Division of Research for guidance on statistical analyses performed.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fimmu.2021.622182/full#supplementary-material

References

1. American Cancer Society. Cancer Statistics Center - Ovary. Available online at: https://cancerstatisticscenter.cancer.org/#!/cancer-site/Ovary (accessed April 9, 2020).

2. Roane BM, Arend RC, Birrer MJ. Review: targeting the transforming growth factor-beta pathway in ovarian cancer. Cancers. (2019) 11:668. doi: 10.3390/cancers11050668

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Ghisoni E, Imbimbo M, Zimmermann S, Valabrega G. Ovarian cancer immunotherapy: turning up the heat. Int J Mol Sci. (2019) 20:2927. doi: 10.3390/ijms20122927

CrossRef Full Text | Google Scholar

4. Ibrahim EM, Refae AA, Bayer AM, Sagr ER. Poly(ADP-ribose) polymerase inhibitors as maintenance treatment in patients with newly diagnosed advanced ovarian cancer: a meta-analysis. Future Oncol. (2020) 16:585–96. doi: 10.2217/fon-2020-0057

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Rossi L, Verrico M, Zaccarelli E, Papa A, Colonna M, Strudel M, et al. Bevacizumab in ovarian cancer: a critical review of phase III studies. Oncotarget. (2016) 8:12389–405. doi: 10.18632/oncotarget.13310

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Cojocaru E, Parkinson CA, Brenton JD. Personalising treatment for high-grade serous ovarian carcinoma. Clin Oncol. (2018) 30:515–24. doi: 10.1016/j.clon.2018.05.008

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Kerliu L, Myruski S, Bhatti A, Soni P, Petrosius P, Pervanas HC, et al. Niraparib for the treatment of recurrent epithelial ovarian, fallopian tube, or primary peritoneal cancer. Ann Pharmacother. (2020) 54:1010–5. doi: 10.1177/1060028020912749

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Zhang L, Conejo-Garcia JR, Katsaros D, Gimotty PA, Massobrio M, Regnani G, et al. Intratumoral T cells, recurrence, and survival in epithelial ovarian cancer. N Engl J Med. (2003) 348:203–13. doi: 10.1056/NEJMoa020177

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Kandalaft LE, Powell DJ, Singh N, Coukos G. Immunotherapy for ovarian cancer: what's next? J Clin Oncol. (2011) 29:925–33. doi: 10.1200/JCO.2009.27.2369

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Liu R, Hu R, Zeng Y, Zhang W, Zhou H-H. Tumour immune cell infiltration and survival after platinum-based chemotherapy in high-grade serous ovarian cancer subtypes: a gene expression-based computational study. EBioMedicine. (2020) 51:102602. doi: 10.1016/j.ebiom.2019.102602

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Liu J, Meng H, Nie S, Sun Y, Jiang P, Li S, et al. Identification of a prognostic signature of epithelial ovarian cancer based on tumor immune microenvironment exploration. Genomics. (2020) 112:4827–41. doi: 10.1016/j.ygeno.2020.08.027

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Ding J, Zhang Q, Chen S, Huang H, He L. Construction of a new tumor immunity-related signature to assess and classify the prognostic risk of ovarian cancer. Aging. (2020) 12:21316–28. doi: 10.18632/aging.103868

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Wu Y, Xia L, Zhao P, Deng Y, Guo Q, Zhu J, et al. Immune profiling reveals prognostic genes in high-grade serous ovarian cancer. Aging. (2020) 12:11398–415. doi: 10.18632/aging.103199

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Bösmüller H-C, Wagner P, Peper JK, Schuster H, Pham DL, Greif K, et al. Combined immunoscore of CD103 and CD3 identifies long-term survivors in high-grade serous ovarian cancer. Int J Gynecol Cancer. (2016) 26:671–9. doi: 10.1097/IGC.0000000000000672

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Montfort A, Owen S, Piskorz AM, Supernat A, Moore L, Al-Khalidi S, et al. Combining measures of immune infiltration shows additive effect on survival prediction in high-grade serous ovarian carcinoma. Br J Cancer. (2020) 122:1803–10. doi: 10.1038/s41416-020-0822-x

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Schillebeeckx I, Armstrong JR, Forys JT, Hiken J, Earls J, Flanagan KC, et al. Analytical performance of an immunoprofiling assay based on RNA models. J Mol Diagn. (2020) 22:555–70. doi: 10.1016/j.jmoldx.2020.01.009

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Maecker HT, McCoy JP, Nussenblatt R. Standardizing immunophenotyping for the Human Immunology Project. Nat Rev Immunol. (2012) 12:191–200. doi: 10.1038/nri3158

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Yu N, Li X, Song W, Li D, Yu D, Zeng X, et al. CD4(+)CD25 (+)CD127 (low/-) T cells: a more specific Treg population in human peripheral blood. Inflammation. (2012) 35:1773–80. doi: 10.1007/s10753-012-9496-8

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Weinberg RA. The Biology of Cancer. 2nd ed. New York, NY: W.W. Norton & Company (2013). p. 960. doi: 10.1201/9780429258794

CrossRef Full Text | Google Scholar

20. LaFranzo NA, Flanagan KC, Quintanilha D. Predictive immune modeling of solid tumors. J Vis Exp. (2020) 156:e60645. doi: 10.3791/60645

PubMed Abstract | CrossRef Full Text | Google Scholar

21. R Core Team. R: A Language and Environment for Statistical Computing. Vienna: Foundation for Statistical Computing (2013).

Google Scholar

22. Morgan M, Davis S. R: A Language and Environment for Statistical Computing. NIH/NCI Genomic Data Commons Access. Available online at: https://bioconductor.org/packages/GenomicDataCommons, http://github.com/Bioconductor/GenomicDataCommons (accessed June 15, 2020).

Google Scholar

23. Li T, Fu J, Zeng Z, Cohen D, Li J, Chen Q, et al. TIMER2.0 for analysis of tumor-infiltrating immune cells. Nucleic Acids Res. (2020) 48:W509–14. doi: 10.1093/nar/gkaa407

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Li T, Fan J, Wang B, Traugh N, Chen Q, Liu JS, et al. TIMER: a web server for comprehensive analysis of tumor-infiltrating immune cells. Cancer Res. (2017) 77:e108–10. doi: 10.1158/0008-5472.CAN-17-0307

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Li B, Severson E, Pignon J-C, Zhao H, Li T, Novak J, et al. Comprehensive analyses of tumor immunity: implications for cancer immunotherapy. Genome Biol. (2016) 17:174. doi: 10.1186/s13059-016-1028-7

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Cerami E, Gao J, Dogrusoz U, Gross BE, Sumer SO, Aksoy BA, et al. The cBio cancer genomics portal: an open platform for exploring multidimensional cancer genomics data. Cancer Discov. (2012) 2:401–4. doi: 10.1158/2159-8290.CD-12-0095

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Gao J, Aksoy BA, Dogrusoz u, Dresdner G, Gross B, Sumer SO, et al. Integrative analysis of complex cancer genomics and clinical profiles using the cBio Portal. Sci Signal. (2013) 6:pl1 doi: 10.1126/scisignal.2004088

CrossRef Full Text | Google Scholar

28. Fawcett T. An introduction to ROC analysis. Pattern Recognit Lett. (2006) 27:861–74. doi: 10.1016/j.patrec.2005.10.010

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Whelan S, Ophir E, Kotturi MF, Levy O, Ganguly S, Leung L, et al. PVRIG and PVRL2 are induced in cancer and inhibit CD8+ T-cell function. Cancer Immunol Res. (2019) 7:257–68. doi: 10.1158/2326-6066.CIR-18-0442

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Bekes I, Löb S, Holzheu I, Janni W, Baumann L, Wöckel A, et al. Nectin-2 in ovarian cancer: how is it expressed and what might be its functional role? Cancer Sci. (2019) 110:1872–82. doi: 10.1111/cas.13992

PubMed Abstract | CrossRef Full Text | Google Scholar

31. James NE, Woodman M, DiSilvestro PA, Ribeiro JR. The perfect combination: enhancing patient response to PD-1-based therapies in epithelial ovarian cancer. Cancers. (2020) 12:2150. doi: 10.3390/cancers12082150

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Kim H-S, Kim J-Y, Lee YJ, Kim SH, Lee J-Y, Nam EJ, et al. Expression of programmed cell death ligand 1 and immune checkpoint markers in residual tumors after neoadjuvant chemotherapy for advanced high-grade serous ovarian cancer. Gynecol Oncol. (2018) 151:414–21. doi: 10.1016/j.ygyno.2018.08.023

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Imai Y, Hasegawa K, Matsushita H, Fujieda N, Sato S, Miyagi E, et al. Expression of multiple immune checkpoint molecules on T cells in malignant ascites from epithelial ovarian carcinoma. Oncol Lett. (2018) 15:6457–68. doi: 10.3892/ol.2018.8101

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Toker A, Nguyen LT, Stone SC, Yang SYC, Katz SR, Shaw PA, et al. Regulatory T cells in ovarian cancer are characterized by a highly activated phenotype distinct from that in melanoma. Clin Cancer Res. (2018) 24:5685–96. doi: 10.1158/1078-0432.CCR-18-0554

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Leffers N, Gooden MJM, de Jong RA, Hoogeboom B-N, ten Hoor KA, Hollema H, et al. Prognostic significance of tumor-infiltrating T-lymphocytes in primary and metastatic lesions of advanced stage ovarian cancer. Cancer Immunol Immunother. (2009) 58:449–59. doi: 10.1007/s00262-008-0583-5

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Curiel TJ, Coukos G, Zou L, Alvarez X, Cheng P, Mottram P, et al. Specific recruitment of regulatory T cells in ovarian carcinoma fosters immune privilege and predicts reduced survival. Nat Med. (2004) 10:942–9. doi: 10.1038/nm1093

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Rowshanravan B, Halliday N, Sansom DM. CTLA-4: a moving target in immunotherapy. Blood. (2017) 131:58–67. doi: 10.1182/blood-2017-06-741033

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Amatore F, Gorvel L, Olive D. Role of Inducible Co-Stimulator (ICOS) in cancer immunotherapy. Expert Opin Biol Ther. (2020) 20:141–50. doi: 10.1080/14712598.2020.1693540

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Conrad C, Gregorio J, Wang Y-H, Ito T, Meller S, Hanabuchi S, et al. Plasmacytoid dendritic cells promote immunosuppression in ovarian cancer via ICOS costimulation of Foxp3(+) T-regulatory cells. Cancer Res. (2012) 72:5240–9. doi: 10.1158/0008-5472.CAN-12-2271

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Oshima T, Sato S, Kato J, Ito Y. Nectin-2 is a potential target for antibody therapy of breast and ovarian cancers. Mol Cancer. (2013) 12:60. doi: 10.1186/1476-4598-12-60

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Drakes ML, Mehrotra S, Aldulescu M, Potkul RK, Liu Y, Grisoli A, et al. Stratification of ovarian tumor pathology by expression of programmed cell death-1 (PD-1) and PD-ligand- 1 (PD-L1) in ovarian cancer. J Ovarian Res. (2018) 11:43. doi: 10.1186/s13048-018-0414-z

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Nabissi M, Marinelli O, Annibali D, Aguzzi C. The controversial role of PD-1 and its ligands in endometrial cancer and other gynaecological malignancies. Front Oncol. (2019) 9:1073. doi: 10.3389/fonc.2019.01073

CrossRef Full Text

Keywords: tumor immunology, immune profiling, LAG-3, ICOS, T regulatory cells, CTLA-4, PVRL2, ovarian cancer

Citation: James NE, Miller K, LaFranzo N, Lips E, Woodman M, Ou J and Ribeiro JR (2021) Immune Modeling Analysis Reveals Immunologic Signatures Associated With Improved Outcomes in High Grade Serous Ovarian Cancer. Front. Oncol. 11:622182. doi: 10.3389/fonc.2021.622182

Received: 27 October 2020; Accepted: 10 February 2021;
Published: 05 March 2021.

Edited by:

Sarah M. Temkin, Anne Arundel Medical Center, United States

Reviewed by:

Tonya J. Webb, University of Maryland, Baltimore, United States
Angeles Alvarez Secord, Duke University, United States

Copyright © 2021 James, Miller, LaFranzo, Lips, Woodman, Ou and Ribeiro. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Jennifer R. Ribeiro, jrribeiro@wihri.org

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.