Skip to main content

ORIGINAL RESEARCH article

Front. Oncol., 24 January 2023
Sec. Gynecological Oncology
This article is part of the Research Topic New Molecular Approaches to Improve Gynecological Cancer Management View all 11 articles

Predictors of residual disease after debulking surgery in advanced stage ovarian cancer

Farnoosh Abbas-Aghababazadeh,&#x;Farnoosh Abbas-Aghababazadeh1,2†Naoko Sasamoto&#x;Naoko Sasamoto3†Mary K. TownsendMary K. Townsend4Tianyi HuangTianyi Huang5Kathryn L. TerryKathryn L. Terry3Allison F. VitonisAllison F. Vitonis3Kevin M. EliasKevin M. Elias3Elizabeth M. PooleElizabeth M. Poole6Jonathan L. HechtJonathan L. Hecht7Shelley S. TworogerShelley S. Tworoger4Brooke L. Fridley*Brooke L. Fridley1*
  • 1Department of Biostatistics & Bioinformatics, H. Lee Moffitt Cancer Center and Research Institute, Tampa, FL, United States
  • 2University Health Network, Princess Margaret Cancer Center, Toronto, ON, Canada
  • 3Department of Obstetrics and Gynecology, Brigham and Women’s Hospital and Harvard Medical School, Boston, MA, United States
  • 4Department of Cancer Epidemiology, H. Lee Moffitt Cancer Center and Research Institute, Tampa, FL, United States
  • 5Department of Medicine, Channing Division of Network Medicine, Brigham and Women’s Hospital, Boston, MA, United States
  • 6Sanofi Genzyme, Global Medical Affairs, Cambridge, MA, United States
  • 7Department of Pathology, Beth Israel Deaconess Medical Center, Boston, MA, United States

Objective: Optimal debulking with no macroscopic residual disease strongly predicts ovarian cancer survival. The ability to predict likelihood of optimal debulking, which may be partially dependent on tumor biology, could inform clinical decision-making regarding use of neoadjuvant chemotherapy. Thus, we developed a prediction model including epidemiological factors and tumor markers of residual disease after primary debulking surgery.

Methods: Univariate analyses examined associations of 11 pre-diagnosis epidemiologic factors (n=593) and 24 tumor markers (n=204) with debulking status among incident, high-stage, epithelial ovarian cancer cases from the Nurses’ Health Studies and New England Case Control study. We used Bayesian model averaging (BMA) to develop prediction models of optimal debulking with 5x5-fold cross-validation and calculated the area under the curve (AUC).

Results: Current aspirin use was associated with lower odds of optimal debulking compared to never use (OR=0.52, 95%CI=0.31-0.86) and two tissue markers, ADRB2 (OR=2.21, 95%CI=1.23-4.41) and FAP (OR=1.91, 95%CI=1.24-3.05) were associated with increased odds of optimal debulking. The BMA selected aspirin, parity, and menopausal status as the epidemiologic/clinical predictors with the posterior effect probability ≥20%. While the prediction model with epidemiologic/clinical predictors had low performance (average AUC=0.49), the model adding tissue biomarkers showed improved, but weak, performance (average AUC=0.62).

Conclusions: Addition of ovarian tumor tissue markers to our multivariable prediction models based on epidemiologic/clinical data slightly improved the model performance, suggesting debulking status may be in part driven by tumor characteristics. Larger studies are warranted to identify those at high risk of poor surgical outcomes informing personalized treatment.

Introduction

Ovarian cancer has a 5-year survival <50%, as most cases are diagnosed at late stages (1). Optimal debulking with no or minimal residual disease during cytoreductive surgery is predictive of survival (2). Randomized trials showed 20% higher risk of death and 25% higher risk of progression among patients with residual tumor >10mm versus 1-10mm (3). Identifying women who may have poor surgical outcomes is critical to defining appropriate treatment, including use of neo-adjuvant chemotherapy to reduce tumor burden (4).

Studies of preoperative predictors of suboptimal debulking had not led to a reproducible model. For example, findings are mixed regarding the predictive value of preoperative blood CA125 (5) and radiologic features from computed tomography scans (6). Data is limited regarding epidemiologic factors related to debulking, with mixed results for obesity and hormone therapy (7, 8). Furthermore, prior studies observed that tumor molecular characteristics can predict residual disease, identifying gene expression signatures in migration, invasion, and stromal activation pathways (913). However, no studies to date have assessed a comprehensive prediction model for debulking outcomes. Thus, our objectives were to identify epidemiological characteristics and tumor markers associated with residual disease and build a prediction model of optimal debulking status after primary debulking surgery in treatment naïve, advanced stage, invasive epithelial ovarian cancer patients.

Materials and methods

The Nurses’ Health Study (NHS) is a prospective cohort study established in 1976 enrolling 121,000 female nurses ages 30-55 years from 11 US states (14). NHSII was established in 1989 enrolling 116,429 female nurses ages 25-42 years from 14 US states (15). Women provided demographic, lifestyle, reproductive, and medical information biennially. Self-reported ovarian cancer diagnosis were confirmed by pathology report review (94%) or linkage to tumor registries. A gynecologic pathologist abstracted information from pathology and surgical reports on stage, histology, grade, and residual tumor (optimal debulking: residual tumor <2cm; suboptimal debulking: residual tumor ≥2cm; unknown). This definition was used because most cases were diagnosed when this threshold was used to define optimal debulking. The study protocol was approved by the institutional review boards of the Brigham and Women’s Hospital and Harvard T.H. Chan School of Public Health, and those of participating registries as required.

The New England Case-Control Study (NECC) is a population-based case control study of ovarian cancer enrolling over three phases (1992-1997, 1998-2002, 2003-2008) from New Hampshire and Eastern Massachusetts (16); 2,203 (71%) of eligible cases identified using area hospital registries agreed to participate. Participants completed in-person interviews on demographics, lifestyle, reproductive factors, and medical history one year prior to ovarian cancer diagnosis. Surgical and pathological reports were reviewed to confirm diagnosis and abstract stage, histology, and grade. Optimal vs. suboptimal debulking was defined as residual disease <1cm vs. ≥1cm or unknown. The study was approved by the Institutional Review Board of Brigham and Women’s Hospital, Boston, Massachusetts and Dartmouth College, Hanover, New Hampshire.

Epidemiologic factors included age (years, continuous), body mass index (BMI; kg/m2, continuous), smoking (never, current, former), duration of smoking (pack-years, continuous), oral contraceptive use (OC; never, ever), parity (0, 1, 2+), menopausal status (premenopausal, postmenopausal never used hormone therapy [HT], postmenopausal ever used HT), aspirin use (never, current, past), family history of breast or ovarian cancer (no, yes), and history of hysterectomy, tubal ligation or Cesarean section (ever, never) at least one year before diagnosis.

Details on ovarian tumor block collection has been described previously (17). We retrieved formalin fixed paraffin embedded (FFPE) blocks with primary ovarian tumor from 631 invasive cases (330 NHS, 86 NHSII, 215 NECC). Blocks were reviewed for histology, invasiveness, and grade by a gynecologic pathologist, using 2014 WHO diagnostic criteria, circling areas of tumor for tissue microarrays (TMA) with two 1.0mm or three 0.6mm cores per case (17). We used histology, invasiveness, and grade from the slide review and record abstraction otherwise.

Immunohistochemistry (IHC) staining was performed on TMA slides within two weeks of sectioning (Supplementary Table S1). IHC markers, except pSMAD2/3, were evaluated by a gynecologic pathologist based on the proportion of cells staining positive (0-3+) or mutant/abnormal or wild type (TP53). For pSMAD 2/3, we assessed H-score (18) based on intensity and percent staining area, calculated via Definiens Tissue Studio v4.2 suite (Definiens Inc, Germany) with scans from Aperio™ ScanScope AT2 or AT Turbo (Leica Biosystems, Vista, CA) at Moffitt Cancer Center. All IHC markers were evaluated in the tumor epithelial component. For this study, we included IHC markers that have previously been reported to be associated with debulking status (i.e. ADH1B, COL11A1, CXCL14, FABP4, FAP, POSTN, pSmad2/3) and other IHC markers in which data were generated as part of pprior studies (i.e. ADRB2 (19), CD163 (20), CD68 (20), PTGS1 (20), PTGS2 (20), ESR1 (17, 21), ESR2n (22), MAPK (23), MUC1, MUC16, TP53 (23), PGR (17, 21), STAT1, VDRc, VDRn).

Statistical analysis

Imputation

Imputation of missing IHC scores was conducted using k-Nearest Neighbors (kNN; k=5) since some markers were missing in selected TMAs (Supplementary Figure S1). Imputed data had more symmetrical distribution and slightly higher median values versus observed data (Supplementary Figure S2).

Clinical and tissue biomarker predictors of debulking status

Mean and standard deviation (SD) for continuous variables and frequencies and percentage for categorical variables were used to summarize the predictors. Tissue markers were treated continuously. The primary outcome was coded as optimal versus suboptimal debulking. Analyses were conducted including all cases and restricted to type II ovarian tumors (high-grade serous, endometrioid, mixed or poorly differentiated, Transitional/Brenner, carcinosarcoma) (24). Logistic regression was used to examine the association of epidemiologic characteristics and normalized IHC scores with debulking status, adjusting for study site (NECC, NHS/NHSII). Cox proportional hazards regression was used to assess the relationship of debulking status with overall survival to ensure the validity of our debulking measure.

Prediction modeling

Bayesian model averaging (BMA) with logistic regression was used to develop the prediction model for debulking status with only additive effects (i.e., no interaction effects) using the BMA R package (25). BMA was fit using five 5-fold cross-validations (CV), resulting in different training and test sets for each run to improve estimated performance. For each predictor, we present the average posterior mean (APM) and average posterior SD (APSD) across the five 5-fold CV. The receiver operating characteristics curve was calculated from the posterior probabilities. For each fold, we evaluated the discriminatory accuracy using the area under curve (AUC) and calculated the mean for each CV, and the overall AUC by averaging the average AUCs of the 5 CVs and computing the associated standard deviation (SD). We conducted BMA for epidemiological/clinical variables with debulking status (n=593). Then, after creating a single predictive score from the epidemiologic variables, we conducted BMA adding the tumor marker data. Analyses were performed using R, version 4.0.2.

Results

In NHS/NHSII, of the 1,550 incident invasive ovarian cancer cases (1,227 in NHS and 323 in NHSII) diagnosed from 1976-2017, we excluded those with unknown debulking status (n=1,307; 1,067 in NHS and 240 in NHSII) and stage I or II disease (n=75, 33 in NHS and 42 in NHSII). In NECC, of the 1,650 invasive epithelial ovarian cancer cases, we excluded those who did not have information on debulking status (n=1,054) and stage I or II disease (n=171) (Figure 1). As a result, the epidemiologic model included 593 invasive epithelial ovarian cancer cases (NHS=127, NHSII=41, NECC=425), of which 464 (78%) were optimally debulked (type II n=537 and 419 optimally debulked; Table 1). Average age at diagnosis was 56.9 years (SD 12.6) with the majority (>90%) being type II tumors. Characteristics of cases by debulking status were similar (Supplementary Table S2). Cases with optimal versus suboptimal debulking had better overall survival (all: HR=0.60, 95%CI=0.48-0.75; type II: HR=0.63, 95%CI=0.50-0.80). Current vs. never aspirin use was significantly associated with lower odds of optimal debulking (OR=0.52, 95%CI=0.31-0.86), which remained significant for type II tumors (OR=0.47, 95%CI=0.28-0.81; Supplementary Table S3).

FIGURE 1
www.frontiersin.org

Figure 1 Flow chart of the exclusion criteria and study population included in the various prediction modeling.

TABLE 1
www.frontiersin.org

Table 1 Association between demographic/clinical characteristics and debulking status (1=optimally debulked, 0 = sub-optimally debulked) among advanced stage invasive epithelial ovarian cancer cases in NHS/NHSII/NECC.

Tissue IHC markers were available in 166 cases with data on debulking status, of which 135 (81%) were optimally debulked. Distribution of clinical and epidemiologic characteristics were similar to all cases (Supplementary Table S4; Supplementary Table S5). ADRB2 (OR=2.21, 95%CI=1.23-4.41) and FAP (OR=1.91, 95%CI=1.24-3.05) were associated with optimal debulking (Figure 2). Results were similar for type II tumors (Supplementary Figure S3).

FIGURE 2
www.frontiersin.org

Figure 2 Tissue marker associations (odds ratio [OR] and 95% confidence intervals [CI]) with optimal debulking status among advanced stage invasive epithelial ovarian cancer cases in NHS/NHSII/NECC (n=166). All models were adjusted for study (NHS/NHSII and NEC). In the logistic regression models 1 = optimally debulked and 0 = sub-optimally debulked.

Prediction modeling of optimal debulking status using epidemiologic and clinical predictors

We first sought to develop a prediction model using epidemiologic and clinical predictors only. The BMA results for all models are presented in Supplemental File 1. Three predictors had a posterior effect probability ≥20% for selection into the prediction model: current vs. never aspirin use (APM=-0.12, APSD=0.21), parity=1 vs. nulliparous (APM=0.05, APSD=0.16), and postmenopausal ever HT vs. premenopausal (APM=0.05, APSD=0.12 (Figure 3A). For type II tumors, 3 predictors were identified (current aspirin use, APM=-0.18, APSD=0.28; smoking pack-years, APM=-0.001, APSD=0.002, and menopausal status/ever HT use, APM=0.04, APSD=0.15; Supplementary Figure S4). The mean AUC of 0.49 (SD 0.02) for all invasive ovarian cancer (Figure 3B) and 0.53 (SD 0.03) for type II tumors (Supplementary Figure S4).

FIGURE 3
www.frontiersin.org

Figure 3 Prediction modeling of optimal debulking status using epidemiologic and clinical predictors. Average posterior means and associated average posterior probabilities of epidemiologic and clinical predictors being selected in the final prediction model of optimal debulking status and area under the curve (AUC) of the prediction models using Bayesian model averaging among invasive epithelial ovarian cancer cases (n=593) in NHS/NHSII/NEC. We assessed the posterior probability of 11 predictors for possible selection in the final model and conducted five 5-fold cross-validations. The bar chart (A) presents the average posterior mean across the 25 models that were run in total; grey bars denote predictors with average posterior probabilities <20% and yellow bars denote predictors with average posterior probability ≥ 20%. (B) presents the five average AUCs from the five 5-fold cross-validations and in the legend overall AUC, which is the average of the average AUCs from the 5-fold CV, is presented. Type 2 tumors include high-grade serous or poorly differentiated, Transitional/Brenner, Carcinosarcoma, high grade mixed histology.

Prediction modeling of optimal debulking status using tissue markers

Next, we developed a prediction model using tissue IHC markers only. There were 8 biomarkers with posterior effect probability of ≥20% including ESR1 and CD8+ T cells, which were associated with higher odds of optimal debulking (APM from -0.72 to 0.04), while p-SMAD2/3, PTGS2, and ADRB2 had lower odds of optimal debulking (APM of -0.72 to -0.02) (Figure 4A). For type II tumors, there were 7 biomarkers with positive posterior means (APM of 0.02 to 0.2) and 6 biomarkers with negative posterior means (APM of -0.14 to -0.02) (Supplementary Figure S5). These IHC markers resulted in a mean AUC of 0.62 (SD 0.03) for advanced stage invasive cases, and 0.47 (SD 0.1) for type II tumors (Figures 4B; Supplementary Figure S5).

FIGURE 4
www.frontiersin.org

Figure 4 Prediction modeling of optimal debulking status using tissue markers. Average posterior means and associated average posterior probabilities of tissue markers being selected in the final prediction model of optimal debulking status and area under the curve (AUC) of the prediction models using Bayesian model averaging among invasive epithelial ovarian cancer cases (n=166) in NHS/NHSII/NEC. We assessed the posterior probability of 24 tissue marker predictors for possible selection in the final model and conducted five 5-fold cross-validations. The bar chart (A) presents the average posterior mean across the 25 models that were run in total; grey bars denote predictors with average posterior probabilities <20% and yellow bars denote predictors with average posterior probability ≥ 20%. (B) presents the five average AUCs from the five 5-fold cross-validations and in the legend overall AUC, which is the average of the average AUCs from the 5-fold CV, is presented.

Combined prediction model with epidemiologic variables and tissue biomarkers

Lastly, we developed a prediction model including both epidemiologic/clinical predictors and tissue biomarkers. Among the subset of cases with biomarker data, the model with only the clinical prediction score had a mean AUC of 0.58 (SD 0.07) for all cases and 0.62 (SD 0.01) for type II tumors. When tissue biomarkers were added, positive posterior mean was observed for 8 markers including ESR1 (APM=0.67, APSD=0.38) and CD8+ T cells (APM=0.03, APSD=0.12) and a negative posterior mean for p-SMAD2/3 (APM=-0.71, APSD=0.35) and PTGS2 (APM=-0.13, APSD=0.23) (Figure 5A). The clinical prediction score had an average posterior mean of 1.58 (APSD=7.94). The resultant mean AUC was 0.62 (SD 0.04) (Figure 5B). We observed similar results for type II tumors, although the model resulted in an AUC=0.47 (SD 0.1; Supplementary Figure S6).

FIGURE 5
www.frontiersin.org

Figure 5 Prediction modeling of optimal debulking status using epidemiologic/clinical predictors and tissue markers. Average posterior means and associated average posterior probabilities of epidemiologic/clinical predictors and tissue markers being selected in the final prediction model of optimal debulking status and area under the curve (AUC) of the prediction models using Bayesian model averaging among invasive epithelial ovarian cancer cases (n=166) in NHS/NHSII/NEC. We assessed the posterior probability of 24 tissue markers in addition to our base model of clinical predict score, which included epidemiologic/clinical variables that had ≥ 20% posterior probability of being selected in the final prediction model of optimal debulking status (i.e. aspirin, parity, and menopausal status), for possible selection in the final model and conducted five 5-fold cross-validations. The bar chart (A) presents the average posterior mean across the 25 models that were run in total; grey bars denote predictors with average posterior probabilities <20% and yellow bars denote predictors with average posterior probability ≥ 20%. (B) presents the five average AUCs from the five 5-fold cross-validations and in the legend overall AUC, which is the average of the average AUCs from the 5-fold CV, is presented.

Discussion

We simultaneously examined a wide range of potential epidemiologic and molecular predictors of optimal debulking in advanced stage invasive ovarian cancer patients undergoing primary debulking surgery in a population-based study. Relatively few epidemiologic predictors were identified, and they did not have predictive capacity. However, four tumor markers (POSTN, pSMAD2/3, CXCL14, ADH1B, FAP) that have previously been reported to predict suboptimal debulking were selected in our model, although only p-SMAD2/3 was in the same direction but with lower discriminatory performance compared to prior reports. Seven other tumor markers predicted optimal debulking. However, the multivariable prediction model showed discriminatory performance that is not clinically actionable.

Several studies have previously identified ovarian tumor tissue markers that predict debulking status (913), with some studies reporting high discriminatory performance (1113). One recent study sought to validate 20 previously reported molecular signatures using gene expression data and all combinations resulted in poor performance with AUC < 0.65 (10), which is consistent with our observation of poor performance for a tissue marker only prediction model. Notably, the direction of association reported in prior gene expression studies often showed opposite associations using IHC markers as in our study (11, 12). This may be due to the use of protein markers, which do not always correlate with gene expression, and the use of a population-based sample in our study. It is unclear if the protein markers do not provide the same predictive capacity as gene expression or if the high dimensionality of gene expression data is led to overfitting of results. Overall, our work and others support that biologic features of the tumor may differ between optimally and sub-optimally debulked tumors.

Even though aspirin use has been associated with a lower ovarian cancer risk (26), pre-diagnostic current aspirin use was associated with decreased odds of optimal debulking. This is inconsistent with reported null associations between pre-diagnostic aspirin use and ovarian cancer survival (27). Complications during surgery may be one reason for this observation (10) as aspirin users may be more likely to develop hemorrhage-related surgical complications. Parous women had increased odds of optimal debulking, in line with studies reporting modest inverse associations between parity and ovarian cancer survival or risk of aggressive disease (28, 29). HT use also increased odds of optimal debulking, consistent with an international case-control consortium reporting that pre-diagnosis HT was associated with lower risk of having macroscopic residual disease and improved survival among postmenopausal patients (8). Interestingly, pre-diagnostic smoking was associated with decreased odds of optimal debulking among type II tumors, which is consistent its association with worse survival (30) and increased risk of aggressive rapidly fatal disease in high-grade serous tumors (29). Smoking increases systemic inflammation (31) and also has been reported to accelerate migration and invasion of ovarian cancer cells, promoting progression and metastasis (32), which may result in tumors that are more likely to be sub-optimally debulked. Future work, with larger sample sizes, should further explore these relationships to validate our observations.

With respect to the biomarkers, we identified new potential predictors of debulking, including CD8+ T cells and ESR1, which increased odds of optimal debulking. CD8+ T cell infiltration has been associated with improved ovarian cancer survival (33). It is possible that an immune-activated tumor microenvironment results in tumors that are easier to surgically resect, possibly by reducing metastatic spread (34). ESR1 expression has been associated with improved ovarian cancer survival and lower risk of macroscopic residual disease in endometrioid tumors (35). Conversely, PTGS2 and ADRB2 were associated with lower odds of optimal debulking. Both PTGS2, which drives prostaglandin synthesis in the tumor microenvironment, and ADRB2 activation can increase cell migration, enhance cell survival (36), and promote cancer growth and metastasis (37). One study reported that concurrent increased expression of ADRB2 and PTGS2 in ovarian cancer was associated with poor survival (37), suggesting activation of this axis should be explored as a biological pathways driving disease spread, leading to residual disease. Additional large-scale, population-based studies are needed to evaluate the biologic differences between tumors that were optimally versus sub-optimally debulked and evaluate and validate the predictive capacity of these biomarkers above that of clinical and epidemiologic measures.

The strength of our study is that we had detailed epidemiologic/clinical and tumor tissue marker data and applied BMA to develop a multivariable prediction model. Limitations include the number of sub-optimally debulked patients and different definitions of optimal debulking across studies due to change in the criteria over time. We also were unable to evaluate complete cytoreductive surgery, although many prior studies used 1cm of residual disease as the cutoff for defining optimal debulking status. In NHS/NHSII and NECC, many ovarian cancer cases were missing data on debulking status (84% in NHS/NHSII; 64% in NECC), which may not be missing at random and possibly biased the observed results. However, the distribution of epidemiologic factors in our analytic sample were similar between those with and without data on debulking status, suggesting a representative sample. Furthermore, there could be reporting bias of debulking status particularly as many women in NHS/NHSII were treated in community settings, which may explain the high percentage of optimally debulked cases (83%) in our study compared to prior studies (range~40%-90%) (38), although debulking status was strongly associated with survival in our population. In NECC, nearly all cases received surgical care by subspecialist gynecologic oncologists at tertiary academic hospitals, likely leading to improved surgical outcomes (39). Our study was limited by not have an independent validation cohort to validate our prediction models, so conducted internal validation using 5x5-fold cross-validation. We did not have a measure of surgical skills by individual surgeons, which may vary widely due to the population-based nature of our study, or detailed laparoscopic data on tumor spread, both of which have been shown to be related to debulking status. While there were some IHC markers that are known to be more present in the stromal component (e.g. POSTN, COL11A1), our ovarian TMA was created to maximize the tumor epithelial tissue and the IHC scoring was based on the expression in the tumor epithelial compartment. Further studies are necessary to evaluate protein expression in the stromal compartments. Finally, we could not study laparoscopy-based scores, which have reported discriminatory performance ranging widely (AUCs~0.69-0.98), depending on outcome definition (complete and/or optimal cytoreduction) and the proportion of cases undergoing neo-adjuvant chemotherapy (40). Adding molecular factors to existing laparoscopy-based scores could enhance discriminatory ability in the primary debulking setting, which is most critical time to determine the need for neoadjuvant chemotherapy.

Overall, combining information on ovarian tumor tissue markers and epidemiologic/clinical data led to the best model performance, although it is not yet clinically actionable. Our results further support that debulking status may be in part driven by tumor characteristics. Future studies are warranted to validate our findings and integrate these variables into currently used clinical models based on disease spread to identify those at high risk of poor surgical outcomes, which will inform personalized treatment for ovarian cancer.

Data availability statement

The datasets presented in this article are not publicly available because of participant confidentiality and privacy concerns. Further information including the procedures to obtain and access data from the Nurses’ Health Studies is described at https://www.nurseshealthstudy.org/researchers (contact email: nhsaccess@channing.harvard.edu). The NEC data that support the findings of this study are available upon request and review by study leadership.

Ethics statement

The study protocol was approved by the institutional review boards of the Brigham and Women’s Hospital and Harvard T.H. Chan School of Public Health, and those of participating registries as required. The patients/participants provided informed consent to participate in the study.

Author contributions

FA-A: Conceptualization, Data curation, Formal analysis, Writing-Original draft, Writing – review & editing. NS: Conceptualization, Data curation, Writing-Original draft, Writing – review & editing. MT: Data curation, Writing – review & editing. TH: Data curation, Writing – review & editing. KT: Data curation, Writing – review & editing. AV: Data curation, Writing – review & editing. KE: Writing – review & editing. EP: Funding acquisition, Data curation, Writing – review & editing. JH: Data curation, Writing – review & editing. ST: Conceptualization, Funding acquisition, Supervision, Writing – review & editing. BF: Conceptualization, Methodology, Supervision, Writing – review & editing. All authors contributed to the article and approved the submitted version.

Funding

This study was funded by the National Cancer Institute grants UM1 CA186107, P01 CA87969, U01 CA176726, R01 CA054419, P50 CA105009, and the Marsha Rivkin Center for Ovarian Cancer Research Skacel Family Scholar Award. NS was supported by the Department of Defense award W81XWH-21-1-0320 and Marsha Rivkin Center for Ovarian Cancer Research Rivkin Scientific Scholars Award. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Dana-Farber/Harvard Cancer Center is supported in part by an NCI Cancer Center Support Grant # NIH 5 P30 CA06516. This work has been supported in part by the Biostatistics and Bioinformatics Shared Resource and the Analytic Microscopy Core Facility at the H. Lee Moffitt Cancer Center & Research Institute, an NCI designated Comprehensive Cancer Center (P30-CA076292).

Acknowledgments

The authors would like to acknowledge the contribution to this study from central cancer registries supported through the Centers for Disease Control and Prevention’s National Program of Cancer Registries (NPCR) and/or the National Cancer Institute’s Surveillance, Epidemiology, and End Results (SEER) Program. Central registries may also be supported by state agencies, universities, and cancer centers. Participating central cancer registries include the following: Alabama, Alaska, Arizona, Arkansas, California, Colorado, Connecticut, Delaware, Florida, Georgia, Hawaii, Idaho, Indiana, Iowa, Kentucky, Louisiana, Massachusetts, Maine, Maryland, Michigan, Mississippi, Montana, Nebraska, Nevada, New Hampshire, New Jersey, New Mexico, New York, North Carolina, North Dakota, Ohio, Oklahoma, Oregon, Pennsylvania, Puerto Rico, Rhode Island, Seattle SEER Registry, South Carolina, Tennessee, Texas, Utah, Virginia, West Virginia, Wyoming. We thank Dana-Farber/Harvard Cancer Center in Boston, MA, for the use of the Specialized Histopathology Core, which provided histology and immunohistochemistry service.

Conflict of interest

NS reports grants from NCI, DOD, Marsha Rivkin Center for Ovarian Cancer Research. TH report grants from NHLBI. KLT reports grants from NIH. KE reports funding from Abcam; royalties from Aspira Women’s Health; consultant of Bluestar Genomics; personal fees from Expert Institute; and is a member of the Enhanced Recovery After Surgery USA unpaid. EP reports grants from NIH. SST reports grants from NIH/NCI, DOD, Rivkin Center, State of Florida, BMS; personal fees from AACR, Ponce Health Science University, Ovarian Cancer Research Alliance, German Cancer Research Center, Harvard T.H. Chan School of Public Health, and NIH outside of submitted work; and is a member of external advisory committee of California Teachers Study City of Hope and The Tomorrow Project Alberta Cancer Center.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fonc.2023.1090092/full#supplementary-material

Supplementary Figure 1 | Aggregation plots show all combinations of missing (yellow) and non-missing (blue) values across biomarkers, from the highest to lowest frequency among Invasive epithelial ovarian cancer cases in (A) NECC and (B) NHS/NHSII. Imputation was conducted using k-Nearest Neighbors (kNN; k=5) with the VIM R package. Data were assumed to be missing at random or completely at random. To avoid scaling issues and ensure comparability across markers and studies, we applied z-score transformation. The horizontal bars to the right of the grid show the frequencies of the corresponding combinations, while the vertical bars on top of it present the proportions of missing values in each variable.

Supplementary Figure 2 | Box plots represent the distribution of average immunohistochemistry (IHC) scores before (top row) and after (bottom row) applying imputation method among the invasive epithelial ovarian cancer cases in (A) NECC and (B) NHS/NHSII.

Supplementary Figure 3 | Tissue marker associations (odds ratio [OR] and 95% confidence intervals [CI]) with optimal debulking status among invasive epithelial ovarian cancer cases in NHS/NHSII/NECC (n=204) in Type 2 tumors. All models were adjusted for study (NHS/NHSII and NEC). In the logistic regression models 1 = optimally debulked and 0 = sub-optimally debulked. Type 2 tumors include high-grade serous or poorly differentiated, Transitional/Brenner, Carcinosarcoma, high grade mixed histology.

Supplementary Figure 4 | Prediction modeling of optimal debulking status using epidemiologic and clinical predictors in Type 2 tumors. Average posterior means and associated average posterior probabilities of epidemiologic and clinical predictors being selected in the final prediction model of optimal debulking status and area under the curve (AUC) of the prediction models using Bayesian model averaging among invasive epithelial ovarian cancer cases (n=537) in NHS/NHSII/NEC. We assessed the posterior probability of 11 predictors for possible selection in the final model and conducted five 5-fold cross-validations. The bar chart (A) presents the average posterior mean across the 25 models that were run in total; grey bars denote predictors with average posterior probabilities <20% and yellow bars denote predictors with average posterior probability ≥ 20%. (B) presents the five average AUCs from the five 5-fold cross-validations and in the legend overall AUC, which is the average of the average AUCs from the 5-fold CV, is presented. Type 2 tumors include high-grade serous or poorly differentiated, Transitional/Brenner, Carcinosarcoma, high grade mixed histology.

Supplementary Figure 5 | Prediction modeling of optimal debulking status using tissue markers in Type 2 tumors. Average posterior means and associated average posterior probabilities of tissue markers being selected in the final prediction model of optimal debulking status and area under the curve (AUC) of the prediction models using Bayesian model averaging among invasive epithelial ovarian cancer cases (n=150) in NHS/NHSII/NEC. We assessed the posterior probability of 24 tissue marker predictors for possible selection in the final model and conducted five 5-fold cross-validations. The bar chart (A) presents the average posterior mean across the 25 models that were run in total; grey bars denote predictors with average posterior probabilities <20% and yellow bars denote predictors with average posterior probability ≥ 20%. (B) presents the five average AUCs from the five 5-fold cross-validations and in the legend overall AUC, which is the average of the average AUCs from the 5-fold CV, is presented. Type 2 tumors include high-grade serous or poorly differentiated, Transitional/Brenner, Carcinosarcoma, high grade mixed histology.

Supplementary Figure 6 | Prediction modeling of optimal debulking status using epidemiologic/clinical predictors and tissue markers in Type 2 tumors. Average posterior means and associated average posterior probabilities of epidemiologic/clinical predictors and tissue markers being selected in the final prediction model of optimal debulking status and area under the curve (AUC) of the prediction models using Bayesian model averaging among invasive epithelial ovarian cancer cases (n=150) in NHS/NHSII/NEC. We assessed the posterior probability of 24 tissue markers in addition to our base model of clinical predict score, which included epidemiologic/clinical variables that had ≥ 20% posterior probability of being selected in the final prediction model of optimal debulking status (i.e. aspirin, pack-years of smoking, and menopausal status), for possible selection in the final model and conducted five 5-fold cross-validations. The bar chart (A) presents the average posterior mean across the 25 models that were run in total; grey bars denote predictors with average posterior probabilities <20% and yellow bars denote predictors with average posterior probability ≥ 20%. (B) presents the five average AUCs from the five 5-fold cross-validations and in the legend overall AUC, which is the average of the average AUCs from the 5-fold CV, is presented. Type 2 tumors include high-grade serous or poorly differentiated, Transitional/Brenner, Carcinosarcoma, high grade mixed histology.

References

1. American Cancer Society. Cancer facts & figures 2021. Atlanta: American Cancer Society, American Cancer Society (2021).

Google Scholar

2. Elattar A, Bryant A, Winter-Roach BA, Hatem M, Naik R. Optimal primary surgical treatment for advanced epithelial ovarian cancer. Cochrane Database Syst Rev (2011) 8):CD007565. doi: 10.1002/14651858.CD007565.pub2

CrossRef Full Text | Google Scholar

3. du Bois A, Reuss A, Pujade-Lauraine E, Harter P, Ray-Coquard I, Pfisterer J. Role of surgical outcome as prognostic factor in advanced epithelial ovarian cancer: a combined exploratory analysis of 3 prospectively randomized phase 3 multicenter trials: by the arbeitsgemeinschaft gynaekologische onkologie studiengruppe ovarialkarzinom (AGO-OVAR) and the groupe d’Investigateurs nationaux pour les etudes des cancers de l’Ovaire (GINECO). Cancer (2009) 115(6):1234–44. doi: 10.1002/cncr.24149

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Wright AA, Bohlke K, Armstrong DK, Bookman MA, Cliby WA, Coleman RL, et al. Neoadjuvant chemotherapy for newly diagnosed, advanced ovarian cancer: Society of gynecologic oncology and American society of clinical oncology clinical practice guideline. J Clin Oncol (2016) 34(28):3460–73. doi: 10.1200/jco.2016.68.6907

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Kang S, Kim TJ, Nam BH, Seo SS, Kim BG, Bae DS, et al. Preoperative serum CA-125 levels and risk of suboptimal cytoreduction in ovarian cancer: a meta-analysis. J Surg Oncol (2010) 101(1):13–7. doi: 10.1002/jso.21398

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Rutten MJ, van de Vrie R, Bruining A, Spijkerboer AM, Mol BW, Kenter GG, et al. Predicting surgical outcome in patients with international federation of gynecology and obstetrics stage III or IV ovarian cancer using computed tomography: a systematic review of prediction models. Int J Gynecol Cancer (2015) 25(3):407–15. doi: 10.1097/igc.0000000000000368

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Fotopoulou C, Richter R, Braicu EI, Kuhberg M, Feldheiser A, Schefold JC, et al. Impact of obesity on operative morbidity and clinical outcome in primary epithelial ovarian cancer after optimal primary tumor debulking. Ann Surg Oncol (2011) 18(9):2629–37. doi: 10.1245/s10434-011-1637-z

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Brieger KK, Peterson S, Lee AW, Mukherjee B, Bakulski KM, Alimujiang A, et al. Menopausal hormone therapy prior to the diagnosis of ovarian cancer is associated with improved survival. Gynecol Oncol (2020) 158(3):702–9. doi: 10.1016/j.ygyno.2020.06.481

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Torres D, Kumar A, Bakkum-Gamez JN, Weaver AL, McGree ME, Wang C, et al. Mesenchymal molecular subtype is an independent predictor of severe postoperative complications after primary debulking surgery for advanced ovarian cancer. Gynecol Oncol (2019) 152(2):223–7. doi: 10.1016/j.ygyno.2018.11.019

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Heitz F, Kommoss S, Tourani R, Grandelis A, Uppendahl L, Aliferis C, et al. Dilution of molecular-pathologic gene signatures by medically associated factors might prevent prediction of resection status after debulking surgery in patients with advanced ovarian cancer. Clin Cancer Res (2020) 26(1):213–9. doi: 10.1158/1078-0432.CCR-19-1741

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Liu Z, Beach JA, Agadjanian H, Jia D, Aspuria PJ, Karlan BY, et al. Suboptimal cytoreduction in ovarian carcinoma is associated with molecular pathways characteristic of increased stromal activation. Gynecol Oncol (2015) 139(3):394–400. doi: 10.1016/j.ygyno.2015.08.026

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Tucker SL, Gharpure K, Herbrich SM, Unruh AK, Nick AM, Crane EK, et al. Molecular biomarkers of residual disease after surgical debulking of high-grade serous ovarian cancer. Clin Cancer Res (2014) 20(12):3280–8. doi: 10.1158/1078-0432.CCR-14-0445

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Riester M, Wei W, Waldron L, Culhane AC, Trippa L, Oliva E, et al. Risk prediction for late-stage ovarian cancer by meta-analysis of 1525 patient samples. J Natl Cancer Institute (2014) 106(5). doi: 10.1093/jnci/dju048

CrossRef Full Text | Google Scholar

14. Colditz GA, Hankinson SE. The nurses’ health study: lifestyle and health among women. Nat Rev Cancer (2005) 5(5):388–96. doi: 10.1038/nrc1608

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Rockhill B, Willett WC, Hunter DJ, Manson JE, Hankinson SE, Spiegelman D, et al. Physical activity and breast cancer risk in a cohort of young women. J Natl Cancer Institute (1998) 90(15):1155–60. doi: 10.1093/jnci/90.15.1155

CrossRef Full Text | Google Scholar

16. Vitonis AF, Titus-Ernstoff L, Cramer DW. Assessing ovarian cancer risk when considering elective oophorectomy at the time of hysterectomy. Obstetr Gynecol (2011) 117(5):1042–50. doi: 10.1097/AOG.0b013e318212fcb7

CrossRef Full Text | Google Scholar

17. Shafrir AL, Rice MS, Gupta M, Terry KL, Rosner BA, Tamimi RM, et al. The association between reproductive and hormonal factors and ovarian cancer by estrogen-alpha and progesterone receptor status. Gynecol Oncol (2016) 143(3):628–35. doi: 10.1016/j.ygyno.2016.09.024

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Goulding H, Pinder S, Cannon P, Pearson D, Nicholson R, Snead D, et al. A new immunohistochemical antibody for the assessment of estrogen receptor status on routine formalin-fixed tissue samples. Hum Pathol (1995) 26(3):291–4. doi: 10.1016/0046-8177(95)90060-8

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Huang T, Tworoger SS, Hecht JL, Rice MS, Sood AK, Kubzansky LD, et al. Association of ovarian tumor β2-adrenergic receptor status with ovarian cancer risk factors and survival. Cancer Epidemiol Biomarkers Prevention: Publ Am Assoc Cancer Res Cosponsored Am Soc Prev Oncol (2016) 25(12):1587–94. doi: 10.1158/1055-9965.EPI-16-0534

CrossRef Full Text | Google Scholar

20. Barnard ME, Hecht JL, Rice MS, Gupta M, Harris HR, Eliassen AH, et al. Anti-inflammatory drug use and ovarian cancer risk by COX1/COX2 expression and infiltration of tumor-associated macrophages. Cancer Epidemiol Biomarkers Prev (2018) 27(12):1509–17. doi: 10.1158/1055-9965.EPI-18-0346

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Hecht JL, Kotsopoulos J, Hankinson SE, Tworoger SS. Relationship between epidemiologic risk factors and hormone receptor expression in ovarian cancer: results from the nurses’ health study. Cancer Epidemiol Biomarkers Prev (2009) 18(5):1624–30. doi: 10.1158/1055-9965.Epi-08-1214

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Shafrir AL, Babic A, Gates Kuliszewski M, Rice MS, Townsend MK, Hecht JL, et al. Estrogen receptor-β expression of ovarian tumors and its association with ovarian cancer risk factors. Cancer Epidemiol Biomarkers Prev (2020) 29(11):2211–9. doi: 10.1158/1055-9965.Epi-20-0618

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Harris HR, Rice MS, Shafrir AL, Poole EM, Gupta M, Hecht JL, et al. Lifestyle and reproductive factors and ovarian cancer risk by p53 and MAPK expression. Cancer Epidemiol Biomarkers Prev (2018) 27(1):96–102. doi: 10.1158/1055-9965.EPI-17-0609

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Kurman RJ, Shih Ie M. The origin and pathogenesis of epithelial ovarian cancer: a proposed unifying theory. Am J Surg Pathol (2010) 34(3):433–43. doi: 10.1097/PAS.0b013e3181cf3d79

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Hoeting JA, Madigan D, Raftery AE, Volinsky CT. Bayesian Model averaging: A tutorial. Stat Sci (1999) 14(4):382–417.

Google Scholar

26. Trabert B, Poole EM, White E, Visvanathan K, Adami HO, Anderson GL, et al. Analgesic use and ovarian cancer risk: An analysis in the ovarian cancer cohort consortium. J Natl Cancer Institute (2019) 111(2):137–45. doi: 10.1093/jnci/djy100

CrossRef Full Text | Google Scholar

27. Dixon SC, Nagle CM, Wentzensen N, Trabert B, Beeghly-Fadiel A, Schildkraut JM, et al. Use of common analgesic medications and ovarian cancer survival: results from a pooled analysis in the ovarian cancer association consortium. Br J Cancer (2017) 116(9):1223–8. doi: 10.1038/bjc.2017.68

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Poole EM, Merritt MA, Jordan SJ, Yang HP, Hankinson SE, Park Y, et al. Hormonal and reproductive risk factors for epithelial ovarian cancer by tumor aggressiveness. Cancer Epidemiol Biomarkers Prev (2013) 22(3):429–37. doi: 10.1158/1055-9965.Epi-12-1183-t

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Fortner RT, Poole EM, Wentzensen NA, Trabert B, White E, Arslan AA, et al. Ovarian cancer risk factors by tumor aggressiveness: An analysis from the ovarian cancer cohort consortium. Int J Cancer (2019) 145(1):58–69. doi: 10.1002/ijc.32075

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Praestegaard C, Jensen A, Jensen SM, Nielsen TS, Webb PM, Nagle CM, et al. Cigarette smoking is associated with adverse survival among women with ovarian cancer: Results from a pooled analysis of 19 studies. Int J Cancer (2017) 140(11):2422–35. doi: 10.1002/ijc.30600

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Shiels MS, Katki HA, Freedman ND, Purdue MP, Wentzensen N, Trabert B, et al. Cigarette smoking and variations in systemic immune and inflammation markers. J Natl Cancer Institute (2014) 106(11). doi: 10.1093/jnci/dju294

CrossRef Full Text | Google Scholar

32. Jeon SY, Go RE, Heo JR, Kim CW, Hwang KA, Choi KC. Effects of cigarette smoke extracts on the progression and metastasis of human ovarian cancer cells via regulating epithelial-mesenchymal transition. Reprod Toxicol (2016) 65:1–10. doi: 10.1016/j.reprotox.2016.06.012

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Goode EL, Block MS, Kalli KR, Vierkant RA, Chen W, Fogarty ZC, et al. Dose-response association of CD8+ tumor-infiltrating lymphocytes and survival time in high-grade serous ovarian cancer. JAMA Oncol (2017) 3(12):e173290. doi: 10.1001/jamaoncol.2017.3290

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Zhang AW, McPherson A, Milne K, Kroeger DR, Hamilton PT, Miranda A, et al. Interfaces of malignant and immunologic clonal dynamics in ovarian cancer. Cell (2018) 173(7):1755–1769.e22. doi: 10.1016/j.cell.2018.03.073

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Sieh W, Köbel M, Longacre TA, Bowtell DD, deFazio A, Goodman MT, et al. Hormone-receptor expression and ovarian cancer survival: an ovarian tumor tissue analysis consortium study. Lancet Oncol (2013) 14(9):853–62. doi: 10.1016/s1470-2045(13)70253-5

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Carter LE, Cook DP, Collins O, Gamwell LF, Dempster HA, Wong HW, et al. COX2 is induced in the ovarian epithelium during ovulatory wound repair and promotes cell survival†. Biol Reprod (2019) 101(5):961–74. doi: 10.1093/biolre/ioz134

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Nagaraja AS, Dorniak PL, Sadaoui NC, Kang Y, Lin T, Armaiz-Pena G, et al. Sustained adrenergic signaling leads to increased metastasis in ovarian cancer via increased PGE2 synthesis. Oncogene (2016) 35(18):2390–7. doi: 10.1038/onc.2015.302

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Fagotti A, Ferrandina G, Vizzielli G, Fanfani F, Gallotta V, Chiantera V, et al. Phase III randomised clinical trial comparing primary surgery versus neoadjuvant chemotherapy in advanced epithelial ovarian cancer with high tumour load (SCORPION trial): Final analysis of peri-operative outcome. Eur J Cancer (2016) 59:22–33. doi: 10.1016/j.ejca.2016.01.017

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Stewart SL, Cooney D, Hirsch S, Westervelt L, Richards TB, Rim SH, et al. The effect of gynecologic oncologist availability on ovarian cancer mortality. World J Obstet Gynecol (2014) 3(2):71–7. doi: 10.5317/wjog.v3.i2.71

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Engbersen MP, Lahaye MJ, Lok CAR, Koole SN, Sonke GS, Beets-Tan RGH, et al. Peroperative scoring systems for predicting the outcome of cytoreductive surgery in advanced-stage ovarian cancer - a systematic review. Eur J Surg Oncol (2021) 47(8):1856–61. doi: 10.1016/j.ejso.2021.03.233

PubMed Abstract | CrossRef Full Text | Google Scholar

Keywords: ovarian cancer, debulking, residual disease, prediction model, immunohistochemistry, tissue microarray

Citation: Abbas-Aghababazadeh F, Sasamoto N, Townsend MK, Huang T, Terry KL, Vitonis AF, Elias KM, Poole EM, Hecht JL, Tworoger SS and Fridley BL (2023) Predictors of residual disease after debulking surgery in advanced stage ovarian cancer. Front. Oncol. 13:1090092. doi: 10.3389/fonc.2023.1090092

Received: 04 November 2022; Accepted: 06 January 2023;
Published: 24 January 2023.

Edited by:

Sara Ricardo, Universidade do Porto, Portugal

Reviewed by:

Mikel Gorostidi, University of the Basque Country, Spain
Marion Curtis, Mayo Clinic Arizona, United States

Copyright © 2023 Abbas-Aghababazadeh, Sasamoto, Townsend, Huang, Terry, Vitonis, Elias, Poole, Hecht, Tworoger and Fridley. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Brooke L. Fridley, Brooke.Fridley@moffitt.org

These authors have contributed equally to this work and share first authorship

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.