Comparison of the Diagnostic Performances of Ultrasound-Based Models for Predicting Malignancy in Patients With Adnexal Masses

Aim: This study aimed to compare different ultrasound-based International Ovarian Tumor Analysis (IOTA) prediction models, namely, the Simple Rules (SRs) the Assessment of Different NEoplasias in the adneXa (ADNEX) models, and the Risk of Malignancy Index (RMI), for the pre-operative diagnosis of adnexal mass.

Methods: This single-centre diagnostic accuracy study involved 486 patients. All ultrasound examinations were analyzed and the prediction models were applied. Pathology was the clinical reference standard. The diagnostic performances of prediction models were measured by evaluating receiver-operating characteristic curves, sensitivities, specificities, positive and negative predictive values, positive and negative likelihood ratios, and diagnostic odds ratios.

Results: To discriminate benign and malignant tumors, areas under the ROC curves (AUCs) for ADNEX models were 0.94 (95% CI: 0.92–0.96) with CA125 and 0.94 (95% CI: 0.91–0.96) without CA125, which were significantly higher than the AUCs for RMI I-III: 0.87 (95% CI: 0.83–0.90), 0.83 (95% CI: 0.80–0.86), and 0.82 (95% CI: 0.78–0.86), (all P < 0.0001). At a cut-off of 10%, the ADNEX model with CA125 had the highest sensitivity (0.93; 95% CI: 0.87–0.97) compared with the other models. The SRs model achieved a sensitivity of 0.93 (95% CI: 0.86–0.97) and a specificity of 0.86 (95% CI: 0.82–0.89) when inconclusive diagnoses (11.7%) were classified as malignant.

Conclusion: ADNEX and SRs models were excellent at characterising adnexal masses which were superior to the RMI in Chinese patients.

Introduction

Ovarian cancer (OC) has the highest mortality rate and most unfavourable prognosis among the gynaecological malignancies; the average 5-year survival rate is < 50% (1, 2). Currently, transvaginal ultrasound is the most commonly used, non-invasive, affordable imaging technique for pre-operative evaluations of adnexal masses with minimal risk and discomfort to the patient (3–5). And subjective assessments of ultrasound findings by specialists in gynaecological ultrasonography are one of the best means of evaluating adnexal masses in clinical practice (5–10). To optimise the treatment and improve the survival of patients with malignant ovarian tumors, and to avoid unnecessary interventions in and preserve the fertility of patients with benign ovarian tumors, accurately characterising benign and malignant ovarian masses through appropriate staging is essential (11, 12). In particularly, accurate diagnosis of borderline ovarian tumors (BOTs) is critical to ensure timely and appropriate management, especially in women desiring to preserve fertility (13–17).

Several ultrasound-based prediction models have been developed to accurately discriminate between benign and malignant tumors, because the numbers of experienced examiners are insufficient and they are unavailable in some regions (18). The Risk of Malignancy Index (RMI), which accounts for the serum cancer antigen (CA) 125 levels, menopausal status, and the ultrasound findings, is a prediction model that is recommended by many national guidelines (19–21). However, the procedures used to calculate the RMI are time-consuming, and its diagnostic performance is unsatisfactory. The International Ovarian Tumor Analysis (IOTA) group presented a consensus statement about the ultrasound characteristics of adnexal tumors in 2000 (22), and other diagnostic models were subsequently developed and validated, including the Logistic Regression model 2 (23, 24), simple ultrasound-based rules or Simple Rules (SRs) model (25–27), and the Assessment of Different NEoplasias in the adneXa (ADNEX) model (28). The findings from previous external validation studies have shown that the SRs model is easy to use and its diagnostic performance is good, but it is not suitable for all adnexal masses (25–27). Although the ADNEX model is excellent at differentiating between malignant and benign tumors (6, 29–31) and indicating the stages of malignant tumors, there is still no diagnostic accuracy study to compare these models above-mentioned in a Chinese setting.

This study aimed to compare the ADNEX and SRs models, and the RMI regarding their abilities to discriminate between benign and malignant adnexal masses using data from a single oncology centre in China.

Material and Methods

Study Setting and Design

Between June 2017 and June 2018, the study was carried out using data prospectively collected from consecutive patients. It evaluated the diagnostic performances of the ADNEX and SRs models, and variants of the RMI (I–III) within a population of women who underwent surgery to remove adnexal masses at the Department of Obstetrics and Gynaecology in a tertiary referral oncology centre. All of the patients underwent pre-operative transvaginal or transrectal ultrasonography examinations according to the IOTA protocol (22) to assess the morphology of the adnexal masses. Clinicians made the final decisions regarding surgery and clinical judgments.

Participants

The patients were prospectively and consecutively enrolled, and they presented with ≥ 1 ultrasound-diagnosed adnexal mass. The inclusion criteria were ≥ 1 adnexal mass detected by transvaginal or transrectal ultrasonography that was not a physiological cyst, patients who were prepared to undergo surgery based on a clinician’s recommendation, and a time interval of 30 days between ultrasound and surgery.

Participants were excluded from the study if they failed to undergo surgery, they were diagnosed with a recurrence of OC, they had undergone a bilateral adnexectomy previously, they had an ectopic pregnancy, or their clinical data were incomplete. A total of 486 patients were included in the final analysis. The study was approved by the Ruijin Hospital, Shanghai Jiaotong University School of Medicine institutional ethics (Grant No.2018-136).

Data Collection

All patients underwent pre-operative transvaginal or transrectal ultrasonography using Voluson E10 (GE Healthcare) and iU22 (Philips Healthcare) ultrasound machines with 5.0–9.0 MHz and 4.0–8.0 MHz transvaginal probes, and 1.0–5.0 MHz transabdominal probes, and the findings were recorded. When a malignancy was suspected or a mass was too large to be evaluated using transvaginal ultrasonography alone, transabdominal ultrasonography was performed. Two expert ultrasonographers with ≥ 10 years of experience in gynaecological ultrasound assessed the tumors’ pre-operative sonographic morphologies using the IOTA protocol’s nomenclature and methodology (22). After the ultrasound examinations and before the statistical analysis of the data, we applied the ADNEX model and three variants of the RMI to calculate the risk of malignancy without knowledge of the histological findings. When multiple adnexal masses were detected, we analysed the mass with the most complex ultrasonographic morphology, and when masses had similar morphological characteristics, we chose the largest mass (22, 28).

Before the ultrasound examinations, we collected clinical data describing the patients’ ages, menopausal statuses, previous malignancies, and family histories of OC. The patients’ pre-operative CA125 levels were measured using a chemoluminescence technique and an automatic analyser (i2000SN; Abbott AxSYM).

Prediction Models

Three prediction models were used to differentiate between benign and malignant adnexal masses. The ADNEX model is available at no cost on the IOTA website (https://www.iotagroup.org/iota-models-software/adnex-risk-model) or it can be installed as a mobile phone application; it comprises nine predictors, including three clinical and six ultrasound variables (28). After inputting all the predictors objectively, the probability ratios for a benign or a malignant mass are displayed both graphically and numerically. As it is the first multiclass prediction model for adnexal masses, the likelihoods of a mass being a BOT, stage I OC, stages II–IV OC, or a metastasis are presented. The ADNEX model is available in versions that include and exclude the CA125 level, and we evaluated the predictive accuracy of the ADNEX model with and without CA125 in this study.

The SRs model comprises a set of rules based on five ultrasound features that indicate benignity (B-features) and five features that indicate malignancy (M-features) (25–27). A lesion is classified as benign if ≥ 1 B-feature is present in the absence of any M-features, and malignant if ≥ 1 M-feature is present in the absence of any B-features. If both B-features and M-features are present or if none of the features are present, the model yields an inconclusive result.

Three principal variants of the RMI scoring system (RMI-I, RMI-II, and RMI-III) were applied that combined the ultrasound findings, serum CA125 levels, and menopausal status (19–21). The points attributed to patients’ ultrasound findings and menopausal statuses differ for the RMI variants, and these points generate a score; a total score of ≥ 200 points was used as the cut-off for malignancy.

Reference Standard

Pathology was the reference standard used for all patients in this study. Tissue specimens obtained during surgery were analysed by a team of pathologists who specialised in gynaecological pathology and were unaware of the ultrasound findings. The tumors were classified according to the World Health Organization’s guidelines for the classification of tumors (32). The stages of the malignant tumors were defined using the International Federation of Gynecology and Obstetrics 2012 criteria (33).

Statistical Analyses

Basic discrimination between benign and malignant adnexal masses by the ADNEX model with or without the CA125 levels and the three RMI variants was assessed using receiver-operating characteristic curves (ROCs) and summarised by calculating the areas under the curves (AUCs). The prediction methods’ AUCs were compared using the method described by DeLong et al. (34). As AUCs could not be calculated for the SRs model, which is based on categorical variables, the McNemar test was used to assess the model’s discrimination between benign and malignant adnexal masses. Diagnostic performance measures, including the sensitivities, specificities, positive and negative predictive values, positive and negative likelihood ratios, and the diagnostic odds ratios (DORs), were calculated to evaluate the models’ classifications of benign or malignant tumors using cut-off points proposed in previous publications (6, 19–21, 28).

The ultrasonographic and clinical characteristics of, and the CA125 levels associated with the benign and malignant tumors were compared; the chi-square test and Fisher’s exact test were used to analyse the categorical data, and the Mann-Whitney U-test was used to analyse the continuous data. The statistical analyses were conducted using IBM^®SPSS^® software, version 22.0 (IBM Corporation) and MedCalc Statistical Software, version 15.2.2 (MedCalc Software bvba). BOTs were considered malignant for the purposes of the statistical analyses. All of the statistical calculations were performed using 95% confidence intervals (CIs), and a value of P < 0.05 was considered statistically significant.

Results

Clinical Findings and Pathologic Diagnosis

Between June 2017 and June 2018, 591 consecutive women with adnexal tumors who underwent pre-operative ultrasound examinations were prospectively enrolled. The final cohort consisted of 486 women; 105 women met the exclusion criteria and were excluded from study. Figure 1 provides a detailed overview of the patients’ inclusion and exclusion from the study.

FIGURE 1

Figure 1 Flow diagram summarizing the inclusion of patients with adnexal masses in the study. *The inclusion criteria were as follows: (1) patients with at least one adnexal mass detected by transvaginal or transrectal ultrasonography, (2) patients who were prepared to undergo surgery, (3) patients with a time interval between ultrasound and surgery within 30 days, and (4) patients without a previous history of ovarian cancer.

In the final analysis, 486 patients with 366 (75.3%) benign and 120 (24.7%) malignant adnexal masses were included. Table 1 presents the histological results. Endometriomas (19.8%, 96/486) and serous cystadenomas (13.4%, 65/486) were the most common benign diagnoses. Among the malignant masses, 7.0% (34/486) were BOTs, 4.9% (24/486) were stage I OCs, 9.7% (47/486) were stages II–IV OCs, and 3.1% (15/486) were metastases.

TABLE 1

Table 1 Distributions of histology outcomes of 486 adnexal masses.

Table 2 summarises the patients’ clinical characteristics and data describing the ultrasound findings from the benign and malignant tumors. The patients with malignancies were older, were more likely to be post-menopausal and to have a family history of OC, and had higher CA125 levels than those with benign tumors (all P < 0.05). Regarding the ultrasound findings, the malignant tumors had significantly greater diameters, more solid tissue, wider solid tissue components, > 10-cyst locules, more papillary projections, and more ascites compared with the benign masses (all P < 0.001). None of the patients with malignant tumors had acoustic shadows.

TABLE 2

Table 2 Results regarding clinical characteristics and ultrasound features for 486 patients with adnexal mass.

Diagnostic Performance of Adnexal Mass Prediction Models

Table 3 details the diagnostic performances of the adnexal mass prediction models regarding their discrimination between benign and malignant tumors. The AUCs for the ADNEX models for differentiating malignant tumors from benign tumors that did and did not account for the CA125 level were 0.94 (95% CI: 0.92–0.96) and 0.94 (95% CI: 0.91–0.96), respectively. At a cut-off of 10%, the performance of the prediction model that included CA125 was excellent, with a sensitivity of 0.93 (95% CI: 0.87–0.97), a specificity of 0.76 (95% CI: 0.72–0.81), and a DOR of 43.67, and the performance of the prediction model that did not include CA125 had a sensitivity of 0.93 (95% CI: 0.87–0.97), a specificity of 0.74 (95% CI: 0.69–0.79), and a DOR of 40.00.

TABLE 3

Table 3 Diagnostic performance of the prediction models for discrimination between benign and malignant adnexal masses.

The SRs model was applicable to 422 (86.8%) patients with adnexal tumors. Of the tumors with inconclusive diagnoses, 56.3% (36/64) were benign tumors, 20.3% (13/64) were BOTs, 14.1% (9/64) were stage I OCs, 4.7% (3/64) were stages I–IV OCs, 4.7% (3/64) were metastases, approximately 43.8% were malignant histologically, and most of the benign masses (75.0%, 27/36) presented with a solid component. When the masses with inconclusive diagnoses were classified as benign, the SRs model’s diagnostic performance had a sensitivity of 0.69 (95% CI: 0.60–0.77), a specificity of 0.96 (95% CI: 0.93–0.97), and a DOR of 49.44, and when they were categorised as malignant, the SRs model’s diagnostic performance had a sensitivity of 0.93 (95% CI: 0.86–0.97), a specificity of 0.86 (95% CI: 0.82–0.89), and a DOR of 72.33.

The three RMI variants with cut-offs of 200 showed poor diagnostic performances in relation to the adnexal masses. Regarding RMI variants I, II, and III, the AUCs for differentiating malignant tumors from benign tumors were 0.87 (95% CI: 0.83–0.90), 0.83 (95% CI: 0.80–0.86), and 0.82 (95% CI: 0.78–0.86), respectively, with sensitivities of 0.55 (95% CI: 0.46–0.64), 0.61 (95% CI: 0.52–0.70), and 0.53 (95% CI: 0.44–0.63), respectively, and specificities of 0.93 (95% CI: 0.90–0.96), 0.92 (95% CI: 0.89–0.95), and 0. 94 (95% CI: 0.91–0.96), respectively.

Figure 2 shows the ROC curves for the ADNEX model and the RMI variants for differentiating malignant and benign tumors. The ADNEX model with or without CA125 was superior to the RMI variants regarding the diagnosis of malignant and benign tumors. When the SRs model yielded inconclusive results that were classified as malignancies, the model’s diagnostic performance was good.

FIGURE 2

Figure 2 Receiver-operating characteristic curves for the performance of prediction models in detection of malignant adnexal masses. The prediction models including: Assessment of Different NEoplasias in the adneXa (ADNEX) model and three variants of the Risk of Malignancy Index (RMI). The red and green ROC point represents the Simple Rules model applied with inconclusive diagnoses classified as either benign (red) or malignant (green).

Table 4 summarises the pairwise ROC curve comparisons of the ADNEX model with or without CA125 and the RMI, which are expressed as differences in the AUCs. The difference between the AUCs for the ADNEX model with or without CA125 was not significant (AUC difference: 0.0002; 95% CI: 0.01–0.02). Comparisons of the ADNEX model with or without CA125 and the three RMI variants revealed significant differences in the AUCs that ranged from 0.074 to 0.118 (all P < 0.0001). Comparisons of the ADNEX model with and without CA125 with RMI variant I showed the greatest differences in the AUC (AUC difference: 0.074; 95% CIs: 0.039–0.109 and 0.040–0.108, respectively; P < 0.0001). The diagnostic performances of the three RMI variants remained statistically significant for the pre-operative diagnosis of adnexal masses (AUC differences: 0.010–0.044; all P < 0.05).

TABLE 4

Table 4 Pairwise ROC curve comparisons expressed as differences in AUC and P-values for the study.

Discussion

Correctly discriminating between benign and malignant adnexal masses is a crucial starting point for optimal treatment. We compared the diagnostic performances of the ADNEX and SRs models, and the RMI. The RMI was the first prediction model used clinically, and it is the most widely used model in many regions (4, 35, 36). However, our study’s findings showed that the ADNEX model was superior to the three RMI variants at distinguishing between benign and malignant adnexal masses. The ADNEX model with and without CA125 had higher AUCs (both 0.94) than the AUCs generated for the RMI variants that ranged from 0.82 to 0.87. Like previous studies’ findings (6, 28), the ADNEX model showed a better diagnostic performance and a higher level of sensitivity than the RMI in our study. Hence, the ADNEX model might be favoured for pre-operatively differentiating adnexal masses in Chinese patients.

Pre-operative evaluations using the SRs model were robust, with a sensitivity of 0.93 (95% CI: 0.86–0.97) and a specificity of 0.86 (95% CI: 0.82–0.89) for adnexal masses with inconclusive diagnoses that were classified as malignant; these findings are similar to the results from previous studies (6, 26, 30, 37, 38). The IOTA SRs model is widely accepted as an effective prediction model for adnexal masses by clinicians, and its use is recommended in the 2011 Green-top guidelines for the assessment and management of suspected ovarian masses in pre-menopausal women that were developed by the Royal College of Obstetricians and Gynaecologists in the United Kingdom (39). Recently, the American College of Obstetricians and Gynecologists incorporated the SRs model into their clinical practice guidelines for the evaluation and management of adnexal masses (40). Followed in the First International Consensus on Adnexal Masses the SRs model was recommended as the main diagnostic strategy (41). The SRs model is easy to apply in clinical practice, and it can be used for approximately 76-89% of adnexal masses (26, 42). The SRs model was applicable to about 86.8% of the patients in our study. When specialists in gynaecological ultrasonography are not available, classifying tumors as malignant is reasonable following inconclusive diagnoses using the SRs model (26, 43). However, this approach could be biased by the prevalence of malignant tumors within the population, and approximately half of the patients with benign diagnoses might undergo unnecessary interventions (26, 43).

Our analyses determined that 64 patients had tumors with inconclusive diagnoses following the application of the SRs protocol to the ADNEX model with or without CA125 and the three RMI variants. Compared with the three RMI variants, the AUC for the ADNEX model was higher (0.59 vs 0.73), the sensitivity was greater (0.29–0.36 vs 0.89), and the specificities were lower (0.86–0.89 vs 0.33–0.39) (Supplementary Tables 1, 2). Regarding the tumors with inconclusive diagnoses, the prediction models’ AUCs did not differ, which may be attributable to the limited sample size. Nevertheless, regarding the identification of malignant tumors among the masses with inconclusive diagnoses, the ADNEX model yielded slightly higher AUCs and DORs than the three RMI variants.

This is one of the first studies to compare the ultrasound-based IOTA prediction models and the RMI in a population of Chinese patients in strict accordance with the IOTA consensus statement, which is a study strength. Additionally, we prospectively and consecutively enrolled unselected patients, and only patients whose data were complete were included. Moreover, our results were validated within a relatively large total study population between benign and malignant patients, however the sample size in particular subtypes was still limited. The study’s weakness, namely, its single-centre design, may have caused a sampling bias and limited the applicability of the results to other regions. Moreover, the ultrasound examinations were not performed by those with different levels of training experience in our study. More studies in different diagnostic centres with different levels of ultrasound expertise in China are needed to further evaluate the prediction models.

In conclusion, our study’s findings showed that the ADNEX and SRs models performed well in relation to discriminating between benign and malignant adnexal masses, and that both models were superior to the RMI in a Chinese context.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The study was approved by the Ruijin Hospital, Shanghai Jiaotong University School of Medicine institutional ethics (Grant No.2018-136).

Author Contributions

LQ: project development, data collection, data analysis and manuscript writing. QD: data analysis and manuscript editing. MJ: data collection and manuscript writing. FY: provided advice for the manuscript. HC: protocol and project development, appraised and revised the manuscript. WF: appraised and revised the manuscript. All authors contributed to the article and approved the submitted version.

Funding

This work was sponsored by Medical Innovation Project of Shanghai Science and Technology Commission (20Y11914000), The ultrasonic model in diagnosis of ovarian tumors (KY20200033), Shanghai Sailing Program (18YF1414400) and National Natural Science Foundation of China (81901488).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors are grateful to all participants of this study.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fonc.2021.673722/full#supplementary-material

Abbreviations

ADNEX, Assessment of Different NEoplasias in the adnexa; AUC, area under the curve; BOT, borderline ovarian tumor; CA, carbohydrate antigen; Cis, confidence intervals; DOR, diagnostic odds ratio; FIGO, International Federation of Gynecology and Obstetrics; IOTA, International Ovarian Tumor Analysis; LR2, Logistic Regression model 2; MHz, megahertz; OC, ovarian cancer; ROC, receiver-operating characteristics; RMI, Risk of Malignancy Index; SRs, the Simple Rules; WHO, World Health Organization.

References

1. Jayson GC, Kohn EC, Kitchener HC, Ledermann JA. Ovarian Cancer. Lancet (2014) 384:1376–88. doi: 10.1016/s0140-6736(13)62146-7

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Siegel RL, Miller KD, Jemal A. Cancer Statistics, 2019. CA Cancer J Clin (2019) 69:7–34. doi: 10.3322/caac.21551

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Piovano E, Cavallero C, Fuso L, Viora E, Ferrero A, Gregori G, et al. Diagnostic Accuracy and Cost-Effectiveness of Different Strategies to Triage Women With Adnexal Masses: A Prospective Study. Ultrasound Obstet Gynecol (2017) 50:395–403. doi: 10.1002/uog.17320

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Menon U, Gentry-Maharaj A, Hallett R, Ryan A, Burnell M, Sharma A, et al. Sensitivity and Specificity of Multimodal and Ultrasound Screening for Ovarian Cancer, and Stage Distribution of Detected Cancers: Results of the Prevalence Screen of the UK Collaborative Trial of Ovarian Cancer Screening (UKCTOCS). Lancet Oncol (2009) 10:327–40. doi: 10.1016/s1470-2045(09)70026-9

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Fischerova D. Ultrasound Scanning of the Pelvis and Abdomen for Staging of Gynecological Tumors: A Review. Ultrasound Obstet Gynecol (2011) 38:246–66. doi: 10.1002/uog.10054

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Meys EMJ, Jeelof LS, Achten NMJ, Slangen BFM, Lambrechts S, Kruitwagen R, et al. Estimating Risk of Malignancy in Adnexal Masses: External Validation of the ADNEX Model and Comparison With Other Frequently Used Ultrasound Methods. Ultrasound Obstet Gynecol (2017) 49:784–92. doi: 10.1002/uog.17225

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Meys EM, Kaijser J, Kruitwagen RF, Slangen BF, Van Calster B, Aertgeerts B, et al. Subjective Assessment Versus Ultrasound Models to Diagnose Ovarian Cancer: A Systematic Review and Meta-Analysis. Eur J Cancer (2016) 58:17–29. doi: 10.1016/j.ejca.2016.01.007

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Valentin L, Hagen B, Tingulstad S, Eik-Nes S. Comparison of ‘Pattern Recognition’ and Logistic Regression Models for Discrimination Between Benign and Malignant Pelvic Masses: A Prospective Cross Validation. Ultrasound Obstet Gynecol (2001) 18:357–65. doi: 10.1046/j.0960-7692.2001.00500.x

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Van Gorp T, Veldman J, Van Calster B, Cadron I, Leunen K, Amant F, et al. Subjective Assessment by Ultrasound is Superior to the Risk of Malignancy Index (RMI) or the Risk of Ovarian Malignancy Algorithm (ROMA) in Discriminating Benign From Malignant Adnexal Masses. Eur J Cancer (2012) 48:1649–56. doi: 10.1016/j.ejca.2011.12.003

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Timmerman D, Schwärzler P, Collins WP, Claerhout F, Coenen M, Amant F, et al. Subjective Assessment of Adnexal Masses With the Use of Ultrasonography: An Analysis of Interobserver Variability and Experience. Ultrasound Obstet Gynecol (1999) 13:11–6. doi: 10.1046/j.1469-0705.1999.13010011.x

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Valentin L, Ameye L, Testa A, Lécuru F, Bernard JP, Paladini D, et al. Ultrasound Characteristics of Different Types of Adnexal Malignancies. Gynecol Oncol (2006) 102:41–8. doi: 10.1016/j.ygyno.2005.11.015

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Di Legge A, Testa AC, Ameye L, Van Calster B, Lissoni AA, Leone FP, et al. Lesion Size Affects Diagnostic Performance of IOTA Logistic Regression Models, IOTA Simple Rules and Risk of Malignancy Index in Discriminating Between Benign and Malignant Adnexal Masses. Ultrasound Obstet Gynecol (2012) 40:345–54. doi: 10.1002/uog.11167

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Zheng X, Lyu G, Gan Y, Hu M, Liu X, Chen S, et al. Microcystic Pattern and Shadowing are Independent Predictors of Ovarian Borderline Tumors and Cystadenofibromas in Ultrasound. Eur Radiol (2021) 31:45–54. doi: 10.1007/s00330-020-07113-z

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Valentin L, Ameye L, Savelli L, Fruscio R, Leone FP, Czekierdowski A, et al. Adnexal Masses Difficult to Classify as Benign or Malignant Using Subjective Assessment of Gray-Scale and Doppler Ultrasound Findings: Logistic Regression Models do Not Help. Ultrasound Obstet Gynecol (2011) 38:456–65. doi: 10.1002/uog.9030

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Fagotti A, Ludovisi M, De Blasis I, Virgilio B, Di Legge A, Mascilini F, et al. The Sonographic Prediction of Invasive Carcinoma in Unilocular-Solid Ovarian Cysts in Premenopausal Patients: A Pilot Study. Hum Reprod (2012) 27:2676–83. doi: 10.1093/humrep/des231

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Sokalska A, Timmerman D, Testa AC, Van Holsbeke C, Lissoni AA, Leone FP, et al. Diagnostic Accuracy of Transvaginal Ultrasound Examination for Assigning a Specific Diagnosis to Adnexal Masses. Ultrasound Obstet Gynecol (2009) 34:462–70. doi: 10.1002/uog.6444

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Guadagno E, Pignatiello S, Borrelli G, Cervasio M, Della Corte L, Bifulco G, et al. Ovarian Borderline Tumors, a Subtype of Neoplasm With Controversial Behavior. Role of Ki67 as a Prognostic Factor. Pathol Res Pract (2019) 215:152633. doi: 10.1016/j.prp.2019.152633

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Terzic M, Aimagambetova G, Norton M, Della Corte L, Marín-Buck A, Lisón JF, et al. Scoring Systems for the Evaluation of Adnexal Masses Nature: Current Knowledge and Clinical Applications. J Obstet Gynaecol (2020) 41:1–8. doi: 10.1080/01443615.2020.1732892

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Jacobs I, Oram D, Fairbanks J, Turner J, Frost C, Grudzinskas JG. A Risk of Malignancy Index Incorporating CA 125, Ultrasound and Menopausal Status for the Accurate Preoperative Diagnosis of Ovarian Cancer. Br J Obstet Gynaecol (1990) 97:922–9. doi: 10.1111/j.1471-0528.1990.tb02448.x

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Tingulstad S, Hagen B, Skjeldestad FE, Onsrud M, Kiserud T, Halvorsen T, et al. Evaluation of a Risk of Malignancy Index Based on Serum CA125, Ultrasound Findings and Menopausal Status in the Pre-Operative Diagnosis of Pelvic Masses. Br J Obstet Gynaecol (1996) 103:826–31. doi: 10.1111/j.1471-0528.1996.tb09882.x

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Tingulstad S, Hagen B, Skjeldestad FE, Halvorsen T, Nustad K, Onsrud M. The Risk-of-Malignancy Index to Evaluate Potential Ovarian Cancers in Local Hospitals. Obstet Gynecol (1999) 93:448–52. doi: 10.1097/00006250-199903000-00028

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Timmerman D, Valentin L, Bourne TH, Collins WP, Verrelst H, Vergote I. Terms, Definitions and Measurements to Describe the Sonographic Features of Adnexal Tumors: A Consensus Opinion From the International Ovarian Tumor Analysis (IOTA) Group. Ultrasound Obstet Gynecol (2000) 16:500–5. doi: 10.1046/j.1469-0705.2000.00287.x

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Timmerman D, Testa AC, Bourne T, Ferrazzi E, Ameye L, Konstantinovic ML, et al. Logistic Regression Model to Distinguish Between the Benign and Malignant Adnexal Mass Before Surgery: A Multicenter Study by the International Ovarian Tumor Analysis Group. J Clin Oncol (2005) 23:8794–801. doi: 10.1200/jco.2005.01.7632

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Timmerman D, Van Calster B, Testa AC, Guerriero S, Fischerova D, Lissoni AA, et al. Ovarian Cancer Prediction in Adnexal Masses Using Ultrasound-Based Logistic Regression Models: A Temporal and External Validation Study by the IOTA Group. Ultrasound Obstet Gynecol (2010) 36:226–34. doi: 10.1002/uog.7636

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Timmerman D, Testa AC, Bourne T, Ameye L, Jurkovic D, Van Holsbeke C, et al. Simple Ultrasound-Based Rules for the Diagnosis of Ovarian Cancer. Ultrasound Obstet Gynecol (2008) 31:681–90. doi: 10.1002/uog.5365

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Timmerman D, Ameye L, Fischerova D, Epstein E, Melis GB, Guerriero S, et al. Simple Ultrasound Rules to Distinguish Between Benign and Malignant Adnexal Masses Before Surgery: Prospective Validation by IOTA Group. BMJ (2010) 341:c6839. doi: 10.1136/bmj.c6839

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Timmerman D, Van Calster B, Testa A, Savelli L, Fischerova D, Froyman W, et al. Predicting the Risk of Malignancy in Adnexal Masses Based on the Simple Rules From the International Ovarian Tumor Analysis Group. Am J Obstet Gynecol (2016) 214:424–37. doi: 10.1016/j.ajog.2016.01.007

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Van Calster B, Van Hoorde K, Valentin L, Testa AC, Fischerova D, Van Holsbeke C, et al. Evaluating the Risk of Ovarian Cancer Before Surgery Using the ADNEX Model to Differentiate Between Benign, Borderline, Early and Advanced Stage Invasive, and Secondary Metastatic Tumours: Prospective Multicentre Diagnostic Study. BMJ (2014) 349:g5920. doi: 10.1136/bmj.g5920

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Araujo KG, Jales RM, Pereira PN, Yoshida A, de Angelo Andrade L, Sarian LO, et al. Performance of the IOTA ADNEX Model in Preoperative Discrimination of Adnexal Masses in a Gynecological Oncology Center. Ultrasound Obstet Gynecol (2017) 49:778–83. doi: 10.1002/uog.15963

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Sayasneh A, Wynants L, Preisler J, Kaijser J, Johnson S, Stalder C, et al. Multicentre External Validation of IOTA Prediction Models and RMI by Operators With Varied Training. Br J Cancer (2013) 108:2448–54. doi: 10.1038/bjc.2013.224

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Chen H, Qian L, Jiang M, Du Q, Yuan F, Feng W. Performance of IOTA ADNEX Model in Evaluating Adnexal Masses in a Gynecological Oncology Center in China. Ultrasound Obstet Gynecol (2019) 54:815–22. doi: 10.1002/uog.20363

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Meinhold-Heerlein I, Fotopoulou C, Harter P, Kurzeder C, Mustea A, Wimberger P, et al. The New WHO Classification of Ovarian, Fallopian Tube, and Primary Peritoneal Cancer and its Clinical Implications. Arch Gynecol Obstet (2016) 293:695–700. doi: 10.1007/s00404-016-4035-8

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Prat J. Staging Classification for Cancer of the Ovary, Fallopian Tube, and Peritoneum. Int J Gynaecol Obstet (2014) 124:1–5. doi: 10.1016/j.ijgo.2013.10.001

PubMed Abstract | CrossRef Full Text | Google Scholar

34. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the Areas Under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach. Biometrics (1988) 44:837–45. doi: 10.2307/2531595

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Andersen ES, Knudsen A, Rix P, Johansen B. Risk of Malignancy Index in the Preoperative Evaluation of Patients With Adnexal Masses. Gynecol Oncol (2003) 90:109–12. doi: 10.1016/s0090-8258(03)00192-6

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Geomini P, Kruitwagen R, Bremer GL, Cnossen J, Mol BW. The Accuracy of Risk Scores in Predicting Ovarian Malignancy: A Systematic Review. Obstet Gynecol (2009) 113:384–94. doi: 10.1097/AOG.0b013e318195ad17

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Kaijser J, Bourne T, Valentin L, Sayasneh A, Van Holsbeke C, Vergote I, et al. Improving Strategies for Diagnosing Ovarian Cancer: A Summary of the International Ovarian Tumor Analysis (IOTA) Studies. Ultrasound Obstet Gynecol (2013) 41:9–20. doi: 10.1002/uog.12323

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Testa A, Kaijser J, Wynants L, Fischerova D, Van Holsbeke C, Franchi D, et al. Strategies to Diagnose Ovarian Cancer: New Evidence From Phase 3 of the Multicentre International IOTA Study. Br J Cancer (2014) 111:680–8. doi: 10.1038/bjc.2014.333

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Royal College of Obstetricians and Gynecologists BSoGE. Management of Suspected Ovarian Masses in Premenopausal Women (2011). Available at: https://www.rcog.org.uk/en/guidelines-research-services/gtg62/(2011) (Accessed 2011 02 December).

Google Scholar

40. Bulletins-Gynecology. Practice Bulletin No.174: Evaluation and Management of Adnexal Masses. Obstet Gynecol (2016) 128:e210–e26. doi: 10.1097/aog.0000000000001768. http://www.American College of Obstetricians and Gynecologists’ Committee on Practice.

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Glanc P, Benacerraf B, Bourne T, Brown D, Coleman BG, Crum C, et al. First International Consensus Report on Adnexal Masses: Management Recommendations. J Ultrasound Med (2017) 36:849–63. doi: 10.1002/jum.14197

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Nunes N, Ambler G, Foo X, Naftalin J, Widschwendter M, Jurkovic D. Use of IOTA Simple Rules for Diagnosis of Ovarian Cancer: Meta-Analysis. Ultrasound Obstet Gynecol (2014) 44:503–14. doi: 10.1002/uog.13437

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Froyman W, Timmerman D. Methods of Assessing Ovarian Masses: International Ovarian Tumor Analysis Approach. Obstet Gynecol Clin North Am (2019) 46:625–41. doi: 10.1016/j.ogc.2019.07.003

PubMed Abstract | CrossRef Full Text | Google Scholar