A Personalized Management Approach of OHSS: Development of a Multiphase Prediction Model and Smartphone-Based App

Objective: This study aimed to develop multiphase big-data-based prediction models of ovarian hyperstimulation syndrome (OHSS) and a smartphone app for risk calculation and patients’ self-monitoring.

Methods: Multiphase prediction models were developed from a retrospective cohort database of 21,566 women from January 2017 to December 2020 with controlled ovarian stimulation (COS). There were 17,445 women included in the final data analysis. Women were randomly assigned to either training cohort (n = 12,211) or validation cohort (n = 5,234). Their baseline clinical characteristics, COS-related characteristics, and embryo information were evaluated. The prediction models were divided into four phases: 1) prior to COS, 2) on the day of ovulation trigger, 3) after oocyte retrieval, and 4) prior to embryo transfer. The multiphase prediction models were built with stepwise regression and confirmed with LASSO regression. Internal validations were performed using the validation cohort and were assessed by discrimination and calibration, as well as clinical decision curves. A smartphone-based app “OHSS monitor” was constructed as part of the built-in app of the IVF-aid platform. The app had three modules, risk prediction module, symptom monitoring module, and treatment monitoring module.

Results: The multiphase prediction models were developed with acceptable distinguishing ability to identify OHSS at-risk patients. The C-statistics of the first, second, third, and fourth phases in the training cohort were 0.628 (95% CI 0.598–0.658), 0.715 (95% CI 0.688–0.742), 0.792 (95% CI 0.770–0.815), and 0.814 (95% CI 0.793–0.834), respectively. The calibration plot showed the agreement of predictive and observed risks of OHSS, especially at the third- and fourth-phase prediction models in both training and validation cohorts. The net clinical benefits of the multiphase prediction models were also confirmed with a clinical decision curve. A smartphone-based app was constructed as a risk calculator based on the multiphase prediction models, and also as a self-monitoring tool for patients at risk.

Conclusions: We have built multiphase prediction models based on big data and constructed a user-friendly smartphone-based app for the personalized management of women at risk of moderate/severe OHSS. The multiphase prediction models and user-friendly app can be readily used in clinical practice for clinical decision-support and self-management of patients.

Introduction

Ovarian hyperstimulation syndrome (OHSS) is a common and severe iatrogenic complication of ovarian hyperstimulation with an incidence of 2% to 6% of moderate OHSS and 0.1% to 0.2% of severe OHSS (1). OHSS is a self-limiting condition, and the symptoms will alleviate shortly. A small proportion of women with moderate/severe OHSS might have persistent discomforts, especially those with late-onset OHSS who will have extended symptoms till early pregnancy.

A variety of measures have been proposed to be useful in the prevention of OHSS, including decreased gonadotropin consumption, GnRH antagonist protocol for controlled ovarian hyperstimulation (COS), and GnRH agonist (GnRHa) for trigger and cryopreservation of all embryos (2). Despite the increasing methods of preventing OHSS, moderate/severe OHSS still occurred on a worldwide scale (3). Complete prevention of OHSS seems to be impossible. Although rare, the mortality risk can happen in around 1 in 450,000 to 500, 000 women (4). Therefore, an early prediction and prevention of OHSS is critical to reducing the morbidity of OHSS.

One challenge for the management of OHSS is to determine what clinical features may predispose the patient to an increased risk of OHSS. Although several prediction models of OHSS had been proposed, the overall predictive value of the possible influential factors of OHSS has not been well studied. Besides, an easy-to-use tool for clinicians to predict OHSS is not available. Moreover, for patients, lack of prompt and proper diagnosis of OHSS is a common problem. Thus, self-monitoring of OHSS-related symptoms and early detection of OHSS are particularly important for those women.

Here, we established big-data-based multiphase prediction models of OHSS. Furthermore, a user-friendly smartphone-based app was built based on the multiphase prediction models to assist clinicians in decision-making and help patients for self-monitoring. The final purpose of this study is to build a user-friendly tool to aid personalized management of OHSS.

Materials and Methods

Study Population

This was a retrospective cohort study, and the study protocol was in accordance with the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statement (5).

Data were obtained from the database in a locally largest reproductive center from January 2017 to December 2020. Infertile women attempting for in vitro fertilization (IVF) or intra-cytoplasmic sperm injection (ICSI) treatment and who underwent COS were included. Each couple had a unique record number for their medical record, which was saved at a local database. Every couple has signed an informed consent for using their medical record for data analysis. The study protocol was reviewed and approved by the hospital ethical committee (approval number, 2021-117).

Data of women who fulfilled the following criteria were included: 1) women aged 20–40 years old and 2) ovarian stimulation with GnRH agonist protocol or GnRH antagonist protocol. There were 21,566 cases included. Then, cases that met the following criteria were excluded: 1) women with recurrent spontaneous miscarriage or recurrent implantation failure (n = 431), 2) IVF cycles transferred from intrauterine insemination cycles (n = 14), 3) preservation of fertility due to malignancies (n = 2), and 4) no oocytes were retrieved (n = 41). Therefore, 21,078 cases were collected for further analysis. Cases with missing data (see Figure 1 for details) were further excluded, and hence a total of 17,445 COS cycles were finally included for the development of the prediction model. Data from 17,445 cycles were randomly assigned to either the training cohort or the validation cohort at a ratio of 5:1.

FIGURE 1

Figure 1 Flowchart of patients’ data inclusion and exclusion. RSA, repeated spontaneous abortion; RIF, repeated implantation failure; IVF, in vitro fertilization; IUI, intrauterine insemination; AMH, anti-Mullerian hormone; AFC, antral follicle counting; OHSS, ovarian hyper-stimulation syndrome.

Ovarian Stimulation Procedures and Embryo Transfer

Two controlled ovarian stimulation protocols were used in the present study, the GnRH agonist protocol (62.1% in the training cohort and 61.7% in the validation cohort) and the GnRH antagonist protocol (37.9% in the training cohort and 38.3% in the validation cohort). Initial doses of 100 to 300 IU/day of recombinant FSH (Gonal-F, Merck, Switzerland; Puregon, Organon, Netherlands; and urofollitropin, Livzon Group, Zhuhai, China) and/or human menopausal gonadotropin (HMG, Livzon Group, China) were administrated. The protocols were chosen based on patients’ individual conditions (including age, ovarian reservation, weight, previous ovarian stimulation response) and physicians’ preferences. Growth of ovarian follicles and serum follicular-stimulating hormone (FSH), luteinizing hormone (LH), estradiol (E2), and progesterone (P) levels was monitored on a regular basis. The gonadotropin (Gn) doses were adjusted according to the development of follicles and serum hormone levels. Triggering of ovulation was provided if the patients had at least three follicles reaching 17 mm or at least two follicles reaching 18 mm in diameter. Transvaginal oocyte retrieval was scheduled 34 to 36 h following the ovulation triggering with HCG (Ovidrel, Merck, Switzerland; HCG, Livzon Group, China) and/or GnRH agonist (triptorelin acetate, Ferring, Kiel, Germany) injection.

Oocytes were collected with a standard oocyte retrieval procedure under the real-time monitoring of transvaginal sonography. Either IVF or ICSI was provided based on semen quality. Fertilization was observed by the appearance of two pronuclei. Day 3 cleavage embryos were monitored and graded. Fresh embryo transfer was scheduled with cleavage embryo on day 3 or blastocysts on day 5 or 6 following oocyte collection. The remaining embryos or embryos in freeze-all strategies were vitrified as cleavage embryos or blastocysts.

In our center, the freeze-all decision was made based on either one of the following conditions: 1) the number of oocytes retrieved ≥20; 2) the serum level of E2 ≥18,350 pmol/L on trigger day; 3) impaired endometrial receptivity induced by endometrial polyps, submucosal fibroid, intrauterine cavity fluid, untreated hydrosalpinx, etc.; 4) other medical conditions not suitable for fresh embryo transfer as determined by physicians; and 5) patient’s personal reasons who cannot arrange fresh embryo transfer.

Outcomes Measured

The primary outcome was the occurrence of moderate/severe OHSS. The diagnosis of moderate/severe OHSS was based on the criteria as suggested by a consensus of Chinese experts (1). The criteria for moderate OHSS are as follows: 1) the patient presents with abdominal distention, nausea, vomit, or diarrhea; 2) sonography exam shows ovarian enlargement to 8–12 cm and presence of ascites; 3) laboratory test shows hematocrit <0.45, elevated leukocyte (10–15 × 10⁹/l). The criteria for severe OHSS are as follows: 1) patient presents with severe nausea, vomit, dyspnea, notable abdominal pain, oliguria, or even anuria (<300 ml/day or <30 ml/h), rapid increase of body weight (>1 kg/24 h); 2) sonography exam shows ovarian enlargement to >12 cm, presence of tension ascites, pleural effusion, vascular embolism, low blood pressure, or low central venous pressure; and 3) laboratory test shows increased hematocrit (>0.45), elevated leukocyte (>15 × 10⁹/l), hyperkalemia (potassium >5 mmol/l), hyponatremia (sodium < 135 mmol/l), damaged renal function (creatine > 1.0 g/l), and damaged liver function (with increased levels of glutamic oxaloacetic transaminase and glutamic pyruvic transaminase). Women who were suspected to develop OHSS were made an appointment with one specialist. Detailed symptoms, physical examination, transabdominal ultrasound, and laboratory tests were recorded in the medical database. Patients who were confirmed with OHSS were administrated aspirin, letrozole, intravenous albumin, or low molecular heparin, as appropriate.

Variables Evaluated in the Models

Potential variables which might be included in the prediction model were prespecified generally based on clinical experience and literature reports. Basic clinical characteristics including female age, infertile duration, infertile type, infertile factors, height, weight, body mass index (BMI), body surface area (equations estimating the body surface area for the Chinese population were derived from (6)), previous live birth, anti-Mullerian hormone (AMH), antra follicle counting (AFC), and diagnosis as polycystic ovarian syndrome (PCOS) were recorded and extracted from a local database. COS cycle-related parameters, including COS cycle number, COS protocol, Gn duration, initial dose and mean dose of Gn, total dose of Gn, number of follicles with various diameters on trigger day, hormonal levels on trigger day (LH, P, and E2), type of trigger (including HCG alone, GnRHa alone, and dual trigger), number of oocytes retrieved, freeze-all strategy, fertilization type, and origin and type of sperms, were obtained. For those with fresh embryo transfer (ET), the number of cycles with ET, number of embryos for ET, number of top-quality embryos for ET, and number of blastocysts for ET were also recorded.

Statistical Analysis

General Statistical Descriptions and Analysis

All data analyses were conducted using SPSS (version 22.0, IBM Corp., Chicago, Illinois, USA) and R package (version 4.1.2, Vienna, Austria). Continuous data following normal distribution were described as mean ± standard deviation (SD) and were compared using Student’s t-test. Continuous data which were not following the normal distribution were described as median and 25th and 75th percentiles and were compared using the Wilcoxon rank-sum test. Categorical data were described as frequency and percentage and were compared using the chi-square test. A P-value < 0.05 was considered statistically significant. All eligible patients were randomized at a 7:3 ratio to create a developmental (training) cohort and a validation cohort, respectively.

Development of the Prediction Models

The multiphase prediction models were developed based on the developmental (training) cohort at the following point times of critical decision-making: 1) prior to COS, 2) prior to ovulation trigger, 3) on the day of oocyte retrieval, and 4) on the day of fresh embryo transfer. The multistage prediction models were made based on variables at corresponding stages. The selection of variables in the above phases was based on the stepwise regression models and confirmed with LASSO regression which demonstrated to have similar performances of prediction. The final models were fitted using stepwise backward selection and presented as odds ratio (OR) and 95% confidential index (95% CI).

Multiphase nomograms with graphic details to predict moderate/severe OHSS in COS cycles were also generated. Values for each covariate were mapped to the points on a special scale ranging from zero to 100. A total point obtained from each covariate was summed up and corresponded to the possibilities of OHSS.

Validation and Evaluation of the Prediction Model

Internal validations of the prediction models were performed using the validation cohort and were assessed by both discrimination and calibration. The discrimination ability of the models was demonstrated in areas under the curve (AUC, or C-statistic) of the receiver operating characteristic (ROC) curves. A higher C-statistic close to 1 refers to a perfect prediction. Generally, C-statistics above 0.7 are acceptable in clinical settings (7).

Calibration of the multistage prediction models was evaluated by calibration plot and Hosmer–Lemeshow good-of-fit test, which was plotted to compare the predicted probability of OHSS and the actual probability of OHSS.

Clinical decision curves were plotted to evaluate the clinical utility of the multiphase models. The clinical benefit was determined by the net benefit. The value zero represents no benefit, whereas higher values represent more benefit.

Construction of the Smartphone-Based App

The app was constructed as part of the built-in app of the IVF-aid platform, which was developed by our Reproductive Medical Center (software copyright registration number, 7857265). The self-monitoring part of the app, which was named as “OHSS monitor,” has three modules, risk prediction, symptoms monitoring, and treatment monitoring. 1) The “risk prediction module” was used to calculate the individual risk of developing OHSS based on the multiphase of the prediction models. The first three phases of risk prediction can only be accessed by clinicians for assisting clinical decision, and the final phase of risk prediction results will be open to the at-risk patients for OHSS risk alert. All variables of the prediction model could be extracted from the IVF-aid platform with additional data input, and an automatic calculation of OHSS risk would be provided to the high-risk individual. 2) The “symptom monitoring module” includes most frequent discomforts related to OHSS, and the severity of symptoms was further divided into four ranks revised from the Likert scale (8). 3) The “treatment monitoring module” records an individual’s visit to the hospital and their personal medical chart for the details of treatment.

Results

Basic Clinical Characteristics of Patients in the Training and Validation Cohorts

In the present study, 21,566 cases were recorded and scrutinized as demonstrated in Figure 1. After the exclusion of women with missing data (see Figure 1 for details), 17,445 cases with sufficient information were recruited in the final analysis. Among them, 12,211 cases (70.0%) were randomly picked as the training cohort, and the remaining 5,234 cases (30.0%) were selected as the validation cohort. There were 293 patients (2.40%) determined to have moderate/severe OHSS in the training cohort, and 132 (2.52%) in the validation cohort were diagnosed with moderate/severe OHSS. Those women with moderate/severe OHSS were termed as “moderate/severe OHSS group,” and the remaining cases were termed as “control group.”

As demonstrated in Tables 1, 2, the basic clinical characteristics and COS cycle-related characteristics showed no differences between the training cohort and validation cohort. As expected, women in the moderate/severe OHSS group showed decreased age, BMI, body surface area, increased ovarian reservation markers (AMH, AFC, PCOS), and more primary infertility (Table 1). The majority of women with moderate/severe OHSS (88.74%% in the training cohort and 88.64% in the validation cohort) were in their first COS cycle. COS protocols (either GnRH agonist protocol or GnRH antagonist protocol) were comparable between the two groups. However, women with moderate/severe OHSS had decreased doses of Gn but increased number of follicles on trigger day no matter what size of follicles. Besides, women in the moderate/severe OHSS group presented with elevated levels of estradiol on trigger day and had more oocytes retrieved. More of them had HCG alone for trigger, and less of them had all embryos frozen, hence more women with fresh embryo transfer (see Table 2 for details).

TABLE 1

Table 1 Baseline clinical characteristics of the training and validation cohorts.

TABLE 2

Table 2 Controlled ovarian stimulation cycle related characteristics of the training and validation cohorts.

Development of multiphase prediction models

Variables included in different phases are presented in Table 3. Multiphase nomograms corresponding to the abovementioned phases were also constructed and are presented in Supplementary Figure 1 (Supplementary Figure 1A for the first phase, 1B for the second phase, 1C for the third phase, and 1D for the fourth phase prediction).

TABLE 3

Table 3 Multivariable analysis of multiphase variables in predicting OHSS.

The equations of the multiphase prediction models are:

The 1st phase: P (OHSS) = 1 / (1 + exp(- (- 1.7165 - 0.0287 * Female age - 0.456 * Number of COS cycles - 0.4604 * Previous live birth (1) + 0.0484 *AMH - 0.035 * BMI))).

The 2nd phase: P (OHSS) = 1 / (1 + exp(- (- 1.7584 + 0.0276 * Female age - 0.3559 * Number of COS cycles - 0.3784 * Previous live birth (1) + 0.1989 * PCOS (yes) - 0.8059 * GnRH antagonist protocol - 0.1586 * Duration of Gn- 0.0126 * Initial dose of Gn + 0.0664 * Number of follicles (≥ 16 mm) on trigger day + 1.2158 * E2 levels on trigger day (3670 ≤ E2 < 18350 pmol/ml) + 0.8908 * E2 levels on trigger day (E2 ≥ 18350 pmol/ml) - 0.092 * P levels on trigger day + 0.0144 * BMI + 0.0028 * AMH))).

The 3rd phase: P (OHSS) = 1 / (1 + exp(- (- 4.7715 - 0.69 * Number of COS cycles - 0.2899 * Previous live birth (1) + 1.057 * Body surface area - 0.0173 * AFC + 0.4284 * PCOS (yes) -0.27 * GnRH antagonist protocol - 1.6428 * Freeze-all (yes) - 0.1359 * Duration of Gn - 0.0061 * Mean Dose of Gn - 0.0067 * Initial dose of Gn - 2.1356 * Trigger type (GnRHa alone) + 1.9909 * Trigger type (HCG alone) + 0.026 * Number of follicles (≥ 10 mm) on trigger day + 0.0947 * Number of follicles (≥ 18 mm) on trigger day + 0.9162 * E2 levels on trigger day (3670 ≤ E2 < 18350 pmol/ml) + 1.391 * E2 levels on trigger day (E2 ≥ 18350 pmol/ml) + 0.0328 * Number of oocytes retrieved))).

The 4th phase: P (OHSS) = 1 / (1 + exp(- (- 5.5972 - 0.5855 * Number of COS cycles -0.3028 * Previous live birth (1) + 0.9797 * Body surface area - 0.0173 * AFC + 0.4322 * PCOS (yes) - 0.7264 * Freeze-all (yes) - 0.1045 * Duration of Gn - 0.0069 * Mean dose of Gn - 0.0065 * Initial dose of Gn - 2.07 * Trigger type (GnRHa alone) + 1.9291 * Trigger type (HCG alone) + 0.0892 * Number of follicles ( 16 mm) on trigger day + 0.6846 * E2 levels on trigger day (3670 ≤ E2 < 18350 pmol/ml) + 1.5483 * E2 levels on trigger day (E2 ≥ 18350 pmol/ml) + 0.0562 * Number of oocytes retrieved + 0.2702 * Number of cycles with ET + 0.5219 * Number of embryo(s) for ET + 0.1745 * Number of top-quality embryo(s) for ET))).

Performance of the multiphase prediction models

ROC curves of both the training cohort and validation cohort were established and are demonstrated in Figure 2. The AUC or C-statistic in the training cohort in the four phases were 0.628 (95% CI 0.598–0.658, Figure 2A), 0.715 (95% CI 0.688–0.742, Figure 2C), 0.792 (95% CI 0.770–0.815, Figure 2E), and 0.814 (95% CI 0.793–0.834, Figure 2G), respectively. The C-statistics in the validation cohort in the four phases were 0.681 (95% CI 0.638–0.723, Figure 2B), 0.700 (95% CI 0.662–0.737, Figure 2D), 0.773 (95% CI 0.737–0.808, Figure 2F), and 0.796 (95% CI 0.761–0.830, Figure 2H), respectively.

FIGURE 2

Figure 2 ROC curves of the multiphase prediction models in the training cohort (left panel) and validation cohort (right panel). The first phase (training cohort, A; validation cohort, B), the 2nd phase (training cohort, C; validation cohort, D), the 3rd phase (training cohort, E; validation cohort, F), and the fourth phase (training cohort, G; validation cohort, H). ROC, receiver-operator characteristic.

The calibration plot (Figure 3A for the training cohort and 3B for the validation cohort) showed high calibration for the third- and fourth-phase prediction models in both the training and validation cohorts. The third- and fourth-phase prediction models also showed the highest net benefit (Figure 3C for the training cohort and 3D for the validation cohort).

FIGURE 3

Figure 3 Calibration plots and clinical decision curves of the training cohort and validation cohort. Calibration plots of the multiphase prediction models in the training cohort (A) and validation cohort (B). Clinical decision curves of the multiphase models in the training cohort (C) and validation cohort (D). Fit 1 represents the first-phase prediction model, fit 2 represents the second-phase prediction model, fit 3 represents the third-phase prediction model, and fit 4 represents the fourth-phase prediction model.

The OHSS discrimination abilities of the multiphase models in normal responders and hyper-responders were also assessed and presented in Supplemental data (Supplementary Figure 2 for normal responders, and Supplementary Figure 3 for hyper-responders). As expected, the prediction models were reliable in predicting the OHSS in either normal responders or hyper-responders.

Construction of the smartphone-based app

The smartphone-based app named “OHSS monitoring” has been constructed and built in the routinely used hospital database system which was designed for women during the ART treatment procedures, with full access to clinical physicians and partial access to ART patients. The app can provide a predictive risk of OHSS at four different phases of the COS cycle and help to support clinical decision-making (the first to third phases can only be accessed with clinical physicians as a “risk calculator”) and self-monitoring (the fourth phase, accessed with both clinical physicians and OHSS at-risk patients). A brief demo of the app for patients’ self-monitoring is shown in Figure 4.

FIGURE 4

Figure 4 Screenshot of the “OHSS monitor” app from a patient’s view.

Discussion

Here, based on a retrospective cohort of more than 21,000 women from a high-volume reproductive center, we generated quantified and complex multiphase prediction models of OHSS with a smartphone-based user-friendly app for clinicians’ risk calculation and patients’ self-monitoring. These multiphase prediction models can accurately distinguish women who may develop moderate/severe OHSS from those who would not. The predicted and observed risks of OHSS were concordant in the multiphase prediction models. Besides, the multiphase prediction models would be favorable in clinical utility with high net benefit. The multiphase prediction models can be used for the unselected population in the reproductive center with satisfactory discrimination and calibration. The multiphase prediction and self-monitoring app can be readily used in clinical settings.

Several prediction models of OHSS have been reported (9). One particular predictive model recruited PCOS women exclusively (10) and provided a prediction of OHSS after oocyte retrieval, whereas an early prediction prior to oocyte retrieval might be more necessary since physicians can rearrange the ovarian stimulation protocols and ovulation trigger options accordingly. Moreover, the reported prediction model included PCOS patients only with fresh embryo transfer, which might neglect the occurrence of OHSS in the population with all embryos cryopreserved. Other models reported either limited to a couple of selected parameters for prediction (11, 12), or restricted with a certain population (for instance, PCOS population only (10, 13), non-PCOS patients with long GnRH agonist protocol (14), patients with GnRH agonist protocol and HCG for triggering (9), coasted patients in the late follicular phase with long GnRH agonist protocol (15). Besides, the majority of the available predictions were established based on a relatively small sample size. Unlike those previously reported prediction models, multiphase prediction models in the present study focused more on the unselected general population and covered the whole spectrum of the COS process from the initiation of COS to fresh embryo transfer.

The prediction of OHSS has not been considered as an easy task. One main reason is that tons of influencing factors are contributing to the cascade of events leading to OHSS. Several ovarian reservation markers, including age, AFC, and AMH, have been thoroughly invested for their value in predicting OHSS (4, 16). Generally, it is believed that young and slim women with increased ovarian reserve were high-risk populations to develop OHSS (2). It is interesting to detect an association of low dose or duration of gonadotropin usage and increased risk of OHSS in the multiphase prediction models. Although no exact mechanisms were confirmed, the higher risk of OHSS might be due to the improved sensitivity to gonadotropin. To be particular, those hyper-responders of ovarian stimulation would require a low dose of gonadotropin and still possess a higher risk of developing OHSS (17). Elevated E2 levels were well-documented as an etiological factor of OHSS (18). Sustained supraphysiological levels of E2 might be associated with elevated number of oocytes retrieved as well as number of usable embryos (19). It is reasonable that increased E2 levels also contributed to the development of OHSS as indicated in the multiphase prediction models. Results from the multiphase prediction models also showed a similar trend of such featured women with a higher risk of developing OHSS.

It is not surprising that GnRHa for trigger and freeze-all strategy are major indicators involved in the prediction models (20). A single dose of GnRHa for trigger has also been shown to be a highly effective way for OHSS prevention in GnRH antagonist protocol (2). Plenty of evidence has confirmed that GnRHa trigger can notably reduce and even eliminate the occurrence of OHSS (3, 21, 22) Elective cryopreservation of all embryos or freeze-all strategy has been well known for preventing OHSS (2, 22, 23). The secretion of endogenous HCG along with pregnancy contributes to the exacerbation of OHSS symptoms and leads to the late onset of OHSS. Therefore, with a freeze-all strategy, the late onset of OHSS can largely be avoided (2). Even though few cases of OHSS occurred following freeze-all, a reduced severity of OHSS symptoms was observed (24). Undoubtedly, an increased number of follicles on trigger day and an elevated number of oocytes collected would promote the development of OHSS and can be useful in OHSS prediction (25). Fresh embryo transfer was one important trigger for late-onset OHSS. As expected, patients with embryo transfer, especially top-quality embryos, would have a higher risk of OHSS as demonstrated by the multiphase prediction models.

As an iatrogenic complication, the occurrence of OHSS is comprehensive result of patients’ characteristics and physicians’ interventions. Therefore, the first-phase prediction model, which was largely based on patients’ baseline clinical features, showed the lowest ability to discriminate OHSS from the non-OHSS population, whereas the prediction models of the other three phases all demonstrated to have reasonable performances. Besides, the C-statistics increased along with the process of the phases, as the acceptable c-statistic should be at least more than 0.7 (7). This low accuracy of very early phase prediction is probably due to multiple preventive or promotive interventions provided during the following COS process, which might cause a deviation in the prediction of occurrence of OHSS. For instance, it is commonly believed that women with significantly higher AMH levels and low BMI were more likely to develop OHSS. However, for those natively “high-risk” women, several prevention approaches might have already been provided by experienced physicians. In this case, those “high-risk” women might eventually be “low-risk” women for developing OHSS. Given the complexity of assisted reproductive technologies and the human reproduction process, the C-statistics above 0.6 are acceptable in clinical practice (26), and the accuracies of prediction models could improve with more variables introduced into the formulas.

Apps are an ideal tool to transfer the complex nomograms to a simple risk assessment result and can be easily applied in clinical practice. The majority of medical apps available are designed for healthcare information provision. Predictive-modeling-based medical apps only represented 16% of clinical decision support modality (27). Here, we developed a predictive-model-based app from rigorous data analysis of a large database. One major advantage of this predictive-modeling-based app is the individualized risk assessment provided for specific women at specific time points of consultation. Thus, an individualized clinical decision could be made accordingly. Especially, since assisted reproductive care is only available in almost urban areas only, women in low-income settings who do not have easy access to healthcare can particularly benefit from this smartphone-based app to have an early alert of OHSS risk. By automatically collecting values of parameters from the clinical database, the app would indicate an individualized risk score. Based on the predictive risk, physicians can choose an optimal strategy to lower the OHSS risk in the following process. A strict follow-up and careful self-monitoring would be suggested for those at risk of developing OHSS. Taking advantage of the self-monitoring module of the app, patients would be more aware of OHSS-related symptoms, for example, nausea, abdominal distention, and abdominal pain, and make revisit schedules once improved risks of OHSS are alerted by the app. With the help of the self-monitoring module, delayed diagnosis and management of OHSS can be largely reduced and even eliminated. This clinical decision-support and self-monitoring app has tremendous potential to improve the management of OHSS-risky women and enhance the safety of ART, although a prospective study is needed in the future to confirm the performance of this app tool.

Clinical Implications

The proposal of multiphase prediction models of OHSS is particularly important in practical terms. Stratification based on the risk ratio and workflow of women with different risk potentials are summarized in Figure 5. It is of great clinical value to predict OHSS before the initiation of COS based on baseline clinical features; hence, clinicians can provide treatment options to prevent OHSS. During the COS and even after oocyte retrieval, with the availability of more parameters, the assessment of OHSS becomes more accurate, which allows clinicians to provide preventive approaches beforehand. For clinicians, with the indication of individualized OHSS at-risk women, several preventive clinical interventions, including COS protocol chosen, GnRHa triggering, and freeze-all strategy, can be applied appropriately. For at-risk patients, the implementation of this user-friendly app could enhance patients’ access to healthcare of OHSS and improve the quality of care, especially for patients in remote areas. An accurate prediction of OHSS could minimize unnecessary interventions, for instance, repeated revisit, and provide remarkable benefits for both clinicians and patients.

FIGURE 5

Figure 5 Workflow of management of an OHSS patient. OHSS, ovarian hyper-stimulation syndrome; COS, controlled ovarian stimulation.

Strengths and Limitations

This study has evaluated more than 21,000 women and produced big data-based multiphase prediction models for OHSS in the general population with satisfactory accuracies. Right now, multifunction app development in the healthcare field is still in its infancy. Here we present one of the few apps reported in ART with predictive-model-based clinical decision-supporting and self-management function. Comprehensive risk assessment, individualized decision-making, and self-monitoring for at-risk individuals could benefit from this predictive-model-based app. However, a few limitations of this multiphase model are worth mentioning. First of all, the present study did not include mild OHSS, since the retrospective design of this study cannot collect detailed information of women with mild OHSS. Secondly, early onset and late onset of OHSS were not specified in the present study, which should be reevaluated in our prospective study. Moreover, due to the retrospective nature of the study design, a couple of parameters, for example, the doses of HCG for triggering, coasting, and other measures attempting to eliminate OHSS symptoms, were not included in the present models. To be a useful tool in daily practice, the predictive ability of the multiphase model and prediction-model-based app should be further assessed in prospective clinical trials to evaluate their cost-effectiveness, safety, and benefit in real-world settings.

Conclusion

This study developed multiphase prediction models of OHSS based on retrospective big data, to help in the evaluation and decision-making of physicians, which can reliably distinguish patients with a high risk of OHSS. By empowering high-risk patients with the smart-phone based self-monitoring app, early detection and management of OHSS can be easily implemented. The management strategies of patients with a high risk of OHSS would be improved with the help of multiphase prediction models and a predictive-model-based app. A combination of the OHSS prediction model and app could serve as efficient support for personalized management of OHSS. Ongoing prospective validations of the predictive model and the app are performed to further evaluate their clinical utility.

Data Availability Statement

The data underlying this article will be shared on reasonable request to the corresponding authors.

Ethics Statement

The studies involving human participants were reviewed and approved by the Ethical Committee of the Third Affiliated Hospital of Guangzhou Medical University (approval number, 2021-117). The patients/participants provided their written informed consent to participate in this study.

Author Contributions

MC, ZL, and YLin contributed to the study design, data acquisition, data analysis, and manuscript draft writing. YLuo, SL, and QH contributed to the data acquisition and data interpretation. HL and JL contributed to the study design, data interpretation, and draft revision. All authors contributed to the article and approved the submitted version.

Funding

This study was supported by grants from the Science and Technology Program of Guangzhou, China (No. 202102010076, for data acquisition), and The Medical Key Discipline of Guangzhou (2021-2023, for data analysis).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

The authors acknowledge Dr. Chongyang Duan and Dr. Zhiyuan Cao (Division of Statistics, Southern Medical University) for his help in development of the prediction model and app.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fendo.2022.911225/full#supplementary-material

Supplementary Figure 1 | Nomograms of the multi-phase prediction models to predict OHSS in the general population. Nomograms of the 1^st phase (2A), the 2^nd phase (2B), the 3^rd phase (2C), and the 4^th phase (2D). Abbreviations: OHSS, ovarian hyper-stimulation syndrome; COS, controlled ovarian stimulation; AMH, anti-Mullerian hormone; BMI, body mass index; PCOS, polycystic ovarian syndrome; Gn, gonadotropin; E2, estradiol; P progesterone; AFC, antral follicle counting; ET, embryo transfer.

Supplementary Figure 2 | ROC curves of the multi-phase prediction models in the training cohort (left panel) and validation cohort (right panel) in normal-responders. The 1^st phase (training cohort, Supplementary 1A; validation cohort, Supplementary 1B), the 2^nd phase (training cohort, Supplementary 1C; validation cohort, Supplementary 1D), the 3^rd phase (training cohort, Supplementary 1E; validation cohort, Supplementary 1F), and the 4^th phase (training cohort, Supplementary 1G; validation cohort, Supplementary 1H).

Supplementary Figure 3 | ROC curves of the multi-phase prediction models in the training cohort (left panel) and validation cohort (right panel) in hyper-responders. The 1^st phase (training cohort, Supplementary 2A; validation cohort, Supplementary 2B), the 2^nd phase (training cohort, Supplementary 2C; validation cohort, Supplementary 2D), the 3^rd phase (training cohort, Supplementary 2E; validation cohort, Supplementary 2F), and the 4^th phase (training cohort, Supplementary 2G; validation cohort, Supplementary 2H).

References

1. Liu F, Yang Y, Zhang S, Qiao J, Liu J. Consensus on the Diagnosis and Management of Complications in Assisted Reproductive Technologies. Reprod Contracep (2015) 35:431–9. doi: 10.7669/j.issn.0253-357X.2015.07.0431

CrossRef Full Text | Google Scholar

2. European IVF-monitoring Consortium (EIM European Society of Human Reproduction and Embryology (ESHRE), Calhaz-Jorge C, De Geyter C, Kupka MS, de Mouzon J, et al. Assisted Reproductive Technology in Europe, 2013: Results Generated From European Registers by ESHRE. Hum Reprod (Oxford England) (2017) 32:1957–73. doi: 10.1093/humrep/dex264

CrossRef Full Text | Google Scholar

3. Fiedler K, Ezcurra D. Predicting and Preventing Ovarian Hyperstimulation Syndrome (OHSS): The Need for Individualized Not Standardized Treatment. Reprod Biol Endocrinol RB&E. (2012) 10:32. doi: 10.1186/1477-7827-10-32

CrossRef Full Text | Google Scholar

4. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): The TRIPOD Statement. BMJ (Clinical Res ed) (2015) 350:g7594. doi: 10.1136/bmj.g7594

CrossRef Full Text | Google Scholar

5. Stevenson P. Height. Weight. Surface Formula for the Estimation of Surface Area in Chinese Subjects. Chin J Physiol (1937) 12:327–30.

Google Scholar

6. Swets JA. Measuring the Accuracy of Diagnostic Systems. Science (1988) 240:1285–93. doi: 10.1126/science.3287615

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Jebb AT, Ng V, Tay L. A Review of Key Likert Scale Development Advances: 1995-2019. Front Psychol (2021) 12:637547. doi: 10.3389/fpsyg.2021.637547

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Lainas GT, Lainas TG, Sfontouris IA, Venetis CA, Kyprianou MA, Petsas GK, et al. A Decision-Making Algorithm for Performing or Cancelling Embryo Transfer in Patients at High Risk for Ovarian Hyperstimulation Syndrome After Triggering Final Oocyte Maturation With hCG. Hum Reprod Open (2020) 2020):hoaa013. doi: 10.1093/hropen/hoaa013

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Li F, Chen Y, Niu A, He Y, Yan Y. Nomogram Model to Predict the Probability of Ovarian Hyperstimulation Syndrome in the Treatment of Patients With Polycystic Ovary Syndrome. Front Endocrinol (2021) 12:619059. doi: 10.3389/fendo.2021.619059

CrossRef Full Text | Google Scholar

10. Griesinger G, Verweij PJ, Gates D, Devroey P, Gordon K, Stegmann BJ, et al. Prediction of Ovarian Hyperstimulation Syndrome in Patients Treated With Corifollitropin Alfa or rFSH in a GnRH Antagonist Protocol. PLoS One (2016) 11:e0149615. doi: 10.1371/journal.pone.0149615

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Madrazo I, Vélez MF, Hidalgo JJ, Ortiz G, Suárez JJ, Porchia LM, et al. Prediction of Severe Ovarian Hyperstimulation Syndrome in Women Undergoing In Vitro Fertilization Using Estradiol Levels, Collected Ova, and Number of Follicles. J Int Med Res (2020) 48:300060520945551. doi: 10.1177/0300060520945551

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Cao Y, Shi H, Ma Y, Ma L, Zhai J. Effect and Relationship of Seasons on the High Risk of Ovarian Hyperstimulation Syndrome After Oocyte Retrieval in Patients With Polycystic Ovary Syndrome. Front Endocrinol (2020) 11:610828. doi: 10.3389/fendo.2020.610828

CrossRef Full Text | Google Scholar

13. Ashrafi M, Bahmanabadi A, Akhond MR, Arabipoor A. Predictive Factors of Early Moderate/Severe Ovarian Hyperstimulation Syndrome in non-Polycystic Ovarian Syndrome Patients: A Statistical Model. Arch Gynecol Qbstet (2015) 292:1145–52. doi: 10.1007/s00404-015-3723-0

CrossRef Full Text | Google Scholar

14. Aljawoan FY, Hunt LP, Gordon UD. Prediction of Ovarian Hyperstimulation Syndrome in Coasted Patients in an IVF/ICSI Program. J Hum Reprod Sci (2012) 5:32–6. doi: 10.4103/0974-1208.97795

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Salmassi A, Mettler L, Hedderich J, Jonat W, Deenadayal A, von Otte S, et al. Cut-Off Levels of Anti-Mullerian Hormone for The Prediction of Ovarian Response, In Vitro Fertilization Outcome and Ovarian Hyperstimulation Syndrome. Int J Fert Steril (2015) 9:157–67. doi: 10.22074/ijfs.2015.4236

CrossRef Full Text | Google Scholar

16. Bulletti C, Allegra A, Mignini Renzini M, Vaiarelli A. How Fixed Versus Variable Gonadotropin Dose During Controlled Ovarian Stimulation Could Influence the Management of Infertility Patients Undergoing IVF Treatment: A National Delphi Consensus. Gynecol Endocrinol Off J Int Soc Gynecol Endocrinol (2021) 37:255–63. doi: 10.1080/09513590.2020.1770214

CrossRef Full Text | Google Scholar

17. Aboulghar M. Prediction of Ovarian Hyperstimulation Syndrome (OHSS). Estradiol Level has an Important Role in the Prediction of OHSS. Hum Reprod (Oxford England) (2003) 18:1140–1. doi: 10.1093/humrep/deg208

CrossRef Full Text | Google Scholar

18. Mizrachi Y, Horowitz E, Farhi J, Raziel A, Weissman A. Ovarian Stimulation for Freeze-All IVF Cycles: A Systematic Review. Hum Reprod Update. (2020) 26:118–35. doi: 10.1093/humupd/dmz037

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Selman H, Brusco GF, Fiorini F, Barnocchi N, Mariani M, El-Danasouri I. Vitrification is a Highly Efficient Method to Cryopreserve Human Embryos in In Vitro Fertilization Patients at High Risk of Developing Ovarian Hyperstimulation Syndrome. Fert Steril (2009) 91:1611–3. doi: 10.1016/j.fertnstert.2008.12.027

CrossRef Full Text | Google Scholar

20. Iliodromiti S, Blockeel C, Tremellen KP, Fleming R, Tournaye H, Humaidan P, et al. Consistent High Clinical Pregnancy Rates and Low Ovarian Hyperstimulation Syndrome Rates in High-Risk Patients After GnRH Agonist Triggering and Modified Luteal Support: A Retrospective Multicentre Study. Hum Reprod (Oxford England). (2013) 28:2529–36. doi: 10.1093/humrep/det304

CrossRef Full Text | Google Scholar

21. The Eshre Guideline Group On Ovarian Stimulation, Bosch E, Broer S, Griesinger G, Grynberg M, Humaidan P, et al. ESHRE Guideline: Ovarian Stimulation for IVF/ICSI(†). Hum Reprod Open (2020) 2020):hoaa009. doi: 10.1093/hropen/hoaa009

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Roque M, Haahr T, Geber S, Esteves SC, Humaidan P. Fresh Versus Elective Frozen Embryo Transfer in IVF/ICSI Cycles: A Systematic Review and Meta-Analysis of Reproductive Outcomes. Hum Reprod Update. (2019) 25:2–14. doi: 10.1093/humupd/dmy033

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Wada I, Matson PL, Troup SA, Morroll DR, Hunt L, Lieberman BA. Does Elective Cryopreservation of All Embryos From Women at Risk of Ovarian Hyperstimulation Syndrome Reduce the Incidence of the Condition? Br J Obstet Gynaecol (1993) 100:265–9. doi: 10.1111/j.1471-0528.1993.tb15241.x

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Lainas GT, Kolibianakis EM, Sfontouris IA, Zorzovilis IZ, Petsas GK, Tarlatzi TB, et al. Outpatient Management of Severe Early OHSS by Administration of GnRH Antagonist in the Luteal Phase: An Observational Cohort Study. Reprod Biol Endocrinol RB&E. (2012) 10:69. doi: 10.1186/1477-7827-10-69

CrossRef Full Text | Google Scholar

25. Dhillon RK, McLernon DJ, Smith PP, Fishel S, Dowell K, Deeks JJ, et al. Predicting the Chance of Live Birth for Women Undergoing IVF: A Novel Pretreatment Counselling Tool. Hum Reprod (Oxford England). (2016) 31:84–92. doi: 10.1093/humrep/dev268

CrossRef Full Text | Google Scholar

26. Watson HA, Tribe RM, Shennan AH. The Role of Medical Smartphone Apps in Clinical Decision-Support: A Literature Review. Artif Intell Med (2019) 100:101707. doi: 10.1016/j.artmed.2019.101707

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Kahnberg A, Enskog A, Brännström M, Lundin K, Bergh C. Prediction of Ovarian Hyperstimulation Syndrome in Women Undergoing In Vitro Fertilization. Acta Obstetricia Gynecol Scandinavica (2009) 88:1373–81. doi: 10.3109/00016340903287482

CrossRef Full Text | Google Scholar