Prediction models for cardiovascular disease risk among people living with HIV: A systematic review and meta-analysis

Abstract

Background:

HIV continues to be a major global health issue. The relative risk of cardiovascular disease (CVD) among people living with HIV (PLWH) was 2.16 compared to non-HIV-infections. The prediction of CVD is becoming an important issue in current HIV management. However, there is no consensus on optional CVD risk models for PLWH. Therefore, we aimed to systematically summarize and compare prediction models for CVD risk among PLWH.

Methods:

Longitudinal studies that developed or validated prediction models for CVD risk among PLWH were systematically searched. Five databases were searched up to January 2022. The quality of the included articles was evaluated by using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). We applied meta-analysis to pool the logit-transformed C-statistics for discrimination performance.

Results:

Thirteen articles describing 17 models were included. All the included studies had a high risk of bias. In the meta-analysis, the pooled estimated C-statistic was 0.76 (95% CI: 0.72–0.81, I² = 84.8%) for the Data collection on Adverse Effects of Anti-HIV Drugs Study risk equation (D:A:D) (2010), 0.75 (95% CI: 0.70–0.79, I² = 82.4%) for the D:A:D (2010) 10-year risk version, 0.77 (95% CI: 0.74–0.80, I² = 82.2%) for the full D:A:D (2016) model, 0.74 (95% CI: 0.68–0.79, I² = 86.2%) for the reduced D:A:D (2016) model, 0.71 (95% CI: 0.61–0.79, I² = 87.9%) for the Framingham Risk Score (FRS) for coronary heart disease (CHD) (1998), 0.74 (95% CI: 0.70–0.78, I² = 87.8%) for the FRS CVD model (2008), 0.72 (95% CI: 0.67–0.76, I² = 75.0%) for the pooled cohort equations of the American Heart Society/ American score (PCE), and 0.67 (95% CI: 0.56–0.77, I² = 51.3%) for the Systematic COronary Risk Evaluation (SCORE). In the subgroup analysis, the discrimination of PCE was significantly better in the group aged ≤40 years than in the group aged 40–45 years (P = 0.024) and the group aged ≥45 years (P = 0.010). No models were developed or validated in Sub-Saharan Africa and the Asia region.

Conclusions:

The full D:A:D (2016) model performed the best in terms of discrimination, followed by the D:A:D (2010) and PCE. However, there were no significant differences between any of the model pairings. Specific CVD risk models for older PLWH and for PLWH in Sub-Saharan Africa and the Asia region should be established.

Systematic Review Registration: PROSPERO CRD42022322024.

1. Introduction

HIV continues to be a major global health issue. At the end of 2020, there were approximately 37.7 million people living with HIV (PLWH), with 1.5 million people becoming newly infected with HIV (1). The widespread usage of highly active antiretroviral therapy (HAART) drastically reduced death rates and potential years of life lost (2). However, as life expectancy increases, non-AIDS-defining illnesses are becoming increasingly common causes of death, with cardiovascular disease (CVD) accounting for a sizable portion (3, 4). CVDs, a set of heart and blood vessel disorders, are the leading causes of mortality worldwide. An estimated 17.9 million individuals died from CVDs in 2019, accounting for 32% of all global fatalities (5). According to a meta-analysis that included 793,635 participants from 73 studies, the relative risk (RR) of CVD among PLWH was 2.16 [95% confidence interval (CI), 1.68–2.77] compared to non-HIV-infections (6). In addition, it was indicated that HIV infection is a risk-enhancing factor for CVD (7). Moreover, ART may have a negative association with an increased risk of CVD events, especially among PLWH with other risk factors for cardiovascular disease (8, 9). As a result, the prediction and treatment of CVD are becoming even more important issues in current HIV management (4).

Prediction models are useful tools for estimating the probability or risk of specific future occurrences based on the combination of multiple predictors (10, 11). They aid clinicians in making therapeutic decisions and determining subsequent steps in therapy (12). CVD risk prediction is crucial to treatment guidelines and CVD control (13). Using prediction models is beneficial to make patients aware of their condition and to encourage them to adopt a healthy lifestyle (12). A substantial number of prediction models for various cardiovascular outcomes have been established for the general population, such as the Framingham Risk Score (FRS) (14), the Systematic COronary Risk Evaluation (SCORE) (15), and the pooled cohort equations of the American Heart Society/American score (PCE) (16). The usefulness of most of the models remains unclear because of methodological shortcomings, incomplete presentation, and lack of external validation (17).

Considering other potential factors driving CVD risk among PLWH, HIV-specific CVD prediction models have been developed, such as the Data collection on Adverse Effects of Anti-HIV Drugs Study risk equation (D:A:D) (18). Several CVD prediction models developed for the general population have also been validated in PLWH. However, the findings of model performance conflicted. Delabays et al. indicated that general models were valid to predict CVD for PLWH. Adding HIV-specific factors to scores did not result in a clinically significant improvement (19). In contrast, Triant et al. recommended adding HIV-related factors because general algorithms consistently underestimate the risk of CVD for PLWH (20). Whether HIV-specific models perform better than general models remains unknown. There is no consensus on optional CVD risk models for PLWH.

Soares et al. conducted a systematic review to summarize the CVD prediction models used for PLWH (21). However, they only considering studies published before January 31, 2021. Additionally, Soares et al. did not perform a subgroup analysis to ensure that the CVD risk prediction model treated subgroups (such as geographic region, race, and age) fairly in PLWH. The fairness of prediction models is crucial for promoting health equity, as ignoring differences in social determinants of health can result in inaccurate risk stratification in vulnerable groups, further exacerbating existing inequities (22).

Therefore, we aimed to systematically summarize multivariable prediction models for CVD risk among PLWH. Specific objectives included, first and foremost, objectively appraising the risk of bias in papers. Second, we conducted a meta-analysis on discrimination to estimate and compare the models' performance quantitatively. This review was conducted in accordance with the guidelines of the Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) (23). The protocol was registered on PROSPERO (CRD42022322024). The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were used to guide the reporting of our review (24).

2. Methods

2.1. Search strategy

We conducted a three-step search to identify both published studies and gray literatures. First, keywords and search terms were captured from an initial limited search via PubMed/MEDLINE. This informed the development of the search strategy with the help of a librarian. Second, a comprehensive search was conducted using the following databases: PubMed/MEDLINE, MEDLINE (Ovid), Embase (Ovid), CINAHL (EBSCO), and Web of Science. Gray literatures were searched via Google Scholar and Baidu. In PubMed/MEDLINE, we searched for papers in English using MeSH terms ([“HIV” OR “Acquired Immunodeficiency Syndrome” OR “HIV infections”] AND [prognosis OR survival OR mortality OR risk]) combined with the title/abstract keywords ([“progn*” OR “predic*” OR “risk” OR “model*”] AND [“HIV” OR “AIDS” OR “Acquired Immunodeficiency Syndrome” OR “PLWH”] AND [“machine learning” OR “artificial intelligence” OR “algorithm”] AND [“cardiovascular disease” OR “CVD” OR “cardiovascular event*” OR“main cardiovascular adverse event”OR “MACE”]). The search was limited to studies from inception to January 2022. The full search strategies for each database are presented in Supplementary Appendix S1. Finally, references in all included studies were manually reviewed to supplement the database search.

2.2. Inclusion and exclusion criteria

The inclusion criteria were as follows: (1) studies that assessed risk models that predicted the short-term or long-term risk of cardiovascular disease among PLWH. Cardiovascular disease in our review included coronary heart disease (CHD), myocardial infarction (MI), stroke, and other heart and blood vessel disorders (5). (2) Studies that developed new models, conducted external validation of an existing model, and/or updated an existing model. (3) Studies published in English or Chinese. No restrictions were made on the setting (e.g., inpatients, outpatients), prediction horizon (how far ahead the model predicted), or predictors or outcomes. Cross-sectional studies were excluded. Studies that merely carried out risk factor analysis without modeling were further excluded.

2.3. Study screening and selection

All identified citations were imported into EndNote X8 (Clarivate Analytics, PA, United States) to remove duplicates. Two reviewers (JY & ZZ) independently carried out screening and selection. First, titles and abstracts were screened to ascertain potentially relevant studies. Next, we screened the full texts and identified studies that met the inclusion and exclusion criteria. Discrepancies were discussed between two reviewers until a consensus was reached after referring to the protocol.

2.4. Quality appraisal

We appraised the quality of the included articles by using the Prediction model Risk Of Bias ASsessment Tool (PROBAST), which is an assessment tool developed specifically for diagnostic and prognostic prediction model studies (25). Two reviewers (JY & ZZ) assessed the risk of bias and applicability respectively. For the assessment of risk of bias, four domains (participants, predictors, outcome, analysis) with a total of twenty signaling questions were judged. Each signaling question was answered as “yes”, “probably yes”, “probably no”, “no”, or “no information”. Based on the answers to the signaling questions, we used our own judgment to rate the domains as high (−), low (+), or unclear (?) risk of bias. We assessed applicability by using the same first three domains but without signaling questions. Finally, we rated the overall ROB and applicability as high (−), low (+), or unclear (?). Any disagreement was discussed between the two reviewers until a consensus was reached.

2.5. Data extraction

Two reviewers (JY & ZZ) independently extracted information from the included papers, including the authors, year, location, study design, study population, predicted outcomes, predictors, sample size, missing data, modeling method, method of validation, and predictive performance. We extracted data separately for each model when a study described multiple models. Any disagreement was discussed, and a consensus was reached.

2.6. Data synthesis

We performed descriptive analysis to summarize the characteristics of the models. If a model was validated in two or more studies, we applied a random effects meta-analysis model to evaluate discrimination performance (26). Since different studies reported different discrimination metrics, either by Harrell's C-statistic or by area under the curve (AUC), we further pooled them separately. Due to the variability in the validation population, differences in outcome and predictor definitions across studies, and the non-normal distribution of the C-statistic between studies, we transformed the C-statistic using a logit transformation (27). When confidence intervals were not available, we approximated them by using the standard normal distribution (28). We considered C-statistics in the range of 0.50–0.59 to indicate poor, 0.60–0.69 to indicate moderate, 0.70–0.79 to indicate acceptable, 0.80–0.89 to indicate very good, and 0.90 or greater to indicate excellent discrimination (29). The I² statistics were used to quantify the heterogeneity of the studies. In addition, subgroup analyses were performed in terms of sample size, age, and follow-up. The validation cohort was defined as “Mainly White” if more than 50% of the population was white. Since all the studies encompass both white and black individuals, the term “Mainly” was used instead of specifying either “White” or “Black”. Analyses were performed in Stata 17.0 (College Station, TX).

3. Results

3.1. Inclusion of studies

A total of 6,192 records were obtained from the databases. After removal of duplicates, the titles and abstracts of 3,901 articles were screened. The full texts of 76 articles were reviewed, and 63 articles were excluded because they did not meet the predefined inclusion criteria. Finally, 13 articles were included in our systematic review (Figure 1) (19, 20, 30–40).

Figure 1

3.2. Characteristics of eligible studies

Table 1 shows the characteristics of the included studies. Overall, 17 different models were derived from 13 studies (19, 20, 30–40). There were six HIV-specific models, including the D:A:D model (2010), the D:A:D (2010) 10-year version model, the updated full D:A:D (2016) model, the updated reduced D:A:D (2016) model, the HIV MI-1, the HIV MI-2 models. The other 11 models were developed for the general population and validated in PLWH, including the FRS (2004), FRS for CVD risk (2008), FRS for CHD risk (1998), FRS adjusted for Colombia, FRS adaptation for the Spanish population (REGICOR), FRS developed specifically for the Italian population by Progetto CUORE, the Prospective Cardiovascular Munster Study (PROCAM), SCORE, SCORE 2, SCORE adjusted for national data (SCORE-NL), and PCE.

In total, four studies (20, 32–34) described the development process of prediction models, and 13 studies (19, 20, 30–40) conducted model external validation. Four studies were carried out in the United States (20, 30, 32, 39), followed by Italy (n = 2) (31, 37), Switzerland (n = 1) (19), Colombia (n = 1) (35), Spain (n = 1) (36), Germany (n = 1) (38), and Netherlands (n = 1) (40), with two being multicenter studies from several countries (33, 34). The sample size ranged from 369 (31) to 32,663 (34). The number of prediction events varied from 8 (35) to 1,393 (40). Twelve studies were mostly conducted on males, while one study (20) included only male participants.

Among four development studies, three (20, 33, 34) used backward selection to remove nonsignificant predictors, whereas the other one (32) did not report a specific strategy for variable selection. Strategies dealing with missing data varied, including excluding participants with missing data (34) and using single imputation (32); one study did not describe a strategy (33).

Among 13 validation studies, nine studies reported both the calibration and discrimination performance of the models (19, 20, 30–34, 39, 40). Two studies reported only discrimination (35, 38), one study reported only calibration (37), and one study did not report model performance (36). Discrimination was reported as Harrell's C-statistic in six studies (20, 30, 34, 39, 40) and AUC in five studies (19, 31, 33, 35, 38). A variety of methods were used to report the calibration performance of the models, including the observed-expected ratio (n = 5) (30, 31, 33, 39, 40), Hosmer–Lemeshow test (n = 3) (19, 34, 39), calibration plot (n = 3) (30, 32, 40), Greenwood-Nam-D'Agostino (GND) test (n = 3) (20, 32, 40), and Brier score (n = 1) (19).

3.3. Risk of bias

Figure 2 shows a summary of the risk of bias and applicability for all included studies. All of the included studies had a high risk of bias. Ten studies (76.9%) (19, 20, 30–32, 36, 37, 40) were considered to have a low risk of bias in the predictor domain, and all the included studies (19, 20, 30–40) were considered to have a low risk of bias in the outcome domain. The risk of bias related to the participant domain was high in seven studies (53.8%) (30, 33–35, 38–40), mainly because they excluded participants with missing data. Twelve studies were judged to have high risk of bias in the analysis domain (19, 20, 30–33, 35–40). Nine studies ignored or dealt with model overfitting in an inappropriate manner (18, 30–32, 35–39). Seven studies had lower-than-reasonable numbers of participants with the events (20, 30, 31, 35–38). Ten studies did not report the methodologies for continuous and categorical predictor transformation (19, 20, 31, 32, 35–40).

Figure 2

3.4. HIV-specific models

Table 2 shows the characteristics of six HIV-specific models (32–34, 39, 40). In total, four models were developed in multicenter studies (33, 34, 39, 40), while the remaining two models were developed in the US (32). Four models could be used in all age groups (33, 34, 39, 40). Three models were developed to predict the 5-year risk of CVD, including the D:A:D (2010), updated full D:A:D (2016) and reduced models, whereas the other three models were for 10-year risk prediction. For the modeling methods, three models utilized Cox proportional hazards modeling (32, 34), two used Poisson regression modeling (33, 39, 40), and one used lasso and ridge regression (32).

In total, 24 predictors were considered. Age, sex, family history of CVD, systolic blood pressure, Smoking, total cholesterol, high-density liptein cholesterol, and diabetes mellitus (n = 4) (33, 34, 39, 40) were the most common predictors, followed by current abacavir use (n = 3) (34, 39, 40). HIV-specific risk factors were included in four models: three models included current abacavir usage (33, 39, 40), two models included CD4 count (34) and exposure to lopinavir/ritonavir and indinavir (33, 39, 40), and one model included cumulative protease inhibitor (PI) exposure (34) and cumulative nucleoside reverse transcriptase inhibitor (NRTI) exposure (34).

The definitions of CVD outcomes revealed considerable heterogeneity. All six models included myocardial infarction (MI) (32–34, 39, 40), followed by coronary heart disease (CHD) death, carotid artery endarterectomy, invasive coronary artery procedure, and stroke (n = 4) (33, 34, 39, 40).

3.5. Meta-analysis of prediction models

Figure 3 shows the results of the meta-analysis of the eight models. We conducted meta-analysis only on model discrimination. Calibrations were not synthesized due to inconsistent and inadequate data. In total, eight models were validated more than once and included in the meta-analysis. Two validation studies were removed from the meta-analysis because they did not report C-statistics or AUCs (36, 37). The FRS CVD model (2008) was the most widely validated model (n = 7) (20, 31, 33–35, 39, 40), followed by the PCE model (n = 6) (20, 32, 35, 38–40), the D:A:D model (2010) (n = 4) (19, 33, 35, 39), and the SCORE (n = 4) (31, 35, 38, 39). In the meta-analysis, the pooled estimated C-statistic/AUC was 0.76 (95% CI: 0.72–0.81, I² = 84.8%) for the D:A:D (2010), 0.75 (95% CI: 0.70–0.79, I² = 82.4%) for the D:A:D (2010) 10-year risk version, 0.77 (95% CI: 0.74–0.80, I² = 82.2%) for the full D:A:D (2016) model, 0.74 (95% CI: 0.68–0.79, I² = 86.2%) for the reduced D:A:D (2016) model, 0.71 (95% CI: 0.61–0.79, I² = 87.9%) for the FRS CHD model (1998), 0.74 (95% CI: 0.70–0.78, I² = 87.8%) for the FRS CVD model (2008), 0.72 (95% CI: 0.67–0.76, I² = 75.0%) for the PCE model, and 0.67 (95% CI: 0.56–0.77, I² = 51.3%) for the SCORE. However, there were no significant differences between any of the model pairings. Table 3 shows the results of the meta-analysis of C-statistics and AUC respectively. Overall, all of the models showed an acceptable discrimination.

Figure 3

3.6. Subgroup analysis

To explore potential inequities in prediction models and sources of heterogeneity, we conducted subgroup analyses for sample size, age, and follow-up period (Table 4). The discrimination of PCE was significantly better in the group aged ≤40 years than in the groups aged 40–45 years (P = 0.024) and ≥45 years (P = 0.010). The discrimination of the full D:A:D (2016) model was significantly better in the group with sample size ≥10,000 than in the group with sample size 1,000–10,000 (P = 0.0076).

We conducted subgroup analysis to compare different models in North America (20, 30, 39) and Mainly White (20, 33, 34, 39, 40) population separately because all models were validated in these two groups. No significant differences were found (P > 0.05). In both groups, only the SCORE performed poorer than the other models in terms of discrimination, with a C-statistic of 0.59 (0.45, 0.73). No models were developed or validated in Sub-Saharan Africa and the Asia region.

4. Discussion

This systematic review identified 17 prediction models for predicting CVD risk among PLWH. Among them, six were HIV-specific models. All included studies were rated as having a high risk of bias against the PROBAST checklist. Only eight models were externally validated at least once. These were included in the meta-analysis, and all of them showed acceptable discrimination. The FRS was the most widely validated model. Most prediction models had acceptable discrimination. The full D:A:D (2016) model performed the best in terms of discrimination, followed by D:A:D (2010) and PCE. However, there were no significant differences between any of the model pairings. We also found that differences in model discrimination existed when they were stratified by geographic region, sample size, and age.

The full D:A:D (2016) model had higher discrimination for PLWH. However, it remained at a high risk of bias as they only included participants with complete information on all predictors. It is inappropriate to exclude participants with missing data directly. Missing data could be divided into missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR) (41). MCAR and MAR are less problematic, but they seldomly occur (24, 41). Consequently, complete case analysis could lead to selection bias, and the characteristics of participants may differ from those with missing data. A nonrandom sample subset could frequently generate considerable bias that cannot be overcome in estimating model parameters and yielding predictive performance (24, 41, 42). Currently, common strategies for handling missing data include zero imputation, mean imputation, and multiple imputation (43). Multiple imputation is often regarded as a preferred method for avoiding selection bias and statistical power loss (24, 44). Furthermore, it is advisable to consider combining unsupervised and supervised learning methods for imputation (42). Strategies for handling missing data should be chosen with caution depending on the intended application of the prediction model (41).

The D:A:D cohort was the largest HIV cohort in our study. A large sample size was one of the key points in developing robust models that would be more reliable for application in new datasets (45). In contrast, nearly half of the included studies (n = 7) (20, 30, 31, 35–38) had limited sample sizes and numbers of outcome events. For development studies, the sample size should ensure at least 10 events per candidate predictor parameter. For studies validating prediction models, a minimum of 100 events and 100 nonevents are suggested (24). A small sample size may result in model overfitting and inaccurate predictions (45, 46).

Current models applied for PLWH lack HIV-specific predictors. This might be attributed to the fact that the majority of prediction models are developed for the general population. According to our review, frequently used traditional CVD risk predictors included age, smoking status, systolic blood pressure, total cholesterol, and diabetes mellitus. Common HIV-specific predictors included abacavir usage, CD4 count, and exposure to lopinavir/ritonavir and indinavir. Previous studies presented multiple factors that drive the risk of CVD among PLWH (47). Decreased CD4 T-cell counts and increased HIV RNA have been demonstrated to be linked to increased CVD rates (47, 48). A persistently lower or inverted CD4/CD8 ratio has emerged as an independent predictor of CVD risk (48). Analysis from observational studies showed that most PIs, such as ritonavir-boosted darunavir/ritonavir, are associated with progressively increased CVD risk, and the effect is cumulative (49, 50). The results of D:A:D cohort indicated that the incidence rate ratio of using ritonavir-boosted darunavir was 1.59 (95% CI: 1.33–1.91, per 5 years of additional use) compared to non-use (50). Our review indicated that the full D:A:D model (2016) performed better than the reduced model (C-statistics: 0.775 vs. 0.739). This might be due to the reduced model leaving out ART covariates, such as PI and NRTI exposure. We recommend including CD4 counts and the CD4/CD8 ratio, as well as ART covariates (PI and NRTI exposure) as potential predictors in future studies that aim to develop HIV-specific models. In addition, it is critical to understand the role of HIV-specific inflammation and immune activation in conferring CVD risk (48, 51–53). Few models contained immunological-related factors, such as soluble markers of interleukin-1 (IL-1) (53). The feasibility and effectiveness of adding immunological-related factors into models still remain to be assessed. Further study is needed to add HIV-specific predictors to models.

Among published models, most used univariate or multivariate regression analysis and no validation process. The performance of the prediction model would be overestimated if there was no internal or external validation process (54). Internal validation is beneficial to provide a more accurate assessment of model performance in new subjects (54). External validation is necessary to determine a model's reproducibility and generalizability to populations with different characteristics (55). Models that have not been validated should not be recommended for clinical use. We suggest that future studies validate the multiple existing HIV-specific models and compare their performance head to head.

Calibration, one of the important aspects of validating model performance, has remained underreported in many published studies. Calibration refers to the accuracy of absolute risk estimates by comparing how similar the predicted risk to the true (observed) risk in different risk strata (56). It could be assessed in various ways, including the Hosmer–Lemeshow test (H–L test), calibration-in-the-large, Brier score, Cox intercept and slope, and integrated calibration index (57, 58). Although the H–L test was one of the most common proxies for calibration measures, we should consider its limitations, including its vulnerability when increasing the sample size and the arbitrary number of groups (55, 57). We recommend that future studies combine multiple calibration measures to assess the model calibration more comprehensively and to promote comparability of model calibration across studies.

Our subgroup analysis revealed differences in the model's discrimination when stratified by age. We found that the model's discrimination degraded with age. Age is an important independent risk factor for CVD (59). Aging is always accompanied by changes in the heart and vascular system. In addition, older individuals are more susceptible to drug interactions and side effects (60). Frailty, a common symptom in older people, may also be associated with an increased risk of CVD and CVD mortality (61, 62). These characteristics of older adults may lead to changes in the relationship between risk factors and CVD outcomes. Specific CVD risk models for older PLWH should be developed and validated in the future. In addition, the majority of models were developed and validated in America and Europe, whereas there was a lack of models from Asia and Africa. As global statistics revealed, Sub-Saharan Africa and the Asia have the highest burden of HIV-associated CVD (63). Models should be adapted to these regions and validated further in the future due to differences in race/ethnicity, healthcare systems, and lifestyle.

5. Limitations

There are several limitations of our study. First, we included only studies that were in English or Chinese. Articles in other languages should be summarized in future reviews. Second, most data for model development and validation came from developed countries. Caution should be used when applying our findings to individuals from different regions. Third, we conducted a meta-analysis only on discrimination rather than calibration. This is because of the inadequate reporting of calibration in the validation studies.

6. Implications for practice

The full D:A:D (2016) model had higher discrimination for PLWH. In addition, the findings indicated that model performance was associated with age. In clinical practice, health professionals should revalidate models if the population, region, or age to which they are applied is different from the original study. Moreover, only eight models were validated more than once. Models that have not been thoroughly validated are not useful for clinical practice. Health professionals should focus on developing, updating, and validating HIV-specific models and report them in accordance with the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guidelines. Specific CVD risk models for older PLWH, as well as models for Sub-Saharan Africa and the Asia region should be established that CVD risk and prevalence vary in different regions due to culture, local economy and health care policy.

7. Conclusion

Our systematic review summarized the prediction models for CVD in PLWH and conducted a meta-analysis on model discrimination performance. All of the models showed acceptable discrimination. The full D:A:D (2016) model performed the best in terms of discrimination, followed by D:A:D (2010) and PCE. However, all these models were assessed as having a high risk of bias. Future studies should adhere to the TRIPOD guidelines to ensure the quality and applicability of the models. Researchers should focus on developing and validating CVD models for PLWH. Specific CVD risk models for older PLWH, as well as models for Sub-Saharan Africa and the Asia region should be established.

References

• 1.

  World Health Organization. Data on the size of the HIV/AIDS epidemic. Available at:https://www.who.int/data/gho/data/themes/topics/topic-details/GHO/data-on-the-size-of-the-hiv-aids-epidemic?lang=en(Accessed March 30, 2022).

• 2.

  Antiretroviral Therapy Cohort Collaboration. Life expectancy of individuals on combination antiretroviral therapy in high-income countries: a collaborative analysis of 14 cohort studies. Lancet. (2008) 372:293–99. 10.1016/S0140-6736(08)61113-7

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 3.

  FarahaniMMulinderHFarahaniAMarlinkR. Prevalence and distribution of non-AIDS causes of death among HIV-infected individuals receiving antiretroviral therapy: a systematic review and meta-analysis. Int J Std Aids. (2017) 28:636–50. 10.1177/0956462416632428

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 4.

  GhosnJTaiwoBSeedatSAutranBKatlamaC. HIV. Lancet. (2018) 392:685–97. 10.1016/S0140-6736(18)31311-4

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 5.

  World Health Organization. Cardiovascular diseases (CVDs). Available at:https://www.who.int/news-room/fact-sheets/detail/cardiovascular-diseases-(cvds) (Accessed March 30, 2022).

• 6.

  ShahASVStelzleDLeeKKBeckEJAlamSCliffordSet alGlobal burden of atherosclerotic cardiovascular disease in people living with HIV. Circulation. (2018) 138:1100–12. 10.1161/CIRCULATIONAHA.117.033369

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 7.

  GrundySMStoneNJBaileyALBeamCBirtcherKKBlumenthalRSet al2018 AHA/ACC/AACVPR/AAPA/ABC/ACPM/ADA/AGS/APhA/ASPC/NLA/PCNA guideline on the management of blood cholesterol: a report of the American college of cardiology/American heart association task force on clinical practice guidelines. Circulation. (2019) 139:e1082–143. 10.1161/CIR.0000000000000625

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 8.

  IslamFMWuJJanssonJWilsonDP. Relative risk of cardiovascular disease among people living with HIV: a systematic review and meta-analysis. HIV Med. (2012) 13:453–68. 10.1111/j.1468-1293.2012.00996.x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 9.

  BavingerCBendavidENiehausKOlshenRAOlkinISundaramVet alRisk of cardiovascular disease from antiretroviral therapy for HIV: a systematic review. PLoS One. (2013) 8:e59551. 10.1371/journal.pone.0059551

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 10.

  MoonsKGAltmanDGReitsmaJBIoannidisJPAMacaskillPSteyerberyEWet alTransparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med. (2015) 162:W1–73. 10.7326/M14-0698

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 11.

  CollinsGSReitsmaJBAltmanDGMoonsKG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Br Med J. (2015) 350:g7594. 10.1136/bmj.g7594

  • CrossRef
  • Google Scholar

• 12.

  MoonsKGRoystonPVergouweYGrobbeeDEAltmanDG. Prognosis and prognostic research: what, why, and how?Br Med J. (2009) 338:b375. 10.1136/bmj.b375

  • CrossRef
  • Google Scholar

• 13.

  FarzadfarF. Cardiovascular disease risk prediction models: challenges and perspectives. Lancet Glob Health. (2019) 7:e1288–9. 10.1016/S2214-109X(19)30365-1

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 14.

  D'AgostinoRBVasanRSPencinaMJWolfPACobainMMassaroJMet alGeneral cardiovascular risk profile for use in primary care. Circulation. (2008) 117:743–53. 10.1161/CIRCULATIONAHA.107.699579

  • CrossRef
  • Google Scholar

• 15.

  ConroyR. Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project. Eur Heart J. (2003) 24:987–1003. 10.1016/S0195-668X(03)00114-3

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 16.

  GoffDCLloyd-JonesDMBennettGCoadySD'AgostinoRBGibbonsRet al2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American college of cardiology/American heart association task force on practice guidelines. Circulation. (2014) 129:S49–73. 10.1161/01.cir.0000437741.48606.98

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 17.

  DamenJAAGHooftLSchuitEDebrayTPACollinsGSTzoulakiIet alPrediction models for cardiovascular disease risk in the general population: systematic review. Br Med J. (2016) 353:i2416. 10.1136/bmj.i2416

  • CrossRef
  • Google Scholar

• 18.

  Friis-MøllerNSabinCAWeberRet alCombination antiretroviral therapy and the risk of myocardial infarction. N Engl J Med. (2003) 349:1993–2003. 10.1056/NEJMoa030218

  • CrossRef
  • Google Scholar

• 19.

  DelabaysBCavassiniMDamasJBeuretHCalmyAHasseBet alCardiovascular risk assessment in people living with HIV compared to the general population. Eur J Prev Cardiol. (2022) 29:689–99. 10.1093/eurjpc/zwab201

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 20.

  TriantVAPerezJReganSMassaroJMMeigsJBGrinspoonSKet alCardiovascular risk prediction functions underestimate risk in HIV infection. Circulation. (2018) 137:2203–14. 10.1161/CIRCULATIONAHA.117.028975

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 21.

  SoaresCKwokMBoucherKAHajiMEchouffo-TcheuguiJBLongeneckerCTet alPerformance of cardiovascular risk prediction models among people living with HIV: a systematic review and meta-analysis [published online ahead of print, 2022 Dec 28]. JAMA Cardiol. (2022) 8(2):139–49. 10.1001/jamacardio.2022.4873

  • CrossRef
  • Google Scholar

• 22.

  ZouJSchiebingerL. AI Can be sexist and racist—it's Time to make it fair. Nature. (2018) 559:324–26. 10.1038/d41586-018-05707-8

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 23.

  MoonsKGde GrootJABouwmeesterWVergouweYMallettSAltmanDGet alCritical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Med. (2014) 11:e1001744. 10.1371/journal.pmed.1001744

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 24.

  PageMJMcKenzieJEBossuytPMBoutronIHoffmannTCMulrowCDet alThe PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Br Med J. (2021) 372:n71. 10.1136/bmj.n71

  • CrossRef
  • Google Scholar

• 25.

  WolffRFMoonsKGMRileyRDWhitingRFWestwoodMCollinsGSet alPROBAST: a tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med. (2019) 170:51–8. 10.7326/M18-1376

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 26.

  TufanaruCMunnZStephensonMAromatarisE. Fixed or random effects meta-analysis? Common methodological issues in systematic reviews of effectiveness. Int J Evid Based Healthc. (2015) 13:196–207. 10.1097/XEB.0000000000000065

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 27.

  SnellKIEnsorJDebrayTPMoonsKGRileyRD. Meta-analysis of prediction model performance across multiple studies: which scale helps ensure between-study normality for the C-statistic and calibration measures?Stat Methods Med Res. (2018) 27:3505–22. 10.1177/0962280217705678

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 28.

  NCSS. Confidence Intervals for the Area Under an ROC Curve. Available at:https://www.ncss.com/wp-content/themes/ncss/pdf/Procedures/PASS/Confidence_Intervals_for_the_Area_Under_an_ROC_Curve.pdf(Accessed March 10, 2022).

• 29.

  HanleyJAMcNeilBJ. The meaning under a receiver characteristic and use of the area operating (ROC) curve. Radiology. (1982) 143:29–36. 10.1148/radiology.143.1.7063747

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 30.

  AnikpoIAgoviAMACvitanovichMJLonerganFJohnsonMOjhaRP. The data-collection on adverse effects of anti-HIV drugs (D:A:D) model for predicting cardiovascular events: external validation in a diverse cohort of people living with HIV. HIV Med. (2021) 22:936–43. 10.1111/hiv.13147

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 31.

  De SocioGVPucciGBaldelliFSchillaciG. Observed versus predicted cardiovascular events and all-cause death in HIV infection: a longitudinal cohort study. BMC Infect Dis. (2017) 17:414. 10.1186/s12879-017-2510-x

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 32.

  FeinsteinMJNanceRMDrozdDRNingHDelaneyJAHeckbertSRet alAssessing and refining myocardial infarction risk estimation among patients with human immunodeficiency virus. JAMA Cardiol. (2017) 2:155–62. 10.1001/jamacardio.2016.4494

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 33.

  Friis-MøllerNThiébautRReissPWeberRMonforteADDe WitSet alPredicting the risk of cardiovascular disease in HIV-infected patients: the data collection on adverse effects of anti-HIV drugs study. Eur J Cardiovasc Prev Rehabil. (2010) 17:491–501. 10.1097/HJR.0b013e328336a150

  • CrossRef
  • Google Scholar

• 34.

  Friis-MøllerNRyomLSmithCWeberRReissPDabisFet alAn updated prediction model of the global risk of cardiovascular disease in HIV-positive persons: the data-collection on adverse effects of anti-HIV drugs (D:A:D) study. Eur J Prev Cardiol. (2016) 23:214–23. 10.1177/2047487315579291

  • CrossRef
  • Google Scholar

• 35.

  García-PeñaAADe-VriesEAldana-BitarJCáceresEBoteroJVásquez-JiménezJet alCardiovascular risk prediction models in people living with HIV in Colombia. Rev Invest Clin. (2022) 74:23–30. 10.24875/RIC.21000251

  • CrossRef
  • Google Scholar

• 36.

  HerreraSGuelarASorliLVilaJMolasEGrauMet alThe Framingham function overestimates the risk of ischemic heart disease in HIV-infected patients from Barcelona. HIV Clin Trials. (2016) 17:131–9. 10.1080/15284336.2016.1177266

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 37.

  RaggiPDe FrancescoDManicardiMZonaSBellasiAStentarelliCet alPrediction of hard cardiovascular events in HIV patients. J Antimicrob Chemother. (2016) 71:3515–18. 10.1093/jac/dkw346

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 38.

  SchulzCAMavaraniLReinschNAlbayrak-RenaSPotthoffABrockmeyerNet alPrediction of future cardiovascular events by Framingham, SCORE and ASCVD risk scores is less accurate in HIV-positive individuals from the HIV-HEART study compared with the general population. HIV Med. (2021) 22:732–41. 10.1111/hiv.13124

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 39.

  Thompson-PaulAMLichtensteinKAArmonCPalellaFJJrSkarbinskiJChmielJSet alCardiovascular disease risk prediction in the HIV outpatient study. Clin Infect Dis. (2016) 63:1508–16. 10.1093/cid/ciw615

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 40.

  van ZoestRALawMSabinCAVaartjesIvan der ValkMArendsJEet alPredictive performance of cardiovascular disease risk prediction algorithms in people living with HIV. J Acquir Immune Defic Syndr. (2019) 81:562–71. 10.1097/QAI.0000000000002069

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 41.

  SperrinMMartinGPSiskRPeekN. Missing data should be handled differently for prediction than for description or causal explanation. J Clin Epidemiol. (2020) 125:183–87. 10.1016/j.jclinepi.2020.03.028

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 42.

  ZhangXYanCGaoCMalinBAChenY. Predicting missing values in medical data via XGBoost regression. J Healthc Inform Res. (2020) 4:383–94. 10.1007/s41666-020-00077-1

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 43.

  MercaldoSFBlumeJD. Missing data and prediction: the pattern submodel. Biostatistics. (2020) 21:236–52. 10.1093/biostatistics/kxy040

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 44.

  HooglandJBarreveldMDebrayTPAReitsmaJBVerstraelenTEDijkgraafMGet alHandling missing predictor values when validating and applying a prediction model to new patients. Stat Med. (2020) 39:3591–607. 10.1002/sim.8682

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 45.

  RileyRDEnsorJSnellKIEHarrellFEJrMartinGPReitsmaJBet alCalculating the sample size required for developing a clinical prediction model. Br Med J. (2020) 368:m441. 10.1136/bmj.m441

  • CrossRef
  • Google Scholar

• 46.

  CollinsGSOgundimuEOAltmanDG. Sample size considerations for the external validation of a multivariable prognostic model: a resampling study. Stat Med. (2016) 35:214–26. 10.1002/sim.6787

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 47.

  McGettrickPMallonPWGSabinCA. Cardiovascular disease in HIV patients: recent advances in predicting and managing risk. Expert Rev Anti Infect Ther. (2020) 18:677–88. 10.1080/14787210.2020.1757430

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 48.

  TriantVAGrinspoonSK. Epidemiology of ischemic heart disease in HIV. Curr Opin HIV AIDS. (2017) 12:540–47. 10.1097/COH.0000000000000410

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 49.

  LundgrenJMocroftARyomL. Contemporary protease inhibitors and cardiovascular risk. Curr Opin Infect Dis. (2018) 31:8–13. 10.1097/QCO.0000000000000425

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 50.

  RyomLLundgrenJDEl-SadrWReissPKirkOLawMet alCardiovascular disease and use of contemporary protease inhibitors: the D:A:D international prospective multicohort study. Lancet HIV. (2018) 5:e291–300. 10.1016/S2352-3018(18)30043-2

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 51.

  LarsonDWonSHGanesanAMavesRCKronmannKOkuliczJFet alStatin usage and cardiovascular risk among people living with HIV in the U.S. military HIV natural history study. HIV Med. (2022) 23:249–58. 10.1111/hiv.13195

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 52.

  BoccaraFLangSMeulemanCEderhySMary-KrauseMCostagliolaDet alHIV and coronary heart disease: time for a better understanding. J Am Coll Cardiol. (2013) 61:511–23. 10.1016/j.jacc.2012.06.063

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 53.

  HoelHUelandTKnudsenAKjærAMichelsenAESagenELet alSoluble markers of interleukin 1 activation as predictors of first-time myocardial infarction in HIV-infected individuals. J Infect Dis. (2019) 221:506–09. 10.1093/infdis/jiz253

  • CrossRef
  • Google Scholar

• 54.

  SteyerbergEWHarrellFJBorsboomGJEijkemansMJVergouweYHabbemaJD. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol. (2001) 54:774–81. 10.1016/S0895-4356(01)00341-9

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 55.

  RamspekCLJagerKJDekkerFWZoccaliCvan DiepenM. External validation of prognostic models: what, why, how, when and where?Clin Kidney J. (2021) 14:49–58. 10.1093/ckj/sfaa188

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 56.

  AlbaACAgoritsasTWalshMHannaSIorioADevereauxPJet alDiscrimination and calibration of clinical prediction models: users’ guides to the medical literature. JAMA. (2017) 318:1377–84. 10.1001/jama.2017.12126

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 57.

  HuangYLiWMacheretFGabrielRAOhno-MachadoL. A tutorial on calibration measurements and calibration models for clinical prediction models. J Am Med Inform Assoc. (2020) 27:621–33. 10.1093/jamia/ocz228

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 58.

  StevensRJPoppeKK. Validation of clinical prediction models: what does the “calibration slope” really measure?J Clin Epidemiol. (2020) 118:93–9. 10.1016/j.jclinepi.2019.09.016

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 59.

  CostantinoSPaneniFCosentinoF. Ageing, metabolism and cardiovascular disease. J Physiol. (2016) 594:2061–73. 10.1113/JP270538

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 60.

  HinesLEMurphyJE. Potentially harmful drug–drug interactions in the elderly: a review. Am J Geriatr Pharmacother. (2011) 9:364–77. 10.1016/j.amjopharm.2011.10.004

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 61.

  FarooqiMAMGersteinHYusufSLeongDP. Accumulation of deficits as a key risk factor for cardiovascular morbidity and mortality: a pooled analysis of 154 000 individuals. J Am Heart Assoc. (2020) 9:e014686. 10.1161/JAHA.119.014686

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 62.

  VeroneseNCeredaEStubbsBSolmiMLuchiniCManzatoEet alRisk of cardiovascular disease morbidity and mortality in frail and pre-frail older adults: results from a meta-analysis and exploratory meta-regression analysis. Ageing Res Rev. (2017) 35:63–73. 10.1016/j.arr.2017.01.003

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 63.

  FeinsteinMJHsuePYBenjaminLABloomfieldGSCurrierJSFreibergMSet alCharacteristics, prevention, and management of cardiovascular disease in people living with HIV: a scientific statement from the American heart association. Circulation. (2019) 140:e98–124. 10.1161/CIR.0000000000000695

  • Pubmed Abstract
  • CrossRef
  • Google Scholar