Risk factors and clinical prediction models for low-level viremia in people living with HIV receiving antiretroviral therapy: an 11-year retrospective study

Objective: This study explores the risk factors for low-level viremia (LLV) occurrence after ART and develops a risk prediction model.

Method: Clinical data and laboratory indicators of people living with HIV (PLWH) at Hangzhou Xixi Hospital from 5 April 2011 to 29 December 2022 were collected. LASSO Cox regression and multivariate Cox regression analysis were performed to identify laboratory indicators and establish a nomogram for predicting LLV occurrence. The nomogram’s discrimination and calibration were assessed via ROC curve and calibration plots. The concordance index (C-index) and decision curve analysis (DCA) were used to evaluate its effectiveness.

Result: Predictive factors, namely, age, ART delay time, white blood cell (WBC) count, baseline CD4⁺ T-cell count (baseline CD4), baseline viral load (baseline VL), and total bilirubin (TBIL), were incorporated into the nomogram to develop a risk prediction model. The optimal model (which includes 6 variables) had an AUC for LLV after 1-year, 3-year, and 5-year of listing of 0.68 (95% CI, 0.61–0.69), 0.69 (95% CI, 0.65–0.70), and 0.70 (95% CI, 0.66–0.71), respectively. The calibration curve showed high consistency between predicted and actual observations. The C-index and DCA indicated superior prediction performance of the nomogram. There was a significant difference in CD4 levels between LLV and non-LLV groups during the follow-up time. The dynamic SCR, ALT, TG and BG levels and occurrence of complications differed significantly between the high- and low-risk groups.

Conclusion: A simple-to-use nomogram containing 6 routinely detected variables was developed for predicting LLV occurrence in PLWH after ART.

Introduction

Acquired immunodeficiency syndrome (AIDS) caused by the human immunodeficiency virus (HIV) has long represented a major public health threat (Mallewa et al., 2018). Antiretroviral therapy (ART) effectively inhibits HIV replication leading to virological suppression in most people living with HIV (PLWH) (Reeves et al., 2018). However, in approximately 20–30% of PLWH present with a viral load (VL) of between 50 and 999 copies/mL after 6 months of ART, which, according the World Health Organization (WHO), is defined as low-level viremia (LLV) (Lan et al., 2023; Liu et al., 2024).

Importantly, LLV has been associated with an increased risk of virological failure (Elvstam et al., 2023; Hermans et al., 2018), altered immune status (Ruggiero et al., 2018), emergence of mutations associated with drug resistance (Taramasso et al., 2020; Villalobos et al., 2020), as well as serious AIDS- and non-AIDS-related morbidity and mortality (Elvstam et al., 2021; Li et al., 2021; Galli et al., 2020). Therefore, early diagnosis of LLV is critical for the effective long-term management of PLWH.

Causes and predictors of LLV, as well as its clinical significance are still controversial. Several HIV cohort studies have found that the lower baseline CD4⁺ T cell counts and higher baseline VL are associated with LLV risk (Lombardi et al., 2024; Bai et al., 2022). In addition, Zhang et al. revealed that other risk factors associated with included long-term ART, Manchu (a member of an Indigenous people of Manchuria), and subtype B′ infection (Zhang et al., 2020). Another previous study has shown that in addition to these factors, gender, therapy history, co-receptor tropism, and VL at ART initiation could be valuable predictive markers to identify patients at risk for long-term persistent LLV (Vancoillie et al., 2014). Although numerous predictors related to the occurrence of LLV have been reported, it is imperative to identify novel sensitive markers of LLV.

The progression of viremia in PLWH is complex and involves multiple factors; therefore, the use of a single factor to accurately predict the outcome of LLV is not feasible. Nomogram, as a convenient and useful models for clinicians, integrates multiple key factors, and each relevant parameter is scored based on its value (Chen et al., 2022). In the context of AIDS, the nomograms are frequently applied to assess the risk of HIV infection and the incidence of AIDS and non-AIDS diseases (Yang et al., 2022). With the development of ART in recent years, some nomograms have also been established to predict the mortality and outcome of immune reconstitution of PLWH after ART (Jiang et al., 2022; Wang et al., 2024). Currently, little is known about the utility of nomograms in predicting LLV among PLWH after ART.

In this study, we identified some determinants of LLV among PLWH and developed a simple-to-use nomogram to predict the outcome of LLV after ART in these patients.

Methods

Sample size calculation

According previous study, the incidence of LLV was 9% (Elvstam et al., 2021), the sensitivity was 0.9 (Negida et al., 2019), the allowable error L was set at 10%, the test level α = 0.05 (bilateral). The sample size of the diagnostic test was calculated according to the sample size formula.

$\begin{array}{l}\mathrm{N}={\mathrm{Z}}_{\alpha /2}^2\times \frac{{\mathrm{S}}_{\mathrm{N}}\left(1-{\mathrm{S}}_{\mathrm{N}}\right)}{{\mathrm{L}}^2\times \mathrm{P}}=1.96\times 1.96\times 0.9\left(1-0.9\right)/\\ \left({0.1}^2\times 9\%\right)=384.\end{array}$

Considering the shedding rate of about 20%, the total sample size required was 461 cases, at least.

Study population and design

In the initial study, a total of 1816 PLWH were recruited. All enrolled participants received standardized ART treatment between 5 April 2011 and 29 December 2022, at Hangzhou Xixi Hospital, the largest AIDS designated hospital in Zhejiang Province, China. The inclusion criteria were as follows: (1) age > 18 years; (2) have received ART for at least 6 months; (3) the plasma VL was lower than 1,000 copies/mL; (4) followed over the period 2011–2022; (5) with at least two consecutive viremia values/per year, in the period analyzed, carried out at least 1 month apart. The exclusion criteria were as follows: (1) initial ART regimens were unknown; (2) predictor variables were incomplete. The flowchart is shown in Figure 1.

Figure 1

Figure 1. Flowchart of all participants in the study. LASSO, the least absolute shrinkage and selection operator; ART, antiretroviral therapy; DCA, decision curve analysis; C-index, concordance index.

Outcome definitions

In our study, the outcome of interest was LLV, which was defined as the occurrence of at least two consecutive VL measurements of VL ≥ 50 to ≤999 copies/mL after 6 months of ART.

Data collection

Data for the following demographic and clinical variables were collected at baseline via a questionnaire: age (pre-ART), sex, marital status, alcohol consumption status, smoking status, body mass index (BMI), monthly salary, education level, route of transmission, hepatitis B infection status, hepatitis C infection status, complications (high blood pressure, hyperglycemia and others), ART initial regimens, use of cotrimoxazole prophylaxis, ART delay time (months, time from HIV diagnosis to ART initiation) (Bogdanić et al., 2021), and time initiated (years). The data of comorbidities were collected at the last patients evaluation. The laboratory parameters included white blood cell (WBC), baseline CD4⁺ T-cell counts, baseline VL, hemoglobin (HB), total bilirubin (TBIL), platelet (PLT), creatinine (CR), aspartate aminotransferase (AST), alanine aminotransferase (ALT), AST/ALT ratio, total cholesterol (TC), total triglyceride (TG), and blood glucose (BG) required to be longitudinally collected (from baseline to the last patients evaluation).

Predictor selection

The least absolute shrinkage and selection operator (LASSO) Cox method was applied for data dimension reduction and the optimal predictors were selected for the nomogram (Wang et al., 2024). The “glmnet” program package was utilized in the R software, and the “family = Cox” model parameter was selected (Li et al., 2022).

Construction of nomogram

The predictive model was visualized and presented as a nomogram. The nomogram was drawn based on the results of the multivariate Cox regression analysis which used the “rms” package of R software. To calculate the score for each variable at each level, a scoring standard was developed based on the standard regression coefficients of all variables (Wang et al., 2024). Then, the scores of these factors were used to calculate a total score to reflect the probability of LLV in each patient.

Statistical analyses

All statistical analysis was performed using the R statistical software version 4.1.3 (R Foundation for Statistical Computing, Vienna, Austria) and IBM SPSS Statistic version 25.0 (IBM, Armonk, NY, USA). Continuous variables are presented as mean and standard deviation (SD). Categorical variables are presented as frequency and percentage, and the percentages were compared using Pearson’s chi-square tests (Zhang et al., 2023). All line graphs were generated using GraphPad Prism version 8.0.2 (263) (GraphPad Software, San Diego, CA, USA).

A multivariable Cox regression model was applied to determine covariate association with LLV. A hazard ratio (HR) of greater or less than 1 was interpreted as an increased or decreased association with LLV, respectively. We used the “Survival” and “Survminer” packages in R to analyze the LLV data from PLWH. The predictive accuracy of the risk model was assessed using the receiver operating characteristic (ROC) curve. The area under ROC curve (AUC) was plotted with the MedCalc software package version 18.2.1 (MedCalc Software, Ostend, Belgium) using sensitivity and specificity values (Kats et al., 2021). The calibration plot was implied by a 45° diagonal line with 1,000 bootstrap samples (Yu et al., 2020; Yuan et al., 2021). The concordance index (C-index) (Wang et al., 2024) and decision curve analysis (DCA) (Vickers et al., 2019) were plotted to evaluate the clinical usefulness and applicability of net benefits of the model with the best predictive value. Kaplan–Meier curves were drawn for PLWH with or without LLV and compared using log-rank tests. A generalized linear mixed model (GLMM) was used to compare the differences in clinical indexes between two groups of PLWH under different follow-up times. All reported levels of statistical significance were two-sided, and p-values <0.05 were considered to indicate statistical significance (Wang et al., 2023).

Results

Study populations

Based on the inclusion and exclusion criteria, 1,568 PLWH were included in the retrospective analysis (Figure 1). The 248 excluded PLWH comprised 6 patients with unknown initial ART regimens and 242 patients with incomplete clinical data. The mean age of the 1,568 participants was 32.54 years (SD: 11.2); 94.6% were male, while 5.4% were female; and the mean BMI was 22.61 (SD: 3.11). In our study, 11.1% (174/1568) of PLWH had LLV after long-term ART (Table 1). Further details are provided in Table 1.

Table 1

Table 1. Baseline characteristics of all participants (n = 1,568).

Selection of risk predictors

Variables with p less than 0.1 in the Lasso regression analysis were included in the univariate and multivariable Cox regression. As shown in Figure 2, with the increase of λ, the variable coefficients continued to decrease. The optimal harmonic parameter λ was selected through cross-validation. In this case, lambda.min was selected at 0.0087, as shown in Figure 2. Finally, seven optimal parameter values were included into the further analysis.

Figure 2

Figure 2. LASSO Cox regression plot. (A) Plot of partial likelihood deviance; (B) plot of LASSO coefficient profiles. Each color curve represents the LASSO coefficient profile of a feature against the Log (λ) sequence. The values above the figure represent the numbers of variables included in the model, with the corresponding λ shown on the x-axis. λ, lambda.

Establishment of risk score model

We first calculated the cut-off values for six continuous variables that were significantly associated with the outcome of LLV. The optimal cut-off value for age was >30 years, with a sensitivity and specificity of 58.62 and 57.96%, respectively. The baseline CD4 had 61.49% sensitivity and 56.1% specificity at an optimal cut-off value of ≤290 cells/μL. The optimal cut-off value of ART delay time was >1 month, with a sensitivity and specificity of 44.25 and 71.45%, respectively. The HB had 49.43% sensitivity and 67.86% specificity at an optimal cut-off value of ≤143 g/L (Supplementary Table S1). The optimal cut-off value of TBIL was ≤9.8 μmol/L, with a sensitivity and specificity of 54.02 and 65.93%, respectively. The WBC count had 28.74% sensitivity and 77.83% specificity at an optimal cut-off value of >6.86 cells/μL (Supplementary Table S1).

We then used the univariate and multivariable Cox regression analysis to screen out six variables, including age, baseline CD4, ART delay time, baseline VL, TBIL, and WBC count (all p < 0.05, Table 2). As in Figure 3A, the forest maps showed that the age (HR = 1.676; 95% CI: 1.236–2.272; p < 0.001), ART delay time (HR = 1.732; 95% CI: 1.271–2.359; p < 0.001), baseline CD4 (HR = 1.875; 95% CI: 1.368–2.569; p < 0.001), baseline VL (HR = 1.606; 95% CI: 1.310–1.969; p < 0.001), TBIL (HR = 1.698; 95% CI: 1.252–2.303; p < 0.001) and WBC (HR = 1.549; 95% CI: 1.106–2.168; p = 0.011), which were significantly associated with LLV. Another study showed that Normal liver function group and Abnormal liver function group among patients with HIV-1 mono-infection receiving ART.

Table 2

Table 2. Univariate and multivariate cox regression analysis for factors associated with the outcome of LLV for PLWH.

Figure 3

Figure 3. Multivariate Cox regression and ROC curves analysis was used to evaluate the risk factors associated with LLV. (A) Forrest plot of multivariate Cox regression analysis. (B) Time-independent ROC curves of the nomogram for 1-, 3- and 5-year overall survival prediction in the PLWH cohort.

We then performed ROC analysis to assess the ability of different models consisting of 6 independent variables to predict the outcome of LLV among PLWH. Table 3 compared the AUC values of five models for 1-year, 3-year and 5-year occurrence of LLV in PLWH cohort. The combined model 1 (includes six variables) had an AUC for LLV after 1-year, 3-year and 5-year of listing of 0.68 (95% CI, 0.61–0.69), 0.69 (95% CI, 0.65–0.70) and 0.70 (95% CI, 0.66–0.71), respectively (Table 3). The combined model 2 (includes five variables, except for age) had an AUC for LLV after 1-year, 3-year and 5-year of listing of 0.69 (95% CI, 0.64–0.70), 0.69 (95% CI, 0.65–0.70) and 0.69 (95% CI, 0.66–0.71), respectively (Table 3). The combined model 3 (includes five variables, except for WBC) had an AUC for LLV after 1-year, 3-year and 5-year of listing of 0.67 (95% CI, 0.60–0.68), 0.68 (95% CI, 0.63–0.69) and 0.68 (95% CI, 0.65–0.70), respectively (Table 3). The combined model 4 (includes five variables, except for TBIL) had an AUC for LLV after 1-year, 3-year and 5-year of listing of 0.64 (95% CI, 0.58–0.65), 0.67 (95% CI, 0.63–0.68) and 0.70 (95% CI, 0.66–0.71), respectively (Table 3). The combined model 5 (includes five variables, except for delay time) had an AUC for LLV after 1-year, 3-year and 5-year of listing of 0.66 (95% CI, 0.59–0.67), 0.67 (95% CI, 0.63–0.69) and 0.69 (95% CI, 0.66–0.71), respectively (Table 3). Given the model with fewer variables was more susceptible to underfitting and the optimal prediction model should include statistically significant variables, the combined model 1 containing age, baseline CD4, ART delay time, baseline VL, TBIL, and WBC was used to evaluate the occurrence of LLV. Plots of time-dependent for 1-year, 3-year and 5-year AUCs are shown in Figure 3B.

Table 3

Table 3. Predictive values of different models for outcome of LLV after 1, 3, 5 years.

In our risk model, PLWH were divided into two groups according to risk (high vs. low), and the grouping criterion was the median of the risk score. Subsequent survival analysis revealed that the outcome of LLV differed significantly between the two groups (Figure 4A, p < 0.0001). The risk curve shows the relationship between LLV and the risk of PLWH. Figure 4B shows the risk values of PLWH in the two groups. PLWH in the high-risk group had significantly higher incidence of LLV than those in the low-risk group (Figure 4C).

Figure 4

Figure 4. Construction of the risk score model with respect to PLWH with LLV. (A) Kaplan–Meier curves of the overall survival of the two LLV risk groups of PLWH with LLV. (B,C) Risk score curves and scatter plots of the risk of LLV among the PLWH cohorts.

Performance and evaluation of risk nomogram

Based on the above results, 6 variables, including age, baseline CD4, ART delay time, baseline VL, TBIL, and WBC, were incorporated into a risk predictive nomogram for predicting the occurrence of LLV among PLWH. Figure 5 shows the protocol for the use of the risk nomogram. The score of each predictive factor could be calculated, and then the total score was calculated as the sum of scores of 6 predictors. In our model, PLWH with higher score represented the higher risk of LLV after ART.

Figure 5

Figure 5. Nomogram for evaluation of LLV in PLWH. To use the nomogram, a line was first drawn from each parameter value to the score axis for the score; then the points for all the parameters were added. Finally, a line from the total score axis was drawn to determine the occurrence of LLV on the lower line of the nomogram.

Calibration plots were constructed to assess the concordance between predicted and actual outcomes (Yuan et al., 2020). Our calibration curves for each model at 1, 3, and 5 years are shown in Figure 6A. The calibration plots exhibited a high conformance in predicting the occurrence of LLV in the PLWH cohort. In addition, DCA has been widely used to evaluate the efficacy of specific clinical approaches in many studies (Kerr et al., 2016). Our DCA curve revealed that the risk score provided a greater net benefit compared with the clinical model across a wide range (17 to 70%) of risk thresholds (Figure 6B). Across the 1,000 bootstrap resamples, the optimism-corrected C-index for the risk nomogram was close to 0.7 (Figure 6C), which was a more accurate and robust estimate of performance.

Figure 6

Figure 6. Calibration plots, DCA curves, C-index for the predictive nomogram. (A) Calibration curves for 1-, 3- and 5-year survival depict the calibration of nomogram in terms of the agreement between the predicted probabilities and observed outcomes of PLWH cohort. (B) The results of DCA analysis for the nomogram. (C) Time-dependent C-index of the nomogram model.

Dynamic changes in CD4, WBC and TBIL levels between the LLV and non-LLV groups following ART

GLMM analysis showed that there was a significant difference in CD4 levels (p < 0.001, Figure 7A) between LLV and non-LLV groups from T1 to T9 after ART. While the difference in the levels of WBC (p > 0.05, Figure 7B) and TBIL (p > 0.05, Figure 7C) between LLV and non-LLV groups was not obvious during the follow-up time (Supplementary Table S2).

Figure 7

Figure 7. Comparisons of the differences in clinical indexes between different groups after ART. The levels of CD4⁺ T cells (A), white blood cells (B), total bilirubin (C) at different follow-up time points in LLV and non-LLV groups. Serum levels of creatinine (D), alanine aminotransferase (E), aspartate aminotransferase (F), total triglycerides (G), total cholesterol (H), and blood glucose (I) at various different follow-up time points in high-risk and low-risk groups. Wilcoxon matched-pairs signed-rank test with two-tailed p-value was used for comparison between groups. ns: p > 0.05, *: p < 0.05, **: p < 0.01, ***: p < 0.001.

Dynamic changes in metabolic indexes after ART among high and low risk groups

Based on the risk value, all PLWH were divided into two groups: high risk of LLV, comprising 744 patients; and low risk of LLV, comprising 824 patients. A GLMM analysis was performed to compare the differences in SCR, ALT, AST, TG, TC, and BG at different times between the two groups. As shown in Figure 7 and Supplementary Table S3, the levels of SCR (p = 0.01, Figure 7D), and ALT (p = 0.004, Figure 7E) in the LLV high-risk group were significantly lower than those in the low-risk group. The levels of TG (Figure 7G). and BG (Figure 7I) among high-risk PLWH were dramatically higher than in the low-risk PLWH (Both p < 0.05). However, there was no significant difference in the levels of AST (p > 0.05, Figure 7F) and TC (p > 0.05, Figure 7H) between the two groups (see Supplementary Table S3).

In addition, we analyzed the differences in the occurrence of various complications between the two groups. As shown in Table 4, there was a significant difference in the occurrence of high TC (p = 0.029), high TG (p = 0.001), and other complications (including diabetes, coronary heart disease, malignant tumors, and fatty liver) (p = 0.002) between the high-risk and low-risk groups. The occurrence of high BG in high-risk group was comparable to those in the low-risk group (Both p > 0.05, Table 4).

Table 4

Table 4. Comparisons of the complications between high-risk and low-risk groups.

Discussion

In this retrospective study, several predictors associated with LLV in PLWH were identified via the LASSO-Cox and multivariable Cox regression analysis. Among these, age, ART delay time, baseline CD4, baseline VL, TBIL, and WBC count were used to construct a predictive nomogram for the prediction of the outcome of LLV among PLWH after ART. The calibration plots, decision curve, and clinical impact curve demonstrated that the novel model possessed excellent discrimination and calibration performance, and the internal verification indicated the favorable applicability and extrapolation capability of this model.

LLV is a well documented independent risk factor for virologic failure (Li et al., 2021; Joya et al., 2019). Previous studies have shown that higher baseline VL is consistently associated with increased rates of LLV (Elvstam et al., 2021; Chen et al., 2021; Álvarez et al., 2023). For example, Hortensia et al. reported that baseline HIV-1 RNA > 100,000 copies/mL is an independent predictor of higher risk of LLV (Álvarez et al., 2022). In addition, Zhang et al. found that the zenith baseline viral load (VL) above 6 log copies/mL is strongly associated with high-risk LLV (Zhang et al., 2020). Our study is in agreement with these previous studies. Our data showed that the risk scores of LLV in PLWH with baseline VL > 500,000 copies/mL and 100,000 ~ 500,000 copies/mL were significantly higher than those in PLWH with baseline VL < 100,000 copies/mL. These data support that baseline VL is an independent risk factor for LLV.

The present study and previous studies have both confirmed that lower baseline CD4 level is a risk factor associated with LLV (Bai et al., 2022; Lao et al., 2024). In addition, our results found a significant higher risk of LLV in PLWH with baseline CD4 ≤ 290 cells/μL than in those with CD4 > 290 cells/μL. Zhang et al. found that nadir baseline CD4⁺ T cell counts below 200 cells/μL were clearly associated with high risk of LLV (Zhang et al., 2020). These findings highlight that initiation of ART at an early stage among PLWH with high baseline CD4⁺ T-cell counts may avoid the occurrence of LLV and enhance their quality of life.

Compared with PLWH without LLV, those with LLV were more likely to be older at ART initiation (Joya et al., 2019). Our results also showed a significantly higher risk of LLV in PLWH who were > 30 years at ART initiation. However, Wu et al. revealed that the older PLWH (> 50 years) were more likely to have poorer immune responses to HIV treatment than younger PLWH (18–49 years) (Wu et al., 2019). Although this difference could be attributed to the fact that 38.2% of PLWH were above 50 years of age in Wu et al.’s study, while our study population was younger. These findings suggest that older PLWH should be more closely monitored for the occurrence of LLV.

The above three predictors have been reported to be closely linked with LLV, and our data further support the previous findings. In addition, we identify novel predictors of LLV in PLWH. We found that PLWH with ART delay time > 1 month had significantly higher risk of LLV than PLWH with ART delay time ≤ 1 month. These data demonstrate that the longer the delay in initiating ART, the higher the risk of LLV. Previous studies have only suggested that the time from HIV diagnosis to ART initiation is associated with LLV based on univariate analysis, without performing multivariate analysis (Bai et al., 2022). Our data implied the importance of timely diagnosis of HIV infection and initiation of ART at an early stage.

A previous study has shown that serum TBIL levels are higher in ART-naive PLWH than in HIV-positive patients receiving first- and second-line ART (Kolgiri et al., 2018). Our results indicate that there is a significantly higher risk of LLV in PLWH with TBIL ≤9.8 μmol/L group than in those with TBIL >9.8 μmol/L. Therefore, serum TBIL, being an easy-to-measure routine clinical indicator, may serve as a predictor of LLV occurrence.

Previous study showed that the WBC count is lower in PLWH with high VL compared with that in PLWH with low VL (Lin et al., 2019). Our results verified that the risk of LLV in PLWH with WBC counts >6.86 cells/μL was higher than in those with WBC count ≤6.86 cells/μL. These data imply that the WBC count is a risk factor for the occurrence of LLV in PLWH. Of note, considering that WBC detection is simple and widely used in clinical practice, dynamic monitoring of WBC count would be a reliable and convenient strategy for assessment of the risk of LLV occurrence.

In the present study, the above six predictors (age, ART delay time, baseline CD4, baseline VL, TBIL, and WBC count) were combined to construct the predictive models. The present study is the first to report the construction of a nomogram for the prediction of LLV among PLWH after ART. Previous studies have only reported a close relationship between baseline CD4 or baseline VL or age at ART initiation and virologic failure in PLWH after ART. Our results indicate that these factors contribute to the process of LLV. Moreover, our study shows, for the first time, that ART delay time, TBIL, and WBC count are also independent predictors of LLV. Follow-up studies are needed to delve deeper into their roles and mechanisms in LLV. On the other hand, Calibration curves, receiver operating characteristic (ROC) curves, and consistency indexes (C-indexes) were used to estimate the performance of the model (Li et al., 2023; Bai et al., 2018). Decision curve analysis (DCA) curves were used to assess the clinical utility of the model. Our calibration plots (1-, 3-, 5- years), DCA curve and optimism-corrected C-index revealed that this model had excellent performance and clinical utility. The result showed that the nomogram was effective for predicting the occurrence of LLV in the PLWH cohorts. Notably, all predictors in our nomogram can be routinely detected in clinical practice. Thus, detection of these parameters is easy and cost-effective, which makes the dynamic assessment of LLV feasible in PLWH and is also helpful for the optimization of therapies.

The nomogram combining the above 6 risk factors not only showed good performance in terms of clinical prediction, but also enabled the categorization of patients into high- and low-risk groups. Furthermore, differences in the occurrence of high cholesterol, high TG, and other complications (including diabetes) between high- and low-risk groups was observed. Previous studies have shown that persistent viremia is a significant predictor of the development of metabolic syndrome (Squillace et al., 2009). The most common metabolic abnormality among PLWH was hypertriglyceridemia (Ang et al., 2021). These data support that PLWH with high risk of LLV were more likely to develop metabolic syndrome. In other words, PLWH at high risk of LLV need more individualized treatment.

Some limitations of this research should be noted. First, this study was a single-center retrospective analysis, and the nomogram requires further optimization and validation in multi-center PLWH populations. Second, the variables dataset has a missing rate of over 13%, which means that data on vital clinical parameters such as immune cells subpopulations and drug resistance status were not included. It is important incorporate these missing parameters in future studies to enhance the accuracy of the prediction model. Finally, prospective cohort studies should be performed to validate the model in subsequent research.

Conclusion

Our study identifies patient age, ART delay time, baseline CD4, baseline VL, TBIL, and WBC count as risk factors for the occurrence of LLV among PLWH after ART. In this research, we methodically developed an innovative nomogram, using these 6 clinical risk indicators, to predict the occurrence of LLV risk in PLWH. The constructed nomogram model enables individualized diagnostic analysis and is helpful in developing risk-adapted therapy for PLWH.

Data availability statement

The data analyzed in this study is subject to the following licenses/restrictions: the datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request. Requests to access these datasets should be directed to Yi Wang, wangyi_xixi@126.com.

Ethics statement

The studies involving humans were approved by Clinical Research Ethics Committee of the Hangzhou Xixi Hospital. The studies were conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation was not required from the participants or the participants’ legal guardians/next of kin because this study was a retrospective study. Using the clinical data has been approved by the Hangzhou Xixi Hospital ethics committee.

Author contributions

WZ: Writing – review & editing, Writing – original draft, Resources, Project administration, Methodology, Investigation, Funding acquisition, Data curation. JS: Writing – review & editing, Validation, Software, Resources, Investigation, Funding acquisition, Data curation. YinW: Writing – review & editing, Visualization, Validation, Supervision, Resources, Formal analysis, Data curation. EL: Writing – review & editing, Validation, Software, Investigation, Formal analysis. DY: Writing – review & editing, Validation, Supervision, Investigation, Formal analysis. ZZ: Writing – review & editing, Validation, Supervision, Resources, Conceptualization. MZ: Writing – review & editing, Validation, Supervision, Project administration. JY: Writing – review & editing, Validation, Supervision, Resources, Project administration, Funding acquisition, Conceptualization. YiW: Writing – review & editing, Writing – original draft, Visualization, Validation, Supervision, Project administration, Formal analysis, Conceptualization.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This research was supported by grants from the Medical Science and Technology Project of Zhejiang Province (2023KY978), Zhejiang Province Traditional Chinese Medicine Science and Technology Project (2024ZL758), Hangzhou Biomedicine and Health Industry Development Project (2022WJCY174).

Acknowledgments

We thank all authors for insightful discussions and advice. We thank Hangzhou Municipal Health Commission and Health Commission of Zhejiang Province for their financial support.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmicb.2024.1451201/full#supplementary-material

References

Álvarez, H., Mocroft, A., Ryom, L., Neesgaard, B., Edwards, S., Svedhem, V., et al. (2023). RESPOND (international cohort consortium of infectious diseases) study group. Plasma human immunodeficiency virus 1 RNA and CD4⁺ T-cell counts are determinants of Virological nonsuppression outcomes with initial integrase inhibitor-based regimens: a prospective RESPOND cohort study. Clin. Infect. Dis. 77, 593–605. doi: 10.1093/cid/ciad219

PubMed Abstract | Crossref Full Text | Google Scholar

Álvarez, H., Rava, M., Martínez, C., Portilla, J., Peraire, J., Rivero, A., et al. (2022). Predictors of low-level HIV viraemia and virological failure in the era of integrase inhibitors: a Spanish nationwide cohort. HIV Med. 23, 825–836. doi: 10.1111/hiv.13265

PubMed Abstract | Crossref Full Text | Google Scholar

Ang, L. W., Ng, O. T., Boudville, I. C., Leo, Y. S., and Wong, C. S. (2021). An observational study of the prevalence of metabolic syndrome in treatment-experienced people living with HIV in Singapore. PLoS One 16:e0252320. doi: 10.1371/journal.pone.0252320

PubMed Abstract | Crossref Full Text | Google Scholar

Bai, Y., Liu, Z. S., Xiong, J. P., Xu, W. Y., Lin, J. Z., Long, J. Y., et al. (2018). Nomogram to predict overall survival after gallbladder cancer resection in China. World J. Gastroenterol. 24, 5167–5178. doi: 10.3748/wjg.v24.i45.5167

PubMed Abstract | Crossref Full Text | Google Scholar

Bai, R., Lv, S., Hua, W., Su, B., Wang, S., Shao, Y., et al. (2022). Factors associated with human immunodeficiency virus-1 low-level viremia and its impact on virological and immunological outcomes: a retrospective cohort study in Beijing, China. HIV Med. 23, 72–83. doi: 10.1111/hiv.13251

PubMed Abstract | Crossref Full Text | Google Scholar

Bogdanić, N., Bendig, L., Lukas, D., Zekan, Š., and Begovac, J. (2021). Timeliness of antiretroviral therapy initiation in the era before universal treatment. Sci. Rep. 11:10508. doi: 10.1038/s41598-021-90043-7

PubMed Abstract | Crossref Full Text | Google Scholar

Chen, J., Li, L., Chen, T., Yang, X., Ru, H., Li, X., et al. (2022). Predicting the risk of active pulmonary tuberculosis in people living with HIV: development and validation of a nomogram. BMC Infect. Dis. 22:388. doi: 10.1186/s12879-022-07368-5

PubMed Abstract | Crossref Full Text | Google Scholar

Chen, G. J., Sun, H. Y., Chang, S. Y., Cheng, A., Huang, Y. S., Huang, S. H., et al. (2021). Incidence and impact of low-level viremia among people living with HIV who received protease inhibitor- or dolutegravir-based antiretroviral therapy. Int. J. Infect. Dis. 105, 147–151. doi: 10.1016/j.ijid.2021.02.045

Crossref Full Text | Google Scholar

Elvstam, O., Malmborn, K., Elén, S., Marrone, G., García, F., Zazzi, M., et al. (2023). Virologic failure following low-level viremia and viral blips during antiretroviral therapy: results from a European multicenter cohort. Clin. Infect. Dis. 76, 25–31. doi: 10.1093/cid/ciac762

PubMed Abstract | Crossref Full Text | Google Scholar

Elvstam, O., Marrone, G., Medstrand, P., Treutiger, C. J., Sönnerborg, A., Gisslén, M., et al. (2021). All-cause mortality and serious non-AIDS events in adults with low-level human immunodeficiency virus viremia during combination antiretroviral therapy: results from a Swedish Nationwide observational study. Clin. Infect. Dis. 72, 2079–2086. doi: 10.1093/cid/ciaa413

PubMed Abstract | Crossref Full Text | Google Scholar

Galli, L., Parisi, M. R., Poli, A., Menozzi, M., Fiscon, M., Garlassi, E., et al. (2020). Burden of disease in PWH harboring a multidrug-resistant virus: data from the PRESTIGIO registry. Open Forum Infect. Dis. 7:ofaa456. doi: 10.1093/ofid/ofaa456

PubMed Abstract | Crossref Full Text | Google Scholar

Hermans, L. E., Moorhouse, M., Carmona, S., Grobbee, D. E., Hofstra, L. M., Richman, D. D., et al. (2018). Effect of HIV-1 low-level viraemia during antiretroviral therapy on treatment outcomes in WHO-guided south African treatment programmes: a multicentre cohort study. Lancet Infect. Dis. 18, 188–197. doi: 10.1016/S1473-3099(17)30681-3

PubMed Abstract | Crossref Full Text | Google Scholar

Jiang, F., Xu, Y., Liu, L., Wang, K., Wang, L., Fu, G., et al. (2022). Construction and validation of a prognostic nomogram for predicting the survival of HIV/AIDS adults who received antiretroviral therapy: a cohort between 2003 and 2019 in Nanjing. BMC Public Health 22:30. doi: 10.1186/s12889-021-12249-8

PubMed Abstract | Crossref Full Text | Google Scholar

Joya, C., Won, S. H., Schofield, C., Lalani, T., Maves, R. C., Kronmann, K., et al. (2019). Persistent low-level viremia while on antiretroviral therapy is an independent risk factor for Virologic failure. Clin. Infect. Dis. 69, 2145–2152. doi: 10.1093/cid/ciz129

PubMed Abstract | Crossref Full Text | Google Scholar

Kats, L., Goldman, Y., and Kahn, A. (2021). Automatic detection of image sharpening in maxillofacial radiology. BMC Oral Health 21:411. doi: 10.1186/s12903-021-01777-9

PubMed Abstract | Crossref Full Text | Google Scholar

Kerr, K. F., Brown, M. D., Zhu, K., and Janes, H. (2016). Assessing the clinical impact of risk prediction models with decision curves: guidance for correct interpretation and appropriate use. J. Clin. Oncol. 34, 2534–2540. doi: 10.1200/JCO.2015.65.5654

PubMed Abstract | Crossref Full Text | Google Scholar

Kolgiri, V., Nagar, V., and Patil, V. (2018). Association of serum total bilirubin and plasma 8-OHdG in HIV/AIDS patients. Interv. Med. Appl. Sci. 10, 76–82. doi: 10.1556/1646.10.2018.02

PubMed Abstract | Crossref Full Text | Google Scholar

Lan, Y., Ling, X., Deng, X., Lin, Y., Li, J., Li, L., et al. (2023). Drug resistance profile among HIV-1 infections experiencing ART with low-level viral load in Guangdong China during 2011-2022: a retrospective study. Infect. Drug Resist. 16, 4953–4964. doi: 10.2147/IDR.S419610

PubMed Abstract | Crossref Full Text | Google Scholar

Lao, X., Zhang, H., Deng, M., Li, Q., Xiao, Q., He, L., et al. (2024). Incidence of low-level viremia and its impact on virologic failure among people living with HIV who started an integrase strand transfer inhibitors: a longitudinal cohort study. BMC Infect. Dis. 24:8. doi: 10.1186/s12879-023-08906-5

PubMed Abstract | Crossref Full Text | Google Scholar

Li, Q., Chen, M., Zhao, H., Yu, F., Yan, L., Xiao, J., et al. (2021). Persistent low-level viremia is an independent risk factor for Virologic failure: a retrospective cohort study in China. Infect. Drug Resist. 14, 4529–4537. doi: 10.2147/IDR.S332924

PubMed Abstract | Crossref Full Text | Google Scholar

Li, Y., Lu, F., and Yin, Y. (2022). Applying logistic LASSO regression for the diagnosis of atypical Crohn's disease. Sci. Rep. 12:11340. doi: 10.1038/s41598-022-15609-5

PubMed Abstract | Crossref Full Text | Google Scholar

Li, X., Xu, Q., Gao, C., Yang, Z., Li, J., Sun, A., et al. (2023). Development and validation of nomogram prognostic model for predicting OS in patients with diffuse large B-cell lymphoma: a cohort study in China. Ann. Hematol. 102, 3465–3475. doi: 10.1007/s00277-023-05418-9

PubMed Abstract | Crossref Full Text | Google Scholar

Lin, T. Y., Yang, C. J., Liu, C. E., Tang, H. J., Chen, T. C., Chen, G. J., et al. (2019). Clinical features of acute human immunodeficiency virus infection in Taiwan: a multicenter study. J. Microbiol. Immunol. Infect. 52, 700–709. doi: 10.1016/j.jmii.2018.01.005

PubMed Abstract | Crossref Full Text | Google Scholar

Liu, J., Li, C., Sun, Y., Fu, C., Wei, S., Zhang, X., et al. (2024). Characteristics of drug resistance mutations in ART-experienced HIV-1 patients with low-level viremia in Zhengzhou City, China. Sci. Rep. 14:10620. doi: 10.1038/s41598-024-60965-z

PubMed Abstract | Crossref Full Text | Google Scholar

Lombardi, F., Bruzzesi, E., Bouba, Y. R., Di Carlo, D., Costabile, V., Ranzenigo, M., et al. (2024). Factors associated with low-level viremia in people living with HIV in the Italian antiviral response cohort analysis cohort: a case-control study. AIDS Res. Hum. Retrovir. 40, 80–89. doi: 10.1089/AID.2023.0015

PubMed Abstract | Crossref Full Text | Google Scholar

Mallewa, J., Szubert, A. J., Mugyenyi, P., Chidziva, E., Thomason, M. J., Chepkorir, P., et al. (2018). Effect of ready-to-use supplementary food on mortality in severely immunocompromised HIV-infected individuals in Africa initiating antiretroviral therapy (REALITY): an open-label, parallel-group, randomised controlled trial. Lancet HIV 5, e231–e240. doi: 10.1016/S2352-3018(18)30038-9

PubMed Abstract | Crossref Full Text | Google Scholar

Negida, A., Fahim, N. K., and Negida, Y. (2019). Sample size calculation guide - part 4: how to calculate the sample size for a diagnostic test accuracy study based on sensitivity, specificity, and the area under the ROC curve. Adv. J. Emerg. Med. 3:e33. doi: 10.22114/ajem.v0i0.158

PubMed Abstract | Crossref Full Text | Google Scholar

Reeves, D. B., Duke, E. R., Wagner, T. A., Palmer, S. E., Spivak, A. M., and Schiffer, J. T. (2018). A majority of HIV persistence during antiretroviral therapy is due to infected cell proliferation. Nat. Commun. 9:4811. doi: 10.1038/s41467-018-06843-5

PubMed Abstract | Crossref Full Text | Google Scholar

Ruggiero, A., Cozzi-Lepri, A., Beloukas, A., Richman, D., Khoo, S., Phillips, A., et al. (2018). Factors associated with persistence of plasma HIV-1 RNA during Long-term continuously suppressive Firstline antiretroviral therapy. Infect. Dis. 5:ofy032. doi: 10.1093/ofid/ofy032

PubMed Abstract | Crossref Full Text | Google Scholar

Squillace, N., Zona, S., Stentarelli, C., Orlando, G., Beghetto, B., Nardini, G., et al. (2009). Detectable HIV viral load is associated with metabolic syndrome. J. Acquir. Immune Defic. Syndr. 52, 459–464. doi: 10.1097/QAI.0b013e3181b93a23

PubMed Abstract | Crossref Full Text | Google Scholar

Taramasso, L., Magnasco, L., Bruzzone, B., Caligiuri, P., Bozzi, G., Mora, S., et al. (2020). How relevant is the HIV low level viremia and how is its management changing in the era of modern ART? A large cohort analysis. J. Clin. Virol. 123:104255. doi: 10.1016/j.jcv.2019.104255

PubMed Abstract | Crossref Full Text | Google Scholar

Vancoillie, L., Demecheleer, E., Callens, S., Vogelaers, D., Vandekerckhove, L., Mortier, V., et al. (2014). Markers associated with persisting low-level viraemia under antiretroviral therapy in HIV-1 infection. J. Antimicrob. Chemother. 69, 1098–1103. doi: 10.1093/jac/dkt484

PubMed Abstract | Crossref Full Text | Google Scholar

Vickers, A. J., van Calster, B., and Steyerberg, E. W. (2019). A simple, step-by-step guide to interpreting decision curve analysis. Diagn. Progn. Res. 3:18. doi: 10.1186/s41512-019-0064-7

PubMed Abstract | Crossref Full Text | Google Scholar

Villalobos, C., Ceballos, M. E., Ferrés, M., and Palma, C. (2020). Drug resistance mutations in proviral DNA of HIV-infected patients with low level of viremia. J. Clin. Virol. 132:104657. doi: 10.1016/j.jcv.2020.104657

PubMed Abstract | Crossref Full Text | Google Scholar

Wang, F., Kang, X., Li, Y., Lu, J., Liu, X., and Yan, H. (2024). Elucidating hepatocellular carcinoma progression: a novel prognostic miRNA-mRNA network and signature analysis. Sci. Rep. 14:5042. doi: 10.1038/s41598-024-55806-y

PubMed Abstract | Crossref Full Text | Google Scholar

Wang, Y., Li, J., Zhang, W., Liu, S., Miao, L., Li, Z., et al. (2023). Extending the dosing interval of COVID-19 vaccination leads to higher rates of seroconversion in people living with HIV. Front. Immunol. 14:1152695. doi: 10.3389/fimmu.2023.1152695

PubMed Abstract | Crossref Full Text | Google Scholar

Wang, Y., Liu, S., Zhang, W., Zheng, L., Li, E., Zhu, M., et al. (2024). Development and evaluation of a nomogram for predicting the outcome of immune reconstitution among HIV/AIDS patients receiving antiretroviral therapy in China. Adv. Biol. 8:e2300378. doi: 10.1002/adbi.202300378

PubMed Abstract | Crossref Full Text | Google Scholar

Wu, G., Zhou, C., Zhang, X., Zhang, W., Lu, R., Ouyang, L., et al. (2019). Higher risks of Virologic failure and all-cause deaths among older people living with HIV in Chongqing, China. AIDS Res. Hum. Retrovir. 35, 1095–1102. doi: 10.1089/AID.2019.0096

PubMed Abstract | Crossref Full Text | Google Scholar

Yang, Z., Gong, D., Huang, F., Sun, Y., and Hu, Q. (2022). Epidemiological characteristics and the development of prognostic nomograms of patients with HIV-associated cutaneous T-cell lymphoma. Front. Oncol. 12:847710. doi: 10.3389/fonc.2022.847710

PubMed Abstract | Crossref Full Text | Google Scholar

Yu, Y., Wang, X., Li, M., Gu, L., Xie, Z., Gu, W., et al. (2020). Nomogram to identify severe coronavirus disease 2019 (COVID-19) based on initial clinical and CT characteristics: a multi-center study. BMC Med. Imaging 20:111. doi: 10.1186/s12880-020-00513-z

PubMed Abstract | Crossref Full Text | Google Scholar

Yuan, S., Pan, Y., Xia, Y., Zhang, Y., Chen, J., Zheng, W., et al. (2021). Development and validation of an individualized nomogram for early prediction of the duration of SARS-CoV-2 shedding in COVID-19 patients with non-severe disease. J. Zhejiang Univ. Sci. B 22, 318–329. doi: 10.1631/jzus.B2000608

PubMed Abstract | Crossref Full Text | Google Scholar

Yuan, Z., Zhou, B., Meng, S., Jiang, J., Huang, S., Lu, X., et al. (2020). Development and external-validation of a nomogram for predicting the survival of hospitalised HIV/AIDS patients based on a large study cohort in western China. Epidemiol. Infect. 148:e84. doi: 10.1017/S0950268820000758

PubMed Abstract | Crossref Full Text | Google Scholar

Zhang, T., Ding, H., An, M., Wang, X., Tian, W., Zhao, B., et al. (2020). Factors associated with high-risk low-level viremia leading to virologic failure: 16-year retrospective study of a Chinese antiretroviral therapy cohort. BMC Infect. Dis. 20:147. doi: 10.1186/s12879-020-4837-y

PubMed Abstract | Crossref Full Text | Google Scholar

Zhang, W., Liu, S., Miao, L., Fu, A., Bao, J., Zheng, L., et al. (2023). Dynamics of CD4⁺ T-cells and neutralizing antibody responses to three consecutive doses of inactivated COVID-19 vaccines in PLWH. Infect. Drug Resist. 16, 2695–2707. doi: 10.2147/IDR.S409147

PubMed Abstract | Crossref Full Text | Google Scholar