Risk assessment and prediction model of renal damage in childhood immunoglobulin A vasculitis

Objectives: To explore the risk factors for renal damage in childhood immunoglobulin A vasculitis (IgAV) within 6 months and construct a clinical model for individual risk prediction.

Methods: We retrospectively analyzed the clinical data of 1,007 children in our hospital and 287 children in other hospitals who were diagnosed with IgAV. Approximately 70% of the cases in our hospital were randomly selected using statistical product service soltions (SPSS) software for modeling. The remaining 30% of the cases were selected for internal verification, and the other hospital's cases were reviewed for external verification. A clinical prediction model for renal damage in children with IgAV was constructed by analyzing the modeling data through single-factor and multiple-factor logistic regression analyses. Then, we assessed and verified the degree of discrimination, calibration and clinical usefulness of the model. Finally, the prediction model was rendered in the form of a nomogram.

Results: Age, persistent cutaneous purpura, erythrocyte distribution width, complement C₃, immunoglobulin G and triglycerides were independent influencing factors of renal damage in IgAV. Based on these factors, the area under the curve (AUC) for the prediction model was 0.772; the calibration curve did not significantly deviate from the ideal curve; and the clinical decision curve was higher than two extreme lines when the prediction probability was ~15–82%. When the internal and external verification datasets were applied to the prediction model, the AUC was 0.729 and 0.750, respectively, and the Z test was compared with the modeling AUC, P > 0.05. The calibration curves fluctuated around the ideal curve, and the clinical decision curve was higher than two extreme lines when the prediction probability was 25~84% and 14~73%, respectively.

Conclusion: The prediction model has a good degree of discrimination, calibration and clinical usefulness. Either the internal or external verification has better clinical efficacy, indicating that the model has repeatability and portability.

Clinical trial registration: www.chictr.org.cn, identifier ChiCTR2000033435.

Introduction

Immunoglobulin A vasculitis (IgAV), formerly called the Henoch-Schönlein purpura (HSP), is characterized by the deposition of immune complexes on the walls of small vessels and is the most common type III hypersensitivity-mediated vasculitis in children (1). The clinical manifestations of IgAV are nonthrombocytopenic purpura, abdominal pain, gastrointestinal bleeding, arthritis and renal damage. It has been reported that the annual incidence of IgAV is ~3-26.7/100,000 (2), of which 30% to 50% of cases result in renal damage (2–4), that is, IgAV with nephritis (IgAVN). The clinical manifestations of IgAVN are hematuria, proteinuria, nephrotic syndrome and renal dysfunction, which usually occur within 6 months of the course of IgAV (3, 4). Most patients generally have a good prognosis, but repeated illness and a prolonged course of disease can lead to end-stage renal disease, which occurs in ~1 to 10% of children (5, 6), creating huge economic, physical and mental burdens to patients and their families. At present, almost all clinical studies on IgAV focus on the risk factors for IgAVN, and there is still a lack of quantitative tools for individualized risk prediction and benefit assessment of renal damage in children with IgAV. The purpose of this paper was to explore the risk factors for IgAVN, construct a clinical prediction model, and evaluate its clinical efficacy. Finally, the prediction model is shown in the form of a nomogram to provide a scientific quantitative tool for individual risk prediction and assessment of benefits for patients.

Materials and methods

Research subjects

Subjects enrolled in the study were hospitalized in Hunan Children's Hospital from January 2018 to December 2018 and in Hunan Provincial People's Hospital, Changsha Central Hospital and Changsha First People's Hospital from January 2018 to December 2019. The patients were clinically diagnosed with IgAV (7) and met the following conditions: ➀ age <18 years; ➁ follow-up time≥6 months; ➂ no history of lupus nephritis, poststreptococcal nephritis, IgA nephropathy, hepatitis-associated glomerulonephritis and other renal diseases; and ➃ no history of rheumatic connective tissue diseases, renal damage, tumors and other diseases. This study was approved by the Ethics Committee of Hunan Children's Hospital (ethical batch number: HCHLL-2020-48) and was registered in the China Clinical Trials Registry (registration number: ChiCTR2000033435).

Research indicators

Indicators are commonly based on previous literature reports, statistically significant indicators, or indicators considered by the investigator to be clinically relevant. In this study, There are 27 indicators included: sex, age, purpura of external skin of the lower limbs, persistent skin purpura and recurrence, abdominalgia, severe colic, gastrointestinal bleeding, joint involvement, white blood cell count, hemoglobin concentration, platelet count, erythrocyte distribution width, C-reactive protein (CRP), erythrocyte sedimentation rate (ESR), complement C3, complement C4, immunoglobulin G (IgG), IgM, IgA, IgE, triglycerides, total cholesterol, low density lipoprotein, albumin, serum creatinine and uric acid. End point indicator: the occurrence of renal damage in children with IgAV. According to EULAR/PRINTO/PRES criteria for Henoch–Schönlein purpura (7), IgAV with nephritis criteria: Proteinuria > 0.3 g/24 h or > 30 mmol/mg of urine albumin/creatinine ratio on a spot morning sample; Hematuria or red blood cell casts: >5 red blood cells/high power feld or red blood cells casts in the urinary sediment or ≥ 2+ on dipstick.

Statistical methods

SPSS 18.0 software, Stata 15.0 software and R 3.5.1 language were used to analyze the clinical data. The measurement data with a normal distribution are expressed as the means ± standard deviation (SD), and comparisons between the two groups were performed using the independent sample t-test. The measurement data of skewness distribution are expressed as the medians and interquartile ranges (IQR), and the Mann–Whitney U-test was used to determine differences between groups. The counting data are expressed as frequencies (percentages), and the chi-square test was used for the analysis of counting data. Univariate analysis and multifactor logistic regression analysis were used to construct a clinical prediction model for the data of the modeling group. The discrimination, calibration and clinical usefulness of the model were evaluated using the AUC, calibration curve and clinical decision curve (DCA). Internal and external verification datasets were used to further verify the clinical effectiveness of the model, which was finally shown in the form of a nomogram. P < 0.05 were considered statistically significant.

Results

General information

After excluding the cases of loss to follow-up and missing clinical data more than 20% (Figure 1), a total of 1,007 cases were included from Hunan Children's Hospital, including 579 (57.5%) males and 428 (42.5%) females. The median (IQR) age at disease onset was 7.20 (5.64, 9.52) years. A total of 715 cases were randomly selected using SPSS software as a modeling set to establish a clinical prediction model, and the remaining 291 cases were used as the internal verification set. A total of 287 cases, including 155 males (54.0%) and 131 females (46.0%) with a median age of 6.88 (5.16, 9.13) years were included in Hunan Provincial People's Hospital, Changsha Central Hospital and Changsha First People's Hospital as the external verification set. A total of 335 (46.9%) children with IgAV had renal damage. There were 335 (46.9%) cases with IgAVN in the modeling set, 137 cases (46.9%) in the internal verification set and 87 cases (30.3%) in the external verification set.

FIGURE 1

Figure 1. Flowchart of included/excluded cases for studies.

Risk factors

Among the 1,007 cases from Hunan Children's Hospital, the occurrence of renal damage within 6 months of IgAV was the dependent variable, and 27 observation indicators of the study subjects were the independent variables. The results of univariate analysis showed that 15 indicators were statistically significant (Table 1). In this study, 11 of the 15 indicators were diagnosed by collinearity, and the results showed that the tolerance of all variables was close to 1 and variance inflation factor (VIF) <10, indicating that there was no collinearity, and the indicators could be included in logistic regression analysis. Fifteen statistically significant indicators from the univariate analysis were included in the multivariate logistic analysis (Table 2); age, persistent cutaneous purpura, erythrocyte distribution width, complement C₃, immunoglobulin G and triglycerides were independent influencing factors of IgAVN.

TABLE 1

Table 1. Univariate analysis of observation indicators in 1,007 children with IgAV.

TABLE 2

Table 2. Multivariate analysis of 1,007 children with IgAV.

Establishment of a model

The modeling set establishes the multiple regression equation of the prediction model according to the statistically significant variables and their regression coefficients in the multivariate analysis. P is the probability of occurrence of IgAVN with a value range of 0–1. The closer the P value is 1, the greater the probability, and the closer the P value is 0, the smaller the probability. A multivariate logistic regression model was established as follows: LogitP = −3.530 + 0.181X₁ + 0.852X₂ + 0.280X₃ −1.551X₄ −0.123X₅ + 0.676X₆.

Note: X₁: age, X₂: persistent skin purpura, X₃: erythrocyte distribution width, X₄: complement C3, X₅: immunoglobulin G, X₆: triglyceride.

Evaluation and verification of the model

Discrimination

In this study, the prediction probability of IgAVN in the modeling set was expressed by P-m. According to the occurrence of the prediction probability P-m and the actual IgAVN in the modeling set, the ROC curve of the P-m value was constructed using Stata15.0 software (Figure 2), and the discrimination of the IgAVN prediction model was evaluated using the AUC. The ROC curve showed that the AUC of the P-m value in the IgAVN model was 0.772, and the 95% confidence interval was 0.738~0.807. We divided the IgAVN model into the internal verification set and the external verification set, calculated the prediction probability P-in value and P-ex value, respectively, and constructed the ROC curve for these two sets (Figures 3, 4). The AUC of the internal verification set was 0.729, and the 95% CI was 0.671~0.786; the AUC of the external verification set was 0.750, and the 95% CI was 0.688~0.813. There was no significant difference between the model set and two verification sets according to the results of the Z-test used to compare the AUCs.

FIGURE 2

Figure 2. Model set.

FIGURE 3

Figure 3. Internal verification set.

FIGURE 4

Figure 4. External verification set.

Calibration

The calibration curve of the modeling set was constructed using R3.5.1 software (Figure 5); the x-axis represents the prediction probability of IgAVN, and the y-axis represents the actual probability of IgAVN. The slope of Ideal (diagonal) was 1, which represents the ideal curve; Apparent represents the uncalibrated prediction curve and Bias-corrected represents the calibration prediction curve. The chart shows that the uncalibrated prediction curve and the calibration prediction curve fluctuated around the diagonal and did not significantly deviate from the ideal curve. The calibration curve of the internal verification set (Figure 6) and the external verification set (Figure 7) show that the uncalibrated prediction curve and the calibration prediction curve fluctuated around the diagonal and did not significantly deviate from the ideal curve. The goodness of fit for all three groups was assessed using the Hosmer–Lemeshow test, and the difference was not statistically significant.

FIGURE 5

Figure 5. Calibration curve for the model set.

FIGURE 6

Figure 6. Calibration curve for the internal verification set.

FIGURE 7

Figure 7. Calibration curve for the external verification set.

Clinical usefulness

The DCA chart was constructed using Stata15.0 software (Figure 8). In the DCA diagram, the red horizontal line indicates that when the prediction model in this study lacked renal damage, the clinical net benefit was zero; the blue slash indicates that when all IgAV cases had renal damage, the clinical net benefit was a slope that was negative; and the green curve was based on the curve associated with the IgAVN model in this study. As we can see from Figure 7, when the prediction probability P-m was between 15 and 82%, the green curve was higher than the two extreme lines, and the model can benefit from predicting renal damage in IgAV. The DCA diagram of the internal verification set and the external verification set was constructed according to the IgAV renal damage prediction probability P-in value and P-ex value and the actual IgAV renal damage occurrence situation in these sets (Figures 9, 10). It can be concluded from the two figures that when the predicted probability of P-in was ~25~84% and P-ex was ~14%~73%, the green decision curve was higher than the blue slash line and red cross line in extreme cases, suggesting that children can benefit from the model when predicting renal damage in IgAV within the range of prediction probability.

FIGURE 8

Figure 8. DCA of model set.

FIGURE 9

Figure 9. DCA of internal verification set.

FIGURE 10

Figure 10. DCA of external verification set.

Model presentation

The nomogram of the IgAVN prediction model was constructed using the rms package of R3.5.1 software (see Figure 11). The nomogram converts the regression equation into an easy-to-understand visual graph, which makes the results of the prediction model more readable and convenient for the risk assessment of patients. In Figure 10, X1–X6 represent the independent variables, and each variable finds the point on the axis corresponding to the line chart and indicates the score of the variable. The total score of each variable adds up to the total score of the corresponding total score scale, and the final total score corresponds to the reading on the risk axis of renal damage, which indicates the probability of renal damage in the child.

FIGURE 11

Figure 11. Nomogram of the prediction model. X₁, age; X₂, persistent skin purpura; X₃, erythrocyte distribution width; X₄, complement C3; X₅, immunoglobulin G; X₆, triglyceride.

Discussion

IgAV is the most common small vessel inflammatory disease in childhood and worldwide and mainly involves the small vasculature of the skin, joints, gastrointestinal tract and kidneys (8). The clinical manifestations of IgAVN are mainly hematuria and proteinuria, which long-term prognosis depends on the severity of renal damage (9). The influencing factors of IgAVN are complex; a large number of studies have shown that the older the age of the child, the greater the risk of renal damage, which is most likely due to the increase in immune function with age. A systematic review of 13 studies from the year 2000 to 2016 (10) (2,398 children) showed that males older than 10 years of age, gastrointestinal symptoms such as abdominal pain and bleeding, joint involvement, persistent skin purpura, recurrent rash, elevated white blood cells, elevated platelets and decreased complement C₃ can increase the risk of renal involvement. Some studies have shown that erythrocyte distribution width can be used as a marker of IgAV risk and may be related to the severity of the disease (11). At present, it is controversial whether immunoglobulin G is related to IgAVN. In a retrospective analysis of 250 children with IgAV, elevated cholesterol and low-density lipoprotein was identified as an independent risk factor for IgAVN (12). In this paper, the children with IgAV were followed up for half a year, and renal damage was the end point index. Twenty-seven potential predictive indicators, such as age, clinical manifestations, and serological indices were used in univariate analysis and logistic multivariate analysis The results showed that age, persistent purpura, erythrocyte distribution width, complement C₃, immunoglobulin G and triglycerides were considered to be independent influencing factors for IgAVN, which is similar to the findings of previous studies on the prediction risk factors for IgAVN.

The construction of a clinical predictive model based on risk factors is a popular and challenging endeavor in current research. A prediction model is a tool to obtain risk or probability by assigning relative weights to each prediction factor to combine multiple prediction factors (13). Clinically, the establishment of a predictive model is one of the important methods used to transform clinical research into clinical application. However, the establishment of a good clinical prediction model is a complex project. The “Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis” (TRIPOD) (14) published in 2015 has made a relatively complete behavior norm for the establishment of the prediction model, including the construction, evaluation, and verification of the model.

Building a model involves the selection of possible predictors and combining them into a multivariate model, which is usually analyzed by logistic regression or Cox regression. Logistic regression analysis is most commonly used to predict binary events, while Cox regression is often used for long-term prognostic results (15). This paper is a retrospective study, and the content of the study is the risk prediction of renal damage in IgAV. The end point is whether renal damage occurs in IgAV within 6 months of follow-up; thus, it is considered a binary outcome index. Therefore, logistic regression was selected to analyze the predictive factors for IgAVN.

The evaluation model includes the evaluation of discrimination, calibration and clinical usefulness. The degree of discrimination refers to the ability of the prediction model to distinguish between terminal events and nonterminal events; the AUC is generally used to quantify logistic models (14). The closer the AUC is to the value of 1, the better the discrimination of the model, and clinically, the model has a better degree of discrimination when 0.7 < AUC <0.9 (16). The accuracy reflects the degree to which the model correctly estimates the absolute risk; that is, whether the predicted value of the model is consistent with the actual value (16, 17). The positional relationship between the calibration curve and the ideal curve and the Hosmer–Lemeshow test are usually used for evaluation (18). Clinical practicality refers to the clinical net benefit of the patient using the prediction model at a certain threshold probability, which is evaluated by DCA (19). DCA obtains the net benefit value of using the model on this threshold by determining the relationship between the selected predictive probability threshold and the relative values of false-positive and false-negative results (20, 21). The net return is calculated by all possible risk thresholds between the two extremes, that is, zero and maximum risk estimates, representing all negative events and all positive events, respectively (22). If the DCA is higher than two extreme lines, it indicates that the patient can benefit, and the clinical practicability is better (23). In this paper, the AUC of the IgAVN model based on the logistic regression equation was 0.772 (0.738–0.807), suggesting that the renal damage model has a good degree of discrimination. The Hosmer–Lemeshow test of the model, P > 0.05, and the calibration curve around the ideal curve in Figure 2 all indicate that the renal damage model has good accuracy. According to the IgAVN model, the DCA is higher than two extreme lines when the value of P-m is ~15~82%, indicating beneficial effects for the child. This shows that the IgAVN model established in this study has a good degree of discrimination, calibration and clinical usefulness. The model is helpful to effectively identify the risk of IgAVN in the clinic and can guide regular revisits and timely treatment of high-risk children with IgAV.

However, the IgAVN model needs to be validated in other populations before it is used in clinical practice. Model verification is not simply repeating the analysis steps established by the model in other individuals to see if they have the same prediction factors and weights but involves application of the new individuals to the established model. According to the prediction factors in the model and the weight assigned (such as the regression coefficient), the predicted value of the new individual is estimated, and the prediction performance of the model is quantified (24–26). The verification of the model includes internal verification and external verification; internal verification reflects the repeatability of the model, and external verification reflects the portability of the model. In this paper, the split sample verification method was used to select the clinical data of children with 70% IgAV in our hospital as the modeling set and the clinical data of children with 30% IgAV as the internal verification set. The clinical data of children with IgAV in other medical centers were used as the external verification set. In these two sets of data, the two types of verification are related to the discrimination, calibration and clinical usefulness of the evaluation model. In the discrimination evaluation, the AUC of the internal verification set was 0.729 (0.671–0.786), and the AUC of the external verification set was 0.750 (0.688–0.813). The AUC of these verification sets was smaller than that of the modeling set, indicating that the discrimination of the model decreased in the smaller dataset. However, Z test results showed no difference in the AUC between the two sets and the modeling set. The results showed that the prediction model still had good discrimination in the verification data. In the evaluation of calibration, the calibration curves for both the internal and external verification sets fluctuated above and below the ideal curve, and the Hosmer–Lemeshow test showed a P > 0.05. This shows that the application of the IgAVN model in these two groups of data has good calibration. In the evaluation of clinical usefulness, the DCA curves drawn by the internal and external verification groups were higher than two extreme lines when the P-in value was 25–84% and the P-ex value was 14–73%, respectively, indicating that the prediction model has good clinical usefulness. Therefore, the IgAVN model has good discrimination, calibration and clinical usefulness in both the internal verification group and the external verification group, indicating that the renal damage model shows good repeatability and portability.

In addition, to facilitate the renal damage model in the evaluation of patients, the most commonly used method is to visualize the IgAVN model. At present, the model is usually presented as a nomogram in the research of clinical prediction models. The nomogram has the ability to personalize the prediction, enabling it to identify and assess the risk for each patient (27). In this paper, the nomogram of the IgAVN model is constructed using the rms software package of R statistics software, and the relatively complex regression equation is transformed into a simple visual graph, which is helpful in making effective clinical decisions when the renal damage model is widely used in the clinic. As we can see from the nomogram in this study, in a child with IgAV aged 8 years and 2 months with skin purpura for more than 1 month, red blood cell distribution width 13.6%, complement C₃ 0.96 g/L, immunoglobulin G 6.92 g/L and triglycerides 1.88 mmol/L, the probability of renal damage is ~82%. It is suggested that the risk of renal damage is high, and clinicians and family members should remain vigilant and have more frequent visits for reexamination if necessary.

In summary, through univariate and multivariate logistic regression analyses, it was concluded that older age, persistent purpura ≥ 1 month, increased erythrocyte distribution width, decreased complement C₃, decreased immunoglobulin G and elevated triglycerides are independent influencing factors of IgAVN. The IgAVN model based on binary logistic regression has good clinical predictive ability and clinical practicability through the evaluation of clinical efficacy in terms of discrimination, calibration and clinical usefulness. Additionally, internal and external verification of the model revealed good clinical repeatability and portability. The model is presented as a line chart, which provides an effective reference basis for the individual risk assessment of patients, contributes to early warning of potential IgAVN in patients, and maximizes the clinical benefits in children.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by the Institutional Review Board of the Hunan Children's Hospital.

Author contributions

RF, ZL, and ZK contributed to conception and design of the study. MY, MX, YW, MW, and XW provided and organized the data. RF wrote the first draft of the manuscript. ZL reviewed sections of the manuscript. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by Scientific Research Project of Health Commission of Hunan Province in China (202106012359).

Acknowledgments

The authors thank all colleagues who processed medical records, managed patient and assisted with follow-up. We would like to express our gratitude to the patient and their family members, who made this study possible. The authors thank AJE Academic Services (www.aje.cn) for English language editing and review services.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Heineke MH, Ballering AV, Jamin A, Ben Mkaddem S, Monteiro RC, Van Egmond M. New insights in the pathogenesis of immunoglobulin A vasculitis (Henoch-Schönlein purpura). Autoimmun Rev. (2017) 16:1246–53. doi: 10.1016/j.autrev.2017.10.009

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Piram M, Mahr A. Epidemiology of immunoglobulin A vasculitis (Henoch-Schönlein): current state of knowledge. Curr Opin Rheumatol. (2013) 25:171–8. doi: 10.1097/BOR.0b013e32835d8e2a

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Karadag SG, Tanatar A, Sönmez HE, Çakmak F, Kiyak A, Yavuz S, et al. The clinical spectrum of Henoch-Schönlein purpura in children: a single-center study. Clin Rheumatol. (2019) 38:1707–14. doi: 10.1007/s10067-019-04460-1

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Wang K, Sun X, Cao Y, Dai L, Sun F, Yu P, et al. Risk factors for renal involvement and severe kidney disease in 2731 Chinese children with Henoch-Schönlein purpura: A retrospective study. Medicine (Baltimore). (2018) 97:e12520. doi: 10.1097/MD.0000000000012520

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Spasojević-Dimitrijeva B, Kostić M, Peco-Antić A, Kruscić D, Cvetković M, Milosevski-Lomić G, et al. Henoch-Schönlein purpura outcome in children: a ten-year clinical study. Srp Arh Celok Lek. (2011) 139:174–8. doi: 10.2298/SARH1104174S

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Delbet JD, Geslain G, Auger M, Hogan J, Salomon R, Peuchmaur M, et al. Histological prognostic factors in children with Henoch-Schönlein purpura nephritis. Pediatr Nephrol. (2020) 35:313–20. doi: 10.1007/s00467-019-04363-y

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Ozen S, Pistorio A, Iusan SM, Bakkaloglu A, Herlin T, Brik R, et al. EULAR/PRINTO/PRES criteria for Henoch-Schönlein purpura, childhood polyarteritis nodosa, childhood Wegener granulomatosis and childhood Takayasu arteritis: Ankara 2008. Part II: Final classification criteria. Ann Rheum Dis. (2010) 69:798–806. doi: 10.1136/ard.2009.116657

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Song Y, Huang X, Yu G, Qiao J, Cheng J, Wu J, et al. Pathogenesis of IgA vasculitis: an up-to-date review. Front Immunol. (2021) 12:771619. doi: 10.3389/fimmu.2021.771619

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Çakici EK, Gür G, Yazilitaş F, Eroglu FK, Güngör T, Arda N, et al. A retrospective analysis of children with Henoch-Schonlein purpura and re-evaluation of renal pathologies using Oxford classification. Clin Exp Nephrol. (2019) 23:939–47. doi: 10.1007/s10157-019-01726-5

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Chan H, Tang YL, Lv XH, Zhang GF, Wang M, Yang HP, et al. Risk factors associated with renal involvement in childhood henoch-schönlein purpura: a meta-analysis. PLoS ONE. (2016) 11:e0167346. doi: 10.1371/journal.pone.0167346

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Xu H, Li W, Mao JH, Pan YX. Association between red blood cell distribution width and Henoch-Schonlein purpura nephritis. Medicine (Baltimore). (2017) 96:e7091. doi: 10.1097/MD.0000000000007091

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Ma DQ, Li Y, Han ZG, Zheng M, Xu N, Fan XJ. Analysis on kidney injury-related clinical risk factors and evaluation on the therapeutic effects of hemoperfusion in children with Henoch-Schonlein purpura. Eur Rev Med Pharmacol Sci. (2017) 21:3894–9.

PubMed Abstract | Google Scholar

13. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ. (2015) 350:g7594. doi: 10.1136/bmj.g7594

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Moons KG, Altman DG, Reitsma JB, Ioannidis JP, Macaskill P, Steyerberg EW, et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med. (2015) 162:W1–W73. doi: 10.7326/M14-0698

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Mijderwijk HJ, Beez T, Hänggi D, Nieboer D. Clinical prediction models. Childs Nerv Syst. (2020) 36:895–7. doi: 10.1007/s00381-020-04577-8

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Alba AC, Agoritsas T, Walsh M, Hanna S, Iorio A, Devereaux PJ, et al. Discrimination and calibration of clinical prediction models: users' guides to the medical literature. JAMA. (2017) 318:1377–84. doi: 10.1001/jama.2017.12126

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. (2010) 21:128–38. doi: 10.1097/EDE.0b013e3181c30fb2

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Adibi A, Sadatsafavi M, Ioannidis JPA. Validation and utility testing of clinical prediction models: time to change the approach. JAMA. (2020) 324:235–6. doi: 10.1001/jama.2020.1230

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Steyerberg EW, Vergouwe Y. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur Heart J. (2014) 35:1925–31. doi: 10.1093/eurheartj/ehu207

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Li K, Anderson G, Viallon V, Arveux P, Kvaskoff M, Fournier A, et al. Risk prediction for estrogen receptor-specific breast cancers in two large prospective cohorts. Breast Cancer Res. (2018) 20:147. doi: 10.1186/s13058-018-1073-0

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Hendriksen JM, Geersing GJ, Moons KG, de Groot JA. Diagnostic and prognostic prediction models. J Thromb Haemost. (2013) 11 Suppl 1:129–41. doi: 10.1111/jth.12262

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Vickers AJ, van Calster B, Steyerberg EW. A simple, step-by-step guide to interpreting decision curve analysis. Diagn Progn Res. (2019) 3:18. doi: 10.1186/s41512-019-0064-7

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Zeng X, Zhang Y, Kwong JS, Zhang C, Li S, Sun F, et al. The methodological quality assessment tools for preclinical and clinical studies, systematic review and meta-analysis, and clinical practice guideline: a systematic review. J Evid Based Med. (2015) 8:2–10. doi: 10.1111/jebm.12141

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Sharabiani M, Clementel E, Andratschke N, Hurkmans C. Generalizability assessment of head and neck cancer NTCP models based on the TRIPOD criteria. Radiother Oncol. (2020) 146:143–50. doi: 10.1016/j.radonc.2020.02.013

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Su TL, Jaki T, Hickey GL, Buchan I, Sperrin M. A review of statistical updating methods for clinical prediction models. Stat Methods Med Res. (2018) 27:185–97. doi: 10.1177/0962280215626466

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Ratna MB, Bhattacharya S, Abdulrahim B, McLernon DJ. A systematic review of the quality of clinical prediction models in in vitro fertilisation. Hum Reprod. (2020) 35:100–16. doi: 10.1093/humrep/dez258

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Park SY. Nomogram: an analogue tool to deliver digital knowledge. J Thorac Cardiovasc Surg. (2018) 155:1793. doi: 10.1016/j.jtcvs.2017.12.107

PubMed Abstract | CrossRef Full Text | Google Scholar