Skip to main content

SYSTEMATIC REVIEW article

Front. Med., 14 June 2021
Sec. Intensive Care Medicine and Anesthesiology
This article is part of the Research Topic Anesthetic and Critical Care Amid the COVID-19 Pandemic View all 40 articles

The Prognostic Value of Hyponatremia for Predicting Poor Outcome in Patients With COVID-19: A Systematic Review and Meta-Analysis

  • 1Department of Cardiology and Vascular Medicine, Faculty of Medicine Universitas Padjadjaran, Rumah Sakit Umum Pusat Hasan Sadikin, Bandung, Indonesia
  • 2Faculty of Medicine, Universitas Pelita Harapan, Tangerang, Indonesia

Background: This meta-analysis aimed to assess the prognostic value of hyponatremia in patients with COVID-19.

Methods: We performed a systematic literature search on PubMed, Scopus, ScienceDirect, and Wiley up until January 26, 2021. The key exposure was hyponatremia, defined as sodium level below the reference level. The outcome of interest was poor outcome, which was a composite of mortality, severe COVID-19, and prolonged hospitalization. Severe COVID-19 was defined severe CAP or needing ICU care or IMV. The pooled effect estimate was odds ratio (OR). Sensitivity, specificity, positive and negative likelihood ratio (PLR and NLR), diagnostic OR (DOR), and area under curve (AUC) were generated.

Results: There were 11,493 patients from eight studies included in this systematic review and meta-analysis. The incidence of hyponatremia was 24%, and incidence of poor outcome was 20%. Hyponatremia was associated with poor outcome in COVID-19 (OR 2.65 [1.89, 3.72], p < 0.001; I2: 67.2%). Meta-regression analysis showed that the association between hyponatremia and poor outcome was reduced by age (OR 0.94 [0.90, 0.98], p = 0.006) and hypertension (OR 0.96 [0.93, 0.94], p < 0.001). Hyponatremia has a sensitivity of 0.37 [0.27, 0.48], specificity of 0.82 [0.72, 0.88], PLR of 2.0 [1.5, 2.7], NLR of 0.77 [0.69, 0.87], DOR of 3 [2, 4], and AUC of 0.62 [0.58, 0.66] for predicting poor outcome. In this pooled analysis, hyponatremia has a 33% posttest probability for poor outcome, and absence of hyponatremia confers to a 16% posttest probability.

Conclusion: Hyponatremia was associated with poor outcome in patients with COVID-19.

Systematic Review Registration: PROSPERO, CRD42021233592.

Introduction

Coronavirus disease 2019 (COVID-19) is now one of the most common diseases in the world (1). Although most of the patients have mild–moderate symptoms, a significant number of patients develop organ failure and require critical care (2). Identifying patients that require a more intense monitoring is of paramount importance due to limitation of human and medical resources, especially in the developing countries. Patients' characteristics and laboratory features proved to be helpful for this purpose (36).

COVID-19 may develop pneumonia and lead to sepsis that produces high levels of inflammatory cytokines (7, 8). Interleukin (IL)-1β, IL-6, and tumor necrosis factor (TNF) play a pivotal role in the development of sepsis. IL-6 has an inverse correlation with serum sodium which has a mechanism to stimulate hypothalamic arginine vasopressin (912). Several studies showed that pneumonia is associated with syndrome of inappropriate antidiuretic hormone (11, 13, 14); this mechanism may cause hyponatremia. Hyponatremia is related with prolonged time in the ICU and increased mortality rate in sepsis, cardiovascular disease, and chronic kidney disease (11, 15). Hyponatremia has been shown to increase mortality in patients with COVID-19. However, the prognostic value of this finding is still unclear. Several laboratory abnormalities have been shown to indicate poor prognosis; however, it does not directly translate to clinical value. This is the first systematic review and meta-analysis to address whether hyponatremia is a reliable biomarker. This meta-analysis aimed to assess the prognostic value of hyponatremia in patients with COVID-19.

Materials and Methods

This is a Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines compliant meta-analysis. This meta-analysis is registered in PROSPERO (CRD42021233592).

Eligibility Criteria

We included studies that fulfill all of these criteria: (1) observational retrospective and prospective studies, (2) COVID-19 patients, (3) hyponatremia and normonatremia, and (4) mortality/severe COVID-19/prolonged hospitalization/need for intensive unit care (ICU)/invasive mechanical ventilation (IMV).

We excluded the following studies: (1) preprints, (2) case reports, (3) conference abstracts, (4) review articles, and (5) abstract-only publication. Preprints were excluded to reduce bias. Conference abstracts or abstract-only publications were excluded because they often did not contain sufficient information and the analyses were only described in brief.

Search Strategy and Study Selection

We performed a systematic literature search on PubMed, Scopus, ScienceDirect, and Wiley with keywords (“COVID-19” OR “Coronavirus Disease” OR “SARS-COV2”) AND (Hyponatremia OR Hyponatremic OR “Low Sodium”) up until January 26, 2021. We removed the duplicates and screened the title/abstract of the records. This process was performed by two independent authors, and discrepancies were resolved by discussion.

Data Extraction

Data extraction of the included studies was performed by two authors (R.P and I.I) independently. Both authors are medical doctors experienced in conducting systematic review and meta-analysis. The data of interest includes the author, design of the study, baseline characteristics, and outcome of interest. Discrepancies were resolved by discussion.

We made unsuccessful attempts to contact authors for studies that do not sufficiently report their data required for our analysis.

Exposure and Outcome

The key exposure was hyponatremia, defined as sodium level below the reference level for eunatremia. The outcome of interest was poor outcome, which was a composite of mortality, severe COVID-19, and prolonged hospitalization. Severe COVID-19 was defined severe pneumonia or needing ICU care or IMV. The pooled effect estimate was odds ratio (OR). Sensitivity, specificity, positive and negative likelihood ratio (PLR and NLR), diagnostic odds ratio (DOR), and area under the curve (AUC) were generated.

Risk of Bias Assessment

Risk of bias assessment was performed by two authors independently, using the Newcastle–Ottawa Scale (NOS) tool (16). There are three domains in NOS: (1) selection (representativeness, selection of comparator, and ascertainment of exposure), (2) comparability (outcome of interest was not present at the start and the two groups were comparable), and (3) outcome (independency of outcome, adequacy of follow-up, and lost to follow-up). Discrepancies were resolved by discussion.

Statistical Analysis

The incidence of hyponatremia and poor outcome was pooled using the meta-analysis of proportion. The random-effects DerSimonian–Laird method was used to generate the pooled effect estimate in form of OR and its 95% CI. P-values <0.05 were considered as statistically significant. To evaluate heterogeneity of the pooled analysis, we performed the I-squared (I2) and Cochran Q tests, in which a value of <50% or p < 0.10 indicates significant heterogeneity. Sensitivity, specificity, PLR, NLR, DOR, and AUC were generated to evaluate the prognostic value of hyponatremia. To assess small-study effects and publication bias, funnel-plot analysis and Egger's were performed. Trim-and-fill analysis using the linear L0 estimator was performed due to asymmetrical funnel plot. Restricted-maximum likelihood (REML) meta-regression was performed for the association between hyponatremia and poor outcome, using age, male (gender), hypertension, diabetes, and chronic kidney disease as covariates. We performed sensitivity analysis by removing non-cohort studies. We used STATA 16 (StataCorp) to perform meta-analysis.

Results

There were 11,493 patients from eight studies included in this systematic review and meta-analysis (Figure 1) (1724). Table 1 displays baseline characteristics of the included studies. The overall risk of bias as assessed with NOS was moderate. The incidence of hyponatremia was 24% [16%−31%]. The incidence of poor outcome was 20% [14%−35%].

FIGURE 1
www.frontiersin.org

Figure 1. PRISMA Flowchart.

TABLE 1
www.frontiersin.org

Table 1. Baseline characteristics of the included studies.

Hyponatremia was associated with poor outcome in COVID-19 (OR 2.65 [1.89, 3.72], p < 0.001; I2: 67.2%, p = 0.003) (Figure 2). The funnel plot was asymmetrical (Figure 3A), and trim-and-fill analysis resulted in an OR of 2.65 [1.89, 3.72] (Figure 3B). There is no indication for small-study effects (p = 0.231). Meta-regression analysis showed that the association between hyponatremia and poor outcome was reduced by age (OR 0.94 [0.90, 0.98], p = 0.006) and hypertension (OR 0.96 [0.93, 0.94], p < 0.001), but not male (gender) (p = 0.532), diabetes (p = 0.308), and chronic kidney disease (p = 0.177).

FIGURE 2
www.frontiersin.org

Figure 2. Hyponatremia and poor outcome.

FIGURE 3
www.frontiersin.org

Figure 3. Publication bias. Funnel-plot analysis (A) and trim-and-fill analysis (B).

Hyponatremia has a sensitivity of 0.37 [0.27, 0.48], specificity of 0.82 [0.72, 0.88], PLR of 2.0 [1.5, 2.7], NLR of 0.77 [0.69, 0.87], DOR of 3 (2, 4), and AUC of 0.62 [0.58, 0.66] for predicting poor outcome (Figure 4). In this pooled analysis, hyponatremia has a 33% posttest probability for poor outcome, and absence of hyponatremia confers to a 16% posttest probability (Figure 5). Meta-regression and subgroup analysis showed that the sensitivity and specificity did not vary by age, male (gender), hypertension, diabetes, and chronic kidney disease (Figure 6).

FIGURE 4
www.frontiersin.org

Figure 4. SROC curve.

FIGURE 5
www.frontiersin.org

Figure 5. Fagan's nomogram.

FIGURE 6
www.frontiersin.org

Figure 6. Meta-regression analysis.

Sensitivity analysis by removing non-cohort studies showed that hyponatremia was associated with poor outcome (OR 2.88 [1.95, 4.27], p < 0.001; I2: 73.2%, p = 0.002), with sensitivity of 0.35 [0.24, 0.48], specificity of 0.84 [0.75, 0.90], PLR of 2.1 [1.5, 3.0], NLR of 0.77 [0.67, 0.87], DOR of 3 (2, 4), and AUC of 0.64 [0.60, 0.69].

Discussion

Hyponatremia was associated with poor outcome in patients with COVID-19 with a 37% sensitivity and 82% specificity. There was an indication of publication bias as shown by the funnel-plot analysis; however, after trim-and-fill analysis, the result was still significant.

All of the studies except Sarvazad et al. demonstrate a significantly elevated risk of poor outcome in patients with hyponatremia. The study has a small sample size and contributes to the least weight in the pooled analysis. One of the possible explanations for the statistical insignificance is due to the inadequate sample size.

Meta-regression analysis showed that older age and hypertension modify the association, in terms of OR, between hyponatremia, and poor outcome. Both older age and hypertension have been shown to increase mortality in patients (2527). The explanation for this modifying finding is currently unclear. However, neither variable significantly modifies the sensitivity and specificity.

In patients with COVID-19, volume depletion from gastrointestinal fluid losses causes ADH release that leads to hyponatremia (18). Additionally, SIADH is frequently encountered in patients with pneumonia and may contribute to hyponatremia (18, 28). IL-6 levels have been shown to be inversely correlated, and IL-6 itself has been associated with increased severity in patients with COVID-19; thus, hyponatremia may signify cytokine storm (29, 30). Additionally, hyponatremia may cause several neurological complications such as cerebral edema, seizures, and encephalopathy (22). An overly aggressive correction of hyponatremia may lead to central pontine myelinolysis; these factors may contribute to mortality and severity in patients with COVID-19.

Although this meta-analysis showed the association between hyponatremia and poor outcome in COVID-19, it did not necessarily equal to causality. Hyponatremia might be a bystander factor; for example, hyponatremia might only indicate increased IL-6 or other parameters or condition that may actually increase the risk of poor outcome. Patients with hypertension may also experience hyponatremia through diuretics use, and meta-regression analysis indicates that the association between hyponatremia and poor outcome is slightly reduced in this population. This finding may imply that hyponatremia due to the natural history of COVID-19 is more significant than hyponatremia due to other causes, although further investigation is required.

Clinical Implications

Hyponatremia indicates a higher risk for poor outcome in patients with COVID-19; however, due to low sensitivity but high specificity, it can be used to rule in, but not rule out, poor prognosis. Moreover, hyponatremia alone has a poor AUC for predicting poor outcome in patients with COVID-19. Thus, combining hyponatremia with other variables in a prediction model will be more useful rather than using it as a standalone parameter.

Limitations

One limitation of this meta-analysis is publication bias in which there are more positive studies compared to negative studies being published. Most of the studies were retrospective in nature and highly prone to bias. This meta-analysis contained observational studies that either were descriptive or analyze cohorts with a prognostic objective. This meta-analysis established the association between hyponatremia and poor outcome in COVID-19, but did not prove causality. Finally, this meta-analysis provides the assumption that hyponatremia should be a risk factor to be taken into account in future prognostic models that may be constructed when exploring COVID-19 risk factors.

In conclusion, hyponatremia was associated with poor outcome in patients with COVID-19.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author Contributions

MA: conceptualization, investigation, writing—review and editing, and supervision. RP: conceptualization, methodology, software, data curation, formal analysis meta-analysis, investigation, validation, writing—original draft, and writing—review and editing. AW: investigation and writing—original draft. I: data curation, investigation, and writing—original draft. TS and JM: investigation and writing—review and editing. All authors contributed to the article and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Abbreviations

AUC, area under curve; COVID-19, Coronavirus disease 2019; ICU, intensive care unit; IL, interleukin; IMV, invasive mechanical ventilation; NOS, Newcastle–Ottawa Scale; NLR, negative likelihood ratio; OR, odds ratio; PLR, positive likelihood ratio; REML, restricted-maximum likelihood; SIADH, syndrome inappropriate antidiuretic hormone; TNF, tumor necrosis factor.

References

1. WHO. Weekly Epidemiological Update - 2 March 2021. Geneva (2021). Available online at: https://www.who.int/publications/m/item/weekly-epidemiological-update—2-march-2021 (accessed April 27, 2021).

Google Scholar

2. Lim MA, Pranata R, Huang I, Yonas E, Soeroto AY, Supriyadi R. Multiorgan failure with emphasis on acute kidney injury and severity of COVID-19: systematic review and meta-analysis. Can J Kidney Heal Dis. (2020) 7:2054358120938573. doi: 10.1177/2054358120938573

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Pranata R, Supriyadi R, Huang I, Permana H, Lim MA, Yonas E, et al. The association between chronic kidney disease and new onset renal replacement therapy on the outcome of COVID-19 patients: a meta-analysis. Clin Med Insights Circ Respir Pulm Med. (2020) 14:1179548420959165. doi: 10.1177/1179548420959165

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Pranata R, Lim MA, Yonas E, Vania R, Lukito AA, Siswanto BB, et al. Body mass index and outcome in patients with COVID-19: a dose–response meta-analysis. Diabetes Metab. (2021) 47:101178. doi: 10.1016/j.diabet.2020.07.005

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Wibowo A, Pranata R, Astuti A, Tiksnadi BB, Martanto E, Martha JW, et al. Left and right ventricular longitudinal strains are associated with poor outcome in COVID-19: a systematic review and meta-analysis. J Intensive Care. (2021) 9:9. doi: 10.1186/s40560-020-00519-3

PubMed Abstract | CrossRef Full Text | Google Scholar

6. July J, Pranata R. Prevalence of dementia and its impact on mortality in patients with coronavirus disease 2019: a systematic review and meta-analysis. Geriatr Gerontol Int. (2021) 21:172–7. doi: 10.1111/ggi.14107

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Ye Q, Wang B, Mao J. The pathogenesis and treatment of the ‘Cytokine Storm” in COVID-19.' J Infect. (2020) 80:607–13. doi: 10.1016/j.jinf.2020.03.037

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Li H, Liu L, Zhang D, Xu J, Dai H, Tang N, et al. SARS-CoV-2 and viral sepsis: observations and hypotheses. Lancet. (2020) 395:1517–20. doi: 10.1016/S0140-6736(20)30920-X

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Palin K, Moreau ML, Sauvant J, Orcel H, Nadjar A, Duvoid-Guillou A, et al. Interleukin-6 activates arginine vasopressin neurons in the supraoptic nucleus during immune challenge in rats. Am J Physiol Endocrinol Metab. (2009) 296:E1289–99. doi: 10.1152/ajpendo.90489.2008

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Gheorghe G, Ilie M, Bungau S, Stoian AMP, Bacalbasa N, Diaconu CC. Is there a relationship between covid-19 and hyponatremia? Med. (2021) 57:1–7. doi: 10.3390/medicina57010055

CrossRef Full Text | Google Scholar

11. Corona G, Giuliani C, Parenti G, Norello D, Verbalis JG, Forti G, et al. Moderate hyponatremia is associated with increased risk of mortality: evidence from a meta-analysis. PLoS ONE. (2013) 8:e80451. doi: 10.1530/endoabs.32.P864

CrossRef Full Text | Google Scholar

12. Cuesta M, Thompson CJ. The syndrome of inappropriate antidiuresis (SIAD). Best Pract Res Clin Endocrinol Metab. (2016) 30:175–87. doi: 10.1016/j.beem.2016.02.009

CrossRef Full Text | Google Scholar

13. Zilberberg MD, Exuzides A, Spalding J, Foreman A, Jones AG, Colby C, et al. Hyponatremia and hospital outcomes among patients with pneumonia: a retrospective cohort study. BMC Pulm Med. (2008) 8:16. doi: 10.1186/1471-2466-8-16

CrossRef Full Text | Google Scholar

14. Nair V, Niederman MS, Masani N, Fishbane S. Hyponatremia in community-acquired pneumonia. Am J Nephrol. (2007) 27:184–90. doi: 10.1159/000100866

CrossRef Full Text | Google Scholar

15. Sun L, Hou Y, Xiao Q, Du Y. Association of serum sodium and risk of all-cause mortality in patients with chronic kidney disease: a meta-analysis and sysematic review. Sci Rep. (2017) 7:15949. doi: 10.1038/s41598-017-16242-3

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Wells G, Shea B, O'Connell D, Peterson J. The Newcastle-Ottawa Scale (NOS) for assessing the Quality of Nonrandomised Studies in Meta-Analyses. Ottawa: Ottawa Hosp Res Inst (2000). Available online at: http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp (accessed April 27, 2021).

Google Scholar

17. Ruiz-Sánchez JG, Núñez-Gil IJ, Cuesta M, Rubio MA, Maroun-Eid C, Arroyo-Espliguero R, et al. Prognostic impact of hyponatremia and hypernatremia in COVID-19 pneumonia. a HOPE-COVID-19 (Health Outcome Predictive Evaluation for COVID-19) registry analysis. Front Endocrinol (Lausanne). (2020) 11:1–12. doi: 10.3389/fendo.2020.599255

PubMed Abstract | CrossRef Full Text | Google Scholar

18. De Carvalho H, Richard MC, Chouihed T, Goffinet N, Le Bastard Q, Freund Y, et al. Electrolyte imbalance in COVID-19 patients admitted to the Emergency Department: a case–control study. Intern Emerg Med. (2021) 1–6. doi: 10.1007/s11739-021-02632-z. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Zeng QL, Li GM, Ji F, Ma SH, Zhang GF, Xu JH, et al. Clinical course and treatment efficacy of COVID-19 near Hubei Province, China: a multicentre, retrospective study. Transbound Emerg Dis. (2020) 67:2971–82. doi: 10.1111/tbed.13674

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Sarvazad H, Cahngaripour SH, Eskandari Roozbahani N, Izadi B. Evaluation of electrolyte status of sodium, potassium and magnesium, and fasting blood sugar at the initial admission of individuals with COVID-19 without underlying disease in Golestan Hospital, Kermanshah. New Microbes New Infect. (2020) 38:100807. doi: 10.1016/j.nmni.2020.100807

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Hu W, lv X, Li C, Xu Y, Qi Y, Zhang Z, et al. Disorders of sodium balance and its clinical implications in COVID-19 patients: a multicenter retrospective study. Intern Emerg Med. (2020) 1–10. doi: 10.1007/s11739-020-02515-9. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Frontera JA, Valdes E, Huang J, Lewis A, Lord AS, Zhou T, et al. Prevalence and impact of hyponatremia in patients with coronavirus disease 2019 in New York City. Crit Care Med. (2020) 48:e1211–7. doi: 10.1097/CCM.0000000000004605

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Wu Y, Hou B, Liu J, Chen Y, Zhong P. Risk factors associated with long-term hospitalization in patients with COVID-19: a single-centered, retrospective study. Front Med. (2020) 7:1–10. doi: 10.3389/fmed.2020.00315

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Tezcan ME, Dogan Gokce G, Sen N, Zorlutuna Kaymak N, Ozer RS. Baseline electrolyte abnormalities would be related to poor prognosis in hospitalized coronavirus disease 2019 patients. New Microbes New Infect. (2020) 37:100753. doi: 10.1016/j.nmni.2020.100753

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Pranata R, Lim MA, Huang I, Raharjo SB, Lukito AA. Hypertension is associated with increased mortality and severity of disease in COVID-19 pneumonia: a systematic review, meta-analysis and meta-regression. JRAAS J Renin-Angiotensin-Aldosterone Syst. (2020) 21:147032032092689. doi: 10.1177/1470320320926899

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Pranata R, Henrina J, Lim MA, Lawrensia S, Yonas E, Vania R, et al. Clinical frailty scale and mortality in COVID-19: a systematic review and dose-response meta-analysis: clinical frailty scale in COVID-19. Arch Gerontol Geriatr. (2021) 93:104324. doi: 10.1016/j.archger.2020.104324

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Tuty Kuswardhani RA, Henrina J, Pranata R, Anthonius Lim M, Lawrensia S, Suastika K. Charlson comorbidity index and a composite of poor outcomes in COVID-19 patients: a systematic review and meta-analysis. Diabetes Metab Syndr Clin Res Rev. (2020) 14:2103–9. doi: 10.1016/j.dsx.2020.10.022

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Edmonds ZV. Hyponatremia in pneumonia. J Hosp Med. (2012) 7:S11–3. doi: 10.1002/jhm.1933

CrossRef Full Text | Google Scholar

29. Berni A, Malandrino D, Parenti G, Maggi M, Poggesi L, Peri A. Hyponatremia, IL-6, and SARS-CoV-2 (COVID-19) infection: may all fit together? J Endocrinol Invest. (2020) 43:1137–9. doi: 10.1007/s40618-020-01301-w

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Yonas E, Alwi I, Pranata R, Huang I, Lim MA, Yamin M, et al. Elevated interleukin levels are associated with higher severity and mortality in COVID 19 – a systematic review, meta-analysis, and meta-regression. Diabetes Metab Syndr Clin Res Rev. (2020) 14:2219–30. doi: 10.1016/j.dsx.2020.11.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Keywords: coronavirus–COVID-19, sodium, mortality, severe, critical, hyponatremia

Citation: Akbar MR, Pranata R, Wibowo A, Irvan, Sihite TA and Martha JW (2021) The Prognostic Value of Hyponatremia for Predicting Poor Outcome in Patients With COVID-19: A Systematic Review and Meta-Analysis. Front. Med. 8:666949. doi: 10.3389/fmed.2021.666949

Received: 11 February 2021; Accepted: 10 May 2021;
Published: 14 June 2021.

Edited by:

Eric Azabou, Assistance Publique Hopitaux de Paris, France

Reviewed by:

Jesus Rico-Feijoo, Hospital Universitario Río Hortega, Spain
Vernon Chinchilli, Pennsylvania State University, United States

Copyright © 2021 Akbar, Pranata, Wibowo, Irvan, Sihite and Martha. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Mohammad Rizki Akbar, m.r.akbar@unpad.ac.id

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.