Does Routine Anti-Osteoporosis Medication Lower the Risk of Fractures in Male Subjects? An Updated Systematic Review With Meta-Analysis of Clinical Trials

Background: Several epidemiological articles have reported the correlations between anti-osteoporosis medication and the risks of fractures in male and female subjects, but the specific efficacy of anti-osteoporosis medication for male subjects remains largely unexplored.

Objective: The aim of this study was to evaluate the correlation between anti-osteoporosis medication and the risk of fracture in relation to low bone mass [including outcomes of osteoporosis, fracture, and bone mineral density (BMD) loss] in male subjects analyzed in studies within the updated literature.

Methods: Randomized controlled trials (RCTs) that analyzed the effectiveness of a treating prescription for male subjects with osteoporosis (or low BMD) and that focused on the outcomes of fracture were included. Relevant studies from Embase, Web of Science, PubMed, and Chinese database of CNKI were retrieved from inception to January 30th, 2019. Two staff members carried out the eligibility assessment and data extraction. The discrepancies were settled by consultation with another researcher. We calculated the pooled relative risks (RRs) based on 95% confidence intervals (CIs).

Results: Twenty-seven documents (28 studies) with 5,678 subjects were identified. For the category of bisphosphonates, significant results were observed in pooled analyses for decreased risk of the vertebral fracture domain (RR, 0.44 [95% CI, 0.31–0.62]), nonvertebral fracture domain (RR, 0.63 [95% CI, 0.46–0.87]), and clinical fracture domain (RR, 0.59 [95% CI, 0.48–0.72]) compared with those of controls. Participants with bisphosphonates had a 56% (95% CI = 38–69%) lower risk of vertebral fractures, 37% (95% CI = 13–54%) lower risk of nonvertebral fractures, and 41% (95% CI = 28–52%) lower risk of clinical fractures. Furthermore, meta-analyses also demonstrated a decreased risk of the vertebral fracture domain via treatment with risedronate (RR, 0.45 [95% CI, 0.28–0.72]) and alendronate (RR, 0.41 [95% CI, 0.23–0.74]), but not with calcitriol, calcitonin, denosumab, ibandronate, monofluorophosphate, strontium ranelate, teriparatide, or zoledronic acid, compared with that of controls.

Conclusions: This systematic review confirms that bisphosphonates were connected with a decreased risk of vertebral fractures, nonvertebral fractures, and clinical fractures for male subjects with osteoporosis. Future research is needed to further elucidate the role of nonbisphosphonates in treating fractures of osteoporosis subjects.

Introduction

Recent evidence has found that 2 million male subjects in the United States have been affected by osteoporosis, accompanied by 12 million male subjects with high-risk conditions (Gielen et al., 2011). Cumulative data have suggested that osteoporosis-oriented fracture risk among male subjects is substantial, and the reports in the United States showed that 27–30% of fractures occur at ages of ≥ 50 years among male subjects (Burge et al., 2007; King et al., 2009). Worldwide, male subjects aged 50 years and older were estimated to incur 39% of all osteoporotic fractures (Johnell and Kanis, 2006). Also, the mortality rate after osteoporotic fracture in male subjects was found to be 39–52% higher than that in females (Haentjens et al., 2010; Kannegaard et al., 2010). Despite the abovementioned facts, osteoporosis in male subjects was still found to be undertreated or under-recognized, even though nearly one in five male subjects aged 50 years and older meet the criteria of the National Osteoporosis Foundation and other references to receive treatment with anti-osteoporosis therapies (Kiebzak et al., 2002; Ebeling, 2008; Khosla et al., 2008). With increasing longevity of male subjects and a concomitant increase in the proportion of the aging population, fractures and burdens for targeted subjects’ health care are likely to increase in the upcoming years.

Recently, the routine therapies for anti-osteoporosis in male subjects include alendronate, risedronate, and teriparatide (Watts et al., 2012; Viswanathan et al., 2018). Although a series of epidemiological reports have been conducted to assess the correlation between anti-osteoporosis medication and the potential risk of male subjects’ fractures (Ringe et al., 1998; Kurland et al., 2000; Orwoll et al., 2000; Ebeling et al., 2001; Ringe et al., 2001; Trovas et al., 2002; Orwoll et al., 2003; Ringe et al., 2004; Miller et al., 2004; Shimon et al., 2005; Toth et al., 2005; Ringe et al., 2006; Boonen et al., 2009; Ringe et al., 2009; Langdahl et al., 2009; Orwoll et al., 2010a; Orwoll et al., 2010b; Ringe et al., 2010; Boonen et al., 2011; Boonen et al., 2012; Orwoll et al., 2012; Kaufman et al., 2013; Kachnic et al., 2013; Nakamura et al., 2014; Yan, 2014; Peng et al., 2018; Zhao et al., 2018), the outcomes have remained controversial. An inverse link between anti-osteoporosis medication and the risk of male subjects’ fractures was observed in three studies (Orwoll et al., 2000; Ringe et al., 2006; Ringe et al., 2009), whereas no association was found in other studies (Ringe et al., 2001; Ringe et al., 2004; Boonen et al., 2009; Orwoll et al., 2010a; Orwoll et al., 2010b; Orwoll et al., 2012; Peng et al., 2018). The aim of this study was to evaluate the correlation between anti-osteoporosis medication and the risk of fracture in relation to low bone mass (including outcomes of osteoporosis, fracture, and BMD loss) in male subjects analyzed in studies within the updated literature.

Methods

The guidelines reported in the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) (Moher et al., 2009) were strictly followed while performing the literature searches and during the assembling of the results. A protocol for this systematic review was developed before the research began; however, this review was not registered in the Research Registry.

Data Sources and Search Strategies

Randomized controlled trials (RCTs) that analyzed the effectiveness of a treating prescription for male subjects with osteoporosis (or low BMD) and that focused on the outcomes of fracture were included. Relevant studies from Embase, PubMed, Web of Science, and Chinese database of CNKI were searched from the inception to January 30th, 2019. The strategies of free-text terms and MeSH terms were adopted for the databases with relevant key words such as “osteoporosis,” “osteopenia,” “fracture,” “bone density,” “bone mass,” “bone,” “bone disease,” “alendronate,” “risedronate,” “ibandronate,” “teriparatide,” and other routine anti-osteoporosis medications. Also, the search methodology included identification of MeSH words from the abstract and title. Additional articles were taken into consideration by manually rechecking the reference lists of topical review documents. The final results of the article searching were updated on January 30th, 2019. A full electronic search strategy for the PubMed database was included as an additional file (Table S1).

Study Selection

Inclusion criteria were used for identification of the potential articles. The domains of inclusion criteria were as follows: 1) RCT studies published in Chinese or English, 2) studies focused on assessing the effectiveness of a treating prescription for subjects with low bone mass (including outcomes of osteoporosis, fracture, and BMD loss, e.g., age ≥ 18 years, males), 3) studies that provided separate results for male subjects or included male subjects, 4) studies that reported fracture outcomes or provided sufficient data to calculate numbers of male subjects involved, 5) the length of interventions in the studies was at least 6 months, and 6) studies had to address the trial and the control group (i.e., either comparison of intervention as anti-osteoporosis medication versus placebo, active comparators, or another group). Most of the included articles were taken in comparison to a treatment option with vitamin D and calcium, placebo, or both. Very few of the studies had comparative active compounds.

Furthermore, the exclusion criteria were as follows: 1) previous reports of review papers, mechanistic studies, or animal experiments; 2) the subjects were not all found to have low bone mineral density (BMD) (T-score ≤1) or osteoporosis; and 3) reports that published only abstracts. Articles were assessed for inclusion based on two stages. The first step consisted of carefully reviewing the titles and abstracts, while the second step consisted of full-text checking of the articles for indications related to the topics of our meta-analysis. Two staff members carried out the eligibility assessment and data extraction. The discrepancies were settled by consultation with another researcher.

Extraction of Data and Quality Assessment

Data were collected from the articles and included the following items: the characteristics of the trial subjects, number of male subjects, study population, trial/control assessment, comparator for assessed trial/control, periods of the trial, the time of follow-ups, fracture-outcome domains, and results mentioned for fracture-outcome domains in the control and trial groups. As for the domain of fracture outcomes, we mainly gathered data on the numbers of subjects with incident fractures in the trial/control groups. Two researchers performed the above data extractions based on the standardized forms for literature collection. While literature was found and performed in the previous review, we would refer to the source document and recheck whether or not the information was correct. The data extraction was done independently and was further cross-checked for validity. Any discrepancies were resolved by seeking help from a third expert. The quality-assessment tool focused on the risk bias assessment recommended by the Cochrane Collaboration (RevMan 5.3; www.cochrane.org/training/cochrane-handbook). Key domains for the above assessments contained reporting bias, selection bias, detection bias, performance bias, attrition bias, and other sources of biases.

Data Analysis

The clinical characteristics and study quality were performed by narrative synthesis. For the assessment of study quality, we used the approaches that are recommended by the Cochrane Collaboration to evaluate risk of potential bias for the studies included, including schemes to estimate the potential risk of performance, selection, attrition, detection, and reporting biases. Pooled relative risks (RRs) with 95% confidence intervals (CIs) were computed by using a fixed-effects model (FEM) if there was no significant evidence of heterogeneity. If this was not the case, we performed the evaluation using the random-effects model (REM). Between-trial heterogeneity in all the meta-analyses was calculated based on I-squared (I²) statistics and Chi-square tests. For the articles that mentioned the outcomes of fracture for several time points of follow-up, the longest duration times were chosen while conducting meta-analyses. While the statistical heterogeneity was identified, we would conduct the sensitivity analyses to explore the potential sources for the existing heterogeneity. Publication biases were checked and inspected by funnel-plot asymmetry. The analyses above were carried out based on the software of Stata SE, version 14.1 (Stata Corp, College Station, TX, USA). RevMan 5.3 software, recommended by the Cochrane Collaboration, was used to assess study quality and potential sources of biases.

Results

Literature Search and Study Selection

Across the 1,639 documents identified during the initial search, 652 articles were selected for further verification by assessing the details of their full-text manuscripts. Then, 625 documents were excluded for various reasons (Figure 1). Finally, 27 articles (Ringe et al., 1998; Kurland et al., 2000; Orwoll et al., 2000; Ebeling et al., 2001; Ringe et al., 2001; Trovas et al., 2002; Orwoll et al., 2003; Miller et al., 2004; Ringe et al., 2004; Shimon et al., 2005; Toth et al., 2005; Ringe et al., 2006; Boonen et al., 2009; Langdahl et al., 2009; Ringe et al., 2009; Orwoll et al., 2010a; Orwoll et al., 2010b; Ringe et al., 2010; Boonen et al., 2011; Boonen et al., 2012; Orwoll et al., 2012; Kachnic et al., 2013; Kaufman et al., 2013; Nakamura et al., 2014; Yan, 2014; Peng et al., 2018; Zhao et al., 2018) (including 28 studies) met the qualified criteria. The procedure of the trial selection is presented in Figure 1.

FIGURE 1

Figure 1 Preferred reporting items for systematic reviews and meta-analysis flow chart of the literature search.

Study Characteristics

The characteristics of the 27 articles (Ringe et al., 1998; Kurland et al., 2000; Orwoll et al., 2000; Ebeling et al., 2001; Ringe et al., 2001; Trovas et al., 2002; Orwoll et al., 2003; Miller et al., 2004; Ringe et al., 2004; Shimon et al., 2005; Toth et al., 2005; Ringe et al., 2006; Boonen et al., 2009; Langdahl et al., 2009; Ringe et al., 2009; Orwoll et al., 2010a; Orwoll et al., 2010b; Ringe et al., 2010; Boonen et al., 2011; Boonen et al., 2012; Orwoll et al., 2012; Kachnic et al., 2013; Kaufman et al., 2013; Nakamura et al., 2014; Yan, 2014; Peng et al., 2018; Zhao et al., 2018) (including 28 studies) involving 5,678 subjects for this review are showed in Table 1. The studies were reported between 1998 and 2018. The durations of the included trials ranged from 6 to 36 months. The number of male subjects ranged from 24 to 1,199. The bisphosphonate prescriptions were found in the trial group across half of the studies included, and specified domains of alendronate or risedronate were used much more than the other bisphosphonates. Anti-osteoporosis treatment was compared with placebo, calcium, vitamin D, or combination of calcium and vitamin D for most of the studies, while active comparators were seldom observed in the control. Most of the fracture outcomes in the studies focused on vertebral fractures; also, studies contained the assessment of nonvertebral fractures or clinical fractures. The details are summarized in Table 1.

TABLE 1

Table 1 Characteristics of the studies included and fracture outcomes.

Study Quality and Potential Sources of Bias

The risks of bias for the 27 articles (Ringe et al., 1998; Kurland et al., 2000; Orwoll et al., 2000; Ebeling et al., 2001; Ringe et al., 2001; Trovas et al., 2002; Orwoll et al., 2003; Miller et al., 2004; Ringe et al., 2004; Shimon et al., 2005; Toth et al., 2005; Ringe et al., 2006; Boonen et al., 2009; Langdahl et al., 2009; Ringe et al., 2009; Orwoll et al., 2010a; Orwoll et al., 2010b; Ringe et al., 2010; Boonen et al., 2011; Boonen et al., 2012; Orwoll et al., 2012; Kachnic et al., 2013; Kaufman et al., 2013; Nakamura et al., 2014; Yan, 2014; Peng et al., 2018; Zhao et al., 2018) (including 28 studies) are presented in Figures S1, S2. The reporting quality of the included trials was identified to be generally moderate, which provided certain unclear information (i.e., whether concealment of allocation, the random sequence generated, and items of blinding were carried out) for further verified inference. Additionally, inadequate reporting could result in risk validity of the results or potential bias.

Bisphosphonate and the Risk of Osteoporotic Fractures

Thirteen studies (Orwoll et al., 2000; Ringe et al., 2001; Ringe et al., 2004; Ringe et al., 2006; Boonen et al., 2009; Ringe et al., 2009; Orwoll et al., 2010a; Orwoll et al., 2010b; Orwoll et al., 2012; Peng et al., 2018) reported bisphosphonate treatments and the incidence of vertebral fractures. Three studies (Orwoll et al., 2000; Ringe et al., 2006; Ringe et al., 2009) revealed that the subjects with bisphosphonate probably reduced the risk of vertebral fractures, whereas the others (Ringe et al., 2001; Ringe et al., 2004; Boonen et al., 2009; Orwoll et al., 2010a; Orwoll et al., 2010b; Orwoll et al., 2012; Peng et al., 2018) failed to find such a correlation. The pooled-effect estimates showed that statistically significant differences between the two groups (RR, 0.44 [95% CI, 0.31–0.62]) were observed. Moreover, 10 studies (Orwoll et al., 2000; Ebeling et al., 2001; Ringe et al., 2001; Shimon et al., 2005; Ringe et al., 2006; Boonen et al., 2009; Ringe et al., 2009; Orwoll et al., 2010a; Boonen et al., 2012; Peng et al., 2018) reported the correlation between bisphosphonate and the incidence of nonvertebral fractures. In comparison to the control, a statistically significant association between the two groups (RR, 0.63 [95% CI, 0.46–0.87]) was identified. In addition, two studies (Boonen et al., 2009; Ringe et al., 2009) found that the subjects with bisphosphonate had a lower risk of clinical fractures, whereas the others (Orwoll et al., 2000; Ringe et al., 2001; Ringe et al., 2004; Shimon et al., 2005; Ringe et al., 2006; Orwoll et al., 2010a; Orwoll et al., 2010b; Boonen et al., 2011; Boonen et al., 2012; Kachnic et al., 2013; Peng et al., 2018) failed to show such a link. The synthesized evidence for the risk of clinical fractures displayed that there were statistically significant differences between groups (RR, 0.59 [95% CI, 0.48–0.72]). Overall, the results indicated that the patients with bisphosphonate were seemingly associated with lower risk of fracture outcomes, including vertebral fractures, nonvertebral fractures, and clinical fractures (Figure 2).

FIGURE 2

Figure 2 Forest plot of meta-analysis on bisphosphonate and osteoporotic fractures. The size of the diamond and box is positively proportional to the weight assigned to each study, and horizontal lines represent the 95% CI. RR, relative risk; CI, confidence interval.

Alendronate and the Risk of Osteoporotic Fractures

Five studies (Orwoll et al., 2000; Ringe et al., 2001; Ringe et al., 2004; Shimon et al., 2005; Peng et al., 2018) reported alendronate and the incidence of osteoporotic fractures. The RR for the risk of vertebral fractures between groups was found (RR, 0.41 [95% CI, 0.23–0.74]). The outcomes showed that patients with alendronate had a 59% (95% CI, 26%–77%) lower risk of vertebral fractures. Also, the pooled-effect estimates for subjects with alendronate were toward a reduced risk of clinical fractures (RR, 0.54 [95% CI, 0.36–0.79]); however, this was not the case for nonvertebral fractures (RR, 0.70 [95% CI, 0.39–1.25]) when compared to that of the controls (Figure 3).

FIGURE 3

Figure 3 Forest plot of meta-analysis on alendronate and osteoporotic fractures. The size of diamond and box is positively proportional to the weight assigned to each study, and horizontal lines represent the 95% CI. RR, relative risk; CI, confidence interval.

Calcitonin and the Risk of Osteoporotic Fractures

There were two studies (Trovas et al., 2002; Toth et al., 2005) involving calcitonin and the incidence of osteoporotic fractures. Compared with that of the control, no statistically significant association was observed in the pooled analysis for calcitonin and the risk of the vertebral fracture domain (RR, 0.32 [95% CI, 0.05–1.98]), nonvertebral fracture domain (RR, 0.27 [95% CI, 0.01–6.37]), or clinical fracture domain (RR, 0.28 [95% CI, 0.05–1.72]) (Figure S3).

Denosumab and the Risk of Osteoporotic Fractures

Two studies (Orwoll et al., 2012; Nakamura et al., 2014) evaluated denosumab and the incidence of osteoporotic fractures. The meta-analysis revealed that no statistically significant differences were found between groups concerning the risk of the vertebral fracture domain (RR, 0.27 [95% CI, 0.03–2.40]), nonvertebral fracture domain (RR, 1.00 [95% CI, 0.06–15.81]), or clinical fracture domain (RR, 0.37 [95% CI, 0.06–2.38]) (Figure S4).

Risedronate and the Risk of Osteoporotic Fractures

Three studies (Ringe et al., 2006; Boonen et al., 2009; Ringe et al., 2009) reported risedronate and the incidence of vertebral fractures. Meta-analyses from the above trials found a significant reduction in fracture outcomes via administration of risedronate, including the risk of the vertebral fracture domain (RR, 0.45 [95% CI, 0.28–0.72]), nonvertebral fracture domain (RR, 0.59 [95% CI, 0.39–0.88]), and clinical fracture domain (RR, 0.56 [95% CI, 0.42–0.75]) (Figure 4).

FIGURE 4

Figure 4 Forest plot of meta-analysis on risedronate and osteoporotic fractures. The size of diamond and box is positively proportional to the weight assigned to each study, and horizontal lines represent the 95% CI. RR, relative risk; CI, confidence interval.

Calcitriol and the Risk of Osteoporotic Fractures

No statistically significant association was observed between calcitriol and risk of the vertebral fracture domain (RR, 4.62 [95% CI, 0.60–35.32]), nonvertebral fracture domain (RR, 8.46 [95% CI, 0.50–144.21]), or clinical fracture domain (RR, 7.10 [95% CI, 0.99–50.81]) (Figure S5).

Ibandronate and the Risk of Osteoporotic Fractures

For the risk of osteoporotic fractures, no significant effect was identified in the ibandronate group in terms of the vertebral fracture domain (RR, 0.28 [95% CI, 0.03–3.05]), nonvertebral fracture domain (RR, 2.72 [95% CI, 0.13–55.58]), or clinical fracture domain (RR, 0.83 [95% CI, 0.14–4.82]) (Figure S6).

Monofluorophosphate and the Risk of Osteoporotic Fractures

No positive effects were observed between monofluorophosphate and the risk of the vertebral fracture domain (RR, 0.31 [95% CI, 0.10–1.03]), nonvertebral fracture domain (RR, 0.68 [95% CI, 0.24–1.88]), or clinical fracture domain (RR, 0.52 [95% CI, 0.26–1.06]) (Figure S7).

Strontium Ranelate and the Risk of Osteoporotic Fractures

Three studies (Ringe et al., 2010; Kaufman et al., 2013; Yan, 2014) reported strontium ranelate and the incidence of osteoporotic fractures. The overall effects of pooled analyses did not indicate any significant difference between groups concerning the risk of the vertebral fracture domain (RR, 0.79 [95% CI, 0.35–1.78]), nonvertebral fracture domain (RR, 0.52 [95% CI, 0.13–2.00]), or clinical fracture domain (RR, 0.71 [95% CI, 0.36–1.40]) (Figure S8).

Teriparatide and the Risk of Osteoporotic Fractures

Five studies (Kurland et al., 2000; Orwoll et al., 2003; Langdahl et al., 2009; Zhao et al., 2018) (two studies included in Orwoll et al., 2003) evaluated teriparatide and the incidence of osteoporotic fractures. Meta-analysis of the data found that the subjects treated with teriparatide in the trial group were not significantly improved compared to those of the control concerning the reduction of fracture outcome, including the vertebral fracture domain (RR, 0.40 [95% CI, 0.10–1.67]), nonvertebral fracture domain (RR, 0.52 [95% CI, 0.21–1.27]), and clinical fracture domain (RR, 0.47 [95% CI, 0.22–1.02]) (Figure S9).

Zoledronic Acid and the Risk of Osteoporotic Fractures

Four studies (Orwoll et al., 2010b; Boonen et al., 2011; Boonen et al., 2012; Kachnic et al., 2013) reported zoledronic acid and the incidence of osteoporotic fractures. As shown in Figure S10, no statistically significant differences between groups were identified for pooled effects by assessing the vertebral fracture domain (RR, 0.56 [95% CI, 0.19–1.67]), nonvertebral fracture domain (RR, 0.65 [95% CI, 0.21–1.98]), and clinical fracture domain (RR, 0.74 [95% CI, 0.46–1.21]).

Publication Bias

Publication bias of this study was evaluated based on funnel plots and RRs performed from trials involving alendronate, risedronate, bisphosphonates, and the risk of fracture outcomes. To large extents, the points should be displayed symmetrically around the vertical line concerning the pooled RRs while in the absence of publication bias. The shapes of the funnel plot were found to be symmetrically reasonable, which indicated the absence of publication bias (Figures S11–13).

Discussion

Statement of Principal Findings

This review included 27 documents (involving 28 studies) with 5,678 subjects. Meta-analyses of these studies found that for the category of bisphosphonates, significant results were observed in pooled analyses for reduced risk of the vertebral fracture domain (RR, 0.44 [95% CI, 0.31–0.62]), nonvertebral fracture domain (RR, 0.63 [95% CI, 0.46–0.87]), and clinical fracture domain (RR, 0.59 [95% CI, 0.48–0.72]), compared with those of controls. Participants with bisphosphonates had a 56% (95% CI = 38–69%) lower risk of vertebral fractures, 37% (95% CI = 13–54%) lower risk of nonvertebral fractures, and 41% (95% CI = 28–52%) lower risk of clinical fractures. Furthermore, meta-analyses showed a decreased risk of vertebral fractures by treatment with alendronate and risedronate, but not with calcitonin, denosumab, calcitriol, ibandronate, monofluorophosphate, strontium ranelate, teriparatide, or zoledronic acid, as compared to that of the controls.

This systematic review confirms that bisphosphonates were correlated with a lower risk of the vertebral fracture domain, nonvertebral fracture domain, and clinical fracture domain for male subjects with osteoporosis. Because of the existing defects in the methodological quality for the included studies, the definitive correlation involving routine anti-osteoporosis medication and the risk of male subjects’ fracture could not be fully verified by the current evidence. Thus, any recent advice and proposals for clinical practice should be interpreted with caution. Further studies with high-quality designs are needed to validate these findings.

Comparison of Findings With Other Results in the Literature

In this study, we performed a systematic review of the possible protective effect of routine anti-osteoporosis medication on the risk of fractures in male subjects with osteoporosis. A similar article (Nayak and Greenspan, 2017) with 22 studies included on the same subjects had been reported in March 2017. This study found that bisphosphonates as a treatment category significantly lowered the risk of the vertebral fracture domain (RR, 0.368 [95% CI, 0.252–0.537]) and nonvertebral fracture domain (RR, 0.604 [95% CI, 0.404–0.904]) compared to that of controls. In our study, three more studies (Yan, 2014; Peng et al., 2018; Zhao et al., 2018), conducted in China, were included. Three studies (Orwoll et al., 2003; Miller et al., 2004; Shimon et al., 2005) in the previous article (Nayak and Greenspan, 2017) were not included in their meta-analysis, but we rechecked these studies and chose to include them for further analysis. To some extent, our present study provides the opportunity to evaluate the original documents that have been reported in Chinese journals. This also contributes to the possible benefit of assessing published articles and forming a sound basis for further research for similar topics. In addition to assessing the correlation between anti-osteoporosis medications and the risk of fractures, we also performed the meta-analyses on the possibly beneficial effect of bisphosphonate and the risk of fracture in relation to low bone mass (including outcomes of osteoporosis, fracture, and BMD loss) in male subjects. Finally, we found that the pooled-effect estimates of bisphosphonates were observed in the direction of lower risk of the vertebral fracture domain (RR, 0.44 [95% CI, 0.31–0.62]), nonvertebral fracture domain (RR, 0.63 [95% CI, 0.46–0.87]), and clinical fracture domain (RR, 0.59 [95% CI, 0.48–0.72]), compared with those of the controls. Participants with bisphosphonates had a 56% (95% CI = 38–69%) lower risk of vertebral fractures, 37% (95% CI = 13–54%) lower risk of nonvertebral fractures, and 41% (95% CI = 28–52%) lower risk of clinical fractures (Figure 2). These findings were different from the results of Nayak et al. that were published in March 2017 (Nayak and Greenspan, 2017). Furthermore, our meta-analyses also showed a decreased risk of the vertebral fracture domain by treatment with alendronate (RR, 0.41 [95% CI, 0.23–0.74]; Figure 3) and risedronate (RR, 0.45 [95% CI, 0.28–0.72]; Figure 4), compared with that of controls; however, this association was not found among calcitonin, denosumab, calcitriol, ibandronate, monofluorophosphate, strontium ranelate, teriparatide, or zoledronic acid (Figures S3–10).

A number of articles concerning anti-osteoporosis medication and possible risk of fracture have been published and have included randomized controlled studies, case series, controlled studies, case reports, and meta-analyses. However, there was no systematic review focusing upon possible protective effects of bisphosphonates and other routine anti-osteoporosis medications on the risk of fracture of osteoporosis male subjects in relation to low bone mass (including outcomes of osteoporosis, fracture, and BMD loss). To a certain extent, our study is the first to explore the roles of anti-osteoporosis medication and possible risk of the above outcome measures by searching and analyzing the updated evidence of the existing literature.

Possible Explanations and Implications of the Study

Additive benefits for the subjects with male fracture in relation to low bone mass were observed while taking bisphosphonates (i.e., a significant links possibly exist among bisphosphonates, fracture, osteoporosis, and BMD loss in male subjects). Relatively few RCTs have been carried out to evaluate the effectiveness of anti-osteoporosis therapies for male subjects concerning risk reduction of fracture outcomes. These systematic-review findings for individual treatment prescriptions indicated that risedronate and alendronate could decrease the risk of the vertebral fracture domain of male subjects. However, the review for other specified treatment prescriptions did not reveal evidence of sufficient efficacy for reducing the risk of the vertebral fracture domain for male subjects, including treatments with denosumab, calcitriol, calcitonin, ibandronate, monofluorophosphate, strontium ranelate, teriparatide, and zoledronic acid. For the category of bisphosphonates, significant results were observed in pooled analyses for lower risk of vertebral fractures, nonvertebral fractures, and clinical fractures when compared to those of the controls.

These findings highlight the need for additional RCTs with higher quality and better design to focus on the effectiveness of anti-osteoporosis therapies for male subjects that are fully efficacious for improving fracture outcomes. Moreover, these findings call for further studies to assess the effectiveness of non-bisphosphonate prescription options. Also, this review highlights the lack of active comparator RCTs of anti-osteoporosis therapies for male subjects. Additional studies of anti-osteoporosis therapies for male subjects using active comparators (not placebo) may contribute to further verifying the potential effectiveness of varied treatment prescriptions for decreasing the risk of fracture in male subjects. Furthermore, these findings indicate the demand for a larger diversity of subjects in actual clinical studies of anti-osteoporosis therapies for male subjects, because most of the articles greatly enrolled a trial population of white subjects. Finally, no included articles were observed for a relatively longer duration (≥3 years). Therefore, the effectiveness of relatively longer anti-osteoporosis periods to lower the risk of fracture for male subjects remains uncertain. Further studies with longer periods and follow-up times would be helpful for assessing the effectiveness of anti-osteoporosis therapies for more than 3 years on the risk of fracture in male subjects. This process maybe help to clarify the possible fracture-risk reduction benefit for male subjects, similar to those of longer duration anti-osteoporosis studies that have shown the positive effects for females (Black et al., 2012; Cosman et al., 2014).

Strengths of the Review

Several notable strengths have been observed in this systematic review. First and foremost, this study is the most comprehensive systematic review of RCTs concerning effectiveness of potential anti-osteoporosis medication to lower the risk of male subjects’ fractures in relation to low bone mass (including outcomes of osteoporosis, fracture, and BMD loss). Despite a similar article (Nayak and Greenspan, 2017) with 22 studies reported in March 2017, another previous review on a similar topic of osteoporosis treatment efficacy for male subjects was also published in 2011; however, this study only included five articles that reported outcomes as fracture and concluded that the studies were inconclusive in terms of any decrease of fracture risk in male subjects (Schwarz et al., 2011). Our systematic review included 27 RCT articles (with 28 studies) involving anti-osteoporosis therapies for male subjects that reported outcomes of fractures and identified evidence involving the effectiveness of bisphosphonate prescriptions on lowering the risk of the vertebral fracture domain, nonvertebral fracture domain, and clinical fracture domain in male subjects. Furthermore, the risk of bias concerning individual RCTs included in this systematic review was assessed and performed based on the routine criteria that the Cochrane Collaboration recommended (Higgins et al., 2011).

Limitations of the Review

The meta-analyses were limited by the number of similar articles evaluating each individual treatment prescription, with just a few (ranging from one to six) articles including individual meta-analysis of the conducted treatment prescription. Also, meta-analyses with higher quality and better designs should be carried out in the future for investigating different fracture outcomes while assessing the category of bisphosphonates. Moreover, most articles included in this review had a relatively small sample size. The quality of the included studies in this systematic review was moderate. In addition, the findings of our meta-analysis were relatively limited due to the unclear or high risk of bias observed among the studies. Despite the above limitations, this review recommends that bisphosphonates could be adopted as first-line anti-osteoporosis therapies for male subjects, based on the evidence of their effectiveness in lowering the risk of vertebral fractures, nonvertebral fractures, and clinical fractures. The findings of the specified effectiveness concerning bisphosphonate in lowering the risk of fracture in male subjects apply to those individuals with low BMD or osteoporosis based on dual X-ray absorptiometry (DXA) criteria, because the articles included in this review enrolled subjects who were qualified with the above criteria. Further studies should also assess the effectiveness of anti-osteoporosis therapies for male subjects with high risks of fracture that are not identified to have low BMD or osteoporosis based on DXA criteria or prior fracture events.

Conclusion

This systematic review confirms that bisphosphonates are connected with a decreased risk of vertebral fractures, nonvertebral fractures, and clinical fractures for male subjects with osteoporosis. Future studies will be required to further elucidate the role of nonbisphosphonates in treating fractures of osteoporosis subjects.

Ethics Statement

This study is a systemic review with meta-analysis that did not involve experimental subjects or human tissues, and no sensitive data or private information were collected during the above study process.

Author Contributions

JL and W-YY conceived and designed the study. The literature searches, study selection, critical appraisal, data extraction, and contacting authors of included studies for additional information were carried out by L-FZ, B-QP, M-HL, Z-TG, DZ, J-LZ, J-TL and J-KP. G-H L, YC, H-YC, H-TH, QL, Y-HH, J-HL, S-RH, MW and L-FZ conducted the interpretation and analysis of the relevant data. L-FZ, DG, W-XL, QW and JL drafted the paper. JL, A-HO and L-FZ further revised the text. The final version of this article was rechecked and approved by L-FZ, B-QP, G-HL, M-HL, YC, DG, H-YC, J-KP, H-TH, QL, Z-TG, Y-HH, DZ, J-LZ, S-RH, MW, J-TL, J-HL, W-XL, A-HO, QW, W-YY, and JL.

Funding

This work was funded by the China Postdoctoral Science Foundation (No. 2018M633036), the Medical Science Research Foundation of Guangdong Province (No. B2019091), the National Natural Science Foundation of China (No. 81873314), the Project of Guangdong Provincial Department of Finance (No.[2014]157, No.[2018]8), Key Scientific Research Platforms and Research Projects of Universities in Guangdong Province (No. 2018KQNCX041), and the Science and Technology Research Project of Guangdong Provincial Hospital of Chinese Medicine (No. YK2013B2N19, YN2015MS15).

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We thank Amanj Kurdi, Han Raja, Laura Johnson, Gillian Attard (who works within the Production Office in Frontiers), and other editors/reviewers for their helpful comments and suggestions. We thank LetPub (www.letpub.com) for its linguistic assistance during the proof-reading of this manuscript.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar.2019.00882/full#supplementary-material

References

Black, D. M., Bauer, D. C., Schwartz, A. V., Cummings, S. R., Rosen, C. J. (2012). Continuing bisphosphonate treatment for osteoporosis–for whom and for how long? N. Engl. J. Med. 366, 2051–2053. doi: 10.1056/NEJMp1202623

PubMed Abstract | CrossRef Full Text | Google Scholar

Boonen, S., Orwoll, E. S., Wenderoth, D., Stoner, K. J., Eusebio, R., Delmas, P. D. (2009). Once-weekly risedronate in men with osteoporosis: results of a 2-year, placebo-controlled, double-blind, multicenter study. J. Bone Miner. Res. 24, 719–725. doi: 10.1359/jbmr.081214

PubMed Abstract | CrossRef Full Text | Google Scholar

Boonen, S., Orwoll, E., Magaziner, J., Colón-Emeric, C. S., Adachi, J. D., Bucci-Rechtweg, C., et al. (2011). Once-yearly zoledronic acid in older men compared with women with recent hip fracture. J. Am. Geriatr. Soc. 59, 2084–2090. doi: 10.1111/j.1532-5415.2011.03666.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Boonen, S., Reginster, J. Y., Kaufman, J. M., Lippuner, K., Zanchetta, J., Langdahl, B., et al. (2012). Fracture risk and zoledronic acid therapy in men with osteoporosis. N. Engl. J. Med. 367, 1714–1723. doi: 10.1056/NEJMoa1204061

PubMed Abstract | CrossRef Full Text | Google Scholar

Burge, R., Dawson-Hughes, B., Solomon, D. H., Wong, J. B., King, A., Tosteson, A. (2007). Incidence and economic burden of osteoporosis-related fractures in the United States, 2005–2025. J. Bone Miner.Res. 22, 465–475. doi: 10.1359/jbmr.061113

PubMed Abstract | CrossRef Full Text | Google Scholar

Cosman, F., de Beur, S. J., LeBoff, M. S., Lewiecki, E. M., Tanner, B., Randall, S., et al. (2014). Clinician’s guide to prevention and treatment of osteoporosis. Osteoporos. Int. 25, 2359–2381. doi: 10.1007/s00198-014-2794-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Ebeling, P. R. (2008). Clinical practice. Osteoporosis in men. N. Engl. J. Med. 358, 1474–1482. doi: 10.1056/NEJMcp0707217

PubMed Abstract | CrossRef Full Text | Google Scholar

Ebeling, P. R., Wark, J. D., Yeung, S., Poon, C., Salehi, N., Nicholson, G. C., et al. (2001). Effects of calcitriol or calcium on bone mineral density, bone turnover, and fractures in men with primary osteoporosis: a two-year randomized, double blind, double placebo study. J. Clin. Endocrinol. Metab. 86, 4098–4103. doi: 10.1210/jcem.86.9.7847

PubMed Abstract | CrossRef Full Text | Google Scholar

Gielen, E., Vanderschueren, D., Callewaert, F., Boonen, S., et al. (2011). Osteoporosis in men. Best Pract. Res. Clin. Endocrinol. Metab. 25, 321–335. doi: 10.1016/j.beem.2010.08.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Haentjens, P., Magaziner, J., Colón-Emeric, C. S., Vanderschueren, D., Milisen, K., Velkeniers, B., et al. (2010). Meta-analysis: excess mortality after hip fracture among older women and men. Ann. Intern. Med. 152, 380–390. doi: 10.7326/0003-4819-152-6-201003160-00008

PubMed Abstract | CrossRef Full Text | Google Scholar

Higgins, J. P., Altman, D. G., Gøtzsche, P. C., Jüni, P., Moher, D., Oxman, A. D., et al. (2011). The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 343, d5928. doi: 10.1136/bmj.d5928

PubMed Abstract | CrossRef Full Text | Google Scholar

Johnell, O., Kanis, J. A. (2006). An estimate of the worldwide prevalence and disability associated with osteoporotic fractures. Osteoporos. Int. 17, 1726–1733. doi: 10.1007/s00198-006-0172-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Kachnic, L. A., Pugh, S. L., Tai, P., Smith, M., Gore, E., Shah, A. B., et al. (2013). RTOG 0518: Randomized phase III trial to evaluate zoledronic acid for prevention of osteoporosis and Associated fractures in prostate cancer patients. Prostate Cancer Prostatic Dis. 16, 382–386. doi: 10.1038/pcan.2013.35

PubMed Abstract | CrossRef Full Text | Google Scholar

Kannegaard, P. N., van der Mark, S., Eiken, P., Abrahamsen, B. (2010). Excess mortality in men compared with women following a hip fracture. National analysis of comedications, comorbidity and survival. Age Ageing 39, 203–209. doi: 10.1093/ageing/afp221

PubMed Abstract | CrossRef Full Text | Google Scholar

Kaufman, J. M., Audran, M., Bianchi, G., Braga, V., Diaz-Curiel, M., Francis, R. M., et al. (2013). Efficacy and safety of strontium ranelate in the treatment of osteoporosis in men. Clin. Endocrinol. Metab. 98, 592–601. doi: 10.1210/jc.2012-3048

CrossRef Full Text | Google Scholar

Khosla, S., Amin, S., Orwoll, E. (2008). Osteoporosis in men. Endocr. Rev. 29, 441–464. doi: 10.1210/er.2008-0002

PubMed Abstract | CrossRef Full Text | Google Scholar

Kiebzak, G. M., Beinart, G. A., Perser, K., Ambrose, C. G., Siff, S. J., Heggeness, M. H., et al. (2002). Undertreatment of osteoporosis in men with hip fracture. Arch. Intern. Med. 162, 2217–2222. doi: 10.1001/archinte.162.19.2217

PubMed Abstract | CrossRef Full Text | Google Scholar

King, A. B., Tosteson, A. N., Wong, J. B., Solomon, D. H., Burge, R. T., Dawson-Hughes, B. (2009). Interstate variation in the burden of fragility fractures. J. Bone Miner. Res. 24, 681–692. doi: 10.1359/jbmr.081226

PubMed Abstract | CrossRef Full Text | Google Scholar

Kurland, E. S., Cosman, F., McMahon, D. J., Rosen, C. J., Lindsay, R., Bilezikian, J. P. (2000). Parathyroid hormone as a therapy for idiopathic osteoporosis in men: effects on bone mineral density and bone markers. J. Clin. Endocrinol. Metab. 85, 3069–3076. doi: 10.1210/jc.85.9.3069

PubMed Abstract | CrossRef Full Text | Google Scholar

Langdahl, B. L., Marin, F., Shane, E., Dobnig, H., Zanchetta, J. R., Maricic, M., et al. (2009). Teriparatide versus alendronate for treating glucocorticoid-induced osteoporosis: an analysis by gender and menopausal status. Osteoporos. Int. 20, 2095–2104. doi: 10.1007/s00198-009-0917-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Miller, P. D., Schnitzer, T., Emkey, R., Orwoll, E., Rosen, C., Ettinger, M., et al. (2004). Weekly oral alendronic acid in male osteoporosis. Clin. Drug. Investig. 24, 333–341. doi: 10.2165/00044011-200424060-00003

PubMed Abstract | CrossRef Full Text | Google Scholar

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., PRISMA Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement. BMJ 339, b2535. doi: 10.1136/bmj.b2535

PubMed Abstract | CrossRef Full Text | Google Scholar

Nakamura, T., Matsumoto, T., Sugimoto, T., Hosoi, T., Miki, T., Gorai, I., et al. (2014). Clinical trials express: fracture risk reduction with Denosumab in Japanese postmenopausal women and men with osteoporosis: Denosumab Fracture Intervention Randomized Placebo Controlled Trial (DIRECT). J. Clin. Endocrinol. Metab. 99, 2599–2607. doi: 10.1210/jc.2013-4175

PubMed Abstract | CrossRef Full Text | Google Scholar

Nayak, S., Greenspan, S. L. (2017). Osteoporosis treatment efficacy for men: a systematic review and meta-analysis. J. Am Geriatr. Soc. 65, 490–495. doi: 10.1111/jgs.14668

PubMed Abstract | CrossRef Full Text | Google Scholar

Orwoll, E., Ettinger, M., Weiss, S., Miller, P., Kendler, D., Graham, J., et al. (2000). Alendronate for the treatment of osteoporosis in men. N. Engl. J. Med. 343, 604–610. doi: 10.1056/NEJM200008313430902

PubMed Abstract | CrossRef Full Text | Google Scholar

Orwoll, E. S., Scheele, W. H., Paul, S., Adami, S., Syversen, U., Diez-Perez, A., et al. (2003). The effect of teriparatide [human parathyroid hormone (1–34)] therapy on bone density in men with osteoporosis. J. Bone Miner. Res. 18, 9–17. doi: 10.1359/jbmr.2003.18.1.9

PubMed Abstract | CrossRef Full Text | Google Scholar

Orwoll, E. S., Binkley, N. C., Lewiecki, E. M., Gruntmanis, U., Fries, M. A., Dasic, G. (2010a). Efficacy and safety of monthly ibandronate in men with low bone density. Bone 46, 970–976. doi: 10.1016/j.bone.2009.12.034

PubMed Abstract | CrossRef Full Text | Google Scholar

Orwoll, E. S., Miller, P. D., Adachi, J. D., Brown, J., Adler, R. A., Kendler, D., et al. (2010b). Efficacy and safety of a once-yearly i.v. infusion of zoledronic acid 5 mg versus a once-weekly 70-mg oral alendronate in the treatment of male osteoporosis: a randomized, multicenter, double-blind, active-controlled study. J. Bone Miner. Res. 25, 2239–2250. doi: 10.1002/jbmr.119

PubMed Abstract | CrossRef Full Text | Google Scholar

Orwoll, E., Teglbjærg, C. S., Langdahl, B. L., Chapurlat, R., Czerwinski, E., Kendler, D. L., et al. (2012). A randomized, placebo-controlled study of the effects of denosumab for the treatment of men with low bone mineral density. J. Clin. Endocrinol. Metab. 97, 3161–3169. doi: 10.1210/jc.2012-1569

PubMed Abstract | CrossRef Full Text | Google Scholar

Peng, C., Wang, Y. T., Liu, X. B., Qin, W. (2018). Observation on efficacy of alendronate in treatment of osteoporosis men. Eval. Anal. Drug Use Hospitals China 18, 600–602. doi: 10.14009/j.issn.1672-2124.2018.05009

CrossRef Full Text | Google Scholar

Ringe, J. D., Dorst, A., Kipshoven, C., Rovati, L. C., Setnikar, I. (1998). Avoidance of vertebral fractures in men with idiopathic osteoporosis by a three year therapy with calcium and low-dose intermittent monofluorophosphate. Osteoporos. Int. 8, 47–52. doi: 10.1007/s001980050047

PubMed Abstract | CrossRef Full Text | Google Scholar

Ringe, J. D., Faber, H., Dorst, A. (2001). Alendronate treatment of established primary osteoporosis in men: results of a 2-year prospective study. J. Clin. Endocrinol. Metab. 86, 5252–5255. doi: 10.1210/jcem.86.11.7988

PubMed Abstract | CrossRef Full Text | Google Scholar

Ringe, J. D., Dorst, A., Faber, H., Ibach, K. (2004). Alendronate treatment of established primary osteoporosis in men: 3-year results of a prospective, comparative, two-arm study. Rheumatol. Int. 24, 110–113. doi: 10.1007/s00296-003-0388-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Ringe, J. D., Faber, H., Farahmand, P., Dorst, A. (2006). Efficacy of risedronate in men with primary and secondary osteoporosis: results of a 1-year study. Rheumatol. Int. 26, 427–431. doi: 10.1007/s00296-005-0004-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Ringe, J. D., Farahmand, P., Faber, H., Dorst, A. (2009). Sustained efficacy of risedronate in men with primary and secondary osteoporosis: results of a 2-year study. Rheumatol. Int. 29, 311–315. doi: 10.1007/s00296-008-0689-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Ringe, J. D., Dorst, A., Farahmand, P. (2010). Efficacy of strontium ranelate on bone mineral density in men with osteoporosis. Arzneimittelforschung 60, 267–272. doi: 10.1055/s-0031-1296284

PubMed Abstract | CrossRef Full Text | Google Scholar

Schwarz, P., Jorgensen, N. R., Mosekilde, L., Vestergaard, P. (2011). The evidence for efficacy of osteoporosis treatment in men with primary osteoporosis: a systematic review and meta-analysis of antiresorptive and anabolic treatment in men. J. Osteoporos. 2011, 259818. doi: 10.4061/2011/259818

PubMed Abstract | CrossRef Full Text | Google Scholar

Shimon, I., Eshed, V., Doolman, R., Sela, B. A., Karasik, A., Vered, I. (2005). Alendronate for osteoporosis in men with androgen-repleted hypogonadism. Osteoporos. Int. 16, 1591–1596. doi: 10.1007/s00198-005-1879-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Toth, E., Csupor, E., Mészáros, S., Ferencz, V., Németh, L., McCloskey, E. V., et al. (2005). The effect of intranasal salmon calcitonin therapy on bone mineral density in idiopathic male osteoporosis without vertebral fractures–an open label study. Bone 36, 47–51. doi: 10.1016/j.bone.2004.09.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Trovas, G. P., Lyritis, G. P., Galanos, A., Raptou, P., Constantelou, E. (2002). A randomized trial of nasal spray salmon calcitonin in men with idiopathic osteoporosis: effects on bone mineral density and bone markers. J. Bone Miner. Re. 17, 521–527. doi: 10.1359/jbmr.2002.17.3.521

CrossRef Full Text | Google Scholar

Viswanathan, M., Reddy, S., Berkman, N., Cullen, K., Middleton, J. C., Nicholson, W. K., et al. (2018). Screening to prevent osteoporotic fractures: updated evidence report and systematic review for the US preventive services task force. JAMA 319, 2532–2551. doi: 10.1001/jama.2018.6537

PubMed Abstract | CrossRef Full Text | Google Scholar

Watts, N. B., Adler, R. A., Bilezikian, J. P., Drake, M. T., Eastell, R., Orwoll, E. S., et al. (2012). Osteoporosis in men: an endocrine society clinical practice guideline. J. Clin. Endocrinol. Metab. 97, 1802–1822. doi: 10.1210/jc.2011-3045

PubMed Abstract | CrossRef Full Text | Google Scholar

Yan, J. (2014). The evaluation of efficacy and safety of strontium ranelate on osteoporotic bone pain and bone mineral density in male osteoporosis. Lingnan Mod. Clin. Surg. 14, 446–449. doi: 10.3969/j.issn.1009-967X.2014.04.026

CrossRef Full Text | Google Scholar

Zhao, H. B., Ye, J. C., Cai, X. F., Lu, J. A., Li, J. P. (2018). Clinical study of parathyroid normone in the treatment of male subjects with osteoporosis. Guangdong Med. J. 39, 1993–1996. doi: 10.13820/j.CNKI.gdyx.20180717.007

CrossRef Full Text | Google Scholar