Methodological Quality and Risk of Bias Assessment of Cardiovascular Disease Research: Analysis of Randomized Controlled Trials Published in 2017

Background: All randomized-controlled trials (RCTs) are required to follow high methodological standards. In this study, we aimed to assess the methodological quality of published cardiovascular clinical research trials in a representative sample of RCTs published in 2017.

Methods: Cochrane Central Register of Controlled Trials was used to identify cardiovascular clinical research trials with adult participants published in 2017. Overall, 250 (10%) RCTs were randomly selected from a total of 2,419 studies. Data on general trial characteristics were extracted and the risk of bias (RoB) was determined.

Results: Overall, 86% of RCTs have reported at least one statistically significant result, with the primary outcome significant in 69%, treatment favored in 55%, and adverse events reported in 68%. Less than one-third (29%) of trials were overall low RoB, while the other two-thirds were rated unclear (40%) or with high RoB (31%). Sequence generation, allocation concealment, and selective reporting were the domains most often rated with high RoB. Drug trials were more likely to have low RoB than non-drug trials. Significant differences were found in RoB for the allocation concealment and blinding of participants and personnel between industry-funded and non-industry-funded trials, with industry-funded trials more often rated at low RoB.

Conclusion: Almost two-thirds of RCTs in the field of cardiovascular disease (CVD) research, were at high or unclear RoB, indicating a need for more rigorous trial planning and conduct. Prospective trial registration is a factor predicting a lower risk of bias.

Introduction

Randomized controlled trials (RCTs) are widely recognized as the most optimal methodology for causal inference, where humans are prospectively included and randomly allocated to groups to evaluate the efficacy and safety of an intervention (1). The strength of the RCTs comes from the randomization procedure, which ensures that all participants have the same chance of being assigned to each of the study groups (1) and guarantees that the characteristics of the participant are similar through the different groups at the baseline (2). However, the extent to which we can draw final conclusions based on RCTs strongly depends on how rigorous study methodology is; methodological inaccuracies during trial planning and conduct will subsequently reduce the reliability of results and their usability in medical practice (3). Biased results can finally lead to the underestimation or overestimation of the true intervention effect (3).

In cardiovascular disease (CVD) research, RCTs have been widely used to provide reliable knowledge on the best treatment strategies, such as therapies which are able to improve patient’s symptoms, correct disease markers, and improve clinical outcomes (4, 5). However, the risk of bias (RoB) has not been assessed in these studies.

The risk of bias (RoB) reflects the degree to which the results of a trial should be believed (6, 7). To reduce the possibility of RoB in RCT’s, the Cochrane Collaboration introduced a tool designed to appraise RoB (8, 9), involving six domains related to the internal validity of a trial: sequence generation, allocation concealment, blinding, incomplete outcome data, selective outcome reporting, and “other” potential threats to validity (6, 7). The risk of bias assessment enables the assessment of flaws in the trial design, conduct, and analysis that may affect study results (10).

In this study, we aimed to describe the reliability of evidence of cardiovascular diseases from a representative sample of cardiovascular RCTs published in 2017. Specific objectives were to examine: (1) the reliability of published cardiovascular trials using the RoB tool; (2) specific trial characteristics which increase the likelihood of unclear/high RoB; and (3) any potential differences in methodological issues between studies funded by the industry or the academy.

Materials and Methods

Sample Selection

We used the Cochrane CENTRAL Register of Controlled Trials to search for RCTs published in 2017 using subject headings and keywords related to adults (aged ≥ 18 years) and CVDs (such as, atherosclerosis, arrhythmia, cardiomyopathy, heart failure, hypertension, ischemic heart disease, heart attack, angina, sudden death, cardiac arrest, hypercholesterolemia, high blood pressure, CVD, ejection fraction, echocardiography, pericarditis, coronary artery disease, angioplasty, and angiography). The search and the screening of identified studies for eligibility were conducted by the first author (OB). Our search yielded a total of 2,556 studies (Supplementary File 1). Following deduplication, 2,419 studies underwent further analysis. Cochrane CENTRAL was the priority search source as it is the most comprehensive resource available of RCTs, containing publications from MEDLINE and EMBASE, as well as hand-search results, and gray literature (6, 11). Results of the search were randomly ordered in Excel, by the following method: after exporting the search result as an Excel file from Cochrane CENTRAL, we assigned a random number between 0 and 1 to each record using Excel’s random number generator, then reordered them from the smallest to the highest number (12). As a next step, we screened studies consecutively for eligibility, and the first 250 (∼10%) RCTs matching our pre-specified inclusion criteria were selected (Supplementary File 2). Trials were eligible for inclusion if they were published in the year 2017, were written in English, the described results of an RCT in the field of cardiovascular medicine, and included participants aged ≥ 18 years. Decision on the inclusion of a study was made after a careful consideration of the methodology in the full text.

Data Extraction

For data extraction, we used a data extraction sheet already tested and described in a previous study (11), data extracting guide available here: https://doi.org/10.1016/j.jpeds.2017.09.014 (12). The following data were extracted: journal type (e.g., specialty cardiovascular, or general medical), the publication details and characteristics of the published trials (such as study design, intervention, trial conduct, study sample, sample size, presence of a data monitoring committee, research outcomes, and conclusions). Further, we collected information about trial registration. Data extraction was completed by two reviewers (OB, OF): the first reviewer extracted the data and then, the second reviewer double-checked the sample. Conflicts were resolved through discussion and by reaching a consensus. Trial registration and protocol availability were investigated by retrieving information from the publications and via additional Internet searches (in Google and Google scholar). For the internet searches, we used the trial register number, the investigators’ names, and keywords describing the intervention or the condition.

Assessment of Methodological Quality and Reporting

We used the Cochrane RoB assessment tool (13) to evaluate the methodological quality of included RCTs. This tool assesses seven domains: (1) random sequence generation (whether the method used to generate the allocation sequence is described in sufficient detail to allow an assessment of whether it should produce comparable groups); (2) allocation concealment (whether the method used to conceal the allocation sequence is described in sufficient detail to determine whether intervention allocations could have been foreseen before or during enrollment); (3) the blinding of participants and personnel; (4) the blinding of outcome assessors; (5) incomplete outcome data (whether attrition and exclusions were reported, whether missing data were balanced across groups or were related to the outcomes); (6) selective reporting (whether pre-specified outcomes were reported in a pre-specified way); and (7) other bias.

We used the Cochrane RoB tool to assess RoB for the primary outcome. When the primary outcome was not clearly defined, we presumed it was the outcome either (1) described under aims/objectives of the study, (2) the outcome used to determine the sample size, or (3) the first outcome reported in the publication (the first applicable was used). One researcher performed a RoB assessment, while a second researcher was assigned to ensure the correctness of the assessments for each study.

Following Cochrane procedures (7), we classified each domain as low, unclear, or high risk. Then, the overall RoB was determined as follows: low when all domains were assessed as low RoB; unclear when at least 1 domain was assessed as unclear and no domains were assessed as high RoB; and high if any domain was assessed as high RoB (7).

Statistical Analysis

Statistical analyses were conducted by the statistical software R version 4.1.2 (14). The data were analyzed descriptively, using means and standard deviations (SDs) or medians and ranges for continuous variables and proportions for categorical variables. A multivariable logistic regression analysis was conducted to investigate the association between pre-specified study characteristics and the odds of high/unclear RoB. The value of p < 0.05 was considered as a significant result.

Results

Study Design and Reporting Characteristics of the Study Sample

Out of the 2,419 studies identified via search, we included the first 250 randomly selected trials, which met our search inclusion criteria as shown in Figure 1. The publication and trial characteristics of our sample are shown in Table 1. Most of the included trials had a parallel design (92.4%) and were efficacy trials (94.8%). Overall, 20.8% were placebo-controlled trials. An important part of the results of the trial was published in specialty cardiovascular journals (40.0%). In 139 studies (55.6%), the main goal was to evaluate the effects of pharmacological interventions. All geographic areas were represented; the majority of authors were from Europe (37.4%) and North America (26.0%).

FIGURE 1

Figure 1. Flowchart of study selection.

TABLE 1

Table 1. Publication and trial characteristics (N = 250).

The funding source was specified in 91.6% of the included trials: most of the trials were funded by an academic grant or a research institute (37.6%), while industrial and pharmaceuticals funding were reported in 23.2% of the trials.

When analyzing the main results of trials, we observed that at least one statistically significant result was reported in 86.0% of the studies; in these studies, the primary outcome was reported to be statistically significant in 69.2% of the cases. The treatment was favored in 55.6% and control in 3.6%. At least one adverse event was reported in 68% of the trials. A data monitoring committee was reported in 42% and sample size calculation in 60.4%. A total of 83.6% of the studies were registered in one of the clinical trials registries out of the 77.5% were registered in clinicaltrials.gov.

Risk of Bias Assessment

Table 2 shows the RoB assessment results. Overall, 29.2% of the studies were deemed as low RoB, while the remaining studies were at either unclear (39.6%) or high risk (31.2%). We rated the domains sequence generation, allocation concealment, and selective reporting to be the domains most often at high RoB (13.2, 9.6, and 10.4%, respectively).

TABLE 2

Table 2. Risk of bias (RoB) assessments by domain (N = 250).

We investigated whether the RoB was associated with the following variables: type of the intervention (drug vs. non-drug); single or multiple study centers; sample size; the presence of a Data Monitoring Committee; statistical significance of the primary outcome and trial registration (Table 3). Of these variables, trial registration influenced overall RoB to the greatest extent (odds ratio [OR] 0.06, 95% CI 0.03–0.31). Drug trials were more likely to have a low RoB than non-drug trials (OR 0.53, 95% CI 0.29–0.97), and multicenter trials more likely than single-center trials (OR 0.39, 95% CI 0.18–0.80). Other investigated variables did not have a significant influence on the RoB.

TABLE 3

Table 3. Multivariable regression analyses for all included trials, and trials with and without stated funding from the industry*.

We observed the following results after investigating individual RoB intems separately: drug trials (OR 0.39, 95% CI 0.22–0.66) and registered trials (OR 0.39, 95% CI 0.18–0.83) were more likely to have low RoB for random sequence generation. Drug trials (OR 0.51, 95% CI 0.28–0.93), registered trials (OR 0.49, 95% CI 0.26–0.91), and multicenter trials (OR 0.49, 95% CI 0.26–0.91) were more likely to have low RoB for allocation concealment, while trials with a statistically significant result were more likely to have unclear or high RoB (OR 2.59, 95% CI 1.34–5.31). Registered trials (OR 0.18, 95% CI 0.06–0.43), trials larger than 500 participants (OR 0.47, 95% CI 0.24–0.92), and trials with a Data Monitoring Committee (OR 0.50, 95% CI 0.28–0.87) had more often low RoB for the blinding of participants and personnel while the blinding of outcome assessors was more often low RoB in multicenter trials (OR 0.42, 95% CI 0.19–0.89) and registered trials (OR 0.27, 95% CI 0.12–0.63). There were no factors that increased the likelihood of low RoB for incomplete outcome data; however, trials with statistically significant results decreased the likelihood of low RoB for incomplete outcome data (OR 2.40, 95% CI 1.18–5.21). Larger trials with more than 500 participants were more likely to have low RoB for selective reporting (OR 0.44, 95% CI 0.19–0.95). Registered trials (OR 0.23, 95% CI 0.08–0.56) and multicenter trials (OR 0.31, 95% CI 0.16–0.59) were more likely to have low RoB for other biases.

Risk of Bias According to a Funding Source

When funding source was added as an additional independent variable to the multivariable regression model, funding did not seem to influence the likelihood of overall low RoB (industry funding: OR 0.76, 95% CI 0.40–1.45).

In the sub-group of industry-funded trials, multicenter trial and Data Monitoring Committee were factors that increased the likelihood of overall low RoB. None of the investigated factors influenced the overall RoB within the sub-group of trials with non-industry funding (Table 3).

Compared with non-industry funded studies more industry funded studies were rated as low RoB (84.9 vs. 63.9%) and less were rated as unclear (12.3 vs. 26.9%) or high RoB (2.8 vs. 9.2%) for allocation concealment (p < 0.001) (Table 4). More industry funded studies were rated low (66.0 vs. 42.9%) and fewer were rated as unclear (33.0 vs. 51.3%) or high risk (0.94 vs. 5.9%) for the blinding of participants and personnel (p < 0.001).

TABLE 4

Table 4. Risk of bias assessments by domain in studies funded by the industry or non-industry (N = 250).

Discussion

Summary of Main Findings

To our knowledge, this is the first research evaluating the RoB and its association with specific trial characteristics in a randomly selected sample of recently published clinical trials in adult cardiovascular disease. Included trials were mainly parallel RCTs investigating the efficacy of an intervention, with a very diverse trial scope and published in a variety of cardiovascular and general medical journals.

Of the 250 studies included, more than 85% have reported at least one statistically significant result, with the primary outcome significant in 69%. Treatment was favored in 55% of the studies, and adverse events were reported in 68%.

Less than one-third of our samples were overall low RoB, while the other two-thirds were unclear or high RoB. Sequence generation, allocation concealment, and selective reporting were the RoB domains most frequently rated at high risk. Trial registration influenced overall RoB to the greatest extent. Drug trials were more likely to be at low RoB than non-drug trials, and multicenter trials were more likely at a low risk RoB than single-center trials.

In the subgroup of industry-funded trials, multicenter trial and Data Monitoring Committee were factors that increased the likelihood of overall low RoB, while none of the investigated factors influenced the overall RoB within the subgroup of non-industry-funded trials. Significant differences were found in the RoB for the domains allocation concealment and the blinding of participants and personnel between industry-funded and non-industry-funded trials, with industry-funded trials more often rated at low risk.

Strengths and Weaknesses of the Study

We aimed to select a sample of studies representative for all randomized controlled trials published in 2017; we did not exclude published trials based on the country, type of journal, type of participants, or type of intervention. Included trials were randomly selected from all eligible trials. We used the most well-recognized tool for methodological assessment and the results indicated several areas of methodological weaknesses. To increase the reliability of findings, both data extraction and RoB assessment were conducted by two independent researchers.

This study was not pre-registered. Our study is limited by the included trials published in the English language only; conclusions cannot be generalized to trials published in other languages. We could not identify register entries or trial protocols for a subsample of trials, and it was difficult to properly evaluate the selective outcome reporting in these trials. We have not contacted the authors to get additional information about their trials, therefore the assessments are solely based on published information. Additionally, our study focused on the internal validity of trials, but have not assessed factors that may impact the external validity.

Discussion of Findings Considering Other Studies

Of the 250 analyzed trials, more than two-thirds (70.8%) were at high or unclear RoB, which is consistent with previous study results (15, 16). The RoB domains sequence generation, allocation concealment, and selective reporting were rated most often to be at high risk in our study.

Trial registration had the most beneficial effect on RoB in our study. In previous studies, random sequence generation, allocation concealment, and selective reporting were shown to differ between registered and unregistered gynecology and fertility trials (17). Registration was shown to be a factor influencing all RoB domains except selective outcome reporting in a sample of pediatric research trials (6). Trial registration is a factor that has the potential to facilitate higher methodological quality (18).

We found that drug trials were more likely at a low RoB than non-drug trials. This might be in connection with the strict regulations surrounding drug trials. Differences in protocol quality (19) registration and publication tendencies were described already in previous publications (20, 21), between trials with regulated and non-regulated interventions.

In our study, multi-center trials have demonstrated lower RoB compared with single-center trials. This is consistent with the conclusions of Landoni et al., who underlined bias issues characteristic for single-center trials (e.g., local effect bias, selection and performance bias, detection and reporting bias, analysis and attrition bias, concomitant therapy bias, low fragility index, and publication bias). Caution is advised for the results of single-center RCTs (22).

The source of funding may have an important impact on trial planning, conduct, and reporting (23–25). In our study, we investigated the role of funding sources on RoB by comparing trials both funded or not by the industry. In our study, industry-funded studies were rated more often low for the allocation concealment and the blinding of participants and personnel, as non-industry-funded studies. Our findigs on the higher probability of industry-funded trials to have adequate blinding are supported by the results of a similar study conducted in pediatric RCTs (6, 25). The blinding of participants and personnel may not always be feasible, however, studies should attempt blinding wherever possible.

Implications for Practice and Future Research

Our results underline the need of further improvement in the process of planning and performing a clinical trial in the field of CVD research. Trial registration was associated with a larger likelihood of low RoB, therefore mandatory trial registration should be endorsed and enforced by ethic committees, funders, and journal editors. Further favorable trial features associated with the lower RoB were multicenter trials, larger trials with more than 500 participants, and trials with a Data monitoring Committee. Trials not funded by the industry were more often at a high RoB for the allocation concealment and the blinding of participants and personnel, indicating, that studies without industry involvement need to pay greater attention to following certain methodological recommendations. The same is applicable for cardiovascular research trials investigating the effects of a non-drug intervention. In the present study, we focused on the methodological quality of CVD research trials but did not assess reporting issues in detail. This may need further assessed in the future.

Conclusion

Almost two-thirds of RCTs published in 2017 in the field of CVD research were at high or unclear RoB. This indicates a need for more rigorous trial planning and conduct. Prospective trial registration is a factor that predicts the lower RoB.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author Contributions

SL conceptualized the research, supervised the research, and reviewed and edited the manuscript. OB and OF performed the research. OB led data extraction and assessment, responsible for data curation, and wrote the first draft of the manuscript. DN conducted statistical analyses. All authors critically revised and approved the final version of the manuscript.

Funding

SL reports financial support from the Hungarian Academy of Sciences for EBM-related methodological research (János Bolyai Research Scholarship, BO/00498/17/5). The publication costs were funded by the Human Resource Development Operational Programme of Ministry for Human Capacities within the HRDOP-3.6.1-16-2016-00004 Comprehensive Development for Implementing Smart Specialization Strategies at the University of Pécs.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We would like to thank Kinga Amália Sándor-Bajusz for reviewing our manuscript for language accuracy.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fcvm.2022.830070/full#supplementary-material

Abbreviations

RCT, randomized-controlled trials; RoB, risk of bias; CVD, cardiovascular disease.

References

1. Altman D. Practical Statistics for Medical Research. London: Chapman and Hall (1991).

Google Scholar

2. Stolberg HO, Norman G, Trop I. Fundamentals of clinical research for radiologists. Am J Roentgenol. (2004) 183:1539–44. doi: 10.2214/ajr.180.4.1800917

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Higgins JPT, Altman DG, Sterne JAC. Chapter 8: assessing risk of bias in included studies. In: Higgins JPT, Green S editors. London: The Cochrane Collaboration (2011).

Google Scholar

4. Coca SG, Krumholz HM, Garg AX, Parikh CR. Underrepresentation of renal disease in randomized controlled trials of cardiovascular disease. J Am Med Assoc. (2006) 296:1377–84. doi: 10.1001/jama.296.11.1377

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Mathieu S, Boutron I, Moher D, Altman DG, Ravaud P. Comparison of registered and published primary outcomes in randomized controlled trials. JAMA. (2009) 302:977–84. doi: 10.1001/jama.2009.1242

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Hamm MP, Hartling L, Milne A, Tjosvold L, Vandermeer B, Thomson D, et al. A descriptive analysis of a representative sample of pediatric randomized controlled trials published in 2007. BMC Pediatr. (2010) 10:96. doi: 10.1186/1471-2431-10-96

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. Cochrane Handbook for Ssystematic Rreviews of Iinterventions 2nd ed. Chichester: John Wiley & Sons (2019).

Google Scholar

8. Jadad AR. Randomised controlled trials: a user’s guide. BMJ. (1998) 317:1258. doi: 10.1136/bmj.317.7167.1258

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. (1995) 273:408–12. doi: 10.1001/jama.1995.03520290060030

CrossRef Full Text | Google Scholar

10. Furuya-Kanamori L, Xu C, Hasan SS, Doi SA. Quality versus risk-of-bias assessment in clinical research. J Clin Epidemiol. (2021) 129:172–5. doi: 10.1016/j.jclinepi.2020.09.044

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Thomson D, Hartling L, Cohen E, Vandermeer B, Tjosvold L, Klassen TP. Controlled trials in children: quantity, methodological quality and descriptive characteristics of pediatric controlled trials published 1948-2006. PLoS One. (2010) 5:e13106. doi: 10.1371/journal.pone.0013106

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Gates A, Hartling L, Vandermeer B, Caldwell P, Contopoulos-Ioannidis DG, Curtis S. The conduct and reporting of child health research: an analysis of randomized controlled trials published in 2012 and evaluation of change over 5 years. J Pediatr. (2018) 193:237–244.e37. doi: 10.1016/j.jpeds.2017.09.014

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Higgins JPT, Savoviæ J, Page MJ, Elbers RG, Sterne JAC. Assessing risk of bias in a randomized trial. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, editors. Cochrane Handbook for Systematic Reviews of Interventions. Hoboken, NJ: Wiley-Blackwell (2019). p. 205–28. doi: 10.1002/9781119536604.ch8

CrossRef Full Text | Google Scholar

14. R Core Team. R: A Language and Environment for Statistical Computing. Vienna: R Foundation for Statistical Computing (2021).

Google Scholar

15. Pildal J, Hrobjartsson A, Jørgensen KJ, Hilden J, Altman DG, Gøtzsche PC. Impact of allocation concealment on conclusions drawn from meta–analyses of randomized trials. Int J Epidemiol. (2007) 36:847–57. doi: 10.1093/ije/dym087

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Viswanathan M, Ansari MT, Berkman ND, Chang S, Hartling L, McPheeters M, et al. Assessing the Risk of Bias of Individual Studies in Systematic Reviews of Health Care Interventions. Agency for Healthcare Research and Quality Methods Quide for Comparative Effectiveness Reviews. Rockville, MD: AHRQ Publication (2012).

Google Scholar

17. Farquhar CM, Showell MG, Showell EAE, Beetham P, Baak N, Mourad S, et al. Clinical trial registration was not an indicator for low risk of bias. J Clin Epidemiol. (2017) 84:47–53. doi: 10.1016/j.jclinepi.2016.11.011

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Roberts I, Ker K, Edwards P, Beecher D, Manno D, Sydenham E. The knowledge system underpinning healthcare is not fit for purpose and must change. BMJ. (2015) 350:h2463. doi: 10.1136/bmj.h2463

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Lohner S, Gryaznov D, von Niederhäusern B, Speich B, Kasenda B, Ojeda-Ruiz E, et al. Reporting quality of trial protocols improved for non–regulated interventions but not regulated interventions: a repeated cross–sectional study. J Clin Epidemiol. (2021) 139:340–9. doi: 10.1016/j.jclinepi.2021.05.011

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Rokhsefat S, Morra DE, Offringa M, Askie LM, Kelly LE. Trial registration in pediatric surgery trials. J Pediatr Surg. (2018) 53:1273–9. doi: 10.1016/j.jpedsurg.2017.10.049

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Dal-Ré R, Bracken MB, Ioannidis JPA. Call to improve transparency of trials of non–regulated interventions. BMJ. (2015) 350:h1323. doi: 10.1136/bmj.h1323

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Landoni G, Pieri M, Young PJ, Bellomo R. Why do multicenter randomized controlled trials not confirm the positive findings of single center randomized controlled trials in acute care? Minerva Anestesiol. (2019) 85:194–200. doi: 10.23736/S0375-9393.18.13070-7

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Blümle A, Wollmann K, Bischoff K, Kapp P, Lohner S, Nury E, et al. Investigator initiated trials versus industry sponsored trials – translation of randomized controlled trials into clinical practice (IMPACT). BMC Med Res Methodol. (2021) 21:182. doi: 10.1186/s12874-021-01359-x

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Fundytus A, Wells JC, Sharma S, Hopman WM, Del Paggio JC, Gyawali B. Industry funding of oncology randomised controlled trials: implications for design, results and interpretation. Clin Oncol. (2021) 34:28–35. doi: 10.1016/j.clon.2021.08.003

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Sismondo S. Pharmaceutical company funding and its consequences: a qualitative systematic review. Contemp Clin Trials. (2008) 29:109–13. doi: 10.1016/j.cct.2007.08.001

PubMed Abstract | CrossRef Full Text | Google Scholar