Valton Costa
Marcelo Graziano Custodio3*
Felipe Fregni
94% of researchers rate our articles as excellent or good
Learn more about the work of our research integrity team to safeguard the quality of each article we publish.
Find out more
- 1Neuromodulation Center and Center for Clinical Research Learning, Spaulding Rehabilitation Hospital, Massachusetts General Hospital, Harvard Medical School, Boston, MA, United States
- 2Laboratory of Neuroscience and Neurological Rehabilitation, Physical Therapy Department, Federal University of Sao Carlos, São Carlos, Brazil
- 3Global Innovation and Development, Established Pharmaceuticals Division, Abbott Products Operations AG, Allschwil, Switzerland
The scientific method has been established as the optimal approach for systematically gathering and interpreting data on various human phenomena, mainly through the adoption of strict experimental methods, such as controlled randomized trials, which is relevant for clinical decision-making and research, but also weights the regulatory processes for approval of drugs and other medical products. However, a key factor in strict methods is the generalizability of findings that may be limited to specific settings and patient characteristics. This limitation can be addressed by non-experimental methods aimed at investigating populations in naturalistic routine clinical settings, which may offer a more representative reflection of the usage, effectiveness, and safety of healthcare interventions. These approaches can generate the so-called real-world data and the resulting real-world evidence. In this narrative review, we present these concepts, explore the potential applications and advantages of real-world evidence for clinical, research, and regulatory decision-making, and discuss the challenges of employing it, the solutions to improve its generation, and lastly the current level of evidence required for the integration of real-world evidence into regulatory decision-making. We concluded that the advantages of using it, when utilized in a balanced manner, overcome the challenges and, therefore, can offer a time- and cost-saving solution for researchers, the healthcare industry, regulatory agencies, policymakers, and payers towards the patient benefit. The knowledge generated by this approach provides valuable additional insights into medical interventions in real patients under realistic daily scenarios.
1 Introduction
The scientific method has long been established as the optimal approach for systematically gathering and interpreting data on various natural phenomena, including human health. Through experimentation and observation, this process transforms raw data into valuable information, which accumulates and forms the basis of evidence. Scientific knowledge plays a crucial role in generating health information and guiding clinical decision-making for the benefit of individuals and communities. This foundation is encapsulated in the concept of Evidence-Based Medicine, which emphasizes the integration of scientific evidence, clinician expertise, and patient preferences in the decision-making process (1). In research, the accuracy of subsequent steps hinges on the quality of data collection. Poor data collection may result in false assumptions, ultimately leading to healthcare malpractices. To mitigate these risks, clinical research has traditionally adhered to a positivist view, aiming to control for predictable biases and measure outcomes as objectively as possible.
Historically, the gold standard for measuring the effects and efficacy of interventions has been the experimental trials with a randomized and controlled design (RCT). RCTs are widely recognized for ensuring rigorous control over experimental conditions, establishing causality between interventions and outcomes, and minimizing bias in comparing treatments (2, 3). However, there exists a trade-off between experimental control and the generalizability of findings. While well-designed and adequately powered RCTs provide accurate results based on systematic data collection and analysis, their generalizability may be limited to specific settings or patient characteristics defined by study protocols. This limitation can fail to capture the heterogeneity of the real-world population and clinical settings in various aspects (2, 4, 5). In everyday clinical practice, patients are not constrained by study protocols or ideal scenarios; they exhibit biological and demographic heterogeneity (6). Therefore, data collected from patients’ every day lives may offer a more representative reflection of the usage, effectiveness, and safety of healthcare interventions.
Although not entirely precise in its terminology, “real-world data” (RWD) refers mainly to data collected in clinical contexts regarding patient health status and healthcare delivery, derived from routine healthcare practices rather than controlled research settings (4, 7). Certainly, all data collected from real people, whether in research settings or daily clinical practice, are RWD. However, this term emphasizes the sources and settings of data collection, indicating that it originates from subjects in naturalistic situations. This terminology has been widely adopted and consistently used by major regulatory bodies and agencies worldwide (2, 8–10). Consequently, the structured evidence derived from the analysis of this type of data is referred to as “real-world evidence” (RWE).
RWD and the resulting evidence have been extensively utilized by the pharmaceutical industry and developers of healthcare technologies to gain valuable insights into customer desires, needs, and preferences on health products and healthcare services throughout various stages of the product lifecycle. Additionally, RWD is increasingly valuable in other settings, including research on the development of new drugs and healthcare materials, to inform study design and gather insights on the efficacy and safety of health products in real-world settings (see Table 1). RWE can be generated prospectively or retrospectively from observational studies and other sources of RWD. Consequently, regulatory agencies, particularly in high-income countries, where such data is more frequently collected and systematically analyzed, have begun considering RWE in guiding marketing approval and regulatory decisions, as well as monitoring the long-term usage, safety, and effectiveness of drugs and medical technologies (11). In this paper, we aim to explore the potential applications of RWD and RWE, as well as the challenges associated with improving data collection and analysis. We will also examine their utility in clinical decision-making, regulatory purposes, and as a complementary strategy to conventional experimental research for generating supportive evidence.
Table 1
Table 1. Real-world data potential use in healthcare.
2 Sources of RWD
Sources of RWD encompass healthcare databases such as electronic health records and registries, health insurance systems, wearable and mobile devices, social media, and patient-led network platforms (12, 13). They provide valuable insights into healthcare utilization, treatment efficacy, adverse effects, and the natural history of certain conditions. Healthcare databases may document routine clinical appointments, laboratory tests, and medication prescriptions. Registries, whether local or national, compile extensive data on treatment delivery for specific populations and may maintain biobanks. Examples of such sources include the European Cystic Fibrosis Society registries, the British Society for Rheumatology’s national registry (patients on biologic therapy), the National Cancer Registration and Analysis Service, the Clinical Practice Research Datalink and the QResearch in the UK, and the American Academy of Orthopedic Surgeons registry (orthopedic care) (9). Additionally, administrative databases, particularly in the context of health insurance and pharmacological dispensation, offer crucial information on healthcare costs and the long-term effects of health interventions throughout inpatient and outpatient services, surgeries, hospitalizations, diagnostics, laboratory, and pharmacy services. Medicare and Medicaid in the USA and the Ontario Pharmacy Evidence Network in Canada are examples of such databases, which utilize RWD to optimize health information and service delivery (9).
Furthermore, emerging sources of health-related RWD include social media and social network forums, where patients actively contribute data by sharing their treatment experiences, medication adherence, and reasons for avoiding treatment (13). Patient-driven networks, which may include clinicians and researchers, have also emerged to provide platforms for sharing health-related information and personal treatment experiences. Examples of such networks include PatientsLikeMe and the National Patient-Centered Clinical Research Network in the USA, which supports multiple disease studies, and the Accelerated Cure Project, which shares clinical data and biosamples for multiple sclerosis research.
3 Potential uses of RWE
3.1 Advantages of RWD
Many of the advantages of RWD studies stem from the limitations of RCTs. RWD encompasses data collected from groups that are often underrepresented in research due to either ethical, costs, timelines, or strategic considerations, such as children, adolescents, pregnant women, the older adult, and individuals with multiple comorbidities and vulnerabilities (4, 14, 15). It provides insights into how medical interventions perform in daily life scenarios, capturing the complexities and variations in effectiveness encountered in routine clinical practice. These variations may include the challenges faced by healthcare professionals, such as different treatment regimens, patient adherence, duration of follow-up, healthcare delivery dynamics, and resource availability (6). RWD studies often involve larger datasets than RCTs, facilitating subgroup analysis, modeling, and the generalizability of findings (15, 16). Moreover, RWD studies complement traditional evidence from RCTs by uncovering information that may not be attainable through more controlled methods (see Table 2) (3, 17). This approach directly benefits patients by addressing unmet needs often overlooked in conventional healthcare delivery, improving understanding of off-label applications, and identifying the risks and limitations of products and services for community health—all in a more time-efficient manner.
Table 2
Table 2. The advantages and challenges of utilizing RWE.
RWD has been extensively utilized in post-marketing approval pharmacovigilance to monitor long-term drug safety and efficacy. For instance, the USA Food and Drug Administration (FDA) does this through the Sentinel Initiative System (18). Observational studies offer a significant advantage in conducting safety evaluations of drugs and other health products. These studies often utilize retrospective data collected from electronic health records, disease registries, and claim databases, ensuring that data collection is not influenced by the a priori objectives of the study. Furthermore, RWD can be collected through various observational designs, e.g., cohort studies, surveys, and case–control, either prospectively or retrospectively. Alternatively, RWD can be collected in the context of pragmatic clinical trials, which emphasize the naturalistic behavior of patients in using a given treatment. Pragmatic trials might encompass relevant population groups, experienced investigators, standard care as control, and outcomes relevant to the population under study (15, 19). Conversely, retrospective analysis offers a time-saving alternative to the lengthy trial phases of drug testing, providing urgent responses to gaps in the healthcare system. Although retrospective analysis can offer reliable insights into the safety of medications in conditions or subgroups lacking RCT-based evidence, insights need to be interpreted with caution in cases where a given event cannot be compared to a control group (13, 16, 20).
Moreover, RWD can provide valuable insights for guiding early-phase research in the development of medical products. Some sources can contribute patient data for integrating an external comparator in single-arm trials or exploring various clinical outcomes, such as long-term and quality-of-life outcomes and adverse events, which are often overlooked in RCTs. Leveraging RWD in this manner can improve care standards, support epidemiological research, expedite public health policies, and save time and resources (2, 4, 21, 22). Insights from RWD analysis can inform internal research and business decisions within the healthcare industry, guide payers, and facilitate cost analysis (23, 24).
3.2 Challenges of using RWD and possible solutions
RWD often lacks the rigor and standardization observed in RCTs. This results in inconsistent data, compromising the reliability of findings. Confounding factors may be introduced, undermining internal validity. Addressing these issues requires improved data collection, completeness, and consistency (13). Likewise, integrating diverse sources of RWD, such as electronic health records and administrative databases, is complex and time-consuming due to variations in data formats and coding systems. Lack of standardization and interoperability hinders data aggregation and analysis (see Table 2). Efforts to standardize data collection practices and improve interoperability can enhance the usefulness of RWD (12, 25).
RWD studies are susceptible to selection bias and confounding factors due to the non-randomized nature of data collection in real-world settings. Factors such as patient demographics, healthcare-seeking behavior, and provider preferences may influence the likelihood of certain individuals being included in analyses, potentially skewing study results and conclusions. Controlling for these factors requires sophisticated statistical techniques, consideration of study design, and analytical methods. Some advanced techniques, such as propensity score matching, inverse probability weighting, stratification for the generation of statistically comparable controls, sensitivity analyses, and language processing tools, can be useful as it has been used previously with relative success (4, 26–29). Creating interconnected networks between research centers and pharmaceutical and biomedical companies can contribute to the quality and accuracy of RWD. They can collectively store and analyze larger cohort data and share tools and insights for accurate data analysis, which can be integrated into subsequent study designs (30).
RWD often contains sensitive patient information, raising concerns about data privacy and security. Safeguarding patient privacy and complying with data protection regulations, such as the Health Insurance Portability and Accountability Act in the USA and the General Data Protection Regulation in the European Union, present significant challenges for researchers and organizations handling RWD. It may raise the complexity level for the development of strategic therapies for the entire population based on restrictions on data access. However, ensuring adequate technical solutions for data anonymization, encryption, and access controls is essential to mitigate the risk of unauthorized breaches and protect patient confidentiality.
Compliance with study design guidelines and ethical principles is essential to maintain the integrity of RWD studies and protect the rights and welfare of research participants. A few recommendations may help to improve the utilization of RWD and the generation of evidence: (i) improvement of electronic data recordings for research purposes; (ii) enhancing completeness, consistency, and accuracy of data collection and storage (10); (iii) improvement of the interoperability and exchange of electronic health information, facilitating data utilization for different stakeholders; (iv) planning of rigorous observational designs, identification of appropriate databases and proper study registration (28); and (v) transparent and complete reporting of studies, especially after marketing approval of health products, which can be achieved by adopting known guidelines, such as the STROBE for observational studies and the CONSORT for pragmatic trials, potentially improving reporting and validity (16, 31).
4 Regulatory standards for the use of RWE
Regulatory bodies and stakeholders, including legislative actors, have integrated RWE into regulatory decisions concerning medical devices, drugs, and health materials. This integration necessitates rigorous analytic methodology to establish the efficacy and safety of these products. Consequently, regulatory bodies and research centers, particularly in the USA, Europe, and some parts of the world, have taken action to establish optimal utilization of this evidence for regulatory purposes. They have produced guidelines and initiatives to generate standards of practice in using RWE (2, 10, 32–34). However, there is still disagreement regarding the stage at which it should be incorporated (35). Some advocate for its use after marketing approval for surveillance purposes, while others present cases of its use in both pre- and post-approval stages, for instance, to regulate orphan drugs. Instances of drug approval without RCTs by the European Medicine Agency (EMA) and FDA have been reported (10, 16, 36).
The integration of RWE into the regulatory process remains a topic of debate. While some argue for the sufficiency of RWE to support the efficiency and safety of health products, most literature emphasizes its use as a complement to RCTs (4, 9, 12, 35). Before RCTs, it can generate hypotheses to be further investigated in stricter designs aiming to establish efficacy. After RCTs, it holds the potential for evaluating the transferability of novel treatments and findings to broader populations, facilitating external validity. Independently, RWE can be used to assess long-term efficacy, safety, and other outcomes of interest, especially when RCTs are not feasible (28).
The design of studies with hybrid methodologies, combining aspects of RCTs and observational studies, offers promise. For example, using pragmatic trial approaches and external controls in single-arm trials can merge practical aspects with statistical rigor. How does this impact the regulatory process? There is a growing trend in regulatory drug approval processes to embrace RWE in decision-making. This is evident in cases from various parts of the world where RWE has been used to accelerate regulatory approval, extend drug-approved indications, add safety information to drug labels, revoke drug usage, or implement changes. Regulatory agencies and pharmaceutical companies have shown openness to utilizing RWE to find accelerated solutions to meet health system needs, particularly in the development, approval, effectiveness, and safety evaluation of drugs for broader or specific populations, combining RCT results with further evidence from RWD analysis beyond Phase IV trials (2, 21, 36–39).
5 Level of evidence from RWD studies
In scientific research, evidence is hierarchically classified based on ranking systems that assess the reliability, validity, and applicability of evidence from different study designs. While classification systems may vary, they generally prioritize designs with better internal validity, bias control, and the ability to establish causality between interventions and outcomes. According to common evidence evaluation systems, RCTs and systematic reviews based on valid RCTs are on top of the hierarchy (level 1 and 2 evidence), while observational studies, case–control studies, and studies with external controls fall at level 4, positioned between non-randomized controlled cohorts (level 3) and expert opinion and case reports (level 5) (1, 40). Interestingly, a systematic review of nested case–control studies or an observational study with dramatic effects for common harms is considered at the highest hierarchical level (40).
This hierarchical view of evidence guides decisions made in clinical settings for patient treatment, resource allocation, and research implementation, though current works in evidence hierarchy are aware of the complexity of this view. Nonetheless, it is an integral aspect of market authorization of new medical products, serving to prevent ineffective or harmful products from entering the market (41). Decisions are made based on the expected and available evidence and may be subject to regulatory standards about the type and quantity of research supporting the authorization application. Historically, market authorization has predominantly relied on RCTs, particularly for assessing drug effectiveness (42). Therefore, incorporating other study designs into the process presents challenges, as it may be perceived to reduce the reliability of evidence. Regulatory agencies, such FDA and EMA are increasingly highlighting the relevance of RWE in their decision-making processes, complementing evidence from other sources, and supporting pre-authorization and post-approval assessments (8, 18, 34, 42).
Therefore, if different study designs are to be considered solely or in combination with traditional RCTs, they need to be part of an equally reliable process, that is, with a similar level of evidence and the same rates of false positive and false negative cases—the proportion of products that will be granted a license or approval but have not the intended clinical effect or disapproved but the intended benefit—expected for the conventional use of RCT evidence to approve a drug or medical product (41). While there is a willingness to integrate RWE into marketing approval and decision-making, there are inconsistent perceptions regarding what constitutes adequate levels of evidence among regulators and stakeholders. Observational studies face challenges in determining the strength of evidence, particularly in controlling bias, especially in prospective designs. However, we do not have evidence to support the notion that, for relevant outcomes such as treatment effects and adverse events, there are robust differences between RCTs and well-designed observational studies in terms of overestimation and quality of the results, though specific observational designs like case–control may tend to overestimate harm as compared to cohort and cross-sectional studies (43–46).
Thus, an important question is how many studies and what kind of RWD study are necessary, with an emphasis on the quality of the primary data and the expected methodology that must be deemed as convincing evidence for regulatory and licensing purposes. The answer to that question may vary depending on circumstances, but it is agreed that decisions should ideally be based on multiple and cumulative sources of evidence, encompassing experimental and observational studies, retrospective and prospective designs, and pragmatic considerations (2, 20, 41). As new types and sources of RWD become widely available—mostly in the form of big data—regulatory agencies, policymakers, clinicians, and researchers must collaborate to establish standards for regulatory decisions, determining the quantity and quality of studies deemed as high-level evidence.
6 Conclusion
The understanding of real-world evidence has gained traction in recent years, with an increasing number of possible uses, from healthcare product development and approval to the final clinical decision and guidance of public policies. The advantages of RWE use, when utilized in a balanced manner, overcome the challenges and, therefore, can offer a time- and cost-saving solution for researchers, the healthcare industry, regulatory agencies, policymakers, and payers towards the patient benefit. Future research should explore strategies to integrate diverse RWD sources to build robust databases that enhance utility and generalizability for stakeholders. Promising methodologies, such as artificial intelligence-powered approaches to ensure validity, hybrid research designs, and large-scale longitudinal and multicenter RWD cohorts, could significantly contribute to the development and credibility of this type of evidence.
Author contributions
VC: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Visualization, Writing – original draft, Writing – review & editing. MC: Conceptualization, Formal analysis, Funding acquisition, Methodology, Project administration, Resources, Visualization, Writing – review & editing. EG: Data curation, Funding acquisition, Methodology, Resources, Writing – review & editing. FF: Conceptualization, Data curation, Methodology, Project administration, Supervision, Writing – review & editing.
Funding
The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This work received funding from Abbott in the form of payment of the publication processing fees. This funder was not involved in the study design, collection, analysis, interpretation of data, and the writing of this article. This study was also supported by a grant from Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP), SPEC project, grant number 2017/12943-8. VC is a fellow of the Institutional Internationalization Program (CAPES/PrInt/UFSCar) funded by the Brazilian Federal Agency for Support and Evaluation of Graduate Education (CAPES/Ministry of Education of Brazil).
Conflict of interest
MC and EG were employed by Abbott Products Operations AG.
The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The author(s) declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision.
Generative AI statement
The author(s) declare that no Generative AI was used in the creation of this manuscript.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1. Greenhalgh, T, Howick, J, and Maskrey, N. Evidence based medicine: a movement in crisis? BMJ. (2014) 348:g3725. doi: 10.1136/bmj.g3725
2. Baumfeld Andre, E, Reynolds, R, Caubel, P, Azoulay, L, and Dreyer, NA. Trial designs using real-world data: the changing landscape of the regulatory approval process. Pharmacoepidemiol Drug Saf. (2020) 29:1201–12. doi: 10.1002/pds.4932
3. Blonde, L, Khunti, K, Harris, SB, Meizinger, C, and Skolnik, NS. Interpretation and impact of real-world clinical data for the practicing clinician. Adv Ther. (2018) 35:1763–74. doi: 10.1007/s12325-018-0805-y
4. Maissenhaelter, BE, Woolmore, AL, and Schlag, PM. Real-world evidence research based on big data: motivation-challenges-success factors. Onkologe (Berl). (2018) 24:91–8. doi: 10.1007/s00761-018-0358-3
5. Barnish, MS, and Turner, S. The value of pragmatic and observational studies in health care and public health. Pragmat Obs Res. (2017) 8:49–55. doi: 10.2147/POR.S137701
6. Fortin, M, Dionne, J, Pinho, G, Gignac, J, Almirall, J, and Lapointe, L. Randomized controlled trials: do they have external validity for patients with multiple comorbidities? Ann Fam Med. (2006) 4:104–8. doi: 10.1370/afm.516
7. Makady, A, de Boer, A, Hillege, H, Klungel, O, and Goettsch, W. (on behalf of GetReal work package 1). What is real-world data? A review of definitions based on literature and stakeholder interviews. Value Health. (2017) 20:858–65. doi: 10.1016/j.jval.2017.03.008
8. United States Food and Drug Administration (2024). Available at: https://www.fda.gov/ScienceResearch/SpecialTopics/RealWorldEvidence/default.htm.(Accessed April 23, 2024).
9. Dang, A. Real-world evidence: a primer. Pharmaceut Med. (2023) 37:25–36. doi: 10.1007/s40290-022-00456-6
10. Nishioka, K, Makimura, T, Ishiguro, A, Nonaka, T, Yamaguchi, M, and Uyama, Y. Evolving acceptance and use of RWE for regulatory decision making on the benefit/risk assessment of a drug in Japan. Clin Pharmacol Ther. (2022) 111:35–43. doi: 10.1002/cpt.2410
11. Rudrapatna, VA, and Butte, AJ. Opportunities and challenges in using real-world data for health care. J Clin Invest. (2020) 130:565–74. doi: 10.1172/JCI129197
12. Dagenais, S, Russo, L, Madsen, A, Webster, J, and Becnel, L. Use of real-world evidence to drive drug development strategy and inform clinical trial design. Clin Pharmacol Ther. (2022) 111:77–89. doi: 10.1002/cpt.2480
13. Chodankar, D. Introduction to real-world evidence studies. Perspect Clin Res. (2021) 12:171–4. doi: 10.4103/picr.picr_62_21
14. Tan, YY, Papex, V, Chang, WH, Mueller, SH, Denaxas, S, and Lai, A. Comparing clinical trial population representativeness to real-world populations: an external validity analysis encompassing 43895 trials and 5685 738 individuals across 989 unique drugs and 286 conditions in England. Lancet Healthy Longev. (2022) 3:e674–89. doi: 10.1016/S2666-7568(22)00186-6
15. Ford, I, and Norrie, J. Pragmatic trials. N Engl J Med. (2016) 375:454–63. doi: 10.1056/NEJMra1510059
16. Visvanathan, K, Levit, LA, Raghavan, D, Hudis, CA, Wong, S, Dueck, A, et al. Untapped potential of observational research to inform clinical decision making: American Society of Clinical Oncology research statement. J Clin Oncol. (2017) 35:1845–54. doi: 10.1200/JCO.2017.72.6414
17. Booth, CM, and Tannock, IF. Randomised controlled trials and population-based observational research: partners in the evolution of medical evidence. Br J Cancer. (2014) 110:551–5. doi: 10.1038/bjc.2013.725
18. United States Food and Drug Administration (2019) Sentinel system: five-year strategy 2019-2023. U.S. Food and Drug Administration p. 29.
19. Sox, HC, and Lewis, RJ. Pragmatic trials: practical answers to “real world” questions. JAMA. (2016) 316:1205–6. doi: 10.1001/jama.2016.11409
20. Dreyer, NA. Strengthening evidence-based medicine with real-world evidence. Lancet Healthy Longev. (2022) 3:e641–2. doi: 10.1016/S2666-7568(22)00214-8
21. Frieden, TR. Evidence for health decision making - beyond randomized, controlled trials. N Engl J Med. (2017) 377:465–75. doi: 10.1056/NEJMra1614394
22. Sherman, RE, Anderson, SA, Dal Pan, GJ, Gray, GW, Gross, T, Hunter, NL, et al. Real-world evidence - what is it and what can it tell us? N Engl J Med. (2016) 375:2293–7. doi: 10.1056/NEJMsb1609216
23. Malone, DC, Brown, M, Hurwitz, JT, Peters, L, and Graff, JS. Real-world evidence: useful in the real world of US payer decision making? How? When? And what studies? Value Health. (2018) 21:326–33. doi: 10.1016/j.jval.2017.08.3013
24. Garrison, LP Jr, Neumann, PJ, Erickson, P, Marshall, D, and Mullins, CD. Using real-world data for coverage and payment decisions: the ISPOR real-world data task force report. Value Health. (2007) 10:326–35. doi: 10.1111/j.1524-4733.2007.00186.x
25. Rajivlochan, M. Clinical audits and the state of record keeping in India. Ann Neurosci. (2015) 22:197–8. doi: 10.5214/ans.0972.7531.220402
26. Kuss, O, Blettner, M, and Börgermann, J. Propensity score: an alternative method of analyzing treatment effects. Dtsch Aerztebl Int. (2016) 113:597–603. doi: 10.3238/arztebl.2016.0597
27. McMurry, TL, Hu, Y, Blackstone, EH, and Kozower, BD. Propensity scores: methods, considerations, and applications. J Thorac Cardiovasc Surg. (2015) 150:14–9. doi: 10.1016/j.jtcvs.2015.03.057
28. Roche, N, Reddel, H, Martin, R, Brusselle, G, Papi, A, Thomas, M, et al. Quality standards for real-world research. Focus on observational database studies of comparative effectiveness. Ann Am Thorac Soc. (2014) 11:S99–S104. doi: 10.1513/AnnalsATS.201309-300RM
29. Hassett, MJ. Quality improvement in the era of big data. J Clin Oncol. (2017) 35:3178–80. doi: 10.1200/JCO.2017.74.1181
30. Elliott, JH, Grimshaw, J, Altman, R, Bero, L, Goodman, SN, Henry, D, et al. Make sense of health data: develop the science of data synthesis to join up the myriad varieties of health information. Nature. (2015) 527:31–2. doi: 10.1038/527031a
31. Equator Network (2024). Enhancing the quality and transparency of health research. Available at: https://www.equator-network.org/ (Accessed April 29, 2024).
32. Berger, ML, Sox, H, Willke, RJ, Brixner, DL, Eichler, HG, Goettsch, W, et al. Good practices for real-world data studies of treatment and/or comparative effectiveness: recommendations from the joint ISPOR-ISPE special task force on real-world evidence in health care decision making. Pharmacoepidemiol Drug Saf. (2017) 26:1033–9. doi: 10.1002/pds.4297
33. Duke-Margolis Center for Health Policy (2017). A framework for regulatory use of real-world evidence. Available at: https://healthpolicy.duke.edu/sites/default/files/2020-08/rwe_white_paper_2017.09.06.pdf (Accessed August 24, 2024).
34. European Medicine Agency (2021). Guideline on registry-based studies. Available at: https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-registry-based-studies_en.pdf-0 (Accessed April 24, 2024).
35. Graili, P, Guertin, JR, Chan, KKW, and Tadrous, M. Integration of real-world evidence from different data sources in health technology assessment. J Pharm Pharm Sci. (2023) 26:11460. doi: 10.3389/jpps.2023.11460
36. Hatswell, A, Baio, G, Berlin, J, Irs, A, and Freemantle, N. Regulatory approvalof pharmaceuticals without a randomised controlled study: analysis of EMA and FDA approvals 1999–2014. BMJ Open. (2016) 6:e011666. doi: 10.1136/bmjopen-2016-011666
37. Khosla, S, White, R, Medina, J, Ouwens, M, Emmas, C, Koder, T, et al. Real world evidence (RWE) - a disruptive innovation or the quiet evolution of medical evidence generation? F1000Res. (2018) 7:111. doi: 10.12688/f1000research.13585.2
38. Pottegård, A, Klungel, O, Winterstein, A, Huybrechts, K, Hallas, J, Schneeweiss, S, et al. The International Society for Pharmacoepidemiology's comments on the core recommendations in the summary of the heads of medicines agencies (HMA) - EMA joint big data task force. Pharmacoepidemiol Drug Saf. (2019) 28:1640–1. doi: 10.1002/pds.4911
39. Silverman, B. (2018). PharmaIntelligence pink sheet. A baker's dozen of USFDA efficacy approvals using real world evidence. Available at: https://pink.pharmaintelligence.informa.com/PS123648/A-Bakers-Dozen-Of-US-FDA-Efficacy-Approvals-Using-Real-World-Evidence (Accessed April 24, 2024).
40. OCEBM (2011). Levels of evidence working group. The Oxford levels of evidence 2. Oxford Centre for Evidence-Based Medicine Available at: https://www.cebm.ox.ac.uk/resources/levels-of-evidence/ocebm-levels-of-evidence (Accessed at August 12, 2024).
41. Slattery, J, and Kurz, X. Assessing strength of evidence for regulatory decision making in licensing: what proof do we need for observational studies of effectiveness? Pharmacoepidemiol Drug Saf. (2020) 29:1336–40. doi: 10.1002/pds.5005
42. United States Food and Drug Administration (2024). Real-world data: assessing electronic health records and medical claims data to support regulatory decision-making for drug and biological products - guidance for industry Available at: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/real-world-data-assessing-electronic-health-records-and-medical-claims-data-support-regulatory (Accessed August 12, 2024).
43. Golder, S, Loke, YK, and Bland, M. Meta-analyses of adverse effects data derived from randomised controlled trials as compared to observational studies: methodological overview. PLoS Med. (2011) 8:e1001026. doi: 10.1371/journal.pmed.1001026
44. Golder, S, Loke, YK, and Bland, M. Comparison of pooled risk estimates for adverse effects from different observational study designs: methodological overview. PLoS One. (2013) 8:e71813. doi: 10.1371/journal.pone.0071813
45. Concato, J, Shah, N, and Horwitz, RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med. (2000) 342:1887–92. doi: 10.1056/NEJM200006223422507
Keywords: real-world data, real-world evidence, clinical research, regulatory process, decision-making, evidence-based medicine
Citation: Costa V, Custodio MG, Gefen E and Fregni F (2025) The relevance of the real-world evidence in research, clinical, and regulatory decision making. Front. Public Health. 13:1512429. doi: 10.3389/fpubh.2025.1512429
Edited by:
Christopher Tompkins, Brandeis University, United StatesReviewed by:
Chen Wang, Mayo Clinic, United StatesCopyright © 2025 Costa, Custodio, Gefen and Fregni. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Marcelo Graziano Custodio, bWFyY2Vsby5ncmF6aWFub0BhYmJvdHQuY29t