Taxiarchis Botsis
Robert Ball
G. Niklas Norén- 1The Sidney Kimmel Comprehensive Cancer Center, Johns Hopkins University School of Medicine, Baltimore, MD, United States
- 2United States Food and Drug Administration, Silver Spring, MD, United States
- 3Uppsala Monitoring Centre, Uppsala, Sweden
Editorial on the Research Topic
Computational methods and systems to support decision making in pharmacovigilance
The World Health Organization defines pharmacovigilance (PV) as “the science and activities related to the detection, assessment, understanding, and prevention of adverse events or any drug-related problem.” (World Health Organization, 2023) Government agencies, clinical institutions, the pharmaceutical industry, and other entities manage PV programs as vital safeguards for supporting the early detection and analysis of safety signals. These rely primarily on expert judgment and concrete business practices specific to each organization (Ball and Dal Pan, 2022). Computational methods and decision-support systems may enable more timely, consistent, and comprehensive analysis and processing of real-world data (RWD) and may free up time for domain experts to focus on higher-value contributions. As a positive side-effect, their development can help standardize business practices by specifying the steps human reviewers follow. In this Research Topic issue, we primarily invited papers that assess contributions to PV which may improve efficiency in established business practices, minimize manual effort, and maximize the quality of human decision-making by enhancing existing processes transparently. We also welcomed case studies of method implementation and perspectives that might significantly contribute to the domain.
Primary use cases in PV, such as case processing and prioritization or signal detection and evaluation, incorporate several steps that the complete decision-support system should augment. As data, user-related, and other challenges of the entire workflow vary, most efforts, to this date, have delivered solutions addressing more narrowly defined tasks. Historically, computational methods were initially proposed and deployed to enable signal detection and analysis in large databases using only structured data. More recently, several studies have presented methods for processing unstructured free texts, prioritizing case reports for clinical and regulatory review, and improving data quality in spontaneous reporting systems. For example, Painter et al. conducted a meta-analysis of a recent scoping review on machine learning in PV (Kompa et al., 2022) and found that the pharmaceutical industry mainly applied machine learning to process RWD and social media. They also highlighted the need to develop consistent systems that can learn and incorporate human-in-the-loop mechanisms and called for best practices for adopting and validating these systems.
To build effective solutions, existing workflows and business practices must be understood. These are rarely documented in stepwise algorithmic forms that can easily be translated into programmatic tools. It is, therefore, necessary to take a step back and, in close collaboration with the end users, identify the overall desired framework for a selected workflow. The papers in this Research Topic highlight several important aspects of this. Fong et al. pursued a strategy to support the medication error categorization workflow in the US Food and Drug Administration Adverse Event Reporting System (FAERS). Interestingly, they first defined a framework that helped them clarify the use case and then developed a prototype user-centered Artificial Intelligence (AI) application based on this framework. This process might work both ways, i.e., the first version of the framework implemented in a prototype tool could be informed by the pilot use of this prototype and, subsequently, updated to support new rounds of development.
Kreimeyer et al. focused on detecting duplicate cases in FAERS using the complete structured and free-text information from FAERS Individual Case Safety Reports (ICSRs). They built the confidence required to operationalize it by engaging safety reviewers in developing, evaluating, and validating their deduplication as a supplement to human review. Dang et al. developed a natural language processing approach to extract four demographic variables (gender, weight, ethnicity, and race) from FAERS ICSR narratives and complete the missing values in the corresponding structured fields. While their approach excellently retrieved ethnicity, gender, and race, it would meaningfully improve only gender information in the actual practice, as reporters infrequently provide weight, ethnicity, or race in the FAERS narratives. This demonstrates that operationalizing an AI solution must evaluate multiple factors, not solely performance.
Dimitriadis et al. presented an open-source platform that integrated ICSRs, synthetic clinical data, social media data from Twitter, and scientific literature from PubMed. Their prototype web-based tool allowed for various analyses in an interactive user interface. However, a comparison with the EudraVigilance Data Warehouse and Analysis System (EVDAS) provided by the European Medicines Agency revealed that PV experts preferred the existing technology. This study illustrates how implementing new tools in an actual work environment requires significant effort in presenting complete and validated systems and educating end users on their application.
When generative AI hits the headlines and heated discussions ensue (Stokel-Walker, 2023), it is essential to emphasize that PV is a tightly controlled, multi-step process. The rapid improvement of large language models across multiple applications should benefit PV, but to build trust with PV experts, we must systematically engage with users when collecting requirements and designing decision-support systems. Incorporating AI and other sophisticated algorithms into these systems requires knowledgeable interdisciplinary teams, including end-users and software developers, and several validation rounds aligned with the supported tasks and business workflows. Some of the studies in our Research Topic followed this strategy. This is the recipe for success in most cases as long as the right expectations are set from the beginning and the proposed solutions do not bring dramatic changes to existing workflows without proven benefit. Including a human-in-the-loop further ensures that a system’s limitations are carefully examined and that a quality assurance process to control them is in place. Ultimately, as no automated solution is likely to be perfect, decision-support systems must be constructed to take advantage of the respective strengths of machines and humans to improve efficiency, improve quality, and add value beyond what each might achieve on their own.
Author contributions
All authors equally contributed to this work and have approved its publication. The views expressed are those of the authors and not necessarily those of the US FDA.
Conflict of interest
RB and TB are authors on US Patent 9,075,796, “Text mining for large medical text datasets and corresponding medical text classification using informative feature selection.” At present, this patent is not licensed and does not generate royalties. GNN declares no conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Ball, R., and Dal Pan, G. (2022). Artificial intelligence" for pharmacovigilance: Ready for prime time? Drug Saf. 45 (5), 429–438. doi:10.1007/s40264-022-01157-4
Dang, V., Wu, E., Kortepeter, C. M., Phan, M., Zhang, R., Ma, Y., et al. (2022). Evaluation of a natural language processing tool for extracting gender, weight, ethnicity, and race in the US Food and drug administration adverse event reporting system. Front. Drug Saf. Regul. 2, 1020943. doi:10.3389/fdsfr.2022.1020943
Dimitriadis, V. K., Stella, D., Chytas, A., George, I. G., Kakalou, C., Panos, B., et al. (2022). An open-source platform integrating emerging data sources to support multi-modal active pharmacovigilance. Front. Drug Saf. Regul. 2, 1016042. doi:10.3389/fdsfr.2022.1016042
Fong, A., Bonk, C., Vasilchenko, V., De, S., Kovich, D., and Wyeth, J. (2022). Exploring opportunities for AI supported medication error categorization: A brief report in human machine collaboration. Front. Drug Saf. Regul. 2, 1021068. doi:10.3389/fdsfr.2022.1021068
Kompa, B., Hakim, J. B., Palepu, A., Kompa, K. G., Smith, M., Bain, P. A., et al. (2022). Artificial intelligence based on machine learning in pharmacovigilance: A scoping review. Drug Saf. 45 (5), 477–491. doi:10.1007/s40264-022-01176-1
Kreimeyer, K., Dang, O., Spiker, J., Gish, P., Weintraub, J., Wu, E., et al. (2022). Increased confidence in deduplication of drug safety reports with natural language processing of narratives at the US Food and drug administration. Front. Drug Saf. Regul. 2, 918897. doi:10.3389/fdsfr.2022.918897
Painter, J. L., Kassekert, R., and Bate, A. (2023). An industry perspective on the use of machine learning in drug and vaccine safety. Front. Drug Saf. Regul. 3, 1110498. doi:10.3389/fdsfr.2023.1110498
Stokel-Walker, C. (2023). ChatGPT listed as author on research papers: Many scientists disapprove. Nature 613, 620–621. doi:10.1038/d41586-023-00107-z
World Health Organization (2023). What is pharmacovigilance? Available at: https://www.who.int/teams/regulation-prequalification/regulation-and-safety/pharmacovigilance. (Accessed March 31, 2023).
Keywords: pharmacovigilance, decision support, artificial intelligence, quality assurance, real-world data
Citation: Botsis T, Ball R and Norén GN (2023) Editorial: Computational methods and systems to support decision making in pharmacovigilance. Front. Drug Saf. Regul. 3:1188715. doi: 10.3389/fdsfr.2023.1188715
Received: 17 March 2023; Accepted: 14 April 2023;
Published: 21 April 2023.
Edited and Reviewed by:
Gianluca Trifirò, University of Verona, ItalyCopyright © 2023 Botsis, Ball and Norén. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Taxiarchis Botsis, dGJvdHNpczFAamhtaS5lZHU=