Paul Rogers
Dong Wang
Zhiyuan Lu- Division of Bioinformatics and Biostatistics, FDA National Center for Toxicological Research, Jefferson, AR, United States
Introduction and Background
The Medical Information Mart for Intensive Care (MIMIC) is a database of de-identified electronic health records (EHR) associated with patients who stayed in intensive care units (ICU) at the Beth Israel Deaconess Medical Center in Boston, MA (Johnson et al., 2016). Currently on its fourth iteration, the database is maintained by the Massachusetts Institute of Technology’s Laboratory for Computational Physiology. With time frames and release dates shown in Table 1, all four MIMIC versions are accessible and available to the public, supporting the concept of reproducibility within ICU research (Johnson et al., 2018). Among these, MIMIC-III alone contains 53,423 distinct adult hospital admissions to one of the five different ICU departments at the Beth Israel Deaconess Medical Center between 2001 and 2012 (Johnson et al., 2016). These specialized ICU departments include Coronary Care, Cardiac Surgery Recovery, Medical Intensive Care, Surgical Intensive Care, and Trauma Surgical Intensive Care units. MIMIC is populated with data from hospital electronic health records, automated critical care information systems, and the Social Security Administration Death Master File. The two critical care information systems which provided time-stamped physiological measurements and progress notes were the Philips CareVue Clinical Information System and iMDsoft MetaVision ICU. Classes of MIMIC data include patient demographics, billing information, unstructured text (notes), prescription medications, vital signs, laboratory results, and a plethora of physiological measurements.
TABLE 1. Versions of MIMIC.
Due to the granularity of information, MIMIC serves as a real-world data (RWD) foundation for artificial intelligence (AI) and machine learning (ML) research applications. MIMIC places thousands of ICU records with millions of physiological measurements directly into scientists' hands, which serves as a springboard for numerous projects in the development of AI/ML algorithms to support the work of ICU clinicians and staff. Here, we highlight a few of MIMIC’s contributions within the world of AI and ML, referring to this fusion as Artificial Intelligence for Real-World Data (AI4RWD). For the sake of brevity, a limited number of these efforts are mentioned in this short overview.
MIMIC Facilitates AI/ML Research With RWD
As the largest publicly available EHR ICU dataset, MIMIC has received intense interest from the AI/ML community as a common ground for developing and validating new methods. For example, in terms of ICU sepsis treatment, MIMIC has been used in various AI/ML approaches ranging from gradient-boosting to reinforcement and deep learning (Komorowski et al., 2018; Dauvin et al., 2019; Kaji et al., 2019; Hou et al., 2020). Several projects have focused on predicting ICU mortality, producing results that include a decision support tool for determining optimal patient ICU discharge times to improve patient outcomes (McWilliams et al., 2019). There are ICU vital signs monitoring programs, based on fuzzy models, that perform with a high degree of accuracy (Leite et al., 2011). These mentioned studies are a sample out of hundreds that employ MIMIC data to explore and develop AI applications for the critical care environment.
Prediction is the goal of most ML tasks that use the MIMIC EHRs. Generally, approaches to prediction involve the application of statistics, ML, and AI to arrive at post-admission outcomes. Logistic regression, proportional hazards model, random forests, support vector machines, boosting, and neural networks are some of the more common statistical and AI/ML approaches to these problems of prediction (Nemati et al., 2018; Vincent et al., 2018; Lin et al., 2019; McWilliams et al., 2019; Le et al., 2020; Nallabasannagari et al., 2020). These methods have been used extensively in other areas for regression or classification tasks depending on the nature of the response variable (continuous, categorical, time-to-event). The advantages and disadvantages of these methods have been extensively discussed. However, EHR data’s complex and heterogeneous nature can make directly transplanting approaches from other domains difficult. Recent developments in deep neural networks and natural language processing (NLP) have further expanded the scope of these applications (Shickel et al., 2017).
Like other EHR datasets, one significant challenge of working with MIMIC data is the high degree of inherent medical heterogeneity (MIT Critical Data, 2016). In contrast to the more homogeneous input tasks like image processing, MIMIC data spans free text forms (clinical notes, radiological reports), medical codes (ICD9/10), demographic information, time-series for vital signs, laboratory results, and medication records. Extensive data preprocessing is needed for any modeling effort that utilizes this data because feature engineering requirements are significant to the derivation of meaningful outcomes. Deep neural network-based NLP approaches, in turn, provide powerful tools to incorporate information from free-text clinical notes (Alsentzer et al., 2019). These challenges make deep learning development, with its capacity for hierarchical feature construction and long-range dependency modeling, valuable for analyzing EHR data (Shickel et al., 2017).
By applying deep learning and NLP approaches to MIMIC data, notable advances have been made in concept and patient representation and concept extraction in addition to outcome prediction (Beaulieu-Jones et al., 2018; Alsentzer et al., 2019; Huang et al., 2019; Nuthakki et al., 2019; Killian et al., 2020; Magna et al., 2020). The capacity for representation learning is significant because it enables common downstream tasks such as regression or classification. It also opens up the possibility for applications like computational phenotyping and clinical data de-identification. Going forward, MIMIC is expected to continue to be a valuable resource for the development and validation of ML/AI methods with applications to EHR data. One potential area of development that will benefit from MIMIC is causal inference, especially in conjunction with propensity score modeling and targeted maximum likelihood estimation. This holds the promise of harnessing the vast predictive power of AI/ML in the pre-final estimation steps while maintaining the well-established causal modeling framework (Blakely et al., 2021).
MIMIC Serves as a Conduit for Collaboration
Critics of AI and ML methods developed on MIMIC raise concerns about lack of validation in terms of safety and failure to demonstrate real-world effectiveness in the clinical setting. They cite that AI acceptance will be limited without stakeholder engagement and will suffer from a limited understanding of the critical care environment (Herasevich et al., 2020). These criticisms are not without merit, but they fail to recognize that these initial studies establish a basis for implementing an AI- or ML-based medical application designed to support the ICU clinician. Publicly available critical care datasets like MIMIC provide a collaborative “sandbox” that brings together clinicians, academia, medical technology companies, and the pharmaceutical industry. The public availability of MIMIC, combined with the granularity and volume of ICU data, fabricates a “crucible” where data scientists’ and medical practitioners’ ideas and efforts are “forged” together. This merging of disciplines is manifested at international datathons centered around MIMIC including the United Kingdom, China, Australia, Brazil and Singapore. Currently, MIMIC is studied in over 30 countries, with 4,000-plus users in industry and academia (Pollard and Celi, 2017). These alliances create unlimited potential for AI applications within the ICU, ultimately promoting positive patient outcomes and easing clinician workloads.
Utility in Regulatory Science
AI systems that assist in decision-making for individual patients are referred to as clinical AI by the National Academy of Medicine (Matheny et al., 2019). The performance of these systems undergoes intense scrutiny by the U.S. Food and Drug Administration (FDA) and its international counterparts. The challenges faced by these regulatory bodies include the collection of dynamic data (i.e., wearable devices) combined with RWD, such as EHRs; providing effective oversight and validating these systems requires more than a static, limited dataset. Because clinical AI systems are deployed to different institutions servicing different communities, epidemiological drift will likely require some fine-tuning of the algorithms to incorporate data from diverse populations (Matheny et al., 2019). In conjunction with other dynamic ICU EHRs, MIMIC can serve as one building block for calibrating these algorithms over time.
The FDA states that “Artificial intelligence and machine learning technologies have the potential to transform health care by deriving new and important insights from the vast amount of data generated during the delivery of health care every day” (FDA, 2021). In other words, regulatory oversight of clinical AI systems is essential to ensure safety, quality, and most importantly, public trust. MIMIC is a treasure trove of ICU data collected during the daily delivery of critical care to patients. As such, the insights provided by the AI/ML project initiatives, based upon MIMIC, serve as additional support for decision-making within regulatory science. An example of the way MIMIC facilitates the regulatory mission involves the antidiarrheal medication Loperamide. In 2016 the FDA issued a safety communication regarding serious heart problems associated with the use of Loperamide (FDA, 2016). There are 2,309 prescriptions of Loperamide captured within MIMIC-III, allowing further study and modeling of the drug relative to patient demographics, vital signs, and other factors of interest in the critical care environment (Adibuzzaman et al., 2016). Thus, the findings derived from MIMIC can offer opportunities to advance public health by improving and managing critical care outcomes.
Conclusion
MIMIC is an ICU real-world data set that serves as a research catalyst fusing the AI and ML efforts of technologists, academic researchers, regulators, and clinicians together to improve the critical care environment. It is foreseeable that MIMIC, in conjunction with other ICU datasets, may provide regulatory agencies with a bird’s-eye view of the post-market performance of new critical-care AI devices and systems. MIMIC serves as a cornerstone that brings AI and RWD together into an innovative mix, spawning ideas, algorithms and laying the groundwork for new critical care medical technologies.
Author Contributions
PR wrote this opinion piece while DW and ZL revised the text. All authors contributed to the article and approved the submitted version.
Funding
The publication fees are paid by the FDA National Center for Toxicological Research located in Jefferson, AR.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
Adibuzzaman, M., Musselman, K., Johnson, A., Brown, P., Pitluk, Z., and Grama, A. (2016). Closing the Data Loop: An Integrated Open Access Analysis Platform for the MIMIC Database. Comput. Cardiol. (2010) 43, 137–140. doi:10.23919/CIC.2016.7868698
Alsentzer, E., Murphy, J., Boag, W., Weng, W., Jin, D., Naumann, T., et al. (2019). Publicly Available Clinical BERT Embeddings. Paper presented at the 17th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, 6 Febraury 2019 to 6 July 2019. doi:10.18653/v1/w19-1909
Beaulieu-Jones, B. K., Orzechowski, P., and Moore, J. H. (2018). Mapping Patient Trajectories Using Longitudinal Extraction and Deep Learning in the MIMIC-III Critical Care Database. Pac. Symp. Biocomput 23, 123–132. Retrieved from. Available at: https://www.ncbi.nlm.nih.gov/pubmed/29218875
Blakely, T., Lynch, J., Simons, K., Bentley, R., and Rose, S. (2021). Reflection on Modern Methods: when Worlds Collide-Prediction, Machine Learning and Causal Inference. Int. J. Epidemiol. 49 (6), 2058–2064. doi:10.1093/ije/dyz132
Dauvin, A., Donado, C., Bachtiger, P., Huang, K.-C., Sauer, C. M., Ramazzotti, D., et al. (2019). Machine Learning Can Accurately Predict Pre-admission Baseline Hemoglobin and Creatinine in Intensive Care Patients. Npj Digit. Med. 2, 116. doi:10.1038/s41746-019-0192-z
FDA (2021). Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan. Retrieved from. Available at: https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-software-medical-device.
FDA (2016). FDA Drug Safety Communication: FDA Warns about Serious Heart Problems with High Doses of the Antidiarrheal Medicine Loperamide (Imodium), Including from Abuse and Misuse. Available at: https://www.fda.gov/drugs/drug-safety-and-availability/fda-drug-safety-communication-fda-warns-about-serious-heart-problems-high-doses-antidiarrheal.
Herasevich, V., Keegan, M., Johnston, M., and Pickering, B. (2020). Will Artificial Intelligence Change ICU Practice?. ICU Manag. Pract. 19 (4). Available at: https://healthmanagement.org/c/icu/issuearticle/will-artificial-intelligence-change-icu-practice
Hou, N., Li, M., He, L., Xie, B., Wang, L., Zhang, R., et al. (2020). Predicting 30-days Mortality for MIMIC-III Patients with Sepsis-3: a Machine Learning Approach Using XGboost. J. Transl Med. 18 (1), 462. doi:10.1186/s12967-020-02620-5
Huang, J., Osorio, C., and Sy, L. W. (2019). An Empirical Evaluation of Deep Learning for ICD-9 Code Assignment Using MIMIC-III Clinical Notes. Comp. Methods Programs Biomed. 177, 141–153. doi:10.1016/j.cmpb.2019.05.024
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-w. H., Feng, M., Ghassemi, M., et al. (2016). MIMIC-III, a Freely Accessible Critical Care Database. Sci. Data 3, 160035. doi:10.1038/sdata.2016.35
Johnson, A. E. W., Stone, D. J., Celi, L. A., and Pollard, T. J. (2018). The MIMIC Code Repository: Enabling Reproducibility in Critical Care Research. J. Am. Med. Inform. Assoc. 25 (1), 32–39. doi:10.1093/jamia/ocx084
Kaji, D. A., Zech, J. R., Kim, J. S., Cho, S. K., Dangayach, N. S., Costa, A. B., et al. (2019). An Attention Based Deep Learning Model of Clinical Events in the Intensive Care Unit. PLoS One 14 (2), e0211057. doi:10.1371/journal.pone.0211057
Killian, T., Zhang, H., Subramanian, J., Fatemi, M., and Ghassemi, M. (2020). An Empirical Study of Representation Learning for Reinforcement Learning in Healthcare. arXiv preprint, arXiv:2011.11235.
Komorowski, M., Celi, L. A., Badawi, O., Gordon, A. C., and Faisal, A. A. (2018). The Artificial Intelligence Clinician Learns Optimal Treatment Strategies for Sepsis in Intensive Care. Nat. Med. 24 (11), 1716–1720. doi:10.1038/s41591-018-0213-5
Le, S., Pellegrini, E., Green-Saxena, A., Summers, C., Hoffman, J., Calvert, J., et al. (2020). Supervised Machine Learning for the Early Prediction of Acute Respiratory Distress Syndrome (ARDS). J. Crit. Care 60, 96–102. doi:10.1016/j.jcrc.2020.07.019
Leite, C. R., Sizilio, G. R., Neto, A. D., Valentim, R. A., and Guerreiro, A. M. (2011). A Fuzzy Model for Processing and Monitoring Vital Signs in ICU Patients. BioMedical Eng. OnLine 10, 68. doi:10.1186/1475-925X-10-68
Lin, K., Hu, Y., and Kong, G. (2019). Predicting In-Hospital Mortality of Patients with Acute Kidney Injury in the ICU Using Random forest Model. Int. J. Med. Inform. 125, 55–61. doi:10.1016/j.ijmedinf.2019.02.002
Magna, A. A. R., Allende-Cid, H., Taramasco, C., Becerra, C., and Figueroa, R. L. (2020). Application of Machine Learning and Word Embeddings in the Classification of Cancer Diagnosis Using Patient Anamnesis. IEEE Access 8, 106198–106213. doi:10.1109/ACCESS.2020.3000075
McWilliams, C. J., Lawson, D. J., Santos-Rodriguez, R., Gilchrist, I. D., Champneys, A., Gould, T. H., et al. (2019). Towards a Decision Support Tool for Intensive Care Discharge: Machine Learning Algorithm Development Using Electronic Healthcare Data from MIMIC-III and Bristol, UK. BMJ Open 9 (3), e025925. doi:10.1136/bmjopen-2018-025925
MIT Critical Data (2016). Secondary Analysis of Electronic Health Records. Cham (CH): Springer Open.
Matheny, M., Thadaney Israni, S., Ahmed, M., and Whicher, D. (2019). Artificial Intelligence in Health Care: The Hope, the Hype, the Promise, the Peril (Washington, DC: National Academy of Medicine).
Nallabasannagari, A., Reddiboina, M., Seltzer, R., Zeffiro, T., Sharma, A., and Bhandari, M. (2020). All Data Inclusive, Deep Learning Models to Predict Critical Events in the Medical Information Mart for Intensive Care III Database (MIMIC III). arXiv preprint, arXiv:2009.01366.
Nemati, S., Holder, A., Razmi, F., Stanley, M. D., Clifford, G. D., and Buchman, T. G. (2018). An Interpretable Machine Learning Model for Accurate Prediction of Sepsis in the ICU. Crit. Care Med. 46 (4), 547–553. doi:10.1097/CCM.0000000000002936
Nuthakki, S., Neela, S., Gichoya, J. W., and Purkayastha, S. (2019). Natural Language Processing of MIMIC-III Clinical Notes for Identifying Diagnosis and Procedures with Neural Networks. arXiv preprint, arXiv:1912.12397.
Pollard, T. J., and Celi, L. A. (2017). Enabling Machine Learning in Critical Care. ICU Manag. Pract. 17 (3), 198–199. Available at: https://www.ncbi.nlm.nih.gov/pubmed/29130079 .
Shickel, B., Tighe, P. J., Bihorac, A., and Rashidi, P. (2018). Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis. IEEE J. Biomed. Health Inform. 22 (5), 1589–1604. doi:10.1109/JBHI.2017.2767063
Keywords: MIMIC, artificial intelligence, machine learning, ICU, critical care, database
Citation: Rogers P, Wang D and Lu Z (2021) Medical Information Mart for Intensive Care: A Foundation for the Fusion of Artificial Intelligence and Real-World Data. Front. Artif. Intell. 4:691626. doi: 10.3389/frai.2021.691626
Received: 06 April 2021; Accepted: 18 May 2021;
Published: 31 May 2021.
Edited by:
Justin Zhan, University of Arkansas, United StatesReviewed by:
Nouha Arfaoui, Gabes University, TunisiaMaria Hallo, Escuela Politécnica Nacional, Ecuador
Copyright © 2021 Rogers, Wang and Lu. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Paul Rogers, Paul.Rogers@fda.hhs.gov