The potential of virtual triage AI to improve early detection, care acuity alignment, and emergent care referral of life-threatening conditions

Objective: To evaluate the extent to which patient-users reporting symptoms of five severe/acute conditions requiring emergency care to an AI-based virtual triage (VT) engine had no intention to get such care, and whose acuity perception was misaligned or decoupled from actual risk of life-threatening symptoms.

Methods: A dataset of 3,022,882 VT interviews conducted over 16 months was evaluated to quantify and describe patient-users reporting symptoms of five potentially life-threatening conditions whose pre-triage healthcare intention was other than seeking urgent care, including myocardial infarction, stroke, asthma exacerbation, pneumonia, and pulmonary embolism.

Results: Healthcare intent data was obtained for 12,101 VT patient-user interviews. Across all five conditions a weighted mean of 38.5% of individuals whose VT indicated a condition requiring emergency care had no pre-triage intent to consult a physician. Furthermore, 61.5% intending to possibly consult a physician had no intent to seek emergency medical care. After adjustment for 13% VT safety over-triage/referral to ED, a weighted mean of 33.5% of patient-users had no intent to seek professional care, and 53.5% had no intent to seek emergency care.

Conclusion: AI-based VT may offer a vehicle for early detection and care acuity alignment of severe evolving pathology by engaging patients who believe their symptoms are not serious, and for accelerating care referral and delivery for life-threatening conditions where patient misunderstanding of risk, or indecision, causes care delay. A next step will be clinical confirmation that when decoupling of patient care intent from emergent care need occurs, VT can influence patient behavior to accelerate care engagement and/or emergency care dispatch and treatment to improve clinical outcomes.

Introduction

Life-threatening illness requiring emergency care accounts for 14.0% of emergency department (ED) visits in the US, and benefits from the earliest possible detection and care referral to improve patient outcomes and reduce hospital length of stay and care costs (1, 2). If patient understanding/self-perception underestimates actual acuity and risk indicated by symptoms and past medical history, patients may delay care seeking (3–5). Delayed care results in avoidable mortality and drives higher acuity care utilization that is more costly than might have been needed had the patient sought care earlier (3, 6).

Symptomate is a current generation digital virtual triage (VT) engine or “symptom checker” that leverages artificial intelligence (AI) to provide access to authoritative information services beyond physician office hours on a 24/7/365 basis from any device with internet connectivity. VT can avert long ED waits or travel, and helps when individuals are uncertain if symptoms warrant medical attention or what kind of care to seek, or if symptoms worsen. The VT interview asks questions about age, sex, symptoms, medical history, risk factors, and medications and, processing responses on a current basis, selects the next most relevant question via Bayesian probabilities, simulating the manner in which a human clinician processes information. After evaluating all patient data, the VT inference engine computes the probabilities of likely conditions using a statistical algorithm for advanced symptom assessment informed by an underlying medical knowledge base. The VT AI advises the patient-user of probable causes, illness severity, and recommended acuity level of medical care. VT is superior to internet browser search because patients are evaluated systematically to identify/convey a care acuity level using clinical AI algorithms validated by physicians. VT AI improves upon traditional clinical triage protocols or decision trees in its ability to dynamically calculate the probability of possible conditions and adjust the patient interview accordingly. However, VT results are not diagnostic and do not constitute medical consultation, providing information and guidance only. Users are advised to seek immediate medical attention when the VT engine determines that a threat to life exists.

Critical to maximizing the clinical and public health impact of VT AI is identifying functional areas where it can enhance existing diagnostic processes and capabilities. It is unknown whether AI-based VT can augment existing healthcare system capabilities for early identification, care acuity alignment, and rapid care referral of individuals whose self-perception of disease acuity/risk is inaccurate when serious morbidity is actually evolving. Patient delay in seeking urgent care across life-threatening diseases increases mortality (3, 7). When patient perception-understanding of symptoms/risk is decoupled from high actual risk of a severe clinical outcome, if identified reliably by VT in real-time, ED care engagement can be accelerated. VT may thus offer a new opportunity for early life-threatening disease detection, enabling improved, more rapid care acuity alignment and ED referral. VT could enhance existing healthcare system capabilities and efforts to reduce preventable disease sequelae and death. This study evaluates decoupling of actual care acuity need from patient care intent/risk self-perception as detected by a clinically validated VT engine available cost-free online. We examine data on patient perception-actual risk decoupling during severe symptom onset of five leading causes of U.S. hospital admission and death.

Methods

Objective

Evaluate the extent to which patient-users reporting symptoms of severe, acute conditions classified by an AI-based VT engine as warranting emergency care had no pre-VT triage intention to engage appropriate ED care. The extent to which decoupling of patient care intent from actual clinical need occurs for five prevalent diseases could imply a potential real-time use of VT to improve early detection and care acuity alignment and accelerate more effective care referral for emergent conditions.

VT technology platform and data source

The Infermedica Symptomate VT engine is designed for general public use and completes evidence-driven analyses informed by over 800 diseases, 1,500 symptoms, and 200 risk factors. Leveraging AI, machine learning, and natural language processing, VT evaluates symptoms reported by patient-users, suggesting the most probable conditions matching the presentation and history, and refers to the most clinically appropriate and safest possible care. There are no prescribed interview pathways, and in light of new information reported, the VT AI explores various clinical queries and hypotheses (as physicians do). Before VT, patient-users are asked about their care intention. The VT interview concludes with an analysis of the reported symptoms and a recommendation to pursue one of three levels of care acuity: self-care, consult a primary care or specialist physician on an outpatient basis, proceed to an ED or call an ambulance for ED transport.

Virtual triage technologies (including Symptomate) are considered medical device class I in Europe, according to Medical Device Directive (93/42/EEC) and in the US, under the Food, Drug & Cosmetic Act. The FDA currently exercises enforcement discretion, which means that virtual triage technology is not required to comply with FDA regulations related to medical devices.

Data were extracted from VT episodes engaged by patient-users of the Symptomate VT application, a stand-alone VT engine not integrated with/implemented within a health system, and available through the Infermedica website or as a mobile application. Symptomate is available in 24 languages. Over 17 million Symptomate evaluations have been completed since 2012.

VT engine clinical validity

AI-based VT engines require rigorous validation to ensure safety and minimize mistriage, and focus on common diseases by design, with AI built to overtriage to emergency care rather than possibly miss a patient with acute care need. VT accuracy varies across clinical specialties and settings, as determined by the depth of disease-specific data used to train the triage AI. VT validity has been evaluated using clinical vignettes prepared by physicians of various patient symptomatic presentations in different clinical settings (8–11). Symptomate provides safe recommendations in 97.8% of instances (9). VT engines may perform poorly detecting low prevalence conditions or presentations in atypical clinical settings (8). Published studies, while providing a point in time comparison, become quickly outdated due to the rapid evolution of AI-based VT. For example, since the analysis conducted by 2019 Gilbert et al., (9) there have been 29 releases of the Symptomate medical content model and 17 updates to core functionality, including new epidemiological models and triage algorithms, each improving accuracy and safety. In a recent study of 8,008 patient-users of the Infermedica virtual triage engine in an ambulatory care network, 2,830 or 35.0% changed their healthcare seeking action from their initial pre-VT care intent as a result of the VT evaluation and recommendation, indicating that VT output can alter the care seeking behavior of patient-users (12).

Sample selection and eligibility criteria

Symptomate VT interviews were selected using the following criteria: (1) interview completed during a 16-month period of March 2022–June 2023; (2) first ranked condition classified as requiring emergency care; (3) condition probability equal or greater than 0.90 as the first ranked by VT; (4) conditions with minimum 300 interviews; and (5) conditions with crude death rate of 1 per 100,000 and above in CDC Wonder (13). COVID-19 was excluded. Only five conditions with high potential severity/mortality met these eligibility criteria, including myocardial infarction (MI), stroke, asthma, pneumonia, and pulmonary embolism (PE). While the VT engine uses the term/variable “myocardial infraction” in its logic, language used in the VT patient interface was adapted to ensure public comprehension (e.g., “heart attack”). Ischemic stroke (including cerebellar strokes) and hemorrhagic stroke (both subarachnoid and intracerebral) were collapsed into a single category for analysis. Figure 1 conveys the sequential application of eligibility criteria to patient-user interview selection.

Figure 1

Figure 1. Flow chart of virtual triage patient-user eligibility for study inclusion.

Data captured and analyses completed

All data was collected through the Symptomate VT engine database. VT interviews were completed anonymously with no individual identifiers recorded beyond age, gender, and language. Patient-users conveyed informed consent for using their anonymized data in the aggregate devoid of personal identifiers. A dataset of 3,022,882 patient-users was evaluated to investigate the extent of decoupling of patient-user pre-triage care intent and VT-identified high acuity conditions warranting emergency care. The pre-triage care intention survey provided response options of doing nothing; self-care/recover at home; consult a doctor; emergency care with or without ambulance; and not sure/do not know/none. While care intent after VT guidance was queried, completion rates were insufficient to support meaningful analysis.

Pre-interview intent responses of doing nothing, self-care/recovering at home, were grouped and classified as non-professional care intent. Intent to consult an outpatient physician, do nothing, and self-care/recovering at home were grouped and classified as non-urgent care intent. For each condition, frequency of responses indicating pre-triage intent of non-professional care and non-emergent care were determined and assessed for systematic variation by demographics. Google Sheets was used to complete chi-squared tests, assessing for statistically significant differences between demographic groups (age, gender, and language) by care intent responses at the p = 0.05 level. To calculate the aggregated level of indicators in Table 1 encompassing various patient care intents across all conditions included in the analysis, weighted arithmetic means were calculated using each condition’s respective prevalence as its weighted value. Weighted means were calculated across the five diseases by multiplying the number of VT interviews for each condition by the percentage of interviews with stated intent for that condition, then summing and dividing by the sum of interviews.

Table 1

Table 1. Decoupling of patient care intent from virtual triage care referral for five causes of us morbidity and mortality.

Because data were derived from general public Symptomate use and not patient-users within a healthcare system, it was impossible to follow patients’ clinical course to confirm acuity and clinical outcomes.

Adjustment for overtriage to emergency care

VT technology errs on the side of over-caution in care referral because of the potentially serious harm/injury to patients if VT under-refers to ED/ambulance, favoring care referral sensitivity over specificity by design and ethical imperative. Analyses of Symptomate over-referral have indicated that approximately 13% of ED recommendations are over-referrals, which has been declining as its VT AI advances (14). Symptomate performance was compared with widely used teletriage clinical decisioning protocols using clinical vignettes. VT over-referred 12% of patients to ED versus 26% for a leading telephone triage product (Schmitt-Thompson) (15). To enhance precision of estimates of decoupling of patient pre-triage care intent from VT acuity/referral output, intention rates were adjusted for overtriage to ED by a factor of 13%.

Results

Decoupling of patient pre-triage care intent and VT acuity assessment

Intent data were obtained for 12,101 VT patient-user interviews. Across all five conditions, a weighted mean of 38.5% of individuals whose VT indicated a condition requiring emergency care had no pre-triage intent to consult a physician. Furthermore, 61.5% intending to possibly consult a physician had no intent to seek emergency medical care (Table 1). The low frequency of patient-users intending before VT to seek emergency care is striking and may reflect the use of VT quite early in symptom evolution, presumably increasing with an experience of more severe symptoms. In addition, individuals with severe symptoms may bypass VT use in favor of telephonic consultation or in-person visit to an ED.

Table 2 presents adjusted rates factoring in VT safety over-triage to ED and shows that after allowing for a 13% over referral to ED, a weighted average of 33.5% of patient-users across conditions had no intent to seek professional care. One half (53.5%) had no intent to seek emergency care.

Table 2

Table 2. Patient care intent-virtual triage care referral decoupling adjusted for AI safety overtriage to emergency department.

Demographic analyses of pre-triage care intent by age, gender, and language

With respect to demographics, all age groups were statistically significantly different from one another (p = 0.05). The youngest (under 18 are completed by a parent or adult caretaker) and the eldest age groups were more likely to express intent to seek appropriate emergency care for symptoms, with those 18–44 years least likely to, as shown in Table 3.

Table 3

Table 3. Patient-user pre-VT intent by age.

Regarding respondent gender (Table 4), men were more likely (40.7%) than women (37.6%) to indicate a non-professional care intent (p = 0.05). Men were also more likely to state an intent to seek emergency care (9.3%) than women (7.1%) (p = 0.05). The sample breakdown of patient-user respondents by gender, with women comprising over two-thirds of VT interviews, reflects the pattern of all Symptomate users (16). Overall, the lack of perceived urgency of symptoms is similar between genders.

Table 4

Table 4. Patient-user pre-VT intent by gender.

Patient-users in each major language grouping (English, Spanish, French, German, Polish, Russian and other) were statistically significantly different from each other in every intent category (p = 0.05) (Table 5). Those completing VT in Spanish were more likely to express appropriate emergency care intent than patient-users in other language groups (10.2%). Those completing in French were lowest in reporting intent to engage emergency care at 4.7%, but were most inclined to engage non-emergent care including consulting an outpatient physician.

Table 5

Table 5. Patient-user pre-VT intent by interview language.

Discussion

Delays in care seeking during medical emergencies negatively impact patient outcomes, and may result from patients failing to understand, dismissing or underestimating serious symptoms, and/or attempting to self-treat serious clinical problems at home (3). Table 6 summarizes causes and impact of care delay for the five conditions evaluated. Early detection and prompt, acuity appropriate treatment are crucial in improving outcomes of MI, which killed 108,610 Americans in 2018 (13). Almost 48% of VT patient-users with likely MI understood their symptoms required physician care but not immediate/urgent care, yet 28.1% planned to recover at home with no medical care (ED overtriage adjusted). This aligns with national mortality data. Reducing MI care delays focuses on patient education about symptoms warranting early care seeking (3, 18, 19), and can be integrated with VT.

Table 6

Table 6. Causes and impact of care delay for five life-threatening conditions.

In 2019, stroke accounted for 1 in 19 US deaths, the fifth leading cause of mortality (22, 33). Prehospital suspicion of stroke compressed time to computer tomography by 219 minutes and to stroke unit care by 556 minutes, reducing mortality (23, 34). However, 42.0% of patients experienced pre-hospital delays due to little knowledge of stroke symptoms (21). In the present study, 45.6% of patient-users with stroke symptoms intended to pursue non-emergency care, including 24.0% who planned to recover at home or take no action (ED overtriage adjusted). Reducing pre-hospital stroke care delay and acuity of care mismatch is a priority (21, 23). VT can potentially reduce delays in emergency stroke care.

In 2019, asthma caused 1,835,901 US ED visits, 169,330 hospital admissions, and 3,524 deaths (35–37). Many deaths and admissions were preventable and related to care delays (5, 7). In total, 45% of patients did not seek medical care during the final fatal asthma attack (25).

In 2021, 1.4 million US cases of pneumonia were diagnosed in EDs, 47,956 of which were fatal (13). Pneumonia was a concomitant cause of death in 148,498 deaths (13). Timely antibiotic treatment reduced adjusted 30-day hospital mortality by 16.0% (27).

Pulmonary embolism caused 8,809 US deaths in 2018 (13), with untreated mortality of 30.0% (30). In-hospital mortality was 1.6% for early versus 43.2% for delayed PE diagnosis (31). Treatment was delayed for 4.2 days due to patient procrastination (29), with 18–38% of patients symptomatic a week or longer before diagnosis (32). VT well integrated with healthcare system triage and intake can potentially reduce patient underestimation of symptom severity and delays in care seeking for these deadly respiratory illnesses.

Our findings that 53.5% of patients (ED overtriage adjusted) who reported symptoms of these five life-threatening conditions had no pre-triage intent to engage urgent care, and 33.5% had no intent to seek any medical care, aligns with national data. Users of VT may wish to assess their symptoms without engaging local healthcare for convenience and/or to save time and expense. Patients may seek reassurance that their symptoms are relatively benign. This sampling of 12,000 interviews from over three million total VT episodes demonstrates that in a significant percentage, possible existence of a severe underlying cause of symptoms should prompt the patient-user to seek professional medical attention sooner than they would have otherwise planned. The potential to reduce morbidity and mortality by more timely assessment of ambiguous/unrecognized signs and symptoms has significant public health implications. We suspect that use of AI-driven VT may become a valuable addition to under-resourced healthcare systems unable to assess all patients who seek care for uncertain symptoms but who may be at elevated risk for more serious disease.

With respect to limitations of the study, it is pertinent to acknowledge that the data collected may have been affected by certain inaccuracies. For instance, patient-users may have clicked through initial questions regarding their pre-VT care intention or conveyed random responses in order to get to the part of the encounter they were interested in, for example the VT evaluation of symptoms presented. While we have no evidence that this behavior occurred in substantial numbers of VT patient-users, the question warrants further research consideration and detection measures if a prevalent behavior.

In the effort to change patient care seeking behavior when confronted with potentially life threatening symptoms, interventions are needed that improve patient engagement, education, understanding of the clinical signs and symptoms of these acute conditions, and which are effective in activating patients to immediately seek emergency care. Given that many lives are avoidably lost, and the quality and extent of others are diminished, our current inability to effectively engage and educate patients to seek urgent care when confronted with life threatening symptoms is an area demanding far greater behavior change research, along with associated research funding. Indeed, one wonders what could be achieved in understanding and affecting patient care seeking behavior change were this research to be funded at the level of pharmaceutical development and marketing of a new therapeutic to treat patients whose severity of illness could have been lessened, if we were more capable of identifying and activating these patients to earlier care seeking and reduction of care delay.

Clearly, current approaches to achieving these behavioral and care seeking objectives have been largely inadequate. Public awareness campaigns addressing the importance of seeking prompt medical care in cases of acute, life-threatening cerebrovascular symptoms, for example, demonstrate a positive impact (39) but are often episodic rather than continuous, which limits their long-term impact and benefit. Reliable online medical resources exist which educate about the importance of early recognition and care seeking for serious cardiac symptoms as well, but provide general, non-personalized medical information, which can lessen the impact on individual behavior.

Subsequent research should systematically assess patient outcomes in order to fully evaluate the potential of VT to earlier detect severe evolving pathologies, such as those reported here, and effectively motivate and accelerate patient care seeking. Prospective validation in a real world clinical setting could track outcomes of patients whose self-perception and healthcare intent prior to VT is decoupled from actual acuity and urgent care need. Using objective clinical parameters and endpoints, this would assess the role and clinical value of AI-enabled VT within a care delivery and patient engagement ecosystem. If thus validated, VT may offer payers, health systems and providers a real-time vehicle for earlier detection and engagement of at-risk patients to enable better care acuity alignment and rapid ED referral/ambulance dispatch for life-threatening conditions.

Conclusion

The impact of advanced diagnostics and therapeutics is blunted if patients do not rapidly engage appropriate acuity care to receive their benefits. This study suggests that VT AI may help accelerate the moment of disease detection and appropriate acuity level care engagement. VT lets a user know their condition may be more serious than they perceive and they should seek ED care. Integrated within a digital front door, VT can potentially accelerate early detection, care acuity alignment and live engagement/intake to save lives, reduce morbidity, and better utilize healthcare resources by reducing care delays due to patient uncertainty and/or misunderstanding. Future research should validate that VT can influence patient care seeking behavior to enable accelerated ED care referral or ambulance dispatch. If validated, VT may become an integral component in an AI-enhanced clinical information ecosystem that expedites detection and delivery of appropriate urgent medical care.

Currently, when patient care intent is decoupled from actual clinical need by patient misunderstanding or indecision, earlier care detection and accelerated care engagement to reduce care delay is a complex challenge. As remote diagnostic monitoring and telemedical capabilities advance in coming years, so too will AI-based VT. Rapid development in generative AI and large language models will deliver improved user experience of VT AI. The use of natural language to enable patient-users to communicate in a more effective and nuanced way with VT AI will enable greater sensitivity, engendering trust in the results and greater impact on patient-user behavior. Exploiting these emerging potentialities will require investment and multisector scientific expertise converging private industry and public-governmental interests, assets, and capabilities.

The VT industry is not sufficiently resourced to drive the research needed to explore the potential beneficial impact and value of AI-based VT in early detection and acuity-aligned care referral. Furthermore, population databases of what will soon be hundreds of millions of VT episodes across the industry are too valuable a data and research resource to be applied exclusively for purely commercial interest. The VT industry should enable access to its anonymized aggregate data by the US National Institutes of Health and Centers for Disease Control and Prevention, international agencies, academic medical centers, and healthcare organizations to investigate and progress the full potential value of VT in the emerging 21st century health IT and AI ecosystem.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors upon reasonable request.

Ethics statement

The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

GAG: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Validation, Visualization, Writing – original draft, Software, Supervision, Writing – review & editing. AK-K: Conceptualization, Data curation, Formal analysis, Methodology, Validation, Writing – review & editing. GLG: Data curation, Project administration, Writing – review & editing. JR-S: Investigation, Writing – review & editing. SG: Conceptualization, Formal analysis, Resources, Writing – original draft, Writing – review & editing. TP: Investigation, Project administration, Supervision, Writing – review & editing. KK: Data curation, Project administration, Writing – review & editing. NM: Writing – original draft. MP: Conceptualization, Writing – original draft. Writing – review & editing. JJ: Conceptualization, Writing – original draft, Writing – review & editing. IL: Writing – review & editing. PO: Project administration, Supervision, Writing – review & editing.

Funding

The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.

Conflict of interest

GAG, AK-K, GLG, JR-S, SG, TP, KK, NM, MP, JJ, GLG, IKL, PMO were employed by Infermedica, Inc.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Carr, BG, Conway, PH, Meisel, ZF, Steiner, CA, and Clancy, C. Defining the emergency care sensitive condition: a health policy research agenda in emergency medicine. Ann Emerg Med. (2010) 56:49–51. doi: 10.1016/j.annemergmed.2009.12.013

PubMed Abstract | Crossref Full Text | Google Scholar

2. Vashi, AA, Urech, T, Carr, B, Greene, L, Warsavage, T Jr, Hsia, R, et al. Identification of emergency care-sensitive conditions and characteristics of emergency department utilization. JAMA Netw Open. (2019) 2:e198642. doi: 10.1001/jamanetworkopen.2019.8642

PubMed Abstract | Crossref Full Text | Google Scholar

3. Rafi, A, Sayeed, Z, Sultana, AS, and Hossain, G. Pre-hospital delay in patients with myocardial infarction: an observational study in a tertiary care hospital of northern Bangladesh. BMC Health Serv Res. (2020) 20:633. doi: 10.1186/s12913-020-05505-x

PubMed Abstract | Crossref Full Text | Google Scholar

4. Aydoğdu, M, Doğan, NÖ, Sinanoğlu, NT, Oğuzülgen, İK, Demircan, A, Bildik, F, et al. Delay in diagnosis of pulmonary thromboembolism in emergency department: is it still a problem? Clin Appl Thromb Hemost. (2013) 19:402–9. doi: 10.1177/1076029612440164

PubMed Abstract | Crossref Full Text | Google Scholar

5. Quek, JS, Tang, WE, Chen, E, and Smith, HE. Understanding the journeys of patients with an asthma exacerbation requiring urgent therapy at a primary care clinic. BMC Pulm Med. (2022) 22:231. doi: 10.1186/s12890-022-02024-9

PubMed Abstract | Crossref Full Text | Google Scholar

6. Thomas, A, Valero-Elizondo, J, Khera, R, Warraich, HJ, Reinhardt, SW, Ali, HJ, et al. Forgone medical care associated with increased health care costs among the U.S. heart failure population. JACC. Heart Fail. (2021) 9:710–9. doi: 10.1016/j.jchf.2021.05.010

PubMed Abstract | Crossref Full Text | Google Scholar

7. Janson, S, and Becker, G. Reasons for delay in seeking treatment for acute asthma: the patient's perspective. J Asthma. (1998) 35:427–35. doi: 10.3109/02770909809048951

PubMed Abstract | Crossref Full Text | Google Scholar

8. Berry, AC, Cash, BD, Wang, B, Mulekar, MS, van Haneghan, AB, Yuquimpo, K, et al. Online symptom checker diagnostic and triage accuracy for HIV and hepatitis C. Epidemiol Infect. (2019) 147:e104. doi: 10.1017/S0950268819000268

PubMed Abstract | Crossref Full Text | Google Scholar

9. Gilbert, S, Mehl, A, Baluch, A, Cawley, C, Challiner, J, Fraser, H, et al. How accurate are digital symptom assessment apps for suggesting conditions and urgency advice? A clinical vignettes comparison to GPs. BMJ Open. (2020) 10:e040269. doi: 10.1136/bmjopen-2020-040269

PubMed Abstract | Crossref Full Text | Google Scholar

10. Hill, MG, Sim, M, and Mills, B. The quality of diagnosis and triage advice provided by free online symptom checkers and apps in Australia. Med J Aust. (2020) 212:514–9. doi: 10.5694/mja2.50600

PubMed Abstract | Crossref Full Text | Google Scholar

11. Blanchard, S. NHS-backed GP chatbot is branded a public health danger. Daily Mail, (2019). Available at: https://www.dailymail.co.uk/health/article-6751393/NHS-backed-GP-chatbot-branded-public-health-danger.html.

Google Scholar

12. Gellert, GA, Garber, L, Kabat-Karabon, A, Kuszczyński, K, Price, T, and Marcjasz, N, et al. Using AI-based virtual triage to improve acuity-level alignment of patient care seeking in an ambulatory care setting. Int J Healthcare. (2024) in press.

Google Scholar

13. Centers for Disease Control and Prevention, National Center for Health Statistics . National Vital Statistics System, Mortality 2018–2021 on CDC WONDER Online Database, released in 2021. Data are from the Multiple Cause of Death Files, 2018–2021, as compiled from data provided by the 57 vital statistics jurisdictions through the Vital Statistics Cooperative Program. Available at: http://wonder.cdc.gov/ucd-icd10-expanded.html.

Google Scholar

14. Świątkowska, M, and Kosturek, M. Analysis of over triage rate based on production data labeled internally by the clinical validation team. Internal report of Infermedica. (2023)

Google Scholar

15. Gellert, GA, Kuszczyński, K, Marcjasz, N, Jaszczak, J, Price, T, and Orzechowski, PM. A comparative performance analysis of live clinical triage using rules based triage protocols versus AI-based automated virtual triage. J. Hosp Adm. (2023) 13:8. doi: 10.5430/jha.v13n1p8

Crossref Full Text | Google Scholar

16. Gellert, GA, Orzechowski, PM, Price, T, Jaszczak, J, Marcjasz, N, Mlodawska, A, et al. A multinational survey of patient utilization of and value conveyed through virtual symptom triage and healthcare referral. Frontiers in Public Health. (2023). doi: 10.3389/fpubh.2022.1047291

Crossref Full Text | Google Scholar

17. Taghaddosi, M, Dianati, M, Fath Gharib Bidgoli, J, and Bahonaran, J. Delay and its related factors in seeking treatment in patients with acute myocardial infarction. ARYA Atheroscler. (2010) 6:35–41.

Google Scholar

18. De Luca, G, Suryapranata, H, Ottervanger, JP, and Antman, EM. Time delay to treatment and mortality in primary angioplasty for acute myocardial infarction: every minute of delay counts. Circulation. (2004) 109:1223–5. doi: 10.1161/01.CIR.0000121424.76486.20

Crossref Full Text | Google Scholar

19. Leslie, WS, Urie, A, Hooper, J, and Morrison, CE. Delay in calling for help during myocardial infarction: reasons for the delay and subsequent pattern of accessing care. Heart. (2000) 84:137–41. doi: 10.1136/heart.84.2.137

Crossref Full Text | Google Scholar

20. Centers for Disease Control and Prevention . Heart Disease Facts. Available at: https://www.cdc.gov/heartdisease/facts.htm

Google Scholar

21. Mulder, MJHL, Jansen, IGH, Goldhoorn, RB, Venema, E, Chalos, V, Compagne, KCJ, et al. Time to endovascular treatment and outcome in acute ischemic stroke: MR CLEAN registry results. Circulation. (2018) 138:232–40. doi: 10.1161/CIRCULATIONAHA.117.032600

Crossref Full Text | Google Scholar

22. Eddelien, HS, Butt, JH, Amtoft, AC, Nielsen, NSK, Jensen, ES, Danielsen, IMK, et al. Patient-reported factors associated with early arrival for stroke treatment. Brain Behav. (2021) 11:e2225. doi: 10.1002/brb3.2225

Crossref Full Text | Google Scholar

23. Fladt, J, Meier, N, Thilemann, S, Polymeris, A, Traenka, C, Seiffge, DJ, et al. Reasons for prehospital delay in acute ischemic stroke. J Am Heart Assoc. (2019) 8:e013101. doi: 10.1161/JAHA.119.013101

Crossref Full Text | Google Scholar

24. Centers for Disease Control and Prevention . Stroke facts (2023). Available at: https://www.cdc.gov/stroke/facts.htm.

Google Scholar

25. Levy, ML . The national review of asthma deaths: what did we learn and what needs to change? Breathe (Sheff). (2015) 11:14–24. doi: 10.1183/20734735.008914

Crossref Full Text | Google Scholar

26. Centers for Disease Control and Prevention . (2018) Archived National Asthma Data. https://www.cdc.gov/asthma/archivedata/2018/2018_archived_national_data.html

Google Scholar

27. Daniel, P, Rodrigo, C, Mckeever, TM, Woodhead, M, Welham, S, and Lim, WS. Time to first antibiotic and mortality in adults hospitalised with community-acquired pneumonia: a matched-propensity analysis. Thorax. (2016) 71:568–70. doi: 10.1136/thoraxjnl-2015-207513

Crossref Full Text | Google Scholar

28. Centers for Disease Control and Prevention. National Center for Health Statistics, National Hospital Ambulatory Medical Care Survey: 2021 Emergency Department Summary Tables. National Hospital Ambulatory Medical Care Survey: 2021 Emergency Department Summary Tables (cdc.gov) .

Google Scholar

29. Walen, S, Damoiseaux, RA, Uil, SM, and van den Berg, JW. Diagnostic delay of pulmonary embolism in primary and secondary care: a retrospective cohort study. Br J Gen Pract. (2016) 66:e444–50. doi: 10.3399/bjgp16X685201

Crossref Full Text | Google Scholar

30. Bĕlohlávek, J, Dytrych, V, and Linhart, A. Pulmonary embolism, part I: epidemiology, risk factors and risk stratification, pathophysiology, clinical presentation, diagnosis and nonthrombotic pulmonary embolism. Exp Clin Cardiol. (2013) 18:129–38.

Google Scholar

31. Mansella, G, Keil, C, Nickel, CH, Eken, C, Wirth, C, Tzankov, A, et al. Delayed diagnosis in pulmonary embolism: frequency, patient characteristics, and outcome. Respiration. (2020) 99:589–97. doi: 10.1159/000508396

Crossref Full Text | Google Scholar

32. American Lung Association . Learn about pulmonary embolism. American Lung Association (2023). Available at. https://www.lung.org/lung-health-diseases/lungdisease-lookup/pulmonary-embolism/learn-about-pulmonaryembolism#:~:text=Pulmonary%20embolism%20affects%20around%20 900%2C000,that%20travel%20to%20the%20lungs.

Google Scholar

33. Tsao, CW, Aday, AW, Almarzooq, ZI, et al. Heart disease and stroke statistics-2022 update: a report from the American Heart Association. Circulation. (2022) 145:e153– e639.l. doi: 10.1161/CIR.0000000000001052

Crossref Full Text | Google Scholar

34. American Heart Association , 2022 Heart Disease and Stroke Statistics Update Fact Sheet. 2022 Heart Disease and Stroke Statistics Update Fact Sheet At-a-Glance

Google Scholar

35. Centers for Disease Control and Prevention . (2020) Archived National Asthma Data. Available at: https://www.cdc.gov/asthma/archivedata/2020/2020_archived_ national_data.html

Google Scholar

36. Asthma and Allergy Foundation of America . Asthma facts and figures. (2023). Available at: https://aafa.org/asthma/asthma-facts/

Google Scholar

37. Braman, SS . The global burden of asthma. Chest. (2006) 130:4S–12S. doi: 10.1378/chest.130.1_suppl.4S

Crossref Full Text | Google Scholar

38. van Maanen, R, Trinks-Roerdink, EM, Rutten, FH, and Geersing, GJ. A systematic review and meta-analysis of diagnostic delay in pulmonary embolism. Eur J Gen Pract. (2022) 28:165–72. doi: 10.1080/13814788.2022.2086232

Crossref Full Text | Google Scholar

39. Magnusson, C, Herlitz, J, Sunnerhagen, KS, Hansson, PO, Andersson, JO, and Jood, K. Prehospital recognition of stroke is associated with a lower risk of death. Acta Neurol Scand. (2022) 146:126–36. doi: 10.1111/ane.13618

Crossref Full Text | Google Scholar