Effect of clinical decision support systems on emergency medicine physicians' decision-making: A pilot scenario-based simulation study

Background and objectives: Children with congenital heart disease (CHD) are predisposed to rapid deterioration in the face of common childhood illnesses. When they present to their local emergency departments (ED) with acute illness, rapid and accurate diagnosis and treatment is crucial to recovery and survival. Previous studies have shown that ED physicians are uncomfortable caring for patients with CHD and there is a lack of actionable guidance to aid in their decision making. To support ED physicians' key decision components (sensemaking, anticipation, and managing complexity) when managing CHD patients, a Clinical Decision Support System (CDSS) was previously designed. This pilot study evaluates the effect of this CDSS on ED physicians' decision making compared to usual care without clinical decision support.

Methods: In a pilot scenario-based simulation study with repeated measures, ED physicians managed mock CHD patients with and without the CDSS. We compared ED physicians' CHD-specific and general decision-making processes (e.g., recognizing sepsis, starting antibiotics, and managing symptoms) with and without the use of CDSS. The frequency of participants' utterances related to each key decision components of sensemaking, anticipation, and managing complexity were coded and statistically analyzed for significance.

Results: Across all decision-making components, the CDSS significantly increased ED physicians' frequency of “CHD specific utterances” (Mean = 5.43, 95%CI: 3.7–7.2) compared to the without CDSS condition (Mean = 2.05, 95%CI: 0.3–3.8) whereas there was no significant difference in frequencies of “general utterances” when using CDSS (Mean = 4.62, 95%CI: 3.1–6.1) compared to without CDSS (Mean = 5.14 95%CI: 4.4–5.9).

Conclusion: A CDSS that integrates key decision-making components (sensemaking, anticipation, and managing complexity) can trigger and enrich communication between clinicians and enhance the clinical management of CHD patients. For patients with complex and subspecialized diseases such as CHD, a well-designed CDSS can become part of a multifaceted solution that includes knowledge translation, broader communication around interpretation of information, and access to additional expertise to support CHD specific decision-making.

Introduction

Children born with congenital heart disease (CHD) are predisposed to rapid clinical deterioration when they become acutely ill with common childhood illnesses compared to children without CHD (1). CHD refers to the range of anatomic defects of the heart and its great vessels that effect the flow of blood through the heart and the lungs. The increased hemodynamic fragility of these patients stems from the natural history of their underlying CHD (despite corrective interventions) and the burden of residual lesions which also alter the responses of these children to traditional resuscitative measures (1). During these episodes of acute illness, patients with CHD often present to their local emergency departments (ED) where specialized training in pediatric CHD is often limited. Of the 241 million ED visits recorded in United States' Nationwide Emergency Department Sample between 2006 and 2014, CHD patients represented 0.17% of these visits (2). The majority of these patients were under 1 year of age and were more likely to die, require hospital admission, or transfer to specialty centers compared to children without CHD (2).

The incidence of CHD is approximately 8 in 1,000 live births (3). The anatomic variations of CHD are many and complex; one group of patients who are particularly vulnerable and high risk are those born with only one main pumping chamber (ventricle). In this form of CHD, blood from the body and the lungs mixes in the heart creating an oxygen level (saturation) in the blood that is lower (usually <85%) when compared to a patient with two normal ventricles. When ED physicians in the state of Michigan were surveyed on their degree of comfort caring for acutely ill pediatric patients with a single ventricle, they expressed an overall lack of comfort (4). When asked about the expected oxygen saturation in these patients, 52% of general ED physicians and 35% of pediatric ED physicians were unsure of what the expected oxygen saturations should be (4). Moreover, 18% of general ED physicians and 26% of pediatric ED physicians identified the wrong saturations for these patients (4). Some reasons for this lack of comfort and familiarity with CHD patients among ED physicians include limited specific training in acute CHD, lack of available and detailed information about the CHD in a particular patient, the unique and complex medical language used by experts to describe CHDs, and limited access to in-house CHD experts (4–6).

We identified a lack of actionable guidance to support ED physicians in their decision making when confronted with a patient with CHD and intercurrent acute illness (7), and developed a prototype Clinical Decision Support System (CDSS) using the critical decision method (CDM), called MyHeartPass^TM (Figure 1) (8). CDSS are systems that provide clinicians with disease and patient specific clinical knowledge and information to facilitate effective decision making, enhance patient care, and improve outcomes (9, 10). These systems have improved care of patients through a variety of means including drug calculations, making patient information more accessible and providing supplemental information to clinicians (10). Often, CDSS studies focus on evaluating outcomes following the use of a CDSS without showing how or why they fail or succeed in modifying outcomes or decision making (10–15). In this pilot study, we use a scenario-based simulation to analyze the frequency and typology of decisions made using the prototype CDSS compared to usual care without clinical decision support to assess which decision making components are modified.

FIGURE 1

Figure 1. Screenshot of MyHeartPass™ prototype. Sections marked with (i) refer to elements of design pertaining to key decision requirement distinguish the patient's unique physiology based on their unique cardiac anatomy. Elements marked with (ii) capture the elements pertaining to the key decision requirement of explicitly consider CHD specific differential diagnoses to allow a more structured reflection of diagnosis; and (iii) capture the elements pertaining to the key decision requirement of select CHD appropriate interventions for each patient.

Previous work

We designed MyHeartPass^TM using semi-structured interviews of CHD experts (pediatric cardiac intensivists) and ED physicians. The transcripts from CHD experts’ and ED physicians' retrospective recounts of challenging scenarios were used to compare their macrocognitive processes (16, 17). We identified and designed for the following key decision requirements that were least supported: (i) Sensemaking – distinguishing patients’ unique physiology based on their cardiac anatomy, (ii) Anticipation – Considering CHD-specific differential diagnoses to allow a more structured reflection of diagnosis, and (iii) Managing Complexity – selecting CHD-appropriate therapeutic interventions (7).

Methods

Ethics approval was obtained from the Research Ethics Board (REB 1000064567) of a large academic institution. Before participating in the scenarios, all participants were asked brief demographic questions and their degree of comfort managing patients with a specific type of CHD (patients with single ventricle physiology) who are considered most fragile and most likely to acutely present to the ED. This pilot study was conducted as a partial counter-balanced repeated measures pilot study where half the participants completed their first scenario with the CDSS and the second without.

Materials

Interviews were conducted using Microsoft Teams®, a virtual meeting platform, and Miro (Version 0.7.2), an individualized virtual idea board. In scenarios where participants had access to the CDSS, MyHeartPass™ was used as the prototype CDSS for the scenario. Scenario data was provided to all participants based on the scenario they were in (Table 1). In keeping with current practice, for all scenarios, supplemental documents and diagnostic data were made available to participants only when they would ask for it (Table 1). These supplemental documents and diagnostic data are routinely available during clinical care but have to be deliberately accessed by clinicians managing the patient. All participants were presented with MyHeartPass™ at the start of their CDSS Scenario with a brief 5-minute orientation to the CDSS.

TABLE 1

Table 1. List of available data, documents, and diagnostic results made available to participants.

Setting and participants

As the intended end users of MyHeartPass™, pediatric emergency medicine physicians working in a quaternary pediatric hospital were recruited to participate in this pilot study. The ED in the pilot study hospital sees over 50,000 pediatric patients, has 6,000 admissions annually (18), and at the time was staffed with 26 full-time and part-time ED physicians. Of these physicians, 22 met the following inclusion criteria: (a) no formal training in acute pediatric CHD treatment, (b) at least 3 years of experience in the field of pediatric emergency medicine, and (c) actively practicing as staff physicians at the time of the pilot study. The pilot study hospital was a major cardiac center with an average of 550 pediatric cardiac surgeries per year. In addition to caring for these patients post-operatively, the cardiac intensive care unit of this hospital also medically managed pediatric congenital and acquired heart disease patients (19). Based on their training, experience, and role in managing these patients at their highest acuity, pediatric cardiac intensivists were deemed to be experts in the acute management of pediatric patients with CHD. The pediatric cardiac critical care unit of the pilot study hospital was staffed with 9 full-time physicians.

Virtual simulations and scenarios

All virtual simulations were conducted one-on-one on Microsoft Teams®. Participants were free to choose a location of their convenience for this simulation (i.e., their personal office at work or at home). Simulations were run by AA, a pediatric nurse practitioner in the cardiac intensive care unit at the pilot study institution and a doctoral student in human factors engineering with experience conducting medical scenario-based simulations and research interviews. At the onset of the pilot study, participants were briefed on the steps of the pilot study and broadly oriented to MyHeartPass™ and Miro. Given that participants had already used Microsoft Teams® and were familiar with the platform, no orientation to this virtual meeting platform was provided. Participants were also informed that for all scenarios, they can ask for any additional information (i.e., items from the supplemental documents and diagnostic data in Table 1) that they normally would, and those results will be made available to them as they would be under the current standard of care conditions.

Mock scenarios were specifically designed by CHD experts (AA and PL) from the study team for this virtual simulation pilot study. The scenarios resembled the presentation and clinical evolution of acutely ill pediatric patients with CHD in an ED. Specifically, an intercurrent illness was incorporated into all the scenarios and each patient was made to deteriorate based on one of their common CHD specific mechanisms of deterioration. Each scenario was presented in stages, allowing participants to provide their differential diagnoses, thoughts, rational, and actions at each stage. These responses were documented on Miro. Scenarios were stopped when participants identified the right diagnosis and treatment recommendations, after obtaining a complete consultation from cardiology, or after reaching the allotted time, whichever came first. For member checking, each participant was asked to review their actions and decisions on Miro at the culmination of each scenario to ensure nothing was omitted or misinterpreted. Member checking is a validation technique that is used to explore the credibility of the results by seeking feedback from participants (20). A small pilot study of all the scenarios was completed with a CHD expert outside the study team in advance of participant recruitment to ensure scenarios were similar in complexity, clinically accurate and realistic while also ensuring the questions and the simulation itself ran smoothly within the allotted time.

ED physician participants completed two scenarios in total, one with MyHeartPass™ and another without. Each scenario was allotted 30 min to reflect the approximate amount of time these patients would spend in the ED before being seen by cardiology or the rapid response team from intensive care. CHD-expert participants completed all their scenarios without MyHeartPass™ within the same allotted time as the ED physicians.

Data analysis

Scenarios were audio and video recorded, transcribed verbatim, and analyzed using NVivo 12® software. Two coders (AA and TR) with extensive experience coding interview data analyzed the interview transcripts. Coders independently identified utterances of decisions and reviewed their decision utterances for overlap (i.e., did they both identify the same decision utterances) to come to consensus. Once all utterances of decisions were identified, coders independently coded each decision utterance for key decision components based on a priori operational definitions identified in a previous study (Table 2) (8). They also coded based on whether a decision utterance was CHD-specific (e.g., choosing to start an epinephrine infusion to support the function of the heart and increase cardiac output) or a general decision component (e.g., choosing to start antibiotics to treat a potential infection) to evaluate the effect of the CDSS on participants' non-CHD specific (general) decision making (i.e., recognition, investigation, and management of the intercurrent illness and other possible differential diagnoses). Only those decision utterances that were relevant to understanding patient state, diagnosing, and evaluating hypotheses as well as the management of the patient in the scenario were included in the analysis (e.g., prescribing an epinephrine infusion to support patients heart function [relevant] vs. describing how many times they have prescribed epinephrine for these patients in the ED [irrelevant]). The range of relevant utterances was defined based on the range of utterances of CHD experts and the clinical expertise of AA (also a CHD expert). Repeated utterances of the same decision were only counted once (e.g., if a participant asked for an ECG twice without describing a new indication for a repeat ECG, only their first utterance of the decision was counted). Researchers compared coding for inter-rater reliability until k > 0.7 was achieved (21), at which point, a single researcher proceeded to code all subsequent simulations. All coded simulations were charted in a matrix. The matrix consisted of a table that organized the utterances by the simulation in which they occurred, and by the key decision components (CHD-specific vs. general). This matrix summarized the utterances across the key components which facilitated analysis of their frequency and qualitative characteristics. Frequency of utterances were analyzed in a 3 (decision making component: sensemaking vs. anticipation vs. managing complexity) × 2 (format: without CDSS vs. with CDSS) × 2 (type: general non-cardiac vs. CHD-specific) repeated measures analysis of variance (ANOVA). Analyses were conducted using IBM SPSS Version 26 (IBM Corp., 2014), α < 0.05 with Bonferroni correction for pairwise comparisons.

TABLE 2

Table 2. Analytical framework.

Results

Overview

In total, 7 of the eligible 22 pediatric ED physicians (32%) participated in this pilot study, all of whom were able to complete their scenarios within the allotted time. Table 3 shows demographic characteristics of the ED participants. Three of the 9 CHD experts (33%) also participated in the pilot study as study controls.

TABLE 3

Table 3. Ed physician participant characteristics.

Although the three-way interaction was not significant, there was a significant interaction between format (i.e., with and without CDSS conditions) and type (general and CHD-specific utterances). Specifically, across all decision-making components, the CDSS significantly increased ED physicians' frequency of “CHD-specific utterances” (Mean = 5.429) compared to the without CDSS condition (Mean = 2.048) whereas there was no significant difference in frequencies of “general utterances” when using CDSS (Mean = 4.619) compared to without CDSS (Mean = 5.143) [F(1,6) = 24.48, p < 0.01] (Figure 2).

FIGURE 2

Figure 2. Effect of MyHeartPass™ on ED physicians’ general and CHD-specific decision-making. The use of MyHeartPass™ (CDSS) is seen to have increased utterances of CHD-specific decisions without effecting utterances of general decisions. Error bars represent the 95% confidence interval.

Effect of CDSS on each CHD-specific decision components

A detailed review of ED physicians' utterances of decisions reveals the ways in which MyHeartPass™ effects each of their 3 key decision components.

Sensemaking – understanding patient's CHD-specific state

ED physicians always referred to the patients' acceptable baseline oxygen saturations in MyHeartPass™ to not only understand what oxygen saturation they should target for the patient when choosing treatments, but also to understand their risk state. For example, during the blocked Blalock-Taussig (BT) shunt scenario, PEM02 referenced MyHeartPass™, commenting that “here (baseline saturation is) listed between 70 and 80. (It) is actually super helpful because 64 is not as awful as it could be if their saturations were supposed to be at 85 to 95 for example.” Conversely, in the absence of MyHeartPass™, ED physicians treated CHD patients based on what they assumed would be appropriate saturations for the patient until confirming it with consulting CHD experts or the patients' electronic health records.

ED physicians also used the graphical illustration of patients' unique cardiac anatomy and associated description of physiological states in MyHeartPass™ to conceptualize the anatomy and begin to understand the effect of anatomy on patient physiology. For example, PEM04 explained that in patients with Fontan physiology “flow to the lungs is passive and there”s this huge sort of restriction to that because of the pathology in the right lung.” Without MyHeartPass™ however, they did not describe the patients' cardiac physiology or risk state as part of their patient assessment.

Anticipation – formulating CHD-specific differential diagnoses and testing hypotheses

After an initial phase of sensemaking and establishing an understanding of the patient's current state, participants engaged with the decision component of anticipation and formulated differential diagnoses and prescribed diagnostic tests to narrow their list of differentials and test their hypotheses. This was an iterative process where new data modified clinicians' sensemaking which in turn modified their list of possible differential diagnoses. While the use of MyHeartPass ™ improved ED physicians' sensemaking, it was not effective at bridging the gap between identifying heart failure as a possible diagnosis and the specific cardiac cause to explain the failure. For example, when treating the patient with the failing Fontan circulation, PEM07, like most other participants, was able to identify that the patient was in “a septic shock or a cardiogenic shock… the two most likely things and they could be both happening.” However, despite correctly diagnosing the pleural effusion (a non-cardiac diagnosis) on x-ray and then on point of care ultrasound, they were unable to make the link between the pleural effusion and the failure of the patient's Fontan circulation.

Nevertheless, with MyHeartPass™, ED physicians correctly prescribed more physiology based diagnostic investigations (e.g., echocardiography to assess patency of shunts, point of care ultrasound to evaluate cardiac function, and apply oxygen to determine cause for desaturation in certain heart disease), particularly when the test was recommended in the CDSS. For example, when treating the patient with obstructed BT shunt using MyHeartPass™, PEM06 was able to identify a lack of response to supplemental oxygen as a positive diagnostic indicator for “a blocked BT shunt” and proceed to request an echocardiography to confirm this suspicion. This contrasted with when ED physicians did not have MyHeartPass™, they prescribed fewer physiology specific diagnostic tests. For example, they broadly asked to speak with the CHD expert, expecting them to determine the specific questions to ask on echocardiography and the urgency with which the test was required.

Managing complexity – treating the patient with CHD-specific interventions

Participants initiated interventions at various points throughout each scenario. Some interventions were delivered to treat the underlying physiology, such as initiating epinephrine to support the function of the heart or starting positive pressure to offset work of breathing and oxygen consumption. These physiology-based interventions occurred more frequently when ED physicians had access to MyHeartPass™. For example, when PEM03 read the latest echocardiography of the mock patient in MyHeartPass™ said “I would consider starting her on presser like Epinephrine infusion to help with her myocardial demand. Looking at her latest Echo show some decrease systolic function and it says here that she does have systolic dysfunction at baseline, so my concern is … I need her to pump to increase to optimize her cardiac output”. However, in the absence of MyHeartPass™, ED physicians often focused on treating symptoms rather than the CHD physiology. For example, when treating the mock Fontan patient with Fontan failure due to poor pulmonary blood flow in the context of a large pleural effusion, PEM04 considered “volume resuscitation vs. diuretics with regard to, you know, potentially needing to start Epinephrine” without appreciating the effect of each intervention on the failing Fontan circuit.

In general, when ED physicians had access to MyHeartPass™, they readily opted to use inotropes and pressors (e.g., epinephrine) to support blood pressure and augment cardiac output. Without the use of MyHeartPass™, ED physicians were more hesitant to use these drugs, often opting to manage hemodynamics with cautious fluid administration instead.

Effect of CDSS on decision making

The use of MyHeartPass™ did not have any significant influence on ED physicians' general utterances of decision making. Importantly, the use of MyHeartPass™ did not lead to any of the participants missing the key general differential diagnoses (i.e., pneumonia and sepsis) and their associated treatments (i.e., escalation in respiratory support and administration of antibiotics). A summary of these results can be found in Figure 2.

Lastly, participants described MyHeartPass™ as an overall helpful tool that makes patient specific information more readily available to them. Participants using MyHeartPass™ described feeling empowered to advocate for their patients and seek further expertise. For example, PEM06 explains that “You get a real baseline understanding of like where the kid is anatomically. (…) you can speak to the cardiologists, and you could actually start to have a more targeted discussion around. Hey, I see that (…) this kid has this BT shunt and is sating 60. Is it possible there's a shunt blockage?” PEM06 goes on to say that “It'll empower us to advocate for the cardiology fellow to come and do the echo in the ED because that's really what we're doing when we're calling (cardiology). We're trying to figure out should the cardiology fellow come and see the kid immediately in the emergency department? (…) It's very empowering to know that you could you yourself as the Emerge physician can come to the conclusion that yes, I think the cardiology fellows should come down to the ED, and this is why. That's empowering.” Similarly, PEM07 said that “it was nice to have something to go by” when reviewing the patient with the CHD expert.

Discussion

Findings from this pilot study show that the use of a prototype CHD CDSS improves the CHD-specific decision-making of ED physicians, specifically their sensemaking, anticipation, and management of complexity. Our findings show that the use of our prototype CDSS facilitated CHD-specific sensemaking through a greater understanding of patients' unique CHD anatomy, physiology, and expected vital signs. Improvements in anticipation were due to a greater number of CHD-specific diagnostic tests being prescribed by ED physicians with minimal improvement in their ability to identify the appropriate CHD-specific diagnosis. Despite not consistently identifying the correct diagnosis, the use of the CDSS still improved management of complexity in these patients by increasing the number of appropriate CHD-specific interventions ED physicians prescribed. There was also no significant effect on clinicians' general (non-cardiac) sensemaking, anticipation and management of complexity while using the proposed CDSS. Our findings therefore suggest that designing CDSS with a focus on specific decision-making cognitive processes can improve decision making, communication, and confidence among clinicians caring for acutely ill pediatric patients with CHD.

Like the participants in Cashen et al.'s (2011) study, most ED physicians in our pilot study were overall uncomfortable or somewhat uncomfortable managing pediatric patients with CHD without support from pediatric cardiology. In their study, Cashen et al. (2011) found a mismatch between the experience and comfort of general and pediatric ED physicians with single ventricle physiology and the educational priority the acute management of these patients should take. While the total number of pediatric patients with single ventricle physiology presenting to the ED with cardiorespiratory issues is small compared to all other causes of pediatric presentation, these patients are often hemodynamically fragile and deteriorate rapidly leading to a high burden of morbidity and mortality (2). Given the vast knowledge burden expected of ED physicians, additional training in the management of pediatric CHD may be insufficient on its own and made more effective with the use of sociotechnical solutions such as our proposed CDSS that aims to support clinicians' decision making. While not specifically measured, CDSS such as the one proposed here can also provide some reassurance and structure in clinicians' decision making as well.

To explain clinical and diagnostic reasoning, multiple theoretical constructs have been created. In general, they involve collection of clinical data utilized for the formulation of differential diagnoses based on clinicians' inherent knowledge and pattern recognition, and scientific reasoning (22). This is the process of sensemaking or understanding a patient's state. To refine the list of differentials, clinicians prescribe diagnostic tests to gather additional data and apply their broader knowledge of disease probabilities and other patient specific modifiers to ultimately decide on a diagnosis which would drive management decisions (22). Our findings show that the proposed CDSS was effective at augmenting CHD-specific decision making without distracting ED physicians from diagnosing and treating other non-CHD causes of illness based on their presentation. This is particularly important given that treating the intercurrent illness is part of the treatment strategy to improve the cardiac output in children who also develop heart failure given their CHD. The qualitative differences between ED physicians' sensemaking, anticipation, and management of complexity suggest the CDSS will not serve as a replacement to involving CHD experts in the management of these patients. Indeed, the ability to transform the fundamental understanding of a patient's CHD to what could be expected of their physiology in response to an intercurrent illness, specific diagnostic investigations to determine the illness physiology, and the rationale for various treatment choices requires expertise and experience in this area that cannot be replaced with a CDSS. Improving clinicians' sensemaking through the use of CDSS can, however, empower clinicians to have more informed discussions and dialogue with CHD experts, and approach the management of these patients from a primary cardiac perspective while concurrently managing underlying illnesses. This is particularly relevant with CDSSs that make prescriptive recommendations without triggering discussions to ensure relevance of recommendation in the context of the patient. To further support ED physicians' decision making, future designs of this tool should focus on augmenting anticipation through addressing the synthesis gap between sensemaking and anticipation. This involves a greater understanding of the CHD, the implication of patients' physiology, the natural history of the CHD, and the effect of residual lesions on possible causes of heart failure.

Limitations

This pilot study was conducted as a single center pilot study at a major cardiac center, where the existing referral structures and greater CHD exposure of ED physicians may have influenced the findings of this pilot study and made the sample size small. To better understand the effect of CDSS on clinicians' decision making, the study should be expanded to evaluate the effect of variables such as participant role (i.e., trainees vs. nurses, vs. pediatric ED physicians vs. general ED physicians), scenario complexity (i.e., complex CHD vs. simple CHD), setting (i.e., community ED, vs. academic ED, vs. pediatric ED, vs. specialty centre ED), and gender. Despite the use of realistic scenarios, the simulated nature of this pilot study may have also affected participants' responses. However, as participants responses were compared against their own control scenarios, any simulation effect would be equal across both scenarios. The significant findings obtained in this pilot study, therefore, seem to be reflective of the intervention (i.e., CDSS) and not an artifact of simulation. Nevertheless, it is important to repeat this study in a near live simulation setting to understand the effect of the CDSS on team performance and decision making under more realistic circumstances.

Conclusion

In the ED, where decisions are made under the pressure of time and uncertainty, it is crucial to understand how any proposed CDSS considered for this space effects clinicians' decision making. Our proposed CDSS was intended to support ED physicians' sensemaking, anticipation, and management of complexity when managing acutely ill paediatric patients with CHD. In evaluating the effect of this CDSS on ED physicians' understanding of CHD and decision-making pertaining to its acute management, improvements in clinician's sensemaking and treatment decision making were noticed. This included an improvement in understanding the CHD anatomy and its effect on how the blood flows through their unique heart and lungs, the influence of the anatomy on normal baseline physiological vital signs and certain limited vulnerabilities these children had because of it. The CDSS also did not distract clinicians away from considering other crucial non-cardiac diagnoses (such as sepsis) and interventions (such as early administration of antibiotics). However, this prototype did not improve CHD specific diagnostic accuracy or rationalization of diagnostic and treatment choices as it did not improve clinicians' anticipation. While this CDSS was successful at supporting clinicians' decision making as it related to determining appropriate interventions, future iteration of this design should focus on improving anticipation among clinicians. Ideally, these sociotechnical solutions should be part of a broader, multifaceted solution that include education and access to additional expertise, to improve the care of these vulnerable patients.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by Hospital for Sick Children Research Ethics Board. The patients/participants provided their written informed consent to participate in this study.

Author contributions

AA analyzed and interpreted the data as well as conduct the pilot study and drafted the initial manuscript. PL, PCT, and MG contributed to the data interpretation and the study design as well as subsequent editing of the manuscript. PL provided clinical insight while PCT provided expertise on study method and principles of cognitive task analysis. GF facilitated with protocol refinement and added valuable insights from the ED when interpreting the data. All authors contributed to the article and approved the submitted version.

Funding

This project, as part of a PhD studentship was funded by the Natural Sciences and Engineering Research Council of Canada (NSERC: CRE-ATE) and SickKids' Heart Centre Innovation Fund.

Acknowledgments

We would like to thank the staff physicians in the pediatric cardiac intensive care unit and emergency department who consented to participate and publish study findings. We also would like to thank Tiffany Ramanathan who participated in the independent qualitative analysis of the data.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Marino BS, Tabbutt S, Maclaren G, Hazinksi MF, Adatia I, Atkins DL, et al. Cardiopulmonary resuscitation in infants and children with cardiac disease: a scientific statement from the American heart association. Circulation. (2018) 137:e1–92. doi: 10.1161/CIR.0000000000000524

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Edelson JB, Rossano JW, Griffis H, Dai D, Faerber J, Ravishankar C, et al. Emergency department visits by children with congenital heart disease. J Am Coll Cardiol. (2018) 72(15):1817–25. doi: 10.1016/j.jacc.2018.07.055

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Bouma BJ, Mulder BJM. Changing landscape of congenital heart disease. Circ Res. (2017) 120(6):908–22. doi: 10.1161/CIRCRESAHA.116.309302

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Cashen K, Gupta P, Lieh-Lai M, Mastropietro C. Infants with single ventricle physiology in the emergency department: are physicians prepared? J Pediatr. (2011) 159(2):273–277.e1. doi: 10.1016/j.jpeds.2011.01.031

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Grap MJ. Nursing documentation and patient mortality. Am J Crit Care. (2013) 22(4):306–13. doi: 10.4037/ajcc2013426

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Laussen PC, Assadi A, Moga M-A, MacLaren G. Challenges for cardiopulmonary resuscitation in infants and children with cardiac disease in the perioperative setting. J Cardiothorac Vasc Anesth. (2019) 33(10):2618–21. doi: 10.1053/j.jvca.2019.06.024. Available at: https://linkinghub.elsevier.com/retrieve/pii/S105307701930577431358440

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Assadi A, Laussen PC, Freire G, Trbovich P. Understanding clinician macrocognition to inform the design of a congenital heart disease clinical decision support system. Front Cardiovasc Med. (2022) 9(February):1–10. doi: 10.3389/fcvm.2022.767378

CrossRef Full Text | Google Scholar

8. Assadi A, Laussen PC, Freire GF, Ghassemi M, Trbovich P. Decision-Centered design of a clinical decision support system for acute management of pediatric congenital heart disease. Front Digit Heal. (2022):4. doi: 10.3389/fdgth.2022.1016522

CrossRef Full Text | Google Scholar

9. Osheroff JA, Pifer EA, Teich JM. Improving outcomes with clinical decision support: an implementer’s guide. Healthcare Information and Management Systems Society (HIMSS). Chicago (2005).

Google Scholar

10. Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, et al. Clinical decision-support systems. Ann Intern Med. (2012) 157(1):29–43. doi: 10.7326/0003-4819-157-1-201207030-00450

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Geurts D, de Vos-Kerkhof E, Polinder S, Steyerberg E, van der Lei J, Moll H, et al. Implementation of clinical decision support in young children with acute gastroenteritis: a randomized controlled trial at the emergency department. Eur J Pediatr. (2017) 176(2):173–81. doi: 10.1007/s00431-016-2819-2

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Ballard DW, Kuppermann N, Vinson DR, Tham E, Hoffman JM, Swietlik M, et al. Implementation of a clinical decision support system for children with Minor blunt head trauma who are at nonnegligible risk for traumatic brain injuries. Ann Emerg Med. (2019) 73(5):440–51. doi: 10.1016/j.annemergmed.2018.11.011

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Ekstrom HL, Kharbanda EO, Ballard DW, Vinson DR, Vazquez-Benitez G, Chettipally UK, et al. Development of a clinical decision support system for pediatric abdominal pain in emergency department settings across two health systems within the HCSRN. eGEMs. (2019) 7(1):15. doi: 10.5334/egems.282

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Müller L, Srinivasan A, Abeles SR, Rajagopal A, Torriani FJ, Aronoff-Spencer E. A risk-based clinical decision support system for patient-specific antimicrobial therapy (iBiogram): design and retrospective analysis. J Med Internet Res. (2021) 23(12):1–14. doi: 10.2196/23571

CrossRef Full Text | Google Scholar

15. Bi J, Abraham A. An intelligent web-based decision support tool for enhancing asthma guideline adherence. IHI’12 - proc 2nd ACM SIGHIT int heal informatics symp. (2012). p. 51–9. doi: 10.1145/2110363.2110373

16. Klein G, Wright C. Macrocognition: from theory to toolbox. Front. Psychol. (2016) 7(January):1–5. doi: 10.3389/fpsyg.2016.00054

CrossRef Full Text | Google Scholar

17. Klein G, Ross KG, Moon BM, Klein DE, Hoffman RR, Hollnagel E. Macrocognition. IEEE Intell Syst. (2003) 18(3):81–5. Available at: http://ieeexplore.ieee.org/document/1200735/ doi: 10.1109/MIS.2003.1200735

CrossRef Full Text | Google Scholar

18. The Hospital for Sick Children. Emergency Medicine. Available at: https://www.sickkids.ca/en/care-services/clinical-departments/emergency-medicine/ (cited April 11, 2021).

19. The Hospital for Sick Children. Critical Care Medicine. Available at: https://www.sickkids.ca/en/care-services/clinical-departments/critical-care-medicine/ (cited April 11, 2021).

20. Birt L, Scott S, Cavers D, Campbell C, Walter F. Member checking: a tool to enhance trustworthiness or merely a nod to validation? Qual Health Res. (2016) 26(13):1802–11. doi: 10.1177/1049732316654870

PubMed Abstract | CrossRef Full Text | Google Scholar

21. McHugh ML. Interrater reliability: the kappa statistic. Biochem Med. (2012) 22(3):276–82. doi: 10.11613/BM.2012.031

CrossRef Full Text | Google Scholar

22. Thammasitboon S, Cutrer WB. Diagnostic decision-making and strategies to improve diagnosis. Curr Probl Pediatr Adolesc Health Care. (2013) 43(9):232–41. doi: 10.1016/j.cppeds.2013.07.003

PubMed Abstract | CrossRef Full Text | Google Scholar