“Debriefing and Organizational Lessons Learned” (DOLL): A Qualitative Study to Develop a Classification Framework for Reporting Clinical Debriefing Results

Background: The COVID-19 crisis has radically affected our healthcare institutions. Debriefings in clinical settings provide a time for the clinicians to reflect on the successes (pluses) and difficulties (deltas) encountered. Debriefings tend to be well-received if included in the broader management of the unit. The goal of this study was to develop a framework to categorize these debriefings and to assess its worthiness.

Methods: A qualitative approach based on a grounded theory research method was adopted resulting in the “Debriefing and Organizational Lessons Learned” (DOLL) framework. Debriefings were conducted within two Emergency Departments of a Belgian University Hospital during an 8-week period. In the first step, three researchers used debriefing transcripts to inductively develop a tentative framework. During the second step, these three researchers conducted independent categorizations of the debriefings using the developed framework. In step 3, the team analyzed the data to understand the utility of the framework. Chi-square was conducted to examine the associations between the item types (pluses and deltas) and the framework's dimensions.

Results: The DOLL is composed of seven dimensions and 13 subdimensions. Applied to 163 debriefings, the model identified 339 items, including 97 pluses and 242 deltas. Results revealed that there was an association between the frequency of pluses and deltas and the dimensions (p < 0.001). The deltas were mainly related to the work environment (equipment and maintenance) (p < 0.001) while the pluses identified tended to be related to the organization of the unit (communication and roles) (p < 0.001). With leadership's support and subsequent actions, clinicians were more enthusiastic about participating and the researchers anecdotally detected a switch toward a more positive organizational learning approach.

Conclusion: The framework increases the potential value of clinical debriefings because it organizes results into actionable areas. Indeed, leadership found the DOLL to be a useful management tool. Further research is needed to investigate how DOLL may work in non-crisis circumstances and further apply the DOLL into incident reporting and risk management process of the unit.

Introduction

The COVID-19 crisis has affected our healthcare institutions (1). Hospitals had to face major economic (2), structural (3) and organizational (4) challenges in a high workload, uncertain and constantly changing environment. This raised concern about impacts on quality and patient safety (5) and a notable psychological burden on professionals (6). Initiatives, such as the Circle-Up process, were developed to help deal with the consequences of the crisis (7). Circle-up is a process aiming to connect team members and to facilitate organizational learning based on three activities: before shift briefings, during shift peer check-ins and post shift debriefings. The before shift briefing is to decide on a work plan, clarify procedures, roles and responsibilities and to create a team spirit. During a shift, peer check-ins help report on the situation, update team situation awareness and offer support to the clinicians. Post shift debriefings provide a time for the participants to reflect on shift successes and difficulties (7). During the post-shift debriefing, the participants gather to review the entire shift. The value of these team debriefings in the clinical environment appears to be twofold. First, they help share information within the clinical team and with leadership to solidify operations where things were working well and to improve things not working as well as they could (7–12). Secondly, clinical debriefings might help to provide a safe place for controlled psychological support, situational awareness, workload management and operational support (8).

Past research resulted in the development of guidance to conduct clinical debriefings (8, 9, 13, 14). During COVID-19, the Debriefing In Situ COVID-19 to Encourage Reflection and Plus-Delta in Healthcare After Shifts End (DISCOVER-PHASE) was specifically developed to implement a standardized clinical team debriefing program (15). The DISCOVER-PHASE seeks to analyze all types of events encountered and was designed to be used at the end of shift (13). It employs a plus/delta method. Indeed, the importance of learning not only from failure but also from success and leadership's wholehearted and visible commitment to act on things that are going well (pluses) and things that need improvement (deltas) appears essential.

Clinical debriefings are a relatively new area of practice. Studies have investigated the most effective method for conducting debriefings in the clinical environment (8, 9, 15–18). However, evidence regarding the frequency of post-shift debriefings and their embedding in a global strategy is scarce. Based on the scarce findings to date, debriefings appear to hold promise to be a keystone for promoting a learning organization culture and for triggering quality and safety improvement (9, 19). In that perspective, providing feedback on the teams' input is an essential programmatic element that could garner support from the clinician teams and for organizational learning. Strengths and opportunities for improvement should be addressed and comparisons with experiences made to identify what remedial actions should be undertaken and the underlying thought processes for those actions. Such commitments could prove difficult to achieve in the absence of structured and standardized tools for the appropriate processing of the data collected during these debriefings. To date, no formal classification system has been developed to organize debriefing data and allow the results to be easily integrated into a unit's internal and external management, resourcing, and coordination of the department. Therefore, we designed the present study with the aim to develop a framework to categorize results from regularly implemented post-shift debriefings into a useful and actionable framework (DOLL) and to perform an initial assessment of its feasibility and utility in the real world of clinical care. With this idea in mind, the framework was employed during a two-month period of the COVID-19 pandemic.

Materials and Methods

Study Design

We used a three-step approach to conduct this study. The first step relied on an adapted qualitative approach based on grounded theory research to construct the DOLL framework (20). It used content analysis of post-shift debriefings in the clinical environment to move beyond description and to generate a theory grounded through meaningful categories of information. The second step was comprised of implementing post-shift debriefings and categorizing the debriefing content. The third step was to analyze the quality of the data and reflect on the utility of the framework. Standards for Reporting Qualitative Research (SRQR) (21) were used to help guide the study.

Study Site

Data was collected from two Emergency Departments (ED) of a single Belgian University teaching hospital with two geographically separated facilities: Main and Satellite. The Main facility is a tertiary care hospital located in a suburban area, while the Satellite is an urban secondary hospital. The ED from the Main facility was raised under the cultural umbrella of a Public University Teaching Hospital while the second ED history started as part of a private clinic that was merged with the Main Hospital. The two sites combine an annual ED census of approximately 100,000 patients: with the Main handling approximately 57% and the Satellite handling 43%. The department employs approximately 50 physicians and 120 nurses. Nurses are assigned to Main or Satellite while physicians are scheduled at both sites.

Data Collection

Data was collected on 163 debriefings conducted during 8 weeks, from March 10 to May 10 of 2020. The debriefings were led in French. The median number of attendees per debriefing was 5 (IQR: 4–6) and remained stable over the observed period. These debriefings were performed after morning and afternoon shifts among medical and nursing members of the ED in the two facilities. Clinical debriefings had a median duration of 10 min. The durations were longer in the first 2 weeks and decreased by over 50% in the latter half of the study period. The same facilitator led most debriefings (142 of the 163) while five other facilitators performed the remaining debriefings (21 of 163). All debriefers had a clinical background and previous training in Debriefing With Good Judgment (22) and the Plus-Delta method. Of the 163 debriefings, 94 were held at the Satellite and 93 were held in the Main facility. DISCOVER-Phase tool was used to collect the following post-shift debriefing data: date, site, shift, members, “plus/delta” and initiatives proposed. Before the project was undertaken, the facilitator (MP) spent one immersive month becoming familiarized with the then-current ED structures, procedures, equipment, and teamwork habits.

Data Analysis

Building the Classification Framework

Three researchers (MP, GG, TS) from different occupational categories (quality and safety, process manager, and psychologist) read all the transcribed debriefing reports. The different backgrounds provided a multidimensional view for interpreting results. The researchers further self-reflected about a classification framework based on their specific background and the transcribed debriefings. This step led to the identification of three candidate frameworks: (a) “Framework for analyzing risk and safety in clinical medicine” (23), (b) Ishikawa/Fishbone diagram (24) and (c) the “European framework for psychosocial risk management” (25). Once the frameworks were chosen, each researcher refined the existing categories for their candidate framework with the topics raised in all 163 debriefings. Then, the researchers shared and compared their frameworks together to decide which categories were relevant and should be retained or discarded. This broad-then-narrowing of the categories was followed by a comprehensive in-person discussion that resulted in the “Debriefing and Organizational Lessons Learned (DOLL)” framework (Figure 1).

FIGURE 1

Figure 1. Paquay-Ghuysen Debriefing and Organizational Lessons Learned (DOLL) © framework.

Debriefings Categorization and Content Analysis

The researchers started the analysis by individually reading the transcripts several times to become familiar with the data. The research team started collectively the sorting on a subset of the data to reach a reasonable understanding and consistency of the process. Then, the researchers worked independently to classify each item as “plus” or “delta” and then placed items into the DOLL framework. Resulting individual analyses were then compared and discussed by the research team until consensus was reached. For that purpose, the three researchers exposed each item and its classification. When a classification was not the same between the researchers, they reanalyzed the item and collectively reached 100% reconciliation through discussion, curiosity, frames exploration and clarification using a process similar to a learning pathway grid (26). Due to their different professional backgrounds, discussion helped sharing individual perspectives and understandings.

Statistics

Descriptive statistics were performed to summarize the frequencies and percentages of the pluses and deltas in each dimension of the DOLL. Chi-square was conducted to examine the association between the type of items (plus or deltas) and the Dimensions followed by post-hoc tests using adjusted standardized residuals. Fisher exact test was employed. Furthermore, z-test for independent proportions was applied to investigate if there were any significant differences in the number of plusses and deltas across the different dimensions. Bonferroni correction was used for all post-hoc comparisons.

Ethical Considerations

This study was approved by the ethical committee of the Liege University Medicine Faculty with the reference number 2020/252. All participants gave informed consent to participate in the study.

Results

Classification Framework Development

The classification framework developed is depicted in Figure 1. It is composed of 7 Dimensions and 13 Subdimensions. The agreed upon Dimensions and Subdimensions are described in Table 1.

TABLE 1

Table 1. Debriefing and Organizational Lessons Learned (DOLL) framework dimensions and subdimensions.

Debriefing Categorization Using DOLL

The 163 debriefings were evenly distributed between the two sites. Out of these debriefings, 339 items were identified including 97 pluses (29%) and 242 deltas (71%) (Table 2). During the “compare and discuss” process, classification was not the same for 64 items (18%) and needed a second collective analysis to reach 100% consensus.

TABLE 2

Table 2. Percentages of Plus and Delta based on the “Debriefing and Organizational Lessons Learned (DOLL)” framework dimensions.

Pluses were equally distributed between the two sites (49% Main−51% Satellite). Due to the presence of cells with zeros, Fisher exact test was performed. The result revealed that there was an association between the frequency of pluses and deltas and the different Dimensions (p < 0.001). Post-hoc examination using adjusted standardized residuals and Bonferroni correction (i.e., value < 0. 0036 being significant) revealed that there were more pluses expected in the Dimension Individuals (p < 0.001) and Internal unit organization (p < 0.001). On the contrary, there were more deltas than expected in the Dimension Work environment (p < 0.001).

Z-test for independent proportions using Bonferroni correction indicated that there were significantly less deltas in the Dimension Individuals (54.3%, p < 0.05) compared to that of Other hospital units (93%), Work environment (93%), Institution (100%), and Patient (100%). There were significantly less deltas in the Dimension Internal unit organization (42%, p < 0.05) compared to that of Procedures (83%), Work environment (92%), Other hospital units (93%) and Institution (100%).

Within the pluses, there were significantly more pluses in the Dimensions Individuals (46%, p < 0.05) and Internal unit organization (58%, p < 0.05) than that of Other hospital units (7%), Work environment (8%), and Procedure (16%). Results are presented in Table 3.

TABLE 3

Table 3. Number of deltas and pluses across different dimensions.

Debriefing Content Analysis Using DOLL

Table 4 presents the Dimensions, Subdimensions and examples of what participants stated for each Dimension in the DOLL.

TABLE 4

Table 4. Debriefing and Organizational Lessons Learned (DOLL) framework dimensions and subdimensions examples.

Patient

Two items were associated with this dimension and were exclusively deltas. Teams highlighted the difficulty of taking care of extreme and rare clinical cases or patient violence despite having all the procedures and competencies at their disposal.

Individuals

This dimension was mostly marked by the pluses in the Subdimension People behavior (70.3%) and by the deltas in the Subdimension Training and knowledge of the workplace (72.7%). Related to the Dimension, teams positively reported behaviors of solidarity, mutual aid, commitment to the work and the team. On the other hand, teams expressed difficulty in getting familiar with the environment and procedures, especially when it concerned nurses on assignment to the ED from other units. Lastly, Physical and psychological state was mostly characterized by deltas (85%) with participants expressing their stress.

Internal Unit Organization

This Dimension included greater pluses (58%) than deltas (41%). Pluses focused on the Roles, tasks, and responsibilities (60%) while Humans and materials resource management was mainly defined by deltas (70.0%). Teams complimented the established organization, particularly task and role assignment between physicians and nurses. However, the management of teams and materials was problematic for the staff. In this respect, participants explained that the workload was more unpredictable than usual and that it was difficult to anticipate how to adequately allocate human resources. On the other hand, communication was more often reported as positive (69%), which referred mainly to the quality of medical-nursing communication.

Procedures

This Dimension was comprised almost exclusively by deltas (83%). Teams highlighted the changes on an almost daily basis in procedures or the absence of procedures for frequently encountered problems.

Work Environment

This Dimension was also largely described by deltas (93%). Comments by participants revealed that Material and equipment (36%), Computer technology (37%) and Site and infrastructures (23%) were the three Dimensions that mostly needed to be improved.

Other Hospital Units

This Dimension consisted almost exclusively of deltas (92.8%). Teams pointed to a lack of coordination between the ED and other units and a lack of available consultation in the decision-making process. The latter had a negative impact on the ED. Laboratory, medical imaging and pre-anesthetic consultations were cited as the most problematic.

Institution

The Dimension Institution was only defined by deltas (100.0%). Teams mainly emphasized the lack of institutional policy (72.2%) related to important issues (e.g., visits, staff screening, patient flowchart) and communication strategy.

Discussion

As clinical debriefing programs gained in popularity, recent research has been conducted to propose guidelines for debriefing implementation (14) and to engage leaders in the process (9). Recent works also helped to set boundaries in terms of debriefing objectives to ensure efficacy (8). Such results promoted debriefing as a learning tool (reflection on the work done and processes) instead of debriefing as a treating tool (mental health exploration). Using a qualitative approach, we aimed to develop a classification framework and to categorize results from regularly implemented post-shift team debriefings. The DOLL framework is composed of seven dimensions and 13 subdimensions. To the best of our knowledge, this is the first study developing a classification framework for debriefings in a clinical environment. During the first Belgian wave of the COVID-19 crisis, the application of this framework revealed interesting perspectives, allowing for a more structured and standardized analysis of debriefings. Such analysis identified actionable areas and insights, perfectly in line with the experience of the teams, and offered a useful management tool for the leadership.

Considering the debriefing categorization using the DOLL, we found that some Dimensions of the framework were less used than others. The “Patient” Dimension in this study only included two items, probably because very few items mentioned during the debriefings were exclusively patient-related. We also found that the “External unit organization” dimension could be merged into the “Institution” Dimension. This raises questions about the relevance of keeping these Dimensions in the framework. However, the DOLL framework was tested only once and during a short period of the COVID-19 crisis. Rather than eliminating these Dimensions at the time being, it would be worthwhile to further test the current DOLL framework in a non-Covid context to assess their relevance before any modifications.

The analysis of the debriefing content using the DOLL framework reveals differences in the number of plus and delta comments, with more deltas compared to pluses and, interestingly, several inequalities in the distribution of pluses and deltas among Dimensions.

Regarding plus and delta classification, our results highlight the predominantly negative impact of the work environment on the clinicians and their capacity to provide effective and efficient care. A shortage of personal protective equipment (PPE) and daily materials or drug supply challenged the clinicians. (27). According to our results, the fear of infecting family and friends was described as a major stress factor for many participants (28). These findings highlight the need to implement strategies to manage emergency reserves, and to determine situations in which PPE is required and when PPE use may interfere with performance. Along with material availability, the results emphasize the need for efficient supporting services such as IT support, mechanical and electric maintenance, accessibility, etc. Indeed, few studies have identified the latter themes in their findings or chosen these issues as a research topic but those who did have pointed out the importance of a proactive IT Department with a clear strategy (29) and the urgent need to improve interoperability and intuitive user interfaces (30). As part of the healthcare process, support services such as maintenance have the potential to markedly improve patient care when they are immediately available, sensitive to ongoing care priorities and complement an interdisciplinary environment.

Clinician teamwork behaviors were a frequent topic. Debriefings revealed that those behaviors were positively valued. In the face of objectively difficult circumstances, clinicians commented on having greater motivation, engagement and solidarity than before the crisis. Past research has also highlighted that despite the psychological burden, personal risk and work overload, nurses tend to have high work engagement (31) and uphold work values (32). Our data illustrate how the COVID-19 crisis, complemented with post shift debriefings, offered meaning, support and a common cause to clinicians. This raises a fundamental question about how, in the context healthcare's ever-increasing stressful, high-risk environment, we can find ways to support clinicians to avoid burnout and maintain commitment to a high level of patient care. The DOLL framework and its guidelines have been developed and successfully implemented in this experiment. Further testing and refinement of the DOLL is encouraged as it appears to warrant further work (33, 34).

Based on participant remarks, unit management and coordination appeared to improve throughout the two-month data collection. Participants noticed that the debriefing system and the feedback communication loop created with the ED strategic committee were highly effective. The committee was comprised of influential and leadership-designated nurses and physicians. Issues reported during debriefings were systematically brought to the attention of the committee, which set up and communicated action plans in coordination with ED clinicians. This closed loop communication coupled with positive actions was key to success. It provided a sense of empowerment; participants felt listened to and that their point of view was taken seriously. As also stated in previous studies, participants frequently verbalized their approval and willingness to continue participation of the debriefings because they could clearly notice changes that started in the debriefings (9). Similarly, previous work has reported participant satisfaction about debriefings regarding emotional support and organizational learning (7, 15, 35). As debriefing programs gained in popularity, research has been conducted to propose guidelines for implementation (14) and guidelines to engage leaders in the process (9). Recent guidelines also helped to put boundaries in terms of debriefing objectives to ensure efficacy (8). These results promoted debriefing as a learning tool (reflection on the work done and processes) instead of debriefing as a treating tool (mental health exploration). Study participants have also reported that they were more likely to attend debriefings when organization and work were discussed rather than too much emphasis on feelings and mental health. Besides participant satisfaction, what is relatively new is that department leadership was increasingly enthusiastic about the debriefing project as they came to realize how the debriefings and the DOLL were becoming a powerful and effective management tool. Such result points out that debriefings themselves are valuable insofar as a comprehensive structure is implemented to provide effective follow-ups. Perhaps as best practices are developed for clinical debriefing, guidelines for leadership need to be included, e.g., closing the loop on good things that happen as well as the things leaders are working to improve, having some resources to address problems, devoting resources to structuring results of clinical debriefings, etc. Furthermore, there are some of the lessons-learned here that may hold promise for processes such as incident reporting or patients' satisfaction.

In terms of procedures and associated processes, the COVID-19 pandemic required committed effort to adapt procedures and practices under the paradox of quick and safe (36). We found that participants from the present study often reported a lack of organizational procedures usually because the situation was so novel and dynamic in its nature. This issue has already been raised by previous research such as screening at the ED entrances (37), rapid PCR testing for COVID-19 patients (38, 39) or patient handovers (40). Regarding routine, well-established clinical and technical procedures, which were mostly available, teams reported a lack of dissemination and training notwithstanding, especially with regard to the unfolding COVID-19 crisis. Indeed, the availability of established procedures or protocols does not guarantee proper use (41). Poor mastery of procedures logically leads to greater clinician and patient risk such as increased exposure and contamination (42) and reduced effectiveness. Different strategies to enhance effective implementation of procedures were also raised as pluses by the participants. For example, they became interested in having briefings at the beginning of the shift to go through new procedures. Pre-shift briefings seem to have helped master clinical and teamwork skills (such as anticipating and planning, task responsibility, role allocation, etc.), likely allowing for enhanced team and individual performance while maintaining a good margin for patient safety as previously reported (43). Simulation-based training was also pointed out as a major asset by the participants. Participants described simulation as one of the best methods to master clinical and teamwork skills in a safe environment (44). Indeed, the use of simulation has been highly promoted during the COVID-19 crisis to practice, implement, improve procedures (45), to validate procedures (46) and to improve crisis resource management (47, 48).

Regarding the framework itself, the lack of a common taxonomy and the inconsistencies in the results reported in past studies on debriefings impairs a reliable focus on comparisons and trends in the scientific literature (49). The adoption of a common reporting framework for clinical debriefings has the potential to provide a potentially powerful management tool and allow meaningful comparisons even if departments use different debriefing tools or programs. In our experience, regarding the classification of the debriefings, the framework developed allowed for comparisons between Satellite and Main, or Dimensions and the Subdimensions, which helped to refine and focus the larger concepts presented by the Dimensions. Moreover, unlike most classifications used for safety and quality improvement, the framework was developed with the desire of reporting not only issues but also successes. For years, work to improve safety was mainly based on the analysis of undesirable or unsafe written incidents and issues (Safety I) (50). However, that strategy sometimes led to poor outcomes (16). Safer care needs a better and deeper understanding of daily processes, including building on successes (Safety II) and in such context, debriefings can help to identify, analyze, and reinforce the positive practice (9).

Besides the framework itself, its practical use has outlined major benefits and some negative points. In our opinion, the usefulness of the framework further lays in its capacity to clearly illustrate the essence and value of debriefings. This has been a great support for leaders as it helped them to allocate resources and priorities more efficiently. In addition, analyzing the debriefings through main categories helped the leaders to think in terms of processes and workflow rather than problem-solution. The ED found itself developing an ongoing quality improvement programs vis-à-vis the clinical debriefings coupled with favorable action. Before the framework, each plus or delta was reported and addressed individually. When using the framework, more links were made between debriefings and their evolving content over time. Using the framework has also made reporting and traceability more affordable. For future use, resource people could be identified for each Dimension to address the issues raised and to work further on improvement, just as the practice for incident reports or risk assessment. However, to fully benefit from the framework, a specific and visible reporting and management platform would be needed.

Limitations

The debriefing program was part of the COVID-19 quality improvement strategy in our ED. Two different hospitals comprised the experimental sites, but they are located in one larger geographical locale. As the DOLL classification framework and its implementation was based on these data, further research is needed to test the model in different localities and contexts. Interpretation bias is also common in qualitative studies during data collection and analysis. Since most debriefings were conducted by the same trained investigator, content may have been unintentionally affected to some extent. To limit interpretation bias, classification was independently achieved by three different investigators from three different backgrounds and then reconciled to 100% agreement. If such programs are to be implemented, more trained debriefers should be involved. To date, scientific evidence is scarce as to how a debriefer's personality, background or debriefing technique may influence the outcomes of clinical debriefings. Lastly, because leadership responses to debriefings and the DOLL were not systematically quantitatively or qualitatively captured, this is another area that could be further investigated.

Conclusions

We developed a classification framework to help analyze and utilize the content of post-shift clinical debriefings. The practical aim of the project was to maintain a commitment to quality improvement, ensure effective patient care and provide clinicians with ongoing support from colleagues and leadership. Testing the DOLL framework in two different EDs revealed how identifying positive and negative aspects of performance within specific dimensional frames and combining them with guidelines for using the categorized results was a useful and well-received technique. Using such framework allows to increase the potential value of debriefings in the clinical environment by integrating the information gathered into a broader strategy. Debriefings coupled to the framework offer a useful management and leadership tool within a specific department. Links with other existing processes, such as risk management, incident management or patients' complaints, could be investigated to fully take advantage of the potential of debriefings coupled with the DOLL. Our experience convinced us that routinely implementing the technique as part of our department's standard operating procedures has notably enhanced our ongoing practices and commitment to our patients and clinicians.

Data Availability Statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Author Contributions

MP conceived and designed the study, realized data analysis, and wrote the first draft. ND facilitated data transcription and realized critical revision of article for important intellectual content. GG helped with study design, facilitated data transcription, and performed data analysis and interpretation. AD performed data analysis and made critical revision. JP and J-CS realized data collection and editing. AG conceived and designed the study, critical editing, and performed supervision. All authors contributed to the manuscript revision, read, and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

The authors gratefully acknowledge Robert Simon, MAT, MEd, EdD. Center for Medical Simulation & Harvard Medical School (retired), Independent Consultant for scientific guidance, for writing assistance and language editing.

References

1. Peiffer-Smadja N, Lucet JC, Deconinck L, Gérard S, Giordano L, Bendjelloul G, et al. Quelles sont les conséquences de l'épidémie COVID-19 sur l'organisation des soins? Med Mal Infect. (2020) 50:S101. doi: 10.1016/j.medmal.2020.06.206

CrossRef Full Text | Google Scholar

2. Kaye AD, Okeagu CN, Pham AD, Silva RA, Hurley JJ, Arron BL, et al. Economic impact of COVID-19 pandemic on healthcare facilities and systems: International perspectives. Best Pract Res Clin Anaesthesiol. (2021) 35:293–306. doi: 10.1016/j.bpa.2020.11.009

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Phua J, Weng L, Ling L, Egi M, Lim CM, Divatia JV, et al. Intensive care management of coronavirus disease 2019. (COVID-19): challenges and recommendations. Lancet Respir Med. (2020) 8:506–17. doi: 10.1016/S2213-2600(20)30161-2

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Tannenbaum SI, Traylor AM, Thomas EJ, Salas E. Managing teamwork in the face of pandemic: Evidence-based tips. BMJ Quality and Safety. (2021). 30:59–63. doi: 10.1136/bmjqs-2020-011447

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Denning M, Goh ET, Scott A, Martin G, Markar S, Flott K, et al. What has been the impact of COVID-19 on safety culture? A case study from a large metropolitan healthcare trust. Int J Environ Res Public Health. (2020) 17:1–12. doi: 10.3390/ijerph17197034

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Vanhaecht K, Seys D, Bruyneel L, Cox B, Kaesemans G, Cloet M, et al. COVID-19 is having a destructive impact on health-care workers' mental well-being. Int J Qual Heal care J Int Soc Qual Heal Care. (2021) 33:1–6. doi: 10.1093/intqhc/mzaa158

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Rock LK, Rudolph JW, Fey MK, Szyld D, Gardner R, Minehart RD, et al. “Circle Up”: Workflow adaptation and psychological support via briefing, debriefing, and peer support. Nejm Catal Innov Care Deliv. doi: 10.1056/CAT.20.0240

CrossRef Full Text | Google Scholar

8. Kolbe M, Schmutz S, Seelandt JC, Eppich WJ, Schmutz JB. Team debriefings in healthcare: Aligning intention and impact. BMJ. (2021) 374. doi: 10.1136/bmj.n2042

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Seelandt JC, Walker K, Kolbe M. A debriefer must be neutral” and other debriefing myths: a systemic inquiry-based qualitative study of taken-for-granted beliefs about clinical post-event debriefing. Adv Simul. (2021) 6:1–15. doi: 10.1186/s41077-021-00161-5

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Hicks CW, Rosen M, Hobson DB, Ko C, Wick EC. Improving safety and quality of care with enhanced teamwork through operating room briefings. JAMA Surg. (2014) 149:863–8. doi: 10.1001/jamasurg.2014.172

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Tavares W, Eppich W, Cheng A, Miller S, Teunissen PW, Watling CJ, et al. Learning conversations: an analysis of the theoretical roots and their manifestations of feedback and debriefing in medical education. Academic Med. (2020). 95:1020–5. doi: 10.1097/ACM.0000000000002932

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Schmutz JB, Eppich WJ. Promoting learning and patient care through shared reflection: A conceptual framework for team reflexivity in health care. Academic Med. (2017) 1555–63. doi: 10.1097/ACM.0000000000001688

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Mullan PC, Wuestner E, Kerr TD, Christopher DP, Patel B. Implementation of an in situ qualitative debriefing tool for resuscitations. Resuscitation. (2013) 84:946–51. doi: 10.1016/j.resuscitation.2012.12.005

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Szyld D, Arriaga AF. Implementing clinical debriefing programmes. Emerg Med J. (2021). doi: 10.1136/emermed-2021-211133

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Servotte JC, Welch-Horan TB, Mullan P, Piazza J, Ghuysen A, Szyld D. Development and implementation of an end-of-shift clinical debriefing method for emergency departments during COVID-19. Adv Simul. (2020) 5:32. doi: 10.1186/s41077-020-00150-0

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Bentley SK, McNamara S, Meguerdichian M, Walker K, Patterson M, Bajaj K. Debrief it all: a tool for inclusion of Safety-II. Adv Simul. (2021) 6:9. doi: 10.1186/s41077-021-00163-3

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Azizoddin DR, Vella Gray K, Dundin A, Szyld D. Bolstering clinician resilience through an interprofessional, web-based nightly debriefing program for emergency departments during the COVID-19 pandemic. J Interprof Care. (2020) 34:711–5. doi: 10.1080/13561820.2020.1813697

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Diaz-Navarro C, Leon-Castelao E, Hadfield A, Pierce S, Szyld D. Clinical debriefing: TALK© to learn and improve together in healthcare environments. Trends Anaesth Crit Care. (2021). doi: 10.1016/j.tacc.2021.07.004

CrossRef Full Text | Google Scholar

19. Brazil V, Symon B, Twigg S. Clinical debriefing in the emergency department. Emerg Med Australas. (2021) 33:778–9. doi: 10.1111/1742-6723.13834

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Denzin NK, Lincoln YS. Handbook of qualitative research. Denzin NK, Lincoln YS, editors. Handbook of Qualitative Research. Thousand Oaks, CA, US: Sage Publications, Inc. (1994). p. xii, 643–xii, 643.

Google Scholar

21. O'Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research. Acad Med. (2014) 89:1245–51. doi: 10.1097/ACM.0000000000000388

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Rudolph JW, Simon R, Rivard P, Dufresne RL, Raemer DB. Debriefing with good judgment: combining rigorous feedback with genuine inquiry. Anesthesiol Clin. (2007) 25:361–76. doi: 10.1016/j.anclin.2007.03.007

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Vincent C, Taylor-Adams S, Stanhope N. Framework for analysing risk and safety in clinical medicine. BMJ. (1998) 316:1154–7. doi: 10.1136/bmj.316.7138.1154

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Ishikawa K. Guide to quality control. Asian Productivity Organization. Tokyo. (1982) p. 226.

Google Scholar

25. Leka S, Jain A, Cox T, Kortum E. The development of the European framework for psychosocial risk management: PRIMA-EF. J Occup Health. (2011) 53:137–43. doi: 10.1539/joh.O10010

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Lane AS, Roberts C. The learning pathways grid. Int J Qual Methods. (2018) 17:160940691879160. doi: 10.1177/1609406918791605

CrossRef Full Text | Google Scholar

27. Wang X, Zhang X, He J. Challenges to the system of reserve medical supplies for public health emergencies: reflections on the outbreak of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) epidemic in China. Biosci Trends. (2020) 14. doi: 10.5582/bst.2020.01043

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Rose C. Am I part of the cure or am I part of the disease? Keeping coronavirus out when a doctor comes home. N Engl J Med. (2020) 382:1684–5. doi: 10.1056/NEJMp2004768

PubMed Abstract | CrossRef Full Text | Google Scholar

29. He Q, Xiao H, Ming LH, Bei ZB, Wei LC, Fjian Y, et al. Practice in information technology support for fangcang shelter hospital during COVID-19 epidemic in Wuhan, China. J Med Syst. (2021) 45. doi: 10.1007/s10916-021-01721-y

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Topaz M, Peltonen LM, Mitchell J, Alhuwail D, Barakati SS, Lewis A, et al. How to improve information technology to support healthcare to address the COVID-19 pandemic: an international survey with health informatics experts. Yearb Med Inform. (2021). doi: 10.1055/s-0041-1726491

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Giménez-Espert M. del C, Prado-Gascó V, Soto-Rubio A. Psychosocial risks, work engagement, and job satisfaction of nurses during COVID-19 pandemic. Front Public Heal. (2020) 8:566896. doi: 10.3389/fpubh.2020.566896

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Savitsky B, Radomislensky I, Hendel T. Nurses' occupational satisfaction during Covid-19 pandemic. Appl Nurs Res. (2021) 59:151416. doi: 10.1016/j.apnr.2021.151416

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Restauri N, Nyberg E, Clark T. Cultivating meaningful work in healthcare: a paradigm and practice. Curr Probl Diagn Radiol. (2019) 48:193–5. doi: 10.1067/j.cpradiol.2018.12.002

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Both-Nwabuwe JMC, Lips-Wiersma M, Dijkstra MTM, Beersma B. Understanding the autonomy–meaningful work relationship in nursing: a theoretical framework. Nurs Outlook. (2020) 68:104–13. doi: 10.1016/j.outlook.2019.05.008

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Monette DL, Macias-Konstantopoulos WL, Brown DFM, Raja AS, Takayesu JK. A video-based debriefing program to support emergency medicine clinician well-being during the covid-19 pandemic. West J Emerg Med. (2020) 21. doi: 10.5811/westjem.2020.8.48579

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Arabi YM, Azoulay E, Al-Dorzi HM, Phua J, Salluh J, Binnie A, et al. How the COVID-19 pandemic will change the future of critical care. Intensive Care Med. (2021) 47:282–91. doi: 10.1007/s00134-021-06352-y

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Qureshi MN, AlRajhi A. Challenge of COVID-19 crisis managed by emergency department of a big tertiary centre in Saudi Arabia. Int J Pediatr Adolesc Med. (2020) 7:147–52. doi: 10.1016/j.ijpam.2020.08.001

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Rubin R. The challenges of expanding rapid tests to Curb COVID-19. JAMA. (2020) 324:1813–5. doi: 10.1001/jama.2020.21106

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Soltan AAS, Kouchaki S, Zhu T, Kiyasseh D, Taylor T, Hussain ZB, et al. Rapid triage for COVID-19 using routine clinical data for patients attending hospital: development and prospective validation of an artificial intelligence screening test. Lancet Digit Heal. (2021) 3:e78–87. doi: 10.1016/S2589-7500(20)30274-0

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Wasserteil N, Nun A Bin, Mimouni FB, Kasirer Y. Handover of patients: the challenges of COVID-19. J Perinatol. (2020) 40:1453–4. doi: 10.1038/s41372-020-00792-y

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Pokrajac N, Schertzer K, Poffenberger CM, Alvarez A, Marin-Nevarez P, Winstead-Derlega C, et al. Mastery learning ensures correct personal protective equipment use in simulated clinical encounters of COVID-19. West J Emerg Med. (2020) 21:1089–94. doi: 10.5811/westjem.2020.6.48132

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Kwon JH, Burnham CAD, Reske KA, Liang SY, Hink T, Wallace MA, et al. Assessment of healthcare worker protocol deviations and self-contamination during personal protective equipment donning and doffing. Infect Control Hosp Epidemiol. (2017) 38:1077–83. doi: 10.1017/ice.2017.121

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Ryan S, Ward M, Vaughan D, Murray B, Zena M, O'Connor T, et al. Do safety briefings improve patient safety in the acute hospital setting? A systematic review. J Adv Nurs. (2019) 75:2085–98. doi: 10.1111/jan.13984

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Barsuk JH, Cohen ER, Williams M V, Scher J, Jones SF, Feinglass J, et al. Simulation-based mastery learning for thoracentesis skills improves patient outcomes: a randomized trial. Acad Med. (2018) 93:729–35. doi: 10.1097/ACM.0000000000001965

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Li Sharpe E. Innovative approach to procedure skills: A nurse practitioner educator's response to covid-19. J Nurs Educ. (2020) 59:692–6. doi: 10.3928/01484834-20201118-06

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Shelton C, Huda T, Lee A. The role of clinical simulation in preparing for a pandemic. BJA Edu. (2021) 21:172–9. doi: 10.1016/j.bjae.2020.12.006

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Cheung VKL, So EHK, Ng GWY, So SS, Hung JLK, Chia NH. Investigating effects of healthcare simulation on personal strengths and organizational impacts for healthcare workers during COVID-19 pandemic: a cross-sectional study. Integr Med Res. (2020) 9:100476. doi: 10.1016/j.imr.2020.100476

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Wendelboe AM, Miller A, Drevets D, Salinas L, Miller EJ, Jackson D, et al. Tabletop exercise to prepare institutions of higher education for an outbreak of COVID-19. J Emer Manag. (2020) 18:183–4. doi: 10.5055/jem.2020.0463

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Franklin BJ, Gandhi TK, Bates DW, Huancahuari N, Morris CA, Pearson M, et al. Impact of multidisciplinaryteam huddles on patient safety: a systematic review and proposed taxonomy. BMJ Qual Safety. (2020). 29:844–53. doi: 10.1136/bmjqs-2019-009911

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Woodward S. Moving towards a safety II approach. J Patient Saf Risk Manag. (2019) 24:96–9. doi: 10.1177/2516043519855264

CrossRef Full Text | Google Scholar