Effectiveness and feasibility of an interprofessional training program to improve patient safety—A cluster-randomized controlled pilot study

Introduction: Interprofessional healthcare teams are important actors in improving patient safety. To train these teams, an interprofessional training program (IPTP) with two interventions (eLearning and blended learning) was developed to cover key areas of patient safety using innovative adult learning methods. The aims of this study were to pilot test IPTP regarding its effectiveness and feasibility. The trial was registered with DRKS-ID: DRKS00012818.

Methods: The design of our study included both a pilot investigation of the effectiveness of the two interventions (eLearning and blended learning) and testing their feasibility (effectiveness-implementation hybrid design). For testing the effectiveness, a multi-center cluster-randomized controlled study with a three-arm design [intervention group 1 (IG1): eLearning vs. intervention group 2 (IG2)]: blended learning (eLearning plus interprofessional in-person training) vs. waiting control group (WCG) and three data collection periods (pre-intervention, 12 weeks post-intervention, and 24 weeks follow-up) was conducted in 39 hospital wards. Linear mixed models were used for the data analysis. The feasibility of IPTP was examined in 10 hospital wards (IG1) and in nine hospital wards (IG2) using questionnaires (formative evaluation) and problem-focused interviews with 10% of the participants in the two intervention groups. The collected data were analyzed in a descriptive exploratory manner.

Results: Pilot testing of the effectiveness of the two interventions (eLearning and blended learning) showed no consistent differences between groups or a clear pattern in the different outcomes (safety-related behaviors in the fields of teamwork, error management, patient involvement, and subjectively perceived patient safety). Feasibility checks of the interventions showed that participants used eLearning for knowledge activation and self-reflection. However, there were many barriers to participating in eLearning, for example, lack of time or access to computers at the ward. With regard to in-person training, participants stated that the training content sensitized them to patient-safety-related issues in their everyday work, and that awareness of patient safety increased.

Discussion: Although the interventions were judged to be feasible, no consistent effects were observed. A possible explanation is that the duration of training and the recurrence rate may have been insufficient. Another conceivable explanation would be that participants became more sensitive to patient safety-critical situations due to their knowledge acquired through the IPTP; therefore, their assessment post-intervention was more critical than before. In addition, the participants reported high pre-measurement outcomes. Future studies should examine the evidence of the intervention within a confirmatory study after adapting it based on the results obtained.

1. Introduction

Patient safety is a cornerstone of healthcare delivery. It aims to ensure that patients receive the best possible care free from preventable harm. It refers to the prevention of errors and adverse events in healthcare, and encompasses a wide range of issues and concerns, including preventing errors in the administration of medications and in medical procedures, and reducing the risk of infection (Mitchell, 2008). Patient safety is an interprofessional effort that requires the active engagement of different healthcare providers (interprofessional teams), patients, and their families; effective communication and collaboration among them is essential for patient safety (Berry et al., 2020; Dinius et al., 2020). Additionally, patient safety also involves educating patients and their families about their own health and healthcare and encouraging them to actively participate in their caregiving (Wright et al., 2016; Wu and Busch, 2019). Proactive measures for identifying and addressing any potential hazards are also important to reduce the risk of errors and adverse events (Spurgeon et al., 2019).

Improving patient safety is a political goal in healthcare globally (World Health Organization [WHO], 2021). Interprofessional healthcare teams are the relevant actors in achieving this goal. Internationally, especially in North America, training programs exist, but often focus on one specific patient safety topic, for example, the prevention of falls, and infection control. Furthermore, they are often developed for healthcare professionals but do not cover the interprofessional aspects of working together in a team (Reeves et al., 2017; Dinius et al., 2019; Amaral et al., 2023). An exception is the TeamSTEPPS^® 2.0 training program, which is a comprehensive, evidence-based, and commonly used program in North America that was brought to Germany at the same time, but independently of our study.¹

A meta-analysis by Schmutz et al. (2019) showed that interprofessional teamwork has a medium-sized effect on team performance. Therefore, healthcare organizations should bolster interprofessional teamwork to enhance patient safety (Schmutz et al., 2019). Furthermore, teamwork interventions have a positive and significant medium-sized effect on teamwork and team performance (McEwan et al., 2017). In turn, teamwork is associated with improved patient outcomes and increased patient safety (Dinius et al., 2020).

There are different approaches to improve patient safety. Previous research suggests that interventions such as trainings effectively improve teamwork, patient engagement, support of cultural changes, and information technology to subsequently reduce medical errors (Woodward et al., 2010; Amaral et al., 2023). A culture of safety can be built through open discussions regarding adverse events, errors, and their consequences for quality of care (Hofinger, 2009). Furthermore, patients should play an active role in error prevention (Schwappach, 2010), which can be achieved by intensifying patient participation, such as, by involving patients in patient safety management (Wright et al., 2016), by informing patients and encouraging them to participate, providing necessary information promptly and comprehensibly, and enhancing their ability to identify patient safety incidents. However, patient participation in patient safety is lacking in clinical practice. Training programs are required to create a culture of safety that includes patient participation in healthcare processes (Sahlström et al., 2019). Such a culture, in which all professionals of the interprofessional team and patients are seen as equal partners, is still missing. Since the participation of both patients and healthcare professionals is a major factor in high-quality and safe patient-centered care, addressing these topics in training may be advantageous (Quaschning et al., 2013; Hwang et al., 2019).

To address this gap, the KOMPAS project (KOMPAS = German acronym for “Development and evaluation of a complex training program to improve patient safety”) developed and implemented an interprofessional training program (IPTP) with two interventions. These utilized eLearning as well as blended learning, with both covering three key areas of patient safety (teamwork, patient involvement, and error management). The aims of this study were to pilot-test the IPTP by comparing the two types of interventions (eLearning and blended learning) with a waiting control group. The study also aimed to pilot test the effectiveness and the feasibility of these interventions.

The improvement in safety-related behavior regarding teamwork, patient involvement, and error management, as well as subjectively perceived patient safety, was expected to be significantly higher in intervention group 1 (IG1: eLearning) and intervention group 2 (IG2: blended learning: eLearning and interprofessional in-person training) than in the waiting control group (WCG). The greatest improvement was predicted for IG2.

In the course of piloting the effectiveness of the interventions, the following research question and hypothesis were pursued:

1. Do the interventions improve safety-related behavior?

H1: The improvement of safety-related behavior in IG1 and IG2 is significantly higher than in WCG.

In the eLearning course, the participants worked individually on the theoretical foundations for the three key areas. Due to the additional interprofessional in-person training, the participants tried out and consolidated their knowledge. This led us to the following hypothesis:

H2: The greatest improvement in safety-related behavior is in IG2.

2. Do the interventions improve subjectively perceived patient safety?

H3: The improvement of subjectively perceived patient safety in IG1 and IG2 is significantly higher than in WCG.

H4: The greatest improvement of subjectively assessed patient safety is in IG2.

In course of evaluating the feasibility of the interventions the following research questions were investigated:

3. How do trained professionals assess the feasibility of eLearning and interprofessional in-person training?

4. Which facilitators and barriers to implementing the interventions can be identified?

2. Materials and methods

For the following description of the study design and outcomes, intervention, data collection process, and data analysis, the CONSORT guidelines extended for cluster randomized trials (Campbell et al., 2012) and randomized pilot and feasibility trials (Eldridge et al., 2016) were used as standards.

2.1. Study design and outcomes

The design included both a pilot investigation of the effectiveness of the interventions and testing their feasibility. Accordingly, our investigation can best be labeled as a pilot study using a hybrid effectiveness-implementation design (Curran et al., 2012).

2.1.1. Pilot test of the effectiveness

To pilot test the effectiveness of the IPTP, a multi-center cluster-randomized controlled study with a three-arm design [intervention group 1 (IG1)]: eLearning vs. intervention group 2 (IG2): blended learning vs. waiting control group (WCG) and three data collection periods (t1: pre-intervention, t2:12 weeks post-intervention, and t3:24 weeks follow-up) was conducted at three study sites (Freiburg, Hamburg, Bonn) between 2017 and 2020. Randomization took place at the ward/team level (clusters). The teams were randomly assigned to the three study arms by an independent statistician, who was not involved in the recruitment or implementation of the intervention based on a computer-generated randomization sequence with a 1:1:1 treatment allocation ratio. For detailed information see study protocol by Dinius et al. (2019).

The outcome subjectively perceived patient safety was assessed with a single item from the German Hospital Survey on Patient Safety (HSPSC-D) (Gambashidze et al., 2017, value range five level: insufficient, poor, acceptable, very good, and excellent). To measure the safety-related behavior regarding teamwork, error management, and patient involvement situational judgment tests (SJT) (McDaniel et al., 2007; Lievens et al., 2009; Christian et al., 2010) consisting of three self-developed vignettes aligned with the three topics were conducted. The situations depicted in the vignettes were exemplary for situations with special relevance for patient safety (e.g., patient mix-up, adverse drug events, team communication about a doubtful diagnosis). In all three vignettes, the description of the situation was followed by the instruction “Put yourself in the situation and imagine how you actually would react.” The answer categories were developed by the research team and reviewed in an interprofessional expert workshop. Participants were instructed to rank the five response alternatives by assessing the letters A-E depending on which behavior would be best and which behavior would be worst in their perspective. The ranking positions ranged from 1 (this action is the most consistent with my reaction) to 5 (this action is the least consistent with my reaction). The ideal sequence, based on results of the expert workshop, was scored with 30 points (= 4 × 4 + 3 × 3 + 2 × 2 + 1 × 1 + 0 × 0). The worst sequence was scored with 10 points (= 4 × 0 + 3 × 1 + 2 × 2 + 1 × 3 + 0 × 4). For better interpretation of the data, scores were transformed from 0 to 100. Figure 1 shows an example of a vignette on teamwork.

FIGURE 1

Figure 1. Vignette on teamwork. Participants had to rank the five response alternatives by assessing the letters A–E to the ranking positions 1–5.

Owing to the longitudinal study design, the above-mentioned outcomes were assessed at all three data collection periods. The questionnaire during the first data collection period also included sociodemographic information (age, gender, profession, leadership position, duration of hospital affiliation and occupational affiliation in years).

2.1.2. Pilot test of the feasibility

To pilot test the feasibility of the interventions short self-developed written surveys (formative evaluation) and problem-focused individual interviews were used. Participants evaluated eLearning in an online written survey using seven items to assess satisfaction, acceptance, and user-friendliness on a scale of 1 (strongly disagree) to 5 (strongly agree) using adapted items from the two standardized questionnaires: System Usability Scale (SUS) (Brook, 1996) and the measure success inventory (MEI) (Kauffeld et al., 2009). The items are listed in Table 4. The evaluation could be completed after finishing the eLearning intervention.

A short self-developed paper-based survey was distributed by the instructors after finishing the interprofessional in-person training. Participants rated in-person training using nine items on a scale of 1 (strongly disagree) to 5 (strongly agree). The interventions were evaluated with respect to satisfaction and acceptance. The items are listed in Table 5.

Additionally, each intervention also received an overall grade from the participants using the German school grading system from 1 (very good) to 6 (deficient).

Problem-focused individual interviews with 10% of the participants in the intervention groups (IG1 and IG2) were conducted as part of the post-measurement. In these interviews, facilitators, and barriers to implementing training in the participants’ daily work routines were explored. Questions related to context, design, and comprehensibility were also included.

2.2. Intervention

Our intervention consists of two components: eLearning and interprofessional in-person training. In recent years, eLearning has become a standard methodological approach in teaching (George et al., 2014). It offers participants flexibility in terms of place and time of learning. From a learning theory perspective, the same basic concepts are used as they are found in other forms of teaching. Our training based on the following pertinent learning theories:

1.  Adult Learning Theory (Knowles, 1984): We designed for the eLearning interactive, case-based, and experiential learning opportunities that actively engage participants and promote critical thinking and problem-solving skills.

2.  Cognitive Load Theory (Sweller, 1988) means simplifying complex concepts, providing clear and concise instructions, and using multimedia and interactive elements effectively to manage cognitive load and enhance learning outcomes in both components of our intervention.

3.  Social Learning Theory (Bandura, 1977) to conceptualize the interprofessional in-person training. This theory underscores the importance of role modeling and peer learning. In the interprofessional in-person training the participants can learn from experts, engage in discussions and collaborative learning.

For developing the IPTP we considered the key patient safety principles, such as human factors, systems approach, teamwork, communication and error reporting (Lee et al., 2022). Based on the learning objectives of the Patient Safety Curriculum Guide of the World Health Organization [WHO] (2011), the Patient Safety Curriculum Guide of the Patient Safety Action Alliance (Aktionsbündnis Patientensicherheit, 2022), and a focus group study of our research group (Dinius et al., 2017), an IPTP was developed. It consists of two parts: a 3-h eLearning and a three and a half hour interprofessional in-person training. In both the eLearning and the in-person training three learning modules (topics: teamwork, error management, patient involvement), which are essential for patient safety, were conducted. These three topics are interrelated and associated with learning objectives for individual healthcare professionals.

For the development of the KOMPAS eLearning, ELPAS (eLearning Patient Safety), which is used at the Albert-Ludwigs University of Freiburg for training medical students, served as a basis. All modules were programmed with Adobe Captivate and were made available to the participants password-protected via the learning platform Weiterbildungs-Ilias at the Albert-Ludwigs-University of Freiburg. Meanwhile, eLearning is open access.² A video served as an introduction to eLearning, in which participants were introduced to the central learning objective -the improvement of patient safety–and were familiarized with the use of eLearning. This was followed by three learning modules on the key aspects of patient safety. Each module started with an introductory video on the respective topic followed by the related content, which was presented in smaller submodules (completion time 5–10 min) (Figure 2). Participants had the possibility to pause eLearning at any time and continue at a later point in time. The end of each learning module was a short summary of the content (take-home messages).

FIGURE 2

Figure 2. Consort flow diagram.

To obtain the continuing education points (CME points for physicians or continuing education points for voluntarily registered professional nurses) after completing the eLearning modules, all participants completed a final test with 39 questions (13 questions per module). There were five alternative answers to each question, of which one answer was correct. The pass mark was 70%. All participants were given two attempts to pass the final test.

Corresponding to the eLearning design, the interprofessional in-person training also consisted of three modules. Interactive video analysis was conducted in the teamwork module. In this video, the patient is being returned to spontaneous circulation by a team. The participants’ task was to analyze the interprofessional teamwork of the resuscitation team and develop suggestions for improvement based on their analysis. This was followed by group work in which the following questions were to be worked on: “Are there specific situations in which safety concerns are not addressed?” as well as “Are there typical strategies that team members use to avoid having to address safety concerns?” At the end of the teamwork module, the participants were instructed to put themselves in the role of a member of the resuscitation team from the video and to practice the “Speak Up” method. A prepared flipchart of the theory and formulation possibilities of Speak Up was available as an aid.

In the error management module, the participants were given a critical incident reporting system (CIRS) case, which they had to analyze in their respective small groups with regard to cause and effect. The developed worksheet “Identifying Factors Promoting Errors,” which shows examples at different system levels, such as patient factors, team factors, or factors of the working environment, served as an aid. For the cause-effect analysis, participants received a prepared Ishikawa diagram including the “bones”: patient, team, staff/individual, organization/management, and environment. The results of the group work were presented by the participants in a plenary.

In the patient involvement module, a 10-min input was first given on the topic of “communicating an adverse event,” followed by role play on the same topic. The input, based on the brochure “Reden ist Gold” (“Talking is Gold”) by the Patient Safety Action Alliance (Aktionsbündnis Patientensicherheit, 2017), gave an overview of the following points and questions: why communication with patients is important, especially after harm; when, where and with whom communication should happen after an incident; what should and may be said; what is the difference between an apology and an acknowledgment; and what patients want after an incident. The subsequent role play in communication after an adverse event was developed based on a case of CIRS Network Berlin. To facilitate the transfer of the training content into everyday work, the participating teams received two posters that could be hung in the ward room in a clearly visible place: each poster included the essentials of each of the topic areas of teamwork and error management. To remind participants of the contents of the topic area of patient involvement, postcards with key messages were sent to the wards on a monthly basis (e.g., “Your teach-back moment?”).

2.3. Data collection process

Each study site (Freiburg, Hamburg, Bonn) was responsible for recruiting at least 12 wards with 120 participants, so that a total of at least 36 different wards with 360 participants had agreed to participate. The inclusion criteria were: inpatient care teams (1) with at least 10 members, and (2) with an interprofessional composition. We excluded emergency and intensive care due to high regimentations and standardized procedures in teamwork. Furthermore, we excluded pediatrics, because patient involvement would not be comparable to other wards. A local study coordinator per ward (mostly ward manager) supported the research team regarding staff recruitment and data collection at their ward. The following inclusion criteria for study participants were applied: (1) member of an interprofessional inpatient care team (e.g., physician, nurse, therapist), (2) at least 18 years old, and (3) fluent in German.

Data were collected via online or paper-pencil questionnaire according to participants’ preferences. Prior to the start of data collection, the local study coordinators received an Excel list containing so-called one-time passwords (individual access codes) and the URL link to the online survey tool. They transmitted the access data (URL link and passwords) in-house to the participants via email. The stored data set contained a participant number corresponding to the access code (but no contact data) and was thus pseudonymized. This approach allowed data from all three data collection periods to be matched. If participants preferred paper-pencil questionnaires, the local study coordinator received the questionnaires marked with a version number as well as the data protection concept. The version number made it possible to track which wards were assigned to which group. The local study coordinator printed out the questionnaires and the data protection concept and distributed them to the participants. Completed questionnaires were collected in locked boxes. At the end, the questionnaires were entered manually into the online survey platform UniPark by researchers of the corresponding study site, so that all data were available in electronic form.

To increase the response rate in the three data collecting periods, multiple reminders following the Total Design Method of Dillman (2000) were sent to the participants. After sending an initial invitation to the survey, the participants received a first reminder to participate within 2 weeks. Within another 2 weeks, a second reminder containing the link to the questionnaire was sent. For data protection reasons and to prevent participants from recognizing those who had already answered the questionnaire, all thank-you letters and reminders were sent to all potential respondents by e-mail.

For the in-person training, the study coordinators in the hospitals recruited the participants.

The feasibility of eLearning was evaluated using an online questionnaire, whereas a questionnaire for evaluating the interprofessional in-person training was distributed directly after the training. The problem-focused interviews were conducted between June and October 2019. Interview appointments were requested electronically and confirmed via telephone or email. They were organized together with the study coordinators on the wards and held either face-to-face at the wards or over the phone.

2.4. Data analysis

The collected pilot test data was analyzed using descriptive and inferential statistical analyses to describe the sample and test the effectiveness of the interventions according to a modified intention-to-treat approach. These analyses included all participants providing data for at least one measurement and for at least one of the three measurement time points, randomizing all participant data at the ward level. Missing values were considered missing at random, dependent on information in the model, and accounted for using mixed models. The mixed models were calculated by including the intervention group (tree-level factor), time of measurement (three-level factor), interaction between the intervention group and time of measurement, sex (two-level factor), age (continuous variable), occupation (two-level factor), and fixed effects. Ward membership was modeled as having a random effect on the intercept. Statistical significance was set at p < 0.05. As the study was a pilot, no adjustment was made for multiple testing. Intracluster correlation coefficients were calculated to determine the proportion of the total variance that could be explained by ward affiliation. The standardized effect sizes (Cohen’s d) for pairwise group comparisons were calculated by dividing the estimated mean differences by the pooled observed standard deviations. To achieve robustness of the findings, a sensitivity analysis was performed with a subsample of participants from whom pre- and post-measurement data were available. Data were analyzed using IBM SPSS version 26 (IBM Corp., Armonk, NY, USA).

The questionnaires used to measure the feasibility of the interventions were analyzed using descriptive statistics. The interview data were transcribed externally according to the rules of Dresing and Pehl (2015). They were analyzed based on structured content analysis (Mayring, 2022) and used exploratively to cross-check the quantitative data. The categories were deductively derived. For subsequent data analysis, MAXQDA Plus 12 (VERBI Software Company) was used.

2.5. Ethics Statement and registration of the study

The project was approved by the ethics commissions at three study sites (Albert-Ludwigs-University of Freiburg: 4/16_170397, Friedrich-Wilhelm-University of Bonn: 329/17, Medical Association of Hamburg: MC-298/17). Participation was voluntary for the wards and team members. Consent for participation was obtained in written form. The study was registered in the German Register of Clinical Trials (DRKS-ID: DRKS00012818) on August 8, 2017.

3. Results

3.1. Pilot-testing

The study included at the beginning 39 interprofessional teams (mainly nurses and physicians) of different wards (ear, nose and throat wards; surgical wards; internal medicine wards; urology wards; gynecology wards; hematology wards; neurology wards; cardiology wards; orthopedic wards; psychosomatic wards) in 13 German hospitals. Participant characteristics are reported in Table 1. The majority of participants were female and were not older than 40 years of age. Most were nurses with several years of work experience.

TABLE 1

Table 1. Description of the sample (N = 249).

A total of 846 individuals were randomly allocated to one of the three study arms (see Figure 3). The participation rate in interventions was 24.2% the blended learning group (IG 2) and 2.7% in the eLearning group (IG1: Out of 846 persons data from 335 persons were collected at baseline (response rate 39.6%), 191 of 846 persons (22.6%) at 12 weeks post-intervention, and 98 of 846 persons (11.6%) at 24 weeks follow-up. A total of 429 of 846 persons (50.7%) were included in the modified intention-to-treat analysis (see Figure 3).

FIGURE 3

Figure 3. Modules of the eLearning (taken from the study protocol, Dinius et al., 2019).

Table 2 presents the observed means and standard deviations of the outcomes. It is notable that between-group differences at baseline often exceed the average within-group changes across time, suggesting a limited capacity of the cluster-randomization procedure to ensure balanced groups. The SJT for error management had a significant ceiling effect.

TABLE 2

Table 2. Observed values for outcomes.

Table 3 shows the results of effectiveness testing via linear mixed modeling. After 12 weeks, participants with IG2 showed significantly higher teamwork in the SJT than participants in the WCG (p = 0.03, d = 0.42). No other statistically significant between-group differences were observed after 12 weeks. After 24 weeks, the IG2 group was significantly superior to IG1 (p = 0.01, d = 0.90), but not the WCG, which was also statistically significantly superior to IG1 (p = 0.01, d = −0.72). No other statistically significant between-group differences were observed. In the global tests, the mean changes across time were not statistically different between groups.

TABLE 3

Table 3. Results of the pilot test of effectiveness.

Sensitivity analysis did not show statistically significant differences between the groups at the two measurement time points for any of the primary outcomes.

Intracluster correlation coefficients were calculated to determine the variance between wards. The proportion of variance explained by ward affiliation for the primary outcomes ranged from 0.0 (SJT teamwork) to 0.14 (subjectively perceived patient safety) at pre-assessment; from 0.05 (SJT teamwork) to 0.23 (subjectively perceived patient safety) at post- assessment; and from 0.05 (SJT teamwork) to 0.43 (SJT error management) at follow-up-assessment. The substantial contribution of ward affiliation to the overall variance sometimes supported the cluster-randomized approach and the approach of examining patient safety at the ward level.

3.2. Feasibility

3.2.1. Feasibility of eLearning

Overall, adherence to eLearning participation was low despite the possibility of completing it independently of time and location. In both intervention groups, a total of 491 people were invited for eLearning and were entered into the system (IG1: N = 291, IG2: N = 200), of which N = 103 (IG1: N = 43, IG2: N = 60) opted in. This corresponds to a response rate of 20.98%. Due to stipulations in the ethics application, no information can be provided on the number of participants who completed eLearning in full.

The online questionnaire on eLearning was answered by 16 participants after completing the program (response rate 15.53%), and data was analyzed descriptively and exploratively. Participants rated eLearning using seven items on a scale of 1 (strongly disagree) to 5 (strongly agree). Table 4 presents the descriptive statistics for the individual items.

TABLE 4

Table 4. Descriptive evaluation of the eLearning.

Participants rated the comprehensibility of the content and the fact that the different forms of presentation contributed best to the understanding of the content. The design of eLearning was evaluated as the worst (see Table 4).

Additionally, using the German school grading system (1 = very good, 2 = good, 3 = satisfactory, 4 = sufficient, 5 = poor, 6 = deficient), participants awarded the program with an overall average grade of 2.13 (good).

3.2.2. Feasibility of the interprofessional in-person training

For in-person training, 232 invitations were sent to 10 wards of IG2, of which 59 participants took part in the in-person training. The response rate was 25.43%.

A paper-based questionnaire on in-person training was completed by 57 participants (response rate: 96.61%). The item “The presenters were well prepared” received the highest ratings, while the item “The eLearning prepared me well for the in-person training” received the lowest ratings. Overall, the interprofessional in-person training was evaluated as good (Table 5).

TABLE 5

Table 5. Descriptive evaluation of the in-person team training.

Additionally, using the German school grading system (1 = very good, 2 = good, 3 = satisfactory, 4 = sufficient, 5 = poor, 6 = deficient) to evaluate in-person training in total, it was rated by the participants with an average grade of 1.7 (good).

3.3. Qualitative results regarding feasibility

In addition, from the results of 30 problem-focused individual interviews, further indicators regarding the feasibility of the interventions were identified. Facilitating factors and barriers to the successful implementation of the intervention in the daily work routine were explored. Interviews were conducted between June and October 2019, and lasted 20 min on average. The sample consisted mainly of physicians and nurses, with most of them having less than 10 years of professional experience in their job and no leading position. The following categories were applied: user-friendliness, barriers, and facilitators.

The eLearning was described as well structured (“It was well structured, articulated.”, “Structurally it was well done.”), clear, realistic, and understandable, (“So that it was realistic and easy to understand.”, “Basically you couldn’t go wrong.”). The visual design was assessed as appealing (“The design was good and clear.”). Participants particularly emphasized the variety of task formats, visual presentation of the content, and videos used (“The videos and the pictorial representation–I really liked that.”). The processing time is sometimes too long (“But I found that I was taking too much time. It was just too long for me.”). The difficulty level was described as appropriate, and tasks and answer options as understandable (“In terms of difficulty, I actually found it quite ok.”).

Participants stated that they had difficulties integrating eLearning into their daily work routine. Implementation during working hours on the wards was hardly possible due to lack of time and interruptions (“So at work [it] was just not possible for me,” “A point that is relatively difficult when you try to do it on the ward, you don’t have any peace and quiet.”, “Because when the patients see me, yes, they always talk to me”). The possibility of conducting eLearning in sections was mentioned as being positive for feasibility (“You are flexible, you can schedule it yourself, when, how, where, what. You don’t have to do everything at once.”, “However, I then simply did it in sections, then it was easier to do.”). The reminder to carry out eLearning by study coordinators (“I also had to […] be reminded again and again by Ms X, which simply helped me a lot”), and the offer of some hospital wards to participate in eLearning credited as working time were experienced as beneficial (“And we got time off work for the training, so we were sort of released from work.”). On the other hand, a lack of technical equipment (lack of computer workstations), time pressure, and quiet in the daily work routine during the completion of eLearning, and IT security standards had a hindering effect on participation (“Unfortunately, I couldn’t integrate it at all because I couldn’t open it here in my PC.”).

Participants stated that they used eLearning primarily for knowledge activation and self-reflection (“I found it very exciting because you can question yourself, but you probably wouldn’t go into the situation or question on your own.”, “You reflect in a completely different way.”, “And I think I’ve questioned myself a lot more now in relation to the patient.”) and that the subsequent in-person training served to apply this knowledge.

Overall, participation adherence in eLearning was low. Additionally, participants reported that the learning effect of blended learning was higher than that of eLearning alone because of practical testing and the opportunity for interprofessional exchange during in-person training.

Moreover, it led to greater learning through hands-on testing (“This group work in particular, where you can then exchange ideas with one another, where you can also talk to the lecturers, makes more sense, or I learn or I take more with me.”). The ability to ask questions directly has been highlighted (“I found the in-person training easier to understand because I could ask directly if I didn’t understand something.”). Participants emphasized that the training content made them more aware of their everyday work, that their sense of safety was enhanced by the application of learned communication strategies, and that awareness of patient safety was increased (“[.] I feel safer now that I [.] have a scheme like this that I can use to shimmy along,” “[…] you look at certain situations or certain things from a different perspective and question things differently.”, “Afterward we talked about it in the team and it really became clear to everyone how important communication is and that you talk to each other and that you take the initiative and in such situations ask clear questions, give clear answers and so on.”). In addition, the participants reported that their courage to approach colleagues about mistakes increased. “Since then, I’ve been taking a closer look. That’s because the program has brought it to the foremind again.”, “[…] and we could talk more openly about problems and mistakes now,” “[.] as the head of the ward, I observed that the participants returned to the ward with great commitment […] and have spoken more consciously with colleagues about mistakes or even approached colleagues.”).

Facilitation of training participation for in-person training was done through the exemption of teams from their shifts in the ward. Participants particularly emphasized the comprehensible and motivating didactic delivery of the content and indicated a lot of enjoyment and an open atmosphere when working on the tasks in groups and plenary sessions (“The group work was fun for us.”, “Well, that was really, really good, very pleasant atmosphere, I have to say. The learning material that was presented to us was really conveyed in a way that we understood it straight away, and that we were able to get involved very well.”). However, the participation of all professionals from one ward was compromised because of conflicting schedules (“We couldn’t all attend together because our schedules are different.”).

4. Discussion

The aim of the study was to pilot the effectiveness and feasibility of interventions. To this end, a cluster-randomized study on the effectiveness of the training program and a descriptive-explorative study regarding its feasibility were conducted. While the cluster-randomized study on the effectiveness of the interventions did not show consistent differences between the groups or a clear pattern in the different outcomes, the formative study analysis (feasibility study) resulted in a high level of acceptance and stressed the importance of daily work for participating in the intervention. At present, there is insufficient evidence to support our hypothesis that IPTP improve safety-related behavior with regard to teamwork, error management, and patient involvement as well as subjectively perceived patient safety. We can only assume why no empirical effects were found in this study. The main assumptions about this are as follows. First both interventions are not effective because they are carried out once (there are no booster sessions) and they are too short. Second, if the interventions are not tested as planned because of the low response rate, they will not have an effect. Third, the outcome measures did not capture the effects of the interventions reliably and validly. A possible solution would be to check the methodological quality of the SJT. Regarding the implementation of the in-person intervention, a trainer model should be considered in follow-up studies to test the sustainability of the in-person intervention and organizational learning in clinics. Managerial support is especially important; therefore, a step-by-step approach could be a solution for the better utilization of both interventions.

Overall, the professionals participated in the interventions rated the intervention as positive and described the content as extremely helpful for their everyday work. The insights gained in the formative evaluation include the following. First, participating wards in the eLearning intervention should be provided with technical equipment (tablet computers). Second, both eLearning and interprofessional in-person training should explicitly consider working schedules, particularly when seeking to train the entire team. Third, at least two different dates should be offered for each in-person training for employees who otherwise could not take part due to conflicting schedules. Fourth, the importance of participation should be communicated, particularly to ward managers who have a role model function, so that they can participate in training and encourage their employees to participate.

Amaral et al. (2023) summarized studies on the effectiveness of patient safety training, stating that there are still few studies that test patient safety training programs. Most studies focus on the development of training programs and do not provide evaluation data. Furthermore, training programs are mostly designed for special healthcare professional groups, and the interprofessional aspect is not explicitly considered (Dinius et al., 2019). Amiri et al. (2018) conducted a RCT (randomized controlled trial, RCT) and showed a significant improvement in safety culture. However, the dimensions of non-punitive responses to errors reported as adverse events did not improve, indicating that additional actions are necessary (Amiri et al., 2018). Overall, there is moderate to high-quality evidence that team training has a positive impact on healthcare team processes and patient outcomes (Weaver et al., 2014). Several studies have reported positive outcomes, such as patient-centered communication, improved clinical outcomes, collaborative practice, reduction of clinical error rates, and improved team behavior (Reeves et al., 2013). A current systematic review and meta-analysis (Agbar et al., 2023) including 16 studies and 6,559 participants from healthcare professional staff showed that the interventions have a positive effect on safety culture. However, the interventions varied between the studies and there was a significant heterogeneity among the studies assessing patient safety culture (Agbar et al., 2023, p. 1471). Furthermore, the effects were no longer significant after the exclusion of studies with low quality scores (Agbar et al., 2023). The integrative review of Lee et al. (2022) identified also “non-significant and inconsistent relationships between safety culture and patient safety and quality of care outcomes” (p. 279). Several different factors could contribute to these inconsistent and non-significant results, such as the lack of a theoretical framework, inconsistent outcome criteria, missing validity of the instruments, etc (Lee et al., 2022). Overall, the study situation is inconsistent, so that the present study fits into the previous picture. Interprofessional training has become increasingly common in recent years. The KOMPAS training program was explicitly interprofessional, but in some clinics, only nurses participated. In addition to the lack of doctors, the participation of health professionals in the training sessions and surveys was very low. The dropout rate was 22.5%. The reasons for dropout included refusal to participate in part of the contacts and difficult accessibility. The healthcare professionals struggled with the lack of support, lack of resources, time constraints, or conflicting priorities in their clinics that hindered an adequate training implementation. Furthermore, the evaluation methods used rely on self-report measures that do not capture the true impact.

As most interventions include narrative reviews on the nature of interprofessional interventions to promote patient safety by Reeves et al. (2017), our intervention is an educational intervention to address individuals’ skills and behaviors. The key principles (human factors, systems approach, teamwork, communication, and error reporting) of our patient safety training are comparable to other training programs (Lee et al., 2022).

To measure the feasibility and effects of the intervention, we conducted surveys, as most of the reported studies in the narrative review. However, in contrast to the vast majority (86 out of 89) of the studies included in the review, we applied a mixed-methods approach and conducted qualitative interviews. Most studies have concentrated on nurses and physicians in acute care, as in our study. A dearth of studies has reported changes in safety behaviors, which we aimed to measure using SJTs. The choice of SJTs as our main outcome measures could be a crucial aspect of the inconsistent and non-significant results.

4.1. Limitations

The moderate response rates and exclusion of pediatric, emergency, and intensive care wards limit the generalizability of the study results only to the target population of individuals. Concurrently, the inclusion of hospitals from different regions in Germany may strengthen the generalizability of the results to the populations of targeted organizations. As participation was voluntary, we must assume a selection bias because we cannot exclude that we mainly reached motivated and well-functioning interprofessional teams. A serious limitation was the low proportion of invited participants who received the interventions. In addition, the internal validity of the pilot effectiveness test is likely to be limited by the imperfectly balanced groups at baseline, missing controls of possible co-interventions, and the high attrition rate across measurements. Concerning the instrument, we developed specific SJTs based on current literature and experts’ opinions on the primary outcomes. However, these are not psychometrically validated instruments. Patient safety is measured using only one ordinal-scale item, which possibly restricts the variability of participants’ responses. Future studies should use objectively measured outcomes to assess patient safety.

Moreover, it is worth considering the appropriateness of utilizing RCTs for assessing these interventions. There is a significant challenge evaluating changes and improvement in practice at this level. We encountered several challenges during the study. Given that both the tested interventions and the circumstances can be considered “complex,” it is possible that other (e.g., multi-level designs, sequential designs, non-randomized designs) might have suited better to our research aims than a RCT (Skivington et al., 2021).

The curriculum was feasible and judged as relevant and useful, but the participation rate was very low and we had a high drop-out rate. This can be due to the wrong implementation strategy in a very complex setting with a high workload. This means that an actual study with the developed intervention was not conducted. For further studies, the design and implementation of the intervention in particular must be redesigned with a comprehensive participatory research approach.

4.2. Future development

Future studies should further examine the effectiveness of the intervention within a confirmatory study after the implementation of the intervention has been further developed based on the current results. To anchor the knowledge comprehensively in clinics, a multi-stage procedure should be chosen, starting with clinic and ward managers. This approach is intended to communicate the importance of participating in such an intervention to ward managers, who function as role models, so that they can disseminate on the topic top-down to their staff. As a result, improved adherence to participation in the intervention and evaluation could be encouraged.

To implement the intervention in a sustainable way, the wards should offer booster sessions after the intervention. Furthermore, individual and group coaching sessions are proven methods to increase participants’ self-efficacy and further steps for evaluating the training program include consideration of the patients’ perspectives and the organizational preconditions. To consider these long-term outcomes, longitudinal mixed methods and multilevel studies are required.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by the Albert-Ludwigs-University of Freiburg: 4/16_170397, Friedrich-Wilhelm-University of Bonn: 329/17, Medical Association of Hamburg: MC-298/17. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

LK designed the pilot effectiveness trial, performed data analysis, and interpreted the results. MK developed the initial article, whereas JD, NE, LH, CB, AH, and SP-H read the manuscript critically and provided relevant corrections, additions, and comments. All the authors established the project application, collectively provided the basis for this study, critically reviewed, and approved the final manuscript.

Funding

This project was funded by the German Innovation Fund (Federal Joint Committee, https://innovationsfonds.g-ba.de/). Funding code 01VSF16014 (KOMPAS). The article processing charge was funded by the German Research Foundation (DFG) and the University of Freiburg in the funding program Open Access Publishing.

Acknowledgments

We would like to thank all participants and study coordinators in the hospitals for their cooperation. We would also like to thank Editage (www.editage.com) for English language editing.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

1. ^ https://www.teamstepps.de/
2. ^ https://wb-ilias.uni-freiburg.de/ilias.php?ref_d=291255&cmd=render&cmdclass=ilrepositorygui&cmdnode=111&baseclass=ilrepositorygui

References

Agbar, F., Zhang, S., Wu, Y., and Mustafa, M. (2023). Effect of patient safety education interventions on patient safety culture of health care professionals: Systematic review and meta-analysis. Nurse Educ. Pract. 67:103565. doi: 10.1016/j.nepr.2023.103565

PubMed Abstract | CrossRef Full Text | Google Scholar

Aktionsbündnis Patientensicherheit (2017). Reden ist Gold: Kommunikation nach einem Zwischenfall. Available online at: https://www.aps-ev.de/wp-content/uploads/2016/08/APS_Reden_ist_Gold_2017.pdf (accessed March 08, 2023).

Google Scholar

Aktionsbündnis Patientensicherheit (2022). Wege zur Patientensicherheit: Katalog für Kompetenzen in der Patientensicherheit. Available online at: https://www.aps-ev.de/wp-content/uploads/2022/05/2022_APS_Lernzielkatalog.pdf (accessed March 08, 2023).

Google Scholar

Amaral, C., Sequeira, C., Albacar-Riobóo, N., Coelho, J., Pinho, L. G., and Ferré-Grau, C. (2023). Patient safety training programs for health care professionals: A scoping review. J. Patient Saf. 19, 48–58. doi: 10.1097/PTS.0000000000001067

PubMed Abstract | CrossRef Full Text | Google Scholar

Amiri, M., Khademian, Z., and Nikandish, R. (2018). The effect of nurse empowerment educational program on patient safety culture: A randomized controlled trial. BMC Med. Educ. 18:158. doi: 10.1186/s12909-018-1255-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Bandura, A. (1977). Social learning theory. New York, NY: General Learning Press.

Google Scholar

Berry, J. C., Davis, J. T., Bartman, T., Hafer, C. C., Lieb, L. M., Khan, N., et al. (2020). Improved safety culture and teamwork climate are associated with decreases in patient harm and hospital mortality across a hospital system. J. Patient Saf. 16, 130–136. doi: 10.1097/PTS.0000000000000251

PubMed Abstract | CrossRef Full Text | Google Scholar

Brook, J. (1996). SUS-A quick and dirty ability scale. Usability Eval. Ind. 189, 4–7.

Google Scholar

Campbell, M. K., Piaggio, G., Elbourne, D. R., Altman, D. G., and Consort Group (2012). Consort 2010 statement: Extension to cluster randomised trials. BMJ. 345:e5661. doi: 10.1136/bmj.e5661

PubMed Abstract | CrossRef Full Text | Google Scholar

Christian, M. S., Edwards, B. D., and Bradley, J. C. (2010). Situational judgement tests: Constructs assessed and a meta-analysis of their criterion-related validities. Pers. Psychol. 63, 83–117. doi: 10.1111/j.1744-6570.2009.01163.x

CrossRef Full Text | Google Scholar

Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M., and Stetler, C. (2012). Effectiveness-implementation hybrid designs: Combining elements of clinical effectiveness and implementation research to enhance public health impact. Med. Care 50, 217–226. doi: 10.1097/MLR.0b013e3182408812

PubMed Abstract | CrossRef Full Text | Google Scholar

Dillman, D. A. (2000). Mail and internet surveys: the tailored design method, 2nd Edn. New York, NY: Wiley.

Google Scholar

Dinius, J., Gaupp, R., Becker, S., Göritz, A. S., and Körner, M. (2017). Patient safety in hospitals: What we do and what we need-focus groups with stakeholders of hospitals in Southern Germany. J. Patient Saf. 17:e1062–e1068. doi: 10.1097/PTS.0000000000000452

PubMed Abstract | CrossRef Full Text | Google Scholar

Dinius, J., Hammer, A., Manser, T., Bergelt, C., Kriston, L., and Körner, M. (2019). Piloting and evaluating feasibility of a training program to improve patient safety for inter-professional inpatient care teams – study protocol of a cluster randomized controlled trial. Trials 20:386. doi: 10.1186/s13063-019-3448-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Dinius, J., Philipp, R., Ernstmann, N., Heier, L., Göritz, A. S., Pfisterer-Heise, S., et al. (2020). Interprofessional teamwork and its association with patient safety in German hospitals – A cross sectional study. PLoS One. 15:e0233766. doi: 10.1371/journal.pone.0233766

PubMed Abstract | CrossRef Full Text | Google Scholar

Dresing, T., and Pehl, T. (2015). Praxisbuch interview. Transkription & analyse: anleitungen und regelsysteme für qualitative forschende, 6th Edn. Marburg: Dr. Dresing und Pehl GmbH.

Google Scholar

Eldridge, S. M., Chan, C. L., Campbell, M. J., Bond, C. M., Hopewell, S., Thabane, L., et al. (2016). PAFS consensus group. CONSORT 2010 statement: Extension to randomised pilot and feasibility trials. BMJ 24:i5239. doi: 10.1136/bmj.i5239

PubMed Abstract | CrossRef Full Text | Google Scholar

Gambashidze, N., Hammer, A., Brösterhaus, M., and Manser, T. WorkSafeMed Consortium (2017). Evaluation of psychometric properties of the German Hospital Survey on Patient Safety Culture and its potential for cross-cultural comparisons: A cross-sectional study. BMJ Open. 7:e018366. doi: 10.1136/bmjopen-2017-018366

PubMed Abstract | CrossRef Full Text | Google Scholar

George, P. P., Papachristou, N., Belisario, J. M., Wang, W., Wark, P. A., Cotic, Z., et al. (2014). Online eLearning for undergraduates in health professions: A systematic review of the impact on knowledge, skills, attitudes and satisfaction. J Glob. Health 4:010406. doi: 10.7189/jogh.04.010406

PubMed Abstract | CrossRef Full Text | Google Scholar

Hofinger, G. (2009). Lernen aus Fehlern im Krankenhaus. Systemische Fehlersicht und Zwischenfall-Berichtssysteme [Learning from mistakes in hospitals. A system perspective on errors and incident reporting systems]. Unfall 112, 604–609. doi: 10.1007/s00113-009-1609-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Hwang, J. I., Kim, S. W., and Chin, H. J. (2019). Patient participation in patient safety and its relationships with nurses’ patient-centered care competency, teamwork, and safety climate. Asian Nurs. Res. 13, 130–136. doi: 10.1016/j.anr.2019.03.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Kauffeld, S., Brennecke, J., and Strack, M. (2009). “Erfolge sichtbar machen: das Maßnahmen-Erfolgs- Inventar (MEI) zur Bewertung von Trainings,” in Handbuch kompetenzentwicklung, ed. S. Kauffeld (Stuttgart: Schäffer-Pöschel), 55–78.

Google Scholar

Knowles, M. (1984). The adult learner: A neglected species, 3rd Edn. Houston, TX: Gulf.

Google Scholar

Lee, S. E., Morse, B. L., and Kim, N. W. (2022). Patient safety educational interventions: A systematic review with recommendations for nurse educators. Nurs. Open 9, 1967–1979. doi: 10.1002/nop2.955

PubMed Abstract | CrossRef Full Text | Google Scholar

Lievens, F., Sackett, P. R., and Buyse, T. (2009). The effects of response instructions on situational judgment test performance and validity in a high-stakes context. J. Appl. Psychol. 94, 1095–1101. doi: 10.1037/a0014628

PubMed Abstract | CrossRef Full Text | Google Scholar

Mayring, P. (2022). Qualitative Inhaltsanalyse: Grundlagen und Techniken. Basel: Beltz.

Google Scholar

McDaniel, M. A., Hartman, N. S., Whetzel, D. L., and Grubb, W. L. (2007). Situational judgement tests, response instructions, and validity: A meta-analysis. Pers. Psychol. 60, 63–91. doi: 10.1111/j.1744-6570.2007.00065.x

CrossRef Full Text | Google Scholar

McEwan, D., Ruissen, G. R., Eys, M. A., Zumbo, B. D., and Beauchamp, M. R. (2017). The effectiveness of teamwork training on teamwork behaviors and team performance: A systematic review and meta-analysis of controlled interventions. PLoS One 12:e0169604. doi: 10.1371/journal.pone.0169604

PubMed Abstract | CrossRef Full Text | Google Scholar

Mitchell, P. H. (2008). “). Defining patient safety and quality care,” in Patient safety and quality: an evidence-based handbook for nurses, ed. R. G. Hughes (Rockville, MD: Agency for Healthcare Research and Quality).

Google Scholar

Quaschning, K., Körner, M., and Wirtz, M. (2013). Analyzing the effects of shared decision-making, empathy and team interaction on patient satisfaction and treatment acceptance in medical rehabilitation using a structural equation modeling approach. Patient Educ. Couns. 91, 167–175. doi: 10.1016/j.pec.2012.12.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Reeves, S., Clark, E., Lawton, S., Ream, M., and Ross, F. (2017). Examining the nature of interprofessional interventions designed to promote patient safety: A narrative review. Int. J. Qual. Health Care. 29, 144–150. doi: 10.1093/intqhc/mzx008

PubMed Abstract | CrossRef Full Text | Google Scholar

Reeves, S., Perrier, L., Goldman, J., Freeth, D., and Zwarenstein, M. (2013). Interprofessional education: Effects on professional practice and healthcare outcomes (update). Cochrane Datab. Syst. Rev. 2013:CD002213. doi: 10.1002/14651858.CD002213.pub3

PubMed Abstract | CrossRef Full Text | Google Scholar

Sahlström, M., Partanen, P., Azimirad, M., Selander, T., and Turunen, H. (2019). Patient participation in patient safety-An exploration of promoting factors. J. Nurs. Manag. 27, 84–92. doi: 10.1111/jonm.12651

PubMed Abstract | CrossRef Full Text | Google Scholar

Schmutz, J. B., Meier, L. L., and Manser, T. (2019). How effective is teamwork really? The relationship between teamwork and performance in healthcare teams: A systematic review and meta-analysis. BMJ open 9:e028280. doi: 10.1136/bmjopen-2018-028280

PubMed Abstract | CrossRef Full Text | Google Scholar

Schwappach, D. L. (2010). Review: Engaging patients as vigilant partners in safety: A systematic review. Med. Care Res. Rev. 67, 119–148. doi: 10.1177/1077558709342254

PubMed Abstract | CrossRef Full Text | Google Scholar

Skivington, K., Matthews, L., Simpson, S. A., Craig, P., Baird, J., Blazeby, J. M., et al. (2021). A new framework for developing and evaluating complex interventions: Update of Medical Research Council guidance. BMJ 374:n2061. doi: 10.1136/bmj.n2061

PubMed Abstract | CrossRef Full Text | Google Scholar

Spurgeon, P., Sujan, M. A., Cross, S., and Flanagan, H. (2019). Building safer healthcare systems: a proactive, risk based approach to improving patient safety. Cham: Springer Nature.

Google Scholar

Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cogn. Sci. 12, 257–285. doi: 10.1207/s15516709cog1202_4

CrossRef Full Text | Google Scholar

Weaver, S. J., Dy, S. M., and Rosen, M. A. (2014). Team-training in healthcare: A narrative synthesis of the literature. BMJ Qual. Saf. 23, 359–372. doi: 10.1136/bmjqs-2013-001848

PubMed Abstract | CrossRef Full Text | Google Scholar

Woodward, H. I., Mytton, O. T., Lemer, C., Yardley, I. E., Ellis, B. M., Rutter, P. D., et al. (2010). What have we learned about interventions to reduce medical errors? Annu. Rev. Public Health 31, 479–971 following497. doi: 10.1146/annurev.publhealth.012809.103544

PubMed Abstract | CrossRef Full Text | Google Scholar

World Health Organization [WHO] (2011). WHO patient safety curriculum guide: Multiprofessional edition. Available online at: https://cdn.who.int/media/docs/default-source/patient-safety/9783000606267-ger9b27b60a-ce75-43a0-a53d-c45b52ed9aac.pdf?sfvrsn=35da6b35_1 (accessed January 23, 2023).

Google Scholar

World Health Organization [WHO] (ed.) (2021). Towards eliminating avoidable harm in health care: draft global patient safety action plan. Available online at: https://cdn.who.int/media/docs/default-source/patient-safety/gpsap/final-draft-global-patient-safety-action-plan-2021-2030.pdf?sfvrsn=fc8252c5_5, 2021-2030 (accessed March 03, 2023).

Google Scholar

Wright, J., Lawton, R., O’Hara, J., Armitage, G., Sheard, L., Marsh, C., et al. (2016). Improving patient safety through the involvement of patients: Development and evaluation of novel interventions to engage patients in preventing patient safety incidents and protecting them against unintended harm. Southampton: National Institute on Handicapped Research Journals Library.

Google Scholar

Wu, A. W., and Busch, I. M. (2019). Patient safety: A new basic science for professional education. GMS J. Med. Educ. 36:Doc21. doi: 10.3205/zma001229

PubMed Abstract | CrossRef Full Text | Google Scholar