The Implementation Process of Two Evidence-Based Protocols: A Spatial Neglect Network Initiative

Introduction: Spatial neglect, a neurocognitive disorder of lateralized spatial attention, is prevalent among stroke survivors especially in inpatient rehabilitation facilities (IRFs). The ultimate goal of the project was to improve spatial neglect care in inpatient rehabilitation and trained as many OTs as possible using both tools in their regular practices as the means to achieve our overall objective. Therefore, we conducted a project aimed at implementing two evidence-based protocols, one for assessment (KF-NAP^®) and the other for treatment (KF-PAT^®), and share the implementation process, which included barriers and facilitators identified during and after the process, and implementation outcomes.

Methods: Sixteen IRFs were involved. The Knowledge-To-Action Cycle was used to describe the process of knowledge inquiry (training), translating knowledge (implementation) and evaluating the use of knowledge in clinical practice (outcomes). Barriers and strategies were reported using the Consolidated Framework for Implementation Research and identified through a survey, after the study concluded.

Results: Thirty-two therapists at the participating sites were trained to some level of the KF-NAP and KF-PAT. Throughout the project and also once after it finished, different barriers were identified by researchers and clinicians, who then determined together actions to eliminate or minimize the barriers. For example, multiple sites reported: “not having time to train other staff at their hospital due to high patient volume and other responsibilities.”

Discussion: The project shared our implementation process which demonstrated the importance of using implementation methods and incorporating a researcher-clinician partnership, not only for knowledge generation but also knowledge translation. Frequent communications and exchanging information with stakeholders at different levels, may be determinant to the success of each implementation phase. Further research is needed.

Introduction

Spatial neglect is a neurocognitive disorder that is characterized by the inability to attend to, perceive, and orient to the space that is contralateral to the injured or damaged cerebral hemisphere (1). It affects 20–40% of stroke survivors (2) and individuals with other acquired brain injuries (3). Since the 1980 s, devastating impacts of spatial neglect on rehabilitation progress, functional recovery, community reintegration, and caregiver burden have been demonstrated in various studies conducted by independent research groups around the world (4–10). Furthermore, spatial neglect prolongs inpatient rehabilitation and increases the risks of falls and injuries (5, 8, 10, 11). Treatments and assessments have been developed, examined, and recommended as guidelines, by organizations such as the American Heart/Stroke Association (12), Canadian Stroke Association (13), and the Australian Stroke Foundation (14). Nonetheless, in practice, it has been arbitrary whether individuals with spatial neglect are provided the recommended evidence-based treatment and assessment services. The problem may be related to the hurdles to achieve knowledge translation, dissemination, and implementation (15–18).

The ultimate goal of the wider research project was to improve spatial neglect care. To achieve this wider goal, we sought to implement two evidence-based standardized protocols, one for assessment and the other for treatment. The assessment protocol was the Kessler Foundation Neglect Assessment Process (KF-NAP^®), and the treatment protocol was the Kessler Foundation Prism Adaptation Treatment (KF-PAT^®). In this article, we aim to report the implementation process, which included barriers and facilitators identified during and after the process, and share implementation outcomes using quantitative and qualitative information.

In order to report the implementation process systematically, we incorporated methods from the implementation science literature (19). First, we followed the Knowledge-to-Action (KTA) cycle (20) to describe the progression from knowledge dissemination, protocol implementation, to outcome evaluation. The KTA cycle provides a “map” for how to move knowledge into action and encouraged revisiting phases of the action cycle as many times as necessary (Figure 1). The KTA cycle has been widely used in practice because it captures the complexities of real-world application and encourages transformation of knowledge that has been generated in research settings to promote use of evidence-based practices in the clinic (21).

FIGURE 1

Figure 1. Knowledge-to-action cycle. Adapted from Graham et al. (20) and phases are numbered in this version was our iteration.

Second, we used the Consolidated Framework for Implementation Research (CFIR), a framework of constructs related to implementation (22), to report barriers identified during the implementation process and the subsequent strategies used to address each barrier (23, 24). The CFIR is organized into five domains (intervention characteristics, outer setting, inner setting, characteristics of the individuals involved and the process of implementation) which provides organization and specificity to evaluate the project's impact (25). There are multiple examples of pragmatic research projects that use the KTA and the CFIR in a rehabilitation setting. Studies suggest that using the CFIR may increase the replicability and generalizability of study findings (26–28), and the KTA cycle can contribute to positive changes in stroke rehabilitation practices (21).

This project implemented the KF-NAP and the KF-PAT. The KF-NAP is a standardized method to administer and score the 10-item Catherine Bergego Scale (CBS) during daily activities (29). The items include gaze orientation, limb awareness, auditory attention, personal belongings, dressing, grooming, navigation, collisions, meals, and cleaning after meals. The scoring uses a scale between 0 and 3 for each item, with the total score ranging between 0 and 30 (the higher the number, the more severe the neglect is). The KF-PAT is a standardized protocol to deliver prism adaption treatment (PAT) (30). PAT is one of the treatment approaches recommended for stroke rehabilitation by the latest guidelines of the American Heart/Stroke Association (12). Both the KF-NAP and KF-PAT, and related materials such as clinician-oriented manuals and equipment, were developed through clinical research and trials over the past decade by our research team (31–35) and thus were the preferred choices in the present implementation project.

The discipline that participated in this project was occupational therapy (OT). Conventionally, in the United States, OT is the discipline in neurorehabilitation providing care related to visuospatial deficits, and is the discipline known to document the observable symptoms of spatial neglect, such as head and eye deviation. Thus, the present project was focused on integrating the two evidence-based protocols into inpatient OT (31–35). In addition, both the KF-NAP and KF-PAT were developed in and for the inpatient rehabilitation care through studies and clinical trials with much involvement of occupational therapists (OTs) (31–35). In the present project, we trained as many OTs as possible using both tools in their regular practices as the means to achieve our overall objective (i.e., to implement both tools) and move closer to our ultimate goal (i.e., to improve spatial neglect care).

Materials and Methods

Participants and Setting

Sixteen inpatient rehabilitation facilities (IRFs) across 11 different states in the United States participated in this assessment and treatment implementation project through an agreement with the research center in New Jersey. The agreement included OT training and de-identified clinical information sharing. Twelve sites were on the East Coast (New Jersey, Ohio, Florida, Pennsylvania, Georgia, Maryland, New York), one on the West Coast (California), two in the Southwest (Texas, Arizona) and one in the Midwest (Missouri). The project was approved by the research center's Institutional Review Board (IRB) and the local IRB of each hospital that had a research infrastructure. IRFs without a research infrastructure were attached to the research center's IRB protocol through a federal assurance agreement. Directors of rehabilitation at each site nominated one or two lead OTs (i.e., implementation champions) for project participation. A total of 32 champions were trained to use the KF-NAP and KF-PAT throughout the project. They implemented the protocols in their practice, participated in monthly calls, tracked information regarding implementation, provided feedback to the research center, and were encouraged to train their peers. The implementation information being tracked included de-identified patient clinical records. Patient outcomes were reported separately (36, 37).

The project was initiated in June 2017, and the therapist user feedback completed in March 2021 marking the end of the project. OTs at three sites that had participated in our previous research (31–35) and KF-NAP and KF-PAT development were more familiar with either tools than OTs at other sites. OTs at sites that joined the project earlier might be more experienced with either tool than OTs at sites that joined the project later over the years. While the project was not designed as a research study, lead OTs served as “study participants,” representing their sites, in the user feedback survey at the end of project, and the consent form was waived. Evaluation of the implementation outcomes was based on responses to the survey and information shared about de-identified clinical records.

Procedures

Following the KTA cycle (Figure 1), the research team led and was actively involved with hospital management leaders and clinicians in the knowledge dissemination (center of the KTA cycle), protocol implementation (Phase 1 to Phase 5), and outcome evaluation (Phase 6). Knowledge here refers to the two protocols of the KF-NAP and the KF-PAT. Phase 6 required the research team to step back and conduct outcome evaluations. This project did not move into Phase 7.

Knowledge Dissemination

Knowledge dissemination involved the lead OTs participating in a formal 2-day training. The training took place at either the research center, the IRF where the OTs worked, or a specified IRF that would host a few groups of OTs at the same time. Sixty percentage of the lead OTs received the training at their sites, and the remaining 40% traveled to the location where the training was provided. Completing the training would enable them to reach Level 2 of competency on both protocols (Table 1), and they were instructed and encouraged to fulfill the requirements for the highest level of competency (Level 3) on their own. Level 3 competency would qualify the therapists to be able to teach their colleagues how to administer the KF-NAP or KF-PAT. This entailed using Table 1's criteria to guide the training process. For example, the trainer scheduled time for each therapist to first observe them completing each protocol, and then had the trainees perform the assessment and treatment protocols under supervision.

TABLE 1

Table 1. Competency levels and criteria.

The 2-day training was taught by the same members of the research team, who developed the protocols and have therefore extensive knowledge on the topic of spatial neglect. The curriculum started with didactics about spatial neglect and then the two protocols, hands-on practice using both protocols under the trainer's supervision and ended with discussions on implementation. The lead OTs were provided training materials including the Manuals, lecture handouts, and pre-recorded video tutorials. Fidelity of both the assessment and intervention were discussed when these materials were being distributed. This discussion included that the Manuals were mandatory to use, the training of other staff members must include the lecture handouts, video tutorials should be reviewed prior to the in-person supervision and hands-on-training, and competency forms should be used. The lead OTs were also instructed about how the ways in which to communicate with the research team.

In addition, the research team offered lectures to other disciplines of therapists, medical staff, case managers, and trainees during new employee trainings, regular staff meetings, and schedule meetings outside of working hours. The lectures focused on the mechanisms, clinical presentations, and consequences of spatial neglect. These knowledge dissemination activities were developed to help coworkers of the OTs understand why the KF-NAP and KF-PAT were being implemented in their facilities. Thus, consistent with “culture,” a construct of the inner setting domain within the CFIR framework, an environment friendly for spatial neglect care was cultivated, potentially beneficial for the implementation project.

Implementation Process

Daily Practice and Frequent Communications

After the training, the lead OTs returned to their daily clinical activities and started to gain experience by completing the protocols independently with their own patient case load. They were instructed to assess all neurological patients for spatial neglect using the KF-NAP within 4 days after admission, regardless of whether patients' symptoms were clearly observable or not. In addition, the lead OTs were also instructed to treat patients using the KF-PAT Portable Kit¹ when spatial neglect was confirmed through the KF-NAP, and assess these patients using the KF-NAP again after completing 10 sessions of PAT or before IRF discharge.

In addition to implementing both the KF-NAP and the KF-PAT in their clinical practice, the lead OTs were asked to document why the assessment and treatment sessions were not performed or performed in a way deviated from the standardized manuals. The research team provided a spreadsheet template to the lead OTs and required that it be filled out with de-identified clinical records of patients who were assessed using the KF-NAP and who were treated using the KF-PAT. The spreadsheet was submitted to the research team every quarter. This spreadsheet was one way that the barriers to implementation and strategies trialed were communicated. The other way was during the monthly calls.

The research team hosted monthly 1-h conference calls with all participating IRFs, represented by the lead OTs. This is an example of the KTA cycle being used to monitor outcomes. A total of 37 conference calls were conducted from June 2017 to September 2020, and they were recorded to document the meeting and later used to create meeting minutes. These minutes were shared via email to all attendees and the lead OTs who were not able to attend. During the calls, the OTs and the research team had the opportunity to share progress related to the training and implementation. The OTs shared any facilitators or barriers implementing either protocol that they may have listed in their spreadsheet or that were new and therefore being reported for the first time. For example, a facilitator was that OTs exchanged experiences about obtaining leadership support when integrating both protocols into their clinical practice. Barriers often shared were related to managing time and resources. When a barrier was shared, other OTs on the call would offer suggestions that worked or did not work for them, and the research team would help determine a resolution. However, if no resolution was suggested during the call or if the solution suggested was not agreed upon, the research team would follow up with the clinicians and sometimes their supervisors after the call. The OTs were encouraged to contact the research team via email at any time for further questions and comments. The calls also allowed the research team to clarify details in the assessment or treatment protocols that might have been forgotten from the training. In addition, starting at the June 2018 meeting, the research team would ask an OT to share a patient case to the group. The cases presented usually included a short medical history and details of either protocol used with that patient. The attendees could offer their comments and asked questions. The sharing of cases facilitated engagement during the meetings as well as encouraged discussions and comradery.

Engaging Other Stakeholders

While the project was focused on integrating the evidence-based protocols into the OT's clinical practice, there were other stakeholders who played significant roles. Because the OTs were involved in the implementation project, there were increased discussions about spatial neglect during care team meetings attended by all disciplines that provided medical and therapy services. In addition, other disciplines were able to easily and informally observe how the KF-NAP and KF-PAT were administered in spaces shared by all disciplines. Also, at several participating sites, the OTs conducted in-services with physicians, nurses, and nutrition staff to share information about this implementation project. This involvement of multiple disciplines could be one action that facilitated the implementation progress, especially in Phase 3 and Phase 4 of the KTA cycle (Figure 1). Also, the IRF administrative leadership were engaged in discussions with the Chief Executive Officers and supported the implementation of both protocols after reviewing implementation goals of the project. Directors of Rehabilitation were informed with the project progress and were encouraged to offer comments and suggestions throughout the project.

Addressing Barriers

The implementation procedures evolved and adapted to different hospital contexts as we continuously learned from OTs' experience using either KF-NAP or KF-PAT protocol with patients and interacting with their colleagues, and from administrators' guidance on hospital regulatory and operational standards. From time to time, a situation would occur that was not expected, or a solution was not immediately available, and we classified the situation as a barrier. When barriers were identified, via verbal or written report, then strategies were carefully customized based on the needs of the setting, and then implemented. A strategy and related actions were decided and executed based on the available resources and contextual situations at the time, usually right after the barrier was identified, rather than based on a pre-determined decision-making roadmap. We followed the CFIR framework (22) to organize the identified barriers and summarize strategies and actions taken to eliminate the barriers (Table 2). Barriers were identified in 4 of the 5 CFIR domains. In these domains, there were no identified barriers in some constructs, but a number of barriers in other constructs. Throughout the process, two limiting factors—time and staff—emerged frequently across different constructs and domains. Time was limited against the administration of the KF-NAP and KF-PAT. Shortage of trained staff trained on the two protocols was a common barrier to assessing all patients with neurological conditions and treating patients with spatial neglect. Collaborating with clinicians and their managers, we offered potential solutions. Some of the barriers identified were resolved shortly such as those identified in the Intervention Characteristics domain (Domain I, Table 2), some barriers ultimately required multiple strategies to fully address the problem such as those identified in the Inner Setting domain (Domain III), and others remained challenging such as several barriers identified in the Process domain (Domain V).

TABLE 2

Table 2. Barrier assessment by the Consolidated Framework for Implementation Research (CFIR) with actions to remove the barrier.

Implementation Outcomes

We evaluated the implementation outcomes using quantitative and qualitative information that was collected during the 4-year period. The study also included collecting patient outcomes, which was reported separately (36, 37).

Numbers of Therapists Trained and Patients Receiving the Care

We used the number of OTs trained to indicate the success of knowledge dissemination and the number of patients receiving spatial neglect care through either the KF-NAP or KF-PAT to indicate the success of knowledge implementation. In April 2020, the research team asked the lead OTs to report the number of therapists they trained on both protocols, and what competency level they reached. In November 2020, we completed the collection of de-identified clinical records shared by participating sites. Note that patient outcomes (i.e., improvement in spatial neglect and rehabilitation outcomes) were reported in separate articles (36, 37).

Lead OTs were interviewed to provide context of the number of therapists trained and the number of patients receiving spatial neglect care. Responses to the interviews were summarized in writing with no audio records. Interview responses were reviewed by Authors PC and CGS.

User Feedback Survey

After the last monthly call (September 2020), a survey was sent to the lead OTs via an online platform, Survey Gizmo. The OTs were asked to report on the IRF that they represented, based on their own experience. See the survey in Appendix 1. The goal was to identify any sustaining barriers and additional facilitators to implementing either protocol now that the research team stepped back. Categorical responses were summarized and described in percentages. The qualitative answers were reviewed by Authors KH, PC, and CGS to enable the identification of emerged categories. We each separately read the data to determine reoccurring information, discussed and compared notes via conference calls and came to an agreement on how to code the data into similar groups or categories.

Results

Number of Therapists Trained

Table 3 reports that overall, 169 OTs at the participating sites were trained to some level of the KF-NAP, and among them, 81 (47.9%) reached the highest level (level 3) of competency. One hundred forty-one OTs were trained for the KF-PAT, and 110 (78.0%) reached the highest level of competency. Site 9 reported that they only implemented the KF-NAP at the beginning of the project and decided not to continue because an ongoing study was using the CBS following the original non-standardized questionnaire format (38). Nonetheless, Site 9 continued using the KF-PAT in their care.

TABLE 3

Table 3. Participating rehabilitation hospitals and occupational therapists (OTs) trained.

Number of Patients Receiving Care

OTs assessed a total of 4,454 patients for spatial neglect using the KF-NAP, and 2,491 (56%) of them had the syndrome. 1,078 (43%) of the patients with spatial neglect were treated using the KF-PAT for at least one session.

To understand why more than half of the patients with spatial neglect did not receive PAT, we had discussions with the lead OTs during the conference calls and reviewed the OT's field notes in the de-identified clinical records in order to determine categories. Also, while the COVID pandemic led to a long pause in 2020 of no treatment using the KF-PAT Portable Kits across all participating sites, there were various factors that contributed to different implementation rates of either protocol in different sites, and therefore We classified the 15 participating sites (excluding Site 9) into three categories: early adopters, additional trained staff needed, and developing facilities.

Five sites (Sites 1, 2, 3, 4, and 6) were the first sites to be trained (aka early adopters). Therefore, many staff OTs, in addition to the implementation champions (lead OTs), were trained during the project. Although the rate of lead OT changes was 50% among these first trained sites, the other OTs were able to step up and take on the lead OT roles. These five sites collectively assessed 3,698 (83%) of all assessed patients and treated 783 (73%) of all treated patients across 16 sites. Based on the shared clinical records, the median CBS scores of PAT-untreated patients at these sites ranged from 2.5 to 3.75, which is a mild level of severity. In the de-identified clinical records and field notes, OTs reported that other deficits such as upper extremity impairment, rather than spatial neglect, was prioritized in OT sessions because the neglect was not severe.

Eight sites (Sites 5, 7, 8, 10, 12, 13, 15, and 16) were classified as “additional trained staff needed.” The median CBS scores of PAT-untreated patients at these sites ranged from 7.5 to 17.38, which is a wide range covering all levels of neglect severity. These sites relied on their lead OTs to implement both protocols while other OTs may or may not have integrated either protocol in their practice. Thus, only patients under lead OTs' care were assessed for spatial neglect, and other OTs who were not trained on the KF-NAP themselves, had to refer patients with neglect signs on their caseloads to the lead OTs for KF-NAP assessment. This indicated that patients whose neglect symptoms were less apparent may have not been identified and therefore not referred. Furthermore, not all patients who were confirmed with spatial neglect received PAT because the work load was too high for the lead OTs. Sites reported in field notes: “we do not have time to train other therapists in these protocols because of other responsibilities.” In addition, when a lead OT left, there was a lag in time during the transition and thus the average 45% change rate of implementation champions among these sites became a significant barrier. These eight sites planned to have additional staff trained early in Year 2020, which however was soon on pause due to the COVID pandemic. The pandemic caused particular challenges for Sites 15 and 16 who had joined the project a few months prior to the shutdown of research activities.

Lastly, four sites fell into the category of “developing facilities.” The management team was at the early stage of development in Sites 15 and 16 (also classified as “additional trained staff needed”) and Sites 11 and 14. These sites were having a hospital-wide staffing call, in order to recruit more therapist to work at their hospitals. Therefore, the combination of situations resulted in low numbers of identified individuals with neglect. That is, the infra-structure was not ready to fully support the implementation project.

User Feedback Survey

Fifteen sites (93.75%) responded to the User Feedback survey. The quantitative aspect of the survey results was summarized in Figure 2. Multiple responses to open-ended questions were able to be categorized. First, the top three barriers related to the KF-NAP not being administered 100% of the time were: (1) lack of time to train the rest of staff on unit to perform the assessment, (2) patients being discharged earlier than expected, and (3) patients requiring other considerations at discharge such as extensive family training. It was also reported by multiple individuals that patients at multiple sites did not always receive all 10 KF-PAT sessions because of three reasons, categorized as: (1) short length of stay, (2) other clinical goals being prioritized, and (3) a lack of trained staff to assist with carrying out all 10 sessions. Besides the implementation of the two protocols, responses on the other aspects of the project included categories: (1) time consuming research documentation, (2) helpful monthly conference calls, and (3) supportive leadership.

FIGURE 2

Figure 2. Summary of quantitative survey results in percentage.

Discussion

There is still work to be done related to translating research into practice and decreasing the research-to-practice gap in the rehabilitation settings despite many efforts being made by many teams (26, 28). The shorter time it takes for the latest evidence to be applied to clinical practice, the greater chance for patients to receive better care (39). This manuscript highlights an implementation project that used the KTA cycle as the process model to assist the transfer of scientific knowledge into clinical practice. Specifically, the KTA cycle helped to guide the project development, report the results and provide specific information for future reproducibility. The CFIR was another implementation tool that was used in this project to assist with the categorization of barriers identified as well as provided a way to organize strategies that were trialed. Similar to others conducting rehabilitation implementation research (26), we found the use of the frameworks to be a strength of the study because the researcher-clinician team had a “road map” to guide implementation of the protocols as well as evaluate outcomes of the implementation.

The close collaboration between researchers and clinicians was key to achieve knowledge translation. The research team and the OTs had frequent communications through the de-identified clinical records and the monthly conference calls. This suggests that rather than a one-way, top-down instruction provision from the research team to the OTs, participants worked together and modified certain aspects on how to administer the KF-NAP and KF-PAT at specific sites. This is one example of how we used the KTA cycle and made an adaptation to fit the local context. However, it was important to the researchers that the core elements of both protocols remained unchanged, in other words, fidelity was maintained. For instance, regarding the KF-NAP protocol, as long as the therapist assessed tasks that relate to skin care or hair care (including facial) then they could be creative to what they ask the patient to complete (e.g., applying makeup instead of washing their face, which is the task suggested in the manual).

Most barriers identified during the implementation process were aligned with previous studies that offered reasons for the difficulties in knowledge translation and evidence-based practice (EBP) implementation in stroke care and rehabilitation (16, 40). One barrier, however, was unexpected. This was when consulting neuro-optometrists questioned whether administering PAT was within the scope of OT practice. The strategy addressing this barrier was to be collaborative, transparent, and be open to inter-professional learning. More specifically, the research team-initiated discussions with the neuro-optometrists about the mechanisms of prism adaptation and offered treatment demonstrations to share the procedures of the KF-PAT. Both KF-NAP and KF-PAT protocols were in use as part of clinical practice by the end of the implementation period. This suggests the tools were accepted by the OTs and adopted into the standard of care (41).

Another interesting finding was that focusing on a single discipline has the potential to change the overall quality of care in the multidisciplinary inpatient rehabilitation system. Shown in a prior study, only 31% of spatial neglect cases were mentioned in care team meetings, which potentially impeded provision of comprehensive care to all patients with spatial neglect (42). In the present project, the change of OT practice via the implementation of both spatial neglect assessment and treatment protocols increased the awareness of spatial neglect care among other disciplines. The CBS scores via the KF-NAP (indicating severity of spatial neglect) and improvements observed after PAT were discussed during care team meetings attended by all disciplines that provided medical and therapy services. Other disciplines could observe the procedures of both protocols easily as OTs worked in the same space with them. Thus, a new vocabulary was created and understood by all the care team members. This became an inner-setting facilitator (22) that emerged during the implementation process, different from facilitators provided by the research team and hospital administrative leaderships. Thus, the researcher-clinician collaboration is critical to initiate knowledge translation, and clinician buy-in and subsequent spontaneous inter-disciplinary communications are essential to strengthen the translation.

The therapists' time to complete the assessment and treatment was a limitation and was the most reported barrier. For the KF-NAP, the difficulty was related to the second (i.e., “post-treatment”) assessment, which is essential in order to measure changes in spatial neglect severity from before to after treatment. The second assessment which was to occur before the patients' discharge was difficult to administer when scheduling conflicts occurred more often at discharge than at admission. Time limitation with the assessment also affected treatment delivery. As recommended by the KF-PAT protocol, patients should complete the full treatment course which includes 10 sessions. In the present project, OTs at several sites shared that they prioritized other therapy activities over PAT when knowing that there was insufficient time to provide the recommended 10 sessions of PAT. This is due to the fact that the length of stay is usually estimated and pre-determined by insurers. Extending approved length of stay in an inpatient rehab solely for the purpose of completing the KF-NAP protocol may be applicable for a small percentage of patients with private insurers. For patients covered by Medicare's prospective payment plan, the hospital may elect to extent a patient's length of stay if the interdisciplinary team feels the additional days to complete the protocol would outweigh benefits of the projected discharge date. Among 2019 Medicare beneficiaries, for example, the average length of stay is 17, 16, and 15 days for patients with stroke, traumatic brain injury, and non-traumatic brain injury, respectively. Even if patients are assessed using the KF-NAP within the first few days of admission, there may be <10 full treatment days for a therapist to provide the recommended 10 once-daily PAT sessions. This factor in addition to many other factors may contribute to a decision not to provide PAT to certain patients.

The limited number of trained OTs in either protocol was another major barrier. This seems inconsistent with the fact that about 170 therapists were trained to integrate the KF-NAP and about 140 therapists were trained to use the KF-PAT in their practice across 16 IRFs. However, 10 participating sites primarily relied on lead OTs in administering the protocols, Who were unable to train other OTs due to time constraints, other clinical duties, and factors related to readiness of certain hospitals. The overall 45% change rate of lead OTs created lags of researcher-clinician communication and further slowed down the implementation progress. Regarding user feedback, 46.7% of survey respondents reported that it would be beneficial to have meetings or calls with their individual leadership, so they could share any continued barriers and determine strategies together. Almost 90% of the survey respondents mentioned that they would be interested in a yearly refresher course, on both protocols. This may help sustain implementation and also ensure fidelity (43). Thus, after collectively providing spatial neglect care to more than 4,500 patients in the context of this implementation project, OTs saw the need of continuing implementing KF-NAP and KF-PAT in their practice.

Study Limitations

The project was initially driven by researchers and fueled by a collaborative effort shared by researchers and clinicians (including hospital administrative leaders). This is a strength but also the limitation of the project such that the outcomes may not be generalizable to facilities that have little access to researchers, especially researchers knowledgeable about implementation science. Another limitation was the inability to evaluate which strategies that were used toward eliminating a given barrier during the implementation process, had the best success rates vs. other strategies. It was not our priority to determine the best strategy but to offer solutions at the time when a barrier was present. Therefore, we cannot comment on the recommended strategies or which strategies should be trialed first. Further investigations formally testing outcomes such as feasibility, adoption and acceptability of delivering the protocols are needed to identify all the challenges to maintaining the implementation, and to determine how to overcome those challenges (43).

Conclusion

The project demonstrated a researcher-clinician partnership in not only knowledge generation but also knowledge translation (e.g., dissemination and implementation of knowledge to be applied clinically). Evidence-based protocols can be implemented through multiple, tireless iterations of barrier reduction and problem solving with active participation of practitioners and practical support from leaders. There were no unintended consequences of the implementation efforts. Frequent communications and exchanging information with stakeholders at different levels, may be determinant to the success of each implementation phase. The results of the present project appeared promising in EBP implementation for spatial neglect care. However, further efforts are needed to promote the persistent inclusion of EBP for spatial neglect as the standard of care in inpatient rehabilitation. We also suggest the following future implementation efforts: (1) a pre-trial consultation with organizational leadership could ensure that sufficient clinician time can be blocked out, and (2) enabling all staff to receive training and deliver the intervention with higher levels of fidelity.

Data Availability Statement

The data is not available to be shared.

Ethics Statement

The studies involving human participants were reviewed and approved by Kessler Foundation. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author Contributions

KH and PC designed the study, analyzed and interpreted the data, and then wrote the manuscript. PC, KH, and CG-S interpreted the qualitative component of the study. JM contributed to editing the manuscript and was essential in data collection efforts. CG-S was essential to study member recruitment and edits to the entire manuscript. AB was key to study design and edited the entire manuscript. RG edited the manuscript. All authors read and approved the final manuscript.

Funding

This work was supported by the Wallerstein Foundation for Geriatric Improvement and the Charles and Ann Serraino Foundation. Contents in this paper do not represent the policy of the funding agencies.

Conflict of Interest

The KF-NAP and KF-PAT are registered trademarks of Kessler Foundation in the United States. JM and PC are employees of Kessler Foundation.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We thank all the occupational therapists and their supervisors of the participating sites for implementing the KF-NAP^® and KF-PAT^® protocols in their practice and for sharing data for our analysis. We thank Kena Patel, Marinos Pylarinos, Grace Wells, Maria Mawhinney, Louis Varillas, Tamara Burdinoska, and Emma Kaplan for administrative support and data management. We acknowledge Geoffrey Hill for his contribution to the design and development of the KF-PAT equipment.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frhs.2022.839517/full#supplementary-material

Footnotes

1. ^KF-PAT Portable Kit, Stoelting, 620 Wheat Lane, Wood Dale, IL, USA 60191.

References

1. Heilman KM, Valenstein E, Watson RT. Neglect and related disorders. Semin Neurol. (2000) 20:463–70. doi: 10.1055/s-2000-13179

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Esposito E, Shekhtman G, Chen P. Prevalence of spatial neglect post-stroke: a systematic review. Ann Phys Rehabil Med. (2021) 64:101459. doi: 10.1016/j.rehab.2020.10.010

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Chen P, Ward I, Khan U, Liu Y, Hreha K. Spatial neglect hinders success of inpatient rehabilitation in individuals with traumatic brain injury: a retrospective study. Neurorehabil Neural Repair. (2016) 30:451–60. doi: 10.1177/1545968315604397

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Buxbaum LJ, Ferraro MK, Veramonti T, Farne A, Whyte J, Ladavas E, et al. Hemispatial neglect subtypes, neuroanatomy, and disability. Neurology. (2004) 62:749–56. doi: 10.1212/01.WNL.0000113730.73031.F4

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Chen P, Fyffe DC, Hreha K. Informal caregivers' burden and stress in caring for stroke survivors with spatial neglect: an exploratory mixed-method study. Top Stroke Rehabil. (2017) 24:24–33. doi: 10.1080/10749357.2016.1186373

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Chen P, Hreha K, Kong Y, Barrett AM. Impact of spatial neglect on stroke rehabilitation: evidence from the setting of an inpatient rehabilitation facility. Arch Phys Med Rehabil. (2015) 96:1458–66. doi: 10.1016/j.apmr.2015.03.019

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Nijboer TCW, Kollen BJ, Kwakkel G. The impact of recovery of visuo-spatial neglect on motor recovery of the upper paretic limb after stroke. PLoS ONE. (2014) 9:e100584. doi: 10.1371/journal.pone.0100584

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Oh-Park M, Hung C, Chen P, Barrett AM. Severity of spatial neglect during acute inpatient rehabilitation predicts community mobility after stroke. PMR. (2014) 6:716–22. doi: 10.1016/j.pmrj.2014.01.002

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Wee JYM, Hopman WM. Comparing consequences of right and left unilateral neglect in a stroke rehabilitation population. Am J Phys Med Rehabil. (2008) 87:910–20. doi: 10.1097/PHM.0b013e31818a58bd

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Yoshida T, Mizuno K, Miyamoto A, Kondo K, Liu M. Influence of right versus left unilateral spatial neglect on the functional recovery after rehabilitation in sub-acute stroke patients. Neuropsychol Rehabil. (2020) 1–22. doi: 10.1080/09602011.2020.1798255. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Navarro M-D, Lloréns R, Noé E, Ferri J, Alcañiz M. Validation of a low-cost virtual reality system for training street-crossing. A comparative study in healthy, neglected and non-neglected stroke individuals. Neuropsychol Rehabil. (2013) 23:597–618. doi: 10.1080/09602011.2013.806269

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Winstein CJ, Stein J, Arena R, Bates B, Cherney LR, Cramer SC, et al. Guidelines for adult stroke rehabilitation and recovery: a guideline for healthcare professionals from the American heart association/American stroke association. Stroke. (2016) 47:e98–e169. doi: 10.1161/STR.0000000000000098

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Hebert D, Lindsay MP, McIntyre A, Kirton A, Rumney PG, Bagg S, et al. Canadian stroke best practice recommendations: stroke rehabilitation practice guidelines, update 2015. Int J Stroke Off J Int Stroke Soc. (2016) 11:459–84. doi: 10.1177/1747493016643553

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Australian Clinical Practice Guidelines-Stroke Rehabilitation,. Stroke Foundation. (2021). Available online at: https://files.magicapp.org/guideline/11009797-9966-4e79-9344-afdf363f3503/published_guideline_5020-6_2.pdf (accessed December 18, 2021).

15. Chen P, Pitteri M, Gillen G, Ayyala H. Ask the experts how to treat individuals with spatial neglect: a survey study. Disabil Rehabil. (2018) 40:2677–91. doi: 10.1080/09638288.2017.1347720

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Petzold A, Korner-Bitensky N, Salbach NM, Ahmed S, Menon A, Ogourtsova T. Determining the barriers and facilitators to adopting best practices in the management of poststroke unilateral spatial neglect: results of a qualitative study. Top Stroke Rehabil. (2014) 21:228–36. doi: 10.1310/tsr2103-228

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Bayley MT, Hurdowar A, Richards CL, Korner-Bitensky N, Wood-Dauphinee S, Eng JJ, et al. Barriers to implementation of stroke rehabilitation evidence: findings from a multi-site pilot project. Disabil Rehabil. (2012) 34:1633–8. doi: 10.3109/09638288.2012.656790

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Fillion B, Rochette A, Girard A. Challenges of being a scholarly clinician as perceived by stroke rehabilitation professionals. Disabil Rehabil. (2014) 36:521–8. doi: 10.3109/09638288.2013.797516

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the expert recommendations for implementing change (ERIC) project. Implement Sci IS. (2015) 10:21. doi: 10.1186/s13012-015-0209-1

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, et al. Lost in knowledge translation: time for a map? J Contin Educ Health Prof. (2006) 26:13–24. doi: 10.1002/chp.47

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Janzen S, McIntyre A, Richardson M, Britt E, Teasell R. Building a knowledge to action program in stroke rehabilitation. Can J Neurol Sci J Can Sci Neurol. (2016) 43:619–25. doi: 10.1017/cjn.2016.258

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. (2009) 4:50. doi: 10.1186/1748-5908-4-50

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients' care. Lancet Lond Engl. (2003) 362:1225–30. doi: 10.1016/S0140-6736(03)14546-1

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Powell BJ, Fernandez ME, Williams NJ, Aarons GA, Beidas RS, Lewis CC, et al. Enhancing the impact of implementation strategies in healthcare: a research agenda. Front Public Health. (2019) 7:1–9. doi: 10.3389/fpubh.2019.00003

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Powell BJ, Proctor EK, Glass JE. A systematic review of strategies for implementing empirically supported mental health interventions. Res Soc Work Pract. (2014) 24:192–212. doi: 10.1177/1049731513505778

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Leavy B, Joseph C, Kwak L, Franzén E. Implementation of highly challenging balance training for Parkinson's disease in clinical practice: a process evaluation. BMC Geriatr. (2021) 21:96. doi: 10.1186/s12877-021-02031-1

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Pastva AM, Coyle PC, Coleman SW, Radman MD, Taylor KM, Jones SB, et al. Movement matters, and so does context: lessons learned from multisite implementation of the movement matters activity program for stroke in the comprehensive postacute stroke services study. Arch Phys Med Rehabil. (2021) 102:532–42. doi: 10.1016/j.apmr.2020.09.386

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Bowden MG, Monsch ED, Middleton A, Daughtry C, Powell T, Kraft SV. Lessons learned: the difficulties of incorporating intensity principles into inpatient stroke rehabilitation. Arch Rehabil Res Clin Transl. (2020) 2:100052. doi: 10.1016/j.arrct.2020.100052

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Chen P, Hreha K. KF-NAP 2015 Manual. Kessler Foundation Learning Center. (2015). Available online at: https://www.kflearn.org/courses/kf-nap-2015-manuals

30. Chen P, Hreha K. KF-PAT 2020 Manual. Stoeling. Available online at: https://www.kflearn.org/courses/kf-pat-2018-manual (accessed April 21, 2020).

31. Chen P, Chen CC, Hreha K, Goedert KM, Barrett AM. Kessler foundation neglect assessment process uniquely measures spatial neglect during activities of daily living. Arch Phys Med Rehabil. (2015) 96:869–76.e1. doi: 10.1016/j.apmr.2014.10.023

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Chen P, Hreha K, Fortis P, Goedert KM, Barrett AM. Functional assessment of spatial neglect: a review of the catherine bergego scale and an introduction of the kessler foundation neglect assessment process. Top Stroke Rehabil. (2012) 19:423–35. doi: 10.1310/tsr1905-423

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Fortis P, Chen P, Goedert KM, Barrett AM. Effects of prism adaptation on motor-intentional spatial bias in neglect. Neuroreport. (2011) 22:700–5. doi: 10.1097/WNR.0b013e32834a3e20

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Chen P, Goedert KM, Shah P, Foundas AL, Barrett AM. Integrity of medial temporal structures may predict better improvement of spatial neglect with prism adaptation treatment. Brain Imag Behav. (2014) 8:346–58. doi: 10.1007/s11682-012-9200-5

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Goedert KM, Chen P, Foundas AL, Barrett AM. Frontal lesions predict response to prism adaptation treatment in spatial neglect: a randomised controlled study. Neuropsychol Rehabil. (2018) 30:1–22. doi: 10.1080/09602011.2018.1448287

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Chen P, Hreha K, Gonzales-Snyder C, Rich T, Gillen R, Barrett AM. Effects of prism adaptation treatment on spatial neglect and rehabilitation outcome: dosage matters.

Google Scholar

37. Chen P, Diaz-Segarra N, Hreha K, Kaplan E, Barrett AM. Prism adaptation treatment improves inpatient rehabilitation outcome in individuals with spatial neglect: a retrospective matched control study. Arch Rehabil Res Clin Transl. (2021) 3:100130. doi: 10.1016/j.arrct.2021.100130

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Azouvi P. Functional consequences and awareness of unilateral neglect: study of an evaluation scale. Neuropsychol Rehabil. (1996) 6:133–50. doi: 10.1080/713755501

CrossRef Full Text | Google Scholar

39. Juckett LA, Robinson ML, Wengerd LR. Narrowing the gap: an implementation science research agenda for the occupational therapy profession. Am J Occup Ther. (2019) 73:7305347010p1–6. doi: 10.5014/ajot.2019.033902

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Purvis T, Moss K, Denisenko S, Bladin C, Cadilhac DA. Implementation of evidence-based stroke care: enablers, barriers, and the role of facilitators. J Multidiscip Healthc. (2014) 7:389–400. doi: 10.2147/JMDH.S67348

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Proctor EK. Leverage points for the implementation of evidence-based practice. Brief Treat Crisis Interv. (2004) 4:227–42. doi: 10.1093/brief-treatment/mhh020

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Chen P, McKenna C, Kutlik AM, Frisina PG. Interdisciplinary communication in inpatient rehabilitation facility: evidence of under-documentation of spatial neglect after stroke. Disabil Rehabil. (2013) 35:1033–8. doi: 10.3109/09638288.2012.717585

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. (2011) 38:65–76. doi: 10.1007/s10488-010-0319-7

PubMed Abstract | CrossRef Full Text | Google Scholar