The Nottwil Standard-Development and Implementation of an International Classification of Functioning, Disability and Health-Based Clinical Standard Assessment for Post-acute Rehabilitation After Newly Acquired Spinal Cord Injury

Introduction: Assessments during rehabilitation of spinal cord injury (SCI) align with the World Health Organization's classifications and national quality requirements. This paper aims to report on the development and first implementation experiences of an institutional standard of assessments performed after newly acquired SCI.

Setting: Specialized SCI acute care and post-acute rehabilitation clinic in Switzerland.

Methods: A situation analysis of an interdisciplinary post-acute SCI rehabilitation program was performed. The results informed a subsequent consensus-based selection of assessments, and an information and implementation strategy. Linking to the ICF Core Set for SCI in post-acute settings and ICF Generic-30 Set was performed. The Nottwil Standard was piloted for 18 months.

Results: Situation analysis: A battery of 41 assessments were irregularly performed during initial rehabilitation after newly aquired SCI. Selection of assessments: A multidisciplinary group of clinicians agreed on 10 examinations, 23 assessments and two questionnaires that make up the Nottwil Standard. In total, 55 ICF categories are covered, including most of the ICF Generic-30 Set categories. The implementation strategy included Executive Board commitment, a structured improvement project, guidelines for documentation and assessments, a manual controlling system, and staff training on the Nottwil Standard. Pilot phase: 54 persons with paraplegia and 42 with tetraplegia (75 male; 21 female) were included. Twenty-seven assessments out of 33 assessments were performed in more than 80% of all observed patients' rehabilitation.

Conclusion: Implementation of a standard assessment schedule was feasible but required a well-structured process with good communication strategy and controlling mechanism, and full engagement of involved professions.

Introduction

Quality clinical care of persons with spinal cord injury (SCI) after a newly acquired SCI demands a comprehensive and accurate assessment of their medical and functioning needs (1–4). A major challenge is determining the assessment tools, clinical examinations and other sources of information (collectively referred to as “assessment tools” from now on) to employ in the assessment. Ideally, such a battery of assessment tools is part of standard assessment procedures and reflects current rehabilitation practice. The development of such a standard should rely on a number of principles that guide its development.

Guiding Principles

The first guiding principle is the application of WHO classifications, specifically the International Classification of Diseases (ICD) (5) and the International Classification of Functioning, Disability and Health (ICF) (6), to meet the objective of SCI rehabilitation, i.e., optimization of the person's functioning (2, 7). The ICF is central to the development of an assessment standard that relies on a four-step approach of standardized documentation of functioning (8). While the ICD can be used to diagnose disease and injury, the ICF can be used to describe the functioning of a person from a bio-psycho-social perspective. The ICF encompasses categories that are hiearchically organized under the following components: body functions (e.g., shoulder pain, muscle tone functions in wheelchair-using patients with SCI), body structures (e.g., shoulder joint or arms), activities and participation (e.g., moving around using a wheelchair, work, participation in wheelchair sports) and environmental factors (e.g., barrier-free buildings). ICF categories contain a letter, i.e., b for body functions, s for body structures, d for activities and participation and e for environmental factors) and a series of numbers representing the four levels of detail in the hierarchy, from least detailed chapter level (e.g., b2 Sensory functions and pain) to the most detailed fourth level (e.g., b28014 Pain in upper limb) (6).

The four-step approach encompasses deciding on (1) what ICF domains (or categories) to document; (2) what perspective to take; (3) what data collection tools to apply; and 4) which approach to use for reporting. ICF Core Sets, short lists of ICF categories for specific health conditions and settings (9, 10), can be used to define what to document. The ICF Generic-7 and Generic-30 Sets as a minimum set of categories independent of health condition and setting, can also be used in the first step and across the care continuum (11, 12). For the rehabilitation care of persons with newly acquired SCI, the ICF Core Set for SCI in post-acute care (13) would be most appropriate to use. This ICF Core Set was developed in a multiple stage consensus process in which experienced SCI specialists from different professions and countries across the world defined the most relevant categories that can be used during this rehabilitation phase.

The ICF is also key to the second principle, i.e., aligning with international initiatives to scale up rehabilitation. A trailblazing initiative has been led by the Physical and Rehabilitation Medicine (PRM) Section and Board of the European Union of Medical Specialists (UEMS-PRM) that reflects the crucial interaction between practice, science and governance (policy) (14). The UEMS-PRM implementation action plan calls for, among other things, the “identification of data collection tools that cover the ICF domains included in the clinical assessment schedules [CLAS] of specific rehabilitation service [types]” (14–16). Aligned with the aforementioned four-steps approach, a CLAS is the specification of functioning aspects to document [using ICF Core Sets (9, 10) and ICF Generic Sets (11, 12)], for whom and when, and the data collection tools to employ (16, 17). Given that a CLAS designated for a specific health condition should cover all relevant aspects of functioning and contextual factors relevant for persons with that health condition, the ICF Core Set for SCI in post-acute care should be used for specifying the CLAS for the rehabilitation of persons with newly acquired SCI. Assessment tools should be administered as soon as possible at the beginning and at the end of an intervention (17). The UEMS-PRM action plan also calls for developing national ICF-based rehabilitation quality management strategies that are consistent with existing clinical quality management systems (14). This is related to the third principle.

The third principle is the consideration of national clinical quality management requirements for the reporting of health data. In Switzerland, functioning data from rehabilitation institutions are reported to the Swiss National Association for Quality Development in Hospitals and Clinics (ANQ) [https://www.anq.ch/de/fachbereiche/rehabilitation/]. In addition to functioning data, health condition(s), the Swiss Classification of Operations (CHOP) codes for interventions and procedures (18, 19), and the definition and achievement of rehabilitation goals according to ANQ criteria (20).

The fourth principle is the consideration of evidence provided by the SCI-specific cohort studies, e.g., Swiss SCI cohort study (SwiSCI; https://www.swisci.ch) and the European Spinal Cord Injury cohort study (EMSCI; https://www.emsci.org), and SCI-specific research platforms and resources on outcomes, e.g., the Spinal Cord Injury Research Evidence (https://scireproject.com/ and) or the Spinal Cord Outcomes Partnership Endeavor (SCOPE) (21, 22) in developing robust assessment standards. Especially relevant for developing standards for the assessment of persons with newly acquired SCI are the S2e guidelines for outcome measures in initial rehabilitation after the onset of a SCI (23, 24) led by the German-speaking Medical Society of Paraplegia (DMGP). Scientific evidence also encompasses information about validity and reliability of assessment tools (25).

The fifth principle relates to the implementation of the assessment standard. For an assessment standard to be feasibly implemented, it should adhere to the requirements of insurers, i.e., that the provision of care is suitable, feasible, cost-efficient and is based on scientific evidence (26). Essential for effective implementation of an assessment standard are health professionals trained in applying the tools contained in the assessment standard, including knowing the appropriate timing for applying specific assessment tools (27), as well as care provider leaders who support its implementation (28). Moreover, the feasibility of implementating an assessment standard is enhanced with the availability of a supportive infrastructure, e.g., health information system (HIS) and administrative support (28).

Initiating the Project

The impetus for developing the Nottwil Standard is illustrative of what UEMS-PRM highlights as the interaction between practice, science and governance (policy). The clinic's decision-makers recognized the need to implement international recommendations for outcome measures and evidence-based rehabilitation of persons with newly acquired SCI (practice), and to meet requirements of financing-relevant stakeholders (e.g., insurances) for rehabilitation quality as well legal requirements (governance/policy). Furthermore, the clinic and its partner research institute envisioned the translation of cohort study results, namely from SwiSCI and EMSCI in rehabilitation quality management. The decision to initiate the project was made at a workshop of clinic and research institute leaders in 2015. Subsequently, a workshop was held in January 2016 at the research institute, during which the UEMS-PRM implementation action plan was developed (14).

The objective of this paper is to report on the development of an assessment standard (called Nottwil Standard) for use in the rehabilitation of persons after newly acquired SCI according to the guiding principles and to report on the first experiences in implementing it.

Methods

Design

This study is an implementation study starting with an observational situational analysis, presenting the participatory consensus process and ending with an observational analysis after the implementation of the newly developed standard.

Setting

This project took place in an acute inpatient rehabilitation and outpatient clinic specialized for SCI and under the auspices of its integrated quality management and multiproject management unit. The clinic is part of a larger organization that also includes a research institute and is governed by a foundation. Since 2006 the clinic has been developing an ICF-oriented culture, striving to increasingly implement ICF elements in interprofessional clinical management.

Preparatory Activities

Situation Analysis

An observational study, a situation analysis was conducted that involved an analysis of retrospective data from patients (adults ≥18 years old) admitted for rehabilitation after newly acquired SCI from December 2014 to December 2015 (3, 27). Among the data analyzed were the assessment tools used by physicians, physiotherapists, occupational therapists and nurses, and assessment procedures, including adherence to administration recommendations (e.g., timing). The situation analysis results were considered in the development of the Nottwil Standard.

Developing the Nottwil Standard

The project was led by a rehabilitation physician with experience in rehabilitation quality management research methodology and conducted by a multidisciplinary core project team (CPT) consisting of the project leader, the rehabilitation department head, the chief physician of rehabilitation, the head of corporate development and scientific assistant. The CPT was supported by an expanded project team (EPT) representing all relevant professions involved in the routine SCI rehabilitation process, including peer counselors, who represented the perspective of a person with SCI. The development of the Nottwil Standard was driven by an inclusive and consensus-based approach. Content and milestones were discussed, revised and approved by the CPT and EPT. The overall project proceeded in alignment with the CLAS concept (16, 17) and the four-step approach (8).

Step 1: Defining the Domains to Document

In a first step, the CPT defined the domains (or ICF categories) based on the ICF Generic-7 and−30 Sets (11, 12), the results of the situation analysis (3, 27) and in line with the guidelines on outcome measures of the DMGP. (23, 24). The CPT and the EPT decided to select assessment tools that are able to measure the defined ICF categories. If no assessments were available to cover specific categories, the patient's status in that category would be narratively described.

Step 2: Deciding What Perspective to Take

The CPT and EPT prioritized clinical and health care professional (HCP)-administered assessment tools to measure the ICF category, as these were deemed objective measures of functioning. To reflect the patient's perspective, patient-reported outcome measures (PROMs) were also selected. Associated with perspective is the decision about which profession(s) are responsible for assessing which aspect of functioning. In turn, this also guided the decision on the assessment tools to include. In some cases, more than one profession was defined as responsible.

Step 3: Identifying What Data Collection Tools to Apply

Identifying the data collection tools to apply mirrors the project's aim, i.e., to develop the Nottwil Standard. In addition to applying the results of steps 1 and 2, this step considered the results of the situational analysis, specifically which assessment tools were employed to measure different aspects of patient functioning and health, how and how often assessment tools were used, as well as the recommended toolkit defined in the DMGP guidelines (24) and the assessment tools recommended in the SwiSCI and EMSCI studies. Furthermore, suggestions for additional assessment tools recommended by HCPs based on recent scientific evidence were also considered.

Step 4: Deciding on When to Assess

The CPT defined the timepoint for assessment with the CLAS recommendation of the UEMS-PRM (16), the recommended timepoints from the SwiSCI (4, 12, 24 weeks post-injury and discharge) (29) and the EMSCI (1, 4, 12, 24, 48 weeks post-injury) (30) studies in mind.

Toward Implementation of the Nottwil Standard

The implementation of the Nottwil Standard was documented in an implementation plan that outlined its integration in routine practice and in the existing documentation system, and the meetings with the EPT and all involved professional groups (physicians, therapists and nurses). To facilitate the integration of the Nottwil Standard in the clinic's electronic documentation system, documentation and process-based management representatives of the clinic's information technology department were also involved.

As Switzerland is a multi-language country, language diversity in the development of the Nottwil Standard was deemed important. Thus, where possible, PROMs in the Swiss languages of German, French, and Italian or English were included in the Nottwil Standard. These are the same languages used in the SwiSCI study (31).

To ensure the smooth implementation of the Nottwil Standard, a 1-year pilot study was conducted.

Pilot Study

The pilot study was approved by the ethical committee (EKNZ Req-2020-01416) as a quality assurance project. The aim of the pilot implementation and quality assurance project was to evaluate the compliance with the standard. Data collection took place and included all patients admitted for initial rehabilitation after 1 July 2019 and discharged before 31 December 2020. Baseline patient characteristics, e.g., gender, age, completeness and level of lesion, as well as admission data, e.g., date of SCI onset and time of assessment, were recorded by the scientific assistant.

During the pilot study, the CPT regularly collected feedback and suggestions for improvement from the clinical staff beyond the EPT. The CPT and EPT discussed the collected input in two half-year feedback meetings, and the Nottwil Standard was adapted accordingly. The CPT and EPT regularly informed their respective clinical teams about changes that impacted the application of the Nottwil Standard during the pilot study.

Results

Situation Analysis

In total, 41 assessment tools were administered, 10 of which were administered more than once per patient. Of these 10 tools, the most frequently used were Spinal Cord Independence Measure III (SCIM) (32, 33), skin assessment and the Manual Muscle Test (34). The results show that outcome measures for motor activity, mobility and self-care were administered regularly, while measures for the autonomous nervous system, mental health and participation were not. Furthermore, neurological assessments like the International Standards for Neurological Classifications of SCI (ISNCSCI) (35) were not administered consequently nor at the recommended time points. Furthermore, the battery of assessment tools at the time did not cover the spectrum of categories of the ICF Core Set for post-acute care (3, 27).

Defining the Domains to Document

The included ICF categories are presented in Table 1.

TABLE 1

Table 1. Overview of the categories of the ICF Generic-30 Set (12). Post-acute SCI Set (13), and categories resulting from linking the Nottwil Standard to the ICF.

The Nottwil Standard does not contain assessment tools that cover the following categories of the ICF Generic-30 Set and ICF Core Set for SCI in post-acute care due to the lack of adequate and established assessment tools that measure these categories: d230 Carrying out daily routine, d240 Handling stress and other psychological demands, d570 Looking after one's health, d640 Doing housework, d660 Assisting others, d710 Basic interpersonal interactions and d770 Intimate relationships.

Deciding What Perspective to Take

The following professions were defined as those responsible for administering the Nottwil Standard: physicians (paraplegiology, neurology, urology, radiology, gynecology, pain management, hand surgery), nursing, physiotherapy, occupational therapy, psychology, social service, speech therapy, nutritional therapy, vocational counselor. See Table 2 for the list of assessment tools and responsible professions. Two PROMs, the Hospital Anxiety and Depression Scale (HADS) (36) and the SCI Quality of Life Basic Data Set (37), were also included in the Nottwil Standard.

TABLE 2

Table 2. The Nottwil Standard assessment tools, responsible professions and compliance of all assessments during the pilot study.

Identifying What Data Collection Tools to Apply

The Nottwil Standard version used for the pilot study contained 10 clinical examinations, 23 assessment instruments and two questionnaires. In addition, 7 tools were added specifically for the assessment of patients with tetraplegia (Table 3) and 5 tools to assess patients with walking ability (Table 4). Several assessment tools recommended in the DGMP guidelines and by the HCPs were also included: the International Standard of Autonomic Function in Spinal Cord Injury (ISAFSCI) (38), Spinal Cord Injury Pain Instrument (SCIPI) (39), International Spinal Cord Injury Pain classification (ISCIP) (40), 10 Meter Walk Test for Spinal Cord Injury (10 MWT) (41, 42) and WHO-QoL BREF (43), the Aktivitätstest zur Mobilität im Rollstuhl (activity test for mobility in wheelchair; AMR) (44, 45), the Bogenhausener Dysphagia Score (BODS) (46), and magnetic resonance imaging (MRI) of the brain and spinal cord. The Nottwil Standard tools were organized according to the ICF components of body functions and structures and activities and participation and ICF categories, and quality of life. See Tables 2–4.

TABLE 3

Table 3. The Nottwil Standard assessment tools, responsible professions and compliance of all assessments for 42 patients with tetraplegia during pilot study.

TABLE 4

Table 4. The Nottwil Standard assessment tools, responsible professions and compliance of all assessments for patients with walking ability during the pilot study.

Deciding on When to Assess

The time points for conducting the assessment were defined as follows: admission (0–2 weeks after admission) and discharge (0–3 weeks before discharge). For specific assessment tools, additional time points were considered clinically relevant. See Table 2.

Toward Implementation of the Nottwil Standard

Other than the assessment tools recommended by the DMGP guidelines (i.e., ISAFSCI, SCIPI, ISCIP, 10 MWT, WHO-QoL BREF) or the HCPs (i.e., AMR, BODS and MRI), the assessment tools included in the Nottwil Standard had already been in routine use in the clinic and integrated in the HIS. The newly introduced assessments were initially introduced in paper form. The ISAFSCI was later integrated into the clinic's HIS.

In terms of the controlling mechanism during the pilot study, when an assessment was not conducted as described in the Nottwil Standard, reminders were manually sent to the responsible person(s), and reasons for non-performance were documented in the HIS for the specific patient. This documentation was visible to all clinical staff involved in the rehabilitation of that patient. The development of an automated reminder system is planned.

Pilot Study

Forty-two patients with tetraplegia and 54 patients with paraplegia (75 were male and 21 female) were included in the pilot study.

All senior (n = 10) and junior (n = 19) physicians, 40 physiotherapists, 20 occupational therapists, 8 psychologists, 5 nutrition specialists, 6 social workers participated in the training on the Nottwil Standard. While many of the assessment tools were already in routine use, physicians had to learn to use the ISAFSCI, ISCIP and SCIPI, and physiotherapists had to be trained to use the 10 MWT.

The following assessments were administered at a 100% compliance level in accordance with the Nottwil Standard recommendations: ISNCSCI, urological examination, manual muscle tests, range of motion testing and SCIM III. See Table 2. The lowest compliance rates were observed for ISAFSCI (35%), ISCIP (35%), SCIPI (35%), (HADS 41%). Assessments using WHOQOL BREF was not implemented at all.

In summary, the Nottwil Standard (see Tables 2–4) covers the complexity of functioning associated with SCI, including but not limited to neuromuscular functions (e.g., 10 MWT, manual muscle test), functions of the autonomic nervous system (e.g., ISAFSCI, blood pressure), pain (e.g., numeric rating scale for pain), mental/psychological functions (e.g., HADS), bladder and bowel management (e.g., SCIM III), mobility (e.g., AMR), participation in work and social life (e.g., SCIM III, ANQ goals), influence of assistive devices (e.g., ANQ goals). Since the functioning of persons with SCI can differ greatly depending on whether the person is living with paraplegia or tetraplegia, or can or cannot walk, the Nottwil Standard also contains specific assessment tools for these sub-populations of persons with SCI.

Discussion

In this paper, we reported on the development of the Nottwil Standard, an ICF-based assessment standard for use in the rehabilitation of persons after newly acquired SCI and on the first experiences in implementing it. Developing the Nottwil Standard not only met the challenge of determining a battery of tools that support a comprehensive and accurate assessment of health and functioning of patients with SCI, it showed that it is feasible to develop and implement it in an interprofessional and participatory manner. Furthermore, this project reflects the potential for real-life clinical application of the ICF that also promotes the clinical quality management.

Potential for Real-Life Clinical Application of the ICF

Although the ICF was launched in 2001, the implementation of the ICF in the clinical management of individuals with SCI took over a decade. This is consistent with findings of a mixed method examination of the extent of ICF diffusion in clinical rehabilitation (not only SCI) between 2001 and 2010. This study showed that clinical implementation of the ICF at the time was rare and called for more large-scale research to address the need for best practice recommendations implementing the ICF in clinical rehabilitation (47). In terms of SCI care, there were early implementation efforts, e.g., in developing an ICF-based electronic tool for use in the long-term clinical follow-up of patients with SCI (48). The ICF has also been key in framing outcomes in SCI care, including the International SCI data sets (49). Efforts to implement the ICF in SCI rehabilitation gained momentum with the development of the ICF Core Set for SCI for the post-acute care and the ICF Generic-30 Set (12, 13), For example, the DGMP guidelines for outcome measures in inital rehabilitation after the onset of a SCI calls for using the ICF Core Set for SCI in post-acute care in selecting the outcome measures to use (24). This was one of the drivers for developing the Nottwil Standard.

The ICF and its underlying comprehensive biopsychosocial perspective also stimulated the decision to include the additional assessment tools suggested by the HCP, i.e., ISAFSCI for autonomic functioning, SCIPI and ISCIP for pain examination, AMR for wheelchair mobility, BODS for swallowing function, the brain and spinal MRI for nerve structural changes, the Montreal Cognitive Assessment for neurocognitive functioning (50) and WHO-QoL for the quality of life evaluation. This helped to ensure that the Nottwil Standard comprehensively covered as many SCI-relevant functioning areas as possible. Ultimately, the aim of establishing such a comprehensive standard for assessing functioning and health of patients with SCI is to improve quality of care.

Quality Management in SCI Rehabilitation

The value of employing the ICF in clinical quality management has been recognized at the national and international level. At the international level, the UEMS-PRM has developed the European Framework for rehabilitation service types and corresponding CLAS as ICF-based standards for improving rehabilitation quality in Europe (14–16). At the national level, the ANQ, the national organization responsible for ensuring quality hospitals and clinics in Switzerland, calls for using the ICF in participation goal-setting (20). For this reason, the Nottwil Standard includes participation goal-setting based on ANQ criteria. The ANQ also calls for employing specific functioning-based instruments for reporting outcomes to the ANQ for further developing and improving quality in Swiss hospitals and clinics. These instruments reflect concrete ICF categories (51).

Ensuring clinical quality in SCI rehabilitation goes beyond ICF implementation. The pilot study showed that the successful implementation of the Nottwil Standard requires the commitment at the institutional level, active involvement of clinicians and an effective information-sharing strategy. Active involvement of the clinicians encompassed training on the Nottwil Standard and continuous discussion, evaluation and adaptation of its use (ongoing). These discussions, the controlling mechanism put in place during the pilot study and the dissemination of information on the status of the pilot study and planned adaptations of the Nottwil Standard were all elements of the information-sharing strategy.

Culture of Change

The implementation of the Nottwil Standard constitutes a change in the way assessments are done in the rehabilitation of newly injury patients with SCI. Managing change as a result of the Nottwil Standard can be viewed from the perspective of Kotter's eight steps of change management in health care: “increase urgency, building guiding teams, get the vision right, communicate for buy-in, enable action, create short-term wins, don't let up, make it stick” (52). The sense of urgency to develop and implement the Nottwil Standard has its roots in the DGMP guidelines (24) for outcome measures and ANQ goal-setting and outcomes reporting criteria (20), which promotes ICF implementation in rehabilitation, and has been building up with the results of the situation analysis (3, 27). Building guiding teams was satisfied with the establishment of a cohesive coalition between the CPT and the EPT and involving all relevant professions. Getting the vision right and communicating for buy-in are related. Clearly communicating the reasons for the Nottwil Standard and regularly communicating the status of the implementation plan were deemed as important and realized through regular meetings and information-sharing with clinical staff. Regular information-sharing and the active involvement in implementing the Nottwil Standard was the opportunity for clinical staff to connect the results of the assessments and the impact on clinical management. The HCP also had to learn that the Nottwil Standard does not hinder individualized management but rather promotes a comprehensive assessment of the patient while simplifying the complexity of the patient's health and functioning. Enabling action was reflected in the support of the clinic's management by investing necessary resources (setting up the CPT and EPT, time for meetings, engaging the IT department) in the project. Creating short-term wins can be seen in the acknowledgment of clinical staff that routine data-based assessments according to the Nottwil Standard was possible. The last two steps (don't let up, making change stick) are ongoing. The Nottwil Standard must undergo continuous improvement based on scientific developments and an evolving clinical and organization environment (7, 53, 54).

Limitations

Some limitations are noteworthy to mention. First, despite efforts to achieve comprehensiveness of the Nottwil Standard by including assessment tools that cover as many SCI-relevant ICF categories as possible, no adequate assessment tools were found for seven categories of the combined ICF Generic-30 Set and ICF Core Set for SCI in post-acute care. Alternatives to established assessment tools for assessing these categories, e.g., single item-questions, will be explored. Second, the Nottwil Standard was developed for implementation in a specific rehabilitation facility. Applying the Nottwil Standard to other hospitals and clinics will require additional testing and possible adaptation. Nevertheless, the principles of continuous improvement can be used in different settings. Lastly, the controlling mechanism nor all the assessment tools had been integrated in the HIS during the pilot study. Thus, the results of the pilot study may have been influenced by this lack of an automated reminder system, as well as the inability to electronically extract relevant data from performed assessments. Respective updates to the HIS are ongoing.

Conclusion

The Nottwil Standard is an ICF-based assessment standard for a comprehensive and accurate assessment of health and functioning of persons after newly acquired SCI. It was developed by an interprofessional group of rehabilitation professionals in a consensus-oriented collaborative process and guided by a set of principles. The Nottwil Standard can be feasibly integrated in routine practice and in the existing HIS. Implementation also requires a well-structured process with a good communication strategy and controlling mechanism, and full engagement of the involved multiprofessional clinical staff. Further development activities include the integration of all the Nottwil Standard assessment tools in the clinic's HIS and deciding on how to assess the SCI-relevant ICF categories that the Nottwil Standard should cover but for which no assessment tool had yet been found. Lastly, since the ultimate aim of the Nottwil Standard is the continuous improvement of rehabilitation quality, an evaluation of impact of the Nottwil Standard on care quality, for example in terms of patient and staff satisfaction, is warranted.

Data Availability Statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Ethics Statement

The ethical committee Ethikkommission Nordwestschweiz (EKNZ Req-2020-01416) approved the study. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

PL and AS-S collected the data and were responsible for the methodological quality. MS, PL, and AS-S drafted the first version of the manuscript and all authors approved the last version and gave feedback to the final manuscript. All authors drafted the study, the development and implementation of the Nottwil standard.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The handling editor declared a past collaboration with several of the authors MS and GS.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

Many thanks to the members of the expanded project team who played an integral role in the success of this project: Markus Berger, Marianne Boller, Irene Buss, Jessica Decker, Inge Eriks Hoogland, Yvonne Häberli, Tobias Jörger, Eva Kissling, Sabrina Koch-Borner, Gunther Landmann, Dirk-Henner Leuth, Romina Miracco, Nuria Morais Leitao, Tobias Pötzel Cordula Ruf, Jürgen Schneider, Stefan Staubli, Adriana Styger, Inge-Marie Velstra, Rajeev Verma, Robert Werder. The authors also thank Pia Scherer for her administrative and coordinative support of the project during the multiproject management phase.

Abbreviations

SCI, Spinal cord injury.

References

1. Swiss Paraplegic Research. Translating Interventions Into Real-life Gains - A Rehab-Cycle Approach. (2021). Available online at: www.icf-casestudies.org.

2. Bickenbach J, Officer A, Shakespeare T, Von Groote PM. International Perspectives on Spinal Cord Injury (IPSCI). Geneva: World Health Organization (2013).

PubMed Abstract | Google Scholar

3. Lampart P, Gemperli A, Baumberger M, Bersch I, Prodinger B, Schmitt K, et al. Administration of assessment instruments during the first rehabilitation of patients with spinal cord injury: a retrospective chart analysis. Spinal Cord. (2018) 56:322–31. doi: 10.1038/s41393-017-0039-x

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Prodinger B, Ballert CS, Brach M, Brinkhof MW, Cieza A, Hug K, et al. Toward standardized reporting for a cohort study on functioning: the swiss spinal cord injury cohort study. J Rehabil Med. (2016) 48:189–96. doi: 10.2340/16501977-2026

PubMed Abstract | CrossRef Full Text | Google Scholar

5. World Health Organization. International Statistical Classification of Diseases and Related Health Problems, 11th Edn (ICD-11). Geneva: World Health Organisation. Available online at: https://www.who.int/classifications/classification-of-diseases (accessed June 3, 2021).

Google Scholar

6. World Health Organization. International Classification of Functioning, Disability and Health. Geneva: World Health Organization (2001).

Google Scholar

7. Stucki G, Bickenbach J. The implementation challenge and the learning health system for sci initiative. Am J Phys Med Rehabil. (2017) 96(2 Suppl 1):S55–60. doi: 10.1097/PHM.0000000000000672

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Stucki G, Prodinger B, Bickenbach J. Four steps to follow when documenting functioning with the international classification of functioning, disability and health. Eur J Phys Rehabil Med. (2017) 53:144–9. doi: 10.23736/S1973-9087.17.04569-5

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Selb M, Cieza A. ICF core sets. In: Bickenbach J, Cieza A, Selb M, Stucki G, editors. ICF Core Sets: Manual for Clinical Practice, 2nd Edn. Göttingen: Hogrefe Publishing (2020). p. 15–22.

10. Selb M, Escorpizo R, Kostanjsek N, Stucki G, Ustun B, Cieza A. A guide on how to develop an international classification of functioning, disability and health core set. Eur J Phys Rehabil Med. (2015) 51:105–17.

PubMed Abstract | Google Scholar

11. Cieza A, Oberhauser C, Bickenbach J, Chatterji S, Stucki G. Towards a minimal generic set of domains of functioning and health. BMC Public Health. (2014) 14:218. doi: 10.1186/1471-2458-14-218

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Prodinger B, Cieza A, Oberhauser C, Bickenbach J, Ustun TB, Chatterji S, et al. Toward the international classification of functioning, disability and health (ICF) rehabilitation set: a minimal generic set of domains for rehabilitation as a health strategy. Arch Phys Med Rehabil. (2016) 97:875–84. doi: 10.1016/j.apmr.2015.12.030

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Kirchberger I, Cieza A, Biering-Sorensen F, Baumberger M, Charlifue S, Post MW, et al. ICF Core Sets for individuals with spinal cord injury in the early post-acute context. Spinal Cord. (2010) 48:297–304. doi: 10.1038/sc.2009.128

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Stucki G, Zampolini M, Juocevicius A, Negrini S, Christodoulou N. Practice, science and governance in interaction: European effort for the system-wide implementation of the international classification of functioning, disability and health (ICF) in Physical and Rehabilitation Medicine. Eur J Phys Rehabil Med. (2017) 53:299–307. doi: 10.23736/S1973-9087.16.04436-1

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Stucki G, Zampolini M, Selb M, Ceravolo MG, Delargy M, Varela Donoso E, et al. European framework of rehabilitation services types: the perspective of the physical and rehabilitation medicine section and board of the european union of medical specialists. Eur J Phys Rehabil Med. (2019) 55:411–7. doi: 10.23736/S1973-9087.19.05728-9

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Selb M, Zampolini M, Delargy M, Kiekens C, Stucki G, Study Group Clinical Assessment S. Specifying clinical assessment schedules for the European framework of rehabilitation service types: the perspective of the physical and rehabilitation medicine section and board of the european union of medical specialists. Eur J Phys Rehabil Med. (2019) 55:834–44. doi: 10.23736/S1973-9087.19.05961-6

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Prodinger B, Scheel-Sailer A, Escorpizo R, Stucki G, moderators UPIW rapporteurs. European initiative for the application of the International classification of functioning, disability and health: development of clinical assessment schedules for specified rehabilitation services. Eur J Phys Rehabil Med. (2017). 53:319–32. doi: 10.23736/S1973-9087.16.04438-5

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Swiss DRG. Online Definition Handbuch SwissDRG 8.0 SwissDRG. (2014). Available online at: https://manual80.swissdrg.org/?locale=de

19. Swiss Federal Statistics Office. Swiss Classification of Operations CHOP. Schweizerische Operationsklassifikation CHOP 2020 Swiss Federal Statistics Office (2020). Available online at: https://www.bfs.admin.ch/bfs/de/home/statistiken/kataloge-datenbanken/publikationen.assetdetail.9286160.html

20. Swiss National Association for Quality Development in Hospitals and Clinics (ANQ). National Measurement Rehabilitation Process Handbook. Bochum; Berlin: Nationaler Messplan Rehabilitation Verfahrenshandbuch (2021). Available online at: https://www.anq.ch/wp-content/uploads/2020/05/ANQ_REHA_Verfahrenshandbuch_2021.pdf

21. Alexander MS, Anderson KD, Biering-Sorensen F, Blight AR, Brannon R, Bryce TN, et al. Outcome measures in spinal cord injury: recent assessments and recommendations for future directions. Spinal Cord. (2009) 47:582–91. doi: 10.1038/sc.2009.18

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Lammertse DP. The spinal cord outcomes partnership endeavor (scope) sci clinical trials tables. Top Spinal Cord Inj Rehabil. (2016) 22:288–315. doi: 10.1310/sci2204-288

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Tomaschek R, Gemperli A, Rupp R, Geng V, Scheel-Sailer A. German-speaking medical sci society (DMGP) ergebniserhebung guideline development group. a systematic review of outcome measures in initial rehabilitation of individuals with newly acquired spinal cord injury: providing evidence for clinical practice guidelines. Eur J Phys Rehabil Med. (2019). 55:605–17. doi: 10.23736/S1973-9087.19.05676-4

PubMed Abstract | CrossRef Full Text | Google Scholar

24. German-speaking Medical Society of Paraplegia (DGMP). Guidelines for Outcome Measures in Inital Rehabilitation After the Onset of a SCI. Ergebniserhebung in der erstbehandlung nach neu erworbener Querschnittlähmung (2021). Available online at: https://www.awmf.org/leitlinien/detail/ll/179-012.html

25. Johnston MV, Graves DE. Towards guidelines for evaluation of measures: an introduction with application to spinal cord injury. J Spinal Cord Med. (2008) 31:13–26. doi: 10.1080/10790268.2008.11753976

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Otto S. Evaluation of Obligation to Cover Health Services Based on the WZW Criteria [German: Beurteilung der Leistungspflicht mithilfe der WZW-Kriterien]. Soziale Sicherheit CHSS. (2018). Available online at: https://soziale-sicherheit-chss.ch/artikel/beurteilung-der-leistungspflicht-mithilfe-der-wzw-kriterien/

27. Scheel-Sailer A, Sailer CO, Lampart P, Baumberger M, Berger M, Mueller G, et al. Examinations and assessments in patients with a newly acquired spinal cord injury - retrospective chart analysis as part of a quality improvement project. Swiss Med Wkly. (2020) 150:w20291. doi: 10.4414/smw.2020.20291

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Turner-Stokes L, Williams H, Sephton K, Rose H, Harris S, Thu A. Engaging the hearts and minds of clinicians in outcome measurement - the UK rehabilitation outcomes collaborative approach. Disabil Rehabil. (2012) 34:1871–9. doi: 10.3109/09638288.2012.670033

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Post MW, Brinkhof MW, von Elm E, Boldt C, Brach M, Fekete C, et al. Design of the swiss spinal cord injury cohort study. Am J Phys Med Rehabil. (2011) 90 (11 Suppl. 2):S5–16. doi: 10.1097/PHM.0b013e318230fd41

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Zariffa J, Curt A, Group ES, Steeves JD. Functional motor preservation below the level of injury in subjects with american spinal injury association impairment scale grade a spinal cord injuries. Arch Phys Med Rehabil. (2012) 93:905–7. doi: 10.1016/j.apmr.2011.10.024

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Fekete C, Gurtner B, Kunz S, Gemperli A, Gmunder HP, Hund-Georgiadis M, et al. Inception cohort of the Swiss Spinal Cord Injury Cohort Study (SwiSCI): Design, participant characteristics, response rates and non-response. J Rehabil Med. (2021) 53:jrm00159. doi: 10.2340/16501977-2795

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Catz A, Itzkovich M. Spinal Cord Independence Measure: comprehensive ability rating scale for the spinal cord lesion patient. J Rehabil Res Dev. (2007) 44:65–8. doi: 10.1682/JRRD.2005.07.0123

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Itzkovich M, Gelernter I, Biering-Sorensen F, Weeks C, Laramee MT, Craven BC, et al. The Spinal Cord Independence Measure (SCIM) version III: reliability and validity in a multi-center international study. Disabil Rehabil. (2007) 29:1926–33. doi: 10.1080/09638280601046302

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Medical Research Council. Aids to Examination of the Peripheral Nervous System. Memorandum no. 45. London: Her Majesty's Stationary Office (1976).

35. ASIA and ISCoS International Standards Committee. The 2019 revision of the international standards for neurological classification of spinal cord injury (ISNCSCI)-What's new? Spinal Cord. (2019). 57:815–7. doi: 10.1038/s41393-019-0350-9

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Muller R, Cieza A, Geyh S. Rasch analysis of the hospital anxiety and depression scale in spinal cord injury. Rehabil Psychol. (2012) 57:214–23. doi: 10.1037/a0029287

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Charlifue S, Post MW, Biering-Sorensen F, Catz A, Dijkers M, Geyh S, et al. International spinal cord injury quality of life basic data set. Spinal Cord. (2012) 50:672–5. doi: 10.1038/sc.2012.27

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Krassioukov A, Biering-Sorensen F, Donovan W, Kennelly M, Kirshblum S, Krogh K, et al. International standards to document remaining autonomic function after spinal cord injury. J Spinal Cord Med. (2012) 35:201–10. doi: 10.1179/1079026812Z.00000000053

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Bryce TN, Richards JS, Bombardier CH, Dijkers MP, Fann JR, Brooks L, et al. Screening for neuropathic pain after spinal cord injury with the spinal cord injury pain instrument (SCIPI): a preliminary validation study. Spinal Cord. (2014) 52:407–12. doi: 10.1038/sc.2014.21

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Dijkers MP, Bryce TN. Introducing the international spinal cord injury pain (ISCIP) Classification. Pain Manag. (2012) 2:311–4. doi: 10.2217/pmt.12.35

PubMed Abstract | CrossRef Full Text | Google Scholar

41. van Hedel HJ, Wirz M, Dietz V. Assessing walking ability in subjects with spinal cord injury: validity and reliability of 3 walking tests. Arch Phys Med Rehabil. (2005) 86:190–6. doi: 10.1016/j.apmr.2004.02.010

PubMed Abstract | CrossRef Full Text | Google Scholar

42. van Hedel HJ, Wirz M, Dietz V. Standardized assessment of walking capacity after spinal cord injury: the European network approach. Neurol Res. (2008) 30:61–73. doi: 10.1179/016164107X230775

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Development of the World Health Organization WHOQOL-BREF quality of life assessment. The WHOQOL Group. Psychol Med. (1998). 28:551–8. doi: 10.1017/S0033291798006667

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Scheuer T, Anneken V, Thietje R, Richarz P, Wirtz M editors. Review of the Measurement Quality of the Aktivitätstests zur Mobilität im Rollstuhl (AMR) With Acquired Paraplegia Based on Rasch Analysis. Überprüfung der Messqualität des Aktivitätstests zur Mobilität im Rollstuhl, Fachhochschule Köln (2010).

45. Rollstuhl (AMR) bei erworbener Paraplegie anhand einer Rasch-Analyse]. Internet Edition - Band 93. 20 Rehabilitationswissenschaftlichen Kolloquium; March 2011 Bochum (DRV-Schriften Band 93). Berlin: Deutsche Rentenversicherung Bund. 92–93.

46. Forschungsinstitut für Inklusion durch Bewegung und Sport (FIBS). AMR® - Der Aktivitätstest zur Mobilität im Rollstuhl FIBS. Available online at: https://www.fi-bs.de/erhebungsinstrumente/amr/

47. Bartolome G, Schröter-Morasch H. Swallowing Disorders. Interdisciplinary Diagnostics and Rehabilitation, 6th Edn, [German: Schluckstörungen. Interdisziplinäre Diagnostik und Rehabilitation. 6. Ausgabe]. Munich: Elsevier, Urban & Fischer (2018).

48. Wiegand NM, Belting J, Fekete C, Gutenbrunner C, Reinhardt JD. All talk, no action?: the global diffusion and clinical implementation of the international classification of functioning, disability, and health. Am J Phys Med Rehabil. (2012) 91:550–60. doi: 10.1097/PHM.0b013e31825597e5

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Spreyermann R, Luthi H, Michel F, Baumberger ME, Wirz M, Mader M. Long-term follow-up of patients with spinal cord injury with a new ICF-based tool. Spinal Cord. (2011) 49:230–5. doi: 10.1038/sc.2010.93

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Biering-Sorensen F, Charlifue S, DeVivo M, Noonan V, Post M, Stripling T, et al. International spinal cord injury data sets. Spinal Cord. (2006) 44:530–4. doi: 10.1038/sj.sc.3101930

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Nasreddine ZS, Phillips NA, Bedirian V, Charbonneau S, Whitehead V, Collin I, et al. The montreal cognitive assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc. (2005) 53:695–9. doi: 10.1111/j.1532-5415.2005.53221.x

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Maritz R, Ehrmann C, Prodinger B, Tennant A, Stucki G. The influence and added value of a standardized assessment and reporting system for functioning outcomes upon national rehabilitation quality reports. Int J Qual Health Care. (2020) 32:379–87. doi: 10.1093/intqhc/mzaa058

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Campbell RJ. Change management in health care. Health Care Manag (Frederick). (2008) 27:23–39. doi: 10.1097/01.HCM.0000285028.79762.a1

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Stucki G, Bickenbach J. Functioning information in the learning health system. Eur J Phys Rehabil Med. (2017) 53:139–43. doi: 10.23736/S1973-9087.17.04612-3

PubMed Abstract | CrossRef Full Text | Google Scholar