Consensus-Based Core Set of Outcome Measures for Clinical Motor Rehabilitation After Stroke—A Delphi Study

Introduction: Outcome measures are key to tailor rehabilitation goals to the stroke patient's individual needs and to monitor poststroke recovery. The large number of available outcome measures leads to high variability in clinical use. Currently, an internationally agreed core set of motor outcome measures for clinical application is lacking. Therefore, the goal was to develop such a set to serve as a quality standard in clinical motor rehabilitation poststroke.

Methods: Outcome measures for the upper and lower extremities, and activities of daily living (ADL)/stroke-specific outcomes were identified and presented to stroke rehabilitation experts in an electronic Delphi study. In round 1, clinical feasibility and relevance of the outcome measures were rated on a 7-point Likert scale. In round 2, those rated at least as “relevant” and “feasible” were ranked within the body functions, activities, and participation domains of the International Classification of Functioning, Disability, and Health (ICF). Furthermore, measurement time points poststroke were indicated. In round 3, answers were reviewed in reference to overall results to reach final consensus.

Results: In total, 119 outcome measures were presented to 33 experts from 18 countries. The recommended core set includes the Fugl–Meyer Motor Assessment and Action Research Arm Test for the upper extremity section; the Fugl–Meyer Motor Assessment, 10-m Walk Test, Timed-Up-and-Go, and Berg Balance Scale for the lower extremity section; and the National Institutes of Health Stroke Scale, and Barthel Index or Functional Independence Measure for the ADL/stroke-specific section. The Stroke Impact Scale was recommended spanning all ICF domains. Recommended measurement time points are days 2 ± 1 and 7; weeks 2, 4, and 12; 6 months poststroke and every following 6th month.

Discussion and Conclusion: Agreement was found upon a set of nine outcome measures for application in clinical motor rehabilitation poststroke, with seven measurement time points following the stages of poststroke recovery. This core set was specifically developed for clinical practice and distinguishes itself from initiatives for stroke rehabilitation research. The next challenge is to implement this clinical core set across the full stroke care continuum with the aim to improve the transparency, comparability, and quality of stroke rehabilitation at a regional, national, and international level.

Introduction

Despite the advances of primary and secondary prevention and the availability of acute medical interventions, stroke remains the second most common cause of disability worldwide (1). Because of an aging population and increasing rates of stroke in younger adults, the number of stroke cases is most likely to increase to 1.5 million cases by the year 2025 (2). In respect to resulting challenges to national health systems and social economy, a European Stroke Action Plan was formulated and includes the domains primary prevention, organization of stroke services, management of acute stroke, secondary prevention, rehabilitation, evaluation of stroke outcome/quality assessment, and life after stroke (3). As motor deficits due to stroke lead to limitations in the performance of activities of daily living (ADL), reduced societal participation, and a lower quality of life (4), outcome measures (OMs) in the motor domain comprise a key role in optimizing and monitoring attainable treatment goals and providing transparency regarding the quality along the stroke care continuum (5). An early and systematic administration of OMs could have multiple benefits for clinicians and patients, such as objective monitoring of the recovery process and the facilitation of goal-oriented interprofessional collaboration, and to support the stroke survivor's education. Currently, a significant number of OMs are available for different clinical settings and stages poststroke (6). Consequently, there is a large variability in clinical use, which hampers transparency and the comparability of motor rehabilitation within and across countries.

Clinical guidelines for evidence-based practice regarding stroke operate on a national level and lack international consensus regarding the use of OMs and, more importantly, the timing of measurements. Despite attempts of implementing the evidence resulting from stroke rehabilitation research into clinical stroke rehabilitation by specific clinical guidelines, the adherence across Europe is often insufficient (7). Standards for OMs to use are not commonly practiced, and the administration of OMs in the field of stroke rehabilitation and other areas is surprisingly low (8). Recently, an international group of researchers systematically reviewed existing clinical guidelines on recommendations for upper extremity assessments and concluded that there is a lack of explicit recommendations on OMs in most of the guidelines (9).

Specifically for research purposes, consensus-based recommendations for sensorimotor measurements in stroke rehabilitation trials were developed by the international Stroke Recovery and Rehabilitation Roundtable (SRRR) to set standards for methodological quality for clinical trials on the body functions and activities domains of the International Classification of Functioning, Disability, and Health (ICF) (10). Also, local and national research groups recommended OMs for stroke research (11–13), including specific interventions such as robotics (14), a single poststroke recovery stage (15), and patient-reported outcomes only (16, 17). Although these efforts are very valuable for stroke rehabilitation research, the recommendations cannot be translated one to one into clinical practice, as the requirements on OMs for clinical use might differ by aspects of the administration time, the number of measurement time points, and the length of follow-up. It is also likely that, for clinical practice, a broader spectrum in terms of impairment and disability levels as well as body sections is relevant, when compared to those covered by various research initiatives. Furthermore, the clinical core set should incorporate the patient's multidomain perspective (18) that was not covered by the SRRR research recommendations, and an international group of clinical stakeholders should be involved.

Therefore, the aim of the present study was to develop an international consensus-based core set of OMs with fixed measurement time points for clinical use in motor rehabilitation after stroke, which is relevant for the full stroke rehabilitation pathway. This set is a key ingredient for transparent stroke rehabilitation and allows alignment between regions and countries with the ultimate goal to improve stroke patients' motor outcomes.

Materials and Methods

Identification of Outcome Measures

An initial collection of sensorimotor OMs was compiled based on an extensive search in relevant systematic reviews (6, 10–27), clinical guidelines (28, 29), and electronic rehabilitation measurement databases [e.g., StrokEngine (30) and Shirley Ryan Ability Lab (31)] by two researchers (JP, JV). The OMs had to meet the following inclusion criteria: (1) assess the motor domain, (2) validated for use in stroke patients, and (3) have a good reliability for the stroke population (intraclass correlation coefficient > 0.7). Eligible OMs were allocated to one of the following three sections: the upper extremity, lower extremity, or ADL/stroke-specific section. The constructs of trunk control, balance, and mobility were assigned to the lower extremity section. The ADL/stroke-specific section included a broader variety of constructs, assessing stroke-specific motor-related functions, activities, and participation. Within each section, OMs were classified according to the ICF domains body structure and function, activities, and participation (18).

Delphi Study Design

A Delphi study design was used to develop the consensus-based core set. The three-round Delphi study was conducted from November 2018 until April 2019. Per section, we aimed to have one OM in each ICF domain that could be applied, regardless of stroke severity. In the lower extremity section, one OM per ICF domain had to be applicable for both patients with and without walking ability.

After each round, each expert received an individualized feedback report with details of the previous round's results in reference to their personal rating. In round 1, each OM was presented with details of the measure's construct, costs, time to administer, and clinimetric properties [validity, reliability, and minimal clinically important difference (MCID)] in line with COSMIN recommendations (Figure 1) (32). For each OM, experts had to rate on a 7-point Likert scale: (1) how familiar they were with that measure, (2) its relevance for clinical practice, and (3) its clinical feasibility. The initial set of OMs was then reduced to those, rated with scores of at least five of seven points for both clinical relevance and clinical feasibility. In round 2, the reduced set of OMs had to be prioritized for each section and ICF domain by assigning ranks in ascending order. As for some lower extremity OMs, patients need to be able to walk; a second measure was allowed if the OM ranked first requires walking ability. For each section, the highest-ranked OM within each ICF domain was included in a preliminary core set for the third round. Additionally, the experts designed a specific measurement scheme, indicating their preferred measurement time points poststroke: Within the acute phase (days 1, 3, and 7), early subacute phase (weeks 2, 4, 6, and 10), the late subacute phase (weeks 12, 16, and 20 and month 6) and for the chronic phase (every 3rd and 6th month following). The minimal agreement rate on measurement timing in round 2 was set to at least 50% ± 2%. In round 3, the experts reviewed the aggregated results presented next to their individual rankings and suggested time points and confirmed their agreement. The cutoff rate for minimal agreement on measurement time points in round 3 was 70 ± 5%.

FIGURE 1

Figure 1. Overview of the Delphi process. ICF, International Classification of Functioning, Disability, and Health; OMs, outcome measures.

A clearance certificate for this study was provided by the cantonal ethics committee Zurich (BASEC Nr. Req-2018-00601). Informed consent of the participating experts was not needed.

Rehabilitation Experts

From September to October 2018, personal enquiries were sent to renowned experts of stroke rehabilitation research and with a networking approach via the following organizations: European Stroke Organization, Council of Occupational Therapists in European Countries, Research in Occupational Therapy and Occupational Science, European Network Occupational Therapy, and the European Network of Physiotherapy in Higher Education. It was our goal to recruit a balanced group of international experts with different clinical backgrounds, including medical doctors, physical therapists, occupational therapists, and rehabilitation engineers. Persons were considered eligible if they had expertise in clinical stroke rehabilitation and clinical research or rehabilitation engineering research and hold at least a master of science degree. The experts were kept ignorant about the other participating experts and received no compensation.

Data Collection and Analysis

The participants received detailed information and instructions on a website with access to the first round's electronic survey created with Research Electronic Data Capture (REDCap, Vanderbilt University Medical Center, USA). Rounds 2 and 3 were carried out via personalized electronic forms, sent and responded via email. Responses were filed by hand (JP) and cross-checked for insertion errors (JH, JV). Feedback of results was given after each round, and equivocal responses were followed up by inquiries via email. Rankings and ratings were analyzed as medians and interquartile ranges. The data were analyzed after each round and presented for the next round. Data analysis and visualization were conducted with Microsoft Office Professional Plus 2016 (Microsoft Cooperation, Redmond, WA, USA).

Results

Participants

Written inquiries yielded 46 eligible experts of whom 33 experts from 18 countries participated and completed the first round of the Delphi study with a response rate of 72% (Table 1). Final agreement was given by 27 experts with three participants lost after the first round and three after the second round.

TABLE 1

Table 1. Participant characteristics.

Development of the Core Set for Clinical Motor Rehabilitation After Stroke

In total, 177 OMs were identified, of which 119 met the inclusion criteria and were presented to the experts. Fifty-nine OMs were rated as being relevant and feasible and were consecutively ranked in round 2. In round 3, final agreement for a core set of nine OMs was given (Figure 2, Table 2).

FIGURE 2

Figure 2. Flowchart of outcome measures per Delphi round, section, and ICF domain. ADL, activities of daily living; ICF, International Classification of Functioning, Disability, and Health; OMs, outcome measures.

TABLE 2

Table 2. Core set of outcome measures for clinical motor rehabilitation after stroke.

The highest ranking in the upper extremity section was given to the Upper Extremity Subscale of the Fugl–Meyer Motor Assessment (FMA-UE) in the body functions domain and the Action Research Arm Test (ARAT) for the activities domain. In the lower extremity section, the two OMs with the highest rankings in the body functions domain were the Fugl–Meyer Motor Assessment Lower Extremity Subscale (FMA-LE) and the 10-m Walk Test (10 MWT); for the activities domain, the Timed-Up-and-Go (TUG) and the Berg Balance Scale (BBS) were indicated. Regarding the ADL/stroke-specific section, the highest ranks were given to the National Institutes of Health Stroke Scale for the body functions domain, and the Barthel Index (BI) or Functional Independent Measure (FIM) for the activities domain. Within all sections, the Stroke Impact Scale (SIS) was prioritized first for the participation domain. However, it should be noted that the SIS also provides the patient's perspective on the body functions and activities domains of the ICF. Subsections of the SIS (hand function, mobility, strength) were presented separately for the upper extremity and lower extremity sections and the whole SIS for the ADL/stroke-specific section. Detailed rankings of the OMs per section and ICF domain and details of the OMs, such as clinimetric properties with references and measurement protocols, can be found in the online supplement (Tables S1–S3, S6–S8, respectively).

Measurement Time Points

In round 2, between three and eight measurement time points per OM within ICF domains were proposed by the experts showing consistent agreement in a range from 48 to 90%. Agreement rates for measurements at 6, 10, 12, and 20 weeks were below 50 ± 2% and were not presented in the last round. Because of expert comments, the first two time points were combined and presented in the final round as one measurement to be administered within the first 3 days. Final agreement was given, with agreement rates ranging from 65.2 to 91.3% for a maximum of seven measurement time points for the upper and lower extremity body functions domain to be taken between days 1 and 3; at day 7; at weeks 2, 4, and 12; at 6 months; and every following 6th month. In the activities domain of the upper and lower extremity sections, agreement of measurement time points followed the same scheme but starting at day 7. A deviating scheme was compiled for the ADL/stroke-specific OMs (Table 3). Final agreement rates can be found in the online supplement (Tables S4, S5).

TABLE 3

Table 3. Measurement time points of the core set for clinical motor rehabilitation after stroke.

Discussion

The goal of this Delphi study was to develop a core set of OMs for clinical motor rehabilitation after stroke as a tool to evaluate the quality of stroke rehabilitation at a local, national, and international level. The consensus-based core set contains nine OMs that cover a wide range of measurement constructs within all ICF domains that are applicable to patients with different stroke severity levels. In addition, a framework with fixed measurement time points was established, following a non-linear pattern (33), with more frequent measurements within the first 3 months after stroke and larger measurement intervals in the chronic phase. The core set was developed on the basis of independent opinions of international experts of different health care professions. All experts have comprehensive experience in clinical stroke rehabilitation. This active involvement of clinical stakeholders ensures the set's clinical relevance, feasibility, and applicability.

Core Set for Clinical Motor Rehabilitation After Stroke

The FMA-UE and ARAT are the selected OMs for the upper extremity section and are in line with the minimal set developed for stroke rehabilitation trials (10). Both instruments have excellent clinimetric properties and therefore demonstrate a high measurement quality for clinical stroke rehabilitation (6, 11). In order to guarantee consistent measurements that allow comparing clinical and research findings, standardized measurement procedures should be followed.

The lower extremity section of the core set covers a large variety of constructs within the spectrum of body functions and activities: motor function, gait speed, functional mobility, and balance in sitting and standing. The OMs are feasible and relevant for stroke patients with and without walking ability. Outcome measures on the activities domain are discriminated by the Functional Ambulation Categories (FAC). Hence, the FAC is not per se included as one of the core set's OMs, but it is a screening tool to determine which OMs should at least be applied. The constructs of motor function (FMA-LE) and balance (BBS) should be evaluated in all patients, whereas walking speed (10 MWT) and functional mobility (TUG) should only be assessed in patients with an FAC score of at least three out of five. Comparing these OMs for the lower extremity with those recommended for stroke rehabilitation research (10) clearly shows that although the constructs of functional balance and mobility were not recommended for research, they are found to be relevant for the clinical setting.

The ADLs/stroke-specific OMs section covers the constructs stroke severity (body functions domain) and basic ADLs (activities domain). Within the activities domain, the highest rank is shared by the BI and FIM, which are highly correlated (r = 0.92–0.99) (34). These OMs can be chosen upon individual considerations within stroke services. The FIM requires annual license fees and provides chargeable access to training materials, offers data services, and contains additional socio-cognitive items. The BI might be favorable regarding time and financial resources (online supplement, Table S8).

The majority of the recommended OMs are designed to objectify the patient's observed functional impairment or to evaluate motor capacity in a standardized test environment, which is defined as the “maximum potential of an individual to succeed in the performance of a motor skill” (35). The included capacity measures are complemented by the patient-reported SIS, which is sensitive to change (36). The SIS not only covers the participation domain of the ICF, but also provides the patient's perspective on the body functions and activities domain. With that, it adds an important multidomain perspective to this clinical core set, a perspective on which no consensus was found for stroke rehabilitation research (10).

The core set's OMs are part of the few clinical guidelines that gave specific recommendations on OMs (9), which potentially facilitates implementation at a national level. The responsibility of clinical assessments should be shared by the involved health care professions according to their specialization. The total time to complete the core set lies between 60 and 75 min, depending on the patient's ability to understand and answer questions or to perform the required tasks of the BI or FIM.

Measurement Time Points

The core set provides a refined framework of fixed measurement time points poststroke, with more frequent measurements early after symptom onset and a low frequent monitoring pattern in the chronic phase. This is in accordance with the logarithmic pattern of sensorimotor recovery after stroke, in which the greatest changes on the body functions and activities domains occur within the first 12 weeks after symptom onset (37, 38). In this period, behavioral restitution takes place, and thereafter, changes occur predominantly due to compensational mechanisms, reaching a plateau between ~3 and 6 months poststroke (33). Low-frequent assessment in the chronic phase allows for monitoring the patient's impairments and disabilities. In case of presence or lack of clinically relevant changes, rehabilitation can be restarted, continued, adapted, or completed (29).

The core set's seven consensus-based measurement time points are in line with existing recommendations in national clinical guidelines (29, 39) and stroke rehabilitation research guidelines (10) to assess in all four recovery phases poststroke (40). However, the experts recommended more measurement time points in the subacute phase, when compared to research recommendations. This will provide more detail about the individual motor recovery pattern across different ICF domains and therefore promote personalized rehabilitation and support appropriate discharge and adaptive planning regarding the home environment. The experts did not select admission and discharge as recommended measurement time points. Although we did not investigate the reason for not selecting specific time points by the experts, we hypothesize that this could be explained by the large international variability in both the length of stay and the accessibility to acute clinics and rehabilitation facilities (41). These arbitrary time points impede the comparability on a regional, national, and international level. As the consensus-based time points are a minimum number of required measurement time points, measurements at admission and discharge could be optionally implemented in the local framework to facilitate rehabilitation goal setting and evaluation.

For clinical practice, an important clinimetric requirement of OMs is their responsiveness to clinically meaningful differences. As it is known that these differences depend on the recovery phase poststroke (42), changes between measurements should be related to the poststroke phase-specific MCID. Although results of anchor-based MCIDs were no inclusion criterion for the preselection of OMs in the Delphi study, MCIDs are available from the acute to the chronic stage for most OMs (online supplement Tables S6–S8).

The experts' consensus resulted in a clear measurement pattern for upper and lower extremity body functions and activities. They proposed a scheme with less measurement time points for the ADL/stroke specific section, possibly because these OMs are not valid and responsive at all time points.

Limitations

There are considerations to be made regarding the developed core set for clinical motor rehabilitation after stroke. First, the availability of validated translations and transcultural validations was no inclusion criterion for OMs. However, with the call for international quality standards in stroke care (3), it should be the interest of stroke services on a regional and national level, to allow for translated and validated versions of the core set's OMs. Second, although we aimed for a well-balanced group of experts in terms of clinical background, occupational therapists were underrepresented. It is unlikely that this influenced the final core set, as there were only marginal variations in the rankings between professions. Regarding the balance by regions of practice, Eastern European countries were underrepresented. Third, there is variability in agreement rates of measurement time points after rounds 2 and 3. However, there was a clear difference in agreement rates between the excluded and final recommended measurement time points. Finally, although most of the experts are still clinically active in stroke rehabilitation, many of them are also involved in research, and they may have ranked OMs using both their clinical and research experiences.

Future Directions

In a next step, the core set for clinical motor rehabilitation after stroke should be implemented across the whole stroke care pathway, including stroke units and acute hospitals, rehabilitation facilities, and outpatient centers or private practices. It should be acknowledged that the implementation of standardized tests in the clinic is challenging. Bland and colleagues (43) demonstrated differences in adherence between settings and professions. Especially in the outpatient facilities, standardized assessments were less frequently applied. However, implementation projects have demonstrated that educational programs and assessment training leads to a successful implementation of stroke OMs in clinical practice (44), and these should be taken as a good example. Routinely scheduled time slots for fixed measurement time points could support time and resource efficiency. A reevaluation of the core set's OMs and the adherence of health care professionals to apply this set should be initiated in 5 years. The measurements' results should be fed to national registries to gain insight into the quality of clinical motor rehabilitation in the acute, subacute, and chronic phase poststroke and provide input for actions for improvement. Last but not least, a collaboration of clinicians and researchers should aim for the development of a minimal set of OMs for other important domains in clinical stroke rehabilitation, such as cognition and speech.

Conclusion

The consensus-based core set of OMs for clinical motor rehabilitation after stroke contains nine OMs that cover the main impairments in body functions, activities, and participation on the motor domain and is complementary to recommendations for stroke rehabilitation research. Measurements should be performed at six time points within the first 6 months poststroke, and consecutive monitoring should take place every 6th month in the chronic stage. The core set and its measurement framework should be implemented throughout the whole stroke care continuum and allows benchmarking, with the long-term goal to optimize the quality of poststroke rehabilitation.

Data Availability Statement

The datasets generated for this study are available on request to the corresponding author.

Ethics Statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements (BASEC Nr. Req-2018-00601). Written informed consent from the participants was not required to participate in this study in accordance with the national legislation and the institutional requirements.

Author Contributions

The idea was conceived by AL, JH, and JV. JV designed the study. JP and JV searched the literature for relevant OMs and designed the electronic survey and forms. JP analyzed the responses, drafted, and edited the manuscript that was critically reviewed by JH, GV, AL, and JV. JH and JV reviewed the data entry and analysis done by JP. MA, SE, AF, TK, GK, and TN also participated in the study. All authors approved the study plan, methods, study design, critically read, and approved the final version.

Funding

This work was financially supported by the P & K Pühringer Foundation.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We would like to thank all experts for their participation: MA, Dr. Guna Berzina, Dr. Iris Brunner, Professor Dr. Jane Burridge, Dr. Lina Butenaite, Dr. David Cioncoloni, SE, Dr. Beatriz Fernandes, AF, Professor Dr. Friedhelm Hummel, MSc. Jip Kamphuis, Dr. Samuel Knobel, Dr. Jan Kool, Professor Dr. Hanne Kaae Kristensen, GK, Professor Dr. Birgitta Langhammer, AL, Professor Dr. Habil Jan Mehrholz, Dr. Illa Mihejeva, Professor Dr. Karel Moses, TN, Professor Dr. Hab Józef Opara, MSc. Beatrice Ottiger, Professor Dr. Valerie Pomeroy, Dr. Pavlin Psychouli, MSc. José López Sánchez, Professor Dr. Blanche Staes, Dr. Esther Steultjens, Professor Dr. Katharina Sunnerhagen, Dr. Ina Tarkka, Dr. Annick van Gils, GV, and Dr. Andreas Winkler. Furthermore, we thank Dr. Shehab Anwer for creating the website and providing technical support.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fneur.2020.00875/full#supplementary-material

References

1. Feigin VL, Nichols E, Alam T, Bannick MS, Beghi E, Blake N, et al. Global, regional, and national burden of neurological disorders, 1990–2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet Neurol. (2019) 18:459–80. doi: 10.1016/S1474-4422(18)30499-X

CrossRef Full Text | Google Scholar

2. Béjot Y, Bailly H, Durier J, Giroud M. Epidemiology of stroke in Europe and trends for the 21st century. La Presse Médicale. (2016) 45:e391–8. doi: 10.1016/j.lpm.2016.10.003

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Norrving B, Barrick J, Davalos A, Dichgans M, Cordonnier C, Guekht A, et al. Action plan for stroke in Europe 2018–2030. Eur Stroke J. (2018) 3:309–36. doi: 10.1177/2396987318808719

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Franceschini M, Porta FL, Agosti M, Massucci M. Is health-related-quality of life of stroke patients influenced by neurological impairments at 1 year after stroke? Eur J Phys Rehabil Med. (2010) 46:11.

PubMed Abstract | Google Scholar

5. Grube MM, Dohle C, Djouchadar D, Rech P, Bienek K, Dietz-Fricke U, et al. Evidence-based quality indicators for stroke rehabilitation. Stroke. (2012) 43:142–6. doi: 10.1161/strokeAHA.111.627679

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Sullivan JE, Crowner BE, Kluding PM, Nichols D, Rose DK, Yoshida R, et al. Outcome measures for individuals with stroke: process and recommendations from the American Physical Therapy Association Neurology Section Task Force. Phys Ther. (2013) 93:1383–96. doi: 10.2522/ptj.20120492

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Legido-Quigley H, Panteli D, Brusamento S, Knai C, Saliba V, Turk E, et al. Clinical guidelines in the European Union: mapping the regulatory basis, development, quality control, implementation and evaluation across member states. Health Policy. (2012) 107:146–56. doi: 10.1016/j.healthpol.2012.08.004

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Haigh R, Tennant A, Biering-Sørensen N, Gimby G, Marinček C, Phillips S, et al. The use of outcome measures in physical medicine and rehabilitation within Europe. J Rehabil Med. (2001) 33:273–8. doi: 10.1080/165019701753236464

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Burridge J, Alt Murphy M, Buurke J, Feys P, Keller T, Klamroth-Marganska V, et al. A systematic review of international clinical guidelines for rehabilitation of people with neurological conditions: what recommendations are made for upper limb assessment? Front Neurol. (2019) 10:567. doi: 10.3389/fneur.2019.00567

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Kwakkel G, Lannin NA, Borschmann K, English C, Ali M, Churilov L, et al. Standardized measurement of sensorimotor recovery in stroke trials: consensus-based core recommendations from the Stroke Recovery and Rehabilitation Roundtable. Int J Stroke. (2017) 12:451–61. doi: 10.1177/1747493017711813

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Alt Murphy M, Resteghini C, Feys P, Lamers I. An overview of systematic reviews on upper extremity outcome measures after stroke. BMC Neurol. (2015) 15:6. doi: 10.1186/s12883-015-0292-6

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Sullivan KJ, Tilson JK, Cen SY, Rose DK, Hershberg J, Correa A, et al. Fugl-Meyer Assessment of sensorimotor function after stroke: standardized training procedure for clinical practice and clinical trials. Stroke. (2011) 42:427–32. doi: 10.1161/STROKEAHA.110.592766

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Moore JL, Potter K, Blankshain K, Kaplan SL, O'Dwyer LC, Sullivan JE. A core set of outcome measures for adults with neurologic conditions undergoing rehabilitation: a clinical practice guideline. J Neurol Phys Ther. (2018) 42:174–220. doi: 10.1097/NPT.0000000000000229

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Baker K, Cano SJ, Playford ED. Outcome measurement in stroke: a scale selection strategy. Stroke. (2011) 42:1787–94. doi: 10.1161/STROKEAHA.110.608505

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Bushnell C, Bettger JP, Cockroft KM, Cramer SC, Edelen MO, Hanley D, et al. Chronic stroke outcome measures for motor function intervention trials: expert panel recommendations. Circ Cardiovasc Qual Outcomes. (2015) 8:S163–9. doi: 10.1161/CIRCOUTCOMES.115.002098

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Reeves M, Lisabeth L, Williams L, Katzan I, Kapral M, Deutsch A, et al. Patient-reported outcome measures (PROMs) for acute stroke: rationale, methods and future directions. Stroke. (2018) 49:1549–56. doi: 10.1161/STROKEAHA.117.018912

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Katzan IL, Thompson NR, Lapin B, Uchino K. Added value of patient-reported outcome measures in stroke clinical practice. J Am Heart Assoc. (2017) 6:5356. doi: 10.1161/JAHA.116.005356

PubMed Abstract | CrossRef Full Text | Google Scholar

18. World Health Organization. International Classification of Functioning, Disability and Health: ICF. Geneva: World Health Organization (2001).

Google Scholar

19. Simpson LA, Eng JJ. Functional Recovery Following Stroke: capturing changes in upper-extremity function. Neurorehabil Neural Repair. (2013) 27:240–50. doi: 10.1177/1545968312461719

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Lang CE, Bland MD, Bailey RR, Schaefer SY, Birkenmeier RL. Assessment of upper extremity impairment, function, and activity after stroke: foundations for clinical decision making. J Hand Ther. (2013) 26:104–15. doi: 10.1016/j.jht.2012.06.005

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Santisteban L, Térémetz M, Bleton J-P, Baron J-C, Maier MA, Lindberg PG. Upper limb outcome measures used in stroke rehabilitation studies: a systematic literature review. PLoS ONE. (2016) 11:e0154792. doi: 10.1371/journal.pone.0154792

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Lemmens RJ, Timmermans AA, Janssen-Potten YJ, Smeets RJ, Seelen HA. Valid and reliable instruments for arm-hand assessment at ICF activity level in persons with hemiplegia: a systematic review. BMC Neurol. (2012) 12:21. doi: 10.1186/1471-2377-12-21

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Ashford S, Brown S, Turner-Stokes L. Systematic review of patient-reported outcome measures for functional performance in the lower limb. J Rehabil Med. (2015) 47:9–17. doi: 10.2340/16501977-1889

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Tyson S, Connell L. How to measure balance in clinical practice. A systematic review of the psychometrics and clinical utility of measures of balance activity for neurological conditions. Clin Rehabil. (2009) 23:824–40. doi: 10.1177/0269215509335018

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Van Bloemendaal M, Van de Water ATM, Van de Port IGL. Walking tests for stroke survivors: a systematic review of their measurement properties. Disabil Rehabil. (2012) 34:2207–21. doi: 10.3109/09638288.2012.680649

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Kelly-Hayes PM, Robertson JT, Broderick JP, Duncan PW, Hershey LA, Roth EJ, et al. The American Heart Association stroke outcome classification. Stroke. (1998) 29:1274–80. doi: 10.1161/01.STR.29.6.1274

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Van Peppen RPS, Hendriks HJM, Van Meeteren NLU, Helders PJM, Kwakkel G. The development of a clinical practice stroke guideline for physiotherapists in The Netherlands: a systematic review of available evidence. Disabil Rehabil. (2007) 29:767–83. doi: 10.1080/09638280600919764

PubMed Abstract | CrossRef Full Text | Google Scholar

28. National Institute of Health and Care Excellence. Stroke Rehabilitation in Adults (2013). Available online at: https://www.nice.org.uk/guidance/cg162

Google Scholar

29. Veerbeek JM, Van Wegen EE, Van Peppen RPS, Hendriks HJM, Rietberg M, Van derWees PhJ, et al. KNGF Clinical Practice Guideline for Physical Therapy in PatientsWith Stroke Practice Guidelines. Amersfoort: KNGF (2014).

Google Scholar

30. Heart and Stroke Foundation. Stroke Engine. Available online at: https://www.strokengine.ca/en/find-assessment/ (accessed October 30, 2018).

Google Scholar

31. Shirley Ryan Ability Lab. Rehabilitation Measure Data Base. Available online at: https://www.sralab.org/rehabilitation-measures/ (accessed October 30, 2018).

Google Scholar

32. Mokkink LB, Prinsen CAC, Bouter LM, Vet HCW, de Terwee CB. The Consensus-based Standards for the Selection of health Measurement Instruments (COSMIN) and how to select an outcome measurement instrument. Braz J Phys Ther. (2016) 20:105–13. doi: 10.1590/bjpt-rbf.2014.0143

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Kwakkel G, Kollen B, Lindeman E. Understanding the pattern of functional recovery after stroke: facts and theories. Restor Neurol Neurosci. (2004) 22:281–99. doi: 10.1017/CBO9780511995590.004

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Hsueh I-P, Lin J-H, Jeng J-S, Hsieh C-L. Comparison of the psychometric characteristics of the Functional Independence Measure, 5-item Barthel Index, and 10-item Barthel Index in patients with stroke. J Neurol Neurosurg Psychiatry. (2002) 73:188–90. doi: 10.1136/jnnp.73.2.188

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Oxford University Press. The Oxford Dictionary of Sports Science and Medicine. 3rd ed. Oxford University Press (2006). doi: 10.1093/acref/9780198568506.001.0001

CrossRef Full Text | Google Scholar

36. Lin K, Fu T, Wu C, Hsieh C. Assessing the Stroke-Specific Quality of Life for outcome measurement in stroke rehabilitation: minimal detectable change and clinically important difference. Health Qual Life Outcomes. (2011) 9:5. doi: 10.1186/1477-7525-9-5

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Persson HC, Opheim A, Lundgren-Nilsson Å, Alt Murphy M, Danielsson A, Sunnerhagen KS. Upper extremity recovery after ischaemic and haemorrhagic stroke: part of the SALGOT study. Eur Stroke J. (2016) 1:310–9. doi: 10.1177/2396987316672809

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Persson CU, Danielsson A, Sunnerhagen KS, Grimby-Ekman A, Hansson P-O. Timed Up and Go as a measure for longitudinal change in mobility after stroke–Postural Stroke Study in Gothenburg (POSTGOT). J NeuroEngineering Rehabil. (2014) 11:83. doi: 10.1186/1743-0003-11-83

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Teasell RW, Foley NC, Bhogal SK, Speechley MR. An Evidence-Based Review of Stroke Rehabilitation. Top Stroke Rehabil. (2003) 10:29–58. doi: 10.1310/8YNA-1YHK-YMHB-XTE1

CrossRef Full Text | Google Scholar

40. Bernhardt J, Hayward KS, Kwakkel G, Ward NS, Wolf SL, Borschmann K, et al. Agreed definitions and a shared vision for new standards in stroke recovery research: the Stroke Recovery and Rehabilitation Roundtable taskforce. Int J Stroke. (2017) 12:444–50. doi: 10.1177/1747493017711816

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Langhorne P, Baylan S. Early supported discharge services for people with acute stroke. Cochrane Database Syst Rev. (2017) doi: 10.1002/14651858.CD000443.pub4

CrossRef Full Text | Google Scholar

42. Lang CE, Edwards DF, Birkenmeier RL, Dromerick AW. Estimating minimal clinically important differences of upper-extremity measures early after stroke. Arch Phys Med Rehabil. (2008) 89:1693–700. doi: 10.1016/j.apmr.2008.02.022

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Bland MD, Sturmoski A, Whitson M, Harris H, Connor LT, Fucetola R, et al. Clinician adherence to a standardized assessment battery across settings and disciplines in a poststroke rehabilitation population. Arch Phys Med Rehabil. (2013) 94:1048–53.e1. doi: 10.1016/j.apmr.2013.02.004

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Van Peppen R, Schuurmans M, Stutterheim E, Lindeman E, Van Meeteren N. Promoting the use of outcome measures by an educational programme for physiotherapists in stroke rehabilitation: a pilot randomized controlled trial. Clin Rehabil. (2009) 23:1005–17. doi: 10.1177/0269215509338999

PubMed Abstract | CrossRef Full Text | Google Scholar