Efficiency and Patient-Reported Outcome Measures From Clinic to Home: The Human Empowerment Aging and Disability Program for Digital-Health Rehabilitation

Background: The recent exponential growth of Digital Health (DH) in the healthcare system provides a crucial transformation in healthcare, answering to alarming threats related to the increasing number of Chronic Neurological Diseases (CNDs). New long-term integrated DH-care approaches, including rehabilitation, are warranted to address these concerns.

Methods: The Human Empowerment Aging and Disability (HEAD) rehabilitation program, a new long-term integrated care including DH-care system, was evaluated in terms of efficiency and patient-reported outcome measures (PROMs) in 107 CND patients (30 with Parkinson's Disease, PD; 32 with Multiple Sclerosis, MS; 45 with stroke in chronic stage). All participants followed 1-month of HEAD rehabilitation in clinic (ClinicHEAD: 12 sessions, 3/week), then 1:3 patient was consecutively allocated to 3-months telerehabilitation at home (HomeHEAD: 60 sessions, 5/week). Efficiency (i.e., adherence, usability, and acceptability) and PROMs (i.e., perceived functioning in real-world) were analyzed.

Results: The rate of adherence to HEAD treatment in clinic (≥90%) and at home (77%) was high. Usability of HEAD system was judged as good (System Usability Scale, median 70.00) in clinic and even more at home (median 80.00). Similarly, administering the Technology Acceptance Model 3 questionnaire we found high scores both in clinic/at home (Usefulness, mean 5.39 ± 1.41 SD/mean 5.33 ± 1.29 SD; Ease of use, mean 5.55 ± 1.05 SD/ mean 5.45 ± 1.17 SD, External Control, mean 4.94 ± 1.17 SD/mean 5.07 ± 1.01 SD, Relevance, mean 5.68 ± 1.29 SD/mean 5.70 ± 1.13 SD and Enjoyment, mean 5.70 ± 1.40 SD/mean 6.01 ± 1.08 SD). After ClinicHEAD, participation and autonomy in daily routine was maintained or even ameliorated (PD and stroke > MS). Whereas, increased functionality and participation in the MS group was found only after HomeHEAD intervention.

Discussion: Our results suggest that a tele-health-based approach is both feasible and efficient in providing rehabilitation care to CNDs from clinic to home. Increasing and maintaining participation as well as autonomy in daily routine are promising findings that open up scenarios for the continuity of care at home through DH-care for CNDs.

Introduction

Parkinson Disease (PD), Multiple Sclerosis (MS), and Stroke are the more frequent chronic Neurological Diseases (CNDs) that can lead to significant motor and cognitive disability: worldwide data report 2.5 million people with MS (1), 7.9 to 19 individuals with PD per 100,000 person-year (2) and 5.5 million deaths due to stroke in 2016 (3). In recent years, new models of digital health (DH) enabling continuity of care are increasingly explored as new solutions to the long-term patient maintenance. Also, growing effort has been spent in the development of technology-enabled treatments, able to be carried out outside clinic setting, with promising results (4–14). Especially, telerehabilitation aids in decreasing socioeconomic costs related to these pathologies and their weight on the healthcare system (15–17). Also, technology-enabled rehabilitation at home allows people with chronic diseases to combine pathology management with their everyday social life (5).

To ensure effectiveness of tele-treatment, a continuous double loop communication between home and clinic environment is needed: in this sense, digital health care platforms constitute the central hub through which health professionals can monitor patient performance at home (18) and consequently modify treatment during the whole period of telerehabilitation. Frequency of rehabilitation and duration of treatment are important parameters that should not be overlooked. In fact, there are health care guidelines for clinical practitioners that detail the frequency and duration of rehabilitation activities specifically for different pathologies, such as MS, PD and stroke (19). For example, strength training, reported as efficacious for MS, PD and stroke patients (20–22) should be performed 2–3 days per week to reach benefits on daily living with a duration per session ranging between 10 and 40 minutes. However, little is known regarding frequency and dose treatment guidelines for treatments administered in a home-based setting.

Unfortunately, adherence to home rehabilitation protocols, including telerehabilitation, is a concern (23). People with neurological disorders that could benefit from rehabilitation often do not adhere to a prescribed protocol once they are in their home environment. This could provide serious consequences, such as loss of functioning, pain, muscle wasting etc., that are risks deriving from a lack of rehabilitation not only in an acute condition, but also in a chronic phase. Few recent studies have investigated the factors affecting adherence in order to predict and enhance adherence to telerehabilitation. An interesting work created a quantitative adherence prediction model based on baseline patients' characteristics by individualizing an important predictive role of education, satisfaction about the treatment and psychological profile (24). Another contribution investigated variation of adherence to treatment comparing different modes of cycling treatment administration, such as active or passive exercises, reporting more satisfying adherence to the passive mode of exercises (25). These contributions demonstrated a pivotal interest on the topic.

Another aspect to be considered regarding new DH approaches is the active role of the patient that is empowered and engaged in own care management, with consequences also on perceived care outcome. In particular, an “e-patients” term has been coined to highlight patients involved in decision-making and management of their own care (26). In fact, patient-reported outcome measures (PROMs) are increasingly used as real-world functioning measures that incorporate self-defined assessments of personal well-being during the management of care (27). Recent clinical practice health guidelines promote the integration of these latter measures into long-term care of patients (28).

The present study aims to report results on efficiency measures and PROMs of the Human Empowerment Aging and Disability program (HEAD), a DH-telerehabilitation system for people with chronic neurological diseases. In particular, we tested HEAD treatment during 1-month of rehabilitation program in clinic and during 3-months of HEAD telerehabilitation at home, comparing patients performances for PD, MS, and chronic stroke.

Materials and Methods

Participants

The study was carried out in two steps: ClinicHEAD and HomeHEAD. In the ClinicHEAD (first step) subjects with PD, MS, and chronic stroke (N = 107) were consecutively recruited. They were identified by the neurologists of the clinics from people that periodically receive neurological follow-up (outpatients) from the respective centers: Valduce Hospital Villa Beretta Rehabilitation Center in Lecco (n = 34; 17 stroke, 7 PD, 10 MS), IRCCS Don Carlo Gnocchi Foundation in Milan (n = 43, 12 stroke, 10 PD, 21 MS) and District Clinic San Camillo in Turin (n = 30, 16 stroke, 13 PD, 1 MS).

Inclusion criteria for enrollment were: [a] age range 18–80; [b] diagnosis of PD in stable treatment for at least 3 years and with a Hoehn and Yahr score ≤ 2 (29), diagnosis of MS without relapses in the last 3-months and with an Expanded Disability Status Scale [EDSS (30)] score ≤ 5.5, diagnosis of stroke in chronic phase, at least 6-months after the acute event.

Exclusion criteria for recruitment were the following: [a] Mini Mental State Examination (31) score < 20; [b] presence of disabling pain; [c] upper limb limited passive range of motion; [d] epilepsy; [e] severe deficit of visual acuity and auditory perception; [f] severe deficit in communication and severe dysmetry.

After enrollment and baseline assessment, they were consecutively assigned into the Clinic and Home HEAD programs.

All participants provided written and informed consent to take part in the study.

The HEAD Program

The HEAD rehabilitation was conceived as a multidimensional program for the continuity of care at home for patients with chronic neurological diseases. It aimed to enhance motor and cognitive abilities such as balance, lower and upper body endurance, strength and speed, memory, executive functions, language and dual-tasking activities, in order to improve patient's everyday functional skills. Procedure and contents of each rehabilitative activity was defined by a health professionals' team including neurologists, physiotherapists and neuropsychologists. Table 1 describes all activities included in HEAD rehabilitation program.

TABLE 1

Table 1. List and brief description of HEAD activities.

The HEAD virtual platform represented the hub for communication between the clinic and the patient's home, allowing rehabilitation activities to be administered through a gaming setting in order to work on goal-directed movements in a virtual reality (VR) scenario. This platform was designed as a bridge between clinic and patient's home setting, making a double loop feedback possible between the two environments. Before every rehabilitation session, physiotherapists and psychologists defined the contents of the session, in the sense of type of activities, repetitions and level of difficulty through the HEAD virtual platform. In this manner, although HEAD technology allowed the same setting for each patient, contents of the rehabilitation program were tailored and personalized according to the different needs related to the pathology of the patient and the level of disability. Patients accessed the platform with their own credentials to start each telerehabilitation session and, health professionals were able to tailor rehabilitation along the whole period of treatment by remotely checking the quality of the gaming performance of the patient reported in the platform.

To run the HEAD program, a PC, internet connection and motor capture devices, such as Kinect (Microsoft, WA, USA) and Leap Motion (Leap Motion Inc., CA, USA), were needed.

The rehabilitation activity was embedded in short video clips. Each video clip lasted from 2 to 9 minutes, and was interrupted between 2 to 6 times on the basis of repetitions of the rehabilitation activity. In general, video clips had three main purposes: as motivating breaks that inter-cut the rehabilitative activities, providing emotional and cognitive stimuli regarding the rehabilitative activities, or awarding the participant at the end of the exercise. Many of the motor and cognitive exercises were directly related to the video. Thus, participants had to erase an image just seen, by means of large movements of the arms in order to continue watching the clip. Alternatively, participants were asked to order the sequences of the film clip just seen, or had to answer questions about the content of the film clip. Patients thus actively controlled their viewing of the movie clips and their progression.

Each activity ended with a feedback of the results, according to an algorithm based on the percentage of completion, number of errors and duration of the performance. The scoring was illustrated by stars, with a minimum of 1 to a maximum of 5 stars being awarded.

During the month of treatment in clinic, the HEAD program was supervised by physiotherapists and neuropsychologists and was administered through 45 min sessions 3 times per week. Once at home, the HEAD program was carried out five consecutive days per week, 30–45 min each session.

Each participant accessed the HEAD portal through his personal credentials to perform his own individual daily program consisting of 3–6 neuromotor activities according to his needs. The participants were free to choose the time of day and weekdays in which to carry out the activities. Finally, participants had the opportunity of calling the Help Desk Service or therapists for technical problems or related issues. A phone call by therapists to participants was planned once a week to check for patient compliance.

Measures

All patients recruited were administered a rehabilitation with VR technology for 1-month in clinic (Time 1: ClinicHEAD). After rehabilitation in clinic, patients were consecutively allocated to HEAD telerehabilitation at home for 3-months (Time 2: HomeHEAD) with a ratio of 1:3. This ratio was due to the limited availability of the HEAD technological kits. For this reason, one patient each three was allocated to continue treatment for 3-months at home. In the second step of the study the participants not allocated to the HomeHEAD group were asked to not participate in physical activities different from those that they would usually do during the protocol duration (control group). Subsequent contributions will report efficacy of HEAD treatment based on outcome measures in each CND included in the study.

Efficiency and Patient-Reported Outcome measures were collected after ClinicHEAD rehabilitation (Time 1) and after 3-months of HomeHEAD treatment by clinicians blind to the treatment allocation (Time 2).

Part of data were obtained through questionnaires administered by a psychologist, while remaining data were extracted from the HEAD platform.

The present work, registered as Clinical Trial ID: NCT03025126, provides preliminary data on efficiency of the HEAD protocol.

Baseline Assessment

Patients recruited were screened with:

(1) Montreal Cognitive Assessment [MoCA (32)] as a measure of global cognitive functioning. Conti's correction was adopted to transform scores on the basis of age and years of education of people. This tool allowed a brief screening of cognitive level evaluating different domains: attention, executive functions, memory, language, visual-spatial abilities, abstraction, calculation, and orientation. Score range is 0–30, with a maximum score of 30.

(2) 2 Minute Walk Test [2MWT (33)] for a quantitative analysis of gait endurance. Participants were instructed to walk as far as possible over 2 minutes and the distance covered was collected.

(3) 10 Meter Walk Test [10MWT (34)] to measure gait speed. Participants were required to walk 10 meters while time was measured. The score was obtained by dividing the distance by the time spent to cover it.

Output Measures

System Usability Scale [Brooke (35)] was administered for a measure of perceived easiness of use of the HEAD system. This is a 10-item, 5 point Likert scale (1 = strongly disagree, 5 = strongly agree). Scoring instructions of Brooke (35) were considered. The final score ranges from 10 to 100. A cut-off score, indicating a satisfying level technological system's usability, is 68. Learnability and usability sub-scores were also obtained in accordance to Lewis and Sauro's indications (36, 37).

Adherence to treatment in clinic and at home was calculated by extracting the following indexes from output of the platform: percentage of total sessions performed, mean number of activities performed per session, mean duration of activities performed per session. We analyzed the same indexes singularly per each week of telerehabilitation at home. Additionally, we analyzed the number of sessions per week performed considering 3 session/week as the recommended frequency of rehabilitation, following Kim's et al. (19) indications. We considered as drop-out participants who followed <50% of treatment period in clinic (<2 weeks of treatment) and at home (<6 weeks of treatment).

Technology Acceptance Model-3 (38) was utilized in order to deepen patients' beliefs related to their inclination to experience the HEAD system. This scale, in fact, specifically explores the perceived ease of use, such as the degree of difficulty that the use of a technology system involves, and the perceived usefulness, as the belief that the use of a specific technology system allows improving one's own productivity. For the purpose of the present study, we focus our analysis only on determined domains of the scale: Perceived usefulness, perceived ease of use, perceptions of external control, perceived enjoyment, Job relevance. Response on a 7-point Likert scale were collected (Totally Disagree = 1/ Totally Agree = 7).

Finally, an ad-hoc questionnaire was created with the purpose to investigate specific barriers patients experienced during rehabilitation at home. This tool was additionally administered to patients recruited in Don Gnocchi Foundation for a deepened investigation. The questionnaire was composed by 11 items with a 5-point Likert scale (0 = Absolutely not/Never, 4 = Very much/Always) (see Table S1 to consult the tool). The questionnaire was scored grouping items into 5 groups in order to obtain 5 total indexes related to crucial aspects to investigate barriers encountered during experience of telerehabilitation. In particular, we analyzed answers extracting the following indexes: (1) motivation (mean score of item 6 and 8) [Did you need support of other persons (ex. your son, bride/wife…) to be motivated to perform HEAD activities (did they remind you to do them?)/ When I didn't perform activities it was because I wouldn't], (2) logistics (mean score of item 1 and 2) [Did having HEAD system at home bother you?”/“How much did you need to modify arrangement of furniture to place HEAD technology devices?”], (3) autonomy (mean score of item 5 and 6) [Did you need support of other persons (ex. Your son, bride/wife…) to prepare HEAD technology setting?/ Did you need support of other persons (ex. your son, bride/wife…) to perform HEAD activities?], (4) inclusion in the routine (mean score of item 3, 4 and 9) [Did you modify your routine to include HEAD activities during the week (ex. Did you eat earlier than usual? Did you stop to have nap after lunch?)/Did you renounce to perform other activities to do HEAD program?/ When I didn't perform activities it was because I couldn't], (5) technical problems (mean score of item 10 and 11) [When I didn't perform activities it was because the system did not work/When I didn't perform activities it was because even if the system worked, the internet connection did not].

Patient-Reported Outcome Measures

Two categories of International Classification of functioning [ICF (39)] on activities and participation in daily life were considered as PROMs: Carrying out daily routine (d230) and Recreation and leisure (d920). These categories were extracted by administering the item 3 of EuroQoL EQ-5D-5L (40–43) and the item 12 of Short Form 12 health survey questionnaire [SF12 (44–47)], respectively. These two items were then translated in the 5 ICF qualifiers. Specifically, we associated ICF qualifier 4 (complete problem) to answer “All the time/Extremely,” qualifier 3 (severe-complete problem) to answer “Most of the time/a good bit of the time/Quite bit,” qualifier 2 (moderate-severe problem) to answer “Some of the time/Moderately,” qualifier 1 (mild-moderate problem) to answer “A little of the time/a little bit” and qualifier 0 (no problem) to answer “Never/Not,” following previous mapping works (44, 46).

Statistical Analysis

Statistical analysis was performed using MedCalc^® Software Version 15.2.1.

Normal distribution of variables was checked through Kolmogorov-Smirnov normality test. According to this test, parametric or non-parametric analysis were performed for the comparison among three pathology groups (PD vs. MS vs. stroke).

To analyze efficiency measures such as adherence to HEAD treatment, usability and acceptance of the HEAD system, descriptive statistics were run in each pathology group. Comparison among groups was also reported through ANCOVA or General Linear Model, by covarying for recruitment center. Bonferroni post-hoc test was considered. To analyze ad-hoc questionnaire results we performed descriptive statistics.

To analyze PROMs, we reported the distribution of qualifiers (percentages) of the ICF categories at the different time points (Enrollment T0, post ClinicHEAD T1, and post HomeHEAD T2). Then, patients were classified into delta score ≤ 0, indicating a perception of maintenance or amelioration over time, and delta score > 0, referring a perception of worsening over time. Percentages of sample reporting delta score ≤ 0, reported as stable/ameliorated patients, and delta score > 0, as worsened patients, were calculated and Chi-square χ² was performed. Results were considered statistically significant when p < 0.05.

Results

Participants

Table 2 shows demographic characteristics of sample included in the study.

TABLE 2

Table 2. Summary statistics and comparison results of patient groups at baseline.

ClinicHEAD: All patients enrolled in the study (n = 107) followed a program of 12 sessions of HEAD rehabilitation in clinic. A total of 107 patients with CNDs was composed of 30 people with PD, 32 with MS and 45 with chronic stroke. The three groups were comparable in terms of gender distribution (χ² = 0.89, p = 0.640) and years of education (F = 0.92, p = 0.401) while there was a statistically significant difference in age between MS and the other two groups (F = 11.29, p < 0.001). In terms of global cognitive level, we found a trend for a lower MoCA score in stroke patients compared to those with MS or PD (F = 3.05, p = 0.051). Finally, endurance and velocity assessed through 2MWT and 10MWT scores were significantly higher in PD than in other patients' groups (2MWT: F = 13.72, p < 0.001; 10MWT: F = 10.55, p < 0.001).

HomeHEAD: Thirty-eight patients were then allocated to HomeHEAD treatment after ClinicHEAD period to test the system for the continuity of care at home. In particular, 11 PD, 14 MS, 13 stroke were assigned to telerehabilitation at home. Similarly to the ClinicHEAD sample, patients groups were comparable for gender distribution (χ² = 0.479, p = 0.112) and education (F = 2.79, p = 0.075), but not for age: we reported a statistically significant difference between PD and MS group (F = 3.73, p = 0.034). The three groups did not differ in MoCA score. Instead, endurance assessed through 2MWT was significant higher in PD than in other patients' groups (F = 7.31, p = 0.003) and velocity assessed through 10MWT was major in PD than stroke (F = 4.48, p = 0.019).

The group enrolled for ClinicHEAD and the sub-group allocated to HomeHEAD did not significantly differ in gender distribution (χ² = 1.42, p = 0.116), age (F = 2.00, p = 0.160), education (F = 2.33, p = 0.130), global cognitive level (F = 0.15, p = 0.698), endurance (F = 0.03, p = 0.868), and velocity (F = 0.33, p = 0.569).

Efficiency Measures Results

In Table 3 we report data on efficiency measures after ClinicHEAD (Time 1) and after HomeHEAD (Time 2), in the three pathologies.

TABLE 3

Table 3. Efficiency measures results of treatment in clinic.

After ClinicHEAD we found a high level of adherence to treatment in all three patients' groups (mean score of all sample: 0.92 ± 0.13) with no significant differences among patients' groups (F = 1.23, p = 0.296). In terms of duration of treatment, as number of activities per session and minutes per session, each session consisted of about 40 minutes multidimensional treatment and was composed of about 4–5 activities. We did not find differences among groups in number of activities (F = 0.40, p = 0.675) and minutes of session (F = 0.90, p = 0.408). We registered only 1 drop out. In terms of perceived usability of the HEAD system in clinic, we reported a good level of usability, with a median SUS score of 70.00 in all patients' groups with no differences among pathologies (F = 0.77, p = 0.679). Finally, considering sub-domains of TAM3, such as perceived system Usefulness, Ease of use, External control, Relevance and Enjoyment, data of all groups supported the high level of functionality of HEAD technology (Usefulness: mean score 5.39 ± 1.41; Ease of Use: mean score 5.55 ± 1.05; External Control: mean score 4.94 ± 1.17), the perceived treatment efficacy (Relevance: mean score 5.68 ± 1.29) and the motivating aspects of HEAD contents (Enjoyment: mean score 5.70 ± 1.40). No differences among patients' groups were registered in all TAM3 subscores (Usefulness: F = 2.82, p = 0.065; Ease of Use: F = 0.28, p = 0.760; External control: F = 0.77, p = 0.466; Relevance: F = 1.37, p = 0.260; Enjoyment: F = 0.47, p = 0.629).

At HomeHEAD (Time 2), we registered 7.89% of drops in the whole group. In general, we reported a discrete adherence to treatment at home (mean score in whole group: 0.77 ± 0.22). More specifically, we observed a better adherence to treatment in PD and stroke groups than in the MS group (F = 6.00, p = 0.007). Focusing on duration of treatment, we did not find group differences in the number of activities performed per session (F = 0.81, p = 0.457) nor in the length of treatment per session (F = 0.72, p = 0.497).

Patient assessment of system usability, as shown by SUS score, was higher in the PD group (median: 85.00) compared to stroke (median: 80.00) and MS (median: 67.50) groups. In general, usability of the system at home was estimated as good (median of whole group: 80.00). No differences among pathologies groups were reported (F = 3.14, p = 0.205).

Results of TAM3 questionnaire highlighted high level of functionality of the system at home in all domains explored (Usefulness: 5.33 ± 1.29; Ease of Use: 5.45 ± 1.17; External Control: 5.07 ± 1.01; Relevance: 5.70 ± 1.13; Enjoyment: 6.01 ± 1.08), suggesting that patients perceived HEAD program as useful, easy to use, intuitive, relevant for everyday life and playful. Again, patients' groups did not differ in TAM3 subscores (Usefulness: F = 1.47, p = 0.254; Ease of Use: F = 2.18, p = 0.139; External Control: F = 0.50, p = 0.613; Relevance: F = 0.50, p = 0.616; Enjoyment: F = 0.97, p = 0.396).

Additionally, adherence to single sessions at home for all period of telerehabilitation (60 sessions, 5 sessions/week) was observed. In particular, we analyzed adherence to sessions in each single week of treatment in each pathology group, considering percentage of adherence to session in each week (1 = adherence to 5 sessions/week; 0 = adherence to 0 sessions/week).

Table 4 reports percentage of adherence to sessions in each week (1 = adherence to 5 sessions/week; 0 = adherence to 0 sessions/week) and the mean number of activities per session in each of 12 weeks of treatment at home.

TABLE 4

Table 4. Total adherence to HEAD telerehabilitation along 3-months of treatment at home.

Based on recent data on recommended frequency of treatment to guarantee rehabilitation effectiveness (19), we considered 0.60 as the ideal adherence per week (3 sessions/5 per week = 1.00). We reported a good adherence (more than 88%) of PD group from the second to the eleventh week of treatment. Stroke group showed an adherence > 80% from the first to the eighth week of treatment, followed by a discrete adherence (75%) in the ninth and tenth week. MS group, instead, demonstrated a discontinuous adherence to treatment, by reported an adherence >85% only from second to forth week. Overall, the whole group presented a good adherence (>82%) from the second to the eighth week of treatment (for details, see Table S2).

Ad-hoc questionnaire on barriers possibly experienced at home reported positive data.

People reported to have encountered very few barriers during their experience of HEAD telerehabilitaiton. In particular, all participants were motivated to carry out rehabilitation as mean score of motivation barriers was 0.03 ± 0.13. Also, from a logistical point of view, the presence of the technological kit in one's home was not perceived as burdensome, as suggested by the low mean score of logistical barriers: 0.31 ± 0.36. Another positive result was that people could perform activities in autonomy: although sometimes preparation of technological setting needed help from the caregiver, performing activities did not. In fact, we collected a mean score of autonomy barriers of 0.31 ± 0.44. Furthermore, HEAD rehabilitation resulted well-integrated into patients' routine: we reported a mean score of 0.40 ± 0.30 at inclusion in the routine barriers. Finally, technological problems represented the only barrier that sometimes impeded the performance of activities: mean score of technological issues was 0.94 ± 0.73.

Patient-Reported Outcome Measures Results

The whole sample was classified as moderate-to-severe in autonomy in daily routine (d230 ICF domain) and as mild-to-moderate in socialization (d920 ICF domain) at baseline.

We report in Table 5 percentages of patients who perceived HEAD treatment as successful in their daily living (daily routine and socialization) and who did not, by reporting a change of level of autonomy and socialization in daily life between baseline and after HEAD treatment in all group that experienced ClinicHEAD (Time 1) (n = 107) and in the sub-group who additionally experienced HomeHEAD (Time 2) (n = 38). We considered as stable or ameliorated patients who presented an ICF score change between time points = 0 or ≤ 1, such as a perceived successful effect of HEAD treatment. On the contrary, patients were considered as worsened when they reported an ICF score change ≥ 1.

TABLE 5

Table 5. Changes in autonomy (d230) and participation (d920) after ClinicHEAD and after HomeHEAD.

We did not find differences for T1-baseline measures between ClinicHEAD and HomeHEAD group (d230: p = 0.476; d920: p = 0.278).

ClinicHEAD: Results showed a high percentage of people with PD (97%) who reported a perceived maintenance or amelioration of functioning in daily life after 12 sessions of HEAD treatment in clinic. Also, a high percentage of patients with stroke judged a positive influence of HEAD treatment on participation in daily living (82%) and a discrete part of the group extended this perception to performance in daily routine (74%). We didn't find similar results in MS group, in which only 59% of patients perceived a successful effect of treatment. In general, a significantly higher number of people who perceived treatment as successful on daily life functioning was registered than people who did not (p < 0.001). This latter result appeared evident in PD and stroke groups, but not in the MS group.

HomeHEAD: We observed satisfying results in the PD group after experience at home. In fact, the entire group (100%) judged a successful effect of HEAD on daily routine and 80% of the sample referred to same perception in daily participation after telerehabilitation. Also the stroke group, which showed positive reported outcome after ClinicHEAD rehabilitation, indicated positive results after rehabilitation at home, with 82% of patients registering a positive effect of telerehabilitation on daily routine and 73% reporting benefits also on participation. Interestingly, the MS group, reported positive results in more than 80% of the group in both daily routine and participation after telerehabilitation at home. In general, we found a significant number of patients reporting positive effect of HEAD treatment on daily functioning (d230: p < 0.001; d920: p = 0.003).

Discussion

We tested efficiency of HEAD, a new technology enabled rehabilitation program for the continuity of care at home for people with CNDs, such as PD, MS, and chronic stroke based on key performance indicators. In particular, we explored HEAD usability and acceptability together with patient's adherence to treatment, and PROMs, as perceived functioning in routine and participation in daily life. We observed output and outcome measures in clinic for a training duration of 1-month (12 sessions) and in continuity of care at home for a total duration of 3-months (60 sessions).

Our results highlighted a very high compliance rate in all three patient groups in clinic, with 92% adherence to treatment. This data is extremely relevant given that Evidence-Based Medicine guidelines reported 80% as minimum rate of adherence needed to appraise quality of clinical trials (42). In particular, we found that stroke patients followed on average 94% of sessions, whereas the numbers were 93 and 90% for PD and MS, respectively. This positive data can be explained by two main factors. First, the ease of use of the technological system can have affected participation to treatment. In fact, from the point of view of usability, our patients judged the system in clinic as efficient. This is particularly important, since the potential effect of DH is strictly related to the perceived ease of use of health care systems (43, 46). Also, investigating perceived acceptability together with usability of the system we reported satisfying feedbacks regarding HEAD technology acceptance in all three pathologies. Especially, facility of use, usefulness of the program, a good control of the system, perception of relevance on their everyday life and enjoyment was observed. Second, the modality of implementation of activities can have influenced motivation to treatment. Especially, HEAD activities were presented in a VR setting and recent evidence has supported the role of VR in influencing outcome by adhering to basic principles of rehabilitation such as intensity, environment, bio-feedback and motivation (47). Moreover, each activity was embedded in a multimedia content, consisting of short video clips, thought to be motivating for patients. Video clips have historically been used to elicit emotion and motivate people, and their dynamic nature appears to be useful in eliciting interest as well as providing an optimal artificial model of reality (48–50).

Having demonstrated the efficiency of the HEAD program in clinic, we focused also on efficiency of the system during telerehabilitation at home for 3-months. Usability and acceptance of HEAD system at home was high in the whole sample, suggesting a good functionality of the system in telerehabilitation. The high score of sub-domain “Enjoyment” of TAM-3 supported the motivating feature of the rehabilitation activities, probably due to variability of the contents included in activities during 3-months of rehabilitation. Moreover, the game-setting of the rehabilitative activities probably played a crucial role in enjoyment and consequently in acceptance of DH-treatment. Accordingly, a recent study demonstrated that game elements affect duration of enjoyment during motor exercises (51). Also, focusing on adherence, <8% of patients did not complete the entire treatment program. This is an important result since continuity of care is crucial in order to do not lose functional recovery after discharge to home (52, 53). However, while the mean adherence to treatment was over 80% in all 3 patients' groups in the first 2-months, it was lower in the last month of the program. This may be due to the high intensity of the home program: the HEAD program was administered 5 sessions/week for 3-months while Kim's et al. (19) work indicated a maximum of 2–3 sessions per week in the MS population. When considering 3 sessions/week as ideal adherence over time, we found a globally longer persistence over telerehabilitation weeks, with, for example, high adherence over 11 weeks/12 in PD group. Moreover, we observed different patterns of adherence in distinct pathologies: with a better adherence in PD and stroke than in MS. This lower adherence to treatment of the MS group might be due to the fatigue that this population experiences during treatment and in daily life (54). It is known that MS patients are faced with elevated challenges when following long-term intervention and there is an urgent need of future research focusing on solutions for continuity of care and exercise persistence in the MS population (55, 56). Moreover, we considered the implication of the video contents included in HEAD activities: movie clips were collected from historically famous movies of years 1940–1990 (obtained by RAI, Italian Radio Television s.p.a.) with the purpose to stimulate positive memories of patients and as such they may have been more targeted to an older audience. The younger age of the MS group compared to the other two pathologies may have contributed to the perception of these contents as not engaging enough. This result stresses the importance of tailoring contents of rehabilitation to age of population targeted, further studies are needed to better elucidate this issue.

A DH approach is not without barriers. Moving the rehabilitation context outside the clinic can result in difficulties for patients to manage technology systems on their own and include treatment in their daily routine. We evaluated the frequency of patients' experience of the most common barriers during 3-months of tele-treatment. Patient feedback suggested that all participants were highly motivated, that they included HEAD program in their routine, did not encounter logistical problems and were able to conduct rehabilitation activities in autonomy. The overall positive judgments support the suitability of DH in a context of global transition of CNDs rehabilitation treatment from inside to outside the clinic (57). The only barriers encountered were technology problems, probably due to the innovative features of the technology system. This is likely an obstacle that will be solved in the near future since technology transformation over time will lead to increasingly more tailored and useable systems.

Moving from efficiency to focus on PROMs enabled us to deepen our understanding of patient perceptions regarding effectiveness of treatment on their functioning in everyday life. Recent works recommended the application of these measures as extremely informative (27), especially in the investigation of effects of newly implemented DH treatments that often are conducted in a home-based context. In the present study a consistent number of subjects referenced a positive effect of treatment particularly on performance of activities in daily life (90%) and socialization (78%). Interestingly, we found a maintenance or amelioration of functioning in daily living especially high after treatment at home, with a greater number of patients reporting positive effect on autonomy in their routine after telerehabilitation at home vs. rehabilitation treatment in clinic (90% of subjects vs. 78%). This evidence is crucial, in accordance with suggestion of Steinhubl et al. (58), who, reporting definitions and scenarios fostered by the new mobile health technologies, declare that changes in the care environment are able to provide better outcomes. More importantly, this result is in line with the main goal of rehabilitation itself, that aims at the recovery of the patient's functioning in terms of its utilization in daily living; all in the overall context of the biopsychosocial digital model that foresees a collaboration between clinicians and patient facilitated by DH in the social context of the person (9). Considering socialization in daily life, we found different outcome in the three pathologies. There was a better perception of socialization after the in-clinic HEAD program than following the home telerehabilitation program. On the contrary, MS patients showed an increase in socialization after telerehabilitation at home compared to the clinic. This result may be related to the phenomenon of the baby boomer generation: people born between 1946 and 1964 need to be introduced to technology. Instead, younger people, nearer to millennials, are more familiar with technology systems. Hence, our results could be explained by demographic factors of our population targets: patients with an older age, such as those with PD and stroke, appreciated HEAD potential benefit on socialization more in clinic setting than at home, while younger people, such as the MS sample in our study, are able to report treatment potentiality on their daily life when it is fostered in their home setting.

Targeting three pathologies, MS, PD, and Stroke in this study leads to smaller sample size for each group which could constitute a limitation. However, one of the hallmarks of the proposed study is its adaptability to different functional levels and its suitability for different neurological disorders. In fact, one of the problems other studies have faced in the past is the narrow applicability of their intervention systems (59). The HEAD protocol is developed to overcome this problem. This study proposes to investigate the responsiveness of the tested intervention to the needs of diverse populations with chronic neurological pathologies that are typically seen in rehabilitative practices and need to have access to monitored continuity of rehabilitative activities.

While the sample size of this study is too small to draw conclusive evidence of the efficacy of the proposed intervention, the results support HEAD as a useful and acceptable DH-care system for people with CNDs with positive impacts on the perceived benefits for autonomy and daily life involvement. This is important given the crucial role technology will play in future neurorehabilitation models. Our findings support the notion that intensity and duration of long-term intervention must be tailored to the individual, taking into account also the personalization of contents to maintain an adequate level of engagement in rehabilitation at home.

Data Availability Statement

The datasets generated for this study are available on request to the corresponding author.

Ethics Statement

The studies involving human participants were reviewed and approved by the Ethics Committees of IRCCS Don Gnocchi Foundation, of inter-company of the province of Lecco, Como and Sondrio, and of inter-company “Città della Salute e della scienza” of Turin. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

FB, FM, and MS conceived the study. CG, JJ, and PG carried out the study. CC, CP, and GP collected data. CP, SI, and FB performed statistical analysis and interpreted results. FB, JJ, and SI wrote the manuscript. All authors reviewed and approved the final manuscript.

Funding

This study had been funded by the Fondazione Cariplo, Italy. This funding source had no role in the design of this study and will not have any role during its execution, analyses, interpretation of the data, or decision to submit results.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We would like to thank all patients who participated to the study. We would also thank the HEAD study group to make possible to conduct the present study (alphabetic order): Aggujaro S, Barra G, Bellomo M, Bertoni R, Boccini S, Bonanima M, Borgogno P, Bowman T, Canobbio S, Castagna A, Covarrubias M, Del Principe A, Di Tella S, Enei L, Ferrari A, Ferrarin M, Fini M, Gencarelli N, Giordano A, Manfredini C, Marino C, Martina L, Mendozzi L, Mocarelli P, Montesano A, Nemni R, Perini G, Peverelli M, Proserpio D, Pugnetti L, Ripamonti E, Rossini M, Ruffin G, Saibene FL, Trombini D, Zanfini A.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fneur.2019.01206/full#supplementary-material

References

1. Bisson EJ, Finlayson ML, Ekuma O, Marrie RA, Leslie WD. Accuracy of FRAX^® in people with multiple sclerosis. J Bone Miner Res. (2019) 34:1095–100. doi: 10.1002/jbmr.3682

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Muangpaisan W, Mathews A, Hori H, Seidel D. A systematic review of the worldwide prevalence and incidence of Parkinson's disease. J Med Assoc. (2011) 94:749–55.

PubMed Abstract | Google Scholar

3. Gorelick PB. The global burden of stroke: persistent and disabling. Lancet Neurol. (2019) 18:417–8. doi: 10.1016/S1474-4422(19)30030-4

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Bloem BR, Munneke M. Revolutionising management of chronic disease: the Parkinson Net approach. BMJ. (2014) 348:g1838. doi: 10.1136/bmj.g1838

CrossRef Full Text | Google Scholar

5. Di Tella S, Pagliari C, Blasi V, Mendozzi L, Rovaris M, Baglio F. Integrated telerehabilitation approach in multiple sclerosis: a systematic review and meta-analysis. J Telemed Telecare. (2019). doi: 10.1177/1357633X19850381. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Realdon O, Rossetto F, Nalin M, Baroni I, Cabinio M, Fioravanti R, et al. Technology-enhanced multi-domain at home continuum of care program with respect to usual care for people with cognitive impairment: the Ability-TelerehABILITation study protocol for a randomized controlled trial. BMC Psychiatry. (2016) 16:425. doi: 10.1186/s12888-016-1132-y

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Greenlaw R, Rocchi L, Chiari L, Farella E. Tele-rehabilitation system based on augmented feedback for people with Parkinson's disease: design principles. In: Hu F, editor. Telehealthcare Computing and Engineering: Principles and Design. CRC Press (2013). p. 99–114. doi: 10.1201/b14770.

CrossRef Full Text | Google Scholar

8. Engel GL. The clinical application of the biopsychosocial model. J Med Philos. (1981) 6:101–23.

PubMed Abstract | Google Scholar

9. Ahmadvand A, Gatchel R, Brownstein J, Nissen L. The biopsychosocial-digital approach to health and disease: call for a paradigm expansion. J Med Internet Res. (2018) 20:e189. doi: 10.2196/jmir.9732

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Bloem BR, Rompen L, Vries NM, Klink A, Munneke M, Jeurissen P. ParkinsonNet: a low-cost health care innovation with a systems approach from the Netherlands. Health Aff. (2017) 36:1987–96. doi: 10.1377/hlthaff.2017.0832

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Chen Y, Abel KT, Janecek JT, Chen Y, Zheng K, Cramer SC. Home-based technologies for stroke rehabilitation: a systematic review. Int J Med Inform. (2019) 123:11–22. doi: 10.1016/j.ijmedinf.2018.12.001

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Motl RW, Backus D, Neal WN, Cutter G, Palmer L, McBurney R, et al. Rationale and design of the STEP for MS Trial: comparative effectiveness of Supervised versus Telerehabilitation Exercise Programs for Multiple Sclerosis. Contemp Clin Trials. (2019) 81:110–22. doi: 10.1016/j.cct.2019.04.013

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Kahraman T, Savci S, Ozdogar AT, Gedik Z, Idiman E. Physical, cognitive and psychosocial effects of telerehabilitation-based motor imagery training in people with multiple sclerosis: a randomized controlled pilot trial. J Telemed Telecare. (2019). doi: 10.1177/1357633X18822355 [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Rogerson L, Burr J, Tyson S. The feasibility and acceptability of smart home technology using the Howz system for people with stroke. Disabil Rehabil Assist Technol. (2019) 1–5. doi: 10.1080/17483107.2018.1541103

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Chen JJ. Parkinson's disease: health-related quality of life, economic cost, and implications of early treatment. Am J Manag Care. (2010) 16:S87–93.

PubMed Abstract | Google Scholar

16. Moccia M, Palladino R, Lanzillo R, Triassi M, Brescia Morra V. Predictors of the 10-year direct costs for treating multiple sclerosis. Acta Neurol Scand. (2016) 135:522–8. doi: 10.1111/ane.12630

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Payne KA, Huybrechts KF, Caro JJ, Craig Green TJ, Klittich WS. Long term cost-of-illness in stroke: an international review. Pharmacoeconomics. (2002) 20:813–25. doi: 10.2165/00019053-200220120-00002

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Chau JPC, Lo SHS, Lee VWY, Choi KC, Shum EWC, Hung ZSS, et al. Effectiveness and cost-effectiveness of a virtual multidisciplinary stroke care clinic for community-dwelling stroke survivors and caregivers: a randomised controlled trial protocol. BMJ Open. (2019) 9:e026500. doi: 10.1136/bmjopen-2018-026500

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Kim Y, Lai B, Mehta T, Thirumalai M, Padalabalanarayanan S, Rimmer JH, et al. Exercise training guidelines for multiple sclerosis, stroke, and Parkinson's disease: rapid review and synthesis. Am J Phys Med Rehabil. (2019) 38:613–21. doi: 10.1097/PHM.0000000000001174

CrossRef Full Text | Google Scholar

20. Han P, Zhang W, Kang L, Ma Y, Fu L, Jia L, et al. Clinical evidence of exercise benefits for stroke. Adv Exp Med Biol. (2017) 1000:131–51. doi: 10.1007/978-981-10-4304-8_9

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Kjølhede T, Vissing K, de Place L, Pedersen BG, Ringgaard S, Stenager E, et al. Neuromuscular adaptations to long-term progressive resistance training translates to improved functional capacity for people with multiple sclerosis and is maintained at follow-up. Mult Scler. (2015) 21:599–611. doi: 10.1177/1352458514549402

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Mehrholz J, Kugler J, Storch A, Pohl M, Hirsch K, Elsner B. Treadmill training for patients with Parkinson's disease. Cochrane Database Syst Rev. (2015) 9:CD007830. doi: 10.1002/14651858.CD007830.pub3

CrossRef Full Text | Google Scholar

23. Gutenbrunner C, Chamberlain AC, Ward TB. White book on physical and rehabilitation medicine in Europe. Eur J Phys Rehabil Med. (2006). 42:295. doi: 10.2340/16501977-0028

CrossRef Full Text | Google Scholar

24. Jeong IC, Liu J, Finkelstein J. Factors affecting adherence with telerehabilitation in patients with multiple sclerosis. Stud Health Technol Inform. (2019) 257:189–93. doi: 10.3233/978-1-61499-951-5-189

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Jeong IC, Finkelstein J. Remotely controlled biking is associated with improved adherence to prescribed cycling speed. Technol Health Care. (2015) 623(Suppl 2):S543–9. doi: 10.3233/THC-150992

CrossRef Full Text | Google Scholar

26. Meskó B, Drobni Z, Bényei É, Gergely B, Gyorffy Z. Digital health is a cultural transformation of traditional healthcare. Mhealth. (2017) 3:38. doi: 10.21037/mhealth.2017.08.07

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Chan EKH, Edwards TC, Haywood K, Mikles SP, Newton L. Implementing patient-reported outcome measures in clinical practice: a companion guide to the ISOQOL user's guide. Qual Life Res. (2019) 28: 621–7. doi: 10.1007/s11136-018-2048-4

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Foster A, Croot L, Brazier J, Harris J, O'Cathain A. The facilitators and barriers to implementing patient reported outcome measures in organisations delivering health related services: a systematic review of reviews. J Patient Rep Outcomes. (2018) 2:46. doi: 10.1186/s41687-018-0072-3

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Zhao YJ, Wee HL, Chan YH, Seah SH, Au WL, Lau PN, et al. Progression of Parkinson's disease as evaluated by Hoehn and Yahr stage transition times. Mov Disord. (2010) 25:710–6. doi: 10.1002/mds.22875

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Kurtzke JF. Rating neurologic impairment in multiple sclerosis: an expanded disability status scale (EDSS). Neurology. (1983) 33:1444–52. doi: 10.1212/WNL.33.11.1444

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Folstein MF, Robins LN, Helzer JE. The mini-mental state examination. Arch Gen Psychiatry. (1983) 40:812. doi: 10.1001/archpsyc.1983.01790060110016

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Conti S, Bonazzi S, Laiacona M, Masina M, Coralli MV. Montreal Cognitive Assessment (MoCA)-Italian version: regression based norms and equivalent scores. Neurol Sci. (2015) 36:209–14. doi: 10.1007/s10072-014-1921-3

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Butland RJ, Pang J, Gross ER, Woodcock A, Geddes DM. Two-, six-, and 12-minute walking tests in respiratory disease. Br Med J. (1982) 284:1607–8 doi: 10.1136/bmj.284.6329.1607

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Bohannon RW. Comfortable and maximum walking speed of adults aged 20-79 years: reference values and determinants. Age Ageing. (1997) 26:15–9. doi: 10.1093/ageing/26.1.15

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Brooke J. SUS-A quick and dirty usability scale. Usabil Eval Industry. (1996) 189:4–7.

Google Scholar

36. Borsci S, Federici S, Lauriola M. On the dimensionality of the System Usability Scale: a test of alternative measurement models. Cogn Process. (2009) 10:193–7. doi: 10.1007/s10339-009-0268-9

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Lewis JR, Sauro J. The factor structure of the system usability scale. In: Proceedings of the Human Computer Interaction International Conference. Berlin: Springer (2009). p. 94–103.

Google Scholar

38. Venkatesh V, Bala H. Technology acceptance model 3 and a research agenda on interventions. Decis Sci. (2008) 39:273–315. doi: 10.1111/j.1540-5915.2008.00192.x

CrossRef Full Text | Google Scholar

39. Cieza A, Oberhauser C, Bickenbach J, Chatterji S, Stucki G. Towards a minimal generic set of domains of functioning and health. BMC Public Health. (2014) 14:218. doi: 10.1186/1471-2458-14-218

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Haywood KL, Griffin XL, Achten J, Costa ML. Developing a core outcome set for hip fracture trials. Bone Joint J. (2014) 96-B:1016–23. doi: 10.1302/0301-620X.96B8.33766

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Herdman M, Gudex C, Lloyd A, Janssen M, Kind P, Parkin D, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res. (2011) 20:1727–36. doi: 10.1007/s11136-011-9903-x

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Centre for Evidence-Based Medicine. Randomized Controlled Trails Critical Appraisal Sheet. Critical Appraisal Tools (2005). Available online at: http://www.cebm.net/critical-appraisal/ (accessed December 16, 2015).

Google Scholar

43. Savazzi F, Isernia S, Jonsdottir J, Di Tella S, Pazzi S, Baglio F. Design and implementation of a Serious Game on neurorehabilitation: data on modifications of functionalities along implementation releases. Data Brief. (2018) 20:864–9. doi: 10.1016/j.dib.2018.08.100

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Mayo NE, Poissant L, Ahmed S, Finch L, Higgins J, Salbach NM, et al. Incorporating the International Classification of Functioning, Disability, and Health (ICF) into an electronic health record to create indicators of function: proof of concept using the SF-12. J Am Med Inform Assoc. (2004) 11:514–22. doi: 10.1197/jamia.M1462

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Ware J, Kasinski M, Keller SD. A 12-item short-form health survey: construction of scales and preliminary tests of reliability and validity. Med Care. (1996) 34:220–33. doi: 10.1097/00005650-199603000-00003

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Sousa VEC, Dunn Lopez K. Towards usable e-health. A systematic review of usability questionnaires. Appl Clin Inform. (2017) 8:470–90. doi: 10.4338/ACI-2016-10-R-0170

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Dias P, Silva R, Amorim P, Lains J, Roque E, Pereira ISF, et al. Using virtual reality to increase motivation in poststroke rehabilitation. IEEE Comput Graph Appl. (2019) 39:64–70. doi: 10.1109/MCG.2018.2875630

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Coan JA, Allen JJ. Handbook of Emotion Elicitation and Assessment. New York, NY: Oxford University Press (2007).

Google Scholar

49. Rottenberg J, Ray RD, Gross JJ. Emotion elicitation using films. In: Coan J, Allen JJB, editors. A Handbook of Emotion Elicitation and Assessment. London (2007). p. 9–28.

Google Scholar

50. Schaefer A, Nils FF, Sanchez X, Philippot P. Assessing the effectiveness of a large database of emotion eliciting films: a new tool for emotion researchers. Cogn Emot. (2010) 24:1153–72. doi: 10.1080/02699930903274322

CrossRef Full Text | Google Scholar

51. van der Kooij K, van Dijsseldonk R, van Veen M, Steenbrink F, de Weerd C, Overvliet KE. Gamification as a sustainable source of enjoyment during balance and gait exercises. Front Psychol. (2019) 10:294. doi: 10.3389/fpsyg.2019.00294

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Brennan DM, Mawson S, Brownsell S. Telerehabilitation: enabling the remote delivery of healthcare, rehabilitation, and self management. Stud Health Technol Inform. (2009) 145:231–48. doi: 10.3233/978-1-60750-018-6-231

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Hailey D, Roine R, Ohinmaa A, Dennett L. Evidence of benefit from telerehabilitation in routine care: a systematic review. J Telemed Telecare. (2011) 17: 281–7. doi: 10.1258/jtt.2011.101208

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Heesen C, Bruce J, Gearing R, Moss-Morris R, Weinmann J, Hamalainen P, et al. Adherence to behavioural interventions in multiple sclerosis: follow-up meeting report (AD@MS-2). Mult Scler J Exp Transl Clin. (2015) 1:1–4. doi: 10.1177/2055217315585333

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Heesen C, Bruce J, Feys P, Sastre-Garriga J, Solari A, Eliasson L, et al. Adherence in multiple sclerosis (ADAMS): classification, relevance, and research needs. A meeting report. Mult Scler. (2014) 20:1795–8. doi: 10.1177/1352458514531348

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Rottoli M, La Gioia S, Frigeni B, Barcella V. Pathophysiology, assessment and management of multiple sclerosis fatigue: an update. Expert Rev Neurother. (2017) 17:373–9. doi: 10.1080/14737175.2017.1247695

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Galea MD. Telemedicine in rehabilitation. Phys Med Rehabil Clin N Am. (2019) 30:473–83. doi: 10.1016/j.pmr.2018.12.002

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Steinhubl SR, Muse ED, Topol EJ. Can mobile health technologies transform health care? JAMA. (2013) 310:2395–6. doi: 10.1001/jama.2013.281078

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Gitlin LN. Introducing a new intervention: an overview of research phases and common challenges. Am J Occup Ther. (2013) 67:177–84. doi: 10.5014/ajot.2013.006742

PubMed Abstract | CrossRef Full Text | Google Scholar