2Vita-B Physical: An Intelligent Home Rehabilitation System Based on Microsoft Azure Kinect

Active rehabilitation is an exercise-based program designed to improve the level of function of people with motor disabilities. The effectiveness of such programs is strongly influenced by the correctness of the exercise execution. An exercise done incorrectly could even lead to a worsening of the health status. For this reason, specialists are required to guide the patient during the execution of an exercise. The drastic reduction of the costs of motion tracking systems has paved the way to the implementation of virtual assistant software able to automatically assess the correctness of an exercise. In this paper 2Vita-B Physical is presented, a rehabilitation software system properly designed to support both 1) the patients, by guiding them in the correct execution of an exercise; and 2) the physiotherapists, by allowing them to remotely check the progress of a patient. The motion capturing in 2Vita-B is performed by using the recently released Microsoft Kinect Azure DK. Thus, the system is easy to use and completely non-invasive. Besides the hardware and software requirements of the system, the results of a preliminary usability evaluation of the system conducted with 29 users is also reported. The results achieved are promising and provide evidence of the high usability of 2Vita-B Physical as home rehabilitation system.

1 Introduction

Physical rehabilitation represents the third pillar of the health care system, next to prevention and treatment. The goal of physical rehabilitation is—in general—the full recovery of people with temporary motor impairments. In the case of permanent disorder, physical rehabilitation aims to mitigate the suffering to offer patients a better level of independence.

Over the years, the computer science research community devoted a lot of effort to physical rehabilitation, with a particular focus on home-based rehabilitation. Home rehabilitation allows the patient to undergo treatment without the need to reach a specialized facility on a daily or weekly basis. Also, to minimize the inconvenience and cost of commuting, with home rehabilitation, the patient is allowed to perform rehabilitation activities in comfortable conditions offered by their own home. All this contributes on the one hand to improve the quality of treatment and speed up recovery (Maclean et al., 2002) and on the other hand, reduces hospitalizations and consequently, the costs of the health care system (Han et al., 2005).

Technology in the field of rehabilitation has developed dramatically in recent years (Han et al., 2005). Sensors in healthcare are becoming more and more widespread and their ability to detect even the smallest movements is making them increasingly attractive for the development of innovative solutions. Indeed, these sensors are not only able to measure and monitor vital parameters on a large scale, but also to identify abnormal situations that may represent potential problems (Fern’ndez-Baena et al., 2012). Motion tracking solutions in the field of rehabilitation can provide objective feedback on how a patient’s body is positioned in space. However, motion tracking is traditionally performed through a photogrammetric system (referred to as MoCap), i.e., a system of multiple cameras, which are also emitters of light (which can be red, infrared, or near-infrared), and markers (small spheres) of reflective material (Chang et al., 2013). Such systems, although very reliable, are somewhat expensive, require specific configuration and placement of 3D markers. All these constraints make MoCaps impractical for everyday clinical use (Lange et al., 2011).

The commercialization of the Microsoft Kinect has given a boost to research in this field. The Kinect has a low cost, is portable and allows natural interaction with the patient. Also, scientific studies have shown that the Kinect has adequate accuracy for clinical application (Maclean et al., 2002; Da Gama et al., 2019). This has prompted researchers to design rehabilitation systems that are more accessible to clinical settings and, more importantly, to patients’ home.

This paper describes the 2Vita-B Physical project that, following the aforementioned line of research, aims at implementing a virtual assistant able to provide indications on the correctness and effectiveness of the activity carried out by the patient in the absence of the therapist. The correctness of activity is evaluated through a comparison of the movement performed by the patient with an “ideal” (i.e., correct) movement performed by a physiotherapist. The effectiveness of the rehabilitation activity carried out, instead, is estimated through the analysis of heart rate during the performance and indices of stress properly defined. The system has been designed to be minimally invasive. For this reason, motion tracking is performed by using the Microsoft Azure Kinect, while a wearable device is exploited for the acquisition of vital parameters.

The system also promotes patient motivation by including elements of gamification in the activities. The evaluations obtained by a patient during the execution of a rehabilitation exercise are compared with the results obtained by other patients to create a constructive competition between the patients during their daily (and often boring) rehabilitation activities.

It is worth noting that 2Vita-B Physical is not intended to replace the therapist. Every decision regarding the therapy is always confirmed by the therapist. For this reason, 2Vita-B Physical provides to the therapist both the objective evaluation and the recording of the patient’s performance to allow the latter to corroborate or refute the decision of the system.

A preliminary evaluation of 2Vita-B Physical through a usability study is also performed. The study was conducted with 29 participants who were requested to conduct five rehabilitation exercises. In the end, they were asked to fill in a questionnaire. With the data collected, we followed state-of-art guidelines to provide a final grade of usability for the system, according to responses received. The results showed an overall appreciation of the system.

The rest of the paper is structured as follows. Section 2 describes in the 2Vita-B Physical system, with particular focus on the Rehab Station. Section 3 provides the details on the study conducted to assess the usability of the system. Finally, Section 7 concludes the paper.

2 2 Vita-B Physical Overview

This section provides an overview of the 2Vita-B Physical system reporting in detail all the end-user functionalities and the whole architecture of the system. Finally, a detailed description of the main component, i.e., the rehabilitation station, is reported.

2.1 End-User Functionalities

Figure 1 summarises through a UML use case diagram the actors and the main functionalities of the 2Vita-B Physical system:

• User: the abstract actor that represents a generic user of the system with appropriate credentials for access to system functionality. She can access the system and manage their account.

• Administrator: the person responsible for the administrative management of the 2Vita-B Physical system. She deals with the management of 2Vita-B system users, manages the archive of rehabilitative exercises and initiates the process of enrollment of patients.

• Patient: the person undergoing rehabilitation treatment. She can access their rehabilitation plan to assess their progress and perform a specific rehabilitation exercise.

• Rehabilitation Team Member: abstract user representing a generic unit of personnel (e.g., physiatrist) which is part of the rehabilitation team. She has the possibility of viewing the individual rehabilitation project of a specific patient.

• Physiatrist: physician specialized in physical and rehabilitative medicine, with the knowledge necessary for the prescription and correct rehabilitative therapy. She has the task of defining the individual rehabilitation project, updating the project through appropriate follow-ups and concluding the individual rehabilitation project of the patient.

• Psychologist: a health professional who carries out prevention, diagnosis, intervention, health promotion, habilitation-rehabilitation, support and counselling in the psychological field. She has the task of entering periodic reports on the analysis of the level of awareness of the patient concerning his health problem and behavioural strategies (coping) that is being implemented.

• Physiotherapist: a health professional who develops and implements interventions aimed at prevention, treatment, care and rehabilitation of diseases or dysfunctions in the areas of motor, upper cortical and visceral functions. She has the task of defining the rehabilitation plan, to evaluate the rehabilitation exercises performed by a specific patient and to define the ideal (i.e., correct) movement for a specific rehabilitation exercise.

FIGURE 1

FIGURE 1. 2Vita-B Physical: End-users functionalities.

The rehabilitation process of a patient in the 2Vita-B Physical system starts with the enrollment of the patient. Especially, the system allows the Administrator to enter the personal data of the new patient. Subsequently, the Physiatrist defines the individual rehabilitation program of the patient specifying the name of the program, the date of assignment, the patient’s medical history, the diagnosis, the comorbidity (optional), the type of operation undergone by the patient (optional), the summary and the objectives of the rehabilitation program and the timing of the program. The system allows the Physiatrist to enter the patient’s functional evaluation according to a specific evaluation standard (e.g., International Classification of Functioning, Disability and Health) by specifying the evaluation standard, the category, the attribute and the corresponding evaluation. Furthermore, the system allows the Physiatrist to store information 1) on the drugs taken by the patient (name of the drug taken, start and end date of intake and dose); 2) on the patient’s health status (blood pressure, frequency and heart rhythm, respiratory rate and arterial saturation). Finally, the system allows the Physiatrist to define the rehabilitation team composed, in addition to the physiatrist, of a psychologist and one or more physiotherapists. Once completed the enrollment of the patient by the Physiatrist, the system allows the Physiotherapist to define the motor rehabilitation plan of a patient by defining the plan of the exercises to be performed (execution sequence, date and scheduled duration).

Through a rehabilitation station, the system allows the Patient to carry out a rehabilitation exercise provided by her motor rehabilitation plan and to automatically acquire and evaluate her movements during the execution of a specific exercise. The system also allows the Patient, immediately after performing a rehabilitation exercise, to evaluate her performance, specifying the evaluation (on a scale of 1–5) and any comments on the execution.

Besides the subjective evaluation of the performance given by the patient, the system allows to store other two evaluations:

• an objective evaluation given by the Physiotherapist. Especially, the system allows the Physiotherapist to access the recording of the exercise executions and provide an evaluation (on a scale of 1–5) and any comments on the execution;

• an objective evaluation automatically provided by the 2Vita-B Physical system. Such an evaluation is automatically derived by comparing the patient’s movements with the “ideal” movements.

All these evaluations are particularly useful for the Physiatrist during the rehabilitation program. The subjective evaluation provides insights into the feeling of the patient, while the objective evaluation is useful to assess the correctness of the exercise execution, in particular, and on the effectiveness of the motor rehabilitation plan, in general. As for the objective evaluation, the evaluation of the Physiotherapist is useful to confirm the automatic evaluation and helps to improve the evaluation of the system by exploiting a continuous machine learning strategy.

2.2 Architecture of the System

In this section, the logical and physical architecture of the 2Vita-B Physical system is described at a high level. The architecture is organized according to the three-tier schema (see Figure 2):

• Presentation Layer: contains the graphical interfaces and therefore manages the interaction with the system user. The user interacts with the 2Vita-B Physical system, and in particular with the Presentation Layer, through one of the two possible clients of the system:

• Management Client: this client accesses the software components belonging to the Presentation Layer executable on a web browser both from desktop and mobile devices. Through these components it is possible to perform operations related to the management of users and individual rehabilitation plans in 2Vita-B Physical;

• Rehab Client: this client—through a dedicated Unity application—allows to perform both the operations of acquisition of an ideal movement and the real operations of rehabilitation (e.g., execution of an exercise). To monitor a rehabilitation exercise, the Rehab Client makes use of two dedicated devices, namely the Microsoft Azure Kinect DK,¹ for motion analysis, and a heart rate monitor, for acquiring the heart rate to estimate the stress during the execution of an exercise. The Microsoft Azure Kinect DK is an input device capable of capturing body movement. One of the main advantages provided by the Kinect is the quality of the acquisition results obtained considering the use of a single hardware component that is easy to install. According to the specifications provided by Microsoft, the newly released Kinect presents several new features, including a depth sensor and an RGB camera with resolutions respectively of 1 and 12 MP, an array of seven microphones, two IMU sensors (specifically an accelerometer and a gyroscope) and an external sync connection (to enabling accurate synchronization of multiple Kinect devices). As for the heart rate monitoring, in the context of the 2Vita-B Physical project, the Polar 0H1² device was chosen. This device consists of a comfortable and stable band that is placed on the forearm, is waterproof and has a declared autonomy of 12 h. The real-time detection of heart rate is possible thanks to the optical reading system with 6 LEDs proprietary to Polar.

• Application Layer: it is responsible for the application logic (or business logic) of the system. The application layer has a dedicated module, i.e., AI Module, for the automatic evaluation of the correctness of an exercise.

• Storage Layer: it is responsible for the persistence (storage) of the data inside the system. For security reasons the storage is divided into three sub-archives: archive DB, containing all the personal data of users; rehab DB, containing the health data of patients; monitoring DB, containing the movements and vital parameters acquired during the execution of an exercise.

FIGURE 2

FIGURE 2. Three-tier architecture in the 2Vita-B Physical system.

Using a three-tier schema has several advantages, including the ability to more effectively and efficiently manage data validations on both the Presentation Layer and Application Layer sides, greater scalability, and less coupling of components.

Regarding the physical architecture of the 2Vita-B Physical system, the system is based on a microservice architecture. Figure 3 shows the deployment diagram of the 2Vita-B Physical system. The main communication mechanisms are reported below:

• Communication between the web browser, front-end and back-end management. The user’s browser requests, using the HTTPS protocol, the front-end of the 2Vita-B Physical system. Once the front-end is loaded, it is possible to perform all the operations offered by the interface. According to the user’s actions, REST requests will be forwarded to the backend API-gateway which will check the authentication and authorization and will forward them, if successful, to the interested microservices. The gateway can route the requests since it obtains the list of registered microservices from the discovery-service.

• Communication between the external device, Rehab front-end and back-end. The external devices foreseen in 2Vita-B Physical are the Azure Kinect DK and a heart rate monitor that will be connected to the Rehab front-end (developed in Unity) through, respectively, the USB and BLE (Bluetooth Low Energy) interfaces. Especially, the Rehab front-end will detect the data sent by the sensors, process it and, when required, send it, similarly to the Management front-end, to the back-end API-gateway.

• Communication with the artificial intelligence module. The 2Vita-B Physical system exploits an artificial intelligence module that, through a Machine Learning (ML) algorithm, expresses an evaluation on the execution of an exercise performed by the assisted person, taking into account not only its distance from the ideal movement but also the previous judgments of the physiotherapist on exercises performed similarly. During its actual use, the 2Vita-B Physical system will continue to improve the accuracy of the model through a continuous learning strategy, which involves updating the model based on new assessments made by the physical therapist. The ML module of the 2Vita-B system is accessible, as a stand-alone component via REST interface and deployed via Gitlab pipeline, to rehabilitation-service only.

• Communication between microservices and databases and intercommunication between microservices. Each microservice has its database. The communication between a microservice and its relational DB is done through the JDBC protocol and the Hibernate library, which is among the most used implementations of JPA (Java Persistence API) at the moment. For the communication between the monitoring microservice and its document DB, instead, the MongoDB Wire Protocol and a Java driver are used. Such components allow the manipulation of objects representing the database and the specific collections. As foreseen by the architecture, it is possible the communication between different microservices, which expose specific interfaces to create a communication channel characterized by the exchange of JSON documents. So, if a microservice needs information from another module, it is sufficient to forward its request to the API-gateway which will address it to the target microservice and return the answer.

FIGURE 3

FIGURE 3. 2Vita-B Physical system deployment diagram.

2.3 The Rehab Station

This section describes in detail the main component of 2Vita-B Physical, i.e., the rehabilitation station. Such component—through the latest version of Microsoft Kinect—allows the acquisition of an “ideal” movement for a specific rehabilitation exercise as well as the execution of rehabilitation exercises planned by a specific motor rehabilitation plan. In the following, the latter functionality is described in detail, after a description of studies evaluating the Microsoft Kinect in the context of motion tracking and analysis.

2.3.1 Motion Tracking and Analysis With Microsoft Azure

Microsoft Azure has widely experimented in the past with motion analysis and tracking applications. Obdrvzalek et al. (2012) compared the estimation of the Kinect v1 pose with more sophisticated pose estimation techniques from motion capture data and evaluated precision and robustness. The stated background is the elderly population’s coaching. They used a single Kinect camera mounted about 3 m away from the subject. The authors reported a 30-min coaching session that would be usually extended for the daily workout routine of the elderly. They conclude the paper announcing their acceptance as a low-cost solution for real-time motion capture and body monitoring in health applications using systems such as Kinect.

Kuster et al. (2016) involved twenty subjects in their experimentation aimed at assessing the validity of the Kinect v1 in tracking upper body motion. Therefore, participants were requested to conduct shoulder abduction in frontal and scapula plane, flexion, external rotation and horizontal flexion in two conditions (sitting and standing). They compared the Kinect sensor to a marker-based system. The conclusions of this work report the adequacy of the Kinect v1 sensor to measure arm motion in a clinical setting.

Mortazavi and Nadian-Ghomsheh (2018) studied the stability of Kinect for Range of Motion (ROM) measurement during static stretching exercises. In this study, 13 subjects took part and completed 5 exercises at 2 separate distances from the cameras. The exercises were recorded simultaneously with the Microsoft Kinect v1 and Kinect v2. The results showed that Kinect v2 is more stable compared to Kinect v1.

Clark et al. (2015) investigated the accuracy of the Microsoft Kinect v2 during a variety of static and dynamic balance assessments. The aim of this study was therefore to assess the concurrent validity and reliability of Kinect v2 kinematic data compared to a marker-based benchmark system. Thirty stable adults completed two exercise sessions, separated by 1 week, consisting of static standing balance tests under varying visual conditions (eyes open vs. closed) and accommodating conditions (single limb vs. double limb), and dynamic balance tests consisting of forward and lateral reach and equilibrium maximum measurement. The results of this work showed that concurrent validity between the systems was mostly found for trunk angle data during the dynamic tasks and—as for the static balance—the highest correlations were found for the anterior-posterior range and path length in the static balance.

The latest version of Kinect, i.e., Azure Kinect DK, has been recently evaluated in the context of motion tracking as well (Albert et al., 2020; Ma et al., 2020; Tölgyessy et al., 2021). All these studies considered the Azure Kinect DK as a promising sensor for motion tracking applications. Thus, we are confident that the Azure Kinect DK based motion tracking system of 2Vita-B Physical can provide an accurate analysis of the execution of rehabilitation activities.

2.3.2 Execution of an Exercise

After logging into the system, 2Vita-B shows to the Patient a form containing the exercises to be performed and general information about the motor rehabilitation plan is displayed (Figure 4A). To execute an exercise, the Patient has simply to choose “Execute.” Also, the Patient can scroll through the list of exercises using the arrows on either side of the current exercise tab.

FIGURE 4

FIGURE 4. The GUI of the Rehab Station: (A) list of exercise scheduled according to the motor rehabilitation plan; (B) detailed description of the exercise to be performed; (C) execution of an exercise; (D) replay of a performed exercise.

By clicking on a specific exercise, the Patient can access a detailed description of the exercise as well as the movements to execute (Figure 4B). By clicking on the “Video” button the user can also watch a video of the exercise being performed. At any time, it is possible to move to the “Execution” tab by clicking on it.

A screen with the execution of an exercise is depicted in Figure 4C. The red dot on the left indicates that the movement is being recorded. During the execution of an exercise, there is the avatar performing the ideal movement. Only the avatar’s outline is visibly superimposed on the actual movement by the Patient. Feedback on the performance is also provided (see the smiley face and the bar under the avatars). On the left-hand side, the Patient will also find 1) the overall (average) performance level, 2) the current performance level, 3) the elapsed time, 4) the current repetition and 5) the heart rate.

At the end of the execution, it is also possible to visualize the recorded performance (see Figure 4D). On the left, there is the actual movement (with again the template representing the ideal movement) and on the right the ideal movement. Finally, by moving to the “Considerations” tab, the Patient can provide numerical feedback (from one to five stars) about the perception of the own performance and a textual note. By clicking on “Save” the execution is uploaded online.

3 Usability Evaluation With Potential Patients

This section reports the results of a preliminary study carried out to evaluate the usability of the main component of 2Vita-B Physical, i.e., the Rehab Station.

3.1 Planning of the Study

The context of the study is composed by 29 participants, 21 males and eight females. Table 1 reports some statistics on the subjects involved in this experimentation.

TABLE 1

TABLE 1. Statistics of the participants.

In the context of the study, participants were required to perform five rehabilitation exercises with the support of the Rehab Station of the 2Vita-B Physical system. The five exercises to be perform belong to five different motor rehabilitation plans:

• shoulder rehabilitation (with particular reference to the deltoid muscles of the middle bundles): standing, hold a weight of 1–2 Kg, with the arm outstretched, abduct until reaching 90°, without exceeding shoulder height. Return slowly to starting position.

• elbow rehabilitation: standing, hands crossed behind nape of neck, move hands away from head, extending elbows upward, hold for 4 s and return to starting position.

• hip rehabilitation (with reference to the iliopsoas muscle, rectus femoris muscle, sartorius muscle and pectoralis muscle): standing, with a 1–2 Kg weight, flex the hip until it reaches 90° and return to the starting position (the exercise can be performed with the knee flexed or with the leg extended).

• knee rehabilitation (with reference to the quadriceps muscle and large and small gluteal muscles): standing, holding two dumbbells one in each hand, step forward with your right foot, resting on a step. Concentrate on the left leg, bending it down until the knee touches the floor. The leg to bend is the left, the right leg bends accordingly, not the other way around. Return to the starting position by pushing with the front foot.

• spine rehabilitation (with particular reference to the lumbar muscles): standing, arms along sides, slide left hand along left thigh, tilting trunk, to maximum possible width, hold for 4 s and return to starting position. Repeat on right.

The choice of the exercise was not random. Exercises were properly selected to evaluate the usability of the system during rehabilitation activities involving both upper and lower part of the body.

Each exercise was repeated ten times. The experiment included a single period of performance, during which the subject performed the entire sequence of the five exercises described above. The approximate duration for the execution of all exercises was 30 min. A physiotherapist assisted during the execution of the exercise to provide information, if needed, to the participants. No specific clothing was provided for the performance of the exercises, although we recommended sportswear.

At the end of the execution of all the exercises participants filled in an online questionnaire. The questionnaire aims at measuring the usability of the system and is based on the System Usability Scale (SUS) (Zviran et al., 2006; Salvendy, 2012; Brooke, 2013), a reliable tool for measuring the usability. The questionnaire is reported in Table 2. It consists of a ten item questionnaire with five response options for respondents, from “Strongly agree” (5) to “Strongly disagree” (1). There is a positive tone of the odd-numbered items; the tone of the even-numbered items is negative. Much effort form the scientific literature was dedicated to the assessment of the SUS as usability questionnaire and in many studies the evidence of the validity and sensitivity of the SUS is shown (Bangor et al., 2008; Lewis and Sauro, 2009; Lewis et al., 2015). Before filling-in the questionnaire, participants had also to provide their consent; the data collected is recorded, processed, managed and stored in electronic, automated, and computerized form for the exclusive purposes related to research, in an absolutely anonymous form.

TABLE 2

TABLE 2. The System Usability Scale (SUS) questionnaire (Brooke, 1996) used to evaluate the usability of the system.

The SUS tool allows to compute an overall score based on the answers (values ranging from 1 to 5) provided by each participant. Especially, positively worded questions (odd numbers) contribute to such a score as the selected value by the participant minus one; while the score contribution for negatively worded questions (even numbers) is five minus the selected value. Then, the final score is obtained by multiplying the total of the single item score by 2.5. As a result, the total SUS rating varies from 0 to 100, with 2.5 point intervals between 0 and 100 (Sauro and Lewis, 2012).

Several analytics were also proposed in the literature to convert the raw SUS score into a grade of usability. The most comprehensive approach is the one proposed by Sauro (Sauro, 2011). Especially, to generate a global benchmark for SUS, the author combined the Bangor et al. (2008), Sauro (2011), and Albert and Tullis (2008) datasets containing a total of 446 surveys/usability studies of different kinds of software systems. A survey/study has multiple respondents; most have been between 10 and 30 respondents and some have more than 300. Based on the distribution of SUS score in such a large dataset, Sauro was able to define a table for converting a generic SUS score on a grade of usability ranging form A+ to F.

3.2 Analysis of the Results

The individual SUS score obtained from the participants are depicted in Figure 5. The distribution of raw SUS score presents a mean value of 81.8 with a standard deviation equal to 14.1. According to methodology proposed by Sauro (2011), the achieved average score reached the 90–95 percentile rank. This means that 2Vita-B Physical is more usable than 90–95% of 446 systems composing the dataset by Sauro (2011).

FIGURE 5

FIGURE 5. The SUS scores achieved in the usability study.

The distribution of grades, according to rating scale proposed by Sauro (2011), is depicted in Figure 6. As we can see, in 17 out 29 (about 60%) of the cases the usability of the system reached the maximum grade, i.e., A+. By considering the achieved average SUS score, 2Vita-B Physical obtains an A grade. Only in two cases, the score was F.

FIGURE 6

FIGURE 6. The SUS grades distribution, according to rating scale proposed by Sauro (2011), achieved in the usability study.

Most of the participants have considered the 2vitaB-Physical system highly usable. Particularly, questions 1, 3 and 7 have been marked with a high score by many participants. These results are indicating that:

• if rehabilitation therapy is needed, most of the participants think they would use the 2Vita-B Physical system frequently (question 1).

• many participants have considered the 2Vita-B Physical system easy to use. And also they have considered the system very quick to learn by most of the people (question 3–7).

Many positive comments were provided also in the notes, where participants were free to express their opinion. A common comment is about the importance of the 2Vita-B Physical system because it allows physical rehabilitation sessions to be conducted with the same care as in professional centres. Also, many comments are about the ease of use of the system.

If we take into consideration the participants that filled that questionnaires that generated the 2 F grades, we can notice that, in contraposition to the others, they have considered the system as not easy to use. Also, one of them thinks that many things have to be learned before using the 2Vita-B Physical system. In other cases, one of the participants marked the question 8 with a 5, meaning that the system was found very awkward to use. In the free-text space, one of these participants stated that the tracking—for some movements—was not precise and that a delay could be perceived by observing the avatar.

At the end of the questionnaire, participants had also the possibility to leave some general comments on the system by filling in a free text field. From the analysis of such comments it was possible to derive interesting improvements to the system that we plan to implement in the future:

• It might be preferable to customize the avatar in terms of gender, height, and weight, perhaps acquiring them directly from the user. In addition, several participants would opt for an avatar with a more human appearance or even one that reflects the user;

• Although accurate, it might be useful to further improve the accuracy of the avatar movements so that it shows as accurately as possible how to perform a specific movement;

• There were mixed comments on the monitoring and display of vital parameters. For some, they are distracting; others, however, requested the monitoring of additional vital parameters, such as the level of body hydration, breathing, and EMG for muscle tension.

4 The Viewpoint of the Physiotherapist

The previous study provides an indication of the usability of the system by potential patients that were enrolled to perform physical rehabilitation exercises. However, the 2Vita-B Physical system involves other actors, such as the physiotherapist. In order to get insights on the usefulness of the tool by the viewpoint of an expert, we conducted a structured interview with a physiotherapist with more than 10 years of experience in rehabilitation therapies. A structured interview is a kind of interview characterized by the use of extremely structured and codified questions aiming at generalizing the answers received. In the following we report the experimental procedure and the analysis of the answers received.

4.1 Experimental Procedure

The experimental procedure first provided an in-depth demo of the 2VitaB-Physical system, and then moved on to the evaluation part, in which the specialist was asked the questions shown in Table 3. At the end of the interview, we also collected free considerations and possible suggestions to improve the effectiveness of the 2Vita-B Physical system.

TABLE 3

TABLE 3. The questions asked to the physiotherapist in the context of the structured interview.

4.2 Analysis of the Answers

Regarding home rehabilitation (question 1), the physiotherapist declared that home rehabilitation is a great way to prevent movement for people who have impediments. However, with home rehabilitation, the physiotherapist is unable to assess the environment in which the exercise is being performed. This assessment is critical to ensure the safety of the patient. In addition, some aspects of the patient’s health are better assessed in the presence (e.g., balance problems). Such an assessment allows the physiotherapist to define low-risk, environmentally appropriate exercises. Nevertheless, the physiotherapist confirmed that home rehabilitation is an excellent tool that the patient can use having the perception of being monitored by a professional (question 2). In the specific case of the 2Vita-B system, this happens because of the presence of visual feedback, which is normally missing when doing exercises at home alone.

Turning to the 2Vita-B Physical system (question 3), the physiotherapist considered it a great system, providing practical support to both the physiotherapist and the patient. One of the strong points is the continuous feedback: The system accompanies the patient, who is not in danger of forgetting the exercise. Moreover, besides showing the exercise, the presence of continuous feedback that shows where the patient is making mistakes and how to correct the mistakes helps in the correct execution and safety of the exercise itself. Thus, the physiotherapist considered the presence of an avatar a very useful visual support to the self-correction of the movement by the patient (question 4). However, she also emphasized that the (physical) movement correction done by a physiotherapist is irreplaceable. But if the patient knows how to correct himself, 2Vita-B Physical is a great system. Finally, the physiotherapist considered very clear the visual feedback provided during the execution of an exercise (question 5).

Regarding the actors of the 2Vita-B system, the physiotherapist (question 6) suggested also to add the nurse with the aim to record into the system any problems with scars and/or healing wounds in the patient. Also, the interaction among all the actors was definitely considered of great importance.

Finally, the physiotherapist considered the artificial intelligence component a crucial component to supporting the physiatrist or physiotherapist in the long-term management of a patient. It can be of great help also to the other members of the team as it allows to have a preliminary idea of the progress of the patient’s rehabilitation therapy. Also, the physiotherapist confirmed that the artificial intelligence component needs to take into account compensatory movements when predicting the correctness of an exercise. For example, if a patient fails to extend the arm and compensates for such movement by extending the torso, this should be negatively evaluated.

At the end of the interview, the physiotherapist was asked to provide any feedback to further improve the 2Vita-B Physical system. The first suggestions concerned assistance during a rehabilitation exercise. Specifically, the specialist stated that it could be useful to provide the physiotherapist with the possibility to make a video call with the patient to allow her to assess the environment and the health status of the patient before starting the exercise. Moreover, it could be useful, in addition to a visual feedback, an audio feedback during the execution of an exercise to scan the time during the execution of the exercise, indicating the intermediate phases of the movement. Finally, the physiotherapist highlighted the usefulness of weighing more the errors during simple movements during the automatic evaluation of the correctness of an exercise. Indeed, these errors could indicate serious motor difficulties of the patient or lack of attention in carrying out the exercises.

5 Social Impact of the System

In this section we reflect on the impact that the 2Vita-B Physical might have in practice.

5.1 Improved User Experience for Rehabilitation

The first clear benefit provided by 2Vita-B Physical is the improved user experience compared to traditional rehabilitation activities. The 2Vita-B Physical system, indeed, provides gamification elements, i.e., it uses concepts typical of games (e.g., scores, achievements, and rewards) to make the rehabilitation activity more compelling. Nowadays, gamification is widespread in the health domain: For example, apps that use gamification concepts for fitness and diet (Lister et al., 2014; Wu et al., 2015) revolutionized such fields. The benefits of serious games have been proved also in the context of the assistance of Persons with Dementia (PwD) (Unbehaun et al., 2020). A study of 4-month involving 52 PwD and 25 carers indicated that a videogame-based system promoted activity, launched meaningful connections, and facilitated social benefits in real-life care settings for individuals living with dementia. The system was also reported to partially enrich and support the complex daily routines and challenges, and even initiate time-saving effects for care givers. Finally, results showed that relatives saw the system as a tool for creating new incentives for organized training activities for PwD and encouraging them to participate in home and care facility activities.

The 2Vita-B Physical system aims to achieve a similar goal with rehabilitation. We expect our system to have an impact mostly also because previous work reported the positive results obtained by gamifying rehabilitative activities through the so-called serious games. Serious games for rehabilitation (Postolache et al., 2019) have clear therapeutic goals, such as: Forearm movement, upper limb control, limb angular velocity, hand-eye coordination, endurance, speed, accuracy, range of motion, trunk movements, core posture, balance, and muscle strength. Such systems aim to set tasks, record and display progression so that users feel they are achieving their goals and succeed in improving their physical condition. In fact, to be effective, games should elicit specific movement characteristics in subjects that are considered relevant to the function being trained (Skjæret et al., 2016). In addition, games have play environments that make it easier for older people or people with disabilities to transfer learned (or re-learned) movements into real-world situations. This allows users to have improved interaction with the virtual environment related to an exercise program, and maintains patient engagement in exergaming at a high level (Theng et al., 2009). Finally, exergaming generally offers similar results to conventional therapy (Laufer et al., 2014; Bonnechère et al., 2016), or alternative exercise (Donath et al., 2016), succeeding, in some cases, in providing an improved effect over conventional therapy (Skjæret et al., 2016).

5.2 Reduction of Costs Related to Rehabilitation

The main drawbacks of adopting home rehabilitation systems in practice are related the high costs of the equipment needed to accurately capture the patients’ movements at home. In 2Vita-B Physical, we experimented the use of a cheap yet effective device, the Microsoft Kinect Azure DK. It is worth noting that the original Microsoft Kinect was designed to be used as an interface for video games: Such a device is designed to be both easy to use and reasonably priced for the general public. Moreover, our system allows to reduce other costs related to rehabilitation. First, it rules out the costs linked to the necessity to move to the health facility. Such costs are not only economic-related, but also health-related, since such a practice slightly increases the risks of injuring for the patient. Also, this system allows to optimize the time management of physiotherapists: The activity performed by the patient and the evaluation made of the physician do not need to be synchronized. All things considered, we believe that a system like 2Vita-B Physical allows to sensibly reduce the costs of rehabilitation.

Our expectation is confirmed also by previous work, which focused on the impact in terms of cost of adopting remote rehabilitation on the healthcare costs (Dalal et al., 2019; Veras et al., 2020). Bettger et al. (2020) investigated the impact of a virtual treatment program on healthcare expenditures and clinical outcomes following Total Knee Arthroplasty (TKA) when compared to standard treatment. Virtual treatment with tele-rehabilitation for professional clinical observation dramatically reduced 3-month healthcare expenses as compared to standard home or clinic physical therapy. Furthermore, the results were found to be similar in terms of effectiveness. The authors recommended virtual treatment under physician supervision should be considered for TKA patients.

5.3 2Vita-B Physical Beyond Rehabilitation

In its current shape, the 2Vita-B Physical system is dedicated to patients that need rehabilitation, i.e., its usage is limited to the time interval in which the patient is in need of rehabilitation. However, our system can be adapted to several different scenarios, some of which provide for the use of the system continuously. One example is the training of athletes: Many disciplines require practicing in performing precise movements. The 2Vita-B system can be easily used, in this context, to allow athletes training continuously, thus limiting the necessity of the constant presence of a coach. In other words, the 2Vita-B Physical system could become a virtual personal trainer.

Another similar example, more directed to the mass public, is the possible adaptation of the 2Vita-B Physical system to the fitness domain. Differently from commonly used fitness apps, some of which allow to have optional feedback from a human “personal trainer,” our system would be completely automated and it would help users to learn how to correctly perform exercises.

6 Related Work

In the last decade, a lot of effort has been devoted by the research community to design and implements Kinect-based approaches supporting home rehabilitation (MousaviHondori and Khademi, 2014; Lun and Zhao, 2019). A recent survey on the use of Microsoft Kinect in healthcare has been presented by Lun and Zhao (2019). In the following, we discuss in detail the approaches more relevant for the 2Vita-B Physical system. We also compare the peculiarity of the 2Vita-B Physical system with similar approaches proposed in the literature.

Bamrungthai and Pleehachinda (2015) created a game-based approach to help people recover from strokes. The suggested system used a Microsoft Kinect v1 as a motion-tracking sensor and a basic color-matching puzzle game as the rehabilitation test. For the pre-experimentation, the authors enlisted a group of healthy students and professors, as well as a patient for the real validation of the system. The main objective declared by the authors was to assist stroke patients in regaining control of their hand and upper limb movement under the supervision of a therapist operator. The technology was successfully tested with a patient in a clinic context after successful in-house testing with normal people.

Calin et al. (2017) aimed at improving exergaming systems’ interaction through an advanced gesture analysis which provides detailed feedback to the user regarding the correctness of the exercise performed, in particular on movement amplitude. The idea is to detect the correctness of a movement, providing also intelligently derived information and recommendations on how to improve it. The long-term goal declared by the authors was to develop a virtual rehabilitation assistant that could support people during the execution of a rehabilitation exercise. The authors tested the approach on their personal Kinect 2 databases as well as on the UTD-MHAD publicly accessible gesture database (Kinect 1). They distinguished three primary components of the movement: pose, trajectory, and range of motion. They reached classification accuracy by up to 56% using the Hidden Markow Model-based method.

Hidden Markov Models have also been used by Palma et al. (2015) to recognize differences between the typical performance in physical therapy settings. To assess the method’s efficiency, a wide range of common workout motions and common variations from these motions were considered. Experiments demonstrated that utilizing Hidden Markov Models trained with proper repetitions of the activities, it is feasible to identify deviations from optimality in chosen physical therapy motions. Because the sensor’s coordinate data is sequential, the authors stated in the results that it is easy to train a model for each phase of the motions under consideration, allowing for deviation detection.

Chang et al. (2011) proposed the development of a Kinect-based rehabilitation system to help therapists rehabilitate students. The technology, called Kinerehab, recognizes the student’s joint position automatically and utilizes the information to decide if the student’s motions have met the rehabilitation criteria and if the amount of exercises in a therapy session is adequate. To boost students’ motivation, the system also offered an interactive interface with voice and visual feedback. Two participants with significant mobility difficulties participated in a study conducted to evaluate the proposed system. Results showed that the technology piqued the students’ interest, and both of them expressed the desire to use it when the trial was done.

Khan et al. (2014) presented a visual evaluation system for in-home physical therapy that provides real-time automatic feedback. To reliably analyze the distance score between the partial user sequence and the whole expert sequence, a specific method based on dynamic time warping was developed. Rather than a single number, the proposed approach displayed a colored virtual skeleton image as feedback. The user’s performance is indicated by the varied hues of the limbs. This type of visual feedback allowed the user to quickly comprehend the errors and remedy them. The efficiency of the proposed suggested strategy is demonstrated by preliminary findings from two exercises involving diverse users.

To create an effective rehabilitation system, Roy et al. (2013) created a system that included serious games (mini-games) that processed 3D input using the Microsoft Kinect motion sensor. Such games are gamified versions of the rehabilitation activities that physiotherapists and occupational therapists need their patients to do. The mini-games are designed to address particular limb mobility issues that affect a wide number of people. Using real-time motion capture data, the software animates a person’s in-game avatar. The user’s whole skeleton is monitored, and the functions the user performs in the virtual world are simulated.

Souza et al. (2014) proposed the use of the Microsoft Kinect depth sensor and specific image analysis algorithms to distinguish between healthy volunteers and patients with multiple sclerosis, as well as categorize patients based on the severity of specific motor dysfunctions. Especially, Kinect was utilized to record specific neurologic motions, which indicated upper and lower extremity functions, as well as truncal stability. The study involved 86 healthy volunteers and 72 patients with multiple sclerosis. The results achieved indicate that it is possible to discriminate healthy volunteers from patients with multiple sclerosis.

Ma et al. (2018) conducted a study to assess the spatial accuracy and measurement validity of the Microsoft Kinect V2 sensor as compared to a gold-standard marker-based motion capture system (Vicon). The study was carried out with the help of Mystic Isle, a rehabilitation game that has been shown to improve motor function and daily activity performance in chronic stroke patients. Multi-planar and full-body motions are required in the Mystic Isle game. Games can be performed in a sitting or standing posture, depending on the therapeutic treatment plan, and are designed for people of all abilities. The user may walk around in 3-dimensional space while standing, similar to real-world rehabilitation. The authors found that Mystic Isle gives an accurate assessment of movement when compared to the Vicon system; nevertheless, their findings revealed that the precision of particular movements and planes of measurement is significantly lower.

Table 4 reports a comparison of the approaches discussed above based on several characteristics:

• Operator Needed: indicates whether an operator is needed to assist the patient during the rehabilitation exercise;

• Online Feedback: indicates whether the proposed approach provides an online and continuous feedback during the execution of an exercise to provide indication on the correctness of the performance;

• Expert Avatar Guide: indicates whether the proposed approach displays an avatar that guide the patient during the execution of an exercise;

• Automatic evaluation: indicates whether the proposed approach integrates a component able to automatically classify the correctness of an exercise.

TABLE 4

TABLE 4. Comparative analysis of approaches similar to 2Vita-B Physical proposed in the literature.

The table also reports the version of the Kinect used, the context of the study, and the number of participants involved in the experimentation.

The analysis of the table emphasizes the peculiarities of the 2Vita-B Physical system. Especially, this system is able to put together all the characteristics considered in the table: 1) the 2Vita-B Physical integrates the newly released Azure Kinect, 2) it does not need the intervention of an external operator during the execution of an exercise, 3) it provides real-time feedback and integrates an avatar as an expert guide, 4) it embeds a component of Machine Learning aiming at supporting the physiotherapist by providing an automatic assessment of a rehabilitation exercise. None of the system provides in the literature is able to provide such a complete support during rehabilitation activities. In addition, the system supports the communication and the information exchange from the different stakeholders composing the rehabilitation team.

7 Conclusion

In this paper, the 2Vita-B Physical system was presented, a software system dedicated to home rehabilitation. Thanks to the employment of the newly released Microsoft Azure Kinect DK, 2Vita-B Physical embeds a motion tracking component that allows conducting rehabilitation activities even in the absence of the therapist.

A preliminary evaluation of 2Vita-B Physical through a usability study is also reported. The study was conducted with 29 participants who were requested to conduct five rehabilitation exercises. At the end of all the exercises, participants were asked to fill in a System Usability Scale (SUS) questionnaire. The analysis of the results revealed that 2Vita-B Physical is more usable than 90–95% of 446 systems considered as a benchmark with an overall usability grade equals to A on a scale ranging from A+ to F. The results achieved are very promising and provide evidence of the high usability of the 2Vita-B Physical system as a home rehabilitation system. However, further experimentations are needed to verify whether the positive attitudes in the short term are reflected in actual behavior in the longer term (Nicholson, 2015; Stepanovic and Mettler, 2018). We plan in the future to perform a long-term case study involving real patients aiming at collecting data about the usage of the 2Vita-B Physical system during real rehabilitation activities.

Since the 2Vita-B Physical system involves other actors besides the patient, we also reported the results of a structured interview with a physiotherapist with more than 10 years of experience. Such a study allowed us to collect interesting data about the usefulness of the proposed system from the viewpoint of an expert. Again, the results are promising confirming the benefits that the 2Vita-B Physical system might have in practice.

Besides a long-term case study aiming at evaluating the effectiveness of the 2Vita-B Physical system, in the future, we also plan to design and conduct an experiment aiming at assessing the ability of the system to automatically evaluate the correctness of an exercise.

Data Availability Statement

The raw data supporting the conclusion of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by Comitato Etico c/o Policlinico Militare di Roma “Celio” Piazza Celimontana, 50 - 00184 Roma. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

Conceptualization: AC, MC, RO and JS; Data curation, AC, MC, GL and JS; Formal analysis, AC, MC, NB, GL, and JS; Funding acquisition, MA; Investigation, AC, MC, NB, GL, AL, MN, RO, and JS; Methodology, AC, MC, RO and SR; Resources, AL; Software, AC, MC, JS; Supervision, RO; Validation, SR, and JS; Writing - original draft, GL; Writing - review and editing, NB, AL, RO, SR, and SS.

Funding

This research was funded by Ministry of Defense grant number 20536 (December 13, 2019) “2ViTA-B: Veteran Virtual Training for Aging Blockchain”—Proposal a2018.137.

Conflict of Interest

Author MA was employed by the company Atlantica spa. Authors RO, SS, and JS were employed by the company Datasound srl.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

The authors would like to thank all the subjects participating in the experimental evaluation.

Footnotes

¹https://azure.microsoft.com/en-us/services/kinect-dk/.

²https://www.polar.com/ca-en/products/accessories/oh1-optical-heart-rate-sensor.

References

Albert, J. A., Owolabi, V., Gebel, A., Brahms, C. M., Granacher, U., and Arnrich, B. (2020). Evaluation of the Pose Tracking Performance of the Azure Kinect and Kinect v2 for Gait Analysis in Comparison with a Gold Standard: A Pilot Study. Sensors 20, 5104. doi:10.3390/s20185104

PubMed Abstract | CrossRef Full Text | Google Scholar

Albert, W., and Tullis, T. (2008). Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. 1st Edn. Burlington, MA: Morgan Kaufmann.

Google Scholar

Bamrungthai, P., and Pleehachinda, W. (2015). “Development of a Game-Based System to Support Stroke Rehabilitation Using Kinect Device,” in 2015 International Conference on Science and Technology (TICST) (IEEE), 323–326. doi:10.1109/ticst.2015.7369379

CrossRef Full Text | Google Scholar

Bangor, A., Kortum, P. T., and Miller, J. T. (2008). An Empirical Evaluation of the System Usability Scale. Int. J. Human-Computer Interaction 24, 574–594. doi:10.1080/10447310802205776

CrossRef Full Text | Google Scholar

Bettger, J. P., Green, C. L., Holmes, D. N., Chokshi, A., Mather, R. C., Hoch, B. T., et al. (2020). Effects of Virtual Exercise Rehabilitation in-Home Therapy Compared with Traditional Care after Total Knee Arthroplasty. J. Bone Jt. Surg. 102, 101–109. doi:10.2106/jbjs.19.00695

CrossRef Full Text | Google Scholar

Bonnechère, B., Jansen, B., Omelina, L., and Van Sint Jan, S. (2016). The Use of Commercial Video Games in Rehabilitation: a Systematic Review. Int. J. Rehabil. Res. 39, 277–290. doi:10.1097/mrr.0000000000000190

CrossRef Full Text | Google Scholar

Brooke, J. (2013). Sus: a Retrospective. J. usability Stud. 8, 29–40. doi:10.5555/2817912.2817913

CrossRef Full Text | Google Scholar

Brooke, J. (1996). Sus: A “Quick and Dirty’usability. Milton Park, United Kingdom: Usability evaluation in industry, 189.

Google Scholar

Chang, Y.-J., Chen, S.-F., and Huang, J.-D. (2011). A Kinect-Based System for Physical Rehabilitation: A Pilot Study for Young Adults with Motor Disabilities. Res. Dev. disabilities 32, 2566–2570. doi:10.1016/j.ridd.2011.07.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Chang, Y.-J., Han, W.-Y., and Tsai, Y.-C. (2013). A Kinect-Based Upper Limb Rehabilitation System to Assist People with Cerebral Palsy. Res. Dev. disabilities 34, 3654–3659. doi:10.1016/j.ridd.2013.08.021

CrossRef Full Text | Google Scholar

Clark, R. A., Pua, Y.-H., Oliveira, C. C., Bower, K. J., Thilarajah, S., McGaw, R., et al. (2015). Reliability and Concurrent Validity of the Microsoft Xbox One Kinect for Assessment of Standing Balance and Postural Control. Gait & Posture 42, 210–213. doi:10.1016/j.gaitpost.2015.03.005

CrossRef Full Text | Google Scholar

Călin, A. D., Pop, F. H., and Boian, F. R. (2017). “Improving Movement Analysis in Physical Therapy Systems Based on Kinect Interaction,” in Proceedings of the 31st International BCS Human Computer Interaction Conference (HCI 2017) (Vancouver, BC: HCI), 1–5.

Google Scholar

Da Gama, A. E. F., Chaves, T. d. M., Fallavollita, P., Figueiredo, L. S., and Teichrieb, V. (2019). Rehabilitation Motion Recognition Based on the International Biomechanical Standards. Expert Syst. Appl. 116, 396–409. doi:10.1016/j.eswa.2018.09.026

CrossRef Full Text | Google Scholar

Dalal, H. M., Taylor, R. S., Jolly, K., Davis, R. C., Doherty, P., Miles, J., et al. (2019). The Effects and Costs of home-based Rehabilitation for Heart Failure with Reduced Ejection Fraction: The Reach-Hf Multicentre Randomized Controlled Trial. Eur. J. Prev. Cardiolog 26, 262–272. doi:10.1177/2047487318806358

CrossRef Full Text | Google Scholar

Donath, L., Rössler, R., and Faude, O. (2016). Effects of Virtual Reality Training (Exergaming) Compared to Alternative Exercise Training and Passive Control on Standing Balance and Functional Mobility in Healthy Community-Dwelling Seniors: a Meta-Analytical Review. Sports Med. 46, 1293–1309. doi:10.1007/s40279-016-0485-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Fern’ndez-Baena, A., Susin, A., and Lligadas, X. (2012). “Biomechanical Validation of Upper-Body and Lower-Body Joint Movements of Kinect Motion Capture Data for Rehabilitation Treatments,” in 2012 Fourth International Conference on Intelligent Networking and Collaborative Systems (IEEE), 656–661.

Google Scholar

Han, D., Kuschner, D., and Wang, Y.-f. (2005). “Upper Limb Position Sensing: A Machine Vision Approach,” in Conference Proceedings. 2nd International IEEE EMBS Conference on Neural Engineering (IEEE), 490–493.

Google Scholar

Khan, N. M., Lin, S., Guan, L., and Guo, B. (2014). “A Visual Evaluation Framework for in-home Physical Rehabilitation,” in 2014 IEEE International Symposium on Multimedia (IEEE), 237–240. doi:10.1109/ism.2014.21

CrossRef Full Text | Google Scholar

Kuster, R. P., Heinlein, B., Bauer, C. M., and Graf, E. S. (2016). Accuracy of Kinectone to Quantify Kinematics of the Upper Body. Gait & Posture 47, 80–85. doi:10.1016/j.gaitpost.2016.04.004

CrossRef Full Text | Google Scholar

Lange, B., Chang, C.-Y., Suma, E., Newman, B., Rizzo, A. S., and Bolas, M. (2011). “Development and Evaluation of Low Cost Game-Based Balance Rehabilitation Tool Using the Microsoft Kinect Sensor,” in 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE), 1831–1834. doi:10.1109/iembs.2011.6090521

CrossRef Full Text | Google Scholar

Laufer, Y., Dar, G., and Kodesh, E. (2014). Does a Wii-Based Exercise Program Enhance Balance Control of Independently Functioning Older Adults? a Systematic Review. Cia 9, 1803. doi:10.2147/cia.s69673

CrossRef Full Text | Google Scholar

Lewis, J. R., Brown, J., and Mayes, D. K. (2015). Psychometric Evaluation of the Emo and the sus in the Context of a Large-Sample Unmoderated Usability Study. Int. J. Human-Computer Interaction 31, 545–553. doi:10.1080/10447318.2015.1064665

CrossRef Full Text | Google Scholar

Lewis, J. R., and Sauro, J. (2009). “The Factor Structure of the System Usability Scale,” in International Conference on Human Centered Design (Springer), 94–103. doi:10.1007/978-3-642-02806-9_12

CrossRef Full Text | Google Scholar

Lister, C., West, J. H., Cannon, B., Sax, T., and Brodegard, D. (2014). Just a Fad? Gamification in Health and Fitness Apps. JMIR serious games 2, e9. doi:10.2196/games.3413

PubMed Abstract | CrossRef Full Text | Google Scholar

Lun, R., and Zhao, W. (2019). A Survey of Using Microsoft Kinect in Healthcare. Springer, 445–456. doi:10.4018/978-1-5225-6198-9.ch023

CrossRef Full Text | Google Scholar

Ma, M., Proffitt, R., and Skubic, M. (2018). Validation of a Kinect V2 Based Rehabilitation Game. PloS one 13, e0202338. doi:10.1371/journal.pone.0202338

PubMed Abstract | CrossRef Full Text | Google Scholar

Ma, Y., Sheng, B., Hart, R., and Zhang, Y. (2020). “The Validity of a Dual Azure Kinect-Based Motion Capture System for Gait Analysis: a Preliminary Study,” in 2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) (IEEE), 1201–1206.

Google Scholar

Maclean, N., Pound, P., Wolfe, C., and Rudd, A. (2002). The Concept of Patient Motivation. Stroke 33, 444–448. doi:10.1161/hs0202.102367

PubMed Abstract | CrossRef Full Text | Google Scholar

Mortazavi, F., and Nadian-Ghomsheh, A. (2018). Stability of Kinect for Range of Motion Analysis in Static Stretching Exercises. PloS one 13, e0200992. doi:10.1371/journal.pone.0200992

PubMed Abstract | CrossRef Full Text | Google Scholar

Mousavi Hondori, H., and Khademi, M. (2014). A Review on Technical and Clinical Impact of Microsoft Kinect on Physical Therapy and Rehabilitation. J. Med. Eng. 2014, 1–16. doi:10.1155/2014/846514

CrossRef Full Text | Google Scholar

Nicholson, S. (2015). A Recipe for Meaningful Gamification. Springer, 1–20. doi:10.1007/978-3-319-10208-5_1

CrossRef Full Text | Google Scholar

Obdržálek, Š., Kurillo, G., Ofli, F., Bajcsy, R., Seto, E., Jimison, H., et al. (2012). “Accuracy and Robustness of Kinect Pose Estimation in the Context of Coaching of Elderly Population,” in 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE), 1188–1193. doi:10.1109/embc.2012.6346149

CrossRef Full Text | Google Scholar

Palma, C., Salazar, A., and Vargas, F. (2015). “Hmm Based Evaluation of Physical Therapy Movements Using Kinect Tracking,” in International Symposium on Visual Computing (Springer), 174–183. doi:10.1007/978-3-319-27857-5_16

CrossRef Full Text | Google Scholar

Postolache, G., Carry, F., Lourenço, F., Ferreira, D., Oliveira, R., Girão, P. S., et al. (2019). “Serious Games Based on Kinect and Leap Motion Controller for Upper Limbs Physical Rehabilitation,” in Modern Sensing Technologies (Springer), 147–177. doi:10.1007/978-3-319-99540-3_8

CrossRef Full Text | Google Scholar

Roy, A. K., Soni, Y., and Dubey, S. (2013). “Enhancing Effectiveness of Motor Rehabilitation Using Kinect Motion Sensing Technology,” in 2013 IEEE Global Humanitarian Technology Conference: South Asia Satellite (GHTC-SAS) (IEEE), 298–304. doi:10.1109/ghtc-sas.2013.6629934

CrossRef Full Text | Google Scholar

Salvendy, G. (2012). Handbook of Human Factors and Ergonomics. John Wiley & Sons.

Google Scholar

Sauro, J. (2011). A Practical Guide to the System Usability Scale: Background, Benchmarks & Best Practices. Denver, CO: Measuring Usability LLC.

Google Scholar

Sauro, J., and Lewis, J. R. (2012). Standardized Usability Questionnaires. Amsterdam, Netherlands: Quantifying the user experience, 8.

Google Scholar

Skjæret, N., Nawaz, A., Morat, T., Schoene, D., Helbostad, J. L., and Vereijken, B. (2016). Exercise and Rehabilitation Delivered through Exergames in Older Adults: An Integrative Review of Technologies, Safety and Efficacy. Int. J. Med. Inform. 85, 1–16. doi:10.1016/j.ijmedinf.2015.10.008

CrossRef Full Text | Google Scholar

Souza, M. D., Kamm, C., Burggraaff, J., Tewarie, P., Glocker, B., Dorn, J., et al. (2014). Assessment of Disability in Multiple Sclerosis Using the Kinect-Camera System: A Proof-Of-Concept Study, 139.

Google Scholar

Stepanovic, S., and Mettler, T. (2018). “Gamification Applied for Health Promotion: Does it Really foster Long-Term Engagement? a Scoping Review,” in Proceedings of the 26th European Conference on Information Systems.

Google Scholar

Theng, Y.-L., Dahlan, A. B., Akmal, M. L., and Myint, T. Z. (2009). “An Exploratory Study on Senior Citizens’ Perceptions of the Nintendo Wii: the Case of singapore,” in Proceedings of the 3rd International Convention on Rehabilitation Engineering & Assistive Technology, 1–5.

Google Scholar

Tölgyessy, M., Dekan, M., Chovanec, Ľ., and Hubinský, P. (2021). Evaluation of the Azure Kinect and its Comparison to Kinect V1 and Kinect V2. Sensors 21, 413. doi:10.3390/s21020413

CrossRef Full Text | Google Scholar

Unbehaun, D., Taugerbeck, S., Aal, K., Vaziri, D. D., Lehmann, J., Tolmie, P., et al. (2020). “Notes of Memories: Fostering Social Interaction, Activity and Reminiscence through an Interactive Music Exergame Developed for People with Dementia and Their Caregivers,” in Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–34. doi:10.1080/07370024.2020.1746910

CrossRef Full Text | Google Scholar

Veras, M., Stewart, J., Deonandan, R., Tatmatsu-Rocha, J. C., Higgins, J., Poissant, L., et al. (2020). Cost Analysis of a home-based Virtual Reality Rehabilitation to Improve Upper Limb Function in Stroke Survivors. Glob. J. Health Sci. 12, 1–98. doi:10.5539/gjhs.v12n2p98

CrossRef Full Text | Google Scholar

Wu, Y., Kankanhalli, A., and Huang, K.-w. (2015). Gamification in Fitness Apps: How Do Leaderboards Influence Exercise?. Fort Worth, TX: Wellness, Ageing, Serious Games and Healthcare Communities.

Google Scholar

Zviran, M., Glezer, C., and Avni, I. (2006). User Satisfaction from Commercial Web Sites: The Effect of Design and Use. Inf. Manag. 43, 157–178. doi:10.1016/j.im.2005.04.002

CrossRef Full Text | Google Scholar