- 1School of Information Engineering, Wuhan University of Technology, Wuhan, China
- 2School of Electronic and Electrical Engineering, University of Leeds, Leeds, United Kingdom
The coordinated rehabilitation of the upper limb is important for the recovery of the daily living abilities of stroke patients. However, the guidance of the joint coordination model is generally lacking in the current robot-assisted rehabilitation. Modular robots with soft joints can assist patients to perform coordinated training with safety and compliance. In this study, a novel coordinated path planning and impedance control method is proposed for the modular exoskeleton elbow–wrist rehabilitation robot driven by pneumatic artificial muscles (PAMs). A convolutional neural network-long short-term memory (CNN-LSTM) model is established to describe the coordination relationship of the upper limb joints, so as to generate adaptive trajectories conformed to the coordination laws. Guided by the planned trajectory, an impedance adjustment strategy is proposed to realize active training within a virtual coordinated tunnel to achieve the robot-assisted upper limb coordinated training. The experimental results showed that the CNN-LSTM hybrid neural network can effectively quantify the coordinated relationship between the upper limb joints, and the impedance control method ensures that the robotic assistance path is always in the virtual coordination tunnel, which can improve the movement coordination of the patient and enhance the rehabilitation effectiveness.
Introduction
In China, the lifetime risk of stroke is 39.9%, ranking first globally. Almost 85% of stroke survivors have difficulties with upper limb motor functions (Liu et al., 2020). The human upper limb participates in a wide variety of activities of daily living (ADL) tasks, and its damage will seriously reduce the mobility and quality of life of patients. The injury of the upper limb will be reflected in the abnormal inter-joint coordination (Brokaw et al., 2014). Stroke patients, especially in the late stage of recovery, tend to have abnormal movement problems in the elbow and wrist while the shoulder joint has a relatively lower possibility of injury (Bilić et al., 2001; Squeri et al., 2014). Patients after stroke tend to suffer from upper limb dysfunction, such as the loss of coordination or being unable to perform coordinated movements. The recovery of upper limb coordination is essential for stroke patients to improve their prognosis (Saita et al., 2020). Robot-assisted therapy can provide patients with highly intensive training to intensify their motor function (Hsieh et al., 2018) and help recover their limb movement coordination (Carpinella et al., 2020).
Upper limb rehabilitation robots can be divided into two types, end-effectors or exoskeletons (Zhang et al., 2018). Compared with end-effectors, exoskeleton robots can assist patients in multiple joints which is more suitable for the human body (Zimmermann et al., 2019). It is worth noting that traditional rehabilitation robots are usually driven by rigid motors, which bring problems such as high stiffness and lack of compliance (Dindorf and Wos, 2020). In contrast, pneumatic artificial muscle (PAM) is lightweight and compliant, which meets the requirements of both safety and comfort during rehabilitation (Ghobj et al., 2017). More recently, Santos et al. (2017) developed a modular lower limb rehabilitation exoskeleton composed of telescopic tubular and modular structures. Garrido et al. (2016) designed the 4-degree of freedom (DOF) modular upper limb exoskeleton. As modular robots can be assembled into a variety of configurations to adapt to different requirements and working conditions, it has become a trend to design rehabilitation exoskeletons with soft and modular structures.
Coordination is the ability to maintain context and dependent relationships between different body segments or joints (Broome et al., 2019). To perform coordinated training, it is necessary to conduct a quantitative analysis of the coordination relationship of upper limb joints. There are some studies on the quantitative coordination analysis of upper limbs in the task space (van Dokkum et al., 2014; Johansson et al., 2017). However, as the upper limb is redundant, the coordination in the task space cannot guarantee that the joint space is coordinated as well (Tomita et al., 2017). Thus, joint coordination must be investigated. In previous studies, principal component analysis (Tang et al., 2019), artificial neural networks, and other methods (Fineman and Stirling, 2017) have been used to quantify the coordination of upper limb joints. With coordination quantification, exoskeletons can be controlled to assist human limbs by following the coordination law. In this way, patients can undergo more coordinated and natural training.
Accurate position control is the basis of various robot-assisted training modes. Pneumatic artificial muscles have been adopted in some rehabilitation robots. Model-based (Huang et al., 2018; Wang et al., 2019) and model-free methods (Tu et al., 2017; Giancarlo, 2019) have been proposed to tackle nonlinear and hysteretic characteristics. Considering the difficulties in PAM modeling, model-free methods have been proved to be more advantageous. During robot control, the purpose is to regulate the position and/or speed to assist human joints to achieve coordinated training (Proietti et al., 2016). Progress in kinematic coordination has been made in lower limb rehabilitation (Gui et al., 2017), such as space robots (Lu and Jia, 2019), dual-arm surgical robots (Wu et al., 2019), and bilateral robots (Miao et al., 2020), but there are few examples in upper limb rehabilitation. Brokaw et al. (2013) implemented a time-independent functional training mode on the ARMin III robot, to realize the coordinated training for stroke patients. Crocher et al. (2012) transformed the tracking problem from the task trajectory space to the joint velocity space to realize special coordination among joints. Proietti et al. (2017) also contributed to the coordinated control of upper limb robots.
The current coordinated control of robots is mainly based on force control. When the movement of joints deviates from the reference trajectory, the robot will generate a torque to drive the limb back to the desired trajectory. During this process, however, safety problems may be caused by the interaction force. In addition, the reference trajectories are usually a set of arbitrary curves without following the coordination laws. Maintaining the active intention of the patient while improving movement coordination is crucial in rehabilitation practice. The joint angles specified by one's own data can motivate participation of patients and generate personalized paths. Machine learning methods have been recently used in joint angle prediction. Gholami et al. (2020) chose the convolutional neural network (CNN), Feiyun et al. (2015) proposed the multiple linear regression autoregressive model and Kalman filter (MLRAR-KF), and Xie et al. (2020) took the artificial neural network (ANN), to predict the joint angles. The coordination relationship between different joints and the control compliance should be fully considered. The existing coordination trajectories cannot adapt to different subjects or rehabilitation stages (Deng et al., 2020; Li Z. et al., 2020), while the convolutional neural network-long short-term memory (CNN-LSTM) algorithm can extract complex features and analyze continuous data by combing the advantages of CNN and LSTM, which can be used to generate personalized coordination trajectories.
The purpose of this paper is to propose a coordinated path planning and impedance control method for the elbow and wrist joints with a modular elbow–wrist exoskeleton. A soft modular elbow–wrist exoskeleton was designed for stroke patients and the neural network was used to quantify the coordinated movement of the upper limb joints. Based on the trajectory tracking control, a coordinated control method of the elbow–wrist robot based on impedance adjustment was proposed to help patients perform active training safely and compliantly with the guidance of the coordination relationship. The main contribution includes the quantification of the coordination trajectories with CNN-LSTM from the data collected under ADL tasks and the control of a compliant exoskeleton for interactive coordinated training. The rest of this paper is organized as follows: Section Soft Modular Exoskeleton demonstrates the modular elbow–wrist exoskeleton. In Section Path Planning and Impedance Control for Coordinated Upper Limb Rehabilitation the proposed coordinated control method with a joint coordination model is presented. The coordination quantification and robot control experiments are conducted in Section Experiments and Results. The discussion and conclusion are presented in Section V.
Soft Modular Exoskeleton
Considering the movements of the elbow flexion/extension and wrist flexion/extension, the elbow angle is always greater than 0° and is equal to 180° when the arm is straight. When the palm and forearm are in a straight line, the wrist angle is equal to 0°. The wrist angle is defined as positive in flexion and negative in extension. The elbow range of motion is 30–180°, while the wrist is −30–50° for the common society. The designed elbow–wrist exoskeleton must meet such a range of motion.
The current rehabilitation robots have drawbacks in terms of their fixed structure and insufficient scalability. The modular structure can solve these problems well. Rehabilitation training using modular robots can be customized for different patients or different parts to improve the utilization of the robot. On the other hand, modular robots have distinct structures and are easier to assemble and maintain. Based on the reconfigurable concept, a modular elbow–wrist rehabilitation robot driven by PAMs was designed. The elbow/wrist modules and the reconfigured elbow–wrist exoskeleton are shown in Figure 1. A pulley was used to convert the linear motion of PAMs to the rotary motion of the joint. In the initial state, the PAMs contracted to half of the maximum contractible length. When the PAMs on both sides alternately inflate or deflate, the joint will begin to rotate under the pull rope at the end of the PAMs.
Figure 1. The designed soft modular elbow–wrist rehabilitation exoskeleton driven by PAMs, (A) the elbow module, (B) the wrist module, and (C) the reconfigured with modular parts.
The elbow–wrist rehabilitation robot is composed of an elbow module and a wrist module, and the splicing position is located at the upper arm connecting rod and forearm PAMs bracket. The wrist module realizes different degrees of freedom rehabilitation by replacing the corresponding sub-function modules. When the connecting bolt is installed, the elbow module and the wrist module are connected, and the robot as a whole can perform rehabilitation for the elbow and wrist joint of the patient at the same time. When disassembled, the elbow module and the wrist module are separated and independent of each other. The modular design also makes the device expandable and adjustable, which can suit the coordinated rehabilitation well with the changing paths and/or forces from different joints which can work separately or together. To ensure safety and comfort during rehabilitation, other parts of the elbow–wrist exoskeleton are made of flexible materials fabricated by a three-dimensional (3D) printer to reduce the weight.
Path Planning and Impedance Control for Coordinated Upper Limb Rehabilitation
Coordination Tasks and Data Acquisition
To quantify the coordination relationship of the upper limb in joint space, we selected two representative reach and grasped tasks with moderate range, as shown in Figures 2, 3: drinking water and touching the head. Detail Task 1: a cup is placed on the horizontal table in front of the subject, and the distance is about four-sevenths of the arm length. During the task, the subject is always sitting upright while the abdomen is at the same level as the desktop. In the initial state, the subject holds the cup with the right hand, then raises the cup to the mouth and completes the drinking, finally puts down the cup and returns to the initial state. Detail Task 2: During the whole process, the sitting posture of the subject is the same as in Task 1. In the initial state, the right hand of the subject is placed on the right thigh, and the palm is upward in a horizontal relaxation state. Then the right hand is raised, reaches, and touches the head. Finally, the arm returns to its initial state. There is no limit on the speed in both tasks and each subject should avoid redundant actions. All the subjects were right-handed and have no arm disease or joint injury. The information of the subjects is shown in Table 1. The studies involving the human participants were reviewed and approved by the Ethics Committee of the Wuhan University of Technology. The patients/participants provided their written informed consent to participate in this study.
The motion data of the shoulder, elbow, and wrist joints when completing the tasks were collected using the motion capture system (QUALISYS Miqus M5, Sweden) with 10 marker points as shown in Figure 4. Points A, B, and C are located on the back of the subject. Point B is at the right scapula, and point C is vertically lower than point B. Point D is located on the chest, Points E, F, and G are located on the upper arm, elbow, and forearm, respectively, and points H, I, and J are located on the forearm, wrist, and hand. Thus, the shoulder angle can be represented by the angles between BA and EF, BC and EF, and BD and EF, respectively, as shoulder angle I, shoulder angle II, shoulder angle III. The Elbow angle is represented by the angle between FE and FG, and the wrist angle is represented by the angle between HI and IJ.
Figure 4. Marker points in the motion capture system and the definition of the shoulder, elbow, and wrist angles.
Then, the angle of each joint can be calculated. Take the elbow as an example, assuming that the point E(x1, y1, z1), F(x2, y2, z2), and G(x3, y3, z3), then
Then the elbow angle is
Similarly, the shoulder and wrist angles can be calculated.
Coordination Relationship Quantification
Due to the differences in the subjects, the pre-defined trajectories might not follow the coordination laws. The personalized coordination trajectories are thus required to meet the individual training needs. Using the recorded motion data when completing coordination tasks, we can quantify the coordination relationship of each joint. The CNN is capable of extracting complex features from massive data, and the LSTM model is used to analyze continuous sequence data. In this paper, the CNN-LSTM model was adopted to quantify the coordination relationship of the upper limb joints. The diagram of the CNN-LSTM hybrid neural network is shown in Figure 5, with the network structure and the parameters properly designed or selected to improve the overall performance, including two convolution layers, two pooling layers, two dropout layers, two LSTM layers, and two full connection layers. The dropout layer can effectively suppress the neural network overfitting. The details of the parameters set of each layer are shown in Table 2. To quantify the shoulder-elbow coordination, the angle data of shoulder I, II, III, and the elbow were inputted to the CNN-LSTM model. After training the network parameters, the coordination relationship of the shoulder and elbow joints is established. Similarly, the coordination of the shoulder, elbow, and wrist can be achieved. Then, we can quantify the coordination of other subjects.
Path Planning
The quantified results were used to generate the coordinated trajectory of the upper limb joints. The target users of this method are patients with normal shoulder movements. The data from the shoulder joint of the patient can be used to generate more personalized and coordinated elbow and wrist trajectories for rehabilitation training. The quantification was operated offline to generate a virtual channel, which was common in rehabilitation practice as the online generation may cause unexpected arbitrary trajectories harmful to patients (Deng et al., 2020; Li Z. et al., 2020). During movement, the elbow and wrist angles can be predicted via the CNN-LSTM model which was then used to generate the coordination trajectories. Robot-assisted training paths should be planned to the motivate participation of the patient while following the coordination laws. Patients are allowed to deviate from the desired trajectory, but the range needs to be limited. In this study, the optimal reference path is the coordination trajectory generated by the quantification results, and the coordination channel is a range around the optimal trajectory. The robot trajectory can be corrected around the coordination path according to human contact force, but it cannot exceed the boundary of the virtual channel to the main overall coordination. As shown in Figure 6, such a path planning strategy can retain the initiative of the patient while ensuring that the motion is always coordinated.
Figure 6. The optimal coordination trajectory and the virtual channel for the robot control ensure the motivation and coordination of patients.
Position control is the basis for coordinated control. An improved multiple input single output model-free adaptive controller (MISO-IMFAC) is proposed for the trajectory tracking of the elbow–wrist exoskeleton. Instead of decomposing it into several single pneumatic muscles, MISO-IMFAC controls the robot as an unknown model to enhance the control robustness. The control scheme is as follows, and more details of the MISO-IMFAC can be found in our previous work (Li Y. et al., 2020).
where u(k) ∈ Rm and θ(k) ∈ R1 represent the input and output data at time k respectively. is the pseudo gradient (PG). The control law is obtained from (4), Equation (5) is for PG estimation, and Equation (6) is a method of PG reset. The method only applies the input and output data without modeling the PAMs, which is not sensitive to the nonlinear and time-varying characteristics of the system, so the control algorithm has strong adaptability and robustness.
Robot Impedance Control
The position-based impedance control obtains the desired trajectory by mapping the error between the desired force and the actual force and corrects the final path, as illustrated in Figure 7. The impedance model can be expressed as (7).
where Md, Bd, Kd represent inertia parameters, damping, and stiffness. θ, , represent the actual angle, angular velocity, and angular acceleration of the robot, and θd, , are the desired angle, angular velocity, and angular acceleration. Fd is the desired contact force, and F is the actual contact force.
Generally for coordinated path control, the desired contact force can be assumed as Fd = 0, Equation (7) can be written as:
Figure 7 shows the position-based impedance (admittance) control scheme for the exoskeleton with the PAM control setup. The position correction is generated from the force loop and the original trajectory was corrected by the human interaction. Once the corrected path was received from the impedance model, the MISO-IMFAC algorithm was adopted to calculate the required displacement and control command of each PAM. Then the PAM can be controlled by tuning the air pressure inside via proportional valve regulators (VPPM-6L-G18-0L6H, FESTO, Germany) to drive the robot to track the corrected path. More details of the PAM control setup on a testing platform can be found in our previous work (Ai et al., 2020).
In this study, the impedance control scheme was designed with the following hypothesis considering both coordination laws and the active motivation of patients. When the movement is following the optimal coordination path, the robot can be set highly compliant, so the active intention of the patient can directly correct the original path. With the increase of the distance from the optimal path, the stiffness should gradually increase, to ensure that the robot will always operate within the coordination channel. The inertia and damping parameters also need to be adjusted accordingly. To achieve this goal, assuming that the object is in the artificial potential field (APF), the target must have an attraction to the object while the obstacle must have repulsive force. Regarding the channel boundary as the obstacle and the coordination trajectory as the target, then the impedance parameter at a certain position can be adjusted with the APF method, to encourage the robot to follow the optimal path while maintaining the active freedom of the patient.
The potential function at a point can be expressed as:
where U(q) is the potential function, Uatt(q) is the gravitational potential function, Urep(q) represents the repulsive potential function, and
where ς and η represent gain parameter, d(q, qgoal) and D(q) is the distance between point q and the target and the nearest obstacle respectively. Q* is the obstacle threshold.
The adjustment of impedance parameters with APF is:
where Kd, Bd, Md represent the stiffness, damping, and inertia parameters and Kd0, Bd0, Md0 are their initial values. The positive weight factors are represented by c1, c2 and c3.
The coordination controller of the elbow–wrist robot was designed as shown in Figure 8, which contains three parts: the coordination quantifier to generate the initial path, the impedance parameters adjuster to constrain the motion within the coordination channel, and the position controller to track the final path corrected by human contact force. For specific coordination training tasks, the elbow–wrist angle was generated via the CNN-LSTM model as the originally desired trajectory, i.e., the optimal coordination path. When patients exert contact force, the expected trajectory will be corrected by the impedance/admittance model tuned by the APF. Then MISO-IMFAC algorithm was used to control the robot to follow the corrected trajectory. Considering the range and compliance of the coordination channel, the APF scheme is properly set to adjust the impedance parameters of the robot to ensure the motivation and coordination of the patient during training.
Figure 8. Diagram of the proposed coordinated controller with the coordination quantification, impedance adjustment, and position control.
Experiments and Results
Coordination Relationship Quantification
For the coordination training tasks, all the subjects wore tight clothes to prevent the change of marker positions during movement. The sampling rate of the motion capture system was 100 Hz. The experimental setup of the robotic exoskeleton and the subject wearing the exoskeleton is demonstrated in Figure 9 for the robot-assisted training. The exoskeleton has modular elbow and wrist parts that can work together or separately to meet various training requirements. Drinking water and touching the head are the two selected typical and representative ADL tasks for patient training. Each subject was instructed to perform a task within 1 min, as a trial. Each of the two tasks will be performed with four trials and a break will be taken between the two tasks. These two tasks will be repeated several times to collect enough training data on the different subjects to verify the model prediction effectiveness.
Figure 9. Experiment setup of the robotic exoskeleton and robot-assisted subject training: (a) the developed modular exoskeletons, (b) a subject wearing the elbow–wrist exoskeleton with modules working together, (c) a subject wearing the elbow exoskeleton, and (d) the wrist exoskeleton.
The collected movement data can reflect the joint angle changes during the robot-assisted movement. Figure 10 shows the changes of the shoulder, elbow, and wrist angles of subject S1 under Task 1, in which the time has been normalized. The elbow angle is large initially and gradually decreases and reaches the minimum while the wrist angle increases first and then decreases. The angle changes of the shoulder I and III are similar to the elbow, and shoulder II is similar to the wrist. This is consistent with our daily activity experience. The coordination quantification of the upper limb can be reflected by the movement data, based on which we can extract the coordination features and generate coordinated elbow–wrist trajectories. Figure 11 gives an example of the coordination relationship trained from the CNN-LSTM model.
Figure 10. The changes of the shoulder, elbow, and wrist angles of subject S1 under Task 1: drinking water.
Figures 12, 13 show the joint angle predicted results of different subjects in Task 1 and Task 2, respectively. From the results, though the actual recorded trajectories were easier to obtain and were useful for certain fixed tasks, they have limited effects on the coordinated elbow and wrist training. First, the movements of the elbow and wrist joints of stroke patients are usually abnormal (Bilić et al., 2001; Squeri et al., 2014), so we cannot obtain the recorded trajectories. Even if the elbow–wrist trajectories of the patients are recorded, the coordination between these joints cannot be guaranteed. Thus, a prediction model is required to generate coordinated paths by imitating the movements of normal people. The recorded trajectories of different upper limbs for a specific task are similar in tendency, due to the differences among subjects with various physical parameters or recovery stages, the coordination trajectories are different among subjects. While the adaptability and personality of the coordinated paths can be ensured by the CNN-LSTM using the shoulder data of the patient. The elbow angle predicted by CNN-LSTM has a high coincidence with the actual angle and the results of four subjects with two tasks verified this. In addition, the recorded paths can only suit certain tasks while the quantification model has potential for other tasks.
The current coordination trajectories tend to use pre-defined analytic expressions to describe the coordination of the elbow and wrist joints (Soltani et al., 2017; Ballesteros-Escamilla et al., 2019). However, pre-defined coordination trajectories, similar to recorded trajectories, cannot adapt to different subjects or tasks. In addition, the analytic methods usually plane trajectory in the task space and then calculated the joint motions, which cannot fully reflect the coordination in joint space. In comparison, the CNN-LSTM can be used to directly quantify the coordination relationship in the joint space and possesses adaptability to different patients. Compared with the back propagation (BP) model, the CNN-LSTM model was evaluated by R2 and mean square error (MSE). Figure 14 shows the results of two network models for Tasks 1 and 2, in which S1E represents the elbow of subject S1 and S1W the wrist. It can be seen that the R2 of CNN-LSTM is greater than that of BP for both tasks and has smaller MSE errors. Taking subject S1 in Task1 as an example, based on the CNN-LSTM model and BP model, the average R2 of the elbow joint was 0.926 and 0.911, respectively, while the wrist was 0.848 and 0.807. When using the CNN-LSTM and BP models, the average MSE of the elbow was 39.842 and 47.834, while the wrist was 10.492 and 13.398, respectively. Considering the difference in the range of motion, the errors were normalized. The relative elbow errors of subject S1 in Task1 are 0.007 and 0.009, respectively.
Figure 14. Comparison of the effectiveness of the upper limb joints prediction using the CNN-LSTM and BP models under Tasks 1 and 2.
Coordinated Control
An APF-based impedance adjustment scheme was proposed for the coordinated control of the soft elbow–wrist exoskeleton. Each subject was asked to wear the elbow–wrist exoskeleton to carry out coordinated experiments with the two tasks. The optimal coordination trajectories of elbow and wrist joints were generated by the CNN-LSTM model. Considering the range of the position control and the requirement of the joint coordinated motions, the width of the elbow and wrist coordination channels were set to 10° and 6° in Task 1, and 10° and 3° in Task 2, respectively. Figure 15 shows the coordinated control results in Task 1, in which the impedance effects were reflected by comparing the contact force with the resulted training paths.
Figure 15. Results of the coordination control with human contact force under Task 1, lines indicating the coordination channel, the optimal trajectory, the corrected trajectory (via impedance control), and the actual tracking trajectory (via MISO-IMFAC).
During movement, the subjects can exert arbitrary forces on the exoskeleton robot. The human contact force (top line) directly corrects the optimal trajectory via adaptive impedance control and the actual path (bottom red line) can timely track the desired (bottom blue line) via the MISO-IMFA control, with a partially enlarged view to show the results more clearly. As MISO-IMFAC is a data-driven control method and does not rely on the system model, the PAM hysteresis caused by the change in impedance or admittance did not have a large impact on the control response. Similarly, the experimental results of Task 2 are shown in Figure 16, in which the corrected and actual trajectories were constrained to the virtual coordinated channel. With the human active interaction, the original trajectory was corrected by the adaptive impedance controller. The adopted APF method regards the boundary of the coordinated channel as an obstacle and the optimal trajectory as the target to generate a virtual channel in space. The impedance control based on APF adjustment can correct the desired trajectory according to the active intention of the patient and always constrain it within the scope of the coordination channel to ensure rehabilitation safety and effectiveness. Consequently, both elbow and wrist joints were within the coordination channel and the exoskeleton adaptively assisted subjects to complete the interactive coordinated training tasks.
Figure 16. Results of the coordination control with human contact force under Task 2, lines indicating the coordination channel, the optimal trajectory, the corrected trajectory (via impedance control), and the actual tracking trajectory (via MISO-IMFAC).
To verify the superiority of the proposed coordinated control method, a comparative experiment was carried out under Task 1. Compared with the common impedance controller with fixed parameters, the results shown in Figure 17 indicate that the actual trajectory might be out of the range of the coordination channel, e.g., the path in the black rectangle. As the contact force increases, the actual trajectory approaches the boundary and finally exceeds the range. In comparison, the robot under the proposed impedance control can always move within the coordination channel, as the parameters are adaptively tuned by the APF scheme, which can drive the robot to move toward the optimal path and away from the boundary. The robot movement can be compliantly and interactively corrected by human contact force while always in a virtual tunnel to keep coordination with the joint space, thus the rehabilitation can be conducted within the coordination channel while maintaining the active motivation of the patient and ensuring movement coordination and safety.
Figure 17. Control results without impedance adjustment under Task 1, the actual trajectory might be out of the coordination channel.
Discussion and Conclusion
Robot-assisted upper limb training can certainly enhance rehabilitation efficiency and safety, which is important for the recovery of stroke patients. The purpose of coordinated control is to ensure that the joints of the patients can cooperate with each other to complete a delicate task. The coordinated control of the rehabilitation robot can help realize the coordinated training of the upper limb of patients, so how to identify coordination relationships needs to be tackled. Extracting knowledge of the coordination of human joints during movement and applying this relationship to the coordinated control can improve the effectiveness of rehabilitation (Dounskaia et al., 2020). Meanwhile, the modular design makes the robot expandable for different rehabilitation purposes and reconfigurable for various users. When the robot modules are coordinately controlled, the rehabilitation training will become more physiologically meaningful.
Considering the limitations of quantification in the task space, we adopted the quantitative method in the joint space. Two kinds of coordinated tasks were designed, and the data of the shoulder, elbow, and wrist of four subjects were collected by a motion capture system. The angle changes of the upper limb joints under different tasks are obtained, to generate the patient-specific coordinated trajectories. Though the tendency of angle change is similar when completing the same task, there are big differences among subjects in terms of the trajectory range and speed, so the CNN-LSTM model was designed to generate patient-specific coordination trajectories to be determined by the uniqueness of an individual. In the late stage of rehabilitation when the shoulder joint of the patient has returned to normal, one's own shoulder data will be taken into account to generate personalized and coordinated paths. The value of the CNN-LSTM algorithm is reflected by its ability to quantify the coordination of the joints of different subjects so that the generated coordination trajectories are more physiologically meaningful, and also enhances the adaptability of the trajectory to different patients. The effectiveness and advantages of the CNN-LSTM method were presented by comparing it with the BP algorithm and other related studies (Deng et al., 2020; Li Z. et al., 2020). Results show that CNN-LSTM can generate higher correlation coefficients for both the tasks and four subjects, indicating its better performance. The exercise sets can be specific ADL tasks to improve independent living abilities, and CNN-LSTM can guarantee the adaptability of the trajectory to different patients.
An impedance adjustment strategy was then proposed and a coordinated controller including an impedance/admittance model was designed for the modular robot. Different from previous studies, the impedance control was realized on the elbow–wrist exoskeleton within a coordinated channel to ensure the coordination and compliance of the rehabilitation training. Path planning based on coordination relationship ensures that the robot is always in the virtual coordination channel. The impedance adjustment scheme based on APF was designed to correct the original coordination trajectory according to the active intention of the patient and always constrain it within the scope of the coordination channel. The proposed adaptive impedance controller can keep the final movement path within the coordination channel by adopting the APF tuning scheme to ensure the safety and effectiveness of the coordinated rehabilitation. The effectiveness and advantages of the APF-based impedance control method were presented by comparing it with common impedance controller and some existing studies (Erol and Sarkar, 2008; Li et al., 2019a,b), results showed that the path under common methods might be out of the coordination channel, while the robot under the proposed impedance control can always move within the coordination channel corrected by the human contact force with the parameters adaptively tuned by the APF scheme. Consequently, the rehabilitation can be conducted while maintaining the active motivation of the patient and ensuring movement coordination and safety.
In this study, a CNN-LSTM model was designed to plan coordinated trajectories using data from specific ADL tasks, and an APF-based adaptive impedance controller was proposed for coordinated training within a virtual tunnel to meet the personalized training needs. This work can accurately plan the patient-specific coordination trajectory of the elbow and wrist joints and then correct the actual path according to the active intention of the patient within a coordination channel. The MISO-IMFAC method was adopted to realize the position tracking control of the soft modular exoskeleton. A quantification method was proposed for multi-joint coordination to represent the interrelationship of the upper limb during movement, and a novel coordinated control method in cooperating adaptive impedance model of the elbow–wrist exoskeleton was proposed for coordinated training. The experimental results showed that the CNN-LSTM model can quantify the joint relationship and predict the elbow and wrist joint data well, which is verified on four different subjects with two representative ADL tasks, i.e., drinking water and touching the head. During the coordinated control, the initiative of the subject was retained and the quantitative coordinated relationship can be reflected by the planned path. With the robot assistance, the subject can move around the coordinated trajectory within the virtual channel, ensuring training safety and coordination.
The limitations of this work include that we just verified the effectiveness of the proposed method on healthy subjects and the patients with elbow–wrist disorders but normal shoulder movements will be recruited and the results will be reported in the near future. Only two typical ADL tasks were selected for the coordination quantification and coordinated control experiment. The CNN-LSTM model with adaptability can potentially be used on more ADL tasks, but we have not verified it in this study. In the future, we will add other rehabilitation tasks to further verify the adaptability of the model and to make rehabilitation training more diverse. The CNN-LSTM model design can also be further improved by integrating with patient-specific parameter settings and structure optimization. In addition, physiological signals can accurately reflect the intention of the patient, so the combination of electromyogram (EMG) and electroencephalogram (EEG), with the robot is also the focus of the next stage work.
Data Availability Statement
The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author/s.
Ethics Statement
The studies involving human participants were reviewed and approved by Ethics Committee of Wuhan University of Technology. The patients/participants provided their written informed consent to participate in this study.
Author Contributions
QL, YLiu, and YLi participated in the design of methods, data collection and analysis, and drafted the manuscript. WM participated in the design of the method and participated in the idea selection. CZ helped in the data extraction and manuscript revision. QA and SX participated as a supervisor and modified the manuscript. All authors read and approved the final manuscript.
Funding
This work was supported by the National Natural Science Foundation of China (52075398), the Research Project of Wuhan University of Technology Chongqing Research Institute (YF2021-17), and the Application Foundation Frontier Project of Wuhan S&T Program (2020020601012220).
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnbot.2021.745531/full#supplementary-material
References
Ai, Q., Ke, D., Zuo, J., Meng, W., Liu, Q., Zhang, Z., et al. (2020). High-order model-free adaptive iterative learning control of pneumatic artificial muscle with enhanced convergence. IEEE Trans. Indus. Electr. 67, 9548–9559. doi: 10.1109/TIE.2019.2952810
Ballesteros-Escamilla, M., Cruz-Ortiz, D., Salgado, I., and Chairez, I. (2019). Hybrid position/force output feedback second-order sliding mode control for a prototype of an active orthosis used in back-assisted mobilization. Med. Biol. Eng. Comput. 57, 1843–1860. doi: 10.1007/s11517-019-01987-y
Bilić, R., Kolundzić, R., and Jelić, M. M. (2001). Overuse injury syndromes of the hand, forearm and elbow. Arh. Hig. Rada Toksikol. 52, 403–414.
Brokaw, E. B., Holley, R. J., and Lum, S. P. (2013). Comparison of joint space and end point space robotic training modalities for rehabilitation of interjoint coordination in individuals with moderate to severe impairment from chronic stroke. IEEE Trans. Neural Syst. Rehabil. Eng. 21, 787–795. doi: 10.1109/TNSRE.2013.2238251
Brokaw, E. B., Nichols, D., Holley, R. J., and Lum, S. P. (2014). Robotic therapy provides a stimulus for upper limb motor recovery after stroke that is complementary to and distinct from conventional therapy. Neurorehabil. Neural Repair 28, 367–376. doi: 10.1177/1545968313510974
Broome, K., Hudson, I., Potter, K., Kulk, J., Dunn, A., Arm, J., et al. (2019). A modified reach-to-grasp task in a supine position shows coordination between elbow and hand movements after stroke. Front. Neurol. 10:408. doi: 10.3389/fneur.2019.00408
Carpinella, I., Lencioni, T., Bowman, T., Bertoni, R., Turolla, A., Ferrarin, M., et al. (2020). Effects of robot therapy on upper body kinematics and arm function in persons post stroke: a pilot randomized controlled trial. J. Neuroeng. Rehabil. 17:10. doi: 10.1186/s12984-020-0646-1
Crocher, V., Sahbani, A., Robertson, J., Roby-Brami, A., and Morel, G. (2012). Constraining upper limb synergies of hemiparetic patients using a robotic exoskeleton in the perspective of neuro-rehabilitation. IEEE Trans. Neural Syst. Rehabil. Eng. 20, 247–257. doi: 10.1109/TNSRE.2012.2190522
Deng, M., Li, Z., Kang, Y., Chen, C. L. P., and Chu, X. (2020). A learning-based hierarchical control scheme for an exoskeleton robot in human-robot cooperative manipulation. IEEE Trans. Cybernet. 50, 112–125. doi: 10.1109/TCYB.2018.2864784
Dindorf, R., and Wos, P. (2020). Using the bioelectric signals to control of wearable orthosis of the elbow joint with bi-muscular pneumatic servo-drive. Robotica 38, 804–818. doi: 10.1017/S0263574719001097
Dounskaia, N., Shimansky, Y., Ganter, B. K., and Vidt, E. M. (2020). A simple joint control pattern dominates performance of unconstrained arm movements of daily living tasks. PLoS ONE 15:e0235813. doi: 10.1371/journal.pone.0235813
Erol, D., and Sarkar, N. (2008). Coordinated control of assistive robotic devices for activities of daily living tasks. IEEE Trans. Neural Syst. Rehabil. Eng. 16, 278–285. doi: 10.1109/TNSRE.2008.922668
Feiyun, X., Gao, Y., Shengxin, W., and Zhao, J. (2015). “Prediction of joint angle by combining multiple linear regression with autoregressive (AR) model and Kalman Filter,” in IEEE Conference on Robotics and Biomimetics (Zhuhai), 1139–1144. doi: 10.1109/ROBIO.2015.7418925
Fineman, A. R., and Stirling, A. L. (2017). Quantification and visualization of coordination during non-cyclic upper extremity motion. J. Biomech. 63, 82–91. doi: 10.1016/j.jbiomech.2017.08.008
Garrido, J., Yu, W., and Li, X. (2016). Modular design and control of an upper limb exoskeleton. J. Mech. Sci. Technol. 30, 2265–2271. doi: 10.1007/s12206-015-0843-1
Ghobj, S., Akl, A., El-Farr, A., Ayyash, M., and Abu-Khalaf, J. (2017). “Mechanical design for a cable driven upper limb exoskeleton prototype actuated by pneumatic rubber muscles,” in 2017 International Conference on Research and Education in Mechatronics (REM) (Wolfenbuettel: IEEE), 1–7. doi: 10.1109/REM.2017.8075232
Gholami, M., Napier, C., and Menon, C. (2020). Estimating lower extremity running gait kinematics with a single accelerometer: a deep learning approach. Sensors 20:2939. doi: 10.3390/s20102939
Giancarlo, P. V. (2019). Control strategy of a pneumatic artificial muscle for an exoskeleton application. IFAC-PapersOnLine 52, 281–286. doi: 10.1016/j.ifacol.2019.06.075
Gui, K., Liu, H., and Zhang, D. (2017). “A generalized framework to achieve coordinated admittance control for multi-joint lower limb robotic exoskeleton,” in 2017 International Conference on Rehabilitation Robotics (ICORR), IEEE, 228–233. doi: 10.1109/ICORR.2017.8009251
Hsieh, Y.-w., Lin, K.-c., Wu, C.-y., Shih, T.-y., Li, M.-w., and Chen, C.-l. (2018). Comparison of proximal versus distal upper-limb robotic rehabilitation on motor performance after stroke: a cluster controlled trial. Sci. Rep. 8, 1–11. doi: 10.1038/s41598-018-20330-3
Huang, J., Cao, Y., Xiong, C., and Zhang, H.-T. (2018). An echo state Gaussian process-based nonlinear model predictive control for pneumatic muscle actuators. IEEE Trans. Automat. Sci. Eng. 16, 1071–1084. doi: 10.1109/TASE.2018.2867939
Johansson, G. M., Grip, H., Levin, M. F., and Häger, K. C. (2017). The added value of kinematic evaluation of the timed finger-to-nose test in persons post-stroke. J. Neuroeng. Rehabil. 14, 1–12. doi: 10.1186/s12984-017-0220-7
Li, G., Cai, S., and Xie, L. (2019b). “Cooperative control of a dual-arm rehabilitation robot for upper limb physiotherapy and training,” in IEEE/ASME International Conference on Advanced Intelligent Mechatronics (Hong Kong: IEEE), 278–285. doi: 10.1109/AIM.2019.8868784
Li, G., Fang, Q., Xu, T., Zhao, J., Cai, H., and Zhu, Y. (2019a). Inverse kinematic analysis and trajectory planning of a modular upper limb rehabilitation exoskeleton. Technol. Health Care 27, 123–132. doi: 10.3233/THC-199012
Li, Y., Liu, Q., Meng, W., Xie, Y., Ai, Q., and Xie, Q. S. (2020). “MISO model free adaptive control of single joint rehabilitation robot driven by pneumatic artificial muscles,” in 2020 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM) (Boston, MA: IEEE), 1700–1705. doi: 10.1109/AIM43001.2020.9158805
Li, Z., Zuo, W., and Li, S. (2020). Zeroing dynamics method for motion control of industrial upper-limb exoskeleton system with minimal potential energy modulation. Measurement 163:107964. doi: 10.1016/j.measurement.2020.107964
Liu, Y., Song, Q., Li, C., Guan, X., and Ji, L. (2020). Quantitative assessment of motor function for patients with a stroke by an end-effector upper limb rehabilitation robot. BioMed. Res. Int. 2020, 1–14. doi: 10.1155/2020/5425741
Lu, X., and Jia, Y. (2019). Adaptive coordinated control of uncertain free-floating space manipulators with prescribed control performance. Nonlin. Dyn. 97, 1541–1566. doi: 10.1007/s11071-019-05071-w
Miao, Q., Zhang, M., McDaid, A., Peng, Y., and Xie, Q. S. (2020). A robot-assisted bilateral upper limb training strategy with subject-specific workspace: a pilot study. Robot. Autonom. Syst. 124:103334. doi: 10.1016/j.robot.2019.103334
Proietti, T., Crocher, V., Roby-Brami, A., and Jarrasse, N. (2016). Upper-limb robotic exoskeletons for neurorehabilitation: a review on control strategies. IEEE Rev. Biomed. Eng. 9, 4–14. doi: 10.1109/RBME.2016.2552201
Proietti, T., Guigon, E., Roby-Brami, A., and Jarrassé, N. (2017). Modifying upper-limb inter-joint coordination in healthy subjects by training with a robotic exoskeleton. J. Neuroeng. Rehabil. 14, 1–19. doi: 10.1186/s12984-017-0254-x
Saita, K., Morishita, T., Hyakutake, K., Ogata, T., Fukuda, H., Kamada, S., et al. (2020). Feasibility of robot-assisted rehabilitation in poststroke recovery of upper limb function depending on the severity. Neurol. Medico Chirurgica 60, 217–222. doi: 10.2176/nmc.oa.2019-0268
Santos, d. W., Nogueira, M., Oliveira, S. L. d. G., Peña, C. G. G., and Siqueira, G. A. A. (2017). “Design and evaluation of a modular lower limb exoskeleton for rehabilitation,” in IEEE International Conference on Rehabilitation Robotics (London: IEEE), 447–451. doi: 10.1109/ICORR.2017.8009288
Soltani, R., Zeiaee, A., Langari, R., and Robson, N. (2017). “Reference path generation for upper-arm exoskeletons considering scapulohumeral rhythms,” in 2017 International Conference on Rehabilitation Robotics (ICORR). IEEE, 753–758. doi: 10.1109/ICORR.2017.8009338
Squeri, V., Masia, L., Giannoni, P., Sandini, G., and Morasso, P. (2014). Wrist rehabilitation in chronic stroke patients by means of adaptive, progressive robot-aided therapy. IEEE Trans. Neural Syst. Rehabil. Eng. 22, 312–325. doi: 10.1109/TNSRE.2013.2250521
Tang, S., Chen, L., Barsotti, M., Hu, L., Li, Y., Wu, X., et al. (2019). Kinematic synergy of multi-DoF movement in upper limb and its application for rehabilitation exoskeleton motion planning. Front. Neurorobot. 13:99. doi: 10.3389/fnbot.2019.00099
Tomita, Y., Rodrigues, M. R., and Levin, F. M. (2017). Upper limb coordination in individuals with stroke: poorly defined and poorly quantified. Neurorehabil. Neural Repair 31, 885–897. doi: 10.1177/1545968317739998
Tu, X., Zhou, X., Li, J., Su, C., Sun, X., Han, H., et al. (2017). Iterative learning control applied to a hybrid rehabilitation exoskeleton system powered by PAM and FES. Clust. Comput. 20, 2855–2868. doi: 10.1007/s10586-017-0880-x
van Dokkum, L., Hauret, I., Mottet, D., Froger, J., Métrot, J., and Laffont, I. (2014). The contribution of kinematics in the assessment of upper limb motor recovery early after stroke. Neurorehabil. Neural Repair 28, 4–12. doi: 10.1177/1545968313498514
Wang, T., Chen, X., and Qin, W. (2019). A novel adaptive control for reaching movements of an anthropomorphic arm driven by pneumatic artificial muscles. Appl. Soft Comput. 83:105623. doi: 10.1016/j.asoc.2019.105623
Wu, Q., Li, M., Qi, X., Hu, Y., Li, B., and Zhang, J. (2019). Coordinated control of a dual-arm robot for surgical instrument sorting tasks. Robot. Autonom. Syst. 112, 1–12. doi: 10.1016/j.robot.2018.10.007
Xie, H., Li, G., Zhao, X., and Li, F. (2020). Prediction of limb joint angles based on multi-source signals by GS-GRNN for exoskeleton wearer. Sensors 20:1104. doi: 10.3390/s20041104
Zhang, K., Chen, X., Liu, F., Tang, H., Wang, J., and Wen, W. (2018). System framework of robotics in upper limb rehabilitation on poststroke motor recovery. Behav. Neurol. 2018, 1–14. doi: 10.1155/2018/6737056
Keywords: path planning, rehabilitation robot, impedance control, coordinated rehabilitation, soft exoskeleton
Citation: Liu Q, Liu Y, Li Y, Zhu C, Meng W, Ai Q and Xie SQ (2021) Path Planning and Impedance Control of a Soft Modular Exoskeleton for Coordinated Upper Limb Rehabilitation. Front. Neurorobot. 15:745531. doi: 10.3389/fnbot.2021.745531
Received: 22 July 2021; Accepted: 27 September 2021;
Published: 01 November 2021.
Edited by:
Dingguo Zhang, University of Bath, United KingdomReviewed by:
Xikai Tu, North Carolina State University, United StatesNingbo Yu, Nankai University, China
Copyright © 2021 Liu, Liu, Li, Zhu, Meng, Ai and Xie. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Wei Meng, weimeng@whut.edu.cn