An Adaptive and Hybrid End-Point/Joint Impedance Controller for Lower Limb Exoskeletons

Assist-as-needed (AAN) algorithms for the control of lower extremity rehabilitation robots can promote active participation of patients during training while adapting to their individual performances and impairments. The implementation of such controllers requires the adaptation of a control parameter (often the robot impedance) based on a performance (or error) metric. The choice of how an adaptive impedance controller is formulated implies different challenges and possibilities for controlling the patient's leg movement. In this paper, we analyze the characteristics and limitations of controllers defined in two commonly used formulations: joint and end-point space, exploring especially the implementation of an AAN algorithm. We propose then, as a proof-of-concept, an AAN impedance controller that combines the strengths of working in both spaces: a hybrid joint/end-point impedance controller. This approach gives the possibility to adapt the end-point stiffness in magnitude and direction in order to provide a support that targets the kinematic deviations of the end-point with the appropriate force vector. This controller was implemented on a two-link rehabilitation robot for gait training—the Lokomat^®Pro V5 (Hocoma AG, Switzerland) and tested on 5 able-bodied subjects and 1 subject with Spinal Cord Injury. Our experiments show that the hybrid controller is a feasible approach for exoskeleton devices and that it could exploit the benefits of the end-point controller in shaping a desired end-point stiffness and those of the joint controller to promote the correct angular changes in the trajectories of the joints. The adaptation algorithm is able to adapt the end-point stiffness based on the subject's performance in different gait phases, i.e., the robot can render a higher stiffness selectively in the direction and gait phases where the subjects perform with larger kinematic errors. The proposed approach can potentially be generalized to other robotic applications for rehabilitation or assistive purposes.

Introduction

Exoskeletons for gait rehabilitation or walking assistance in subjects with neurological injuries have flourished in the last decades (Esquenazi et al., 2017). These devices seek to control the leg segments of the user and try to restore a gait pattern that is both physiological (i.e., following kinematic characteristics observed in non-impaired individuals) and safe. An effective rehabilitation device should not only control the movements of a patient's legs, but should also challenge the patient and promote his active participation (Lotze et al., 2003; Hogan et al., 2006). One way to achieve the latter is through adaptation of the robotic support based on the user's capabilities (Cai et al., 2006; Marchal-Crespo and Reinkensmeyer, 2009). This concept is known as Assist-As-Needed (AAN) (Emken et al., 2005).

The simplest and most common method of modifying the level of robotic support is with impedance controllers, where impedance is defined as any dynamic operator that outputs a force (or a torque) from a kinematic input (e.g., displacement, velocity) (Hogan, 1985). In most available exoskeletons, the impedance parameters must be manually adapted by therapists based on their experience [e.g., Lokomat^® (Hocoma AG, Switzerland) (Colombo et al., 2000), LOPES II (University of Twente, The Netherlands) (Meuleman et al., 2016), HAL (Cyberdyne Inc., Japan) (Nilsson et al., 2014)]. New controllers that automatically adapt the impedance of the joints based on the user's performance have been proposed (Emken et al., 2008; Koopman et al., 2013; Maggioni et al., 2015), but not extensively implemented due to safety requirements. The use of adaptive control algorithms increases the compliance of the exoskeleton to the user's movements. Too much compliance (i.e., a low mechanical impedance) can lead to unsafe conditions because support may not be provided against users' errors, which may lead to tripping and injuries. The challenge in implementing adaptive controllers in lower limb exoskeletons is to find an appropriate trade-off between compliance (i.e., freedom of movement) and safety.

The choice of how an adaptive impedance controller is formulated inevitably determines how complex it is to address any potential hazard situation arising from reduced impedance. Here, we analyze the characteristics and limitations of controllers defined in two commonly used formulations: joint and end-point space, exploring especially the implementation of AAN controllers in these two spaces. A comparative analysis of these two approaches has been reported for industrial manipulators (Smith et al., 2014, 2015) but, to the best of our knowledge, such comparison has not been extensively examined within the context of rehabilitation robotics and even less in lower limb applications.

After analyzing the properties of these two approaches for the control of lower limb exoskeletons, we propose an AAN impedance controller that combines the strengths of working in both spaces: a hybrid joint/end-point impedance controller. This controller gives the possibility to adapt the end-point stiffness in magnitude and direction, to provide a support that targets end-point deviations with the appropriate force vector. This controller was implemented and tested on a two-link rehabilitation robot for gait training with actuated hip and knee joints—the Lokomat^®Pro v5 (Hocoma AG, Switzerland). We present the proof-of-concept for this hybrid controller based on simulations and tests conducted with five able-bodied subjects and one subject with walking impairment due to a complete spinal cord injury. The proposed approach can potentially be generalized to other robotic applications for rehabilitation or assistive purposes.

Joint vs. End-Point Space Formulations

Background Concepts

In this paper we analyze the implications of using joint or end-point space formulation for the control of lower limb exoskeletons. We model these systems as two-segment exoskeletons with a shank and a thigh segment. In the swing phase, the system can be modeled as a two-segment pendulum: the upper segment is fixed to the hip center of rotation (CoR) and the end-point corresponds to the ankle position (Kuo and Donelan, 2010). In the stance phase, the model is an inverse two-segment pendulum: after heel contact, the foot can only be moved backwards by the treadmill, hence the end-point of the kinematic chain is the hip CoR and not the ankle joint (Figure 1).

FIGURE 1

Figure 1. Comparison between the swing (Left) and stance (Right) models. During the swing phase, the ankle constitutes the end-point of the kinematic chain. During the stance phase, the end-point is the hip CoR. The force generated by the treadmill acts at the ankle, whose position is constrained in the vertical direction by the treadmill.

To analyze the impedance properties of the joint and end-point control approaches for the control of lower limb exoskeletons, we present the impact that these two formulations have on the end-point stiffness. The stiffness can be visualized as an ellipse, whose major axis indicates the direction of maximum stiffness (Mussa-Ivaldi et al., 1985; Shadmehr, 1993). The stiffness ellipse captures the geometrical features of the force field around a reference position of the end-point. In the force field representation, we can visualize the direction and magnitude of the restoring forces for displacements around the reference trajectory. For further details on the calculation of stiffness ellipses and force field, see the Appendix 1.

Impedance Control Based on Joint Space Formulation

In most exoskeleton devices, the actuators control the flexion and extension of the robotic joints, which roughly align to the human joints. Therefore, it is common to implement impedance controllers that compute the actuators' torques in order to follow reference trajectories defined in joint space (e.g., hip and knee angles). Furthermore, instrumented gait analysis increased our familiarity with angular kinematics and kinetics of the human joints.

A joint controller can be applied both in the stance and the swing phase of gait, because the actual joint trajectory q_act and the reference trajectory q_ref are defined continuously during the whole gait cycle and do not depend on the kinematic configuration (e.g., open chain in swing phase or closed chain in double-support phase). A joint space formulation avoids problems that might arise from inverse kinematics/dynamics calculations, especially in kinematic configurations (specific combinations of hip/knee angles) where the Jacobian matrix is singular.

For a two-link exoskeleton robot, the joint reference trajectory can be expressed as q_ref = [q_hip, q_knee], while q_act refers to the measured angles while the subject is walking. The torques τ_q to control the robotic actuators are provided by a motion controller with stiffness K_q = [K_hip 0; 0 K_knee] and damping B_q = [B_hip 0; 0 B_knee] (Equation 1).

$\begin{array}{ll}{\tau }_q={\text{K}}_q({\text{q}}_{ref}-{\text{q}}_{act})+{\text{B}}_q({\stackrel{·}{\text{q}}}_{ref}-{\stackrel{·}{\text{q}}}_{act})& (1)\end{array}$

Generally, in addition to the control torques τ_q, robotic exoskeletons have a separate component τ_comp, which compensates the inherent robot dynamics such as gravity, friction or inertia (e.g., Riener et al., 2005; Vallery et al., 2009).

Selection of Joint Reference Trajectories

Joint reference trajectories q_ref can be taken from literature (e.g., Winter, 1991; Perry, 1992; Stoquart et al., 2008), or from recordings of able-bodied subjects walking “freely” (i.e., in “transparent mode,” where only τ_comp, but not τ_q, is applied) in the same device to be controlled (Colombo et al., 2000). When determining q_ref, attention must be paid to avoid unwanted contact between the end-point (e.g., the heel or the tip of the foot) and the ground. For example, a small angular deviation at the knee joint may result in a considerable change in foot clearance (Winter, 1992).

One challenge in joint space formulation comes with the high inter-subject variability in angular patterns, which makes it difficult to define joint reference trajectories that fit all subjects. In some exoskeletons, q_ref can be changed manually by the user within some limits (Riener et al., 2010; Meuleman et al., 2016). However, it is difficult to predict whether the subject will have adequate foot clearance and step length, since these also depend on the length of the thigh and shank segments.

Another challenge comes in applications where the users are required to perform a task following visual feedback, e.g., to follow a reference trajectory displayed on the screen. Simultaneous feedback from two or more joint space variables (e.g., hip and knee) is usually quite complex to process (Maggioni et al., 2015).

Impact of Joint Space Formulation on End-Point Stiffness

Potential hazards during walking can come from unwanted interactions between the foot and the floor (or treadmill). Therefore, we examined the forces at the ankle level that may result in such unwanted interactions. These forces were generated by a controller defined in joint space, given foot displacements of different amplitude and directions throughout the swing phase. We obtained the resulting end-point forces (force field) using the Jacobian matrix of the two-links robot (see Appendix 1.2).

In Figure 2, we show the force field for different points during the pre-swing and swing phase. In this case, hip and knee stiffness are constant throughout the gait cycle, but the resulting end-point stiffness varies depending on the angular configuration of the joints. The magnitude and direction of joint torques and end-point forces applied by a joint controller on a real trajectory are presented in Figure 3. Two main requirements for functional walking are adequate foot clearance and foot placement at the end of swing (Gage, 1991; Baker, 2013). Therefore, we examined these two phases in detail. As the reader can appreciate, the restoring forces around the foot are not always directed toward the reference trajectory (note that the reference trajectory is defined in joint space, but it is transformed to end-point space for visualization purposes). Consider the situation where a subject is not able to sufficiently lift the foot from the ground at the beginning of swing phase: as we can see in Figures 2A, 3B, the joint controller is able to provide forces that are directed toward an adequate foot clearance position. On the other hand, if the subject is not able to perform a sufficiently long step (e.g., due to insufficient hip flexion or reduced knee extension at the end of swing), or if his foot is lagging behind the reference position, the actual position of his ankle can fall in an area where the forces rendered by the controller direct the foot toward the ground, instead of lifting it to guarantee a sufficient step length. It is interesting to compare how the same controller acts in the two different spaces; we can obtain insights that are not possible by studying the joint torques and end-point forces in isolation.

FIGURE 2

Figure 2. The force field resulting from a joint space impedance controller is shown at some selected points along the ankle trajectory. The restoring forces do not always point toward the reference position. Two critical points are magnified. (A) Point of maximum foot clearance: the vectors show that enough support is guaranteed if the ankle is below the reference trajectory. The ellipse in black represents the end-point stiffness resulting from the joint stiffness. (B) At the end of the swing phase, if the subject is late with respect to the reference point, it can experience forces directed downwards instead of forward.

FIGURE 3

Figure 3. In correspondence of a real trajectory (blue line) deviating from the reference trajectory (red line), the joint controller generates the torques shown in (A). The same torques can be visualized in end-point space (B) as equivalent end-point forces [see Appendix 1.4]. Refer to the scale for information on the magnitude of the torques and forces. The beginning of the stance phase is marked with a gray circle, while the beginning of the swing phase is marked with a gray square.

Impedance Control Based on End-Point Space Formulation

An alternative option to a joint space formulation is an end-point space formulation (sometimes referred to as task space formulation), in which the reference trajectory is defined according to an anatomical landmark around an end-point. In walking, the definition of end-point depends on the kinematic configuration, e.g., lateral malleolus or foot metatarsal during swing phase; or trochanter during stance phase, as the foot is already placed on the ground. Thus, formulating the problem in end-point space for lower limb exoskeletons may require two different control approaches: one for stance and another one for swing. While the implementation of this approach may be cumbersome in practice, a controller during swing that relies on an end-point space formulation may provide additional benefits compared to a joint space approach. In this paper, we are interested in studying the control of the end-point impedance only in the swing phase of gait.

In an end-point space formulation, the torque applied to the exoskeleton actuators is derived from an end-point force F_x (Equation 2). This force depends on a set of stiffness, K_x = [K_xx, K_xy; K_yx, K_yy], and damping, B_x = [B_xx, B_xy; B_yx, B_yy], parameters and a kinematic error between a measured end-point trajectory, x_act = [x_act y_act], and a reference trajectory, x_ref = [x_ref y_ref]. Note that x_ref and x_act can be calculated in real-time by using forward kinematic equations that depend on the measured joint angles q_ref and q_act and known limb segment lengths of the user (see Appendix 1.1). The accuracy of this calculation, however, depends on the correct measurement of the segments' lengths and on the alignment between the robotic joints and the human joints.

$\begin{array}{ll}{\text{F}}_x={\text{K}}_x({\text{x}}_{ref}-{\text{x}}_{act})+{\text{B}}_x({\stackrel{·}{\text{x}}}_{ref}-{\stackrel{·}{\text{x}}}_{act})& (2)\end{array}$

Using the Jacobian matrix J(q_act), we obtain through inverse dynamics the torque that the joint actuators need to render the force F_x:

$\begin{array}{ll}{\tau }_x=\text{J}{\left[{\text{q}}_{act}\right]}^T{\text{F}}_x& (3)\end{array}$

Selection of End-Point Reference Trajectories

In contrast to joint reference trajectories, end-point trajectories are not widely available in the literature. One could take joint reference trajectories and apply forward kinematics, or obtain such trajectories experimentally. Another approach is to take a few features that ensure that the position of the foot guarantees a safe interaction with the environment, e.g., foot clearance and step length. These features can be easily visualized and adapted in end-point space. The manual adaptation of x_ref is more intuitive for therapists if they reason in end-point space (Emken et al., 2008) and focus on specific gait subtasks (Meuleman et al., 2016), rather than setting hip and knee angular reference trajectories simultaneously.

The subject can be provided with visual feedback regarding the position of his foot and asked to control its trajectory, in a similar way he is required to do in real environments—e.g., by lifting a foot over an obstacle. In gait trainer device literature, similar approaches have been followed when the focus was on ankle height to guarantee appropriate foot clearance in stiff-knee gait (Koopman et al., 2013). Additionally, visual feedback containing information about the end-point is much easier to process (Banala et al., 2009; Koopman et al., 2013; Krishnan et al., 2013) for subjects, whereas it is extremely difficult to adapt behavior based on feedback about hip and knee movements (Maggioni et al., 2015).

Impact of End-Point Space Formulation on End-Point Stiffness

Similar to section Impact of Joint Space Formulation on End-Point Stiffness, we would like now to examine the forces acting at the level of the foot when end-point control is used. By design (Equation 2), at each point of the swing phase, the restoring force for every deviation in Cartesian space is directed toward the reference end-point position (Figure 4), which is the point that could have potentially critical collisions with the environment (e.g., stumbling). The axes of the stiffness ellipse can be modified in magnitude and direction as desired. For example, a higher stiffness in the direction of gravity can be designed. However, singularities exist which prevent the end-point controller from generating joint torques in correspondence of those points (i.e., when the knee is completely extended at the end of swing).

FIGURE 4

Figure 4. The desired force field in task-space is shown at some selected points along the end-point trajectory. The force field always points toward the reference position. Two critical areas are magnified. (A) Point of maximum foot clearance: the circle in black represents the desired end-point stiffness. The arrows show that regardless of the deviation from the reference point, the restoring force results always in a force directed to the reference point. (B) At the end of the swing phase, the desired characteristics of the force field are the same as in (A).

Now consider the end-point forces generated when an end-point controller is used with a real trajectory. The force field set as shown in Figure 4 leads, in the case of the real trajectory presented in Figure 5B, to forces directed toward the reference trajectory in end-point space. Figure 5A shows the same forces transformed to torques Equation (3). As visible in the graph, the joint torques in this case do not always point toward the joint reference trajectory, especially at initial swing, the phase that is crucial for determining a safe foot clearance through an appropriate knee flexion. When the foot is lagging behind the reference trajectory in end-point space, the end-point controller tries to push the foot forward by increasing the hip flexion, while not acting on the knee. This is evident in Figure 5A where, at the point of maximum knee flexion, the torques have an almost null component acting on the knee. This problem might cause insufficient foot clearance and potential undesired foot contact with the treadmill.

FIGURE 5

Figure 5. In correspondence of a real trajectory (blue line) deviating from the reference trajectory (red line), the end-point controller generates the forces shown in (B). The same forces can be visualized in joint space (A) as equivalent joint torques [see Equation (3)]. Refer to the scale for information on the magnitude of the torques and forces. The beginning of the stance phase is marked with a gray circle, while the beginning of the swing phase is marked with a gray square.

Assist-As-Needed Controllers

General Formulation

“Assist-As-Needed” (AAN) refers to a control strategy based on assisting the patient/user only as much as needed to successfully perform a predefined task (Emken et al., 2005). One way to modulate the assistance provided by the robotic device is to modify the mechanical impedance rendered by the exoskeleton. A common AAN algorithm for an impedance controller typically updates a normalized impedance parameter $P(P\in ℝ|0\le P\le 1)$, e.g., stiffness or damping, at every gait step s:

$\begin{array}{cc}{P}_{s+1}=\gamma P_s+f\left(e_s\right)g& (4)\end{array}$

A forgetting factor, $\gamma (\gamma \in ℝ|0<\gamma <1)$, limits the excessive reliance on the robotic assistance provided by the motion controller (the “slacking” effect; Marchal-Crespo and Reinkensmeyer, 2009). A gain $g(g\in ℝ|g>0)$ adjusts the control parameter according to an error function $f\left(e_s\right),f(f:e_s\to [0,1])$, where e_s can be, for example, the kinematic deviation between the reference and actual trajectory of an exoskeleton. The function f may account for physiological kinematic variability (e.g., by defining a “deadband” around the reference trajectory; Banala et al., 2007; Emken et al., 2007). Note that domain of the parameters P, γ and f(e_s) can be different, depending on the behavior one would like to achieve with the AAN algorithm (Marchal-Crespo and Reinkensmeyer, 2009), however, for the examples discussed further in this paper we have selected the ones above.

Joint Space Formulation of an AAN Controller

There are several examples of controllers that adapt the robotic joint impedance of an exoskeleton to the subject's ability to walk - for a review see: (Marchal-Crespo and Reinkensmeyer, 2009; Hussain et al., 2011; Cao et al., 2014). For example, to create a patient-cooperative strategy for the Lokomat, hip and knee impedances were adapted according to the patient's effort (as estimated by the robot force sensors) (Riener et al., 2005). Based on a similar estimation of the subject's active contribution, Hussain adapted the joint impedance of a pneumatic-actuated exoskeleton robot (Hussain et al., 2013). However, both works were based on forces exerted by a limited group of able-bodied subjects, which could heavily compromise their applicability in patients exhibiting clonus or spasticity. In the Lokomat, this dependence on the interaction forces was overcome by implementing an approach called “Path control,” which allows freedom of movement around predefined joint trajectories, while a virtual tunnel of adjustable width guarantees safety (Duschau-Wicke et al., 2010).

In Maggioni et al. (2015), we presented an AAN algorithm that automatically adapts the Lokomat actuators' impedance based on the ability of the subject to follow a reference gait trajectory. In this work, the algorithm described by Equation (4) was applied. The control parameters P were the stiffness K and the damping B of the hip and knee in an impedance joint controller. The estimator of the subject's performance relied on the kinematic deviation between the actual trajectory and the reference. The gait cycle was divided in 30 windows. For each window w and for each step s the joint impedance was defined by one set of parameters, K_{s, w} and B_{s, w}, which was adapted according to the weighted kinematic error performed in each window and every step.

$\begin{array}{ll}{\text{K}}_{s+1,w}={\gamma }_1{\text{K}}_{s,w}+g_1{f_1\left[e_s\right]}_w& (5)\end{array}$

$\begin{array}{ll}{\text{B}}_{s+1,w}={\gamma }_2{\text{B}}_{s,w}+g_2{f_2\left[\stackrel{·}{{\text{e}}_s}\right]}_w& (6)\end{array}$

A set of gains γ₁, γ₂, g₁, g₂ were defined in order to have the impedance decrease slowly in the presence of physiological deviations and to react fast enough in case of large errors. The error weighting function f [e_s]_w consisted of a hyperbolic tangent function of the kinematic error e_s defined for each window w, which allowed physiological deviations from the reference trajectories of the hip and knee joint, while ensuring safety. This means that for each time point of the gait cycle, the subject's hip and knee was allowed to deviate from the reference trajectory within the deadbands defined for each joint, independently from each other and irrespective of the position of the end-point. Suitable deadbands in joint-space can be defined based on normal ranges for hip and knee joint angles (e.g., taking normative data from Winter, 1991; Perry, 1992; Stoquart et al., 2008 or from able-bodied people walking in the device). To study how these angular boundaries result in end-point space, we applied forward kinematics (see Appendix 1.1) to render the resulting boundaries around the end-point (i.e., at the ankle), as illustrated in Figure 6.

FIGURE 6

Figure 6. In the joint controller hip and knee deadbands are defined independently from each other, as shown in the (A) (hip angle) and (B) (knee angle). The reference trajectory (red) is taken from Colombo et al. (2000). The deadbands (black lines) are calculated from the standard deviation of the trajectories of 10 able-bodied subjects walking in the Lokomat with impedance set to 5% of the maximum, which allows freedom of movement. In the AAN algorithm, deviations occurring within the deadbands lead to a null error. The gait cycle is divided in 30 windows (gray lines show the windows' limits). In (C), the resulting reference trajectory (red) and the corresponding deadbands in end-point space are shown. For each window along the swing phase (only 15 are shown for clarity of representation), the gray rhomboid shows the area including all the possible combinations of hip and knee angles within the deadbands shown in (A,B).

Due to the non-linearity of the kinematic transformation and its dependency on the joint configuration, the shape of the boundaries resulting at the end-point is hardly predictable from what can be seen in joint space. During the push-off phase and at the beginning of swing, the boundaries are extremely narrow along the direction of the foot motion. This results in a very strict timing requirement for the subject walking in the robot (i.e., the subject must closely follow the desired ankle position at any time). Even small deviations along the directions of motion can result in a high error, which causes the algorithm to increase the impedance in this specific gait phase. However, in the direction perpendicular to the reference trajectory, higher deviations are allowed, and they could potentially result in insufficient foot clearance. Conversely, during mid-swing, the resulting shape of the joint space deadbands is less conservative along the direction of the trajectory, allowing increased leading or lagging of the foot with respect to a reference position. At the end of swing, again the shape of the boundaries in end-point space changes: here the boundaries allow the subject to perform longer or shorter steps than desired.

End-Point Space Formulation of an AAN Controller

In this type of controller, the parameters P adapted based on Equation (4) are the end-point stiffness and damping (K_x and B_x). In literature, there are several examples of end-point impedance adaptation implemented in exoskeleton and end-effector devices. Among the latter, Emken et al. adapted the end-point impedance of a robot guiding the ankle of the subject (ARTHuR) based on the position and velocity error between the reference and actual ankle trajectories (Emken et al., 2008). Hussein et al. implemented an algorithm for adapting the width of a deadband for velocity deviations in the footplate-based Gait Trainer GT-I (Reha-Stim, Germany): based on the error between actual and desired end-effector velocity; the deadband width was either increased to allow more freedom or decreased to provide more guidance to the subject (Hussein et al., 2009). Other works instead, despite using exoskeleton devices, developed an algorithm that adapted the end-effector impedance or force field and calculated the required joint torques based on end-point information. For example, Koopman et al. developed an adaptive vertical force acting on the ankle to support foot clearance (LOPES, Koopman et al., 2013); Banala et al. designed a force field acting on the ankle to guide the end-point along a virtual tunnel (ALEX, Banala et al., 2007).

Having control over the task space impedance allows the implementation of AAN controllers that provide optimal assistance to the end-point. Indeed, the task space force field can be shaped in order to support the foot only in the directions that are needed. Furthermore, designing the deadbands in end-point space allows requirements such as minimum foot clearance or minimum step length to be set directly.

Summary of Working in Different Spaces

The two controllers show very different features when applied to a two-link exoskeleton and it is not possible to prefer one over the other independently of the application. In Table 1, we summarized the strengths and weaknesses of the two control formulations. The symbols “+” and “–” indicate whether the formulation can adequately address the specific features listed. These aspects have also been nicely addressed in Smith et al. (2015), where the performance of joint and end-point controllers is compared in an industrial manipulator.

TABLE 1

Table 1. Summary of the performances of joint and end-point formulations for the control of a two-link exoskeleton.

Hybrid Joint/End-Point Space Controller With Assist-As-Needed

In section Joint vs. End-Point Space Formulations, we highlighted strengths and weaknesses of the two formulations: joint and end-point space. Here, we propose an adaptive controller that is formulated in both spaces (“hybrid” formulation) and aims to combine the strengths of both approaches. An end-point space component aims at adapting the end-point stiffness in both magnitude and direction to provide a guided foot placement; while a joint space component aims at providing appropriate temporal coordination between hip and knee angles, especially when the kinematic configuration of the exoskeleton is close to a singularity. This hybrid approach also gives the possibility of defining deadbands more intuitively (based on foot position), which gives more control over the interactions with the environment.

The torques applied during the swing phase of gait, τ_swing, are the sum of torques generated by a PD controller based on the end-point position and end-point velocity error (Equation 8), torques generated by a D controller based on the angular velocity error in joint space (Equation 9), and compensation, as illustrated in Figure 7:

$\begin{array}{cc}{\tau }_{swing}={\tau }_{xPD}+{\tau }_{qD}+{\tau }_{comp}& (7)\end{array}$

$\begin{array}{cc}{\tau }_{xPD}=J{\left[q_{act}\right]}^T{K}_x\left(x_{ref}-x_{act}\right)+J{\left[q_{act}\right]}^T{B}_x\left({\stackrel{·}{x}}_{ref}-{\stackrel{·}{x}}_{act}\right)& (8)\end{array}$

$\begin{array}{cc}{\tau }_{qD}=B_{\mathit{q}}({\stackrel{·}{q}}_{ref}-{\stackrel{·}{q}}_{act})& (9)\end{array}$

The end-point controller is designed to control the magnitude and direction of the forces required in task-space. Since the reference trajectories in joint space are derived from trajectories in task space, one can express the controller terms as:

$\begin{array}{llll}{\tau }_{xPD}+{\tau }_{qD}& =& {\text{K}}_{tot}({\text{q}}_{ref}-{\text{q}}_{\text{act}})+{\text{B}}_{tot}({\stackrel{·}{\text{q}}}_{ref}-{\stackrel{·}{\text{q}}}_{act})& (10)\end{array}$

$\begin{array}{llll}{\text{K}}_{tot}& =& {\text{J}\left[q_{\text{act}}\right]}^{\text{T}}{\text{K}}_x\text{J}\left[q_{\text{act}}\right]& (11)\end{array}$

$\begin{array}{llll}{\text{B}}_{tot}& =& {\text{J}\left[q_{\text{act}}\right]}^T{\text{B}}_x\text{J}\left[q_{\text{act}}\right]+{\text{B}}_q& (12)\end{array}$

Note that the stiffness and damping matrices must fulfill the necessary conditions for stability defined in Appendix 2.

FIGURE 7

Figure 7. Control diagram of the adaptive hybrid joint/end-point controller during the swing phase of gait. The transparency of the exoskeleton is obtained through a torque feedback loop (torque controller). The torque controller provides a torque proportional to the error between the desired torque τ_swing and the measured torque τ_int, in order to minimize this same error. Transparency is improved through the optimization of passive dynamics with a method called Generalized Elasticities (Vallery et al., 2009). Detailed information on the low-level control architecture can be found in Riener et al. (2005) and Vallery et al. (2009).

AAN Algorithm

The actual stiffness and damping in end-point space, K_x[N/m] and B_x[Ns/m], are obtained from a normalized stiffness and damping ${\overline{\text{K}}}_x$and ${\overline{\text{B}}}_x$ matrices, which are then scaled according to the specific characteristics of the robot. The normalized joint damping term ${\overline{\text{B}}}_q$ can be adapted according to Equation (6) in section Joint Space Formulation of an AAN Controller. ${\overline{\text{B}}}_x$ can be adapted either with a similar algorithm or coupled to ${\overline{\text{K}}}_x$.

For the term ${\overline{\text{K}}}_x$ we would like an AAN algorithm that adapts both the magnitude and direction of the equivalent stiffness ellipse based on the kinematic errors performed throughout the swing phase.

To achieve this the swing phase is divided into equally sized windows. For each window w and for each step s, we adapt the stiffness based on the weighted error at the previous step, both in magnitude and in direction, as:

$\begin{array}{cc}{\overline{K}}_{x_{s+1,w}}={\gamma }_x{\overline{K}}_{x_{s,w}}+f_{K_x}\left[e_{x_{s,w}}\right]R\left[{\alpha }_{s,w}\right]G_{K}R{\left[{\alpha }_{s,w}\right]}^T& (13)\end{array}$

$\begin{array}{cc}{e}_{x_{s,w}}=\left[\begin{array}{c}{x}_{re{f}_{s,w}}-x_{ac{t}_{s,w}}\\ y_{re{f}_{s,w}}-y_{ac{t}_{s,w}}\end{array}\right]& (14)\end{array}$

$\begin{array}{cc}{\alpha }_{s,w}=\arctan \left(e_{x_{s,w}}\right)& (15)\end{array}$

$\begin{array}{cc}R\left[{\alpha }_{s,w}\right]=\left[\begin{array}{c}\cos {\alpha }_{s,w}-\text{}\sin {\alpha }_{s,w}\\ \sin {\alpha }_{s,w}\cos {\alpha }_{s,w}\end{array}\right]& (16)\end{array}$

The first term, ${\gamma }_x{\overline{\text{K}}}_{x_{s,w}}$, reduces the stiffness ellipse in all directions given a constant forgetting factor, γ_x = 0.9. The second term increases the stiffness in the direction of the kinematic error. The magnitude of this change is controlled by a gain matrix G_K = [0.1 0;0 0.01], which can be seen as a predefined ellipse with axes of fixed length. This ellipse G_K is (i) rotated along the direction of the error, (ii) scaled according to the magnitude of the weighted error f_Kx [e_{xs, w}] and (iii) summed to the stiffness ellipse ${\gamma }_x{\overline{\text{K}}}_{x_{s,w}}$. The error function f_Kx (f_Kx : e_{xs, w} → [0, 1]) is defined for each window w with different shape characteristics (Figure 8). The error functions f_Kx [e_{xs, w}] can be defined with deadbands designed in end-point space. In this way, it is possible to identify requirements for the foot trajectory that ensure a safe interaction between the foot and the treadmill, for example, minimum foot clearance (Begg et al., 2007) and minimum step length (Sekiya et al., 1996). One way of defining the error weighting functions f_Kx [e_{xs, w}] is by using Asymmetric Generalized Gaussian functions (AGGF) (Elguebaly and Bouguila, 2013) which can be designed to have a different variance depending on the gait cycle window. The AGGF allows the width of tolerated kinematic deviations to be defined in all directions independently. An example is presented in Figure 8. By design, ${\overline{\text{K}}}_{x_{s,w}}$and f_Kx [e_{xs, w}] are bounded above by 1, therefore, even in presence of high errors, the eigenvalues of the stiffness matrix will never increase above the initial values. The change in the stiffness matrix between consecutive time steps can be bounded by the necessary stability conditions defined in the Appendix 2.

FIGURE 8

Figure 8. In end-point space the deadbands have been designed as asymmetric Generalized Gaussian functions (AGGF). The weighting functions are shaped differently in different points of the swing phase to prevent kinematic deviations that could result in unsafe interactions with the treadmill (e.g., reduced foot clearance and step length). For each window during the swing phase an AGGF is defined (for clarity of representation, only half of the windows are displayed). Kinematic errors falling within the borders of the respective AGGF result in a null weighted error. Otherwise, the weighted error saturates to 1.

Due to the non-linear and adaptive nature of the controller (and the human) and to the variable impedance profile, it is a daunting task to derive the analytical necessary and sufficient conditions for stability. However, we believe that with the necessary (although not sufficient) conditions defined in the Appendix 2, in combination with a series of safety measures to prevent undesired robot behaviors, the safety of the user can be guaranteed. First and foremost, we made sure that the controller was stable with constant stiffness and damping values throughout the task space. Second, software mechanisms were in place to constrain the stiffness and damping values to the necessary boundaries defined in Appendix 2. The damping was tied to the stiffness to guarantee a critically damped (or overdamped) system throughout the different kinematic configurations. The rate of change of stiffness and damping parameters was constrained. Finally, the safety hardware and software mechanisms of the Lokomat prevented to reach singular configurations and shut down the motors whenever an excessive force or an excessive deviation from the reference trajectory was detected. Before the tests in humans, the controller was tested in real-life simulations on a test-bench as described in section Simulation Results.

Simulation Results

Before testing the AAN hybrid joint/end-point controller in human subjects, we performed simulations of the expected behavior using Matlab (v2013b, Mathworks).

We started from the simple case of a point along the reference trajectory and simulated different types of kinematic error. We wanted to test whether the AAN algorithm in the hybrid controller ensures an adaptation of the stiffness matrix to the direction and magnitude of the error. We simulated two cases: (i) error of unitary magnitude and constant direction (angle α between the error vector and the x axis equals 0) and (ii) error of unitary magnitude but variable direction (with α varying randomly at each step in the interval [0, π /2]). The resulting stiffness ellipses are described in terms of size, shape, and orientation (Mussa-Ivaldi et al., 1985), whereby size indicates the length of the major axis of the ellipse, shape the ratio between the major and minor axis of the ellipse, and orientation the angle between the major axis and the x axis.

In the first simulation (Figure 9—first line), the size along the error direction (length of the ellipse major axis) does not decrease since the error function f_Kx [e_{xs, w}] gives a constant unitary result (Equation 13). The orientation of the ellipse's major axis aligns with the error direction, inducing a force field with maximal restoring forces along the direction of the error and very low forces in every other direction, guaranteeing a compliant behavior of the controller against disturbances in directions other than the error. In the second simulation, as shown in Figure 9—second line, the ellipse orientation follows the error direction and so does the relative force field. The shape of the ellipse depends on how variable the direction of the error was in the previous steps (Equation 13).

FIGURE 9

Figure 9. First line: simulation of an error with constant magnitude and direction (black vector) around a reference point (in red). The stiffness ellipse initial configuration is a circle which adapts step by step to the error. The central force field visible at step 1 consequently changes its characteristics. At step 50, the force field is directed mainly along the direction of the error. This implies that the stiffness is high only in directions parallel to the error. Second line: simulation of an error with constant magnitude and variable direction. The error angle variates randomly between 0 and 90°. The error of the current step is shown in bold black, while the previous vectors are shown in gray. The stiffness ellipse adapts its orientation based on the error direction. The force field represented by the blue vectors adapts accordingly.

In a second phase, we used a robotic test bench to simulate neurological impairments such as spasticity. The test bench uses a bio-inspired model of a human leg implemented on the leg orthosis of a robotic gait trainer (the Lokomat, in this case). In this setup, one leg orthosis is controlled to simulate a human leg (simulated human leg), while the second orthosis (test orthosis) is controlled by the hybrid end-point/joint controller with AAN. The two orthoses are then rigidly connected using two aluminum bars, simulating a physical attachment of the robot to the user's leg. A spastic-like behavior was implemented on the simulated human leg by adding a velocity-dependent torque at the level of the knee joint, which was applied when the knee angular velocity exceeded a certain threshold. A detailed description of the test bench and of the impairment simulation can be found in Maggioni et al. (2016). The physical connection between the two orthoses allowed the hybrid controller implemented on the test orthosis to control the simulated human leg by shaping the stiffness ellipses to the simulated impairment. As expected, the test orthosis with the hybrid joint/end-point controller adapted the end-point stiffness to counteract the deviations of the simulated human leg caused by the spastic-like simulated impairment (Figure 10).

FIGURE 10

Figure 10. Adaptation of the end-point stiffness of the test orthosis during the simulation of a spastic-like behavior in the simulated human leg of the test bench. The initial stiffness ellipses are shown in the first box. The simulated velocity-dependent torque caused deviations of the foot trajectory at mid-swing and at the end of the swing phase. The hybrid controller adapted the stiffness ellipses magnitude and direction to provide targeted support to these deviations.

Experimental Results

The adaptive hybrid joint/end-point controller and the adaptive joint controller were tested with five able-bodied subjects (1 female, age = 27 ± 4.7 years) and one subject with a chronic motor complete Spinal Cord Injury (male, age = 37 years, ASIA B, level of injury = T4, WISCI II = 0/20). The Kantonale Ethikkommission Zürich and Swissmedic approved the study. The aim of this test was first to determine the feasibility and safety of the novel hybrid controller, and subsequently compare the performances of the adaptive hybrid controller to the existing joint adaptive controller (Maggioni et al., 2015). In particular, we hypothesized (i) that this novel controller adapts the magnitude of the stiffness to the subject's ability to follow the reference trajectory and, at the same time, (ii) that the orientation of the stiffness ellipses aligns to end-point deviations. We decided not to test the pure end-point controller on human subjects, due to safety concerns that emerged while doing preliminary tests with a dummy. As foreseen in section Impact of End-Point Space Formulation on End-Point Stiffness, the end-point controller alone was not able to guarantee sufficient foot clearance and avoid potential undesired foot contact with the treadmill.

Methods

Subjects were instructed to follow a given foot trajectory in time and space, which was projected on a screen positioned in front of the Lokomat. The actual and reference ankle trajectories were displayed in different colors and two dots indicated the reference and actual position at every time point. After being set up in the Lokomat, the subjects were allowed to familiarize themselves with walking in the device with the standard impedance controller (impedance was set at the maximum available value). The visual feedback was constantly presented to the subject. In this familiarization phase, the Lokomat gait pattern was adjusted to the subject's gait pattern by tuning the ROM and the offset of the hip and knee angular trajectories. These settings were then kept constant during the subsequent experiment. Once comfortable and accustomed to walking inside the robot, the subject was presented with a familiarization round with the novel AAN hybrid controller as described in section Hybrid Joint/End-Point Space Controller With Assist-as-Needed. The subject was instructed to follow the reference trajectory as closely as possible while the adaptation algorithm adapted the impedance based on the kinematic error of the ankle trajectory. After the familiarization phase, the AAN control was active on the leg under test for 50 steps, while the impedance of the other leg was kept at the maximum available value. To ensure a safe foot clearance during swing, the stiffness in the vertical direction was made 5 times higher than the stiffness in the horizontal direction. While this is not a problem in the case of high impedance, it might become apparent when the adaptation algorithm reduces the impedance below a certain level, especially in patients with walking impairments. The implemented stiffness ${\tilde{\text{K}}}_x$ and damping ${\tilde{\text{B}}}_x$ in the Lokomat were:

$\begin{array}{cc}{K}_x=M_K{\overline{K}}_x& (17)\end{array}$

$\begin{array}{cc}{M}_K=\left[15000;07500\right]\frac{N}{m}& (18)\end{array}$

$\begin{array}{cc}{\tilde{K}}_x=\frac{\left(K_x+K_x^T\right)}{2}& (19)\end{array}$

$\begin{array}{cc}{\tilde{B}}_x=M_B{\overline{B}}_x& (20)\end{array}$

$\begin{array}{cc}{M}_B=\left[400;040\right]\frac{Ns}{m}& (21)\end{array}$

The transformation in Equation (19) guarantees that stiffness matrix is symmetric. In addition, to guarantee the stiffness matrix to remain positive definite after this transformation the following constraint was implemented:

$\begin{array}{ll}\left(-\sqrt{K_{11}{K}_{22}}+\rho \right)<K_{ij}<\left(\sqrt{K_{11}{K}_{22}}-\rho \right)& (22)\end{array}$

For i ≠ j;i,j = 1,2, where

$\begin{array}{ll}\rho =0.1\sqrt{K_{11}{K}_{22}}& (23)\end{array}$

The performance of the AAN hybrid controller was then compared with that of the AAN joint controller (see section Joint Space Formulation of an AAN Controller and Maggioni et al., 2015). For this comparison, subjects were tested in a separate session (scheduled within 4 weeks), while performing the same task using the AAN joint controller.

In the AAN hybrid controller, the magnitude of the end-point stiffness was calculated as the maximum eigenvalue of the stiffness matrix (i.e., the length of the major axis of the stiffness ellipse), averaged over all the windows during the swing phase of each step. The major axis of the stiffness ellipse indicates the direction where the end-point stiffness is maximal. To obtain a measure of the alignment between the direction of maximum stiffness and the position error at the ankle, we calculated the angle between the major axis of the stiffness ellipse and the vector of the end-point error. Only the swing phase of the gait is considered, since the hybrid controller is active only during swing. The weighted kinematic error f_Kx [e_{xs, w}] equals zero when the actual deviation is within the defined deadbands. In this case, the adaptation algorithm (Equation 13) decreases the size of the stiffness ellipse but does not change its orientation. Therefore, we only calculated the alignment in those windows where the weighted error f_Kx [e_{xs, w}] was greater than 0.1. The data of the last 5 steps of the adaptive task were used for the analysis of the final stiffness alignment determined by the algorithm. An average value over all subjects was calculated.

In the joint controller, the magnitude of the stiffness was calculated as the mean of the hip and knee joint stiffness during the swing phase. We then obtained the equivalent end-point stiffness resulting from the joint stiffness matrix (Equation A.10 in Appendix 1). The angle between the major axis of the resulting stiffness ellipse and the direction of the error in end-point space was calculated to estimate the alignment of the force field perceived at the ankle with the error.

Results

All subjects were able to perform the experiment with the adaptive hybrid controller; the subject with SCI required a fixed body weight support equal to 70% of his body weight to use the adaptive hybrid controller.

The overall end-point stiffness decreased over time and converged to a specific value for each subject. The patient reached, as expected, a higher final value than the able-bodied subjects did.

Results (Figure 11) confirmed that the stiffness ellipses start from an initial size and shape (ratio major/minor axis = 5) and, based on Equation (13), subsequently adapt in shape, orientation and size to the errors at the ankle (Figure 12). During adaptation, the size of the stiffness ellipses adapts gradually to the kinematic error occurring in that gait phase. At every step, the orientation of the stiffness ellipses tends to align to the direction of the error in that gait window (Equation 13, second term).

FIGURE 11

Figure 11. In this figure, the normalized adaptive stiffness of the two types of controller (AAN hybrid controller and AAN joint controller) is shown over 50 steps. Each data point represents the mean value over the swing phase of one step. In (A), the adaptive stiffness of the hybrid controller (i.e., the maximum eigenvalue of the ellipse) is displayed. In (B), the adaptive stiffness of the joint controller (i.e., the mean of the normalized hip and knee stiffness) is shown. The data of the patient are visualized in red.

FIGURE 12

Figure 12. Adaptation of the end-point stiffness during the experiment for three different subjects. The initial stiffness ellipses are shown in the first column. To ensure a safe foot clearance during swing, the vertical stiffness maximum value was set higher than the horizontal stiffness. After 25 steps (2nd column) the stiffness ellipses are adapting to the error size and direction. At the last step of the adaptation (3rd column), the ellipses reached their final configuration. On the 1st line, data from the subject with SCI are shown: as expected, the final stiffness ellipses have a bigger size than those achieved by the able-bodied subjects in the 2nd and 3rd line.

In contrast, Figure 13 shows the results for the joint controller, whereby hip and knee joint stiffness adapt separately (section Joint Space Formulation of an AAN Controller) and no coupling terms are present. Hence, the size, shape and orientation of the resulting end-point stiffness depend not only on the actual joint stiffness but also on the configuration of the leg segments (therefore, on the gait phase). It is clear that there is little or no correspondence between the errors performed in task space and the resulting end-point stiffness.

FIGURE 13

Figure 13. Resulting end-point stiffness ellipses caused by the joint controller in the subject with SCI. The resulting end-point stiffness is calculated from the hip and knee stiffness during the last step (50th) of the adaptation. The ellipses appear in the figure as lines since the minor axis is close to a null length. The kinematic error between the reference trajectory (red) and the actual trajectory (blue) of the ankle joint during swing phase is shown by the black vectors.

The alignment between the major axes of the ellipses and the error in the respective time window in the last 5 steps is greater (i.e., the angle is minimum) in the ideal hybrid controller (for K_xx = K_yy) (Figure 14). The joint controller showed the worst performance in terms of alignment.

FIGURE 14

Figure 14. Average alignment (angle) between the major axis of the end-point stiffness ellipses and the direction of the error in the same gait window. First boxplot: alignment of the end-point stiffness ellipses resulting from the joint controller. Second boxplot: alignment of the end-point stiffness ellipses obtained in the experiments with the hybrid controller. Third boxplot: alignment of the end-point stiffness ellipses in the ideal case where the initial vertical stiffness of the hybrid control was set equal to the horizontal stiffness.

If we examine one of the critical points of the late swing phase, i.e., right before heel strike, in further detail (Figure 12), it becomes apparent that, especially in the subject with SCI, the hybrid controller generates an end-point stiffness ellipse rotated in the direction of the error (i.e., the stiffness is higher in the direction along which the error occurred). The adaptive joint controller (Figure 13) instead shows a very small stiffness value in that direction, but a high stiffness in a direction that does not apparently require any support.

Discussion

The aim of our work was to develop an AAN controller for a lower limb exoskeleton which could optimally adapt the support based on the patient's ability to follow a reference trajectory. To achieve this, we examined and discussed the features and disadvantages of joint and end-point space formulation to control exoskeleton robots for the lower limbs. Then, we developed a proof-of-concept novel controller that combines the benefits of joint and end-point formulations: an adaptive hybrid joint/end-point space controller. We presented the results of a software simulation and, finally, the results of the tests on able-bodied subjects and one subject with SCI.

When developing a controller for gait exoskeletons, the choice of the formulation (joint or end-point) highly influences the apparent stiffness and damping rendered by the robot and it has an impact on how the reference trajectories (and safety features around them) are designed. While gait trajectories defined in joint space are closer to the hardware structure of the exoskeleton and similar to what gait analysis presents us, the trajectory of the foot during gait is a precise end-point control task (Winter, 1992). The human achieves certain trajectories in task-space thanks to the fact that the internal models take care of the proper muscle activations that guarantee the correct joint movements (Shadmehr and Mussa-Ivaldi, 1994). In both healthy and pathological conditions, different joint kinematic solutions are adopted to control the position and orientation of the end-point and, in particular, to achieve a safe trajectory of the foot during the swing phase (Winter, 1992). It has thus been hypothesized that the control of the foot trajectory during swing is a major focus of our central nervous system (CNS) during human locomotion (Ivanenko et al., 2003), as also supported by animal studies (Georgopoulos and Grillner, 1989). In contrast to trajectories defined in joint space, end-point trajectories of able-bodied subjects during swing show very little variability during walking on firm level ground and on the treadmill (Winter, 1992; Ivanenko et al., 2002; Awai and Curt, 2014). Instead, during stance phase the main task is not control of the foot trajectory, but rather the support and balance of the body weight. These functional tasks are accomplished by the control of hip, knee and ankle angles in a so called “support synergy” (Winter, 1995). The different tasks performed during swing and stance phase and the different models used for these two phases, support the use of the end-point formulation only during the swing phase of gait.

Depending on the formulation used in the controller, the resulting stiffness properties of the exoskeleton can vary significantly. This results in different magnitudes and directions of supportive torques. Considering the strengths and weaknesses of the joint and end-point formulation for impedance controllers, we proposed a hybrid joint/end-point controller in order to exploit the benefits of the end-point controller in shaping a desired end-point stiffness, while using an additional joint component to guarantee the correct angular trajectories of the joints. In previous research, the concept of a hybrid controller was introduced for an industrial manipulator that was programmed to follow a given end-point trajectory in the presence of external disturbances (both at the end-effector and at the joint level) (Smith et al., 2015). The torques calculated by the end-point controller were complemented with the torques obtained from a joint impedance controller only at those joints that were affected by large disturbance forces. This approach was proven more effective than either end-point or joint control alone to reduce the tracking error in the presence of perturbations at the end-effector and at the joint level.

The control over the end-point stiffness also opens new possibilities when developing a controller with assist-as-needed characteristics. The AAN implemented in end-point space can be directly programmed to adapt the magnitude and the direction of the stiffness based on the error of the subject in task space. Our experiments showed that the controller was capable of adapting the end-point stiffness based on the deviation of the subject from the foot reference trajectory. As expected, the application with a subject with SCI resulted in a higher final end-point stiffness than the able-bodied subjects. When comparing the alignment of the end-point stiffness ellipses generated by the different controllers, we saw that in the hybrid joint/end-point controller the stiffness was better aligned with the error direction. In this way, the controller directs the restoring forces in the direction where they are needed, thus providing a more “specific” support.

Emken et al. (2008) employed a similar approach on the end-effector robotic gait trainer ARTHuR. The end-point stiffness of the robot was adapted with an AAN algorithm that separately adapted horizontal and vertical stiffness. Our approach differs in that the end-point stiffness ellipses align to the direction where the maximum stiffness is required (i.e., the direction of the error). Interestingly, this behavior is close to the way humans adapt their stiffness in response to external disturbances: as shown in Burdet et al. (2001), the CNS can voluntarily control the magnitude, shape and orientation of the end-point stiffness in the upper limb. Moreover, several studies have found that the control of the foot trajectory is the major focus of our CNS during locomotion, both in the unimpaired (Winter, 1992; Ivanenko et al., 2002) and in the impaired spinal cord (Ivanenko et al., 2003; Awai and Curt, 2014). Therefore, a controller for robotic exoskeletons that is shaping the end-point position and the end-point stiffness can be considered as a “bioinspired” solution for the control of robotic devices for human interaction.

A further advantage of the adaptive end-point controller is that the error metric for the algorithm can be defined in task space. This allows us to consider explicitly the interaction between the foot and the environment and the spatio-temporal features of the foot trajectory. As Winter (1992) showed, foot clearance is sensitive to very small angular deviations in any of the joints of the lower limb kinematic chain. This means that, in order to guarantee a safe minimum toe clearance, one would have to design very restrictive deadbands in joint space, which would have a negative impact on freedom of movement for physiological deviations. In contrast, deadbands in task space can be designed to be restrictive only in the directions that are needed for safety, determining how much deviation can be tolerated in the vertical direction (crucial for avoiding stumbling) and in the horizontal direction, which corresponds to leading or lagging with respect to the reference trajectory. Moreover, with the end-point controller, it is possible to present the subjects with visual feedback on the errors in end-point space, which is much easier to process than feedback on joint position (Banala et al., 2009; Koopman et al., 2013; Krishnan et al., 2013), and use the same representation within the error metric of the adaptation algorithm. When used in the experiments with subjects, the adaptive hybrid joint/end-point controller required the use of an additional term to support against gravity: the vertical stiffness was set 5 times higher than the horizontal stiffness. Alternatively, an additional feed-forward term for compensating the effect of gravity could be added. Furthermore, lighter robots (e.g., LOPES; Veneman et al., 2007) would reduce the role of gravity and inertia of the system and thus the need to counteract them. The accuracy of the end-point position can be increased by adding a position sensor which measures directly the x coordinates of the ankle, instead of estimating them from the joint angles. While we derived necessary conditions for stability based on the approach proposed by Kronander and Billard (2016) and we took several precautions to guarantee safety, the stability of the AAN hybrid controller is something that requires further investigation.

The single subject with SCI with whom we tested the adaptive and hybrid end-point/joint controller did not show abnormal muscle activation synergies. Extra care should be taken when using the hybrid control with patients that present abnormal synergies or other strong compensatory movements. There might be cases where, despite an almost physiological end-point trajectory, hip and knee angles remain anomalous (Awai and Curt, 2014). In such cases the hybrid control should be extended by a term that counteracts joint position deviations, as in the approach proposed by Smith et al. (2015), where a joint impedance term was added only when large disturbances at the joint level were detected. Before drawing any conclusions on the benefits of this novel controller in treating subjects with gait disabilities, more tests are needed to study how the controller would react to different impairments such as spasticity.

As a future step, the application of our adaptive hybrid joint/end-point controller concept to other rehabilitation robots, e.g., upper limb exoskeletons [such as the ARMin (Nef et al., 2007), Armeo^®Power (Hocoma AG) or ALEx (Pirondini et al., 2014)] would be of great interest, because a vast body of literature has investigated how humans adapt their upper-limb stiffness based on the task and on external disturbances (Shadmehr, 1993; Burdet et al., 2001) and it would be instructive to use an adaptive controller similar to the one presented in this work to test its interaction with the human arm.

Conclusion

The adaptive controller presented in this paper implements our ideas of a safe controller combining an end-point impedance controller with a joint damping controller into a “hybrid” joint/end-point controller. The controller was tested successfully with able-bodied human subjects and one subject with spinal cord injury. With this approach, it was possible to implement an adaptive controller that shapes the end-point stiffness according to the direction and the magnitude of the error performed at the ankle. In contrast to other applications, the hybrid controller adapts the end-point stiffness to selectively counteract certain errors while leaving the robot compliant in other directions. The adaptive controller proposed in this paper is a patient-cooperative, bio-inspired solution for more human-oriented rehabilitation robots, which fulfills the requirement of “adaptability” identified by many studies in the field of rehabilitation robotics (Iosa et al., 2016) and may be used on other devices, including upper extremity rehabilitation robots.

Author Contributions

SM contributed to the development of the controller described in this work and to the study design, performed the experiments, analyzed the data and wrote the manuscript draft. NR developed the controller described, performed the experiments, analyzed the data and revised the manuscript. LL provided significant advice during the whole study and revised the manuscript. AM-C conceived the study and the controller, supported significantly in the development of the controller, interpretation of the data, and contributed to the manuscript draft. All authors contributed to manuscript revision, read and approved the submitted version.

Funding

SM and LL were supported by the Industrial Academic Training Network Moving Beyond, funded through the European Community's Seventh Framework Programme FP7/2012 (under grant agreement No. 316639). AM-C was supported by the cereneo - Zentrum für Interdisziplinäre Forschung (cefir) Gemeinnützige Stiftung during the writing of this manuscript.

Conflict of Interest Statement

SM, LL, and AM-C at the time of the study worked at Hocoma AG in the R&D Department. SM, NR, and LL currently work at Hocoma AG.

Acknowledgments

The authors thank Dr. Marc Bolliger with the Spinal Cord Injury Center of the University Hospital Balgrist and Prof. Robert Riener for the support received during the study; Dr. Ellen Jaspers, Dr. Jaime E. Duarte, Dr. Nicolas Gerig, and Marcel Menner for providing valuable suggestions during the writing process; the subjects who participated to the experiments; Christopher Jarrett for proofreading the manuscript.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frobt.2018.00104/full#supplementary-material

Abbreviations

AAN, Assist-as-needed; PD, Proportional Derivative control; CNS, Central Nervous System; CoR, Center of Rotation; ROM, Range of Motion; SCI, Spinal Cord Injury.

References

Awai, L., and Curt, A. (2014). Intralimb coordination as a sensitive indicator of motor-control impairment after spinal cord injury. Front. Hum. Neurosci. 8:148. doi: 10.3389/fnhum.2014.00148

PubMed Abstract | CrossRef Full Text

Baker, R. (2013). Measuring Walking: A Handbook of Clinical Gait Analysis, 1st Edn. London: Mac Keith Press.

Banala, S. K., Agrawal, S. K., and Scholz, J. P. (2007). “Active leg exoskeleton (ALEX) for gait rehabilitation of motor-impaired patients,” in Proceedings of the 2007 IEEE 10th International Conference on Rehabilitation Robotics (Noordwijk).

Banala, S. K., Kim, S. H., Agrawal, S. K., and Scholz, J. P. (2009). Robot assisted gait training with active leg exoskeleton (ALEX). IEEE Trans. neural Syst. Rehabil. Eng. 17, 2–8. doi: 10.1109/BIOROB.2008.4762885

PubMed Abstract | CrossRef Full Text | Google Scholar

Begg, R., Best, R., Dell'Oro, L., and Taylor, S. (2007). Minimum foot clearance during walking: strategies for the minimisation of trip-related falls. Gait Posture 25, 191–198. doi: 10.1016/j.gaitpost.2006.03.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Burdet, E., Osu, R., Franklin, D., Milner, T., and Kawato, M. (2001). The central nervous system stabilizes unstable dynamics by learning optimal impedance. Nature 414, 446–449. doi: 10.1038/35106566

PubMed Abstract | CrossRef Full Text | Google Scholar

Cai, L., Fong, A., Otoshi, C., Liang, Y., Burdick, J., Roy, R., et al. (2006). Implications of assist-as-needed robotic step training after a complete spinal cord injury on intrinsic strategies of motor learning. J. Neurosci. 26, 10564–10568. doi: 10.1523/JNEUROSCI.2266-06.2006

PubMed Abstract | CrossRef Full Text | Google Scholar

Cao, J., Xie, S. Q., Das, R., and Zhu, G. L. (2014). Control strategies for effective robot assisted gait rehabilitation: the state of art and future prospects. Med. Eng. Phys. 36, 1555–1566. doi: 10.1016/j.medengphy.2014.08.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Colombo, G., Joerg, M., Schreier, R., and Dietz, V. (2000). Treadmill training of paraplegic patients using a robotic orthosis. J. Rehabil. Res. Dev. 37, 693–700. Available online at: https://www.rehab.research.va.gov/jour/00/37/6/colombo.htm

PubMed Abstract | Google Scholar

Duschau-Wicke, A., von Zitzewitz, J., Caprez, A., Luenenburger, L., and Riener, R. (2010). Path control: a method for patient-cooperative robot-aided gait rehabilitation. IEEE Trans. Neural Syst. Rehabil. Eng. 18, 38–48. doi: 10.1109/TNSRE.2009.2033061

PubMed Abstract | CrossRef Full Text | Google Scholar

Elguebaly, T., and Bouguila, N. (2013). Finite asymmetric generalized Gaussian mixture models learning for infrared object detection. Comput. Vis. Image Underst. 117, 1659–1671. doi: 10.1016/j.cviu.2013.07.007

CrossRef Full Text | Google Scholar

Emken, J. L., Benitez, R., and Reinkensmeyer, D. J. (2007). Human-robot cooperative movement training: learning a novel sensory motor transformation during walking with robotic assistance-as-needed. J. Neuroeng. Rehabil. 4:8. doi: 10.1186/1743-0003-4-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Emken, J. L., Bobrow, J. E., and Reinkensmeyer, D. J. (2005). “Robotic movement training as an optimization problem: designing a controller that assists only as needed,” in Proceedings of the 2005 IEEE 9th International Conference on Rehabilitation Robotics (Chicago, IL), 307–312.

Google Scholar

Emken, J. L., Harkema, S. J., Beres-jones, J. A., Ferreira, C. K., and Reinkensmeyer, D. J. (2008). Feasibility of manual teach-and-replay and continuous impedance shaping for robotic locomotor training following spinal cord injury. IEEE Trans. Biomed. Eng. 55, 322–334. doi: 10.1109/TBME.2007.910683

PubMed Abstract | CrossRef Full Text | Google Scholar

Esquenazi, A., Talaty, M., and Jayaraman, A. (2017). Powered exoskeletons for walking assistance in persons with central nervous system injuries: a narrative review. Phys. Med. Rehabil. 9, 46–62. doi: 10.1016/j.pmrj.2016.07.534

PubMed Abstract | CrossRef Full Text | Google Scholar

Gage, J. (1991). Gait Analysis in Cerebral Palsy. Oxford: Mac Keith Press.

Georgopoulos, A. P., and Grillner, S. (1989). Visuomotor coordination in reaching and locomotion. Science 245, 1209–1210. doi: 10.1126/science.2675307

PubMed Abstract | CrossRef Full Text | Google Scholar

Hogan, N. (1985). Impedance control: an approach to manipulation: Part I-theory. J. Dyn. Syst. Meas. Control. Trans. 107, 1–7. doi: 10.1115/1.3140702

CrossRef Full Text

Hogan, N., Krebs, H. I., Rohrer, B., Palazzolo, J. J., Dipietro, L., Fasoli, S. E., et al. (2006). Motions or muscles? Some behavioral factors underlying robotic assistance of motor recovery. J. Rehabil. Res. Dev. 43, 605–618. Available online at: https://www.rehab.research.va.gov/jour/06/43/5/Hogan.html

PubMed Abstract | Google Scholar

Hussain, S., Xie, S. Q., and Jamwal, P. K. (2013). Adaptive impedance control of a robotic orthosis for gait rehabilitation. IEEE Trans. Cybern. 43, 1025–1034. doi: 10.1109/TSMCB.2012.2222374

PubMed Abstract | CrossRef Full Text | Google Scholar

Hussain, S., Xie, S. Q., and Liu, G. (2011). Robot assisted treadmill training: mechanisms and training strategies. Med. Eng. Phys. 33, 527–533. doi: 10.1016/j.medengphy.2010.12.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Hussein, S., Schmidt, H., and Krüger, J. (2009). “Adaptive control of an end-effector based electromechanical gait rehabilitation device,” in 2009 IEEE 11th International Conference on Rehabilitation Robotics (Kyoto), 366–371.

Google Scholar

Iosa, M., Morone, G., Cherubini, A., and Paolucci, S. (2016). The three laws of neurorobotics: a review on what neurorehabilitation robots should do for patients and clinicians. J. Med. Biol. Eng. 36, 1–11. doi: 10.1007/s40846-016-0115-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Ivanenko, Y. P., Grasso, R., Macellari, V., and Lacquaniti, F. (2002). Control of foot trajectory in human locomotion: role of ground contact forces in simulated reduced gravity. J. Neurophysiol. 87, 3070–3089. doi: 10.11512/jn.00815.2001

PubMed Abstract | CrossRef Full Text | Google Scholar

Ivanenko, Y. P., Grasso, R., Zago, M., Molinari, M., Scivoletto, G., Castellano, V., et al. (2003). Temporal components of the motor patterns expressed by the human spinal cord reflect foot kinematics. J. Neurophysiol. 90, 3555–3565. doi: 10.1152/jn.00223.2003

PubMed Abstract | CrossRef Full Text | Google Scholar

Koopman, B., van Asseldonk, E. H., and van der Kooij, H. (2013). Selective control of gait subtasks in robotic gait training: foot clearance support in stroke survivors with a powered exoskeleton. J. Neuroeng. Rehabil. 10:3. doi: 10.1186/1743-0003-10-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Krishnan, C., Ranganathan, R., Dhaher, Y. Y., and Rymer, W. Z. (2013). A pilot study on the feasibility of robot-aided leg motor training to facilitate active participation. PLoS ONE 8:e77370. doi: 10.1371/journal.pone.0077370

PubMed Abstract | CrossRef Full Text | Google Scholar

Kronander, K., and Billard, A. (2016). Stability considerations for variable impedance control. IEEE Trans. Robot. 32, 1298–1305. doi: 10.1109/TRO.2016.2593492

CrossRef Full Text | Google Scholar

Kuo, A. D., and Donelan, J. M. (2010). Dynamic principles of gait and their clinical implications. Phys. Ther. 90, 157–174. doi: 10.2522/ptj.20090125

PubMed Abstract | CrossRef Full Text | Google Scholar

Lotze, M., Braun, C., Birbaumer, N., Anders, S., and Cohen, L. G. (2003). Motor learning elicited by voluntary drive. Brain 126, 866–872. doi: 10.1093/brain/awg079

PubMed Abstract | CrossRef Full Text | Google Scholar

Maggioni, S., Lunenburger, L., Riener, R., and Melendez-Calderon, A. (2015). “Robot-aided assessment of walking function based on an adaptive algorithm,” in 2015 IEEE 14th International Conference on Rehabilitation Robotics (Singapore), 804–809.

Google Scholar

Maggioni, S., Stucki, S., Lunenburger, L., Riener, R., and Melendez-Calderon, A. (2016). “A bio-inspired robotic test bench for repeatable and safe testing of rehabilitation robots,” in Proc. IEEE RAS EMBS Int. Conf. Biomed. Robot. Biomechatronics (Singapore), 894–899.

Google Scholar

Marchal-Crespo, L., and Reinkensmeyer, D. J. (2009). Review of control strategies for robotic movement training after neurologic injury. J. Neuroeng. Rehabil. 6:20. doi: 10.1186/1743-0003-6-20

PubMed Abstract | CrossRef Full Text | Google Scholar

Meuleman, J., van Asseldonk, E., van Oort, G., Rietman, H., and van der Kooij, H. (2016). LOPES II-design and evaluation of an admittance controlled gait raining robot with shadow-leg approach. IEEE Trans. Neural Syst. Rehabil. Eng. 24, 352–363. doi: 10.1109/TNSRE.2015.2511448

PubMed Abstract | CrossRef Full Text | Google Scholar

Mussa-Ivaldi, F., Hogan, N., and Bizzi, E. (1985). Neural, mechanical, and geometric factors subserving arm posture in humans. J. Neurosci. 5, 2732–2743. doi: 10.1056/NEJM197706162962404

PubMed Abstract | CrossRef Full Text | Google Scholar

Nef, T., Mihelj, M., and Riener, R. (2007). ARMin: a robot for patient-cooperative arm therapy. Med. Biol. Eng. Comput. 45, 887–900. doi: 10.1007/s11517-007-0226-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Nilsson, A., Vreede, K., Häglund, V., Kawamoto, H., Sankai, Y., and Borg, J. (2014). Gait training early after stroke with a new exoskeleton – the hybrid assistive limb: a study of safety and feasibility. J. Neuroeng. Rehabil. 11:92. doi: 10.1186/1743-0003-11-92

PubMed Abstract | CrossRef Full Text | Google Scholar

Perry, J. (1992). Gait Analysis. Normal and Pathological Function. Thorofare, NJ: SLACK Incorporated.

Pirondini, E., Coscia, M., Marcheschi, S., Roas, G., Salsedo, F., Frisoli, A., et al. (2014). “Evaluation of a new exoskeleton for upper limb post-stroke neuro-rehabilitation: preliminary results,” in Replace, Repair, Restore, Relieve – Bridging Clinical and Engineering Solutions in Neurorehabilitation, eds W. Jensen, O. Kæseler Andersen, and M. Akay (Cham: Springer), 637–645. doi: 10.1007/978-3-319-08072-7

CrossRef Full Text | Google Scholar

Riener, R., Lünenburger, L., Jezernik, S., Anderschitz, M., Colombo, G., and Dietz, V. (2005). Patient-cooperative strategies for robot-aided treadmill training: first experimental results. IEEE Trans. neural Syst. Rehabil. Eng. 13, 380–394. doi: 10.1109/TNSRE.2005.848628

PubMed Abstract | CrossRef Full Text | Google Scholar

Riener, R., Lünenburger, L., Maier, I. C., Colombo, G., and Dietz, V. (2010). Locomotor training in subjects with sensori-motor deficits: an overview of the robotic gait orthosis Lokomat. J. Healthc. Eng. 1, 197–216. doi: 10.1260/2040-2295.1.2.197

CrossRef Full Text | Google Scholar

Sekiya, N., Nagasaki, H., Ito, H., and Furuna, T. (1996). The invariant relationship between step length and step rate during free walking. J. Hum. Mov. Stud. 30, 241–257.

Google Scholar

Shadmehr, R. (1993). Control of equilibrium position and stiffness through postural modules. J. Mot. Behav. 25, 228–241. doi: 10.1080/00222895.1993.9942052

PubMed Abstract | CrossRef Full Text | Google Scholar

Shadmehr, R., and Mussa-Ivaldi, F. (1994). Adaptive representation of dynamics during learning of a motor task. J. Neurosci. 14, 3208–3224.

PubMed Abstract | Google Scholar

Smith, A., Yang, C., Ma, H., Culverhouse, P., Cangelosi, A., and Burdet, E. (2014). “Biomimetic joint/task space hybrid adaptive control for bimanual robotic manipulation,” in Proceedings-IEEE International Conference on Control & Automation (ICCA) (Taichung).

Smith, A., Yang, C., Ma, H., Culverhouse, P., Cangelosi, A., and Burdet, E. (2015). Novel hybrid adaptive controller for manipulation in complex perturbation environments. PLoS ONE 10:e0129281. doi: 10.1371/journal.pone.0129281

PubMed Abstract | CrossRef Full Text | Google Scholar

Stoquart, G., Detrembleur, C., and Lejeune, T. (2008). Effect of speed on kinematic, kinetic, electromyographic and energetic reference values during treadmill walking. Neurophysiol. Clin. 38, 105–116. doi: 10.1016/j.neucli.2008.02.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Vallery, H., Duschau-Wicke, A., and Riener, R. (2009). “Optimized passive dynamics improve transparency of haptic devices,” in Proceedings-IEEE International Conference on Robotics and Automation, 301–306. Available online at: http://www.scopus.com/scopus/record/display.url?fedsrfIntegrator=MEKPAPERS-SCOCIT&origin=fedsrf&view=basic&eid=2-s2.0-70350349707

Google Scholar

Veneman, J. F., Kruidhof, R., Hekman, E. E. G., Ekkelenkamp, R., Asseldonk, E. H. F., and Van van der Kooij, H. (2007). Design and evaluation of the LOPES exoskeleton robot for interactive gait rehabilitation. IEEE Trans. Neural Syst. Rehabil. Eng. 15, 379–386. doi: 10.1109/TNSRE.2007.903919

PubMed Abstract | CrossRef Full Text | Google Scholar

Winter, D. A. (1991). Biomechanics and Motor Control of Human Gait: Normal, Elderly and Pathological, 2nd Edn. Waterloo, ON: Waterloo Biomechanics.

Winter, D. A. (1992). Foot trajectory in human gait: a precise and multifactorial motor control task. Phys. Ther. 72, 45–53.

PubMed Abstract | Google Scholar

Winter, D. A. (1995). Human balance and posture control during standing and walking. Gait Posture 3, 193–214.

Google Scholar