The Benefit of Combining Neuronal Feedback and Feed-Forward Control for Robustness in Step Down Perturbations of Simulated Human Walking Depends on the Muscle Function

It is often assumed that the spinal control of human locomotion combines feed-forward central pattern generation with sensory feedback via muscle reflexes. However, the actual contribution of each component to the generation and stabilization of gait is not well understood, as direct experimental evidence for either is difficult to obtain. We here investigate the relative contribution of the two components to gait stability in a simulation model of human walking. Specifically, we hypothesize that a simple linear combination of feedback and feed-forward control at the level of the spinal cord improves the reaction to unexpected step down perturbations. In previous work, we found preliminary evidence supporting this hypothesis when studying a very reduced model of rebounding behaviors. In the present work, we investigate if the evidence extends to a more realistic model of human walking. We revisit a model that has previously been published and relies on spinal feedback control to generate walking. We extend the control of this model with a feed-forward muscle activation pattern. The feed-forward pattern is recorded from the unperturbed feedback control output. We find that the improvement in the robustness of the walking model with respect to step down perturbations depends on the ratio between the two strategies and on the muscle to which they are applied. The results suggest that combining feed-forward and feedback control is not guaranteed to improve locomotion, as the beneficial effects are dependent on the muscle and its function during walking.

1. Introduction

One way of trying to better understand human motor control is to simulate human walking with neuro-musculoskeletal models. These models generate locomotion by the interaction of their neuronal control and musculoskeletal dynamics, allowing to explore the consequences of different control approaches. Several such control approaches have evolved over time. One approach to modeling the control of human walking is to generate all muscle stimulation signals based on reflexes (Geyer and Herr, 2010; Song and Geyer, 2015), which relate proprioceptive feedback of muscle sensory organs to alpha-motor neurones (αMN) in the spinal cord. Another approach is to rely on central pattern generators (CPGs) as the primary source of muscle stimulation (Taga et al., 1991; Hase et al., 2003). CPGs are low-level circuits that can generate rhythmic patterns without external feedback stimuli, representing feed-forward control. Most models combine feed-forward and feedback control to produce walking, either by combining CPGs with general muscle reflexes (Ogihara and Yamazaki, 2001; Hase et al., 2003; Paul et al., 2005; Aoi et al., 2010; Kim et al., 2011; Dzeladini et al., 2014; van der Noot et al., 2015), or by using feed-forward signals (potentially forming a CPG) to set references for specific mono-synaptic stretch reflexes (Günther and Ruder, 2003). All of these models have in common, that no higher level multi-synaptic networks are required for steady-state walking and that they are robust to some perturbations in the environment.

The combination of feed-forward and feedback control is to some extend inspired by experimental observations and physiological evidence. Experiments demonstrate the contribution of reflexes as direct feedback to the muscle stimulation in animal (e.g., Donelan and Pearson, 2004) and human locomotion (Schneider et al., 2000; Grey et al., 2007; van der Linden et al., 2007). On the other hand, the existence of central pattern generators in the spinal cord providing rhythmic feed-forward control signals has been shown in animal (Orlovsky et al., 1999; Ijspeert, 2008) and discussed for human locomotion (MacKay-Lyons, 2002; Minassian et al., 2017). Experimental evidence further suggests that both, feedback and feed-forward commands simultaneously contribute to muscle stimulation in human locomotion (McDonagh and Duncan, 2002; Müller et al., 2015).

The co-existence of both signal types raises the question of what the benefit of their combination is. Kuo (2002) showed in a computer simulation with a simple model, consisting of a pair of antagonistic muscles driving one segment, that the combination of feed-forward and feedback signals allows to increase stability of rhythmic movements. This observation was confirmed in a very simple muscle-driven hopping simulation (Haeufle et al., 2012), where a linear combination of both signals allowed to reject a perturbation in ground level within one hopping cycle. The major reason for the benefit was, that a certain feed-forward contribution allowed to increase the muscle stimulation in preparation for ground contact. If the ground contact was delayed due to a drop in ground level, the leg muscle was already more active, and hence, could generate larger breaking forces early in the stance phase. This allows to partially compensate for the neuronal delays and offers a feed-forward strategy to prepare for ground contact. Although these studies show that the combination of feed-forward and feedback increases the stability and robustness of dynamic repetitive movements, it remains unclear whether these findings extend to more complex tasks such as human walking.

Here we investigate if the combination of feed-forward and feedback signals improves the robustness of walking, a more complex but highly relevant human movement. To this end, we resort to a previously published model of human walking (Geyer and Herr, 2010), which in its original form solely relies on spinal feedback control to generate walking. We extend this feedback control with feed-forward muscle stimulation patterns and study the change in the model's reaction to step down perturbations as a function of the ratio between the two control strategies. This highlights especially the role of feed-forward for unexpected touch-down conditions. We find that the general notion of an advantage of the combination of feed-forward and feedback control does not hold for all muscles, as they serve different purposes in walking.

2. Methods

2.1. Combining Feedback and Feed-Forward

We investigated the combination of feedback and feed-forward signals in the walking model of Geyer and Herr (2010). The model predicts the saggital kinematics and dynamics of human walking, as well as the muscle activities. It has two legs, each composed of three rigid segments (foot, shank, thigh), and an upper body connected by six joints idealized as single degree of freedom hinge-joints (Figure 1). The joints are actuated by 14 Hill-type muscle-tendon units considering the contraction dynamics of active muscle fibers, parallel passive muscle tissue, tendon elasticity, and first-order activation dynamics. For this study, we only modified the neuronal control, all other bio-mechanical properties were kept as in the original publication (see Geyer and Herr, 2010 for more details).

FIGURE 1

Figure 1. The walking model has two legs, actuated by seven Hill-type muscle-tendon units per leg (Geyer and Herr, 2010). Feed-forward signals in the model are triggered on the heel-off of the contralateral leg. A perturbation in the ground level of drop-height h causes a delayed touch down (perturbed TD) with respect to the expected touch down.

In the original formulation of this model, the muscle stimulation was generated solely by proprioceptive feedback. That is, the feedback stimulation signals $u_m^{FB}(t)$ for each muscle m were based on proprioceptive sensory signals P(t) e.g., from muscle spindles, Golgi tendon organs, or the vestibular organ. These signals are subject to neuronal delay ΔP and multiplied by synaptic gains G (Figure 2A). The proprioceptive feedback control strategy was described in detail by Geyer and Herr (2010).

FIGURE 2

Figure 2. Sketch of the approach to generate the feed-forward control signal and the combination of both control schemes. (A) First, the model is driven only by a reflex-based feedback control. Here, the stimulation to the alpha motor neurone (αMN) is recorded to a memory. (B) For the simulations with combined control strategies, the recorded stimulation signal is played back as a feed-forward signal. The feed-forward signal is triggered, when the heel of the contralateral leg leaves the ground (Figure 1). This represents the last known state of the environment. The parameter w linearly scales between feedback (w = 0) and feed-forward (w = 1) control. Modified from Geyer et al. (2003).

We generated feed-forward signals $u_m^{FF}(t)$ from the existing control, by recording the time trace of the original feedback-generated muscle stimulation $u_m^{FB}(t)$ in unperturbed steady-state walking (Animation 9, see Electronic Supplementary Material). This way, a feed-forward muscle stimulation signal can be derived without the need for optimization or models of a central pattern generator (in contrast to Haeufle et al., 2012 or Dzeladini et al., 2014, respectively). The feed-forward signals were triggered by the heel take-off of the contralateral leg, where the leg of interest is at the end of its swing phase. This represents the last point where the biological systems can estimate the ground level without the aid of visual feedback and cause the feed-forward pattern to stimulate the muscles as if the ground was level and no perturbation is to be expected.

To investigate the combination of feedback and feed-forward, we linearly combined the two signals as an input to the α-motor neuron (Figure 2B):

$\begin{array}{cc}{u}_m(t)=(1-w_m)·u_m^{FB}(t)+w_m·u_m^{FF}(t).& (1)\end{array}$

This leaves us with one free parameter for each muscle: the weighting factor w_m which quantifies the relative contribution of the feed-forward signal, with w_m∈[0, 1].

2.2. Effect of the Combination in a Simple Hopping Example

The effect of the combination can be best demonstrated with a simple example. For this, we exemplarily combine a previously published hopping model (Geyer et al., 2003) consisting only of one knee extensor muscle with the proposed linear combination of feed-forward and feedback (Haeufle et al., 2012). As this hopping model has only one muscle and investigates only the rebounding behavior, the effect can be demonstrated more clearly.

The hopping model of Geyer et al. (2003) was driven only by one muscle-tendon unit (Figure 3A). In unperturbed periodic hopping, positive force feedback generates a feedback signal u^FB(t) which is low in the beginning of the stance phase and rises with increasing muscle force during the stance phase (see Figure 3C). This signal was recorded and used as a feed-forward command u^FF(t) (Figure 2). The beginning of the feed-forward signal was triggered on the take-off, the last known state of the environment. Combining both signals according to Equation (1) (Figure 2B) has no effect on unperturbed hopping, as both signals u^FB(t) and u^FF(t) generate the same muscle stimulation. However, if we introduce a perturbation by lowering the ground level, the model behavior changes. The drop in ground level causes a delayed ground contact with higher kinetic energy of the center of mass leading to higher forces in the muscle. For pure feedback, the neuronal response to the perturbation only happens after the delayed ground contact resulting in a delayed sensor signal P(t) and hence, a delayed αMN stimulation (Figure 3C). For pure feed-forward, neither timing nor amplitude change: the feed-forward pattern does not consider the increased duration of the flight phase as it is triggered on the last unperturbed take-off and it does not consider the increased muscle strain. Hence, the muscle is already in a more active state at the instance of the delayed ground contact and does not rise as high (Figure 3C). Combining both according to Equation (1) allows to combine both effects. This can be exploited to increase the stability and reject the perturbation within one hopping cycle (Figure 3B).

FIGURE 3

Figure 3. (A) In a reduced hopping model (Geyer et al., 2003), only one muscle (knee extensor m. vastus) generates the anti-gravity forces. (B) Pure feed-forward (w = 1) and pure feedback (w = 0) control both lead to stable hopping. After a drop in ground level height (h = −0.05m, at t = 0s), the center of mass (CoM) reaches a periodic hopping height within two hopping cycles for the feed-forward control and within several cycles for the feedback control. Combining both control strategies (w = 0.3) allows to compensate for the perturbation within one hopping cycle. Shown here is only the upper part of the CoM trajectory to expand details of the adaptation to the perturbation. (C) The feedback muscle stimulation (w = 0) is generated by positive force feedback which causes rapidly rising stimulation after touchdown with a neuronal delay of ∆P = 0.015s. In presence of a perturbation, the feedback based stimulation (w = 0) can only increase after the actual touchdown, when the muscle is stretched and generates forces in the interaction with the ground. A feed-forward pattern triggered to the last touchdown (w = 1) starts stimulating the muscle shortly after the expected touchdown for level hopping ↓1* and is therefore already more active at the actual touchdown ↓1 in the case of a drop down perturbation (h = −0.05m, at t = 0s). Linearly combining both strategies (w = 0.3) results in an intermediate behavior, which is beneficial to the stability of the hopping pattern.

This application of the control method to the simple hopping model demonstrates the potential benefit, as it would generally be expected. In the following, we will investigate how this concept generalizes to walking. Walking is more complex and the model has to achieve more movement requirements than just stabilizing vertical rebounding behavior, e.g., stabilizing the trunk, generating ground clearance in swing phase, generating push-off for swing leg acceleration, etc.

2.3. Model Analysis

To investigate the robustness of the walking model, we implemented ground perturbations by modifying the ground level

$\begin{array}{cc}{y}_{\text{0}}\text{= 0m + }h\text{,}& (2)\end{array}$

where h is the perturbation height. For the majority of the simulation experiments, the perturbation height was negative, meaning a drop in ground level. We define a walking pattern to be robust, if the model successfully continues to walk for at least 10 s (approximately 20 steps) after the perturbation. In general, this means that the model returned to steady state walking after the perturbation. Although the focus of this study was on step down perturbations, we also investigated small step up perturbations up to the limit of ground clearance (h = 0.014m), as well as walking on downhill slopes.

For the analysis of the model, we changed only the weighting factors w_m between feed-forward and feedback and investigated the robustness by increasing the step down perturbation height h∈{−0.01, −0.02, …} m. First, we modified the feed-forward contribution only for single muscles (w_{m, i}∈{0, 0.1, 0.2, …, 0.9, 1}) while all other muscles were driven by pure feed-forward (w_{m, exept i} = 0). In a second step, we looked into simultaneous feed-forward contributions to several muscles. For this, we performed a systematic search, where we investigated all linear combinations of two muscles with varying feed-forward contributions, while the other five muscles per leg were still only driven by pure feedback signals.

3. Results

The reference model with pure feedback control (w_m = 0 for all muscles) tolerates height perturbations of up to h = −0.03m. Larger perturbations lead to a collapse in the first step after the ground height change (Anim. 0).

3.1. Control Combination Applied to Individual Muscles

Adding a feed-forward contribution to either the knee or the ankle extensor muscles increases walking robustness. For instance, with a feed-forward contribution of w_VAS = 0.4 to the m. vastus (VAS), the model can tolerate larger step down perturbations of up to h = −0.05m. The effect of the feed-forward contribution is similar to what was shown in section 2.1 for the extensor muscle in the hopping model: A pure feedback contribution can act only after ground contact, delaying the onset of muscle stimulation in the case of a step down perturbation (Figure 4A). Adding the feed-forward contribution generates an earlier onset of the muscle stimulation, reducing muscle strain and increasing robustness (Figure 4B, between ↓1* and ↓1). Similarly, a feed-forward contribution to either the m. soleus (SOL, w_SOL = 0.2) or the biarticular m. gastrocnemius (GAS, w_GAS = 0.6) increases the model tolerance to larger step down perturbations (up to h = −0.07m in case of GAS, more than twice the maximum perturbation of the feedback controlled reference model). However, for the other muscles, no level of individual feed-forward contribution increases walking robustness.

FIGURE 4

Figure 4. This figure shows the change in stimulation u_VAS and knee joint angle for increasing perturbation height h with the same feed-forward / feedback combination: 0% in the left column, and 40% in the right column. Step size increases from h = 0m (black) to h = −0.07m. Heel contact is indicated by ↓1, heel-off by ↑2, and toe-off by ↑3. Heel-contact and heel-off marked with *represent the reference case (0%, level walking). (A) It becomes visible, how an increased perturbation first delays and in consequence increases m. vastus stimulation, if no feed-forward contribution is present. (C) The delayed stimulation causes the knee to further flex in the stance phase. In consequence, the model stumbles. (B) For the combined control (40%), the delay is partially compensated by the triggered feed-forward stimulation pattern. (D) Hence, the knee is less flexed and stumbling is prevented for larger perturbations.

Furthermore, for all muscles the level of feed-forward contribution affects walking robustness. For example, the robustness of VAS increases with increasing feed-forward contribution until w_VAS = 0.4 (Table 1, Anim. 1). Larger feed-forward contributions quickly diminish robustness, and no stable walking is possible for w_VAS≥0.6 (Anim. 2). The muscle group most sensitive to the level of feed-forward contribution are the hip flexors HFL, for which w_HFL≥0.3 already destabilizes the walking pattern. On the other extreme, GAS requires relatively high contributions w_GAS≥0.5 (Anim. 4) and is the only muscle that can be stimulated by a pure feed-forward signal (w_GAS = 1) without reducing walking robustness. In effect, the level of feed-forward contribution has to be chosen carefully for each muscle and, in most muscles, cannot be too large without destabilizing the walking pattern, suggesting that, within the spinal control structure proposed by the walking model, a CPG-like drive as the sole provider of muscle stimulations fails to generate walking.

TABLE 1

Table 1. The table shows successful (✓) and unsuccessful (×) walking simulations for variations in perturbation height h and feed-forward contribution ω in m.vastus (VAS).

3.2. Control Combination Applied to Muscle Groups

A systematic search with feed-forward signals applied to several muscles reveals that some combinations can further improve walking robustness while others destabilize walking or change the entire walking pattern. For instance, combining the most robust case for VAS (w_VAS = 0.4) with simultaneous feed-forward contributions to the hip muscles shows that it improves robustness with GLU (Table 2b, Anim. 5) but not with HFL (Table 2a), which destabilized walking for w_HFL≥0.3. On the other hand, a combination of SOL (w_SOL = 0.2) and HFL (w_HFL = 0.4) is robust to perturbations of up to three times the pure feed-forward robustness (h = −0.09m, Anim. 7). Here, the recovery after the perturbation takes several steps, but the steady walking pattern still appears natural. In contrast, the most robust case was found for a combination with GAS (w_GAS = 0.8) and GLU (w_GLU = 0.1), which increased the step tolerance to h = −0.12m but resulted in a slower and unnaturally looking steady walking pattern (Anim. 8).

TABLE 2

Table 2. The tables (a,b) show successful (✓) and unsuccessful (×) walking simulations if two muscles have a feed-forward contribution.

A more in depth description of the resulting walking and stimulation patterns can be found in the Electronic Supplementary Material, along with the animations and the model output data.

3.3. Other Perturbations

The possibility to investigate perturbations with positive heights in this model is very limited. The model has a relatively low ground clearance of only about 0.014 m (see for example Anim. 9). This means, that the maximum perturbation height for positive heights is limited. All such perturbations can directly be compensated by the pure feedback control scheme (w=0, Anim. 10, h = 0.012[m]). Larger step up perturbations lead to an early ball contact in mid-stance resulting in stumbling, which cannot be compensated by just adding feed-forward signals.

Feed-forward contributions to some muscles also allows to walk down steeper slopes than for pure feedback control, where the maximum slope angle is ${\alpha }_{max}=3.4^{°}$. The maximum slope angle ${\alpha }_{max}=5.7^{°}$ for any contribution of m. vastus is achieved with w_VAS = 0.4 (Anim. 11), which is the same feed-forward contribution which leads to the maximum step down perturbation. Also feed-forward contributions to m. soleus lead to larger maximum slope angles (${\alpha }_{max}=5.7^{°}$ for w_SOL = 0.1) while they destabilize the walking if applied e.g., to the hip flexor muscle group (${\alpha }_{max}=2.3^{°}$ for w_HFL = 0.4). Overall, the effect is similar to the step down perturbations.

4. Discussion

Presumably, a combination of feed-forward and feedback contributions should improve control performance. While this notion has been confirmed for comparably simple neuro-musculoskeletal models and tasks (Kuo, 2002; Haeufle et al., 2012), we find that it may be misleading for more complex behaviors. We introduced step down perturbations (modification of ground contact height) in a human walking model and studied the effect of combining feed-forward and feedback control of muscles to the robustness of walking. Our results show that combining the two control types can improve walking robustness when applied to some muscles but not to others, suggesting the benefit of a control combination cannot be presumed and requires to consider individual muscle function.

4.1. Benefit for Robustness Depends On Muscle Function

The anti-gravity muscles (VAS, SOL, GAS) of the walking model benefit from adding feed-forward control. One key function of these muscles is to counteract the gravitational force and reversing the body's vertical motion in stance. Although, overall, legs behave similar to elastic springs in steady running and walking (Blickhan, 1989; McMahon and Cheng, 1990; Geyer et al., 2006; Lipfert, 2010), a drop in ground level results in an increased impact velocity and additional kinetic energy (Müller and Blickhan, 2010, e.g., in running) that has to be dissipated or redirected by the muscles in the leg (Kalveram et al., 2012). As demonstrated for hopping in section 2.1), a feed-forward control for an anti-gravity muscle can ensure a rising stimulation in anticipation of a larger impact due to the drop in ground level. In contrast, a stimulation based purely on feedback control does not sufficiently stimulate the muscle early in the stance phase (Haeufle et al., 2012). In humans as well as in the walking model, the stimulation of VAS (Figure 4) rises in the initial stance phase (Hof et al., 2002; Geyer and Herr, 2010). With a pure feedback control, however, this stimulation can only increase after ground contact (Figure 4A vs. Figure 4B). In the case of an unexpected drop in ground level, the knee bends more than usual (Figure 4C, between ↓¹ and ↓²) due to the additional kinetic energy and the walking pattern cannot be recovered. With the early onset of the feed-forward stimulation, the leg generates larger braking forces dissipating the additional energy and avoiding excessive knee bending (Figure 4D, between ↓¹ and ↓²). As a result, the maximum perturbation height increases (Table 1).

The bi-articular m.gastrocnemius partially works as anti-gravity muscle like m. soleus at the ankle. However, it also flexes the knee. Adding feed-forward to m.gastrocnemius therefore influences knee and ankle joint and has two consequences: Firstly, in the perturbed step (e.g., h = −0.07m), the original feedback model flexes the swing leg's knee less compared to level walking and the stance phase is not capable of keeping the CoM high enough above ground. Combined, this results in an early ground contact of the swing leg's foot (too short stride length) and the model stumbles. Adding feed-forward to m. gastrocnemius increases swing leg knee flexion and therefore prevents stumbling. The second consequence is, that the relatively high feed-forward contributions in m. gastrocnemius, which actually improve robustness up to h = −0.07m (for w_GAS = 0.8), also result in a modification of the gait pattern with longer stance phases and early heel lift-offs. This can be attributed to the bi-articular function of m. gastrocnemius which also coordinates the energy release in the push-off phase (Lipfert et al., 2014) where the timing between ankle and knee joint is crucial (van Ingen Schenau et al., 1987). The feed-forward contribution perturbs this coordination.

Other muscles do not benefit from adding feed-forward control. For instance, feedforward contributions to the hip muscles (HFL and GLU) rather tend to destabilize the model. These muscles primarily serve a different function in walking, maintaining trunk balance, which requires a reactive control scheme (Horak and Nashner, 1986; Tang et al., 2012; Müller et al., 2017). A higher feed-forward contribution reduces the ability of these muscles to adequately react to a perturbed orientation of the trunk, especially for the HFL, which in normal locomotion becomes active in late stance. Only in combination with other muscles, a small feed-forward contribution to GLU improved robustness by partially compensating for the increased forward lean of the trunk early in the perturbed stance phase. Experiments even suggest, that a feed-forward strategy may be employed to erect the trunk in case of an unexpected drop (Müller et al., 2014, 5.5deg for a h = −0.1 m drop). In combination with m. vastus' feed-forward stimulation compensating for the additional kinetic energy, a slight m. gluteus feed-forward stimulation (w_GLU = 0.2 w_VAS = 0.4) to erect the trunk after the perturbation allows to compensate for a maximum drop height of h = −0.06 m.

Similarly, muscles which are active in the swing phase (hip flexor group, hamstrings, and tibialis anterior) do not benefit from a feed-forward contribution in the model. Similar to the trunk stabilization in stance, the leg in swing requires reactive control to achieve proper foot placement, which is critical for maintaining gait stability (Pratt et al., 2006; Hof, 2008). A feedforward contribution triggered at heel-off of the contralateral leg is likely out of sync once the ipsilateral leg enters the swing phase after perturbations. Resetting the feedforward contribution at characteristic transitions in the gait may resolve this issue and has been suggested in the literature for CPGs (Righetti and Ijspeert, 2006; Aoi et al., 2010; Aoi and Tsuchiya, 2011; Li et al., 2017).

4.2. Robustness May Be Improved by More Complex Models for the Signal Combination

We expect that refinements of the feed-forward contributions can improve walking robustness. The feed-forward patterns we tested, were recorded signals as generated by the original feedback scheme. We expect that in human control such feed-forward patterns are learned and optimized by the organism based on the interaction with the feedback control. In addition, the simple linear combination of the control contributions is likely oversimplifying the actual situation. For instance, it is known that the signal contributions are modulated during the walking cycle, as indicated by the H-reflex amplitude modulation of the ankle extensors (e.g., Crenna and Frigo, 1987; Schneider et al., 2000). Such a modulation over the gait cycle of the contributions of different signal types may increase the benefit of a combined feed-forward and feedback control (Song and Geyer, 2015). For instance, GLU could in this way prepare the trunk orientation by feed-forward control around impact and correct the orientation during the late stance phase by pure feedback. All these extensions may allow to bring the result closer to the commonly expected benefit of feed-forward control to robustness.

4.3. Generalization of the Concept to Other Perturbations

The possibility to investigate perturbations with positive heights in this model is very limited, as the normal walking pattern has only about 0.014m ground clearance (see for example Anim. 9). They can directly be compensated by the pure feedback control scheme (w = 0, Anim. 10, h = 0.012[m]).

The results for drop down perturbations generalize to walking down a slope. Here, also, the feed-forward contribution to the anti-gravity muscles is beneficial and allow to compensate for the additional kinetic energy in each step. However, the improvement we found was not as consistent as for the step down perturbations, i.e., some flatter slopes may not be compensated for. This means that an improvement may be achieved, however it may require online tuning of the weighting and may not be possible by simply adding the same feed-forward at all times and cases. For the step down perturbations this is unrealistic, as they appear as a sudden unexpected perturbation where no online tuning is possible. Here they have to (and do) work consistently for all perturbations until a certain maximum height. For a slope, such an adaptation may, however, be feasible as the slope may be detected visually or adapted to within the first or second step.

As demonstrated by Song and Geyer (2015), a more recent version of the model allows to simulate other gaits by varying the control parameters of the feedback terms. This may allow to also study different walking patterns and perturbations in the future. Although the found benefit generalizes to slope down walking, other perturbations, like pushes or slips may not necessarily benefit from the combination.

Validating the response to such perturbations experimentally in humans is still difficult. While the general model approach can reproduce a set of perturbation responses quite well (Song and Geyer, 2017), it is difficult to distinguish between feed-forward and feedback contributions in muscle activity recordings (Müller et al., 2015). Only invasive techniques may be able to shed some light on this (Grey et al., 2007). However, we see potential in studying this in exoskeleton supported or prosthetic walking, by varying the contributions in an augmented control scheme.

4.4. A purpose of Feed-Forward and Feedback Combination Beyond Robustness?

While it is difficult to distinguish in human walking experiments between feed-forward and feedback contributions, perturbation studies suggest the coexistence of both (Schneider et al., 2000; McDonagh and Duncan, 2002; Grey et al., 2007; van der Linden et al., 2007; Müller et al., 2015). The reported benefit of this coexistence to robustness with respect to external perturbations (Kuo, 2002; Haeufle et al., 2012), however, may not be the only purpose of such control combinations. Alternative suggestions include that internal changes of the system such as changes in muscle force may be better compensated by a combined control scheme (Yakovenko et al., 2004). In addition, Dzeladini et al. (2014) demonstrated an increase in the agility of locomotion. By modulating a feed-forward CPG pattern that was combined with the reflex control scheme of the walking model investigated here, they demonstrated changes in walking speed without the need to re-adjust the synaptic gains of the reflex loops. Other potential benefits include the compensation of inevitable feedback delays by internal feed-forward models (Wolpert and Miall, 1996; Wolpert and Flanagan, 2001) and the reduction of control complexity and effort by relying on feed-forward patterns of low neuronal complexity (Haeufle et al., 2014).

5. Conclusion

We have shown with a simulation model of human walking that a simple linear combination of (central pattern generated) feed-forward control and (reflex generated) feedback control can improve walking robustness when compared to isolated feed-forward or feedback control. But we have also shown that this outcome cannot be presumed in general, as the beneficial effects of combining the two strategies are dependent on the muscle and its function during gait. Furthermore, the positive effect was shown for a step down perturbation an may generalize to similar experiments like slope down walking, but not necessarily to other perturbation experiments.

Author Contributions

DH, SSch, and RM conceived of the study and designed the simulation experiments. HG implemented the walking model. BS modified the model and ran the simulations. BS, DH, and RM analyzed the data. All authors contributed to the manuscript. All authors gave final approval for publication.

Funding

The research of DH was supported by the Ministry of Science, Research and the Arts Baden-Württemberg (Az: 33-7533.-30-20/7/2). The research of RM was supported by the German Research Foundation (DFG; MU2970/4-1). SSch was supported by the Deutsche Forschungsgemeinschaft via Cluster of Excellence Simulation Technology (EXC310 SimTech).

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We would like to thank Sten Grimmer for the discussions leading to the idea of this manuscript.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fncom.2018.00080/full#supplementary-material

References

Aoi, S., Ogihara, N., Funato, T., Sugimoto, Y., and Tsuchiya, K. (2010). Evaluating functional roles of phase resetting in generation of adaptive human bipedal walking with a physiologically based model of the spinal pattern generator. Biol. Cybern. 102, 373–387. doi: 10.1007/s00422-010-0373-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Aoi, S., and Tsuchiya, K. (2011). Generation of bipedal walking through interactions among the robot dynamics, the oscillator dynamics, and the environment: stability characteristics of a five-link planar biped robot. Auton. Robots 30, 123–141. doi: 10.1007/s10514-010-9209-9

CrossRef Full Text | Google Scholar

Blickhan, R. (1989). The spring-mass model for running and hopping. J. Biomech. 22, 1217–1227. doi: 10.1016/0021-9290(89)90224-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Crenna, P., and Frigo, C. (1987). Excitability of the soleus H-reflex arc during walking and stepping in man. Exp. Brain Res. 66, 49–60. doi: 10.1007/BF00236201

PubMed Abstract | CrossRef Full Text | Google Scholar

Donelan, J. M., and Pearson, K. G. (2004). Contribution of force feedback to ankle extensor activity in decerebrate walking cats. J. Neurophysiol. 92, 2093–2104. doi: 10.1152/jn.00325.2004

PubMed Abstract | CrossRef Full Text | Google Scholar

Dzeladini, F., van den Kieboom, J., and Ijspeert, A. (2014). The contribution of a central pattern generator in a reflex-based neuromuscular model. Front. Hum. Neurosci. 8:371. doi: 10.3389/fnhum.2014.00371

PubMed Abstract | CrossRef Full Text | Google Scholar

Geyer, H., and Herr, H. (2010). A muscle-reflex model that encodes principles of legged mechanics produces human walking dynamics and muscle activities. IEEE Trans. Neural Syst. Rehabil. Eng. 18, 263–273. doi: 10.1109/TNSRE.2010.2047592

PubMed Abstract | CrossRef Full Text | Google Scholar

Geyer, H., Seyfarth, A., and Blickhan, R. (2003). Positive force feedback in bouncing gaits? Proc. R. Soc. Lond. Ser. B 270, 2173–2183. doi: 10.1098/rspb.2003.2454

CrossRef Full Text

Geyer, H., Seyfarth, A., and Blickhan, R. (2006). Compliant leg behaviour explains basic dynamics of walking and running. Proc. R. Soc. Lond. Ser. B 273, 2861–2867. doi: 10.1098/rspb.2006.3637

PubMed Abstract | CrossRef Full Text | Google Scholar

Grey, M. J., Nielsen, J. B., Mazzaro, N., and Sinkjaer, T. (2007). Positive force feedback in human walking. J. Physiol. 581(Pt 1), 99–105. doi: 10.1113/jphysiol.2007.130088

PubMed Abstract | CrossRef Full Text | Google Scholar

Günther, M., and Ruder, H. (2003). Synthesis of two-dimensional human walking: a test of the lambda-model. Biol. Cybern. 89, 89–106. doi: 10.1007/s00422-003-0414-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Haeufle, D. F. B., Grimmer, S., Kalveram, K.-T., and Seyfarth, A. (2012). Integration of intrinsic muscle properties, feed-forward and feedback signals for generating and stabilizing hopping. J. R. Soc. Interface 9, 1458–1469. doi: 10.1098/rsif.2011.0694

PubMed Abstract | CrossRef Full Text | Google Scholar

Haeufle, D. F. B., Günther, M., Wunner, G., and Schmitt, S. (2014). Quantifying control effort of biological and technical movements: an information-entropy-based approach. Phys. Rev. E 89:012716. doi: 10.1103/PhysRevE.89.012716

CrossRef Full Text | Google Scholar

Hase, K., Miyashita, K., Ok, S., and Arakawa, Y. (2003). Human gait simulation with a neuromusculoskeletal model and evolutionary computation. J. Visual. Comput. Animat. 14, 73–92. doi: 10.1002/vis.306

CrossRef Full Text | Google Scholar

Hof, A. L. (2008). The ‘extrapolated center of mass' concept suggests a simple control of balance in walking. Hum. Mov. Sci. 27, 112–125. doi: 10.1016/j.humov.2007.08.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Hof, A. L., Elzinga, H., Grimmius, W., and Halbertsma, J. P. K. (2002). Speed dependence of averaged EMG profiles in walking. Gait Posture 16, 78–86. doi: 10.1016/S0966-6362(01)00206-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Horak, F. B., and Nashner, L. M. (1986). Central programming of postural movements: adaptation to altered support-surface configurations. J. Neurophysiol. 55, 1369–1381. doi: 10.1152/jn.1986.55.6.1369

PubMed Abstract | CrossRef Full Text | Google Scholar

Ijspeert, A. J. (2008). Central pattern generators for locomotion control in animals and robots: a review. Neural Netw. 21, 642–653. doi: 10.1016/j.neunet.2008.03.014

PubMed Abstract | CrossRef Full Text | Google Scholar

Kalveram, K. T., Haeufle, D. F. B., Seyfarth, A., and Grimmer, S. (2012). Energy management that generates terrain following versus apex-preserving hopping in man and machine. Biol. Cybern. 106, 1–13. doi: 10.1007/s00422-012-0476-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Kim, Y., Tagawa, Y., Obinata, G., and Hase, K. (2011). Robust control of CPG-based 3D neuromusculoskeletal walking model. Biol. Cybern. 105, 269–282. doi: 10.1007/s00422-011-0464-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Kuo, A. D. (2002). The relative roles of feedforward and feedback in the control of rhythmic movements. Motor Control 6, 129–145. doi: 10.1123/mcj.6.2.129

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, W., Szczecinski, N. S., and Quinn, R. D. (2017). A neural network with central pattern generators entrained by sensory feedback controls walking of a bipedal model. Bioinspir. Biomimet. 12:065002. doi: 10.1088/1748-3190/aa8290

CrossRef Full Text | Google Scholar

Lipfert, S., Gunther, M., Renjewski, D., and Seyfarth, A. (2014). Impulsive ankle push-off powers leg swing in human walking. J. Exp. Biol. 217:1831. doi: 10.1242/jeb.107391

CrossRef Full Text | Google Scholar

Lipfert, S. W. (2010). Kinematic and Dynamic Similarities between Walking and Running. Ph.D. thesis, Friedrich-Schiller University, Jena.

MacKay-Lyons, M. (2002). Central pattern generation of locomotion: a review of the evidence. Phys. Ther. 82, 69–83. doi: 10.1093/ptj/82.1.69

PubMed Abstract | CrossRef Full Text | Google Scholar

McDonagh, M. J. N., and Duncan, A. (2002). Interaction of pre-programmed control and natural stretch reflexes in human landing movements. J. Physiol. 544, 985–994. doi: 10.1113/jphysiol.2002.024844

PubMed Abstract | CrossRef Full Text | Google Scholar

McMahon, T. A., and Cheng, G. C. (1990). The mechanics of running: how does stiffness couple with speed? J. Biomech. 23(Suppl. 1), 65–78. doi: 10.1016/0021-9290(90)90042-2

CrossRef Full Text

Minassian, K., Hofstoetter, U. S., Dzeladini, F., Guertin, P. A., and Ijspeert, A. (2017). The human central pattern generator for locomotion. Neuroscientist 3, 649–663. doi: 10.1177/1073858417699790

CrossRef Full Text | Google Scholar

Müller, R., and Blickhan, R. (2010). Running on uneven ground: leg adjustments to altered ground level. Hum. Mov. Sci. 29, 578–589. doi: 10.1016/j.humov.2010.04.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Müller, R., Haeufle, D. F. B., and Blickhan, R. (2015). Preparing the leg for ground contact in running: the contribution of feed-forward and visual feedback. J. Exp. Biol. 218, 451–457. doi: 10.1242/jeb.113688

PubMed Abstract | CrossRef Full Text | Google Scholar

Müller, R., Rode, C., Aminiaghdam, S., Vielemeyer, J., and Blickhan, R. (2017). Force direction patterns promote whole body stability even in hip-flexed walking, but not upper body stability in human upright walking. Proc. R. Soc. A 473:20170404. doi: 10.1098/rspa.2017.0404

CrossRef Full Text | Google Scholar

Müller, R., Tschiesche, K., and Blickhan, R. (2014). Kinetic and kinematic adjustments during perturbed walking across visible and camouflaged drops in ground level. J. Biomech. 47, 2286–2291. doi: 10.1016/j.jbiomech.2014.04.041

PubMed Abstract | CrossRef Full Text | Google Scholar

Ogihara, N., and Yamazaki, N. (2001). Generation of human bipedal locomotion by a bio-mimetic neuro-musculo-skeletal model. Biol. Cybern. 84, 1–11. doi: 10.1007/PL00007977

PubMed Abstract | CrossRef Full Text | Google Scholar

Orlovsky, G., Deliagina, T. G., and Grillner, S. (1999). Neuronal Control of Locomotion: From Mollusc to Man, 1st Edition. Oxford: Oxford University Press.

Google Scholar

Paul, C., Bellotti, M., Jezernik, S., and Curt, A. (2005). Development of a human neuro-musculo-skeletal model for investigation of spinal cord injury. Biol. Cybern. 93, 153–170. doi: 10.1007/s00422-005-0559-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Pratt, J., Carff, J., Drakunov, S., and Goswami, A. (2006). “Capture point: a step toward humanoid push recovery,” in 2006 6th IEEE-RAS International Conference on Humanoid Robots (Genova), 200–207.

Google Scholar

Righetti, L., and Ijspeert, A. J. (2006). “Programmable central pattern generators: an application to biped locomotion control,” in Proceedings 2006 IEEE International Conference on Robotics and Automation (Orlando, FL), 1585–1590.

Google Scholar

Schneider, C., Lavoie, B. a., and Capaday, C. (2000). On the origin of the soleus H-reflex modulation pattern during human walking and its task-dependent differences. J. Neurophysiol. 83, 2881–2890. doi: 10.1152/jn.2000.83.5.2881

PubMed Abstract | CrossRef Full Text | Google Scholar

Song, S., and Geyer, H. (2015). A neural circuitry that emphasizes spinal feedback generates diverse behaviours of human locomotion. J. Physiol. 593, 3493–3511. doi: 10.1113/JP270228

PubMed Abstract | CrossRef Full Text | Google Scholar

Song, S., and Geyer, H. (2017). Evaluation of a neuromechanical walking control model using disturbance experiments. Front. Comput. Neurosc. 11:15. doi: 10.3389/fncom.2017.00015

CrossRef Full Text | Google Scholar

Taga, G., Yamaguchi, Y., and Shimizu, H. (1991). Self-organized control of bipedal locomotion by neural oscillators in unpredictable environment. Biol. Cybern. 65, 147–159. doi: 10.1007/BF00198086

PubMed Abstract | CrossRef Full Text | Google Scholar

Tang, K. S., Honegger, F., and Allum, J. H. J. (2012). Movement patterns underlying first trial responses in human balance corrections. Neuroscience 225, 140–151. doi: 10.1016/j.neuroscience.2012.09.004

PubMed Abstract | CrossRef Full Text | Google Scholar

van der Linden, M. H., Marigold, D. S., Gabreels, F. J., and Duysens, J. (2007). Muscle reflexes and synergies triggered by an unexpected support surface height during walking. J. Neurophysiol. 97, 3639–3650. doi: 10.1152/jn.01272.2006

PubMed Abstract | CrossRef Full Text | Google Scholar

van der Noot, N., Ijspeert, A. J., and Ronsse, R. (2015). “Biped gait controller for large speed variations, combining reflexes and a central pattern generator in a neuromuscular model,” in 2015 IEEE International Conference on Robotics and Automation (ICRA) (Seattle, WA), 6267–6274.

Google Scholar

van Ingen Schenau, G. J., Bobbert, M. F., and Rozendal, R. H. (1987). The unique action of bi-articular muscles in complex movements. J. Anat. 155, 1–5.

PubMed Abstract | Google Scholar

Wolpert, D. M., and Flanagan, J. R. (2001). Motor prediction. Curr. Biol. 11, R729–R732. doi: 10.1016/S0960-9822(01)00432-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Wolpert, D. M., and Miall, R. C. (1996). Forward models for physiological motor control. Neural Netw. 9, 1265–1279. doi: 10.1016/S0893-6080(96)00035-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Yakovenko, S., Gritsenko, V., and Prochazka, A. (2004). Contribution of stretch reflexes to locomotor control: a modeling study. Biol. Cybern. 90, 146–155. doi: 10.1007/s00422-003-0449-z

PubMed Abstract | CrossRef Full Text | Google Scholar