High-level motor planning allows flexible walking at different gait patterns in a neuromechanical model

Humans can freely adopt gait parameters like walking speed, step length, or cadence on the fly when walking. Planned movement that can be updated online to account for changes in the environment rather than having to rely on habitual, reflexive control that is adapted over long timescales. Here we present a neuromechanical model that accounts for this flexibility by combining movement goals and motor plans on a kinematic task level with low-level spinal feedback loops. We show that the model can walk at a wide range of different gait patterns by choosing a small number of high-level control parameters representing a movement goal. A larger number of parameters governing the low-level reflex loops in the spinal cord, on the other hand, remain fixed. We also show that the model can generalize the learned behavior by recombining the high-level control parameters and walk with gait patterns that it had not encountered before. Furthermore, the model can transition between different gaits without the loss of balance by switching to a new set of control parameters in real time.

1 Introduction

Human locomotion is amazingly flexible. We can avoid obstacles of different sizes and shapes, precisely step to a suitable location with a variety of swing foot trajectories (Zhang et al., 2020), and walk with a wide range of overall gait patterns (Inman et al., 1981; Ackermann and van den Bogert, 2012; Steele et al., 2012). Despite plenty of empirical studies on human walking behavior, however, the concrete neuromuscular processes that generate and regulate human locomotion are still not well understood. During locomotion, the central nervous system must generate a stable, rhythmic movement pattern that moves the body in a certain direction in space with a relatively constant speed while keeping movements in other directions to a minimum (Inman et al., 1981). For each step, the swing leg must be moved to a new location over an appropriate time, and errors in either time or location of the step will perturb the overall stability of the walking body. The stance leg, on the other hand, needs to generate forces against the ground that prevent the body from collapsing due to gravitational forces and propel the body forward to maintain a steady movement speed (Reimann et al., 2020). Furthermore, as the body traverses throughout the gait cycle, movement generation and control for the two legs must dynamically switch between stance and swing. Despite these challenges, humans are not only capable of easily performing steady state locomotion but also of smoothly adapting their locomotion patterns to external requirements (Zhang et al., 2020). They can walk on their tiptoes, step over obstacles, or bend their knees while walking. Their gait patterns are often characterized by high-level parameters such as walking speed, step length, step width, and stepping cadence, and they can generally freely vary these parameters and walk with a variety of different gait patterns (Nilsson and Alf, 1987).

The flexibility of human walking behavior is contrasted by the highly repetitive nature of the gait cycle (Clark, 2015). A large body of research has shown that stable locomotion patterns are generated solely by spinal structures in insects (Mantziaris et al., 2020), lampreys (Ayers et al., 1983), and cats (Perret et al., 1988; Kiehn, 2016). Less is known about how humans generate stable, rhythmic walking patterns or how the flexibility of high-level human movement generation is integrated with the rhythmic, repetitive patterns usually associated with spinal structures. One way to test conjectures that integrate high-level motor planning, low-level spinal modules and reflexes, and the musculoskeletal biomechanics of the body and environment is to develop computational models those include all factors of interest. In such models, it is possible to independently manipulate specific factors and observe the resulting effect on the movement pattern (Allen and Ting, 2016; De Groote and Antoine, 2021). Most neuromuscular models for walking use spinal circuits to generate the rhythmic movement patterns. These models are based either on the central pattern generators (Taga, 1995; Van der Noot et al., 2018; Aoi et al., 2019) or finite state machines that organize the model’s behavior on the basis of its state using specialized reflex modules (Günther and Ruder, 2003; Song and Geyer, 2015; Ong et al., 2019). They can reproduce human walking behavior (Song and Geyer, 2015; Ong et al., 2019) and are, to some degree, flexible. Existing models of this style can walk at different speeds (Günther and Ruder, 2003; Van der Noot et al., 2018; Ong et al., 2019; Di Russo et al., 2021), change the walking direction (Song and Geyer, 2015; Van der Noot et al., 2018), step over obstacles (Taga, 1998), or vary their gait parameters (Di Russo et al., 2021).

Flexibility in existing models is largely limited to specific variations, such as speed modulation or increasing the toe clearance during swing. We postulate that these limitations of existing neuromuscular models are due to their almost exclusively spinal nature and the lack of supraspinal motor planning and control. Experimental studies of human walking suggest a duality of steps as both 1) part of the rhythmic movement patterns of the whole body and 2) reaching movements with the foot (Clark, 2015). We previously presented a model that attempts to bridge this gap by integrating high-level, voluntary movement planning for the swing leg with low-level, habitual control (Ramadan et al., 2022). The model could avoid obstacles, vary movement speed and walking direction, and perform goal-directed movements with the swing leg by executing a motor plan to reach a kinematic goal, without re-optimizing model parameters. Since generation of planned, voluntary movements is restricted to the swing leg, variations of overall gait parameters such as step length, cadence, and the resulting movement speed are limited. The movement speed could be varied by increasing trunk lean and exploiting the interaction between balance control and speed, but step length and cadence could not be controlled independently in the way humans are clearly capable of.

Here, we develop a model of walking that combines planned, voluntary movements with habitual, reflexive control for both the swing and stance leg, with the goal of reaching a similar level of flexibility in walking as observed in humans. To challenge flexibility, our goal is to have the model capable of walking at a large range of different gaits, represented here by the two gait parameters step length and cadence. The high-level controller represents movement goals as a set of eight control parameters, consisting of desired kinematic states for joint angles or body parts (see Section 2.1 for details). For any set of control parameters, the controller plans a movement to a goal in the kinematic task space. The task-level movement plan is then transformed into descending motor commands that integrate with spinal structures using a combination of internal models and neural networks. Our specific research goals are 1) to show that it is possible to generate stable walking patterns as a planned and voluntary movement, with gait parameters spanning the range typically adopted by humans. To this end, we use evolutionary optimization to find sets of control parameters that will generate a given gait pattern. This results in a large set of individually learned sets of high-level control parameters, each of which produces a stable walking pattern with different gait parameters. 2) To test whether the learned gaits can be generalized by interpolation in the space of control parameters to walk with gait patterns that were not previously learned. Success would show that the high-level voluntary control is sufficient to generate any desired gait pattern within a reasonable range. 3) To test whether the model can transition between different gaits in real time. Success would show that the motor behavior generated by the model is robust, without losing balance when transitioning between stable states.

In Section 2, we introduce the model used in this work. Section 3 describes the optimizations conducted to learn high-level parameter sets and presents the approach used to walk at and transition between arbitrary cadences and step lengths. Section 4 describes the results obtained from the simulation experiments, and in Section 5, we discuss insights and conclusion from the results and limitations as well as possible model extensions.

2 Model

We present a neuromechanical model of human locomotion that is based on the work of Ramadan et al. (2022) and spans high-level movement planning and coordination, spinal reflex arcs, muscle physiology, and skeletal biomechanics. Figure 1 provides an overview of the major model components. A finite state machine organizes the gait phases and switches each leg between early swing, late swing, and stance phases, depending on feedback from ground contact and measured movement time. In the supraspinal layer, a volition module represents the task-level movement goals for the swing and stance leg. Movement goals are desired kinematic states, that is, positions, velocities, or accelerations of joint angles or specific body parts. A planning module generates a task-level motor plan to reach any given movement goal. A movement plan is generated in the form of a trajectory that moves the task variable from its current state to the desired state in a given time. Movement plans are updated in real time based on sensory feedback. A combination of a neural network and an explicit internal model then transforms the high-level motor plans into descending motor commands by inverting dynamics, forces, the muscle model, and the stretch reflex. The resulting descending commands interface with the neural control modules in the spinal cord to execute the movement plan. The spinal control includes a generic stretch reflex for each muscle. This stretch reflex uses proprioceptive information about muscle length and velocity as input, compares the input to an activation threshold, and activates motor neurons in proportion to the difference between the sensed muscle state and reference threshold. The reference threshold is modulated by the descending motor commands. In addition to this general stretch reflex, the stance leg is controlled by specific functional reflex modules that (1) generate compliant leg behavior, 2) prevent knee overextension, and 3) balance the trunk. Motorneural activation is fed into Hill-type muscle models that actuate a three-dimensional biomechanical model. The following section provides a detailed description of the innovations made to the work presented in Ramadan et al. (2022). For details on the adapted parts of the model, please refer to Ramadan et al. (2022).

FIGURE 1

FIGURE 1. Overview of the model architecture. In the supraspinal layer, motor goals for the swing leg are desired kinematic states, set by a volition module and modulated by balance control feedback. A motor plan toward these goal joint configurations is generated by minimal jerk trajectories that can be updated during execution. For the stance leg, the goal for propulsion is a desired forward acceleration of the trunk, transformed into a joint-level motor plan by an inverse kinematics module. An internal inverse model comprising biomechanics, muscle moment arms, muscle activation properties, and the spinal stretch reflex transforms the motor plan into descending commands that execute the planned movement. The descending commands are integrated with the stretch reflex in the spinal layer. The stance leg is additionally controlled by three functional reflex modules that stabilize the trunk (Sharbafi and Seyfarth, 2015), generate leg compliance and prevent overextension of the knee (Song and Geyer, 2015). Spinal motor neurons activate Hill-type muscle–tendon units that actuate the biomechanical model in the environment. A finite-state machine organizes switches between early swing phase, late swing phase, and stance phase. The concrete feedback paths are described in detail in the equations below.

2.1 High-level task variables

2.1.1 Propulsion

The model generates propulsion by pushing against the ground with the stance leg. We define the task variable as a desired forward acceleration of the trunk center of mass a_x. This desired acceleration is constant and applied throughout the stance phase of each leg. The role of the constant acceleration from the propulsion module is to offset the deceleration generated by friction and energy dissipation from viscoelastic elements in the muscles and connective tissue. This deceleration is consistent throughout the gait cycle, though the average level of energy loss depends on the chosen gait pattern and speed. We chose constant acceleration as the simplest form to counter the consistent energy loss, instead of a more complex, state-dependent term. Specifically, there is no desired speed that is explicitly controlled via a sensory feedback loop.

In order to execute the planned whole-body acceleration with the DoF of the stance leg, we transform a_x into desired joint accelerations for the stance leg. We compute the current velocity vector v_com of the trunk CoM as

$v_{\text{com}}=\left[\begin{array}{c}\hfill v_{x,\mathrm{c}\mathrm{o}\mathrm{m}}\hfill \\ \hfill v_{y,\mathrm{c}\mathrm{o}\mathrm{m}}\hfill \\ \hfill v_{z,\mathrm{c}\mathrm{o}\mathrm{m}}\hfill \\ \hfill {\omega }_{x,\mathrm{c}\mathrm{o}\mathrm{m}}\hfill \\ \hfill {\omega }_{y,\mathrm{c}\mathrm{o}\mathrm{m}}\hfill \\ \hfill {\omega }_{z,\mathrm{c}\mathrm{o}\mathrm{m}}\hfill \end{array}\right]=\left[\begin{array}{c}\hfill j_x\hfill \\ \hfill j_y\hfill \\ \hfill j_z\hfill \\ \hfill j_{rx}\hfill \\ \hfill j_{ry}\hfill \\ \hfill j_{rz}\hfill \end{array}\right]\left[\begin{array}{c}\hfill {\dot{\theta }}_{\text{hipfl}}\hfill \\ \hfill {\dot{\theta }}_{\text{hipab}}\hfill \\ \hfill {\dot{\theta }}_{\text{knee}}\hfill \\ \hfill {\dot{\theta }}_{\text{ankle}}\hfill \\ \hfill {\dot{\theta }}_{\text{ball}}\hfill \end{array}\right]=\mathbf{J}{\dot{\theta }}_{\text{st}},(1)$

where v_i and ω_i are the translational and rotational components of the trunk velocity, J = J(θ_st) the Jacobian matrix of partial derivatives relating changes in trunk configuration to changes in joint angles, θ, with row vectors j_*, and ${\dot{\theta }}_{*}$ are the angular velocities of the corresponding joints. We use the Jacobian that relates the trunk CoM to the four degrees of freedom of the corresponding stance leg and an extra hinge joint at the balls of the foot for motion of the foot segment around the contact during pushing off, for a total of five degrees of freedom represented as θ_st.

Deriving Eq. 1 by time yields

$a_{\text{com}}=\dot{\mathbf{J}}\left(\theta \right){\dot{\theta }}_{\text{st}}+\mathbf{J}\left(\theta \right){\ddot{\theta }}_{\text{st}}\approx \mathbf{J}\left(\theta \right){\ddot{\theta }}_{\text{st}}.(2)$

Here we neglect the velocity-dependent term using $\dot{J}(\theta )$. This is reasonable because the leg configuration changes relatively slowly during stance. The resulting mapping of the desired acceleration of the whole-body CoM to joint accelerations is sufficiently accurate to generate walking movements, which is the overall goal, showing that neglecting this term is acceptable. The vector a_com is six dimensional comprising both translational and rotational degrees of freedom. The stance leg model, however, has only four degrees of freedom that are actuated by muscles, and we have the constraint of being unable to apply torques at the unactuated degree of freedom at the football, and the hip abduction degree of freedom, although actuated, does not move the trunk in the anterior–posterior direction. We solve the inverse kinematic problem while accounting for these constraints as

${\widetilde{\stackrel{̈}{\theta }}}_{\text{st}}={\left[\begin{array}{c}\hfill j_x\hfill \\ \hfill j_{rz}\hfill \\ \hfill m_{\text{ball}}\hfill \\ \hfill m_{\text{hipab}}\hfill \end{array}\right]}^{+}\left[\begin{array}{c}\hfill {\tilde{a}}_x\hfill \\ \hfill 0\hfill \\ \hfill 0\hfill \\ \hfill 0\hfill \end{array}\right](3)$

where j_x, j_rz are the components of the trunk CoM Jacobian that affect forward translation and rotation in the sagittal plane. The additional two constraint rows ensure that the torques at the hip abduction joint and the unactuated joint at the ball of the foot will be zero, where m_ball and m_hipab are the rows of the mass matrix (compare Eq. 5 below) that relate torques in these two degrees of freedom to accelerations across all five joints. The tilde in ${\widetilde{\stackrel{̈}{\theta }}}_{\text{st}}$ and ${\tilde{a}}_x$ indicates that these are the desired quantities determined by the high-level controller, instead of the actual kinematic states.

2.1.2 Swing leg task variables

Freely choosing a gait pattern requires the ability to flexibly modify the kinematic trajectory of the swing leg. In Ramadan et al. (2022), we presented a neuromuscular modeling approach to generate flexible swing leg movements that we followed and expanded in this study. The model generates a motor plan to bring a kinematic task variable from its current state to a goal state in a desired time, using a minimal jerk trajectory. A minimal jerk trajectory is a trajectory that transports a variable from an initial to a goal state in a given time, while observing kinematic constraints (Flash and Hogan, 1985). It has been widely adopted as a model for human reaching movement generation (Engelbrecht, 2001) and extended to generate joint angle trajectories (Andani and Bahrami, 2012). We represent the movement goal as the desired joint angles for hip flexion, knee flexion, and ankle flexion. As in Ramadan et al. (2022), we subdivide the swing phase into an early and a late swing phase and define a set of target joint angles for the end of each subphase. Specifically for ankle flexion, we define only one target joint angle for the entire swing phase as the task variable, since ankle flexion does not change its direction of movement during the swing phase in normal human walking (Lim et al., 2017). The total movement time T_sw of the swing phase is another control parameter. This results in a total number of six task variables for the swing leg that are used as control parameters to modulate the overall gait pattern: ${\theta }_{\text{sw,\hspace{0.17em}hipfl}}^{\text{tar,\hspace{0.17em}early}},{\theta }_{\text{sw,\hspace{0.17em}knee}}^{\text{tar,\hspace{0.17em}early}},{\theta }_{\text{sw,\hspace{0.17em}hipfl}}^{\text{tar,\hspace{0.17em}late}},{\theta }_{\text{sw,\hspace{0.17em}knee}}^{\text{tar,\hspace{0.17em}late}},{\theta }_{\text{sw,\hspace{0.17em}ankle}}^{\text{tar}}$ and T_sw. The target hip flexion angle is additionally modulated by a feedback control law:

${\tilde{\theta }}_{\text{sw,\hspace{0.17em}hipfl}}={\theta }_{\text{sw,\hspace{0.17em}hipfl}}^{\text{tar},\ast }+c_d^{\ast }\widehat{d}+c_v^{\ast }\widehat{v}-{\widehat{\varphi }}_{\text{trunk}},(4)$

for balance, where $\widehat{d}$ and $\widehat{v}$ are the time-delayed horizontal difference between the center of pressure (CoP) and the trunk position and its velocity; $c_d^{\ast }$ and $c_v^{\ast }$ are feedback gains, with the ^∗ indicating early and late swing; and ${\widehat{\varphi }}_{\text{trunk}}$ is the time-delayed trunk orientation. Eq. (4) is applied independently for the sagittal and frontal plane orientations of the thigh. The control parameters of the frontal plane, however, are not treated as control variables in this study. All details about the implementation of movement goals, minimal jerk trajectories, and balance control have been introduced in Ramadan et al. (2022). Time delays are adopted from Song and Geyer (2015) and have been reported in Ramadan et al. (2022).

2.1.3 Trunk reference lean

The forward lean of the trunk during locomotion has a substantial effect on the gait. It changes the mass distribution within the body, moving the CoM forward relative to the feet and increasing the lever arm of the gravitational force around the pivot point at the stance foot ankle (Reimann et al., 2020). In Ramadan et al. (2022), modulation of the trunk lean reference angle was used to change the average walking speed of the model. In humans, faster walking is associated with increased forward lean of the trunk (Ahmad Sharbafi and Seyfarth, 2017). Here, we use trunk reference lean as one of several high-level control variables to modulate an overall gait pattern. To regulate trunk lean, we define a reference angle for the stance leg hip flexion joint as a supraspinal movement goal. This reference angle ${\theta }_{\text{st,\hspace{0.17em}hipfl}}^{\text{ref}}$ is directly sent into the spinal cord as a descending command, where it interacts with the force-modulated compliant-hip reflex module (see Section 2.3.1). This parameter is constant and does not require movement planning or the modulation by an internal model. Trunk balance in the lateral direction is controlled as in Ramadan et al. (2022).

2.2 Internal model

The supraspinal control modules for propulsion and the swing leg movement generate motor plans that specify desired accelerations of individual joint angles. We transform these accelerations into descending commands that modulate the reflex arcs in the spinal cord to generate motorneural activation patterns that execute the planned movement. To this end, we use a sequence of inverse models, implemented as a combination of neural networks and explicit algebraic equations that invert the forward equations for spinal control, muscle physiology, and biomechanics, with some simplifying approximations. The following section describes the individual components of this sequence of internal models.

2.2.1 Inverse dynamics

Motor plans at the task level are represented by vectors of desired joint accelerations: ${\ddot{\theta }}_{\text{st}}$ for the stance leg and ${\ddot{\theta }}_{\text{sw}}$ for the swing leg. Executing the motor plan means realizing these planned joint accelerations. We compute torque profiles that realize the planned joint acceleration by an inverse model of the biomechanics. For the stance leg, we approximate the relationship between the joint accelerations and joint torques by

${\tilde{\tau }}_{\text{st}}=\mathbf{M}\left(\widehat{\theta }\right){\widetilde{\stackrel{̈}{\theta }}}_{\text{st}}.(5)$

The mass matrix $\mathbf{M}(\widehat{\theta })$ relates joint accelerations to joint torques as part of the equation of motion, and $\widehat{\theta }$ are the time-delayed sensor estimates of the body configuration. Neglecting the velocity-dependent torques is reasonable because the leg moves relatively little during the stance phase. The gravitational components, which do contribute significantly during the stance phase, are assumed to be compensated by spinal reflexes (see Section 2.3.1). This mapping of the desired joint acceleration to joint torques, in combination with the spinal reflexes countering the gravitational forces, is sufficiently accurate to generate walking movements, as shown in the Results section below, indicating that neglecting these terms is acceptable. The resulting vector of the desired stance leg joint torques, ${\tilde{\tau }}_{\text{st}}$, is composed of five components (see Section 2.1.1). The component of the unactuated degree of freedom at the ball of the foot, however, is zero due to the constraints used in Eq. 3 and is disregarded for further considerations.

For the swing leg, we solve the complete equation of motion by including gravitational and interaction terms for the four actuated joints of the swing leg, resulting in a vector of desired joint torques ${\tilde{\tau }}_{\text{sw}}$. For details on the inverse dynamics applied at the swing leg, please refer to Ramadan et al. (2022).

2.2.2 Inversion of muscle force generation and stretch reflex

Here, the goal is to find a descending motor command that interfaces with the stretch reflex to generate the desired joint torques ${\tilde{\tau }}_{\text{st}}$ and ${\tilde{\tau }}_{\text{sw}}$. We first find a vector of muscle forces that generates the desired joint torques, then define a descending command that will interface with the spinal stretch reflex to generate the desired force at each muscle.

Since the number of muscles exceeds the number of joints of the model, there is an infinite number of possible muscle force vectors F that potentially realize a desired torque vector $\tilde{\tau }$. From the space of the muscle force vectors that realize the desired torque, we select the force vector that minimizes the total amount of squared muscle forces across all muscles and contains only positive muscle forces. Therefore, we solve the constrained minimization problem

$\tilde{F}=\underset{\begin{array}{c}\mathbf{R}\left(\theta \right)\tilde{F}=\tau \\ {\tilde{F}}_i\ge 0\end{array}}{\min }\sum _i{\tilde{F}}_i^2,(6)$

using the interior point algorithm, where R(θ) is the matrix of moment arms mapping muscle forces to joint torques. In order to reduce computational load of the model, we trained a feed-forward neural network that approximates this minimum. Details on the training of the network can be found in Section 6.1 in the Appendix.

To find a motorneural stimulation level $\tilde{S}$ that generates the desired force ${\tilde{F}}_i$ for each muscle, we invert the characteristics of the Hill-type muscle model as

${\tilde{S}}_i=\frac{{\tilde{F}}_i}{F_{i,\mathrm{m}\mathrm{a}\mathrm{x}}{f}_l\left(l_{\text{ce}}\right)f_v\left(v_{\text{ce}}\right)},(7)$

where F_i,max is the maximal force; f_l and f_v represent the force-length and force-velocity characteristics; and l_ce and v_ce are the length and lengthening velocity of the Hill-type muscles (Geyer et al., 2003). Finally, to define a descending command that interfaces with the spinal stretch reflex to generate the desired level of stimulation ${\tilde{S}}_i$, we invert the stretch reflex as

$u=\frac{{\tilde{S}}_i+h}{K_l}-l_{\text{ce}},(8)$

where K_l is the gain and h is the resting level of the spinal stretch reflex (see Section 2.3.2). For details on these inversions and the associated simplifying approximations, please see Ramadan et al. (2022).

2.3 Spinal control mechanisms

In addition to the high-level motor planning and control described above, the model relies on various spinal reflex pathways. Each muscle has a general stretch reflex mapping the proprioceptive feedback about the muscle length directly to motorneural activation of the same muscle. During the stance phase, additional reflex arcs are used to realize specific functions during locomotion. This section describes these reflex mechanisms in detail.

2.3.1 Force-modulated compliant hip

To stabilize the trunk, we use an approach following the force-modulated compliant hip mechanism (FMCH) introduced by Sharbafi and Seyfarth (2015). In human experiments, it has been observed that the hip torque at the stance leg generated to balance the trunk can be approximated by a force-modulated spring:

${\tau }_{\text{st,\hspace{0.17em}hipfl}}=c{F}_s\left({\theta }_{\text{st,\hspace{0.17em}hipfl}}-{\theta }_{\text{st,\hspace{0.17em}hipfl}}^{\text{ref}}\right).(9)$

Here, F_s is the force that the combined stance leg joint torques apply to the trunk segment at the hip, θ_{st, hipfl} is the stance leg hip joint flexion angle, and c is a constant gain factor. The reference hip angle ${\theta }_{\text{hipfl}}^{\text{ref}}$ is the descending command that is generated in the supraspinal layer.

We approximate this behavior by activating the biarticular hip muscles as

$S_{\text{FMCH},i}={\left[c_i{F}_s\left(t-\mathrm{\Delta }t\right)\left({\theta }_{\text{st,\hspace{0.17em}hipfl}}\left(t-\mathrm{\Delta }t\right)-{\theta }_{\text{st,\hspace{0.17em}hipfl}}^{\text{ref}}\right)\right]}^{+},(10)$

where i indicates one of the two biarticular muscles spanning the hip and knee joints: hamstring and rectus femoris. We restrict the balance control to these two muscles because human experiments suggest that trunk balance is mainly realized by biarticular muscles (Sarmadi et al., 2019; Schumacher et al., 2019). It has also been shown that the use of biarticular muscles has biomechanical advantages when generating horizontal forces (Hof, 2001). The leg force F_s is approximated as the force

$F_s=\frac{F_{\text{VAS}}{m}_{\text{VAS}}+F_{\text{SOL}}{m}_{\text{SOL}}}{0.032}.(11)$

Here, F_* and m_* are the forces and moment arms of the soleus and vastus groups, which are the two monoarticular muscles that generate compliant leg behavior and act on the hip joint center. The factor 0.032 approximates the transformation from the joint torques to a force vector acting on the hip. The feedback about muscle force is assumed to be provided by Golgi tendon organs and feedback about the joint angle is measured by a combination of muscle spindle and Golgi tendon organ feedback (Kistemaker et al., 2013; Prochazka, 2013).

2.3.2 Generic stretch reflex and functional reflex modules

Each muscle is always innervated by the generic stretch reflex:

$S_{\text{STR}}={\left[K_l\left({\widehat{l}}_{\text{ce}}+u\right)+K_v\left({\widehat{v}}_{\text{ce}}+\dot{u}\right)-h\right]}^{+}(12)$

The stretch reflex uses proprioceptive feedback from the muscle spindles, mapping the time-delayed sensor estimates of length ${\widehat{l}}_{\text{ce}}$ and velocity ${\widehat{v}}_{\text{ce}}$ of the contractile element to motorneural stimulation S_STR of the Hill-type muscle, relative to a threshold u that is modulated by the descending command. The gain factors K_l and K_v are constant and the same for all muscles, and h represents the resting level for the neural activity. The rate of change of the descending command, $\dot{u}$, is computed as the numerical derivative of u. Time delays for the neural control structures are reported in Ramadan et al. (2022).

In addition to this generic stretch reflex, we adopt a subset of the functional reflex modules for the stance leg used in Ramadan et al. (2022), first described by Song and Geyer (2015) and Geyer and Herr (2010). These reflexes 1) generate compliant leg behavior and 2) prevent knee overextension by mapping proprioceptive information about muscle length, velocity, and force from muscle spindles and Golgi tendon organs to motorneural stimulation. For details about these functional reflex modules, please refer to Song and Geyer (2015) and Ramadan et al. (2022).

2.4 Integration of different reflexes

The generic stretch reflex, modulated by descending commands according to the motor plan, and the functional reflex modules are integrated in the spinal cord. During swing, the leg muscles are exclusively activated by the generic stretch reflex stimulation S_STR described in Eq. (12). During stance, we integrate the generic stretch reflex with the dedicated reflex modules that implement force-modulated compliant hip behavior (S_FMCH) and the compliant stance leg (S_CL) and prevent knee overextension (S_PKO) by adding the components to

$S=S_{\text{STR}}+{\left[S_{\text{FMCH}}+S_{\text{CL}}+S_{\text{PKO}}\right]}^{+},(13)$

to generate the total motorneural stimulation S that activates each Hill-type muscle.

2.5 Biomechanics and muscle model

We adapt the biomechanics and muscle model from Ramadan et al. (2022). The biomechanics model is three dimensional and has a total of 14 degrees of freedom. Internal degrees of freedom are four actuated joints at each leg, representing hip flexion/extension, hip abduction/adduction, knee flexion/extension, and ankle plantar/dorsiflexion. Six free-body degrees of freedom at the trunk segment allow the model to move freely in space. Ground reaction forces are implemented using four contact spheres at each foot, two at the heels and two at the balls. Muscle tendon units are modeled as the standard Hill-type muscles. For more details, please refer Ramadan et al. (2022).

3 Optimization and simulation studies

Our first goal is to establish that the proposed model may generate stable walking patterns as a planned, voluntary movement, with gait parameters spanning the range typically adopted by humans. Target gaits are represented by two gait parameters for step length and cadence. The model behavior is parameterized by a set of eight control parameters, representing the target angle for the swing leg hip flexion (${\theta }_{\text{sw,\hspace{0.17em}hipfl}}^{\text{tar,\hspace{0.17em}early}}$, ${\theta }_{\text{sw,\hspace{0.17em}hipfl}}^{\text{tar,\hspace{0.17em}late}}$) and knee flexion (${\theta }_{\text{sw,\hspace{0.17em}knee}}^{\text{tar,\hspace{0.17em}early}}$, ${\theta }_{\text{sw,\hspace{0.17em}knee}}^{\text{tar,\hspace{0.17em}late}}$) at the end of the early and late swing phases, the swing leg ankle at the end of the swing $({\theta }_{\text{sw,\hspace{0.17em}ankle}}^{\text{tar}})$, the total movement time of the leg swing (T_sw), the reference angle for the stance leg hip flexion to regulate trunk lean $({\theta }_{\text{st,\hspace{0.17em}hipfl}}^{\text{ref}})$, and the constant propulsion to maintain forward velocity$({\tilde{a}}_x)$. We also include the initial walking speed $v_y^{\text{init}}$, as an optimization parameter. Adding the initial speed facilitates learning, since the model does not have to relax to a steady state from an arbitrarily chosen initial speed. The initial walking speed has no bearing on the resulting steady state gait, however, it is not considered a control parameter. In addition to the eight control parameters, the model has eight gain parameters for balance control and 27 parameters for the spinal reflexes (Ramadan et al., 2022), totaling 43 parameters.

As a first step to achieve our goal, we optimize all 43 model parameters once to find a parameter set for stable walking, without constraining the gait pattern. In a second step, we optimize the eight control parameters and the initial walking speed of the model v^init to find settings for specific targets for the gait parameters step length and cadence, while leaving all other parameters fixed. This optimization is repeated multiple times with different target gait parameters, to form a library of control parameter sets for different gaits spanning the range of normal human walking. We then analyze whether this control approach allows generalization by interpolating between learned gaits to generate new, previously not learned gait patterns. We also test whether it is possible to transition between different gait patterns in real time without the loss of stability. The following sections describe these steps in detail.

3.1 Optimization of self-selected gait

The presented neuromuscular model contains a total number of 43 control parameters for high-level goal representation and supraspinal balance control, as well as spinal parameters such as reflex gains, resting lengths, and basis stimuli.

We perform one single optimization of all 43 model parameter to find a model that walks at a self-selected step length and cadence. We adopt the evolutionary algorithm used in Song and Geyer (2015) and Ramadan et al. (2022), which is based on the covariance-matrix adaptation technique (Hansen et al., 2006). As a cost function, we define

$J_{\mathit{cost}}=\left\{\begin{array}{cc}{c}_0-x_{\text{fall}}\hfill & \mathrm{i}\mathrm{f}\mathrm{f}\mathrm{a}\mathrm{l}\mathrm{l}\hfill \\ \frac{1}{2}{c}_0+d_{\text{steady}}\hfill & \mathrm{e}\mathrm{l}\mathrm{s}\mathrm{e}.\hfill \end{array}\right.(14)$

The first part of the cost function generates basic walking without falling and the second part generates steady locomotion. The constant c₀ = 10³ is a normalization factor, x_fall is the distance the model walked before falling, and d_steady measures the “steadyness” of the gait (see Ramadan et al., 2022). The model that results from this first optimization walks with a cadence of 101 steps per minute with a step length of 0.86 m.

3.2 Optimization of high-level control parameters

To show that it is possible to generate stable walking patterns as a planned, voluntary movement, we now find sets of control parameters that walk at specific values for step length and cadence. We use the same optimization technique as in Section 3.1, but only optimize the set of eight high-level control parameters and the initial velocity of the model. To find a desired gait pattern, we expand the cost function in Eq. 14 in the following way:

$J_{\mathit{cost}}=\left\{\begin{array}{cc}{c}_0-x_{\text{fall}}\hfill & \mathrm{i}\mathrm{f}\mathrm{f}\mathrm{a}\mathrm{l}\mathrm{l}\hfill \\ c_1+|\left(v-v_{\text{tar}}\right)|\hfill & \mathrm{i}\mathrm{f}|\left(v-v_{\text{tar}}\right)|>{\delta }_{\text{v}}\hfill \\ c_2+|\left(cad-{cad}_{\text{tar}}\right)|\hfill & \mathrm{i}\mathrm{f}|\left(cad-{cad}_{\text{tar}}\right)|>{\delta }_{\text{cad}}\hfill \\ c_3+d_{\text{steady}}\hfill & \mathrm{e}\mathrm{l}\mathrm{s}\mathrm{e}.\hfill \end{array}\right.(15)$

The first part of the cost function again enforces stable walking without falling. The second and third parts of the cost function ensure that the model walks at a desired cadence cad_tar and speed v_tar, where the speed is determined by the cadence and step length sl_tar. All gait parameters are measured over the last 10 s of 20 s simulated walking. The last part of the cost function generates stable locomotion as in Eq. (14). The constants δ_v = 0.02 m/s and δ_cad = 2 steps/min are tolerance margins for the desired speeds and cadences. The constants c₀ = 10³, c₁ = 500, c₂ = 250, and c₃ = 125 are normalization factors that ensure that the different parts of the cost function are realized in the required order. We found that without such tolerance margins, the model behavior was constrained too tightly and the optimization often failed to converge. All parameters except the control parameters are adopted from the result of the optimization procedure in Section 3.1.

We found a total number of 98 control parameter sets with cadence and step length values that cover the entire range usually adopted in human walking. Figure 2 shows the resulting gait patterns of these 98 models in the gait parameter space spanned by step length and cadence (blue dots). The colored region illustrates the normal range of human walking. We defined this range as all points in the cadence–step length domain for which cadence, step length, and speed all fall within the interval containing 95% of human experimental data around the mean for each parameter (Inman et al., 1981). This results in a region that is delimited by three sets of two lines, one along the horizontal axis for cadence, one along the vertical axis for step length, and a third along the diagonal for speed. Going forward, we will refer to this set as the normal human walking region. Target gait parameters during the optimization procedure were manually selected to gradually cover the whole normal human walking region. The actual gaits resulting from each optimization were partially stochastic due to the randomness in the optimization algorithm and the tolerance in the cost function. Initial conditions for each optimization were either hand-tuned or determined by linear combinations of the five nearest neighbors in the step length–cadence domain. Going forward, we will refer to this collection of 98 control parameter sets as the basis point library.

FIGURE 2

FIGURE 2. Basis Point Library. Gait patterns in the cadence–step length domain resulting from the 98 sets of control parameters found by optimization. The shaded area is the normal human walking region, delimited by the intervals for cadence, step length, and speed, containing 95% of human data for each parameter. Dashed purple lines indicate the interval limits.

3.3 Generalization from learned gaits

The optimization procedure described in Section 3.2 provides a general approach to generate walking at any step length and cadence combination usually adopted by humans. However, each gait is still “learned” by an optimization procedure. Humans, on the other hand, can spontaneously walk at any desired combination of step length and cadence (Nilsson and Alf, 1987), within a certain range, and adopt new gaits very quickly, on a time scale that is usually considered too fast for learning or adaptation (Edwards, 2010), more in the range of parameter estimation (Buckingham et al., 2009). Here our goal is to test whether the learned gaits in the basis point library can be generalized to walk at gaits that were not previously learned.

To generalize the learned patterns to new gaits, we interpolate between the existing points in the basis point library based on proximity in the cadence–step length domain. Given a target gait as a combination of step length and cadence, we select five nearest neighbors from the basis point library, based on Euclidian distance in the cadence–step length domain. We then define the new gait as a linear combination of these five nearest neighbors in the control parameter space, resulting in a new control parameter vector that is supposed to result in a gait with the desired step length and cadence combination. This approach is elaborated in Section 6.2 in the Appendix. We apply this approach to a grid of target gaits evenly spaced in the cadence–step length domain, with the results as shown below in Section 4.2.

3.4 Transitions between gaits in real time

Our last goal is to test whether the model can transition between different gaits in real time without the loss of stability. This is not trivial because even gaits that are close in the two-dimensional cadence–step length domain might be distant in the eight-dimensional space of control parameters, representing differences in the gait pattern not captured by step length and cadence.

To this end, we generate transitions between gaits by switching to a new set of control parameters at fixed points in time, regardless of the state of gait cycle. In a first simulation study, we perform targeted switches along different paths in the cadence–step length domain, representing either continuous speeding up or slowing down, by increasing or decreasing both cadence and step length simultaneously, or modulations of step length and cadence in opposite directions, in combinations that leave speed largely invariant. In a second simulation study, we switch randomly to a new gait within a certain distance in the cadence–step length domain at fixed points in time. The results of these simulation experiments are shown below in Section 4.3.

4 Results

4.1 Optimization of high-level control parameters

The goal of this section is to find control parameter sets that generate stable walking gaits that cover the normal human walking region, that is, the range of cadence–step length combinations usually adopted by humans. We performed optimizations following the procedure described in Section 3.2. This process resulted in a total of 98 sets of control parameters covering most of the normal human walking region. These 98 gaits are shown as blue dots in Figure 2. For each of these 98 sets of high-level control parameters, the model walked without falling for 100 s.

Since the optimization process was partially stochastic and included tolerance margins for the target cadence and step length in the cost function, we did not use a hard exit criterion for this process. We decided to stop the process when gaits covering most of the normal human walking region were successfully found, and further improvements were slow. We found gaits with cadences ranging from 85 to 140 steps/min and step lengths ranging from 0.53 to 0.97 m, with several solutions lying outside the normal human walking region. As Figure 2 shows, gaits with short, slow steps (cadence ≤110 s/min and step length ≤0.75 m) were rarely found. Optimizations in this area tended to not converge during the stabilization phase of optimization. Furthermore, we found one isolated gait with a step length of 0.3 m (not shown in Figure 2). However, we did not investigate gaits beyond human usual walking behavior because convergence appeared to be significantly harder. The 98 solutions are used as a basis point library for subsequent simulation studies to test generalization and transition.

4.2 Generalization from learned gaits

Finding the basis points required the optimization of high-level control parameters for each desired gait. Here, we test whether it is possible to generalize between these learned basis points and walk at gaits that were not previously learned, by interpolating the control parameters from the basis point library, and without having to re-optimize a new set of control parameters.

In a first simulation experiment, we tested walking at an evenly spaced grid of 19 target gaits loosely spanning the region of normal human walking and extending beyond the 95% confidence limits by 10% in the cadence and step length directions. For each target gait, we simulated the model using control parameters from the linear recombination of the five nearest neighbor basis points in the cadence–step length space. For details on the interpolation, see Appendix 6.2. For each set of control parameters, we considered the resulting gait as “stable” if the model walked for at least 20 s without falling.

Figure 3 shows the results of this simulation study in the cadence–step length space. Out of the 19 generated parameter sets, 12 successfully walked for at least 20 s. The target gaits for these successful sets are shown as blue dots in Figure 3. For the remaining seven parameter sets, the models did not walk successfully for 20 s but fell after an average of 3.05 ± 2.12 s. The target gaits for these unsuccessful sets are shown as red crosses in Figure 3. For the successful gaits, we measured the average cadence and step length over the last 10 s of walking to determine how close the actual gait was to the target gait. The actual gaits are shown as green dots in Figure 3, connected to the target gaits by dashed lines.

FIGURE 3

FIGURE 3. Generalization results for 19 gaits. Blue points are target gaits placed on a lattice in the cadence–step length domain. Green points are the actual gaits that resulted from planning to walk at the target gaits by generalizing the previously learned gaits. Pairs of planned and the resulting actual gaits are connected by dotted lines. Red crosses represent target gaits for which the generalization did not result in stable walking.

Overall, the solutions of this simulation study can be divided into three groups. The first group of solutions generates walking gaits very close to the target gait. Nine of the 19 solutions are in this first group covering large portions of the search space. Three points correspond to the second group, all in a region corresponding to gaits with slow and short steps. The third group of solutions did not generate stable walking. The seven gaits in this group were all located outside the range usually adopted by humans.

In a second simulation experiment, we investigated the same range of gaits, but increased the grid resolution to get a more fine-grained sampling of the region in the cadence–step length space where the generalization performs well. This resulted in a total number of 340 target gaits, out of which 179 walked at least 20 s without falling. Figure 4 shows the successful (blue dots) and the unsuccessful (red crosses) target gaits. The mean (±STD) distance between the target and realized gaits is 0.0297 ± 0.0520 m in step length and 0.9618 ± 1.757 steps/min in cadence. Throughout large portions of the investigated range, generalization was successful, with the model walking for at least 20 s without falling. Very fast ($>$1.65 m/s) and slow ($<$1.2 m/s) walking gaits, however, frequently led to unstable gaits. Target gaits in the 10 percent margin around the normal human walking region also often led to falls.

FIGURE 4

FIGURE 4. Generalization results for 340 gaits. Blue points are target gaits placed on a lattice in the cadence–step length domain. Red crosses represent target gaits for which the generalization did not result in stable walking.

4.3 Transition between states

In the previous section, we showed that the model can generalize the previously optimized set of basis points and independently select gaits with cadence and step length within the normal human walking region. In this section, we investigate the model’s ability to transition between different gaits during locomotion in real time. In a first simulation experiment, we select six different paths through the cadence–step length domain. Each path is a sequence of seven gaits, visualized as circles connected by dotted lines in cadence–step length space as shown in Figure 5. For three of these paths, movement speed is systematically changed while the ratio of cadence and step length remains similar. For the other three paths, the movement speed remains similar while the ratio of cadence and step length is systematically changed. For each of these paths through the cadence–step length space, we initialize the model with the parameter vector of the first basis point, shown as green dots in Figure 5. After 20 s of walking, and irrespective of the current state of the gait cycle, we switch the control parameters to the subsequent set to test whether the model can transition to the new gait in real time. We repeat this switch to the next parameter set every 20 s, simulating a total of 140 s per path. The initial velocity parameter is only used once at the simulation onset.

FIGURE 5

FIGURE 5. Gait transitions along six paths in the cadence–step length domain. The dotted lines indicate the target paths and solid lines indicate the actual paths. Starting points along the respective walks are highlighted as squares.

All gait transitions in this simulation study were completed successfully, without the model falling. The solid dots and connecting lines in Figure 5 show the step length and cadence realized by the model over the last 10 s of each 20 s walking period. The average (maximal) error between the desired and realized gait parameters across all paths and transitions was 0.53 (4.60) cm for step length and 0.25 (1.08) steps/min for cadence. Gait transitions were less accurate in regions of cadence–step length space where generalization tended to fail (see Section 4.2 above). This is particularly pronounced in Walk 4, shown in dark red in Figure 5, which leads through an area of the step length–cadence space that consists of short, slow steps.

Time courses of step length and cadence for two selected paths are shown in Figure 6. Figures 6A,B illustrate the outcome variables step length and cadence for Walk 3, corresponding to the light blue path in Figure 5. The red lines indicate the target step lengths and cadences, and blue dots show measured step lengths and cadences for each stride. Both step length and cadence consistently relax toward their target values for all transitions. In some cases, the gait pattern briefly oscillates around the target values during the initial relaxation, before stabilizing close to the target values.

FIGURE 6

FIGURE 6. Time courses of step length and cadence for Walks 3 and 4 (see Figure 5). Time courses of step length and cadence are depicted in panels (A,B) for Walk 3 and in panels (C,D) for Walk 4. Target step lengths and cadences are indicated by red lines. Blue dots show the actual cadences and step length for each stride.

Figures 6C,D show the same time courses for Walk 4, corresponding to the dark red path in Figure 5. Cadence and step length behave generally similar to Walk 3. In the stretch between 40 and 60 s, the step length oscillates strongly before stabilizing at a value $\approx 5$ cm below its target value. Between 60 and 80 s, the step length initially changes in the wrong direction, then slowly relaxes toward the target value, but without reaching steady state within the 20 s.

We performed a second simulation experiment to further test how well the model can transition between different gaits. In this experiment, we tested whether the ability to transition between different gaits is preserved when generalizing the learned basis points to new gaits, as described above. We simulated a total number of 180 walks with a maximal duration of 120 s. We initialized the model at one of three selected basis points with similar speed and different relationships of step length and cadence (see Figure 7), then updated the control parameters to a new set every 20 s to generate a gait transition. The new control parameter set was determined by first randomly drawing new values for target step length and cadence from an interval around the current values, using five different interval sizes chosen as narrow (1 step/min or cm), medium narrow (3 step/min or cm), medium (5 step/min or cm), medium large (7.5 step/min or cm), or large (10 step/min or cm). For these target gait parameters, we then determined a control parameter set by linear recombination of neighboring basis points, as described in Section 3. After a transition, the model either successfully continues to walk at a new gait or fails to recover from the transition and falls. Figure 7 shows how the success rate of transitions develops with increasing range of the gait parameter update.

FIGURE 7

FIGURE 7. Success of transition between randomized gaits. Data are the success rates of transitions to a random new gait relative to all attempted transitions. The horizontal axis is the radius of the interval from which the new gait is randomly drawn. The different colors show different starting points for individual walks.

5 Discussion

We presented a neuromuscular model of human locomotion that is capable of voluntarily controlling both the swing and stance leg according to a kinematic motor plan. The model can adopt different gait patterns, with step lengths and cadences covering the region of normal human walking, defined as the gait parameter intervals covering 95% of experimental data for human walking. The model combines biomechanics, muscle physiology, spinal reflex loops, and supraspinal neural processes in a physiologically plausible way. The supraspinal layer generates a movement plan from a set of high-level control parameters that define a goal state for the swing leg, propulsion for the stance leg, a reference stance hip angle for balancing the trunk, and the desired movement time. The kinematic movement plan is transformed into descending motor commands that interface with the spinal cord, using a combination of neural networks and explicit internal models. The spinal layer integrates the descending commands with reflex arcs that activate muscles based on the feedback from muscle spindles and Golgi tendon organs. The model can walk in a wide range of step lengths and cadences by adopting different high-level control parameters for swing leg movement and timing, propulsion and trunk balance that were learned using evolutionary optimization. We found that the model can generalize between the previously learned gaits to some degree and walk with new gaits within the same region. The model can furthermore transition between different gaits in real time by switching to a new set of control parameters without losing stability.

5.1 Volitional and habitual control of walking

Stable locomotion requires coordinated interaction between the different components of sensorimotor control involved in walking (Prochazka and Gorassini, 1998; Bauby and Kuo, 2000). This coordination is particularly important when leaving a steady state to change the gait pattern. To increase step length, for instance, it is necessary to extend the hip of the swing leg further. Increased step length, however, will lead to other changes in movement kinematics and dynamics, such as increased vertical movement of the CoM, which increases the loss of kinetic energy at each step (Reimann et al., 2019). Hence, it is also necessary to adjust the propulsion with the stance leg and maintain a constant speed at the new gait pattern. Changing the gait voluntarily, thus, requires the ability to actively manipulate both the kinematics of the swing leg and the kinetics of the entire model.

In Ramadan et al. (2022), we presented an integrative neuromuscular model that combines the ability to control the swing leg as a planned, voluntary movement with habitual, reflexive control of the stance leg. The model presented here extends the voluntary control approach to the stance leg, adding the ability to control propulsion by pushing off the ground with the stance leg, as well as controlling trunk balance by modulating the stance leg hip flexion angle. This enabled us to regulate the gait pattern of the model by selecting a set of high-level control parameters. We showed that with this high-level control approach, the model was capable of walking with a wide range of gaits, quantified by the gait parameters of cadence and step length. The model could walk at gaits covering the entire region of normal human walking by switching to a different set of high-level control parameters, while leaving the parameters governing the behavior of the low-level reflexes unchanged. A relatively simple generalization approach based on linear recombination of previously learned gaits allowed the model to walk at gaits that it had not previously learned. Furthermore, the model could transition between different gaits in real time without the loss of stability.

Existing neuromechanical models of walking are mostly based on spinal control mechanisms such as reflexes and central pattern generators (Taga, 1995; Geyer and Herr, 2010). These control schemes generate a stable walking movement pattern that can be modified to some degree by re-tuning the neural feedback loops that map sensory information to muscle activation. Van der Noot et al. (2018) showed that it is possible to generalize between different sets of learned behaviors by interpolating between different sets of spinal control parameters. Di Russo et al. (2021) identified polynominal functions of a subset of reflex parameters that can be used to modify the step length and cadence. The control approaches used by these models are largely habitual and coordination between the individual components involved in locomotion emerges from parameters tuning rather than from an associated motor plan. In the model presented here, the gait pattern is not determined by a specific tuning of the reflexes but by a set of high-level movement parameters. Reflexes, however, still play an important role for the model as they substantially simplify the control problem. Our model uses positive force feedback from the Golgi tendon organs (Prochazka and Gorassini, 1998) to keep the stance leg stretched and compliant, countering gravitational forces during the stance phase. Furthermore, a reflex arc built of combined feedback from muscle spindles and from Golgi tendon organs solves the problem of balancing the trunk upright.

5.2 Structure of solution space

We used high-level control parameters to generate walking movement with different gaits. The eight-dimensional control parameter space is spanned by swing leg hip and knee flexion target joint angles for early and late swing, trunk lean reference, swing leg ankle target, propulsion, and step time. An additional parameter, initial velocity, was used in the optimization but did not affect the steady-state gait pattern. The two-dimensional task space of gait parameters is spanned by step length and cadence. We optimized the high-level control parameters to find solutions at different points in task space, for a total number of 98 different models covering the entire normal human walking region. We also showed that it is possible to interpolate between points in control space based on proximity in task space to generate gaits at new points in task space.

The generalization of solutions, however, is limited, and models using recombined parameter sets occasionally lose balance and fall. This particularly applies to the extrapolation of solutions beyond the region in the cadence–step length space covered by the learned parameters set. But even within this region, interpolation is limited in some areas. One potential reason for this limitation is redundancy of the eight-dimensional control parameter space over the two-dimensional task space. Two parameter sets can lie very close to each other in the task space, yet be substantially different from each other in the control parameter space, resulting in two gait patterns that have similar step lengths and cadences but are different in other aspects, such as swing leg kinematics. Recombining these two solutions to a new gait pattern can then be problematic, since the average control parameter set interpolated between them might not lead to a stable gait. To analyze the dimensionality of the solutions in the control parameter space, we ran a principle component analysis (Abdi and Williams, 2010) on the 98 sets in the basis point library. Figure 8 shows the explanatory power for each principal component. The first two principle components explain less than 65% of the variability in the data, indicating that step length and cadence are not the only gait properties that change within the basis point library. Only the first four principle components explain $>90\%$ of the variability of the data. The finding clarifies that the task variables cadence and step length, chosen here to quantify a gait pattern, are not sufficient to completely describe the complexity of the gait patterns in the basis point library. However, extending the choice of output variables is difficult and beyond the scope of this work. One possibility for additional constraints on the resulting gait patterns would be to minimize the metabolic energy expended by the models. It is plausible that variability in the data could be reduced by expanding the cost function by further constraints such as the minimization of metabolic energy (Falisse et al., 2019; Ong et al., 2019).

FIGURE 8

FIGURE 8. Principle component analysis of control parameters. The blue bars show the percentage of variance explained by the respective principle components and the red line shows the cumulative explained variance.

5.3 Limitations and scope

We presented a neuromuscular model of human locomotion that can independently change step length and cadence within a range usually adopted by humans. However, interpolation between the obtained solutions is limited, and generalization tended to fail for very fast and slow walking speeds. One reason for this limitation could be a lack of flexibility in lateral balance control. Humans use two main mechanisms for lateral balance control, modulation of the foot placement location at each new step, and ankle roll during single stance (Reimann et al., 2019). Human experiments show that the relative importance of the ankle roll mechanism increases at lower speeds and step frequencies (Fettrow et al., 2019). Our model has only one degree of freedom at the ankle joint, for ankle flexion, so the ankle roll mechanism is not available for balance control in the frontal plane. We speculate that extending the biomechanical model to add ankle roll movement and control can improve stability at slower gaits and increase the ability of the model to walk and generalize between gaits, particularly in the region with short and slow steps, where humans rely more strongly on the ankle roll mechanism for balance control (Fettrow et al., 2019).

A second potential reason for the limited generalization of our model is the assumption that balance control parameters for foot placement remain constant across all gaits. The foot placement controller adapts the foot placement location in proportion to the current position and velocity of the trunk center of mass relative to the stance foot (Ramadan et al., 2022). The trunk kinematics in the frontal plane change substantially with movement speed and stepping cadence, so it is plausible that the gain parameters of the foot placement controller might change in humans depending on movement speed. When analyzing unperturbed human walking at different speeds, Wang and Srinivasan (2014) did not find significant differences in the slopes of linear models relating foot placement location to the kinematic CoM state at mid-stance, which are closely related to balance control gains. Stimpson et al. (2018) found that the explanatory power of the kinematic CoM state at mid-stance to predict foot placement changes is reduced for very slow walking speeds, but did not report how the slopes of these relationships change with speed. Based on this experimental evidence, we chose to keep the gain parameters fixed in the present model. Adding the balance control gains to the high-level control parameters could potentially improve balance control at high or low speeds, which might lead to better generalization in these regions.

In the current model, we use a trunk balance mechanism that solely relies on spinal feedback, based on Sharbafi and Seyfarth (2015). Human experiments, however, show that supraspinal feedback can play an important role in balancing the trunk (Shumway-Cook and Horak, 1986).

The particular set of high-level control parameters used here was developed from a combination of historical and functional reasons. The parameters for kinematic goal state of the swing leg at the end of the early and late stance phases were adopted from a robotic model (Yin et al., 2007) and used in a slightly different form in Ramadan et al. (2022). To control cadence and step length, we added control mechanisms and parameters for propulsion and step time. Our goal here was to show that it is possible to generate stable walking with different gait patterns as a planned, voluntary movement using a small set of high-level kinematic control parameters. The rationale for this choice is that high-level representations of movement generally use kinematic variables in task space, rather than low-level variables on the execution level (Schwartz et al., 1988; Churchland et al., 2012). Whether the specific choice of kinematic variables, based on Yin et al. (2007), is reasonable or whether humans use different variables, such as leg length and leg angle in space, is a question for future research.

Data availability statement

The source code for the model implementation can be found at https://github.com/Rachidramadan1990/FlexibleGait.

Author contributions

Conceptualization: RR, HR; data curation: RR, HR; formal analysis: RR, HR; funding acquisition: HR; investigation: RR, HR, FM; methodology: RR, HR; project administration: RR, HR; resources: HR; software: RR, FM; supervision: HR, RR; validation: RR, HR; visualization: RR, HR; writing—original draft preparation: RR, HR, FM.

Funding

HR was funded by the National Science Foundation (NSF CRCNS 1822568). RR was funded by the German Federal Ministry of Education and Research (01GQ1803).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, editors, and reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Abdi, H., and Williams, L. J. (2010). Principal component analysis. WIREs. Comp. Stat. 2 (4), 433–459. doi:10.1002/wics.101

CrossRef Full Text | Google Scholar

Ackermann, M., and van den Bogert, A. J. (2012). Predictive simulation of gait at low gravity reveals skipping as the preferred locomotion strategy. J. Biomechanics 45 (7), 1293–1298. doi:10.1016/j.jbiomech.2012.01.029

PubMed Abstract | CrossRef Full Text | Google Scholar

Ahmad Sharbafi, Maziar, and Seyfarth, A. (2017). How locomotion sub-functions can control walking at different speeds? J. Biomechanics 53, 163–170. doi:10.1016/j.jbiomech.2017.01.018

PubMed Abstract | CrossRef Full Text | Google Scholar

Allen, Jessica L., and Ting, Lena H. (2016). “Why is neuromechanical modeling of balance and locomotion so hard?” in Neuromechanical modeling of posture and locomotion (Berlin: Springer), 197–223.

CrossRef Full Text | Google Scholar

Andani, M. E., and Bahrami, Fariba (2012). Comap: A new computational interpretation of human movement planning level based on coordinated minimum angle jerk policies and six universal movement elements. Hum. Mov. Sci. 31 (5), 1037–1055. doi:10.1016/j.humov.2012.01.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Aoi, Shinya, Ohashi, Tomohiro, Bamba, Ryoko, Fujiki, Soichiro, Tamura, Daiki, Funato, Tetsuro, et al. (2019). Neuromusculoskeletal model that walks and runs across a speed range with a few motor control parameter changes based on the muscle synergy hypothesis. Sci. Rep. 9 (1), 369. doi:10.1038/s41598-018-37460-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Ayers, Joseph, Carpenter, Gail A., Scott, Currie, and Kinch, James (1983). Which behavior does the lamprey central motor program mediate? Science 221 (4617), 1312–1314. doi:10.1126/science.6137060

PubMed Abstract | CrossRef Full Text | Google Scholar

Bauby, Catherine E., and Kuo, Arthur D. (2000). Active control of lateral balance in human walking. J. Biomechanics 33 (11), 1433–1440. doi:10.1016/s0021-9290(00)00101-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Buckingham, Gavin, Cant, Jonathan S., and Goodale, Melvyn A. (2009). Living in A Material world: How visual cues to material properties affect the way that we lift objects and perceive their weight. J. Neurophysiology 102 (6), 3111–3118. doi:10.1152/jn.00515.2009

PubMed Abstract | CrossRef Full Text | Google Scholar

Churchland, Mark M., Cunningham, John P., Kaufman, Matthew T., Foster, Justin D., Paul, Nuyujukian, Ryu, Stephen I., et al. (2012). Neural population dynamics during reaching. Nature 487 (7405), 51–56. doi:10.1038/nature11129

PubMed Abstract | CrossRef Full Text | Google Scholar

Clark, David J. (2015). Automaticity of walking: Functional significance, mechanisms, measurement and rehabilitation strategies. Front. Hum. Neurosci. 9, 246. doi:10.3389/fnhum.2015.00246

PubMed Abstract | CrossRef Full Text | Google Scholar

De Groote, Friedl, and Falisse, Antoine (2021). Perspective on musculoskeletal modelling and predictive simulations of human movement to assess the neuromechanics of gait. Proc. R. Soc. B 288, 20202432. doi:10.1098/rspb.2020.2432

PubMed Abstract | CrossRef Full Text | Google Scholar

Di Russo, Andrea, Stanev, Dimitar, Armand, Stéphane, and Ijspeert, Auke (2021). Sensory modulation of gait characteristics in human locomotion: A neuromusculoskeletal modeling study. PLoS Comput. Biol. 17 (5), e1008594. doi:10.1371/journal.pcbi.1008594

PubMed Abstract | CrossRef Full Text | Google Scholar

Edwards, W. H. (2010). Motor learning and control: From theory to practice. Boston: Cengage Learning.

Google Scholar

Engelbrecht, Sascha E. (2001). Minimum principles in motor control. J. Math. Psychol. 45 (3), 497–542. doi:10.1006/jmps.2000.1295

PubMed Abstract | CrossRef Full Text | Google Scholar

Falisse, Antoine, Gil, Serrancolí, Dembia, Christopher L., Gillis, Joris, Jonkers, Ilse, and De Groote, Friedl (2019). Rapid predictive simulations with complex musculoskeletal models suggest that diverse healthy and pathological human gaits can emerge from similar control strategies. J. R. Soc. Interface 16 (157), 20190402. doi:10.1098/rsif.2019.0402

PubMed Abstract | CrossRef Full Text | Google Scholar

Fettrow, Tyler, Reimann, Hendrik, Grenet, David, Crenshaw, Jeremy, Higginson, Jill, and Jeka, John (2019). Walking cadence affects the recruitment of the medial-lateral balance mechanisms. Front. Sports Act. Living 1, 40. doi:10.3389/fspor.2019.00040

PubMed Abstract | CrossRef Full Text | Google Scholar

Flash, Tamar, and Hogan, Neville (1985). The coordination of arm movements: An experimentally confirmed mathematical model. J. Neurosci. 5 (7), 1688–1703. doi:10.1523/jneurosci.05-07-01688.1985

PubMed Abstract | CrossRef Full Text | Google Scholar

Géron, Aurélien (2019). Hands-on machine learning with scikit-learn, keras, and TensorFlow: Concepts, tools, and techniques to build intelligent systems. Sebastopol: O’Reilly Media, Inc.

Google Scholar

Geyer, Hartmut, and Herr, Hugh (2010). A muscle-reflex model that encodes principles of legged mechanics produces human walking dynamics and muscle activities. IEEE Trans. Neural Syst. Rehabil. Eng. 18 (3), 263–273. doi:10.1109/tnsre.2010.2047592

PubMed Abstract | CrossRef Full Text | Google Scholar

Geyer, Hartmut, Seyfarth, A., and Blickhan, Reinhard (2003). Positive force feedback in bouncing gaits? Proc. R. Soc. Lond. B 270, 2173–2183. doi:10.1098/rspb.2003.2454

PubMed Abstract | CrossRef Full Text | Google Scholar

Günther, Michael, and Ruder, Hanns (2003). Synthesis of two-dimensional human walking: A test of the lambda-model. Biol. Cybern. 89 (2), 89–106. doi:10.1007/s00422-003-0414-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Hansen, Nikolaus (2006). “The cma evolution strategy: A comparing review,” in Towards a new evolutionary computation: Advances in the Estimation of distribution algorithms, studies in fuzziness and soft computing. Editors Jose A. Lozano, Pedro Larrañaga, Iñaki Inza, and Endika Bengoetxea (Berlin, Heidelberg: Springer), 75–102.

Google Scholar

Hof, A. L. (2001). The force resulting from the action of mono- and biarticular muscles in a limb. J. Biomechanics 34 (8), 1085–1089. doi:10.1016/s0021-9290(01)00056-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Inman, Verne Thompson, Ralston, H. J., and Todd, Frank (1981). Human walking. Pennsylvania: Williams & Wilkins.

Google Scholar

Kiehn, Ole (2016). Decoding the organization of spinal circuits that control locomotion. Nat. Rev. Neurosci. 17 (4), 224–238. doi:10.1038/nrn.2016.9

PubMed Abstract | CrossRef Full Text | Google Scholar

Kistemaker, Dinant A., Knoek Van Soest, Arthur J., Wong, Jeremy D., Kurtzer, Isaac, and Gribble, Paul L. (2013). Control of position and movement is simplified by combined muscle spindle and Golgi tendon organ feedback. J. Neurophysiology 109 (4), 1126–1139. doi:10.1152/jn.00751.2012

PubMed Abstract | CrossRef Full Text | Google Scholar

Lim, Yoong Ping, Lin, Yi-Chung, and Pandy, M. G. (2017). Effects of step length and step frequency on lower-limb muscle function in human gait. J. Biomechanics 57, 1–7. doi:10.1016/j.jbiomech.2017.03.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Mantziaris, Charalampos, Bockemühl, Till, and Büschges, Ansgar (2020). Central pattern generating networks in insect locomotion. Dev. Neurobiol. 80 (1-2), 16–30. doi:10.1002/dneu.22738

PubMed Abstract | CrossRef Full Text | Google Scholar

Nilsson, Johnny, and Thorstensson, Alf (1987). Adaptability in frequency and amplitude of leg movements during human locomotion at different speeds. Acta Physiol. Scand. 129 (1), 107–114. doi:10.1111/j.1748-1716.1987.tb08045.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Ong, Carmichael F., Geijtenbeek, Thomas, Hicks, Jennifer L., and Delp, Scott L. (2019). Predicting gait adaptations due to ankle plantarflexor muscle weakness and contracture using physics-based musculoskeletal simulations. PLoS Comput. Biol. 15 (10), e1006993. doi:10.1371/journal.pcbi.1006993

PubMed Abstract | CrossRef Full Text | Google Scholar

Perret, C., Cabelguen, J.-M., and Orsal, D. (1988). “Analysis of the pattern of activity in “knee flexor” motoneurons during locomotion in the cat,” in Stance and motion (Berlin: Springer), 133–141.

CrossRef Full Text | Google Scholar

Prochazka, Arthur, and Gorassini, Monica (1998). Ensemble firing of muscle afferents recorded during normal locomotion in cats. J. Physiology 507 (1), 293–304. doi:10.1111/j.1469-7793.1998.293bu.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Prochazka, Arthur (2013). “Proprioceptor models,” in Dieter jaeger and ranu jung (New York, NY: Springer), 1–20. Encyclopedia of Computational Neuroscience.

Google Scholar

Ramadan, Rachid, Geyer, Hartmut, Jeka, John J., Schöner, G., and Reimann, Hendrik (2022). A neuromuscular model of human locomotion combines spinal reflex circuits with voluntary movements. Sci. Rep. 12, 8189. doi:10.1038/s41598-022-11102-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Reimann, Hendrik, Ramadan, Rachid, Fettrow, Tyler, Hafer, Jocelyn F., Geyer, Hartmut, and Jeka, J. (2020). Interactions between different age-related factors affecting balance control in walking. Front. Sports Act. Living 2, 94. doi:10.3389/fspor.2020.00094

PubMed Abstract | CrossRef Full Text | Google Scholar

Reimann, Hendrik, Fettrow, Tyler, Grenet, David, Thompson, Elizabeth D., and Jeka, John J. (2019). Phase-dependency of medial-lateral balance responses to sensory perturbations during walking. Front. Sports Act. Living 1, 25. doi:10.3389/fspor.2019.00025

PubMed Abstract | CrossRef Full Text | Google Scholar

Sarmadi, Alireza, Schumacher, Christian, Seyfarth, A, and Maziar, Ahmad Sharbafi (2019). Concerted control of stance and balance locomotor subfunctions—Leg force as a conductor. IEEE Trans. Med. Robot. Bionics 1 (1), 49–57. doi:10.1109/tmrb.2019.2895891

CrossRef Full Text | Google Scholar

Schumacher, Christian, Berry, Andrew, Lemus, Daniel, Rode, Christian, Seyfarth, André, and Vallery, Heike (2019). Biarticular muscles are most responsive to upper-body pitch perturbations in human standing. Sci. Rep. 9 (1), 14492. doi:10.1038/s41598-019-50995-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Schwartz, Andrew B., Kettner, Ronald E., and Georgopoulos, Apostolos P. (1988). Primate motor cortex and free arm movements to visual targets in three- dimensional space. I. Relations between single cell discharge and direction of movement. J. Neurosci. 8 (8), 2913–2927. doi:10.1523/jneurosci.08-08-02913.1988

PubMed Abstract | CrossRef Full Text | Google Scholar

Sharbafi, Maziar A., and Seyfarth, A. (2015). “Fmch: A new model for human-like postural control in walking,” in 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 5742–5747.

CrossRef Full Text | Google Scholar

Shumway-Cook, A., and Horak, Fay Bahling (1986). Assessing the influence of sensory interaction on balance: Suggestion from the field. Phys. Ther. 66 (10), 1548–1550. doi:10.1093/ptj/66.10.1548

PubMed Abstract | CrossRef Full Text | Google Scholar

Song, Seungmoon, and Geyer, Hartmut (2015). A neural circuitry that emphasizes spinal feedback generates diverse behaviours of human locomotion: A spinal feedback circuitry generating human locomotion behaviors. J. Physiol. 593 (16), 3493–3511. doi:10.1113/jp270228

PubMed Abstract | CrossRef Full Text | Google Scholar

Steele, Katherine M., van der Krogt, M., Schwartz, Michael H., and Delp, Scott L. (2012). How much muscle strength is required to walk in a crouch gait? J. Biomechanics 45 (15), 2564–2569. doi:10.1016/j.jbiomech.2012.07.028

PubMed Abstract | CrossRef Full Text | Google Scholar

Stimpson, Katy H., Heitkamp, Lauren N., Horne, Joscelyn S., and Dean, Jesse C. (2018). Effects of walking speed on the step-by-step control of step width. J. Biomechanics 68, 78–83. doi:10.1016/j.jbiomech.2017.12.026

PubMed Abstract | CrossRef Full Text | Google Scholar

Taga, Gentaro (1998). A model of the neuro-musculo-skeletal system for anticipatory adjustment of human locomotion during obstacle avoidance. Biol. Cybern. 78 (1), 9–17. doi:10.1007/s004220050408

PubMed Abstract | CrossRef Full Text | Google Scholar

Taga, Gentaro (1995). A model of the neuro-musculo-skeletal system for human locomotion. Biol. Cybern. 73 (2), 97–111. doi:10.1007/bf00204048

PubMed Abstract | CrossRef Full Text | Google Scholar

Van der Noot, Nicolas, Ijspeert, Auke, and Ronsse, Renaud (2018). Bio-inspired controller achieving forward speed modulation with a 3D bipedal walker. Int. J. Robotics Res. 37 (1), 168–196. doi:10.1177/0278364917743320

CrossRef Full Text | Google Scholar

Wang, Yang, and Srinivasan, Manoj (2014). Stepping in the direction of the fall: The next foot placement can be predicted from current upper body state in steady-state walking. Biol. Lett. 10 (9), 20140405. doi:10.1098/rsbl.2014.0405

PubMed Abstract | CrossRef Full Text | Google Scholar

Yin, KangKang, Kevin, Loken, and Michiel van de Panne, (2007). Simbicon: Simple biped locomotion control. ACM Trans. Graph. 26 (3), 105. doi:10.1145/1276377.1276509

CrossRef Full Text | Google Scholar

Zhang, Yajie, Smeets, Jeroen B. J., Brenner, Eli, Verschueren, S., and Duysens, J. (2020). Fast responses to stepping-target displacements when walking. J. Physiol. 598 (10), 1987–2000. doi:10.1113/jp278986

PubMed Abstract | CrossRef Full Text | Google Scholar

6 Appendix

6.1 A1: Neural network for muscle inversion

We use a feed forward neural network as part of the inverse model to map desired torques $\tilde{\tau }$ to desired muscle forces $\tilde{F}$. The mapping is dependent on the current joint configuration θ, since moment arms of the biomechanical model depend on the body configuration. The network is composed of an input layer i, one hidden layer r, and an output layer f. The neurons in i receive the desired torque $\tilde{\tau }$ and joint configuration θ as input. The layer r consists of 1,000 hidden neurons and is connected to i via weight matrix W. The layer f has one neuron for each muscle and is connected to r via weight Z.

In order to generate input training data, we sample 500,000 random torque and joint angle combinations from physiologically determined limits and 1,000,000 torque and joint angle combinations obtained from the walking model presented in Ramadan et al. (2022). From the input training data, we determined muscle forces that realize the respective input torques depending on the input joint configuration, using a constrained optimization algorithm to minimize the sum of all forces under the constraint that forces remain positive (fmincon in MATLAB).

For the training, we initialize the weights W and Z randomly and used tanh as the activation function in the r layer. To ensure resulting muscle forces are not negative, we also used the activation function ReLu in the f layer. For minimizing the cost mean squared error, we used the Adam optimizer (Géron, 2019).

We implement the network only for nine muscles in the sagittal plane, as the moment arms in the sagittal plane are modeled independent from the frontal plane. The mapping in the frontal plane is computed analytically since it does not involve redundancies.

6.2 A2: Interpolation between control parameter sets

Simulating the walking model defines a mapping Φ from the eight-dimensional space of control parameters, $\mathcal{M}$, to the two-dimensional space $\mathcal{N}$ of gait parameters cadence and step length. To walk at any desired combination of cadence c and step length s, we need to invert this mapping to find control parameters m = Φ⁻¹(n) that will generate a walking pattern with the desired gait parameters n = (c, s). To find this m, we interpolate between existing basis points m_b in the space of control parameters developed in Section 3.2, for which we know the function values Φ(m_b) = n_b relating gait parameters n_b to the control parameter set m_b used to generate this gait pattern.

Let $n_{\text{des}}=\left(\begin{array}{c}\hfill c_{\text{des}},s_{\text{des}}\hfill \end{array}\right)$ be any combination of desired cadence and step length. We use the gait

$n_1=\underset{n\in N_b}{\mathrm{arg\; min}}\Vert n-n_{\text{des}}\Vert (\mathrm{B1})$

from the basis point library that is closest to the desired gait as the starting point, and the four next-closest gaits n_i, i = 2 … 5 as supporting points for the interpolation, where $N_b\subset \mathcal{N}$ is the set of gaits in the basis point library. Let m_i = Φ⁻¹ (n_i) be the control parameter sets associated with these five gaits. We define direction vectors

$n_{\text{i}}^{\prime }=n_i-n_1(\mathrm{B2})$

$n_{\text{des}}^{\prime }=n_{\text{des}}-n_1(\mathrm{B3})$

$m_{\text{i}}^{\prime }=m_i-m_1,(\mathrm{B4})$

with 2 ≤ i ≤ 5, and combine them to matrices $\mathbf{M}={(m_2^{\prime },m_3^{\prime },m_4^{\prime },m_5^{\prime })}^T$ and $\mathbf{N}={(n_2^{\prime },n_3^{\prime },n_4^{\prime },n_5^{\prime })}^T$ and define the direction vector in control parameter space as

$w_{\text{des}}^{\prime }=\mathbf{W}{\mathbf{M}}^{-1}{n}_{\text{des}}^{\prime }.(\mathrm{B5})$

Finally, we calculate the control parameter vector as

$m_{\text{des}}=m_1+m_{\text{des}}^{\prime }.(\mathrm{B6})$