- 1Institute of Biomedical Engineering, Boğaziçi University, Istanbul, Türkiye
- 2Institute for Modelling and Simulation of Biomechanical Systems, University of Stuttgart, Stuttgart, Germany
- 3Huawei Turkey R&D Center, Istanbul, Türkiye
Advancements in instrumentation support improved powered ankle prostheses hardware development. However, control algorithms have limitations regarding number and type of sensors utilized and achieving autonomous adaptation, which is key to a natural ambulation. Surface electromyogram (sEMG) sensors are promising. With a minimized number of sEMG inputs an economic control algorithm can be developed, whereas limiting the use of lower leg muscles will provide a practical algorithm for both ankle disarticulation and transtibial amputation. To determine appropriate sensor combinations, a systematic assessment of the predictive success of variations of multiple sEMG inputs in estimating ankle position and moment has to conducted. More importantly, tackling the use of nonnormalized sEMG data in such algorithm development to overcome processing complexities in real-time is essential, but lacking. We used healthy population level walking data to (1) develop sagittal ankle position and moment predicting algorithms using nonnormalized sEMG, and (2) rank all muscle combinations based on success to determine economic and practical algorithms. Eight lower extremity muscles were studied as sEMG inputs to a long-short-term memory (LSTM) neural network architecture: tibialis anterior (TA), soleus (SO), medial gastrocnemius (MG), peroneus longus (PL), rectus femoris (RF), vastus medialis (VM), biceps femoris (BF) and gluteus maximus (GMax). Five features extracted from nonnormalized sEMG amplitudes were used: integrated EMG (IEMG), mean absolute value (MAV), Willison amplitude (WAMP), root mean square (RMS) and waveform length (WL). Muscle and feature combination variations were ranked using Pearson’s correlation coefficient (r > 0.90 indicates successful correlations), the root-mean-square error and one-dimensional statistical parametric mapping between the original data and LSTM response. The results showed that IEMG+WL yields the best feature combination performance. The best performing variation was MG + RF + VM (rposition = 0.9099 and rmoment = 0.9707) whereas, PL (rposition = 0.9001, rmoment = 0.9703) and GMax+VM (rposition = 0.9010, rmoment = 0.9718) were distinguished as the economic and practical variations, respectively. The study established for the first time the use of nonnormalized sEMG in control algorithm development for level walking.
1. Introduction
Lower limb amputations include transtibial amputation, i.e., a surgical procedure to fully remove the lower limb below the knee, and ankle disarticulation, i.e., surgical removal of the foot at the ankle joint. The incidence rates of these interventions are high, reaching, e.g., over 11,500 cases alone in the UK each year (Isaacs-Itua and Sedki, 2018) and from 2003 to 2013, about half of those were transtibial amputations (Ahmad et al., 2016). Ascribed to a simple, low economic cost and robust design, energy storing and returning prostheses with elastic and damping characteristics (Brackx et al., 2013) dominate the commercial lower limb prostheses market (Gehlhar et al., 2020). However, because of lack of mechanical power generation and autonomous adaptation they can only provide lower than normal gait speeds with high metabolic energy costs that leads to early fatigue (Vucina et al., 2005; Brackx et al., 2013; Xu et al., 2021). Central to these limitations is the absence of a sensory feedback to characterize user dependent needs (Tucker et al., 2015).
Current studies on powered prostheses, which devices overcome the indicated shortcomings and assist users’ gait at walking speeds close to normal (Nasr et al., 2021) do utilize several sensors including inertial measurement units (Culver et al., 2018; Shultz and Goldfarb, 2018), pressure sensors (Attal et al., 2018; Liu et al., 2019), force sensors (Sup et al., 2008; Quintero et al., 2018; Lenzi et al., 2019), and mechanical sensors, e.g., load cells, position, velocity and current sensors (Huang et al., 2011; Spanias et al., 2018; Khademi and Simon, 2021). However, such sensors can be energy demanding and feature latency in signal outputs for matching human motion resulting in compromised autonomous adaptation (Zhang et al., 2019). On the other hand, surface electromyograms (sEMG) can be used in the assistive robotic system controllers (Huang et al., 2016; Ferris and Schlink, 2017; Spanias et al., 2018; Li et al., 2019; Hosseini et al., 2020; Yin et al., 2020; Hunt et al., 2021; Kyeong et al., 2022; Zhang et al., 2023) gait classification (Huang et al., 2009; Varol et al., 2010; Miller et al., 2013; Liu et al., 2017; Khademi and Simon, 2021) and predicting lower limb joint kinetics and kinematics (Sepulveda et al., 1993; Zhang et al., 2012; Chen et al., 2018; Jephil et al., 2020; Zabre-Gonzalez et al., 2021), which serve well the purpose of the autonomous adaptation. Asif et al. (2021) and Fleming et al. (2021) reported that biomimetic models, artificial neural networks, and support vector machines are widely used for such applications. However, there are several issues to take into account in using sEMG for such purpose one related to preprocessing or avoiding of that, and two related to the number of sensors to be utilized.
First, sEMG signals have complicated and random patterns, making real-time analysis difficult (Reaz et al., 2006; Chowdhury et al., 2013). Halaki and Ginn (2012) reported that amplitude normalization can make interpretation of raw sEMG more accurate for detecting muscle activation. However, the implementation of a real-time normalization imposes additional difficulties in determining the reference value and leads to an elevated computational cost as the necessary periodic calibration of such reference can take several minutes (Tanaka et al., 2022) and disrupt real-time use. Fluctuations across sEMG channels caused by muscle fatigue and variability in skin impedance can also obscure the reference value over longer periods of time (Scheme and Englehart, 2011; Khushaba et al., 2014; Triwiyanto et al., 2018; Ameri et al., 2020) and make sEMG normalization unsuitable for real-time processing (Tanaka et al., 2022). One possibility could be finding static reference values for each targeted muscle through some pre-testing such as a few minutes of stable walking at a certain condition and normalizing the sEMG signals to those references in the real-time implementation. In the amputee prosthesis users, this could be utilized before adding any control to the device provided that it can work passively. This can serve limiting the computation cost of normalization, which needs to be tested in new studies. However, prolonged walking has been shown to cause muscle fatigue (Chambers et al., 2019) and consequently decreased sEMG amplitudes (Schlink et al., 2021). More importantly, studies have shown variations in sEMG amplitudes with changes in walking speed (den Otter et al., 2004), which may compromise accuracy of static reference values. Currently, the use of nonnormalized sEMG supported by feature extraction implementation is the plausible method to be utilized (Fleming et al., 2021). With the aim of achieving autonomous adaptation to improve the human-machine coordination performance in exoskeletons, Ma et al. (2020) used time-domain features of sEMG of eight leg muscles collected from healthy subjects in order to estimate the knee joint angle by using a long short-term memory structure (LSTM). A similar approach was used by Foroutannia et al. (2022) to develop an LSTM architecture that utilizes time-domain features of leg muscles in the prediction of hip joint kinematics.
Second, sEMG signals can be contaminated by noise caused by cross talk and skin conductivity change, or artifacts originating at the skin-electrode interface, signal amplifiers, and external sources (De Luca et al., 2010). Therefore, using a large number of sEMG sensors may cause processing complexity (Hussain et al., 2020), complications in sensor setup design, and impose difficulty in daily use (Fleming et al., 2021). In different studies, the number of sEMG amplitudes utilized for predicting joint kinematics and kinetics varies from eleven (e.g., Huang et al., 2009, 2011; Du et al., 2013; Hargrove et al., 2013; Young et al., 2014; Spanias et al., 2016; Liu et al., 2017) to four (e.g., Hoover et al., 2012; Spanias et al., 2018). Zaffir et al. (2021) developed and compared different types of neural network models for estimating dorsiflexion for robotic ankle-foot orthoses with similar concerns of minimizing the number of muscle inputs and eliminating mechanical sensors. They utilized sEMG data of four leg muscles of healthy participants. Extracted time-domain features were used as inputs, and the best performance was shown by the LSTM neural network.
Third, the original cause of the amputation, the choice of surgical technique and factors such as residual limb length, shape, and subsequent muscle atrophy can affect muscle activity and, consequently, sEMG signals (Isakov et al., 1996; Smith, 2004; Tintle et al., 2010; Ranz et al., 2017). In transtibial amputation, the residual limb muscle availability gets compromised if the amputation is performed closer to the knee (Huang and Ferris, 2012), whereas most muscle mass remains intact in ankle disarticulation (Isaacs-Itua and Sedki, 2018).
Therefore, minimizing the number of sEMG muscle sources utilized in a joint kinematics and kinetics prediction algorithm will make the controller of a powered ankle prosthesis economic, and limiting the use of lower leg muscles will make it practical for both ankle disarticulation and transtibial amputation. Recently, we developed neural network-based algorithms to predict ankle position and moment using all combinations of sEMG amplitudes of several lower leg muscles (Keleş and Yucesoy, 2020). This approach methodologically paved the way for achieving a user specific algorithm development procedure as the most suitable sEMG variations that either minimizes the total number of sensors or those of lover leg muscles could be identified. However, the use of normalized sEMG exclusively impedes a real-time applicability. In addition, the number of available muscles limited to five restricts a versatile applicability, given the inter-individual differences of potential users. In the present study, the methodology is extended for competency by (i) utilizing nonnormalized sEMG signals coupled with feature extraction and (ii) making muscle input selection comprehensive via the usage of eight lower leg muscles.
The aims were to (1) implement our extended methodology for developing algorithms that predict sagittal ankle kinematics and kinetics during level walking using nonnormalized lower limb sEMG signals, and (2) rank the extensive range of muscle combination variations according to their success to facilitate a user specific selection of sensor inputs for economic and practical control algorithms that can be used in powered ankle prostheses.
2. Methods
2.1. Data utilized and summary of data collection procedures
Gait data are required for developing, training, and testing neural networks. For that purpose, sEMG amplitudes, sagittal ankle angle, and moment of level walking trials were acquired from the open access data by Lencioni et al. (2019). The data includes (I) trials of level walking at different speeds (0.3–2.3 m/s) collected from fifty able-bodied subjects (25 males, 25 females, age range, 6–72 years, body mass: 18.2–110 kg, body height: 116.6–187.5 cm) and (II) in addition to joint kinematics and kinetics data also nonnormalized sEMG data of tibialis anterior (TA), soleus (SO), medial gastrocnemius (MG), peroneus longus (PL), rectus femoris (RF), vastus medialis (VM), biceps femoris (BF) and gluteus maximus (GMax), which were recorded simultaneously with motion capturing during the level walking trials. Ankle joint kinematics and kinetics data were recorded using a 9-camera motion capture system (SMART system, BTS, Garbagnate Milanese, Italy), total-body LAMB marker set and two force plates (Kistler, Winterthur, Switzerland). The sEMG amplitudes were recorded using an 8-channel wireless EMG recording system (ZeroWirePlus, Cometa, Bareggio, Italy) unilaterally on the dominant side with electrode locations in agreement with SENIAM (Hermens et al., 2000), using pre-amplified self-adhesive Ag-AgCl electrodes (Medtronic Kendall, USA, 24 mm electrode diameter, 10 mm active part diameter, bipolar configuration, 20 mm inter-electrode distance). sEMG amplitudes were band-pass filtered (10 Hz – 400 Hz) before sampling to reduce the aliasing effect, but not normalized. Synchronous data acquisition was managed by the proprietary software of the motion capture system.
In each experiment, the subjects were equipped with the LAMB total-body marker set, which includes retro-reflective markers (12 mm diameter) on lower limbs. As required by the LAMB protocol, additional markers were placed on the medial part of the lower limbs for the preliminary static calibration trial and were removed during the dynamic trials. Subjects were required to wear tight clothes or swimsuits and markers were attached on the skin above bony landmarks with double-sided adhesive tape.
For the level-ground walking task, subjects were initially asked to walk five trials at their natural speed. Then, they were asked to perform the following ten trials while progressively increasing (first 5 trials) or decreasing (latter 5 trials) their speed. No precise indications about gait speed or cadence in order not to induce gait alterations were given.
2.2. Neural network development
All neural networks to be developed feature the following: (1) The long-short term memory (LSTM) neural network architecture was used. LSTM is a deep learning method that is a form of the recurrent neural network model that can make predictions for time series in real-time (Kaushik et al., 2020; Yi et al., 2021). This architecture has been shown to be highly efficient in sEMG-based human motion pattern recognition for identifying the user’s movement (Song et al., 2020). (2) sEMG features, calculated over sEMG amplitudes of leg muscles, were used as LSTM inputs to predict ankle position and moment (see Figure 1 for a block diagram). A similar input integration into LSTM has been shown to be highly efficient in estimating joint angles (Ma et al., 2020; Zaffir et al., 2021; Foroutannia et al., 2022). Presently, two separate LSTMs were developed and trained for predicting ankle joint position and moment. The reason for not using a single LSTM with multiple outputs was (i) to reduce complexity so that LSTMs can be trained and tuned efficiently (Kendall et al., 2018; Crawshaw et al., 2020), (ii) to avoid possible overfitting, which can cause inaccurate predictions for new data (Rebuffi et al., 2017; Zhang and Yang, 2021) and (iii) to achieve a better computational performance (Rebuffi et al., 2017).
2.2.1. sEMG feature selection
sEMG features have been used widely as inputs for sEMG-based robotic controllers (Hudgins et al., 1993; Chan and Green, 2007; Angkoon et al., 2009). Presently, five time-domain sEMG features that can be used efficiently in real-time applications (Phinyomark et al., 2011) were tested and evaluated, which are: integrated EMG (IEMG), mean absolute value (MAV), Willison amplitude (WAMP), root mean square (RMS) and waveform length (WL). Their mathematical definitions are presented in Table 1. Since the window size to calculate sEMG features should be less than 300 ms for real-time implementation (Phinyomark et al., 2011), the window size was selected as 150 ms (Huang et al., 2011). To test the success of sEMG features, all possible combinations (in total 31) were studied: 5 single features (e.g., IEMG), 10 two feature combinations (e.g., IEMG+MAV), 10 three feature combinations (e.g., IEMG+MAV + WAMP), 5 four feature combinations (e.g., IEMG+MAV + WAMP+RMS) and 1 five feature combination (i.e., IEMG+MAV + WAMP+RMS + WL). In order to determine the best performing sEMG feature variation, sEMG amplitudes of the muscle combination MG + BF + GMax were used based on the success of this muscle combination in ankle position and moment predictions (Keleş and Yucesoy, 2020).
2.2.2. Muscle selection
sEMG amplitudes of the eight lower extremity muscles included in the dataset were used: the TA, SO, MG, and PL in the lower leg and the RF, BF, VM, and GMax in the upper leg. To test the success of the use of sEMG amplitudes in neural networks inputs for the prediction of ankle position and moment, all possible combinations (in total 255) were studied: 8 single muscles (e.g., TA), 28 two muscle combinations (e.g., TA + SO), 56 three muscle combinations (e.g., TA + SO+MG), 70 four muscle combinations (e.g., TA + SO+MG + PL), 56 five muscle combinations (e.g., TA + SO+MG + PL + RF), 28 six muscle combinations (e.g., TA + SO+MG + PL + RF + BF), 8 seven muscle combinations (e.g., TA + SO+MG + PL + RF + BF + VM), and 1 eight muscle combination (i.e., TA + SO+MG + PL + RF + BF + VM + GMax).
The best performing variation including the minimum total number of muscle inputs is referred to as the economic variation. The variation performing better than the economic variation and including the minimum number of lower leg muscle inputs is referred to as the flexible variation. The best performing variation including the minimum total number of only upper leg muscles is classified as practical variation.
2.3. Neural network training
The open access data (Lencioni et al., 2019) contains 844 level walking trials at various walking speeds. Randomly selected 675 datasets (80% of the total) were classified and used as training datasets, and the rest were classified and used as validation datasets. Both training and validation datasets contain the same range of walking speed data (0.3–2.3 m/s) which enabled the LSTMs to be trained and tested for a wide range of walking speeds. The feature extraction function outputs for the TA muscle are exemplified in Figure 2 (see Supplementary Document 1 for the remainder of the muscles). For each muscle within each dataset, the features were extracted and were used in the training and validation of LSTMs developed, which was done separately for ankle joint position and moment predictions. LSTMs were trained within MATLAB, using the Adam optimizer because of its computational efficiency and its success in non-stationary data (Kingma and Ba, 2015). Trainings were stopped after 200 epochs (e.g., Zhang and Ma, 2023). The selection of the number of hidden units for joint position and moment predictor LSTMs was based on previous studies aiming at utilization of sEMG in joint kinetics and kinematics predictions (Kim et al., 2020; Bao et al., 2021), such that a range of 250 units was chosen for both. 200 hidden units for joint position and 50 hidden units for joint moment predictor LSTMs were determined (Figure 1) as the outcome of a systematical variation of the number of hidden units within this range to yield the highest correlations for ankle position and moment changes in the sagittal plane.
Figure 2. Mean and standard deviation of sEMG signals and its extracted features for the TA muscle. (A) the sEMG signals and the corresponding (B) IEMG, (C) MAV, (D) RMS, (E) WL and (F) WAMP features.
2.4. Evaluation of the developed neural networks
Normal Q-Q plots for the residuals between original data and LSTM response were used to illustrate if the residuals follow a normal distribution. Two-way ANOVA for repeated measures (factors: %GC and muscle combination) was performed separately for joint position and moment predictions based on correlation coefficients. If significant main effects were found, Bonferroni post hoc tests were performed to further locate significant within-factor differences. One-way ANOVA was further used for temporal success assessment based on one-dimensional statistical parametric mapping (SPM) across the gait cycle (GC) using F-statistics (Pataky, 2010). Differences between the original data and estimated outputs were considered significant at p < 0.05.
To evaluate the outcome of the developed sEMG feature and muscle combinations (i.e., the best performing one out of all possible 31 sEMG feature combinations x all 255 possible muscle combinations) for joint position and moment predictions, the following were used: (1) Pearson’s correlation coefficient (r) between the original data and LSTM response was calculated. r = 0.90 was selected as a strong correlation threshold. A miscorrelation score was defined as the sum of deviation of correlation values from one. (2) Root-mean-square errors (RMSE) between the original data and LSTM response were calculated. An RMSE score was defined as the mean of position and moment RMSE values normalized to their respective peak value among all sEMG feature or muscle combinations. (3) SPM analysis was conducted to localize significantly different %GC points between the original data and LSTM response, total number of which are expressed as a percentage of the entire GC. An SPM Score was defined as the mean of those percentages for ankle position and moment predictions. (4) An overall error score was defined as the product of miscorrelation, RMSE, and SPM scores.
3. Results
3.1. sEMG feature selection
Table 2 shows the correlation coefficients, miscorrelation scores, RMSE values, RMSE scores, SPM results, and SPM scores of all sEMG feature combinations calculated over sEMG amplitudes of MG + BF + GMax and provides their ranking based on their overall error score.
Table 2. The comparison of the performance of sEMG feature variations utilizing the muscle combination MG + BF + GMax.
The five feature combination (IEMG+MAV + RMS + WAMP+WL) shows a strong correlation (rposition = 0.9006 and rmoment = 0.9663) and ranks 19th amongst all.
Four feature combinations: Only IEMG+MAV + RMS + WL shows a strong correlation (rposition = 0.9095 and rmoment = 0.9747) and ranks 6th amongst all.
Three feature combinations: 2 of 10 variations show strong correlations (rposition > 0.90 and rmoment > 0.97). IEMG+RMS + WL shows the strongest correlation (rposition = 0.9031 and rmoment = 0.9710) and ranks 4th amongst all variations.
Two feature combinations: 2 of 10 variations show strong correlations (rposition > 0.90 and rmoment > 0.97). IEMG+WL shows the strongest correlation (rposition = 0.9006 and rmoment = 0.9742) and ranks 1st amongst all variations.
Single features: Only WL shows a strong correlation (rposition = 0.9006 and rmoment = 0.9814) and ranks 11th amongst all.
Results show that IEMG+WL yields the best position and moment prediction performance. SPM analysis of the IEMG+WL (Figure 3) showed that significant localized differences between the actual and estimated outputs are limited to 10 points (1–3%, 8–9%, 25–28%, and 54% GC) for ankle position, whereas no significantly different points were localized for ankle moment.
Figure 3. Temporal success assessment for IEMG+WL. The normal Q-Q plots for the residuals between original data and LSTM response for ankle (A) position and (B) moment. Mean and standard deviation of estimated NN response vs. original data for ankle (C) position and (D) moment, as a function of % stride. Statistical parametric mapping using F-statistics (SPM{F}) for ankle (E) position and (F) moment as a function of % stride shows values below the threshold (i.e., for p < 0.05, F*position = 9.26, F*moment = 9.36). IEMG abbreviates integrated EMG and WL abbreviates waveform length.
3.2. Muscle selection
ANOVA (factors: %GC and muscle combination) showed significant main effects on joint position predictions, but no interaction. ANOVA (factors: %GC and muscle combination) showed significant main effects on joint moment predictions and a significant interaction. Post hoc testing showed significant effects of muscle combination based on comparisons of correlation coefficients for (i) 239 out of 465 combinations for joint position predictions, and (ii) 227 out of 465 combinations for joint moment predictions.
Table 3 shows the correlation coefficients, miscorrelation scores, RMSE values, RMSE scores, SPM results, and SPM scores of best performing 30 muscle combinations determined utilizing the sEMG feature IEMG+WL followed by selected exclusively upper leg muscle combinations, and provides their ranking based on their overall error score (see Supplementary Document 2 for all variations).
Table 3. The best performing 30 muscle combinations determined by utilizing the sEMG feature IEMG+WL.
The eight muscle combination (BF + MG + GMax+PL + RF + SO+TA + VM) shows strong correlation (rposition = 0.9205 and rmoment = 0.9695) and ranks 7th amongst all variations.
Seven muscle combinations: All variations show strong correlations (rposition > 0.90 and rmoment > 0.97). BF + MG + GMax+RF + SO+TA + VM shows the strongest correlation and ranks 4th amongst all variations (rposition = 0.9246 and rmoment = 0.9803).
Six muscle combinations: 26 of 28 variations show strong correlations (rposition > 0.90 and rmoment > 0.95). BF + MG + GMax+PL + RF + SO shows the strongest correlation and ranks 3rd amongst all variations (rposition = 0.9134 and rmoment = 0.9714).
Five muscle combinations: All variations show strong correlations (rposition > 0.90 and rmoment > 0.95). MG + GMax+PL + RF + TA shows the strongest correlation and ranks 17th amongst all variations (rposition = 0.9099 and rmoment = 0.9779).
Four muscle combinations: 66 of 70 variations show strong correlations (rposition > 0.90 and rmoment > 0.96). MG + GMax+PL + VM shows the strongest correlation and ranks 5th amongst all variations (rposition = 0.9101 and rmoment = 0.9725).
Three muscle combinations: 45 of 56 variations show strong correlations (rposition > 0.90 and rmoment > 0.96). MG + RF + VM shows the strongest correlation and ranks 1st amongst all variations (rposition = 0.9099 and rmoment = 0.9707).
Two muscle combinations: 21 of 28 variations show strong correlations (rposition > 0.90 and rmoment > 0.96). RF + TA shows the strongest correlation and ranks 11th amongst all variations (rposition = 0.9174 and rmoment = 0.9776).
Single muscles: 2 of 8 variations show strong correlations (rposition > 0.90 and rmoment > 0.97). PL shows the strongest correlation and ranks 69th amongst all variations (rposition = 0.9001 and rmoment = 0.9703).
Economic variation is PL which ranks 69th amongst all variations. SPM analysis (Figure 4) showed that significant localized differences between the actual and estimated outputs are limited to 1–3%, 12–13%, 30–31%, and 97–100% GC for the position and, 45–53% and 91–98% GC for the moment prediction.
Figure 4. Temporal success assessment for PL. The normal Q-Q plots for the residuals between original data and LSTM response for ankle (A) position and (B) moment. Mean and standard deviation of estimated NN response vs. original data for ankle (C) position and (D) moment, as a function of % stride. Statistical parametric mapping using F-statistics (SPM{F}) for ankle (E) position and (F) moment as a function of % stride shows values below the threshold (i.e., for p < 0.05, F*position = 9.25, F*moment = 9.38). PL abbreviates peroneus longus.
Flexible variation is MG + RF + VM, which ranks the 1st amongst all variations. Table 3. further shows muscle variations with significantly different correlation coefficients compared to MG + RF + VM. SPM analysis (Figure 5) showed that significant localized differences between the actual and estimated outputs occur between 1–3% GC for the position and, 83–86% GC for the moment prediction.
Figure 5. Temporal success assessment for MG + RF + VM. The normal Q-Q plots for the residuals between original data and LSTM response for ankle (A) position and (B) moment. Mean and standard deviation of estimated NN response vs. original data for ankle (C) position and (D) moment, as a function of % stride. Statistical parametric mapping using F-statistics (SPM{F}) for ankle (E) position and (F) moment as a function of % stride shows values below the threshold (i.e., for p < 0.05, F*position = 9.32, F*moment = 9.11). MG abbreviates medial gastrocnemius, RF abbreviates rectus femoris, and VM abbreviates vastus medialis.
Practical variation is GMax+VM, which ranks the 106th amongst all variations. Table 3. further shows muscle variations with significantly different correlation coefficients compared to GMax+VM. SPM analysis (Figure 6) showed that significant localized differences between the actual and estimated outputs occur between 1–2%, and 89–95% GC for the position and 63–68% 74–91%, and 99–100% GC for the moment prediction.
Figure 6. Temporal success assessment for GMax+VM. The normal Q-Q plots for the residuals between original data and LSTM response for ankle (A) position and (B) moment. Mean and standard deviation of estimated NN response vs. original data for ankle (C) position and (D) moment, as a function of % stride. Statistical parametric mapping using F-statistics (SPM{F}) for ankle (E) position and (F) moment as a function of % stride shows values below the threshold (i.e., for p < 0.05, F*position = 9.29, F*moment = 9.44). GMax abbreviates gluteus maximus and VM abbreviates vastus medialis.
4. Discussion
Recent technological improvements in hardware development have advanced the field of lower limb prostheses (Azocar et al., 2018; Elery et al., 2018; Lenzi et al., 2018). However, an advanced autonomous adaptation is still required to achieve seamless and natural ambulation (Fleming et al., 2021). Therefore, studies focused on sEMG-based algorithms for prosthesis controllers have started to accumulate (Dawley et al., 2013; Wang et al., 2013; Huang et al., 2016; Spanias et al., 2018). Recently, we developed neural networks showing that ankle position and moment changes during level walking can be predicted by using normalized sEMG amplitudes of leg muscles (Keleş and Yucesoy, 2020). However, the use of normalized sEMG amplitudes was a limitation which is addressed in the present study.
Souza et al. (2014) indicated that neuromuscular control schemes merging timing-based approaches could advance prosthesis control. Since the action patterns of walking occur in a time series (Song et al., 2020) and sEMG amplitudes show variability in time and are cyclic (Souza et al., 2014), in order to make accurate predictions of joint movement using data from past is essential. LSTM architecture contains a set of memory blocks (Ma et al., 2020) that can maintain its state over time (Greff et al., 2017), enabling the structure to remember the previous inputs (Song et al., 2020). Consequently, LSTM is widely used owing to its capability of generating accurate outputs for time series: the action patterns of walking occur in a time series (Song et al., 2020), and LSTM generates outputs based on the inputs from the past (Greff et al., 2017), making it suitable for predicting joint kinematics and kinetics (Ma et al., 2020). Also, the sEMG feature extraction is used widely, where features are extracted within a sliding window and used as inputs to predict joint kinetics or kinematics (Bi et al., 2019; Gupta et al., 2020; Zhang et al., 2021; Chen et al., 2022; Rabe and Fey, 2022; Truong et al., 2023). Previous studies have shown that using sEMG features as input (e.g., Spanias et al., 2015, 2018), the LSTM structure (e.g., Ren et al., 2022), and their combined implementations (e.g., Song et al., 2020) provide a successful prediction of intended motion. Although prediction of ankle position and moment was not sought after, these studies demonstrated that accurate predictions using time-domain sEMG features and LSTM is plausible. Therefore, an LSTM structure that predicts ankle position and moment during level walking using features extracted from nonnormalized sEMG amplitudes was implemented in the present study. Results showed that the developed LSTMs can predict ankle position and moment during level walking with up to 0.9292 and 0.9813 correlations, respectively. Note that, Huihui et al. (2018) reported an RMSE of 4.80° in ankle position prediction, whereas Hahn and O’Keefe (2008) reported an RMSE of 0.10 Nm/kg in ankle moment prediction. However, those authors utilized 5 and 7 muscles, respectively. Ma et al. (2020) estimated the knee joint angle using 8 muscles with an RMSE of 4.60°, whereas Farmer et al. (2014) estimated the ankle joint angle using 3 lower leg muscles and reported an RMSE of 5.40°. By utilizing only three muscles, the present LSTM structure yielded a compatible success with 4.5923° and 0.1072 Nm/kg RMSE.
Although studies aiming to develop sEMG-based lower limb prosthesis controllers have increased in the last decade, there is still no commercialized product that relies on neuromuscular input (Fleming et al., 2021). Song et al. (2020) predicted the movement patterns using normalized sEMG amplitudes of RF, VM, vastus lateralis, and semitendinosus muscles to extract features within a sliding window size of 1,024 ms. However, a window size less than 300 ms was suggested to support implementation of feature extraction in real-time prosthetic control (Phinyomark et al., 2011). Also, sEMG normalization is a non-specific method in terms of the required data to determine the reference value (Halaki and Ginn, 2012), e.g., the large window size can be a major limitation for the required computation and real-time applicability. Spanias et al. (2015) and Spanias et al. (2018) utilized the same number of neuromuscular input and achieved the use of nonnormalized sEMG amplitudes collected from RF, tensor fasciae latae, semitendinosus, and adductor magnus to extract features within a window size lower than 300 ms. Nonetheless, they have implemented the additional mechanical sensor, which can be energy consuming and could be a trade-off in autonomous adaptation due to its latency in generating the output for the human locomotion (Zhang et al., 2019). Utilizing solely nonnormalized sEMG amplitude was achieved but the number of neuromuscular inputs was increased. In order to recognize the locomotion modes, Huang et al. (2011) utilized sEMG amplitudes of RF, VM, sartorius, vastus lateralis, gracilis, biceps femoris long head, semitendinosus, biceps femoris short head, and adductor magnus muscles. Hargrove et al. (2013) developed a pattern recognition algorithm using sEMG amplitudes of BF, RF, VM, vastus lateralis, sartorius, gracilis, adductor magnus, and tensor fasciae latae. Chen et al. (2018) developed a deep belief network-based algorithm to estimate lower limb joint angles including the ankle and utilized sEMG amplitudes collected from BF, VM, RF, MG, TA, SO, MG, semitendinosus, vastus lateralis, and sartorius. Huihui et al. (2018) implemented the use of feature extraction and nonnormalized sEMG amplitudes collected from the external gastrocnemius, gastrocnemius, SO, TA, and tibialis longus in order to estimate ankle joint angle. Wang et al. (2019) used sEMG amplitudes of RF, VM, BF, semitendinosus, and gastrocnemius to estimate knee joint angles. These studies show that the prosthesis controller can be advanced with several sEMG inputs. However, such implementation may bear difficulties regarding the processing complexity (Hussain et al., 2020). Au et al. (2008) limited the use of neuromuscular input size and utilized TA and gastrocnemius muscles in order to develop a powered ankle prosthesis controller, whereas Au et al. (2005) and Farmer et al. (2014) used the same muscles and included the SO to predict the ankle position. Similarly, Zabre-Gonzalez et al. (2021) developed a neural network-based algorithm to predict ankle position and moment using TA and MG muscles. Jephil et al. (2020) utilized the same muscles with the addition of the lateral gastrocnemius to estimate the ankle joint torque and angle. These studies show that the neuromuscular input size can be reduced by using the lower leg muscles. However, Huang and Ferris (2012) showed that sEMG amplitudes patterns of the TA, gastrocnemius medial head, and gastrocnemius lateral head in transtibial amputees, recorded within the prosthetic socket during walking, have high inter-subject variability. Therefore, limiting lower leg muscle inputs or using solely upper leg muscles can increase the compatibility of the prediction algorithms for different levels of amputations. In the present study, with the goal of real-time applicability, we developed LSTMs to predict ankle position and ankle moment using time-domain features extracted within 150 ms from non-normalized sEMG amplitudes. The proposed LSTM structure eliminates the need for external mechanical sensor inputs which cause high processing complexity. Also, the presented structure is capable of generating strongly correlated predictions by using sEMG amplitudes of GMax and VM muscles, addressing the compatibility issue. Furthermore, none of the previous studies provided a systematic analysis for the selection of muscles to be used as input. We recently developed a methodology to systematically study all possible variations of sEMG inputs from lower leg muscles (Keleş and Yucesoy, 2020). Yet, the implementation of this methodology to level walking was limited to the usage of five muscle inputs (TA, MG, RF, BF, and GMax) and more importantly to normalized sEMG data exclusively. Presently, we extended the systematic analysis by including also SO, PL, and VM muscles. Note that, indeed the PL muscle was shown to be relevant as even a single input, whereas the analyses indicated that VM is important in limiting the usage of lower leg muscles. However, the bigger contribution of the present study was the extension of the methodology to make it suitable for a real-time implementation, which now involves utilization of features extracted from nonnormalized sEMG signals as inputs to the LSTM architecture. Integration of the SPM analysis outcome into the performance metric is also a new approach allowing for a better tracking of specific %GC’s showing significant differences between the original data and LSTM response. These present achievements provide a good versatility by increasing the choices of upper and lower leg muscles and improve the user specific applicability of sEMG-based ankle position and moment prediction procedures in powered ankle prosthesis control algorithms.
The present study shows that the LSTM is suitable for ankle position and moment prediction by using the time-domain features extracted from the nonnormalized sEMG amplitudes as input. Our comprehensive time-domain sEMG feature selection analysis showed that the best performing variation is IEMG+WL, which is frequently implemented in sEMG-based prosthesis controller studies (Fleming et al., 2021) and is feasible to be used in real-time applications (Phinyomark et al., 2011). Presently, we used the muscle combination MG + BF + GMax for sEMG feature selection. Previous studies with a similar aim of utilizing sEMG in predicting joint kinematics and kinetics conducted either a feature (e.g., Phinyomark et al., 2011) or a muscle combination selection (e.g., Wang et al., 2021), or report only the utilized feature per muscle (Phinyomark et al., 2011; Li et al., 2021). Note that, taking into account the challenging nature of determining which specific parameter or combination of parameters is responsible for an improved neural network output (Goodfellow et al., 2016), a separate evaluation of the effects of multiple parameters has been suggested (Molnar, 2020). The complexity of the neural networks can consequently be reduced for an efficient training (Kendall et al., 2018). Our present approach, which involves already a large space of possible muscle combinations (255) is in concert with that. To the best of our knowledge studies combining the possible sets of both sEMG feature selection and muscle selection are very rare and one which did cover both involves only 11 muscle combinations (Khan et al., 2021). In the present study, the muscle combination (i.e., MG + BF + GMax) used in sEMG feature selection process was chosen based on our previous study (Keleş and Yucesoy, 2020), which was shown to provide successful predictions, while minimizing the use of lower leg muscles. The feature combination IEMG+WL selected as a result, if put to the test with also other muscle combinations that stood out in Keleş and Yucesoy (2020) yields consistently the preferable results (see Supplementary Document 3 for details): for BF + MG + TA (the three muscle combination providing improved predictions without aiming at minimizing the use of lower leg muscles), for MG + TA (the successful combination with minimum muscle inputs) and for BF + MG + Gmax+RF + TA (the best-performing muscle combination), IEMG+WL shows the best performance among other feature combinations yielding strongly correlated predictions (minimally, rposition = 0.9161 and rmoment = 0.9739). Our comprehensive muscle selection analysis showed that 225 of 255 variations provide strong correlations for ankle position and moment prediction, ascribed to the flexibility of using all combinations of a wide range of muscles. A specific assessment seeking for the economic variation showed that PL alone provides the best performance in minimizing the number of muscle inputs. This is a lower leg muscle with limited accessibility but, strongly correlated predictions were shown to be achievable by utilizing solely upper leg muscles. The practical variation achieving the best performance by minimizing utilized upper leg muscles, is comprised of GMax+VM which are highly relevant for sEMG-based prosthesis controller development (Fleming et al., 2021). On the other hand, the flexible variation, i.e., MG + RF + VM showed that the best performance can be achieved by using a combination of upper and lower leg muscles.
The limitations of the present study need to be addressed. The position and moment predicting LSTMs were developed, trained and tested using data from healthy subjects only. Note that, the open database by Lencioni et al. (2019) involving nonnormalized sEMG amplitudes of various leg muscles is a rarity, but the lack of an open database that consists of amputee data limits further testing. However, Huang and Ferris (2012) reported that transtibial amputees are able to voluntarily activate their leg muscles with several of them producing activation profiles similar to healthy controls during voluntary dorsiflexion and plantar flexion. They also demonstrated that amputees are capable of generating consistent sEMG amplitude patterns from stride to stride, which supports the approach utilized in this study for developing position and moment predicting LSTMs. On the other hand, in case sEMG amplitude patterns show differences in amputees compared to those of healthy subjects, the proposed infrastructure can be adapted by re-training it with data collected from amputees. This will also facilitate a patient specific algorithm development. Also, the present study focuses on the prediction of ankle position and moment during level walking only, yet the LSTMs can be advanced for different locomotion tasks such as stair ascending and descending. Training their LSTM and using time-domain features extracted from nonnormalized sEMG amplitudes of eight muscles involving RF, BF, TA, and SO, Lu et al. (2022) achieved lower limb joint angles prediction for additional locomotion tasks. Therefore, the developed methodology is suitable for such prediction algorithms. Overall, the accuracy of ankle movement predictions should be studied after re-training the developed LSTM structure with a large database including nonnormalized sEMG amplitudes collected from lower extremity amputees and different locomotion tasks can be adapted. Note that unlike surface electrodes, electrodes within the socket of the prosthetic device may compromise user’s comfort, and long-term use may negatively affect sensor lifetime. The use of, e.g., sEMG knit band sensor (Lee et al., 2018) i.e., a silver-plated conductive yarn for electrodes knitted with a moisture-wicking technical yarn can make implementation much easier and improve comfort. Additionally, high density sEMG can help avoiding problems related to bipolar electrode placement (Ison et al., 2016) and improve data quality (Rojas-Martínez et al., 2012).
In bodily motion, humans have the sensation of position and moment changes in their limbs and such proprioception is essential to human motor control (Clites et al., 2018). Consequently, to support natural ambulation, powered prostheses require an advanced autonomous adaptation, which can be achieved with a real-time implementation of joint position and moment prediction (Fleming et al., 2021). The present study serves this purpose by utilizing nonnormalized sEMG amplitudes and LSTM architecture for ankle joint position and moment predictions in level-ground walking. This structure can be implemented in real-time robotic control applications (e.g., Li et al., 2021) and used to generate reference inputs for advanced powered prosthesis controllers such as impedance control (Aghasadeghi et al., 2013; Wu et al., 2022).
In conclusion, a novel LSTM approach utilizing exclusively lower limb nonnormalized sEMG amplitudes was developed. Its feasibility for predicting ankle angle and moment during level walking of a healthy population was evaluated by testing five time-domain features (IEMG, MAV, WAMP, RMS, and WL) and eight leg muscle combinations (TA, SO, MG, PL, RF, VM, BG, and GMax). PL and GMax+VM performed best in predicting ankle motion while minimizing the total number of sEMG inputs and minimizing the use of lower leg muscles, respectively. The best performing variation MG + RF + VM combines upper and lower leg muscle inputs. The versatile LSTM architecture utilizing nonnormalized sEMG for sensor inputs makes the algorithms get closer to being implemented in a real-time implementation. Moreover, the comprehensive testing protocol developed involving ranking of all muscle combinations according to their success will facilitate user specific solutions. However, in order to translate an offline application into online applicable powered ankle prosthesis control algorithms, the present implementation needs to be extended with other movement types, followed by training and testing of the algorithms for data collected from amputee participants in future studies.
Data availability statement
The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding author.
Author contributions
AK, RT, and CY contributed to conception and design of the study. CY received the funding and edited sections of the manuscript. AK developed the algorithms, performed the analyses, and wrote the first draft of the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.
Funding
This work was supported by the Scientific and Technological Research Council of Turkey (TÜBI TAK) under grant 120E272 to CY. Open access publication fees were covered by European University of Brain and Technology - Research and Innovation funded by the European Commision under grant 101035817.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnins.2023.1158280/full#supplementary-material
References
Aghasadeghi, N., Zhao, H., Hargrove, L. J., Ames, A. D., Perreault, E. J., and Bretl, T. (2013). “Learning impedance controller parameters for lower-limb prostheses” in 2013 IEEE/RSJ international conference on intelligent robots and systems (Piscataway: IEEE), 4268–4274.
Ahmad, N., Thomas, G. N., Gill, P., and Torella, F. (2016). The prevalence of major lower limb amputation in the diabetic and non-diabetic population of England 2003-2013. Diabetes Vasc. Dis. Res. 13, 348–353. doi: 10.1177/1479164116651390
Ameri, A., Akhaee, M. A., Scheme, E., and Englehart, K. (2020). A deep transfer learning approach to reducing the effect of electrode shift in EMG pattern recognition-based control. IEEE Trans. Neural Syst. Rehabil. Eng. 28, 370–379. doi: 10.1109/TNSRE.2019.2962189
Angkoon, P., Chusak, L., and Pornchai, P. (2009). A novel feature extraction for robust EMG pattern recognition. J. Med. Eng. Technol. 40, 4, 149–154. doi: 10.48550/arXiv.0912.3973
Asif, M., Tiwana, M. I., Khan, U. S., Qureshi, W. S., Iqbal, J., Rashid, N., et al. (2021). Advancements, trends and future prospects of lower limb prosthesis. IEEE Access 9, 85956–85977. doi: 10.1109/ACCESS.2021.3086807
Attal, F., Amirat, Y., Chibani, A., and Mohammed, S. (2018). Automatic recognition of gait phases using a multiple-regression hidden Markov model. IEEE/ASME Trans. Mechatron. 23, 1597–1607. doi: 10.1109/TMECH.2018.2836934
Au, S., Berniker, M., and Herr, H. (2008). Powered ankle-foot prosthesis to assist level-ground and stair-descent gaits. Neural Netw. 21, 654–666. doi: 10.1016/j.neunet.2008.03.006
Au, S.K., Bonato, P., and Herr, H. (2005). An EMG-position controlled system for an active ankle-foot prosthesis: an initial experimental study, in: Proceedings of the 2005 IEEE 9th international conference on rehabilitation robotics. ICORR 2005, Chicago, IL, USA
Azocar, A.F., Mooney, L.M., Hargrove, L.J., and Rouse, E.J., (2018). Design and characterization of an open-source robotic leg prosthesis, in: Proceedings of the IEEE RAS and EMBS international conference on biomedical robotics and biomechatronics.
Bao, T., Zaidi, S. A. R., Xie, S., Yang, P., and Zhang, Z.-Q. (2021). A CNN-LSTM hybrid model for wrist kinematics estimation using surface electromyography. IEEE Trans. Instrum. Meas. 70, 1–9. doi: 10.1109/TIM.2020.3036654
Bi, L., Feleke, A., and Guan, C. (2019). A review on EMG-based motor intention prediction of continuous human upper limb motion for human-robot collaboration. Biomed. Signal Process. Control 51, 113–127. doi: 10.1016/j.bspc.2019.02.011
Brackx, B., Van Damme, M., Matthys, A., Vanderborght, B., and Lefeber, D. (2013). Passive ankle-foot prosthesis prototype with extended push-off. Int. J. Adv. Robot. Syst. 10:101. doi: 10.5772/55170
Chambers, A. J., Haney, J. M., Huppert, T., and Redfern, M. S. (2019). The effect of prolonged walking with intermittent standing on erector spinae and soleus muscle oxygenation and discomfort. J. Sports Sci. Med. 18, 337–343.
Chan, A.D.C., and Green, G.C. (2007). Myoelectric control development toolbox. In Proceedings of the 30th conference of the canadian medical & biological engineering society (CMBEC), Toronto, ON, Canada
Chen, C., Yu, Y., Sheng, X., and Zhu, X. (2022). Non-invasive analysis of motor unit activation during simultaneous and continuous wrist movements. IEEE J. Biomed. Heal. Informatics 26, 2106–2115. doi: 10.1109/JBHI.2021.3135575
Chen, J., Zhang, X., Cheng, Y., and Xi, N. (2018). Surface EMG based continuous estimation of human lower limb joint angles by using deep belief networks. Biomed. Signal Process. Control 40, 335–342. doi: 10.1016/j.bspc.2017.10.002
Chowdhury, R. H., Reaz, M. B. I., Bin Mohd Ali, M. A., Bakar, A. A. A., Chellappan, K., and Chang, T. G. (2013). Surface electromyography signal processing and classification techniques. Sensors (Switzerland). 13, 12431–12466. doi: 10.3390/s130912431
Clites, T. R., Carty, M. J., Ullauri, J. B., Carney, M. E., Mooney, L. M., Duval, J. F., et al. (2018). Proprioception from a neurally controlled lower-extremity prosthesis. Sci. Transl. Med. 10, 8373. doi: 10.1126/scitranslmed.aap8373
Crawshaw, M. (2020). Multi-task learning with deep neural networks: a survey. arXiv Prepr. arXiv2009.09796. doi: 10.48550/arXiv.2009.09796
Culver, S., Bartlett, H., Shultz, A., and Goldfarb, M. (2018). A stair ascent and descent controller for a powered ankle prosthesis. IEEE Trans. Neural Syst. Rehabil. Eng. 26, 993–1002. doi: 10.1109/TNSRE.2018.2819508
Dawley, J.A., Fite, K.B., and Fulk, G.D., (2013). EMG control of a bionic knee prosthesis: exploiting muscle co-contractions for improved locomotor function, in: IEEE international conference on rehabilitation robotics, Seattle, WA USA.
De Luca, C. J., Donald Gilmore, L., Kuznetsov, M., and Roy, S. H. (2010). Filtering the surface EMG signal: movement artifact and baseline noise contamination. J. Biomech. 43, 1573–1579. doi: 10.1016/j.jbiomech.2010.01.027
den Otter, A., Geurts, A. C., Mulder, T., and Duysens, J. (2004). Speed related changes in muscle activity from normal to very slow walking speeds. Gait Posture 19, 270–278. doi: 10.1016/S0966-6362(03)00071-7
Du, L., Zhang, F., He, H., and Huang, H., (2013). Improving the performance of a neural-machine interface for prosthetic legs using adaptive pattern classifiers, in: Proceedings of the annual international conference of the IEEE engineering in medicine and biology society, EMBS
Elery, T., Rezazadeh, S., Nesler, C., Doan, J., Zhu, H., and Gregg, R. D. (2018). Design and benchtop validation of a powered knee-ankle prosthesis with high-torque, low-impedance actuators. IEEE Int. Conf. Robot. Autom. 2018, 2788–2795. doi: 10.1109/ICRA.2018.8461259
Farmer, S., Silver-Thorn, B., Voglewede, P., and Beardsley, S. A. (2014). Within-socket myoelectric prediction of continuous ankle kinematics for control of a powered transtibial prosthesis. J. Neural Eng. 11:056027. doi: 10.1088/1741-2560/11/5/056027
Ferris, D. P., and Schlink, B. R. (2017). Robotic devices to enhance human movement performance. Kinesiol. Rev. 6, 70–77. doi: 10.1123/kr.2016-0040
Fleming, A., Stafford, N., Huang, S., Hu, X., Ferris, D. P., and Huang, H. H. (2021). Myoelectric control of robotic lower limb prostheses: a review of electromyography interfaces, control paradigms, challenges and future directions. J. Neural Eng. 18:041004. doi: 10.1088/1741-2552/ac1176
Foroutannia, A., Akbarzadeh-T, M.-R., and Akbarzadeh, A. (2022). A deep learning strategy for EMG-based joint position prediction in hip exoskeleton assistive robots. Biomed. Signal Process. Control 75:103557. doi: 10.1016/j.bspc.2022.103557
Gehlhar, R., Chen, Y., and Ames, A.D. (2020). Data-driven characterization of human interaction for model-based control of powered prostheses, 2020 IEEE/RSJ international conference on intelligent robots and systems (IROS). Las Vegas, NV: IEEE
Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., and Schmidhuber, J. (2017). LSTM: a search space odyssey. IEEE Trans. Neural Networks Learn. Syst. 28, 2222–2232. doi: 10.1109/TNNLS.2016.2582924
Gupta, R., Dhindsa, I. S., and Agarwal, R. (2020). Continuous angular position estimation of human ankle during unconstrained locomotion. Biomed. Signal Process. Control 60:101968. doi: 10.1016/j.bspc.2020.101968
Hahn, M. E., and O’Keefe, K. B. (2008). A neural network model for estimation of net joint moments during normal gait. J. Musculoskelet. Res. 11, 117–126. doi: 10.1142/S0218957708002036
Halaki, M., and Ginn, K. (2012). “Normalization of EMG signals: to normalize or not to normalize and what to normalize to?” in Computational intelligence in electromyography analysis – a perspective on current applications and future challenges. ed. G. R. Naik (Rijeka: InTech)
Hargrove, L. J., Simon, A. M., Young, A. J., Lipschutz, R. D., Finucane, S. B., Smith, D. G., et al. (2013). Robotic leg control with EMG decoding in an amputee with nerve transfers. N. Engl. J. Med. 369, 1237–1242. doi: 10.1056/nejmoa1300126
Hermens, H. J., Freriks, B., Disselhorst-Klug, C., and Rau, G. (2000). Development of recommendations for SEMG sensors and sensor placement procedures. J. Electromyogr. Kinesiol. 10, 361–374. doi: 10.1016/S1050-6411(00)00027-4
Hoover, C. D., Fulk, G. D., and Fite, K. B. (2012). The design and initial experimental validation of an active myoelectric transfemoral prosthesis. J. Med. Devices, Trans. ASME 6, 011005. doi: 10.1115/1.4005784
Hosseini, M., Meattini, R., San-Millan, A., Palli, G., Melchiorri, C., and Paik, J. (2020). A sEMG-driven soft ExoSuit based on twisted string actuators for elbow assistive applications. IEEE Robot. Autom. Lett. 5, 4094–4101. doi: 10.1109/LRA.2020.2988152
Huang, S., and Ferris, D. P. (2012). Muscle activation patterns during walking from transtibial amputees recorded within the residual limb-prosthetic interface. J. Neuroeng. Rehabil. 9:55. doi: 10.1186/1743-0003-9-55
Huang, H., Kuiken, T. A., and Lipschutz, R. D. (2009). A strategy for identifying locomotion modes using surface electromyography. IEEE Trans. Biomed. Eng. 56, 65–73. doi: 10.1109/TBME.2008.2003293
Huang, S., Wensman, J. P., and Ferris, D. P. (2016). Locomotor adaptation by transtibial amputees walking with an experimental powered prosthesis under continuous myoelectric control. IEEE Trans. Neural Syst. Rehabil. Eng. 24, 573–581. doi: 10.1109/TNSRE.2015.2441061
Huang, H., Zhang, F., Hargrove, L. J., Dou, Z., Rogers, D. R., and Englehart, K. B. (2011). Continuous locomotion-mode identification for prosthetic legs based on neuromuscular - mechanical fusion. I.E.E.E. Trans. Biomed. Eng. 58, 2867–2875. doi: 10.1109/TBME.2011.2161671
Hudgins, B., Parker, P., and Scott, R. N. (1993). A new strategy for multifunction myoelectric control. IEEE Trans. Biomed. Eng. 40, 82–94. doi: 10.1109/10.204774
Huihui, C., Farong, G., Chao, C., and Taixing, T. (2018). Estimation of ankle angle based on multi-feature fusion with random forest, in: Proceedings of the 2018 37th Chinese Control Conference (CCC). Wuhan
Hunt, G., Hood, S., and Lenzi, T. (2021). Stand-up, squat, lunge, and walk with a robotic knee and ankle prosthesis under shared neural control. IEEE Open J. Eng. Med. Biol. 2, 267–277. doi: 10.1109/OJEMB.2021.3104261
Hussain, T., Iqbal, N., Maqbool, H. F., Khan, M., Awad, M. I., and Dehghani-Sanij, A. A. (2020). Intent based recognition of walking and ramp activities for amputee using sEMG based lower limb prostheses. Biocybern. Biomed. Eng. 40, 1110–1123. doi: 10.1016/j.bbe.2020.05.010
Isaacs-Itua, A., and Sedki, I. (2018). Management of lower limb amputations. Br. J. Hosp. Med. 79, 205–210. doi: 10.12968/hmed.2018.79.4.205
Isakov, E., Burger, H., Gregorič, M., and Marinček, Č. (1996). Stump length as related to atrophy and strength of the thigh muscles in trans-tibial amputees. Prosthetics Orthot. Int. 20, 96–100. doi: 10.3109/03093649609164425
Ison, M., Vujaklija, I., Whitsell, B., Farina, D., and Artemiadis, P. (2016). High-density electromyography and motor skill learning for robust long-term control of a 7-DoF robot arm. IEEE Trans. Neural Syst. Rehabil. Eng. 24, 424–433. doi: 10.1109/TNSRE.2015.2417775
Jephil, P. B., Acharaya, P., Xu, L., Guo, K., Yu, H., Watsford, M., et al. (2020). “Estimation of ankle joint torque and angle based on S-EMG signal for assistive rehabilitation robots” in Biomedical signal processing. Series in BioEngineering. ed. G. Naik (Singapore: Springer)
Kaushik, S., Choudhury, A., Sheron, P. K., Dasgupta, N., Natarajan, S., Pickett, L. A., et al. (2020). AI in healthcare: time-series forecasting using statistical, neural, and ensemble architectures. Front. Big Data 3:4. doi: 10.3389/fdata.2020.00004
Keleş, A. D., and Yucesoy, C. A. (2020). Development of a neural network based control algorithm for powered ankle prosthesis. J. Biomech. 113:110087. doi: 10.1016/j.jbiomech.2020.110087
Kendall, A., Gal, Y., and Cipolla, R. (2018). Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 7482–7491. doi: 10.48550/arXiv.1705.07115
Khademi, G., and Simon, D. (2021). Toward minimal-sensing locomotion mode recognition for a powered knee-ankle prosthesis. I.E.E.E. Trans. Biomed. Eng. 68, 967–979. doi: 10.1109/TBME.2020.3016129
Khan, S. M., Khan, A. A., and Farooq, O. (2021). Pattern recognition of EMG signals for low level grip force classification. Biomed. Phys. Eng. Express 7:065012. doi: 10.1088/2057-1976/ac2354
Khushaba, R. N., Takruri, M., Miro, J. V., and Kodagoda, S. (2014). Towards limb position invariant myoelectric pattern recognition using time-dependent spectral features. Neural Netw. 55, 42–58. doi: 10.1016/j.neunet.2014.03.010
Kim, D., Koh, K., Oppizzi, G., Baghi, R., Lo, L.-C., Zhang, C., et al. (2020). “Simultaneous estimations of joint angle and torque in interactions with environments using EMG” in 2020 IEEE international conference on robotics and automation (ICRA) (Piscataway: IEEE), 3818–3824.
Kingma, D.P., and Ba, J.L. (2015). Adam: a method for stochastic optimization 3rd international conference on learning representations, ICLR 2015 – conference track proceedings San Diego
Kyeong, S., Feng, J., Ryu, J. K., Park, J. J., Lee, K. H., and Kim, J. (2022). Surface electromyography characteristics for motion intention recognition and implementation issues in lower-limb exoskeletons. Int. J. Control. Autom. Syst. 20, 1018–1028. doi: 10.1007/s12555-020-0934-3
Lee, S., Kim, M.-O., Kang, T., Park, J., and Choi, Y. (2018). Knit band sensor for myoelectric control of surface EMG-based prosthetic hand. IEEE Sensors J. 18, 8578–8586. doi: 10.1109/JSEN.2018.2865623
Lencioni, T., Carpinella, I., Rabuffetti, M., Marzegan, A., and Ferrarin, M. (2019). Human kinematic, kinetic and EMG data during different walking and stair ascending and descending tasks. Sci. Data 6:309. doi: 10.1038/s41597-019-0323-z
Lenzi, T., Cempini, M., Hargrove, L., and Kuiken, T. (2018). Design, development, and testing of a lightweight hybrid robotic knee prosthesis. Int. J. Robot. Res. 37, 953–976. doi: 10.1177/0278364918785993
Lenzi, T., Cempini, M., Hargrove, L. J., and Kuiken, T. A. (2019). Design, development, and validation of a lightweight nonbackdrivable robotic ankle prosthesis. IEEE/ASME Trans. Mechatronics 24, 471–482. doi: 10.1109/TMECH.2019.2892609
Li, T.-H. S., Kuo, P.-H., Cheng, C.-H., Hung, C.-C., Luan, P.-C., and Chang, C.-H. (2021). Sequential sensor fusion-based real-time LSTM gait pattern controller for biped robot. IEEE Sensors J. 21, 2241–2255. doi: 10.1109/JSEN.2020.3016968
Li, Z., Yuan, Y., Luo, L., Su, W., Zhao, K., Xu, C., et al. (2019). Hybrid brain/muscle signals powered wearable walking exoskeleton enhancing motor ability in climbing stairs activity. IEEE Trans. Med. Robot. Bionics 1, 218–227. doi: 10.1109/TMRB.2019.2949865
Liu, H., Huang, Q., and Tong, Z. (2019). Simulation and analysis of a full-active electro-hydrostatic powered ankle prosthesis, in: 2019 19th international conference on advanced robotics, ICAR 2019 Belo Horizonte, Brazil
Liu, M., Zhang, F., and Huang, H. H. (2017). An adaptive classification strategy for reliable locomotion mode recognition. Sensors (Switzerland) 17. doi: 10.3390/s17092020
Lu, Y., Wang, H., Zhou, B., Wei, C., and Xu, S. (2022). Continuous and simultaneous estimation of lower limb multi-joint angles from sEMG signals based on stacked convolutional and LSTM models. Expert Syst. Appl. 203:117340. doi: 10.1016/j.eswa.2022.117340
Ma, X., Liu, Y., Song, Q., and Wang, C. (2020). Continuous estimation of knee joint angle based on surface electromyography using a long short-term memory neural network and time-advanced feature. Sensors 20:4966. doi: 10.3390/s20174966
Miller, J. D., Beazer, M. S., and Hahn, M. E. (2013). Myoelectric walking mode classification for Transtibial amputees. I.E.E.E. Trans. Biomed. Eng. 60, 2745–2750. doi: 10.1109/TBME.2013.2264466
Molnar, C. (2020). Interpretable Machine Learning: A Guide For Making Black Box Models Explainable, Leanpub Publishing.
Nasr, A., Laschowski, B., and McPhee, J. (2021). Myoelectric control of robotic leg prostheses and exoskeletons: a review, in: Proceedings of the ASME design engineering technical conference. Springer, Berlin
Pataky, T. C. (2010). Generalized n-dimensional biomechanical field analysis using statistical parametric mapping. J. Biomech. 43, 1976–1982. doi: 10.1016/j.jbiomech.2010.03.008
Phinyomark, A., Hirunviriya, S., Nuidod, A., Phukpattaranont, P., and Limsakul, C. (2011). “Evaluation of EMG feature extraction for movement control of upper limb prostheses based on class separation index” in 5th Kuala Lumpur International Conference on Biomedical Engineering 2011. IFMBE Proceedings. eds. N. A. A. Osman, W. A. B. W. Abas, A. K. A. Wahab, and H. N. Ting, vol. 35 (Berlin, Heidelberg: Springer)
Quintero, D., Villarreal, D. J., Lambert, D. J., Kapp, S., and Gregg, R. D. (2018). Continuous-phase control of a powered knee-ankle prosthesis: amputee experiments across speeds and inclines. IEEE Trans. Robot. 34, 686–701. doi: 10.1109/TRO.2018.2794536
Rabe, K. G., and Fey, N. P. (2022). Evaluating electromyography and Sonomyography sensor fusion to estimate lower-limb kinematics using Gaussian process regression. Front. Robot. AI 9:716545. doi: 10.3389/frobt.2022.716545
Ranz, E. C., Wilken, J. M., Gajewski, D. A., and Neptune, R. R. (2017). The influence of limb alignment and transfemoral amputation technique on muscle capacity during gait. Comput. Methods Biomech. Biomed. Engin. 20, 1167–1174. doi: 10.1080/10255842.2017.1340461
Reaz, M. B. I., Hussain, M. S., and Mohd-Yasin, F. (2006). Techniques of EMG signal analysis: detection, processing, classification and applications. Biol. Proced. Online 8, 11–35. doi: 10.1251/bpo115
Rebuffi, S.-A., Bilen, H., and Vedaldi, A. (2017). Learning multiple visual domains with residual adapters. Adv. Neural Inf. Process. Syst 30
Ren, B., Zhang, Z., Zhang, C., and Chen, S. (2022). Motion trajectories prediction of lower limb exoskeleton based on long short-term memory (LSTM) networks. Actuators 11:73. doi: 10.3390/act11030073
Rojas-Martínez, M., Mañanas, M. A., and Alonso, J. F. (2012). High-density surface EMG maps from upper-arm and forearm muscles. J. Neuroeng. Rehabil. 9:85. doi: 10.1186/1743-0003-9-85
Scheme, E., and Englehart, K. (2011). Electromyogram pattern recognition for control of powered upper-limb prostheses: state of the art and challenges for clinical use. J. Rehabil. Res. Dev. 48:643. doi: 10.1682/JRRD.2010.09.0177
Schlink, B. R., Nordin, A. D., Brooks, C. N., and Ferris, D. P. (2021). Fatigue induces altered spatial myoelectric activation patterns in the medial gastrocnemius during locomotion. J. Neurophysiol. 125, 2013–2023. doi: 10.1152/jn.00602.2020
Sepulveda, F., Wells, D. M., and Vaughan, C. L. (1993). A neural network representation of electromyography and joint dynamics in human gait. J. Biomech. 26, 101–109. doi: 10.1016/0021-9290(93)90041-C
Shultz, A. H., and Goldfarb, M. (2018). A unified controller for walking on even and uneven terrain with a powered ankle prosthesis. IEEE Trans. Neural Syst. Rehabil. Eng. 26, 788–797. doi: 10.1109/TNSRE.2018.2810165
Smith, D. G. (2004). “General principles of amputation surgery” in Atlas of amputations and limb deficiencies. Surgical, prosthetic, and rehabilitation principles 1st Ed, American Academy of Orthopaedic Surgeons.
Song, J., Zhu, A., Tu, Y., Huang, H., Arif, M. A., Shen, Z., et al. (2020). Effects of different feature parameters of sEMG on human motion pattern recognition using multilayer perceptrons and LSTM neural networks. Appl. Sci. 10, 3358. doi: 10.3390/APP10103358
Souza, J. M., Fey, N. P., Cheesborough, J. E., Agnew, S. P., Hargrove, L. J., and Dumanian, G. A. (2014). Advances in transfemoral amputee rehabilitation: early experience with targeted muscle reinnervation. Curr. Surg. Rep. 2:51. doi: 10.1007/s40137-014-0051-4
Spanias, J. A., Perreault, E. J., and Hargrove, L. J. (2016). Detection of and compensation for EMG disturbances for powered lower limb prosthesis control. IEEE Trans. Neural Syst. Rehabil. Eng. 24, 226–234. doi: 10.1109/TNSRE.2015.2413393
Spanias, J. A., Simon, A. M., Finucane, S. B., Perreault, E. J., and Hargrove, L. J. (2018). Online adaptive neural control of a robotic lower limb prosthesis. J. Neural Eng. 15:016015. doi: 10.1088/1741-2552/aa92a8
Spanias, J.A., Simon, A.M., Ingraham, K.A., and Hargrove, L.J. (2015). Effect of additional mechanical sensor data on an EMG-based pattern recognition system for a powered leg prosthesis, in: International IEEE/EMBS conference on neural engineering, NER. Montpellier, France
Sup, F., Varol, H.A., Mitchell, J., Withrow, T., and Goldfarb, M. (2008). Design and control of an active electrical knee and ankle prosthesis, in: Proceedings of the 2nd biennial IEEE/RAS-EMBS international conference on biomedical robotics and biomechatronics, BioRob 2008. 523–528
Tanaka, T., Nambu, I., Maruyama, Y., and Wada, Y. (2022). Sliding-window normalization to improve the performance of machine-learning models for real-time motion prediction using electromyography. Sensors 22:5005. doi: 10.3390/s22135005
Tintle, S. M., Keeling, J. J., Shawen, S. B., Forsberg, J. A., and Potter, B. K. (2010). Traumatic and trauma-related amputations: part I: general principles and lower-extremity amputations. J. Bone Jt. Surg. A. 92, 2852–2868. doi: 10.2106/JBJS.J.00257
Triwiyanto, T., Wahyunggoro, O., Nugroho, H. A., and Herianto, H. (2018). Muscle fatigue compensation of the electromyography signal for elbow joint angle estimation using adaptive feature. Comput. Electr. Eng. 71, 284–293. doi: 10.1016/j.compeleceng.2018.07.026
Truong, M. T. N., Ali, A. E. A., Owaki, D., and Hayashibe, M. (2023). EMG-based estimation of lower limb joint angles and moments using long short-term memory network. Sensors 23:3331. doi: 10.3390/s23063331
Tucker, M. R., Olivier, J., Pagel, A., Bleuler, H., Bouri, M., Lambercy, O., et al. (2015). Control strategies for active lower extremity prosthetics and orthotics: a review. J. Neuroeng. Rehabil. 12:1. doi: 10.1186/1743-0003-12-1
Varol, H. A., Sup, F., and Goldfarb, M. (2010). Multiclass real-time intent recognition of a powered lower limb prosthesis. I.E.E.E. Trans. Biomed. Eng. 57, 542–551. doi: 10.1109/TBME.2009.2034734
Vucina, A., Hudec, M., and Raspudic, V. (2005). Kinematics and forces in the above-knee prosthesis during the stair climbing. Int. J. Simul. Model. 4, 17–26. doi: 10.2507/IJSIMM04(1)2.033
Wang, J., Dai, Y., and Si, X. (2021). Analysis and recognition of human lower limb motions based on electromyography (EMG) signals. Electronics 10:2473. doi: 10.3390/electronics10202473
Wang, J., Kannape, O. A., and Herr, H. M. (2013). “Proportional EMG control of ankle plantar flexion in a powered transtibial prosthesis” in IEEE international conference on rehabilitation robotics, 1–5.
Wang, J., Wang, L., Miran, S. M., Xi, X., and Xue, A. (2019). Surface electromyography based estimation of knee joint angle by using correlation dimension of wavelet coefficient. IEEE Access 7, 60522–60531. doi: 10.1109/ACCESS.2019.2913959
Wu, R., Li, M., Yao, Z., Liu, W., Si, J., and Huang, H. (2022). Reinforcement learning impedance control of a robotic prosthesis to coordinate with human intact knee motion. IEEE Robot. Autom. Lett. 7, 7014–7020. doi: 10.1109/LRA.2022.3179420
Xu, X., Xu, X., Liu, Y., Zhong, K., and Zhang, H. (2021). Design of bionic active–passive hybrid-driven prosthesis based on gait analysis and simulation of compound control method. Biomed. Eng. Online 20:126. doi: 10.1186/s12938-021-00962-9
Yi, C., Jiang, F., Zhang, S., Guo, H., Yang, C., Ding, Z., et al. (2021). Continuous prediction of lower-limb kinematics from multi-modal biomedical signals. IEEE Trans. Circuits Syst. Video Technol. 32, 2592–2602. doi: 10.1109/TCSVT.2021.3071461
Yin, G., Zhang, X., Chen, D., Li, H., Chen, J., Chen, C., et al. (2020). Processing surface EMG signals for exoskeleton motion control. Front. Neurorobot. 14:40. doi: 10.3389/fnbot.2020.00040
Young, A. J., Kuiken, T. A., and Hargrove, L. J. (2014). Analysis of using EMG and mechanical sensors to enhance intent recognition in powered lower limb prostheses. J. Neural Eng. 11:056021. doi: 10.1088/1741-2560/11/5/056021
Zabre-Gonzalez, E. V., Riem, L., Voglewede, P. A., Silver-Thorn, B., Koehler-McNicholas, S. R., and Beardsley, S. A. (2021). Continuous myoelectric prediction of future ankle angle and moment across ambulation conditions and their transitions. Front. Neurosci. 15:709422. doi: 10.3389/fnins.2021.709422
Zaffir, M. A. B. M., Nuwantha, P., Arase, D., Sakurai, K., and Tamura, H. (2021). Comparison of deep neural network models and effectiveness of emg signal feature value for estimating dorsiflexion. Electron 10, 2767. doi: 10.3390/electronics10222767
Zhang, K., de Silva, C.W., and Fu, C. (2019). Sensor fusion for predictive control of human-prosthesis-environment dynamics in assistive walking: a survey. doi: 10.48550/arXiv.1903.07674
Zhang, Q., Iyer, A., Sun, Z., Kim, K., and Sharma, N. (2021). A dual-modal approach using electromyography and Sonomyography improves prediction of dynamic ankle movement: a case study. IEEE Trans. Neural Syst. Rehabil. Eng. 29, 1944–1954. doi: 10.1109/TNSRE.2021.3106900
Zhang, Q., Lambeth, K., Sun, Z., Dodson, A., Bao, X., and Sharma, N. (2023). Evaluation of a fused sonomyography and electromyography-based control on a cable-driven ankle exoskeleton. IEEE Trans. Robot. 39, 2183–2202. doi: 10.1109/TRO.2023.3236958
Zhang, F., Li, P., Hou, Z.-G., Lu, Z., Chen, Y., Li, Q., et al. (2012). sEMG-based continuous estimation of joint angles of human legs by using BP neural network. Neurocomputing 78, 139–148. doi: 10.1016/j.neucom.2011.05.033
Zhang, X., and Ma, M. (2023). Research on sEMG feature generation and classification performance based on EBGAN. Electron 12:1040. doi: 10.3390/electronics12041040
Keywords: powered ankle prosthesis, surface electromyogram (sEMG), long short-term memory neural network, feature extraction, lower limb amputation
Citation: Keleş AD, Türksoy RT and Yucesoy CA (2023) The use of nonnormalized surface EMG and feature inputs for LSTM-based powered ankle prosthesis control algorithm development. Front. Neurosci. 17:1158280. doi: 10.3389/fnins.2023.1158280
Edited by:
Jit Muthuswamy, Arizona State University, United StatesReviewed by:
Qiang Zhang, University of North Carolina at Chapel Hill, United StatesAhad Behboodi, National Institutes of Health (NIH), United States
Copyright © 2023 Keleş, Türksoy and Yucesoy. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Can A. Yucesoy, Y2FuLnl1Y2Vzb3lAYm91bi5lZHUudHI=