Eric J. Earley
Anton Berneving
Jan Zbinden
Max Ortiz-Catalan- 1Center for Bionics and Pain Research, Mölndal, Sweden
- 2Department of Electrical Engineering, Chalmers University of Technology, Gothenburg, Sweden
- 3Operational Area 3, Sahlgrenska University Hospital, Gothenburg, Sweden
- 4Department of Orthopedics, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
As the demand for prosthetic limbs with reliable and multi-functional control increases, recent advances in myoelectric pattern recognition and implanted sensors have proven considerably advantageous. Additionally, sensory feedback from the prosthesis can be achieved via stimulation of the residual nerves, enabling closed-loop control over the prosthesis. However, this stimulation can cause interfering artifacts in the electromyographic (EMG) signals which deteriorate the reliability and function of the prosthesis. Here, we implement two real-time stimulation artifact removal algorithms, Template Subtraction (TS) and ε-Normalized Least Mean Squares (ε-NLMS), and investigate their performance in offline and real-time myoelectric pattern recognition in two transhumeral amputees implanted with nerve cuff and EMG electrodes. We show that both algorithms are capable of significantly improving signal-to-noise ratio (SNR) and offline pattern recognition accuracy of artifact-corrupted EMG signals. Furthermore, both algorithms improved real-time decoding of motor intention during active neurostimulation. Although these outcomes are dependent on the user-specific sensor locations and neurostimulation settings, they nonetheless represent progress toward bi-directional neuromusculoskeletal prostheses capable of multifunction control and simultaneous sensory feedback.
Introduction
Ziegler-Graham et al. (2008) estimated that the number of people living with amputations in the US would more than double by the year 2050. Furthermore, it is estimated that there are more than 1 million annual limb amputations globally (Advanced Amputee Solutions LLC, 2012). This poses a significant challenge as the demand for prosthetic devices with reliable and multi-functional control for intuitive use in daily life increases.
During the last decades, most of the development has lead in the direction of introducing powered prostheses where movement control is decoded from surface electromyogram (sEMG) using myoelectric pattern recognition (Hudgins et al., 1993; Englehart and Hudgins, 2003; Zheng et al., 2021). Further work has also been performed to improve control resolution and reliability through Targeted Muscle Reinnervation (TMR), where nerves in the remaining limb are innervated into existing musculature to increase the number of EMG channels and improve prosthesis controllability (Kuiken et al., 2004).
However, using sEMG for prosthesis control comes with a multitude of problems as the signal quality is heavily dependent on environmental conditions and susceptible to motion artifacts and myoelectric crosstalk (Ortiz-Catalan et al., 2014). To remedy this, recent work extending the concept of bone-anchored (osseointegrated) prostheses to also include bi-directional electrical communication has allowed electrodes to be implanted and connected directly through the implant to the prosthesis. This has improved controllability and general prosthesis usability over classical myoelectric prostheses (Ortiz-Catalan et al., 2014, 2020b; Mastinu et al., 2018).
The bi-directional communication additionally allows neurostimulation to provide sensory feedback to the user (Ortiz-Catalan et al., 2014, 2020a). By placing spiral cuff electrodes around nerves in the residual limb, somatosensory (touch) sensations can be elicited through neurostimulation. However, due to the nature of electrical signals, and the fact that the electrical stimulation pulses are often larger in amplitude compared to the underlying EMG signal, the stimulation pulses can also be picked up by the nearby EMG electrodes. This creates unwanted artifacts in the activation patterns used to detect the user’s intent and leads to reduced pattern recognition performance and robustness (Hartmann et al., 2015).
The problem of stimulation artifacts (SAs) is not only present in the area of prosthesis control, but also applies to any system involving closed-loop neuromodulation (Zhou et al., 2018). Literature on the topic of removing unwanted signals from recorded biosignals is expansive; for example, removal of ocular artifacts is of great interest for those using electroencephalography (EEG) for brain-computer interface applications (Nolan et al., 2010; Sreeja et al., 2018; Jafarifarmand and Badamchizadeh, 2019). However, research involving prosthesis control applications is lacking. Moreover, the existing literature mostly focuses on removing electrocardiogram (ECG) artifacts from the EMG signals (Marque et al., 2005; Zhou et al., 2007) and do not consider the artifacts caused by neurostimulation for providing tactile feedback. Furthermore, while prior work has investigated SAs and closed-loop myoelectric control (Hartmann et al., 2015), stimulation and recording were conducted via surface electrodes placed on the skin. Therefore, there is a need to develop and test methods for real-time stimulation artifact removal (SAR) using implanted EMG electrodes.
In this paper, we implement two real-time stimulation artifact removal algorithms, Template Subtraction (TS) and ε-Normalized Least Mean Squares (ε-NLMS), and investigate their performance in offline and real-time myoelectric pattern recognition with two transhumeral amputees implanted with nerve cuff and EMG electrodes. Using offline analysis, we show that both algorithms are capable of significantly improving signal-to-noise ratio (SNR) and pattern recognition accuracy of artifact-corrupted EMG signals. Furthermore, both algorithms improved real-time decoding during a Motion Test (Kuiken et al., 2009; Ortiz-Catalan et al., 2013) performed during active neurostimulation.
Materials and methods
This study was approved by the Swedish regional ethical committee in Gothenburg (DNR: 769-12). All participants provided written informed consent prior to participation in the study. Data and code related to this study are freely available on the Open Science Framework (Earley et al., 2022a).
Hardware
The embedded hardware was based on previous work (Mastinu et al., 2017). It comprises a TM4C123GH6PM 32-bit ARM Cortex-M4F main microcontroller unit (MCU) with floating point unit clocked at 80 MHz (Texas Instruments, Dallas, TX, USA) and a secondary MSP430G2755 16-bit mixed signal MCU at 16 MHz (Texas Instruments, Dallas, TX, USA). The secondary MCU handles stimulation waveform generation and signal acquisition through a RHS2116 digital electrophysiology stimulator and amplifier chip (Intan Technologies, Los Angeles, CA, USA), capable of sampling from implanted electrodes and stimulating the extraneural spiral-cuff electrodes within the arm of a person with a neuromusculoskeletal prosthesis (Integrum AB, Mölndal, Sweden). During stimulation pulse generation, the secondary MCU ceases signal acquisition (Figure 1A, in red), effectively blanking the EMG signal to avoid saturating the EMG channels and quickly return the electrode potential to baseline after each stimulation pulse (Hartmann et al., 2015). This blanking removes most of the SA spike but leaves the longer lasting exponential tail still in the signal (O’Keeffe et al., 2001; Zhou et al., 2018). However, by sampling the stimulation channel, a reference signal highly correlated with the stimulation artifact can be acquired for later use by SAR algorithms.
Figure 1. The Template Subtraction (TS) algorithm identifies stimulation artifacts from the original signal x(t) [(A), in blue] immediately following neurostimulation [(A), in red] and averages them using exponential filters (B) to form an artifact template (C). This template is subtracted from a newly-identified artifact [(D), in orange)] to yield an estimated artifact-free signal (E).
Algorithm selection and description
Two algorithms were selected with focus on ease of implementation, computational complexity, and ability to perform SAR in real-time during closed-loop control of a prosthetic hand. Both algorithms were implemented such that they were to be executed immediately following the delivery of a neurostimulation pulse, aligning with the first non-blanked sample after stimulation pulse generation.
Template Subtraction
We implemented a TS algorithm based on a first order infinite impulse response (IIR) filter, similar to the ones presented by Keller and Popovic (2001) and Azin et al. (2007). The algorithm was chosen due to its simplicity and recursive formulation, yielding a computationally efficient implementation for usage in real-time on an embedded device.
By averaging artifacts immediately following stimulation pulses using multiple exponential filters with filter learning rate α, a representative template of length N is constructed which is subtracted from the original signal x(t) to yield an estimated artifact-free signal as illustrated in Figure 1.
Let Wk(i) represent the learned artifact at sample i = 1,…,N after the end of stimulation pulse k. Assuming that tk is the sample index at which stimulation pulse k ends, i.e., when the stimulation artifact is present in the EMG signal, the recursive update is defined as:
and the estimated artifact-free signal is then given by subtracting the template:
ε-Normalized Least Mean Squares
We also implemented a variant of the common Least Mean Squares (LMS) adaptive filter, namely the ε-NLMS filter. The ε-NLMS algorithm is an improved version of the standard LMS algorithm, yielding better performance for signals with intervals of larger and lower signal energy such as speech signals and overall faster convergence (Sayed, 2008). The ε-NLMS algorithm is slightly more computationally expensive than the standard LMS algorithm, but has still been successfully used to remove neurostimulation artifacts in real-time (Basir-Kazeruni et al., 2017).
As the adaptive filter relies on a reference signal highly correlated with the SA (in our case, samples from the stimulation channel itself), the algorithm can adapt to varying artifact waveforms without requiring completely relearning of the weights. Figure 2 illustrates the working principle of the adaptive filter.
Figure 2. The ε-Normalized Least Mean Squares (ε-NLMS) adaptive filter algorithm convolutes the reference signal u(t) with filter weights w to predict the artifact waveform (A). The predicted artifact is subtracted from the original signal x(t) to yield an estimated artifact-free signal (B), which is then also used to update the filter weights after each sample (C).
Let x(t) be the original signal, u(t) = [u(t)u(t−1)…u(t−N−1)] a vector containing the N previous samples from the reference u(t), and wt a weight vector of length N at sample t. The estimated artifact-free signal is then given as:
and the weights are updated according to the ε-NLMS update rule:
where ϵ > 0 is a small constant for avoiding division by zero, and α is a positive learning rate parameter.
Offline evaluation
An offline evaluation to identify optimal algorithm parameters and investigate the sensitivity to parameter variation was conducted with one participant with a transhumeral amputation and an osseointegrated e-OPRA implant system (Integrum AB, Mölndal, Sweden) connected to a prosthetic arm as previously reported (Ortiz-Catalan et al., 2020b). Signals were sampled at 1,000 Hz from the four channels normally used to control hand open, hand close, pronate, and supinate functions using direct control. In addition, the stimulation channel was sampled as a reference for the ε-NLMS algorithm. Signals were processed with a 50 Hz notch filer to reduce electrical interference and passed through a second order digital high-pass filter at 20 Hz to remove signal bias. Square biphasic, asymmetric, and stimulation pulses (Günter et al., 2019) were applied at the stimulation frequency through the cuff electrode around the median nerve.
Initial data collection
We performed an initial data collection to obtain EMG signals both with and without SAs for offline evaluation. During the experiment, the participant was asked to perform three tasks: (i) no movement, (ii) repeated hand close and hand open, or (iii) repeated pronation and supination. Signals in scenario (i) were recorded for 10 s, while scenarios (ii) and (iii) were recorded for 15 s each. During the first half of each recording, stimulation was applied in the form of multiple successive pulses, where four sets of suitable stimulation parameters were chosen based on the participant’s detection threshold—pulse amplitudes of 300–450 μA, pulse frequencies of 30–50 Hz, and a pulse width of 150 μs. In total, 12 recordings were obtained containing both artifact-free and artifact-contaminated EMG signals.
Algorithm parameter selection
To select the values of hyperparameters N and α for each algorithm, we devised an optimization scheme based on semi-synthetic signals. By using the data collected during scenario (i) and superimposing pure stimulation artifacts a(t) from scenario (ii) and (iii) onto the artifact-free EMG signals x(t), the Root Mean Square Error (RMSE) between the true artifact-free signal x(t) and the estimated artifact-free signal , , can be used as the optimization objective (Liang and Lin, 2002; Li et al., 2019). This approach has been used in previous work for evaluating SAR algorithm performance (Liang and Lin, 2002; De Clercq et al., 2006; Waddell et al., 2009), and provides a method by which the outcome is compared to the true artifact-free signal x(t), which cannot be known in in vivo testing. Due to the large variance in signal amplitude, the parameters were optimized for each channel individually and selected based on the median optimal RMSE of each of the 32 semi-synthetic signal combinations.
Sensitivity analysis
Since selecting the algorithm hyperparameters is a complicated task when no existing signals are available for use in the above-described optimization scheme, we investigated the change in algorithm performance with respect to small deviations from the optimal parameters for N and α.
Using the same set of semi-synthetic signals as the optimization procedure, the algorithms’ hyperparameters were varied before applying them to the signals. However, using the RMSE between the true EMG signal x(t) and the estimated artifact-free signal as a performance metric can be problematic for comparing performance between channels due to the large variation in signal amplitude. The RMSE is completely usable when optimizing algorithm performance, but as soon as comparing signals of different fundamental amplitudes, another metric is required. Additionally, the RMSE is difficult to interpret and relate in terms of absolute performance.
Therefore, a more general metric based on SNR, as employed in Basir-Kazeruni et al. (2017), was used for evaluating the algorithms’ performance related to changes in the hyperparameters. The metric, hereby denoted as ΔSNR or SNR improvement, measures the change in SNR in decibels when applying the algorithm. It is not dependent on the absolute amplitude of the signals, but rather on the relative energy content of the true signal x(t), artifact a(t), and estimated artifact-free signal , as
The identified optimized parameters and the sensitivity analyses were used to guide the selection of parameters for the real-time evaluations (see section “Real-time evaluations”).
Pattern recognition performance
To evaluate the impact of the algorithms on the pattern recognition performance, signals were manually labeled with both the intended movement and the presence or absence of simultaneous stimulation (Figure 3). Both SAR algorithms were applied to pre-recorded signals before separating the samples into time windows of 200 samples with 150 samples of overlap. In accordance with the findings by Hartmann et al. (2015), blanked samples were removed from signals before extracting Mean Absolute Value (MAV), Zero Crossings (ZC), Slope Sign Changes (SSC), and Waveform Length (WL) from each window (Hudgins et al., 1993).
Figure 3. Overview of the process used for evaluating offline myoelectric pattern recognition performance of the two algorithms on signals containing stimulation artifacts.
The feature vectors from time windows without SAs were used to train a one-layer neural network using Rectified Linear Unit (ReLU) and softmax activation functions for predicting the intended prosthesis movement. The feature vectors used for training were randomly split into a 60% training set, 20% validation set and a 20% testing set, and the training set was augmented by adding additional 10 dB SNR noise to the feature vectors after being normalized. The network was trained and evaluated in MATLAB (MathWorks Inc., Natick, MA, USA).
The remaining feature vectors, calculated on time windows containing SAs, both raw and processed by each algorithm, were fed to the trained network to evaluate the network and algorithm performance. The training and evaluation procedure was performed 100 times to account for the inherent randomness during data splitting and training.
Real-time evaluations
To evaluate the real-time implications for prosthesis controllability, Motion Tests were performed with two participants with transhumeral amputation (Kuiken et al., 2009) as implemented in BioPatRec (Ortiz-Catalan et al., 2013). Both participants were home users of the osseointegrated e-OPRA implant.
First, a one layer neural network was trained on the standard features MAV, ZS, SSC, and WL (Hudgins et al., 1993) calculated from time windows at 500 Hz of length 200 ms, with 150 ms overlap on eight EMG channels to predict the prosthesis movements hand open, hand close, pronate, supinate, and rest. No stimulation was active during the training and thus no SAs were included in the training data.
For each Motion Test, the participant was instructed to perform each movement multiple times in randomized order and hold the correct movement for a total of 1 s (non-continuous) within a time limit of 5 s while measuring the time to completion. The trial would conclude after the 1 s of cumulative correct movement predictions were achieved; for unsuccessful trials, timed outcomes were set to the maximum time (5 s). First, a test without stimulation was performed to assess the baseline controllability of the prosthesis. Then, stimulation as a train of successive stimulation pulses at 20 Hz was enabled during the test; stimulation amplitude and pulse width were selected such that control of the prosthesis was impaired, but limited control of the prosthesis was still possible. Finally, the two algorithms were enabled on all signal channels and two additional motion tests (one for each algorithm) were performed. Guided by the outcomes from the sensitivity analysis, the TS algorithm was implemented with a learning rate α = 0.06 and a template length N = 25, and the ε-NLMS was implemented with a learning rate α = 0.035 and a filter length N = 25. The order of SAR algorithms was randomized.
Statistical analysis
For the offline evaluation, where the finite boundaries of accuracy skewed the outcomes, the Wilcoxon rank sum test was performed to assess if there was a significant difference in classification accuracy for signals containing SAs and after applying the TS and ε-NLMS algorithms. Corrections for multiple comparisons were made using Holm–Bonferroni corrections.
For the Motion Test, where outcomes were either discretized or skewed due to incomplete trials, the Wilcoxon rank sum test was performed to assess if there was a significant difference in completion rate (percentage of Motion Test trials completed successfully), completion time (duration of each trial, maximum 5 s for unsuccessful trials), movement accuracy (percentage of correct predictions during each trial), and selection time (elapsed time before first correct prediction, maximum 5 s for unsuccessful trials) between movements with and without SAs, and between the artifact-corrupted trials and the trials with SAR enabled. Corrections for multiple comparisons were made using Holm–Bonferroni corrections.
Results
Offline evaluation
Using the initial data collected, we performed a series of offline evaluations of the two algorithms to investigate their sensitivity to tuning of hyperparameters and effect on MPR classification accuracy.
Sensitivity analysis
Learning rate
As can be seen in Figure 4, larger learning rates α generally led to more variability in the ΔSNR for both algorithms, indicating that a larger learning rate improves SNR more for some semi-synthetic signals while causing a reduced improvement for others. For some parameter values, the ΔSNR also yielded negative changes, meaning that the algorithms induced more artifact noise into the signal than what was effectively removed. This was especially prominent for ε-NLMS, which displayed an overall larger performance variation.
Figure 4. ΔSNR increased with an increasing learning rate α to a point, after which further increases in the learning rate worsened the change. The sensitivity of both Template Subtraction (TS) (blue) and ε-Normalized Least Mean Squares (ε-NLMS) (red) showed similar trends. Higher learning rates α led to increased variability (shaded regions), but the α determined by the parameter optimization (dashed vertical lines) provided a good balance between median performance and variability.
By comparing the ΔSNR curves with the dashed lines representing the optimal parameter value for minimizing the median RMSE, it is evident that both performance metrics (RMSE and ΔSNR) were seemingly well correlated. The optimal parameters (dashed lines) generally provided a sound trade-off between the median and lower quartile SNR improvement. For example, consider the performance of the TS algorithm (blue) for the supinate channel (lower right) in Figure 4—by purely optimizing median ΔSNR, the optimal parameter would be α=0.04 which would yield a larger ΔSNR variability and even cause the lower quartile to lie below zero. Instead, the RMSE optima provided similar median and upper quartile ΔSNR performance, but with a considerably better lower quartile performance.
In summary, an increased learning rate α led to increased performance variability and optimizing for RMSE provided a good balance between median performance and variability. Furthermore, an excessive learning rate caused negative ΔSNR improvement, indicating that α needs to be tuned sufficiently low for correct functionality, especially for ε-NLMS.
Template/filter length
Considering the sensitivity when the template/filter length N was varied in Figure 5, the choice of N does not seem to be as crucial for determining the algorithms’ performance as the learning rate α.
Figure 5. ΔSNR generally improved with an increasing length N for Template Subtraction (TS) (blue), however ε-Normalized Least Mean Squares (ε-NLMS) (red) required a minimum length be met before improvement was seen. Variability of the outcome (shaded region) did not appear to relate to the length N, nor did the value determined via parameter optimization (dashed vertical lines) consistently align with the maximum ΔSNR.
One may observe that after a certain length the performance stayed constant. This is a logical behavior, considering that the number of samples between each stimulation pulse is dependent on the sampling and stimulation frequency. In the offline evaluation, the semi-synthetic signals were sampled at 1,000 Hz and contained stimulation pulses generated at 30–50 Hz, leading to a maximum of roughly 33 samples between each pulse. It is therefore understandable that the algorithm performance stayed constant once N ≥ 30 as both algorithms are reset to start over again once a new pulse is detected.
Another interesting observation is that TS behaved similarly across all channels. When the template length was increased from zero, the SNR improvement was immediate and continued to increase when the template was extended. Considering ε-NLMS, however, a too short filter (N ≤ 7 in this case) decreased the SNR between the residual artifact and the true EMG signal when applying the algorithm, rather than removing the stimulation artifact. It thus seems important that the filter length is chosen long enough to correctly reduce the artifact signal power.
In contrast to the learning rate, both algorithms’ performance consistently improved as the template/filter length N increased. However, ε-NLMS required a sufficient length before any SNR improvement was noticed while TS improved the SNR immediately when increasing the length from zero. This indicates that ε-NLMS may require more care when manually tuning the filter length. Optimizing for RMSE seemed to provide a reasonable trade-off between performance and variability, although the observation was not as evident as for the learning rate.
Offline pattern recognition
Using the constructed semi-synthetic signals containing artifacts, we evaluated the effect of stimulation on the MPR performance. In total, n = 100 training and evaluation iterations were performed to account for the inherent randomness of the training data split and network initialization. The total accuracy, defined as the portion of the predictions that were correct, decreased significantly between artifact-free signals (median [IQR]: 95.0% [94.5%, 95.7%]) and signals containing raw SAs (48.1% [46.4%, 49.5%]; Wilcoxon rank sum: p < 0.001), and was significantly improved after applying either of TS (78.8% [77.0%, 80.2%], p < 0.001) or ε-NLMS (77.1% [74.9%, 78.7%], p < 0.001) compared to artifact-corrupted signals (see Figure 6). The accuracy with either SAR algorithm did not reach the same levels as the artifact-free test signals (p < 0.001) although TS performed better than ε-NLMS (p < 0.001).
Figure 6. Offline pattern recognition accuracy was significantly reduced for signals containing stimulation artifacts (SAs) but both Template Subtraction (TS) and ε-Normalized Least Mean Squares (ε-NLMS) significantly restored the PR accuracy, although without reaching the same level as for the test set of artifact-free signals.
Motion Tests
The Motion Test was used to determine the effect of the two artifact removal algorithms on real-time myoelectric pattern recognition. Participants were generally able to complete movements when no stimulation was provided (96% completion rate), movements were significantly affected during stimulation (20% completion rate; Wilcoxon rank sum: p < 0.001, Figure 7A). This completion rate was significantly improved by using either TS (56%, p = 0.029) or ε-NLMS (52%, p = 0.041).
Figure 7. Real-time Motion Test outcomes were deteriorated by neurostimulation, but performance was recovered when using artifact removal algorithms. Template Subtraction (TS) resulted in a significantly improved completion rate (A), and ε-Normalized Least Mean Squares (ε-NLMS) significantly reduced the movement completion time (B) and increased the movement accuracy (C). Neither algorithm had a significant effect on selection time (D). Colored whisker plots represent pooled median and interquartile range. Circle and square points represent data from the two participants, and the gray horizontal lines represent their individual medians.
Additional outcome measures from the Motion Test also provide insight into how the artifact removal algorithms affected myoelectric decoding. The completion time describes the amount of time taken to perform the defined movement for 1 s, within a 5 s trial. Completion time showed generally good performance without stimulation (median [IQR]: 1.26 s [1.01, 2.34]), which significantly increased during neurostimulation (5 s [5, 5], p < 0.001, Figure 7B). Completion time was significantly recovered when using TS (4.31 s [1.46, 5], p = 0.017) and ε-NLMS (3.30 s [1.07, 5], p = 0.020), as compared to using raw artifact-corrupted signals. The movement accuracy indicated that the high rate of correct predictions without stimulation (80.0% [44.0%, 96.4%]) similarly diminished with active sensory feedback (0% [0, 0], p < 0.001, Figure 7C). Again similarly, this performance was significantly restored with TS (23.3% [0, 60.8], p = 0.021) and ε-NLMS (25.6% [0, 80.4], p = 0.030), as compared to using raw artifact-corrupted signals. Selection time did not show the same trend; while uncorrupted selection time (0.05 s [0.05, 0.05]) was still significantly hindered by the application of neurostimulation (0.41 s [0.05, 5], p < 0.001), performance was not significantly different when using TS (0.23 s [0.07, 1.93], p = 0.890) or ε-NLMS (0.18 s [0.05, 2.09], p = 0.907, Figure 7D).
Taken together, these results suggest that, for movements that are significantly impacted by neurostimulation artifacts, the use of TS and ε-NLMS artifact removal algorithms can help to restore some of the lost performance by making affected movements easier and quicker to achieve.
Discussion and conclusion
In this paper, we implemented and evaluated two algorithms, TS and ε-NLMS, to remove artifacts caused by neurostimulation used for somatosensory feedback, with the aim to restore control to myoelectric prosthesis users. We evaluated the algorithms’ sensitivity to hyperparameters and their effect on offline and online pattern recognition performance.
Considering the algorithms’ sensitivity to variations in the two hyperparameters α and N, the results suggest that an increase in the learning rate α generally leads to higher variability with increased performance for some semi-synthetic signals and worsened performance for others. The increased variability additionally led to a negative ΔSNR caused by more noise being introduced in the signal than what was removed in terms of artifacts. On the other hand, optimizing the RMSE seemed to provide a good balance between median performance and performance variability, suggesting that the RMSE is a useful optimization metric for automatic selection of hyperparameters.
Neither algorithm was negatively affected by an increase in the length parameter N, but the improvement was constant once the inter-pulse period was reached. Additionally, ε-NLMS required a sufficient filter length (N ⪆ 7) to yield improvements in SNR, while TS provided a consistent improvement starting from N = 1. This indicates that the ε-NLMS adaptive filter is dependent on enough previous samples on the reference channel to be able to predict the artifact waveform correctly. Furthermore, TS seems to require a larger N to reach the same performance as ε-NLMS. Given the two algorithms’ different approaches to predicting the artifact, this behavior is expected. The TS approach, of constructing a template for each of the N samples following the stimulation pulse, naturally requires a large enough N depending on the length of the artifact in the time domain for enough effect. On the other hand, when basing the prediction on the N last samples, as in ε-NLMS, a lower but large enough N is sufficient to adequately predict the artifact.
When comparing the two algorithms in offline analyses, there is no clear evidence supporting that one is significantly better than the other. In the evaluation of the algorithms’ sensitivity to variations in the hyperparameters, both algorithms performed similarly in terms of achieved SNR improvement. ε-NLMS, however, seemed to require more care when selecting parameters since a too high learning rate α quickly degraded the lower quartile performance and a too low filter length N induced more noise in the signal, ultimately causing a negative SNR improvement. In this respect, TS, which had a larger range of values of α that provided sufficient performance and showed consistent performance increase for all values of N, gives the impression of being easier to tune manually. TS is additionally more computationally efficient than ε-NLMS due to its simpler recursive update formulation, but may require a longer template length to achieve the same performance; ε-NLMS, on the other hand, may be more computationally intensive due to dot product calculation performed every sample, but require a smaller filter length.
Regarding the real-time Motion Test outcomes, both TS and ε-NLMS showed improvements to the movement completion rate, completion times, and accuracy, suggesting that either algorithm can be used to recover raw signals corrupted by SAs. However, it should also be noted that these outcomes are dependent on numerous other factors. Our offline investigations suggest that hyperparameter selection can greatly impact the performance of the SAR algorithms, and more so that the optimal hyperparameter values may differ substantially between channels. In the real-time tests, we opted to use the same hyperparameters for all channels, demonstrating a general-use scenario, however, fine tuning and selection of hyperparameters may lead to further improved SAR effectiveness.
As is often the case with prosthetics research, user variability will also drastically impact the need for and benefit of using SAR algorithms. In this study, two participants with transhumeral neuromusculoskeletal prostheses tested our algorithms; despite the fact that the amputation level and technology was similar between our participants, differences in surgical reconstruction, electrode and nerve cuff placement, and prosthesis use meant that different muscles were used to control the same prosthesis movements, and depending on the proximity of the nerve cuffs to each electrode the relative strength of SAs could differ. Thus, stimulation parameters had been selected for each participant to achieve a balance of performance; if SAs were too small, control of the prosthesis would not be affected and SAR algorithms would demonstrate no benefit, and if SAs were too large, the raw signal would be unrecoverable with SAR and the control would remain compromised. The stimulation parameters selected for this study managed this balance, but as a result the outcomes of this study must be interpreted within this basis—outside of this balanced stimulation window, the benefit of these SAR algorithms is reduced.
Related to this limitation is the number of participants included in the study. A major factor influencing the study size is the limited number of individuals using the neuromusculoskeletal prosthesis in daily life. Not all users in this cohort experience SAs which compromise the control of their prosthesis. As mentioned previously, all users have differences in limb presentation and surgical intervention which affect the presence and severity of SAs. Thus, the approach taken in this study is one of patient-centered design, in which the participant’s prosthetic system was customized to their unique limb presentation and needs: stimulation intensity was set such that the sensation could be felt reliably during daily activities, and SAR algorithms were tuned such SAs could be removed without compromising control of the prosthetic hand. This approach has yielded a system which benefits our participants, however, the translation of this solution to other users will depend on the limb presentation, surgical reconstruction, selection of implanted devices, and other factors.
The TS and ε-NLMS algorithms are both well suited to learn and counteract static artifacts, which makes it particularly suited for stimulation paradigms such as discrete event-driven sensory feedback (Cipriani et al., 2014), which have consistent and repeatable stimulation patterns. In particular, the TS assumptions of a stable artifact profile may make it the preferred choice for this application due to its lower computational demands. However, most sensory feedback research sets the stimulation intensity proportional to the feedback measurement (typically grip force). Both TS and ε-NLMS can adapt to changing stimulation artifacts, however they will always lag behind a proportional feedback scheme—the stimulation template for TS will have a maximum rate of change limited by the learning rate, and linear correlation between reference and signal may not be constant across stimulation intensities. While these algorithms may still reduce the SA, they may under- or overcorrect and continue to leave residual artifacts in the signals. We expect that ε-NLMS would outperform TS in this regard, as it uses the stimulation signal as a reference for its artifact removal, however, SAR methods which take the current stimulation parameters into account may be able to circumvent this issue entirely, which is an area we plan to investigate in a future study.
Overall, when possible, it is best for implanted electrodes to be configured in such a way as to minimize the potential for SAs. Evaluation methods such as cross-channel impedance measurement may be used to identify configurations with a higher likelihood for SAs (Earley et al., 2022b), and the use of biopolar electrodes may help to reduce the likelihood of cross-talk. However, monopolar configurations can allow for a greater number of unique muscle sources for a given number of electrodes, which is of particular importance for implanted electrodes. In these cases, SAR algorithms may be able to make up the difference and improve the control of prosthetic limbs while simultaneously permitting sensory feedback, allowing for bidirectional and closed-loop control of prostheses which serve to improve the independence and quality of life of people with amputations.
Data availability statement
The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found below: https://osf.io/y473e/.
Ethics statement
The studies involving human participants were reviewed and approved by the Swedish Regional Ethical Committee, Gothenburg, Sweden (DNR: 769-12). The patients/participants provided their written informed consent to participate in this study.
Author contributions
AB, EE, and JZ developed and implemented the algorithms and ran the experiments. EE and AB developed the experimental protocols, analyzed the data, prepared the figures, and drafted the manuscript. MO-C envisioned the study and obtained funding for this project. All authors read and approved the final manuscript.
Funding
This work was supported by the Promobilia Foundation, the IngaBritt and Arne Lundbergs Foundation and the Swedish Research Council (Vetenskapsrådet).
Acknowledgments
We would like to acknowledge and thank our participants for their contributions and feedback to this study.
Conflict of interest
Author MO-C has consulted for Integrum AB.
The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Advanced Amputee Solutions LLC, (2012). Amputee statistics you ought to know. Available online at: AdvancedAmputees.com (accessed July 11, 2022).
Azin, M., Chiel, H. J., and Mohseni, P. (2007). “Comparisons of FIR and IIR implementations of a subtraction-based stimulus artifact rejection algorithm,” in Proceedings of the annual international conference of the IEEE engineering in medicine and biology, Lyon, 1437–1440. doi: 10.1109/IEMBS.2007.4352570
Basir-Kazeruni, S., Vlaski, S., Salami, H., Sayed, A. H., and Marković, D. (2017). “A blind adaptive stimulation artifact rejection (ASAR) engine for closed-loop implantable neuromodulation systems,” in Proceedings of the international IEEE/EMBS conference on neural engineering, NER, Shanghai, 186–189. doi: 10.1109/NER.2017.8008322
Cipriani, C., Segil, J. L., Clemente, F., ff Weir, R. F., and Edin, B. (2014). Humans can integrate feedback of discrete events in their sensorimotor control of a robotic hand. Exp. Brain Res. 232, 3421–3429. doi: 10.1007/s00221-014-4024-8
De Clercq, W., Vergult, A., Vanrumste, B., Van Paesschen, W., and Van Huffel, S. (2006). Canonical correlation analysis applied to remove muscle artifacts from the electroencephalogram. IEEE Trans. Biomed. Eng. 53, 2583–2587. doi: 10.1109/TBME.2006.879459
Earley, E. J., Berneving, A., and Ortiz-Catalan, M. (2022a). Neurostimulation artifact removal for implantable sensors improves signal clarity and decoding of motor volition. Open Science Framework. Available online at: https://osf.io/y473e/ (accessed August 28, 2022).
Earley, E. J., Mastinu, E., and Ortiz-Catalan, M. (2022b). “Cross-channel impedance measurement for monitoring implanted electrodes,” in Proceedings of the 2022, 44th annual international conference of the IEEE engineering in medicine and biology society (EMBC), Glasgow. doi: 10.1109/EMBC48229.2022.9871954
Englehart, K., and Hudgins, B. (2003). A robust, real-time control scheme for multifunction myoelectric control. IEEE Trans Biomed Eng 50, 848–854. doi: 10.1109/TBME.2003.813539
Günter, C., Delbeke, J., and Ortiz-Catalan, M. (2019). Safety of long-term electrical peripheral nerve stimulation: Review of the state of the art. J. NeuroEng. Rehabil. 16:16. doi: 10.1186/s12984-018-0474-8
Hartmann, C., Došen, S., Amsuess, S., and Farina, D. (2015). Closed-loop control of myoelectric prostheses with electrotactile feedback: Influence of stimulation artifact and blanking. IEEE Trans. Neural Syst. Rehabil. Eng. 23, 807–816. doi: 10.1109/TNSRE.2014.2357175
Hudgins, B., Parker, P., and Scott, R. N. (1993). A new strategy for multifunction myoelectric control. IEEE Trans. Biomed. Eng. 40, 82–94. doi: 10.1109/10.204774
Jafarifarmand, A., and Badamchizadeh, M. A. (2019). EEG artifacts handling in a real practical brain–computer interface controlled vehicle. IEEE Trans. on Neural Syst. Rehabil. Eng. 27, 1200–1208. doi: 10.1109/TNSRE.2019.2915801
Keller, T., and Popovic, M. R. (2001). “Real-time stimulation artifact removal in EMG signals for neuroprosthesis control applications,” in Proceedings of the 6th annual IFESS conference, (January 2014), Cleveland, OH, 4–6.
Kuiken, T. A., Dumanian, G. A., Lipschutz, R. D., Miller, L. A., and Stubblefield, K. A. (2004). The use of targeted muscle reinnervation for improved myoelectric prosthesis control in a bilateral shoulder disarticulation amputee. Prosthet. Orthot. Int. 28, 245–253. doi: 10.3109/03093640409167756
Kuiken, T. A., Li, G., Lock, B. A., Lipschutz, R. D., Miller, L. A., Stubblefield, K. A., et al. (2009). Targeted muscle reinnervation for real-time myoelectric control of multifunction artificial arms. JAMA 301, 619–628. doi: 10.1016/S0276-1092(09)79632-4
Li, Y., Chen, J., and Yang, Y. (2019). A method for suppressing electrical stimulation artifacts from electromyography. Int. J. Neural Syst. 29:1850054. doi: 10.1142/S0129065718500545
Liang, H., and Lin, Z. (2002). Stimulus artifact cancellation in the serosal recordings of gastric myoelectric activity using wavelet transform. IEEE Trans. Biomed. Eng. 49, 681–688. doi: 10.1109/TBME.2002.1010851
Marque, C., Bisch, C., Dantas, R., Elayoubi, S., Brosse, V., and Pérot, C. (2005). Adaptive filtering for ECG rejection from surface EMG recordings. J. Electromyogr. Kinesiol. 15, 310–315. doi: 10.1016/j.jelekin.2004.10.001
Mastinu, E., Brånemark, R., Aszmann, O., and Ortiz-Catalan, M. (2018). “Myoelectric signals and pattern recognition from implanted electrodes in two TMR subjects with an osseointegrated communication interface,” in Proceedings of the annual international conference of the IEEE engineering in medicine and biology society, (EMBS), Honolulu, HI, 5174–5177. doi: 10.1109/EMBC.2018.8513466
Mastinu, E., Doguet, P., Botquin, Y., Hakansson, B., and Ortiz-Catalan, M. (2017). Embedded system for prosthetic control using implanted neuromuscular interfaces accessed via an osseointegrated implant. IEEE Trans. Biomed. Circuits Syst. 11, 867–877. doi: 10.1109/TBCAS.2017.2694710
Nolan, H., Whelan, R., and Reilly, R. B. (2010). FASTER: Fully automated statistical thresholding for EEG artifact rejection. J. Neurosci. Methods 192, 152–162. doi: 10.1016/j.jneumeth.2010.07.015
O’Keeffe, D. T., Lyons, G. M., Donnelly, A. E., and Byrne, C. A. (2001). Stimulus artifact removal using a software-based two-stage peak detection algorithm. J. Neurosci. Methods 109, 137–145. doi: 10.1016/S0165-0270(01)00407-1
Ortiz-Catalan, M., Brånemark, R., and Håkansson, B. (2013). BioPatRec: A modular research platform for the control of artificial limbs based on pattern recognition algorithms. Source Code Biol. Med. 8:11. doi: 10.1186/1751-0473-8-11
Ortiz-Catalan, M., Håkansson, B., and Brånemark, R. (2014). An osseointegrated human-machine gateway for long-term sensory feedback and motor control of artificial limbs. Sci. Transl. Med. 6:257re6. doi: 10.1126/scitranslmed.3008933
Ortiz-Catalan, M., Mastinu, E., Sassu, P., Aszmann, O., and Brånemark, R. (2020b). Self-contained neuromusculoskeletal arm prostheses. N. Engl. J. Med. 382, 1732–1738. doi: 10.1056/NEJMoa1917537
Ortiz-Catalan, M., Mastinu, E., Greenspon, C. M., and Bensmaia, S. J. (2020a). Chronic use of a sensitized bionic hand does not remap the sense of touch. Cell Rep. 33:108539. doi: 10.1016/j.celrep.2020.108539
Sreeja, S. R., Sahay, R. R., Samanta, D., and Mitra, P. (2018). Removal of eye blink artifacts from EEG signals using sparsity. IEEE J. Biomed. Health Inf. 22, 1362–1372. doi: 10.1109/JBHI.2017.2771783
Waddell, C., Pratt, J. A., Porr, B., and Ewing, S. (2009). “Deep brain stimulation artifact removal through under-sampling and cubic-spline interpolation,” in Proceedings of the 2009 2nd international congress on image and signal processing, CISP’09, Tianjin. doi: 10.1109/CISP.2009.5301199
Zheng, M., Crouch, M., and Eggleston, M. S. (2021). Surface electromyography as a natural human-machine interface: A review. IEEE Sensors J. 22, 9198–9214.
Zhou, A., Johnson, B. C., and Muller, R. (2018). Toward true closed-loop neuromodulation: Artifact-free recording during stimulation. Curr. Opin. Neurobiol. 50, 119–127. doi: 10.1016/j.conb.2018.01.012
Zhou, P., Lock, B., and Kuiken, T. A. (2007). Real time ECG artifact removal for myoelectric prosthesis control, physiological measurement. IOP Publ. 28, 397–413. doi: 10.1088/0967-3334/28/4/006
Keywords: neurostimulation, artifact removal, implantable electrodes, prosthesis control, osseointegration, myoelectric pattern recognition, sensory feedback
Citation: Earley EJ, Berneving A, Zbinden J and Ortiz-Catalan M (2022) Neurostimulation artifact removal for implantable sensors improves signal clarity and decoding of motor volition. Front. Hum. Neurosci. 16:1030207. doi: 10.3389/fnhum.2022.1030207
Received: 28 August 2022; Accepted: 27 September 2022;
Published: 19 October 2022.
Edited by:
Jules Anh Tuan Nguyen, University of Minnesota Twin Cities, United StatesReviewed by:
Josefina Gutierrez, National Institute of Rehabilitation Luis Guillermo Ibarra Ibarra, MexicoDavid J. Warren, The University of Utah, United States
Copyright © 2022 Earley, Berneving, Zbinden and Ortiz-Catalan. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Max Ortiz-Catalan, bWF4b0BjaGFsbWVycy5zZQ==
†These authors have contributed equally to this work and share first authorship