A hybrid sensory feedback system for thermal nociceptive warning and protection in prosthetic hand

Background: Advanced prosthetic hands may embed nanosensors and microelectronics in their cosmetic skin. Heat influx may cause damage to these delicate structures. Protecting the integrity of the prosthetic hand becomes critical and necessary to ensure sustainable function. This study aims to mimic the sensorimotor control strategy of the human hand in perceiving nociceptive stimuli and triggering self-protective mechanisms and to investigate how similar neuromorphic mechanisms implemented in prosthetic hand can allow amputees to both volitionally release a hot object upon a nociceptive warning and achieve reinforced release via a bionic withdrawal reflex.

Methods: A steady-state temperature prediction algorithm was proposed to shorten the long response time of a thermosensitive temperature sensor. A hybrid sensory strategy for transmitting force and a nociceptive temperature warning using transcutaneous electrical nerve stimulation based on evoked tactile sensations was designed to reconstruct the nociceptive sensory loop for amputees. A bionic withdrawal reflex using neuromorphic muscle control technology was used so that the prosthetic hand reflexively opened when a harmful temperature was detected. Four able-bodied subjects and two forearm amputees randomly grasped a tube at the different temperatures based on these strategies.

Results: The average prediction error of temperature prediction algorithm was 8.30 ± 6.00%. The average success rate of six subjects in perceiving force and nociceptive temperature warnings was 86.90 and 94.30%, respectively. Under the reinforcement control mode in Test 2, the median reaction time of all subjects was 1.39 s, which was significantly faster than the median reaction time of 1.93 s in Test 1, in which two able-bodied subjects and two amputees participated. Results demonstrated the effectiveness of the integration of nociceptive sensory strategy and withdrawal reflex control strategy in a closed loop and also showed that amputees restored the warning of nociceptive sensation while also being able to withdraw from thermal danger through both voluntary and reflexive protection.

Conclusion: This study demonstrated that it is feasible to restore the sensorimotor ability of amputees to warn and react against thermal nociceptive stimuli. Results further showed that the voluntary release and withdrawal reflex can work together to reinforce heat protection. Nevertheless, fusing voluntary and reflex functions for prosthetic performance in activities of daily living awaits a more cogent strategy in sensorimotor control.

1 Introduction

The human sensory system is equipped with rich receptors at fingertips and vast neural circuits in the spinal cord and brain for perceiving the external environment (Johansson and Westling, 1984; Johansson and Flanagan, 2008). Nociceptors serve to protect the skin from noxious thermal stimuli by eliciting pain sensations. The nociceptive afferents inform the brain of potentially hazardous objects and initiate protective motor actions by quickly retracting the hand from the heat source, a behavior known as the withdrawal reflex (Henrich et al., 2021), resulting from the seamless sensorimotor pathways of nociceptive temperatures (Witney et al., 2004; Muniak et al., 2007; Sieck, 2019). Advanced prosthetic hands are emerging with artificial skin embedded with integrated nanosensors (Yang et al., 2019; Romeo et al., 2021; Li et al., 2022). Some flexible sensors made of materials such as hydrogel, textile, and paper could be susceptible to heat damage (Islam et al., 2020; Pinelli et al., 2020; Tai et al., 2020; Yang et al., 2021; Hardman et al., 2022; Liu et al., 2022). It is important to protect the integrity of the prosthetic hand for sustainable function since amputees may have reduced or no sensation in the residual limb, rendering them unable to perceive nociceptive temperatures or make the necessary motor adjustments for protection.

Recent research in providing temperature perception and protection in prosthetic hands has advanced significantly. Utilizing Peltier sensors, researchers have provided feedback on various temperature levels (range from 15°C to 40°C) to subjects, facilitating the perception of heat through thermal stimulation (Ueda and Ishii, 2016; Moqadam et al., 2021; Iberite et al., 2023). However, it is not clear how a nociceptive warning can be achieved. Moreover, the advent of novel sensors embedded in electronic skins has enabled prosthetic hands or robot to rapidly detect a nociceptive stimulus and to evoke protective withdrawal reflexes (Chen et al., 2022; Liu et al., 2022; Neto et al., 2022; Shi et al., 2023; Wang et al., 2023). Yet, these automatic control strategies do not involve the sensory loop with the human nociception. It remains a challenge not only to warn against harmful temperatures but also take withdraw actions via reflexive, voluntary, or hybrid control. There are two modes in which voluntary action and withdrawal reflex can work together in the human hand (Lynn et al., 2016; Henrich et al., 2021). The reinforcement control mode demonstrates the subjects’ motor intention to rush away from danger, while the contradictory mode indicates the subjects’ ability to inhibit the action of withdrawal reflex and represents that the subjects are able to tolerate this danger after careful consideration. These two control modes are both essential in practical operation in daily life. In this paper, we will focus only on the feasibility of individual control modes and a simple interactive control in which the neuromorphic withdrawal reflex reinforces voluntary release upon a nociceptive warning. Subject’s intention to inhibit the withdrawal reflex action requires careful consideration of more cognitive and behavioral factors (Ghosh et al., 2014; Lynn et al., 2016). Thus, interactive control where voluntary control may overrule the withdrawal reflex will not be investigated in this study. The objective here is to develop and verify the sensorimotor circuits for nociceptive warning and protection in prosthetic hands.

Restoring sensorimotor ability for amputees via invasive or non-invasive approaches has become an emerging technology (Bensmaia et al., 2020; Lan et al., 2023). In invasive sensory feedback technologies, intraneural electrodes including the longitudinal intrafascicular electrode (LIFE) (Dhillon et al., 2004), transverse intrafascicular multichannel electrode (TIME) (Raspopovic et al., 2014), Utah slanted electrode array (USEA) (George et al., 2020), and flat interface nerve electrode (FINE) (Tan et al., 2014) not only enable the perception of contact forces but also help amputees identify object properties, such as size, compliance, (George et al., 2020), texture (Oddo et al., 2016), and shape (D’Anna et al., 2017). Results also found that these approaches improved the amputees’ sense of embodiment, and self-confidence in manipulations of daily life (Graczyk et al., 2018; Cuberovic et al., 2019). In non-invasive technologies, alternative feedback methods based on electrotactile or mechanotactile stimulation can establish a distinct feedback mapping pathway through training and learning (Saunders and Vijayakumar, 2011; Dosen et al., 2017; Aboseria et al., 2018; Markovic et al., 2018). These alternative feedback approaches have been applied to restore the identification of objects (Akhtar et al., 2018; Dideriksen et al., 2020) and grasp control (Dosen et al., 2015; Schweisfurth et al., 2016). Compared with invasive somatotopic sensory feedback approaches, these alternative approaches showed less robustness and high cognitive process in the manipulation tasks (Valle et al., 2020). Transcutaneous electrical nerve stimulation (TENS) on the skin surface can induce somatotopic sensations that do not require complicated implant surgery or additional training. Using TENS based on evoked tactile sensations (ETS), amputees were able to stably perceive information such as forces and apertures (Chai et al., 2015; Zhang et al., 2022b). The finger-specific sensations have become a promising approach for amputees to provide a sensory foundation for noxious temperatures (Hao et al., 2020). In this study, we applied TENS to restore the perception of forces and nociception.

There were four specific aims in this study. First, a rapid algorithm was validated to predict the steady-state temperature from the slow response of a thermosensitive sensor to establish further thermal warning action timely. Second, a hybrid sensory coding strategy was designed to allow the delivery of prosthetic contact forces and nociceptive heat warnings via TENS to evoke dangerous awareness and voluntary protection. Third, a biomimetic withdrawal reflex control strategy was implemented on a tendon-driven prosthetic hand with antagonistic neuromorphic muscle models to reinforce voluntary release. Finally, the closed-loop system was tested and verified in able-bodied and amputee subjects to demonstrate the feasibility of providing nociceptive warning and protection in both a voluntary and a reinforcing manner.

2 Methods and materials

2.1 Subjects

A total of six subjects were recruited, including four able-bodied individuals (H1–H4, 2 males and 2 females, 26 ± 2.0 years old) and two male forearm amputees with intact projected finger maps as shown in Figure 1A (A1–A2, 62 ± 2.0 years old). The study was approved by the Ethical Committee of Human and Animal Studies at Shanghai Jiao Tong University (E2020021I), and all subjects were informed of the experimental procedures and signed an informed consent form before the experiments. Two able-bodied subjects (H1–H2) and two amputees participated in Test 1 and all six subjects participated in Test 2 due to variation.

Figure 1

Figure 1. Experimental protocol and workflow diagram. (A) The antagonistic tendon-driven prosthetic hand was anchored to the table. Subjects activated their sEMG signals to grasp a tube containing water of different temperatures after receiving the instructions from the experimenter. (B) Test flowchart showing subject responded differently based on perception.

2.2 Experimental setup

The experimental setup was illustrated in Figure 1A: (1) an electrical stimulator and a sEMG acquisition system, (2) an antagonistic tendon-driven prosthetic hand, (3) a brass tube, (4) force sensors (Flex Force A201, Tekscan, United States), (5) a thermosensitive temperature sensor (LMT70YFQT, Texas Instruments, United States), (6) customized sEMG electrodes, (7) customized stimulation electrodes, (8) neuromorphic muscle model chips (Spartan-6, Xilinx, United States), and (9) a trigger button.

A tendon-driven prosthetic hand (2) controlled by a pair of antagonistic muscle models was used in this study (Xie et al., 2022), which was modified from a 3D-printed open-source hand (InMoov, 2012, France). This prosthetic hand was fixed to a table, and a highly thermally conductive brass tube (50 mm in diameter, 1 mm in thickness, 250 mm in height) was in front of it (3). A temperature sensor (5) was placed on the index finger, and two force sensors (4) were placed on the index and thumb fingers. Two steel cables with a radius of 0.5 mm were passed through and fixed to the medial and lateral sides of a finger to mimic the finger extensor and flexor. Each cable was connected to a linear motor (LAF 30, Inspire Robotics Inc., Beijing, China), and two linear motors drive a finger through the position-force proportional and integral control in extension and flexion motions within a single degree of freedom. Two motors decoded the target muscle forces corresponding to the antagonistic muscle models. The maximum grip force in a single finger was 15 N. The antagonistic muscles were implemented by two Hill muscle models, respectively, (Shadmehr and Wise, 2005; Mileusnic et al., 2006). The muscle force was generated by the real-time computational model via a neuromorphic chip (8) (Niu et al., 2014, 2017). Muscle contraction force was mainly calculated by alpha motor commands based on a force-alpha motor command relationship (Cheng et al., 2000; Mileusnic et al., 2006; Song et al., 2008). Subjects used their sEMG signals to grasp the tube with a pinch grip as shown in Figure 1B. Electrical stimulator and sEMG acquisition system (1) acquired sEMG signals and implemented a multi-channel, programmable TENS sensory stimulation function. Since sEMG signals and electrical stimulation signals may interfere with each other, we used hardware blanking and software digital filtering to eliminate the artifacts, and two channels of sEMG signals (6) were sampled in 2 kHz (Yu et al., 2022). This system also acquired signals from two channels of force sensors, one channel of temperature sensor, and one channel of trigger button, and converted the corresponding force and temperature signals into multi-channel stimulation pulses, which were transmitted to the projected finger map area of the amputee through customized Ag/AgCl stimulation electrodes (7) to evoke tactile sensations (Zhang et al., 2022b). Stimulation currents were biphasic, charge-balanced, cathode-first pulse trains with a 10 μs interval between pulses. Force and temperature signals were collected with a sampling frequency of 100 Hz and a trigger button (9) was used to mark the perceptual events. In this study, normalized sEMG was used to represent the alpha motor command, with the sEMG normalized to 0 and 1 when the subject was resting and exerting maximum voluntary contraction force, respectively. More detailed information was published in previous research (Xie et al., 2022; Zhang et al., 2022b, Zhang Z. et al., 2022c).

2.3 Steady-state temperature prediction algorithm

Temperature sensors exhibit various slopes of voltage output at different temperatures. Typically, the thermosensitive temperature sensor we utilized in this study requires a response time of 6 s to detect a steady-state temperature as shown in Figure 2A. Such a long response time does not allow for the timely detection of nociceptive temperatures that could pose a danger. To shorten the response time of the temperature sensor and provide timely heat warnings, we developed a steady-state temperature prediction algorithm based on the voltage slope of the sensor. The temperature sensor was calibrated by placing it on a thermostatic heating plate (DB-XAB, Lichen Bangxi Instrument Inc., China) for 10 s to establish the relationship between output voltage and constant temperature, ranging from 30.0°C to 80.0°C with a step of 5.0°C. Ten repeat measurements were taken for each temperature.

Figure 2

Figure 2. Accuracy of steady-state temperature prediction algorithm. (A) Response curves of the temperature sensor measuring thermostatic plate at different temperatures. (B) Prediction algorithm diagram at 45.0°C based on four identification times. (C) The k − ΔT relationship curve. (D) The errors of the prediction algorithm correspond to four identification times with *p < 0.05.

The temperature calibration and tests were carried out at room temperature, and the initial temperature was set at 20.0°C. We recorded the temperature at initial contact ($T_{\text{initial}\text{state}}$) and steady-state after contact for four fixed sliding times ($T_{\text{steady}\text{state}}$). Then, a Savitzky–Golay filter was used to calculate the slope ($k_{\text{initial}\text{state}}$) of sensor voltage at $T_{\text{initial}\text{state}}$ for a fixed sliding time (Kordestani and Zhang, 2020), which was defined as the identification time. We chose four identification times including 0.35 s, 0.50 s, 0.75 s, and 1.00 s. Based on the relationship of the temperature difference ($\mathrm{\Delta }T$) between the initial-state and steady-state, and the initial slope ($k_{\text{initial}\text{state}}$) using Eq. 1, a linear relationship could be used to predict the steady-state temperature as shown in Eq. 2, where $a$ and $b$ coefficients were obtained by linear fitting of initial slope and temperature difference under four sliding time lengths. Prediction error was defined as the percentage of absolute error between the actual temperature set on the thermostatic plate and the temperature predicted by the algorithm. In this study, the nociceptive temperature threshold ideally considered both amputee safety and prosthetic material limitations and could be set based on the lower of the two limits. Building upon the understanding that the TRPV2 ion channel in human nociceptors triggers pain sensations at 52.0°C (Wang et al., 2023), this study set a hypothetical noxious temperature warning threshold at 60.0°C as a proof of concept.

$\begin{array}{ll}\mathrm{\Delta }T=a{k}_{\mathit{\text{initial}}\mathit{\text{state}}}+b& (1)\end{array}$

$\begin{array}{ll}{T}_{\mathit{\text{steady}}\mathit{\text{state}}}=\mathrm{\Delta }T+T_{\mathit{\text{initial}}\mathit{\text{state}}}& (2)\end{array}$

2.4 Hybrid sensory strategy

Previous research has verified that ETS elicited by TENS restored digit-specific sensations for amputees via the direct neural pathway (Hao et al., 2020). The stability of the ETS location and area, as well as the sensory thresholds for each sensation were also verified (Chai et al., 2015). Tingling sensation was evoked by high frequency or amplitude and was described as a tolerable tingling sensation but no pain and no muscle contraction (Amer-Cuenca et al., 2011; Johnson et al., 2015). The characterization of tingling could be used to encode the nociceptive warning to indicate that prosthetic hand was touching something hot. Buzz sensation was described as a strong and continuous high frequency vibration and has been shown a wide pulse width modulation range and high sensitivity (Chai et al., 2015; George et al., 2019; Zhang et al., 2022a), it was chosen to encode force information.

We identified our participants’ perceived locations of their fingers on their residual limb using a 2 mm diameter metal pen with a rounded tip and placed stimulating electrodes over these areas. Stimulation frequency was set at 50 Hz. Amplitude and pulse width ranges were customized for each subject based on the rapid assessment strategy developed in our previous study (Yang et al., 2020; Zhang et al., 2022b). In this protocol, the amplitude was increased from an initial value of 1 mA in 0.5 mA step until the buzz or tingling sensations were first clearly perceived. Under the determined amplitude, the pulse width was initially set at 20 μs and gradually increased in 20 μs step. The minimum pulse width was defined as the sensory threshold when the buzz or tingling modality was first clearly perceived ($W_{\text{buzz}:\min }$,$W_{\text{tingling}:\min }$), and the maximum pulse width was defined as the value at which the sensory modality began to change ($W_{\text{buzz}:\max }$). Each electrical stimulus used to assess the sensory threshold had a duration of 3 s, with a rest period of 5 s before the next stimulus. The stimulation parameters of all subjects for these two sensory modalities were summarized in Supplementary Table S1. The maximum ($F_{\text{max}}$) and minimum grip force ($F_{\text{min}}$) generated by subjects when grasping the tube were tested and mapped to pulse width using Eq. 3. To reduce discomfort for the subjects, a constant pulse width was encoded using the sensory threshold at which the tingling sensation first appeared, as shown in Eq. 4. When the temperature sensor did not detect a nociceptive temperature, the stimulation of the index and thumb stimulation sites was a buzz modality. As soon as a harmful temperature was detected, stimulation parameters were immediately switched to the tingling modality in the index finger stimulation site, while stimulation of the thumb finger was paused.

$W=\{\begin{array}{c}\hfill 0\mathit{\text{if}}F<F_{\text{min}}\hfill \\ \hfill \frac{W_{\text{buzz}:\max }-W_{\text{buzz}:\min }}{F_{\text{max}}-F_{\text{min}}}\ast \left(F-F_{\text{min}}\right)+W_{\text{buzz}:\min }\text{if}{F}_{\text{min}}\le F<F_{\text{max}}\hfill \\ \hfill W_{\text{buzz}:\max }\text{if}F\ge F_{\text{max}}\hfill \end{array}\text{ (3)}$

$\begin{array}{ll}W=W_{\text{tingling}:\min }\text{if}F\ge 0.1N\text{and}T\ge {60.0}^{°}\mathrm{C}& (4)\end{array}$

2.5 Withdrawal reflex control strategy

Neuromorphic muscle control technology mimics the biomechanical properties of the human hand through calculations involving motor neurons and muscle models. Experimental evidence has demonstrated that this method has significantly superior control results compared to traditional proportional control methods (Niu et al., 2021; Luo et al., 2021a,b). To reconstruct the control loop of nociceptive temperature, we designed a neuromorphic withdrawal reflex circuit.

In scenarios where the detected temperature was below 60.0°C, the bionic withdrawal reflex was not triggered, and no reflex compensation was superimposed on the alpha motor commands. Once the prosthetic hand grasped an object ($F\ge 0.1N$) and the temperature sensor predicted a steady state temperature exceeding 60.0°C, the withdrawal reflex circuit was activated. The voltage output of the temperature sensor was converted into the presynaptic current ($I_{\mathrm{n}\mathrm{c}\mathrm{p}}$) to excite the interneuron. We utilized the Izhikevich neuron model to simulate the interneuron, calculating a binary spike train, membrane potential ($V_{\text{itn}}$), membrane recovery variable ($U_{\text{itn}}$) induced by a nociceptive temperature (Izhikevich, 2003), as shown in Eqs. 5, 6.

$\{\begin{array}{c}\hfill {V_{\text{itn}}}^{\prime }=0.04{V_{\text{itn}}}^2+5V_{\text{itn}}+140-U_{\text{itn}}+I_{\mathrm{n}\mathrm{c}\mathrm{p}}\hfill \\ \hfill {U_{\text{itn}}}^{\prime }=a\left(b{V}_{\text{itn}}-U_{\text{itn}}\right)\hfill \end{array}\text{ (5)}$

$\text{if}{V}_{\text{itn}}\ge 30\text{mV},\text{then}\{\begin{array}{c}\hfill V_{\text{itn}}\leftarrow c\hfill \\ \hfill U_{\text{itn}}\leftarrow U_{\text{itn}}+d\hfill \end{array}\text{ (6)}$

Upon reaching the firing threshold, the excitatory interneuron generated the postsynaptic current ($I_{\text{itn}}$). We adjusted the rise and fall time parameters of the synaptic model (${\tau }_r=0.08s,{\tau }_f=0.19s,V_m=5.2\mathit{\text{mV}}$) (Glowatzki and Fuchs, 2002; Niu et al., 2017), as delineated by Eq. 7. The excitatory postsynaptic current also activated an inhibitory interneuron, which generated an inhibitory postsynaptic current (Li et al., 2015). The excitatory postsynaptic current served as reflex compensation to the extensor alpha motor command, while the inhibitory postsynaptic current was directed to the flexor alpha motor command. Consequently, the biomimetic withdrawal reflex response was manifested by a reduction in flexor activity preventing further grasping, and an increase in extensor activity facilitating a rapid release of the grip. These synaptic computations were realized through filtering by a host computer. To validate the consistency between the postsynaptic current generated by the filter and the synaptic model, a constant current was input into the interneuron model (Figure 3A). After the interneuron model consistently produces the membrane potential and a binary spike train as shown in Figures 3B,C, the synaptic model begins to compute postsynaptic current as shown in Figure 3D. If the interneuron is not activated or is not activated enough to produce a binary spike train, the synaptic current will be 0.

$I_{\text{itn}}\left(t\right)=\{\begin{array}{c}\hfill V_{\mathrm{m}}\left(e^{-\frac{t}{{\tau }_{\mathrm{f}}{V}_{\mathrm{m}}}}-e^{-\frac{t}{{\tau }_{\mathrm{r}}{V}_{\mathrm{m}}}}\right),\text{if}t\ge 0\hfill \\ \hfill 0,\text{otherwise}\hfill \end{array}\text{ (7)}$

Figure 3

Figure 3. A constant current input from interneuron layer generated a series of spike to synapse layer including model and filter outputs. The parameters of interneuron model were as following: a = 0.1 ms⁻¹, b = 0.2 ms⁻¹, c = −65 mV, d = 2 mV/ms.

2.6 Experimental protocols

We first validated the feasibility of the neuromorphic withdrawal reflex control strategy without subject participation. Based on the validation of reflexive heat protection, Test 1 and Test 2 with the subjects’ participation were designed to investigate how similar neuromorphic mechanisms implemented in prosthetic hands could allow amputees to release the hot object upon a nociceptive warning in voluntary and reinforcement mode. Subjects were not informed of the experimental conditions during the two tests.

The validation of the neuromorphic withdrawal reflex in the prosthetic hand under no-load condition was executed by a host computer that directly issued constant alpha motor commands (α = 0.7 for the flexor, α = 0.2 for the extensor). Building upon the verification of temperature prediction accuracy, we tested the prosthetic hand grasping a tube at two temperatures: one above the warning threshold at 65.0°C and another below the threshold at 30.0°C, each repeated 40 times. If the predicted temperature exceeded the warning threshold, the biomimetic withdrawal reflex circuit was triggered. Interneurons and synapses calculated the excitatory postsynaptic current, which was then superimposed onto the extensor alpha motor neuron as reflex compensation. Conversely, a negative reflex compensation value decreased the flexor alpha motor command.

Test 1 aimed to validate the feasibility of providing nociceptive warning and voluntary protection through a psychophysical experiment (Osborn et al., 2018; Valle et al., 2018). Water at nine different temperatures was placed inside the tube (30.0°C, 40.0°C, 50.0°C, 55.0°C, 57.5°C, 62.5°C, 65.0°C, 70.0°C, and 80.0°C). Each temperature was tested in ten trials and randomly assigned during the test. Subjects used sEMG signals to grasp the tube under instruction. Subjects’ auditory and visual senses were occluded with earmuffs and eye masks. The experimenter signaled the start and end of each trial by tapping the subject’s shoulder. A buzz sensation indicated the water was below the warning threshold, prompting subjects to maintain their grasp for 5 s until a subsequent shoulder tap. A tingling sensation indicated water above 60.0°C, signifying potentially harmful heat and necessitating an immediate release of the grasp by the prosthetic hand, while the subjects were asked to press a button with their contralateral hand to record perception time as shown in Figure 1B. Data were provided by only two able-bodied individuals (H1, H2) and two amputees (A1, A2) due to data collection constraints.

The purpose of Test 2 was to test the integration of the withdrawal reflex strategy with the hybrid sensory strategy and to validate the feasibility of providing nociceptive warning and protection in a reinforcement way. The setup and requirements for Test 2 were identical to those of Test 1. Similarly, the test involved two temperatures set at 65.0°C and 30.0°C, with each temperature undergoing ten repeated trials. Once the predicted temperature exceeded the warning threshold, the biomimetic withdrawal reflex circuit was activated, switching the stimulation modality to the nociceptive temperature warning. To confirm the reinforcement of voluntary release and withdrawal reflex, subjects were instructed to voluntarily open the prosthetic hand and press a button, aligning their actions with the reflex response upon perceiving the tingling sensation. If a tube heated at 30.0°C was grasped, subjects were asked to maintain the grasp for 5 s upon perceiving the buzz sensation. All participants took part in Test 2.

2.7 Data processing and statistical analysis

All control programs were written in C# language and ran on the Visual Studio 2019 platform. Data processing was completed using Visual Studio Code 2020 in Python. In Test 1, the probability of perceiving the tingling sensation after ten trials was statistically computed for each temperature setting. A sigmoid function was used to fit the quantitative relationship between the tested temperatures and the probabilities of nociceptive temperature perception, revealing subjects’ ability to perceive nociceptive temperature warnings via a hybrid sensory strategy. The 50% probability point of the fitted curve, known as the point of subjective equality (PSE) (Valle et al., 2018), represented the subjective nociceptive temperature warning threshold perceived through the sensory strategy. We defined the following performance indices to measure the subjects’ perception of nociceptive temperature warnings and the speed of their response. The time required for sliding window of the temperature prediction algorithm to output a response was defined as the identification time (IT). Action time (AT) was defined as the time interval from grasping the tube to the change in alpha motor commands. Perception time (PT) was determined as the time interval from when the steady-state temperature was predicted to when the trigger button was pressed. Execution time (ET) was defined as the time interval from when the heat warning temperature was predicted to when the index fingertip force disappeared. Reaction time (RT) was the total time from grasp to release and was equal to the sum of the IT and ET. With reference to Figure 2D, the boxes indicated the interquartile range (IQR), the line within the boxes depicted the median, and the whiskers represented maximum and minimum values of the performance index for subject participation in each test. No dataset passed the one-sample Kolmogorov–Smirnov test, which indicated that all datasets were not normally distributed. The Wilcoxon signed rank test with Bonferroni correction for nonparametric tests was performed on pairwise comparison. The results of all tests were also analyzed in between-subjects comparison, and there were not significantly different. Horizontal continuous lines denoted statistically significant differences in the dispersions. All data were combined for statistical analysis using IBM SPSS Statistics 26.0.

3 Results

3.1 Accuracy of temperature prediction algorithm

Time-temperature response curves are shown in Figure 2A. Figure 2B used the example of 45.0°C to illustrate how steady-state temperature could be predicted with four different identification times based on slopes. Figure 2C showed that all the initial slopes were strongly correlated with the difference between the steady-state and initial-state temperatures under four identification times. Figure 2D depicted prediction accuracy with four identification times in each temperature block for ten trials. Prediction errors were low across the range of tested temperatures with average errors of 13.68 ± 10.19%, 11.00 ± 8.70%, 8.30 ± 6.00%, and 7.70 ± 6.20%. The average errors of the lower identification times (0.35 s and 0.50 s) were significantly different than those associated with the higher identification times (0.75 s and 1.00 s), and there was no significant difference between 0.75 s and 1.00 s. Considering prediction speed and accuracy, we selected 0.75 s as the identification time for subsequent tests. These results illustrated the robustness and accuracy of this algorithm, which was much faster than the sensor’s response time and contributed to the fast and safe detection of nociceptive temperatures.

3.2 Performance of withdrawal reflex strategy

The response of the withdrawal reflex circuit was evaluated. With constant activation of alpha command in Figure 4A, the two servo motors followed the antagonistic muscle forces computed by the neuromorphic muscle models (Figure 4B) and grasped the tube as shown in the fingertip force trajectories (Figure 4C). After the identification time, the temperature exceeded the warning threshold in Figure 4D, the interneuron and synapse models produced synaptic currents (Figure 4E). The synaptic currents inhibited the flexor muscle and excited the extensor muscle immediately. This action caused the hand to open, thus releasing the grasped tube during the execution time. If the temperature was below the warning threshold, the withdrawal reflex control loop remained inactive, with alpha command, neuromorphic muscle forces, and contact forces all remaining unchanged and no synaptic output. These results demonstrated the bionic withdrawal reflex’s ability to tune alpha command while detecting noxious temperatures under system no-load condition, verifying the feasibility of reconstructing a bionic withdrawal control loop using neuromorphic muscle control technology in the prosthetic hand.

Figure 4

Figure 4. Constant alpha commands controlled by host computer demonstrated the reflexive protection to 65.0°C temperature.

3.3 Performance of hybrid sensory strategy

Figure 5A illustrated the fitted psychophysical curves based on response probability for a range of temperature blocks from 30.0°C to 80.0°C in four able-bodied and two amputee subjects. The horizontal axis was the tested temperature and the vertical axis was the probability of perceiving a tingling sensation at each tested temperature. The 50% probability point of the fitted psychophysical curves were the PSE for each subject, which could demonstrate the nociceptive perception ability. Figure 5B showed that the PSE for each subject’s perception of harmful temperatures was not significantly different from each other. The average PSE in the nociceptive temperature threshold was 61.3°C, which was close to the warning threshold internally set at 60.0°C represented by a red horizontal line. Figure 5C presented confusion matrices for the recognition of buzz and tingling sensations with the six subjects achieving average success rates of 86.90 and 94.30%. Figures 5D,E presented the infrared imaging of a subject controlling a prosthetic hand to hold the tube with hot water and release the tube, which was highlighted in the yellow dotted box.

Figure 5

Figure 5. Perception response to nociceptive stimulation of four able-bodied and two amputee subjects. (A) Nociceptive perception probability curves in all subjects, red line represented the 50% probability. (B) Point of subjective equality (PSE) for each subject’s perception of nociceptive temperatures compared to the internally set heat warning threshold in red line (p > 0.05). (C) Confusion matrices represented all subjects’ identification accuracy of buzz and tingling sensations. (D,E) Infrared imaging of grasping and releasing of tube with hot water. The yellow dotted box highlighted the grasp and release actions.

These results suggested that despite individualized hybrid sensory coding strategies, amputees were able to distinctly perceive the warning of nociceptive temperatures and force. These results also demonstrated that a hybrid sensory strategy could provide feed-forward sensory commands for subsequent adjustment of protection motor commands.

3.4 Nociceptive warning and voluntary protection

Figure 6A depicted the representative complete process of one subject (H2) volitionally controlling the sEMG to open the prosthetic hand upon perceiving a tingling sensation at 62.5°C, 65.0°C, 70.0°C, 80.0°C. The subject grasped the tube using the flexor sEMG as shown in Figure 6A(a). After an identification time of 0.75 s, the detected temperature was above the warning threshold, the system switched from the buzz stimulus modality to the tingling stimulus modality (Figures 6A(b)). The subject increased the extensor sEMG significantly (Figures 6A(c,d)) after the perceived tingling sensation (Figures 6A(e)), leading to a gradual reduction in prosthetic hand force and ultimately opening the hand. Figure 6B showed four performance indices for H1, H2, and A1, A2. The time indices of AT, PT, ET, and RT showed no significant difference between each subject (p > 0.05). Test 1 confirmed the feasibility of reconstructing the sensory circuit for noxious temperatures in a prosthetic hand using TENS based on ETS technology, amputees could voluntarily control the opening of the prosthetic hand after perceiving a tingling warning sensation.

Figure 6

Figure 6. The time responses in Test 1 for voluntary opening actions to four nociceptive temperatures at 62.5°C, 65.0°C, 70.0°C, 80.0°C. (A) Representative data from subject H2 for contact force, stimulus pulse width, sEMG signals and alpha motor commands. Dark lines represented the average values and light shadow area represented mean ± standard deviation. (B) Performance indices of voluntary responses for two able-bodied and two amputees in Test 1.

3.5 Nociceptive warning and reinforcement protection

Test 2 demonstrated the results of the joint operation between the withdrawal reflex control strategy and the hybrid sensory strategy. This representative process from subject H2 was described in Figure 7A, where the changes in sEMG trailed but strengthened these in alpha motor commands following the nociceptive warning as shown in Figure 7A(c–e). This was demonstrated as soon as the synaptic current was firing (Figure 7A(b)), and the extensor motor command immediately reached the maximum value from excitatory action and the flexor motor command disappeared due to the inhibitory effect. The sEMG signals remained with no significant changes until the subject perceived a tingling sensation (Figure 7A(f)). Release of hand was indicated by the disappearance of fingertip forces (Figure 7A(a)). This showed that the nociceptive withdrawal reflex could accelerate the protection process of the prosthetic hand in addition to the subject’s voluntary action for the first time. There was no significant difference between four able-bodied and two amputee subjects in the performance of AT, PT, ET, and RT as shown in Figure 7B.

Figure 7

Figure 7. A total of six subjects (four able-bodied subjects and amputees) performed Test 2 in the cooperated reflex and voluntary control at 65.0°C nociceptive temperature. (A) Representative data from subject H2 to nociceptive heat stimulation with voluntary action and withdrawal reflex. Dark lines represented the average values and light shadow areas represented mean ± standard deviation. (B) Performance indices of six subjects in Test 2.

Test 2 proved the effectiveness of reinforcement of voluntary release and withdrawal reflex, thus verifying the feasibility of reconstructing a sensorimotor closed-loop for noxious temperature in a prosthetic hand and restoring the amputee’s ability to warn and shield against harmful temperature.

3.6 Time index analysis

With the involvement of the bionic withdrawal reflex, the action time of Test 2 was equal to the identification time (0.75 s) as shown in Figure 8, and was significantly faster than the action time of Test 1 with a median point of 1.32 s (IQR: 0.16), which demonstrated that the bionic withdrawal reflex was able to move away from the nociceptive temperatures in the first instance. Perception time belonged to the subjects’ sensory ability, so there was no significant difference between Test 1 (median point: 0.33 s, IQR: 0.12) and Test 2 (median point: 0.35 s, IQR: 0.09). The execution time that the prosthetic hand opened at the first time with maximal force in Test 2 (median point: 0.62 s, IQR: 0.07) was significantly faster than in Test 1 (median point: 1.19 s, IQR: 0.14), which required subjects to voluntarily increase the extensor muscle force to open. Similarly, Test 2 (median point: 1.39 s, IQR: 0.05) was still significantly faster than Test 1 (median point: 1.93 s, IQR: 0.13) in terms of reaction time. These results could be explained by the following neurophysiological properties of the human hand. The volitional control commands induced by sEMG go through the brain to make decisions belonging to the voluntary control, while the withdrawal reflex was equivalent to the short-latency spinal reflex, which prioritized the occurrence of the voluntary control to protect the human hand in the fastest way after sensing nociception (Andersen, 2007; Zangrandi et al., 2021). Results further illustrated that our study reconstructed a natural stereotypical sequence of hand activity in prostheses, effectively restoring the amputee’s capacity for both warning and protection against nociceptive temperatures.

Figure 8

Figure 8. Comparison of the time indices observed in the two tests. There was a significant difference in the AT, ET and RT between Test 1 and Test 2, suggested the effectiveness of the biomimetic withdrawal reflex in expediting the motor response, ***p < 0.001. No significant difference in PT was found between Test 1 and Test 2, both of which involved participant engagement.

4 Discussion

Perceiving nociceptive sensations and avoiding potential dangers for amputees is valuable, but it is not clear whether a nociceptive temperature warning and protection could be achieved in current prosthetic hand solutions (Bensmaia and Miller, 2014; Manz et al., 2022). In response to this need, we proposed reconstructing a sensorimotor circuit for nociceptive temperature detection in a prosthetic hand to restore the amputees’ ability to warn and protect against noxious temperatures. Two tests were carried out to investigate how similar neuromorphic mechanisms implemented in prosthetic hand could allow amputees to release a hot object upon a nociceptive warning. Test 1 validated the feasibility of providing nociceptive warning and voluntary protection, while Test 2 was integrated with the neuromorphic withdrawal reflex strategy to provide nociceptive warning and protection in a reinforcing way. Results confirmed that integrating the hybrid sensory strategy with the neuromorphic withdrawal reflex control strategy created a voluntary or reinforcement protection effect. These strategies restored the amputees’ sensorimotor ability to warn and protect against nociceptive temperatures and enabled a prosthetic hand to provide a similar neuromorphic mechanism, which is in line with natural human motor control and proprioception.

The steady-state temperature prediction algorithm displayed robustness and accuracy in promptly detecting nociceptive temperature. This temperature prediction algorithm shortened the long response time of the thermosensitive sensor. Among the four different identification times for accuracy, 0.75 s was the value that balanced response speed and prediction accuracy. Compared to the long response time of 6 s, our identification time was significantly faster than the sensor’s own response time (Figure 2D). This prediction algorithm based on the sensor output characteristics could be applied to other sensors with the same long response time (Figure 2B).

The evoked hybrid sensation modalities could be perceived by switching the stimulation mode from the buzz range to the tingling range. Previous studies demonstrated that both buzz and tingling sensations could be perceived and discriminated clearly (Zhang et al., 2022b). In this study, we encoded the force and nociceptive temperature warning corresponding to buzz and tingling sensations. Both subjects achieved high perception success rates for force and nociceptive warning (Figure 5E). This hybrid sensory strategy was a further application for previous studies in ETS-based sensory feedback (Hao et al., 2020). In particular, the nociceptive warning perception performance of 94.3% was similar compared to those reported in the literature that involved temperature perception (range from 15°C to 40°C) at 97.2% (Iberite et al., 2023) or 88% (Ueda and Ishii, 2016). Based on the high perception performance, we found that there was no significant difference between the average PSE of subjects and the actual temperature warning threshold (p > 0.5, Figure 4A,B), which provided a feed-forward input for the adjustment of subsequent motion commands. Force perception was also important in regulating grip events. In a safe condition that the predicted temperature was below the warning threshold, subjects could maintain their grip force for 5 s with the linear buzz sensation feedback. These results demonstrated the ability of subjects to interpret the multi-modality tactile information based on the hybrid sensory strategy. Besides, a nociceptive warning could also encode information about a sharp object, which was also an important cue for prosthesis protection (Osborn et al., 2018). It is very useful to code hybrid information including normal and bursty states. Such a hybrid sensory coding strategy enriched the resolution of sensations and allowed subjects to interpret tactile information without heavy cognitive load.

The human reaction under nociceptive perception provides an excellent template for prosthetic hands to learn from. Inspired by the natural withdrawal reflex, a biomimetic withdrawal reflex for noxious temperatures was designed to generate a pair of reflex compensations to antagonistic muscles. The outputs of interneuron and synapse models manifested the biologically plausible spiking characteristics in the spinal cord as shown in Figure 3. The evoking of the withdrawal reflex was a consequence of the excitability and inhibition of the spinal reflex and descending to alpha motor commands. The increment in extensor muscle force and decrement in flexor muscle force corresponded to the reflex compensation that was superimposed on a pair of alpha motor commands. Figure 4 depicted the feasibility of adjusting reflexive alpha motor commands upon detection of nociceptive temperature. Results revealed that these alpha motor command adjustments could generate well-behaved muscle forces, which were consistent with the previous research (Niu et al., 2021; Zhang Z. et al., 2022). This withdrawal reflex originated in the spinal cord and was a demonstration of the short-latency spinal reflex, which was a subconscious action to unpredictable perturbation (Andersen, 2007; Zangrandi et al., 2021). This neuromorphic short-latency spinal reflex strategy could be also applied to other subconscious protective actions, such as grip force compensation and regulation upon detecting slippage (Osborn et al., 2016).

Since it was difficult to recognize potential heat hazards with the naked eye (Figures 5D,E), perceiving force and nociceptive temperature and adjusting motor commands based on current sensory feedback are valuable abilities of the human sensorimotor control system (Andersen, 2007; Moayedi and Davis, 2013). The withdrawal reflex and voluntary movement, whether synergistic or opposing, represent reinforcement and contradictory control modes. Both demonstrate the subject’s sensorimotor ability to nociceptive temperature, which are crucial for daily life activities. In this study, we focused only on the reinforcement mode as a proof of concept to verify the proposed sensorimotor strategies for prosthetic hands. Test 1 implemented and tested the voluntary release and Test 2 combined the neuromorphic withdrawal reflex as a reinforcement mode. Results in Test 1 proved that amputees restored this valuable ability to perceive force and nociceptive warning, allowing them to volitionally control the prosthetic hand after perceiving a tingling sensation (Figure 6). Figure 7 verified that the withdrawal reflex control strategy could tune motor commands and accelerate voluntary release, demonstrating the feasibility of the reflexive protection mechanism. There was no significant difference between each subject in all time indices despite amputees lack of various receptors. However, the AT, ET, and RT in Test 2 showed a significant difference from Test 1, and the PT was not significantly different from Test 1 (Figure 8), which confirmed that subjects were able to reinforce the protective effect in response to heat hazards.

There are still the following limitations. First, a constant threshold of 60.0°C was used for noxious temperature warnings. Considering that amputees mainly rely on the interaction with the contralateral heath hand in their daily activities, this study has verified the feasibility of mimicking the human hand’s nociceptive perception and withdrawal mechanism in a prosthetic hand. The nociceptive threshold temperature could be appropriately set to a critical value. This critical value in our approach could also be adjusted to avoid other potential damage using other sensors or prosthetic hands if needed. Second, the nociceptive temperature warning and protection study was tested on only two amputee subjects. The performance trends of both subjects were consistent and demonstrated that the neurocognitive ability of amputees was capable of processing two modalities of sensory information effectively. In future studies, this approach will be evaluated in more forearm amputee subjects to obtain more valuable data.

This study investigated and demonstrated the sensorimotor ability of amputees to perceive and protect from nociceptive temperature based on hybrid sensory feedback and neuromorphic withdrawal reflex strategies. Since this work was a proof-of-concept study, it focused on a temperature warning system that may provide protection for modern prosthetic hands from thermal damage. Furthermore, a nociceptor with learning capabilities will be designed to judge whether the current temperature is dangerous or safe based on synaptic plasticity (Wang et al., 2023). Future utilization of model-based software and neuromorphic hardware computation methods may facilitate the inclusion of additional neurons or even neural networks, enabling concise judgment of temperature warning threshold and precise modulation of withdrawal reflex. A linear sensory coding strategy to indicate pain intensity will be designed in the future (Coghill et al., 1999; Moayedi and Davis, 2013). Subsequent considerations will also include using the subject’s sEMG as an additional input to reflex model to modulate the reflex intensity. A more sophisticated control scheme will be employed to manage more complex scenarios in activities of daily life.

5 Conclusion

In this study, nociceptive temperature was detected swiftly by a steady-state temperature prediction algorithm with good robustness and accuracy, which compensated for the slow response time and improved the precision in the detection of nociceptive temperature. A psychophysical test showed that the proposed hybrid sensory strategy could restore the sensory ability of amputees to perceive force under safe conditions and nociceptive warnings when the temperature of grasped objects exceeded the warning threshold. Amputees were able to warn and protect against nociceptive temperature through voluntary release or reinforced by neuromorphic withdrawal reflex verified the feasibility of reconstructing the sensorimotor circuits for nociception and protection in prosthetic hands. This study confirmed that this prosthetic hand was capable of recapturing the hierarchical properties of heat warning and protection characteristic of the human hand, which was conducive to restoring the amputee’s ability to warn and protect against thermal nociceptive stimulus, and also provided embodiment and safety for amputees. More control schemes including reflex and voluntary intention in heat protection will be discussed in the future.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding author.

Ethics statement

The studies involving humans were approved by Ethical Committee of Human and Animal Studies at Shanghai Jiao Tong University (E2020021I). The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

AX: Data curation, Formal analysis, Software, Validation, Writing – original draft. CL: Data curation, Software, Writing – original draft. C-hC: Data curation, Software, Writing – original draft. TL: Resources, Writing – review & editing, Formal analysis. CD: Conceptualization, Supervision, Writing – review & editing, Project administration. NL: Funding acquisition, Project administration, Supervision, Writing – review & editing, Conceptualization.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This work was supported in part by grants from the National Key R&D Program of China under Grant 2017YFA0701104, in part by the National Natural Science Foundation of China under Grant 81630050, and in part by the Science and Technology Commission of Shanghai Municipality under Grant 20DZ2220400.

Acknowledgments

We would like to extend our gratitude to Zhuozhi Zhang and Jie Zhang for valuable comments and technical support for this study. Additionally, we express our appreciation to Chuanxin Niu and Manzhao Hao for their invaluable suggestions on experimental design. Finally, we are grateful to the participants for their involvement in the study.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The author(s) declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnins.2024.1351348/full#supplementary-material

References

Aboseria, M., Clemente, F., Engels, L. F., and Cipriani, C. (2018). Discrete Vibro-tactile feedback prevents object slippage in hand prostheses more intuitively than other modalities. IEEE Trans. Neural Syst. Rehabil. Eng. 26, 1577–1584. doi: 10.1109/TNSRE.2018.2851617

PubMed Abstract | Crossref Full Text | Google Scholar

Akhtar, A., Sombeck, J., Boyce, B., and Bretl, T. (2018). Controlling sensation intensity for electrotactile stimulation in human-machine interfaces. Science Robotics 3:eaap9770.

Google Scholar

Amer-Cuenca, J. J., Goicoechea, C., Girona-López, A., Andreu-Plaza, J. L., Palao-Román, R., Martínez-Santa, G., et al. (2011). Pain relief by applying transcutaneous electrical nerve stimulation (TENS) during Unsedated colonoscopy: a randomized double-blind placebo-controlled trial. Eur. J. Pain 15, 29–35. doi: 10.1016/j.ejpain.2010.05.006

PubMed Abstract | Crossref Full Text | Google Scholar

Andersen, O. K. (2007). Studies of the Organization of the Human Nociceptive Withdrawal Reflex.: focus on sensory convergence and stimulation site dependency. Acta Physiol. 189, 1–35. doi: 10.1111/j.1748-1716.2007.01706.x

PubMed Abstract | Crossref Full Text | Google Scholar

Bensmaia, S. J., and Miller, L. E. (2014). Restoring sensorimotor function through Intracortical interfaces: progress and looming challenges. Nat. Rev. Neurosci. 15, 313–325. doi: 10.1038/nrn3724

PubMed Abstract | Crossref Full Text | Google Scholar

Bensmaia, S. J., Tyler, D. J., and Micera, S. (2020). Restoration of sensory information via bionic hands. Nat. Biomed. Eng. 7, 443–455. doi: 10.1038/s41551-020-00630-8

Crossref Full Text | Google Scholar

Chai, G., Sui, X., Li, S., He, L., and Lan, N. (2015). Characterization of evoked tactile sensation in forearm amputees with transcutaneous electrical nerve stimulation. J. Neural Eng. 12:066002. doi: 10.1088/1741-2560/12/6/066002

PubMed Abstract | Crossref Full Text | Google Scholar

Chen, Z., He, Y., Tao, X., Ma, Y., Jia, J., and Wang, Y. (2022). Thermal nociception of ionic skin: TRPV1 Ion Channel-inspired heat-activated dynamic ionic liquid. The Journal of Physical Chemistry Letters 13, 10076–10084. doi: 10.1021/acs.jpclett.2c02952

PubMed Abstract | Crossref Full Text | Google Scholar

Cheng, E. J., Brown, I. E., and Loeb, G. E. (2000). Virtual muscle: a computational approach to understanding the effects of muscle properties on motor control. J. Neurosci. Methods 101, 117–130. doi: 10.1016/S0165-0270(00)00258-2

PubMed Abstract | Crossref Full Text | Google Scholar

Coghill, R. C., Sang, C. N., Maisog, J. M., and Iadarola, M. J. (1999). Pain intensity processing within the human brain: a bilateral, distributed mechanism. J. Neurophysiol. 82, 1934–1943. doi: 10.1152/jn.1999.82.4.1934

PubMed Abstract | Crossref Full Text | Google Scholar

Cuberovic, I., Gill, A., Resnik, L. J., Tyler, D. J., and Graczyk, E. L. (2019). Learning of artificial sensation through long-term home use of a sensory-enabled prosthesis. Front. Neurosci. 13:853. doi: 10.3389/fnins.2019.00853

Crossref Full Text | Google Scholar

D’Anna, E., Petrini, F. M., Artoni, F., Popovic, I., Simanić, I., Raspopovic, S., et al. (2017). A somatotopic bidirectional hand prosthesis with transcutaneous electrical nerve stimulation based sensory feedback. Sci. Rep. 7:11306. doi: 10.1038/s41598-017-11306-w

PubMed Abstract | Crossref Full Text | Google Scholar

Dhillon, G. S., Lawrence, S. M., Hutchinson, D. T., and Horch, K. W. (2004). Residual function in peripheral nerve stumps of amputees: implications for neural control of artificial limbs. J. Hand Surg. Am. 29, 605–615. doi: 10.1016/j.jhsa.2004.02.006

PubMed Abstract | Crossref Full Text | Google Scholar

Dideriksen, J. L., Mercader, I., and Dosen, S. (2020). Closed-loop control using Electrotactile feedback encoded in frequency and pulse width. IEEE Trans Haptics. 13, 818–824. doi: 10.1109/TOH.2020.2985962

Crossref Full Text | Google Scholar

Dosen, S., Markovic, M., Somer, K., Graimann, B., and Farina, D. (2015). EMG biofeedback for online predictive control of grasping force in a myoelectric prosthesis. J. Neuroeng. Rehabil. 12:55. doi: 10.1186/s12984-015-0047-z

Crossref Full Text | Google Scholar

Dosen, S., Markovic, M., Strbac, M., Belic, M., Kojic, V., Bijelic, G., et al. (2017). Multichannel Electrotactile feedback with spatial and mixed coding for closed-loop control of grasping force in hand prostheses. IEEE Trans. Neural Syst. Rehabil. Eng. 25, 183–195. doi: 10.1109/TNSRE.2016.2550864

PubMed Abstract | Crossref Full Text | Google Scholar

George, J. A., Kluger, D. T., Davis, T. S., Wendelken, S. M., Okorokova, E. V., He, Q., et al. (2019). Biomimetic sensory feedback through peripheral nerve stimulation improves dexterous use of a bionic hand. Sci. Robotics 4:eaax2352. doi: 10.1126/scirobotics.aax2352

Crossref Full Text | Google Scholar

George, J. A., Page, D. M., Davis, T. S., Duncan, C. C., Hutchinson, D. T., Rieth, L. W., et al. (2020). Long-term performance of Utah slanted electrode arrays and intramuscular Electromyographic leads implanted chronically in human arm nerves and muscles. J. Neural Eng. 17:056042. doi: 10.1088/1741-2552/abc025

PubMed Abstract | Crossref Full Text | Google Scholar

Ghosh, A., Rothwell, J., and Haggard, P. (2014). Using voluntary motor commands to inhibit involuntary arm movements. Proc. R. Soc. B Biol. Sci. 281:20141139. doi: 10.1098/rspb.2014.1139

PubMed Abstract | Crossref Full Text | Google Scholar

Glowatzki, E., and Fuchs, P. A. (2002). Transmitter release at the hair cell ribbon synapse. Nat. Neurosci. 5, 147–154. doi: 10.1038/nn796

PubMed Abstract | Crossref Full Text | Google Scholar

Graczyk, E. L., Resnik, L., Schiefer, M. A., Schmitt, M. S., and Tyler, D. J. (2018). Home use of a neural-connected sensory prosthesis provides the functional and psychosocial experience of having a hand again. Sci. Rep. 8:9866. doi: 10.1038/s41598-018-26952-x

Crossref Full Text | Google Scholar

Hao, M., Chou, C.-H., Zhang, J., Yang, F., Cao, C., Yin, P., et al. (2020). Restoring finger-specific sensory feedback for Transradial amputees via non-invasive evoked tactile sensation. IEEE Open J. Eng. Med Biol. 1, 98–107. doi: 10.1109/OJEMB.2020.2981566

PubMed Abstract | Crossref Full Text | Google Scholar

Hardman, D., Thuruthel, T. G., and Iida, F. (2022). Self-healing ionic Gelatin/glycerol hydrogels for strain sensing applications. NPG Asia Mater. 14:11. doi: 10.1038/s41427-022-00357-9

Crossref Full Text | Google Scholar

Henrich, M. C., Frahm, K. S., and Andersen, O. K. (2021). Tempo-spatial integration of nociceptive stimuli assessed via the nociceptive withdrawal reflex in healthy humans. J. Neurophysiol. 126, 373–382. doi: 10.1152/jn.00155.2021

PubMed Abstract | Crossref Full Text | Google Scholar

Iberite, F., Muheim, J., Akouissi, O., Gallo, S., Rognini, G., Morosato, F., et al. (2023). Restoration of natural thermal sensation in upper-limb amputees. Science 380, 731–735. doi: 10.1126/science.adf6121

PubMed Abstract | Crossref Full Text | Google Scholar

Islam, G. M., Nazmul, A. A., and Collie, S. (2020). Textile sensors for wearable applications: a comprehensive review. Cellulose 27, 6103–6131. doi: 10.1007/s10570-020-03215-5

PubMed Abstract | Crossref Full Text | Google Scholar

Izhikevich, E. M. (2003). Simple model of spiking neurons. IEEE Trans. Neural Netw. 14, 1569–1572. doi: 10.1109/TNN.2003.820440

PubMed Abstract | Crossref Full Text | Google Scholar

Johansson, R. S., and Flanagan, J. R.. (2008). “Tactile Sensory Control of Object Manipulation in Humans,” in The Senses: A Comprehensive Reference. Academic Press, 67–86.

Google Scholar

Johansson, R. S., and Westling, G. (1984). Roles of Glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects. Exp. Brain Res. 56:237997. doi: 10.1007/BF00237997

PubMed Abstract | Crossref Full Text | Google Scholar

Johnson, M. I., Paley, C. A., Howe, T. E., and Sluka, K. A. (2015). Transcutaneous electrical nerve stimulation for acute pain. Cochrane Database Syst. Rev. 2021:2. doi: 10.1002/14651858.CD006142.pub3

Crossref Full Text | Google Scholar

Kordestani, H., and Zhang, C. (2020). Direct use of the Savitzky–Golay filter to develop an output-only trend line-based damage detection method. Sensors 20:1983. doi: 10.3390/s20071983

PubMed Abstract | Crossref Full Text | Google Scholar

Lan, N., Zhang, J., Zhang, Z., Chou, C.-H., Rymer, W. Z., Niu, C. M., et al. (2023). Biorealistic hand prosthesis with compliance control and noninvasive Somatotopic sensory feedback. Prog Biomed Eng 5:023001. doi: 10.1088/2516-1091/acc625

Crossref Full Text | Google Scholar

Li, S., Cheng, Z., Hao, M., He, X., Marquez, J. C., Niu, C. M., et al. (2015). Coordinated alpha and gamma control of muscles and spindles in movement and posture. Front. Comput. Neurosci. 9:e122. doi: 10.3389/fncom.2015.00122

Crossref Full Text | Google Scholar

Li, Y., Zhao, M., Yan, Y., He, L., Wang, Y., Xiong, Z., et al. (2022). Multifunctional biomimetic tactile system via a stick-slip sensing strategy for human-machine interactions. Npj Flexible Electronics 6:46. doi: 10.1038/s41528-022-00183-7

Crossref Full Text | Google Scholar

Liu, F., Deswal, S., Christou, A., Baghini, M. S., Chirila, R., Shakthivel, D., et al. (2022). Printed synaptic transistor–based electronic skin for robots to feel and learn. Sci. Robot. 7:eabl7286. doi: 10.1126/scirobotics.abl7286

Crossref Full Text | Google Scholar

Liu, W., Xie, R., Zhu, J., Jiansheng, W., Hui, J., Zheng, X., et al. (2022). A temperature responsive adhesive hydrogel for fabrication of flexible electronic sensors. Npj Flexible Electronics 6:68. doi: 10.1038/s41528-022-00193-5

Crossref Full Text | Google Scholar

Luo, Q., Niu, C. M., Chou, C.-H., Liang, W., Deng, X., Hao, M., et al. (2021a). Biorealistic control of hand prosthesis augments functional performance of individuals with amputation. Front. Neurosci. 15:783505. doi: 10.3389/fnins.2021.783505

Crossref Full Text | Google Scholar

Luo, Q., Niu, C. M., Liu, J., Chou, C.-H., Hao, M., and Lan, N. (2021b). Evaluation of model-based biomimetic control of prosthetic finger force for grasp. IEEE Trans. Neural Syst. Rehabil. Eng. 29, 1723–1733. doi: 10.1109/TNSRE.2021.3106304

PubMed Abstract | Crossref Full Text | Google Scholar

Lynn, M. T., Demanet, J., Krebs, R. M., Van Dessel, P., and Brass, M. (2016). Voluntary inhibition of pain avoidance behavior: an fMRI study. Brain Struct. Funct. 221, 1309–1320. doi: 10.1007/s00429-014-0972-9

PubMed Abstract | Crossref Full Text | Google Scholar

Manz, S., Valette, R., Damonte, F., Gaudio, L. A., Gonzalez-Vargas, J., Sartori, M., et al. (2022). A review of user needs to drive the development of lower limb prostheses. J. Neuroeng. Rehabil. 19:119. doi: 10.1186/s12984-022-01097-1

Crossref Full Text | Google Scholar

Markovic, M., Schweisfurth, M. A., Engels, L. F., Farina, D., and Dosen, S. (2018). Myocontrol is closed-loop control: incidental feedback is sufficient for scaling the prosthesis force in routine grasping. J. Neuroeng. Rehabil. 15:81. doi: 10.1186/s12984-018-0422-7

Crossref Full Text | Google Scholar

Mileusnic, M. P., Brown, I. E., Lan, N., and Loeb, G. E. (2006). Mathematical models of proprioceptors. I. Control and transduction in the muscle spindle. J. Neurophysiol. 96, 1772–1788. doi: 10.1152/jn.00868.2005

PubMed Abstract | Crossref Full Text | Google Scholar

Moayedi, M., and Davis, K. D. (2013). Theories of pain: from specificity to gate control. J. Neurophysiol. 109, 5–12. doi: 10.1152/jn.00457.2012

PubMed Abstract | Crossref Full Text | Google Scholar

Moqadam, B., Saeed, A. S., Asheghabadi, F. N., Jafarzadeh, H., Khosroabadi, A., Alagheband, A., et al. (2021). Conceptual method of temperature sensation in bionic hand by extraordinary perceptual phenomenon. J. Bionic Eng. 18, 1344–1357. doi: 10.1007/s42235-021-00112-w

PubMed Abstract | Crossref Full Text | Google Scholar

Muniak, M. A., Ray, S., Hsiao, S. S., Dammann, J. F., and Bensmaia, S. J. (2007). The neural coding of stimulus intensity: linking the population response of Mechanoreceptive afferents with psychophysical behavior. J. Neurosci. 27, 11687–11699. doi: 10.1523/JNEUROSCI.1486-07.2007

PubMed Abstract | Crossref Full Text | Google Scholar

Neto, J., Chirila, R., Dahiya, A. S., Christou, A., Shakthivel, D., and Dahiya, R. (2022). Skin-inspired Thermoreceptors-based electronic skin for biomimicking thermal pain reflexes. Adv. Sci. 9:2201525. doi: 10.1002/advs.202201525

Crossref Full Text | Google Scholar

Niu, C. M., Jalaleddini, K., Sohn, W. J., Rocamora, J., Sanger, T. D., and Valero-Cuevas, F. J. (2017). Neuromorphic meets Neuromechanics, part I: the methodology and implementation. J. Neural Eng. 14:025001. doi: 10.1088/1741-2552/aa593c

PubMed Abstract | Crossref Full Text | Google Scholar

Niu, C. M., Luo, Q., Chou, C.-h., Liu, J., Hao, M., and Lan, N. (2021). Neuromorphic model of reflex for Realtime human-like compliant control of prosthetic hand. Ann. Biomed. Eng. 49, 673–688. doi: 10.1007/s10439-020-02596-9

PubMed Abstract | Crossref Full Text | Google Scholar

Niu, C. M., Nandyala, S. K., and Sanger, T. D. (2014). Emulated muscle spindle and spiking afferents validates VLSI neuromorphic hardware as a testbed for sensorimotor function and disease. Front. Comput. Neurosci. 8. doi: 10.3389/fncom.2014.00141

Crossref Full Text | Google Scholar

Oddo, C. M., Raspopovic, S., Artoni, F., Mazzoni, A., Spigler, G., Petrini, F., et al. (2016). Intraneural stimulation elicits discrimination of textural features by artificial fingertip in intact and amputee humans. eLife 5:e09148. doi: 10.7554/eLife.09148

Crossref Full Text | Google Scholar

Osborn, L. E., Dragomir, A., Betthauser, J. L., Hunt, C. L., Nguyen, H. H., Kaliki, R. R., et al. (2018). Prosthesis with neuromorphic multilayered E-dermis perceives touch and pain. Sci. Robot. 3:eaat3818. doi: 10.1126/scirobotics.aat3818

Crossref Full Text | Google Scholar

Osborn, L., Kaliki, R. R., Soares, A. B., and Thakor, N. V. (2016). Neuromimetic event-based detection for closed-loop tactile feedback control of upper limb prostheses. IEEE Trans Haptics 9, 196–206. doi: 10.1109/TOH.2016.2564965

PubMed Abstract | Crossref Full Text | Google Scholar

Pinelli, F., Magagnin, L., and Rossi, F. (2020). Progress in hydrogels for sensing applications: a review. Mater. Today Chem. 17:100317. doi: 10.1016/j.mtchem.2020.100317

Crossref Full Text | Google Scholar

Raspopovic, S., Capogrosso, M., Petrini, F. M., Bonizzato, M., Rigosa, J., Di Pino, G., et al. (2014). Restoring natural sensory feedback in real-time bidirectional hand prostheses. Sci. Transl. Med. 6:222ra19. doi: 10.1126/scitranslmed.3006820

Crossref Full Text | Google Scholar

Romeo, R. A., Lauretti, C., Gentile, C., Guglielmelli, E., and Zollo, L. (2021). Method for automatic slippage detection with tactile sensors embedded in prosthetic hands. IEEE Trans. Med. Robot. Bionics 3, 485–497. doi: 10.1109/TMRB.2021.3060032

Crossref Full Text | Google Scholar

Saunders, I., and Vijayakumar, S. (2011). The role of feed-forward and feedback processes for closed-loop prosthesis control. J. Neuroeng. Rehabil. 8:60. doi: 10.1186/1743-0003-8-60

PubMed Abstract | Crossref Full Text | Google Scholar

Schweisfurth, M. A., Markovic, M., Dosen, S., Teich, F., Graimann, B., and Farina, D. (2016). Electrotactile EMG feedback improves the control of prosthesis grasping force. J. Neural Eng. 13:056010. doi: 10.1088/1741-2560/13/5/056010

PubMed Abstract | Crossref Full Text | Google Scholar

Shadmehr, R., and Wise, S. P.. (2005). A mathematical muscle model Supplementary Documents for ‘Computational Neurobiology of Reaching and Pointing’ (Cambridge, MA: MIT Press), pp 1–18.

Google Scholar

Shi, Y., Hua, Q., Dong, Z., Wang, B., Dai, X., Niu, J., et al. (2023). Neuro-inspired Thermoresponsive nociceptor for intelligent sensory systems. Nano Energy 113:108549. doi: 10.1016/j.nanoen.2023.108549

Crossref Full Text | Google Scholar

Sieck, G. C. (2019). Physiology in perspective: anatomy and physiology—structure and function in biology. Physiology 34, 379–380. doi: 10.1152/physiol.00029.2019

PubMed Abstract | Crossref Full Text | Google Scholar

Song, D., Raphael, G., Lan, N., and Loeb, G. E. (2008). Computationally efficient models of neuromuscular recruitment and mechanics. J. Neural Eng. 5:8. doi: 10.1088/1741-2560/5/2/008

PubMed Abstract | Crossref Full Text | Google Scholar

Tai, H., Duan, Z., Wang, Y., Wang, S., and Jiang, Y. (2020). Paper-based sensors for gas, humidity, and strain detections: a review. ACS Appl. Mater. Interfaces 12, 31037–31053. doi: 10.1021/acsami.0c06435

PubMed Abstract | Crossref Full Text | Google Scholar

Tan, D. W., Schiefer, M. A., Keith, M. W., Anderson, J. R., Tyler, J., and Tyler, D. J. (2014). A neural Interface provides long-term stable natural touch perception. Sci. Transl. Med. 6:257ra138. doi: 10.1126/scitranslmed.3008669

PubMed Abstract | Crossref Full Text | Google Scholar

Ueda, Y., and Ishii, C. (2016). Development of a feedback device of temperature sensation for a myoelectric prosthetic hand by using Peltier element. 2016 International Conference on Advanced Mechatronic Systems (ICAMechS), 488–493. Melbourne, Australia: IEEE

Google Scholar

Valle, G., D’Anna, E., Strauss, I., Clemente, F., Granata, G., Di Iorio, R., et al. (2020). Hand control with invasive feedback is not impaired by increased cognitive load. Front. Bioeng. Biotechnol. 8:287. doi: 10.3389/fbioe.2020.00287

Crossref Full Text | Google Scholar

Valle, G., Mazzoni, A., Iberite, F., D’Anna, E., Strauss, I., Granata, G., et al. (2018). Biomimetic Intraneural sensory feedback enhances sensation naturalness, tactile sensitivity, and manual dexterity in a bidirectional prosthesis. Neuron 100, 37–45.e7. doi: 10.1016/j.neuron.2018.08.033

PubMed Abstract | Crossref Full Text | Google Scholar

Wang, F., Olvera, J. R. G., Thakor, N., and Cheng, G. (2023). A bio-plausible approach to realizing heat-evoked nociceptive withdrawal reflex on the upper limb of a humanoid robot. IEEE Robot. Autom. Lett. 8, 3398–3405. doi: 10.1109/LRA.2023.3268624

Crossref Full Text | Google Scholar

Witney, A. G., Wing, A., Thonnard, J.-L., and Smith, A. M. (2004). The cutaneous contribution to adaptive precision grip. Trends Neurosci. 27, 637–643. doi: 10.1016/j.tins.2004.08.006

PubMed Abstract | Crossref Full Text | Google Scholar

Xie, A., Chou, C., Luo, Q., Zhang, Z., and Lan, N. (2022). Antagonistic Control of a Cable-Driven Prosthetic Hand with Neuromorphic Model of Muscle Reflex. 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 732–735.

Google Scholar

Yang, F., Hao, M.-Z., Zhang, J., Chou, C.-H., and Lan, N. (2020). An Experimental Protocol for Evaluating Pulse Width Modulation Ranges of Evoked Tactile Sensory Feedback in Amputees *. 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 3869–3872. Montreal, QC, Canada: IEEE

Google Scholar

Yang, S., Li, C., Wen, N., Shihong, X., Huang, H., Cong, T., et al. (2021). All-fabric-based multifunctional textile sensor for detection and discrimination of humidity, temperature, and strain stimuli. J. Mater. Chem. C 9, 13789–13798. doi: 10.1039/D1TC02755G

Crossref Full Text | Google Scholar

Yang, J. C., Mun, J., Kwon, S. Y., Park, S., Bao, Z., and Park, S. (2019). Electronic skin: recent Progress and future prospects for skin-attachable devices for health monitoring, robotics, and prosthetics. Adv. Mater. 31:1904765. doi: 10.1002/adma.201904765

Crossref Full Text | Google Scholar

Yu, Y., Chou, C.-H., Zhang, J., Hao, M., and Lan, N. (2022). Development of Myoelectric Control Module for Prosthetic Hand with Artifact Removal during Sensory Electrical Stimulation. Proceedings of the 15th International Joint Conference on Biomedical Engineering Systems and Technologies, 118–125.

Google Scholar

Zangrandi, A., D’Alonzo, M., Cipriani, C., and Di Pino, G. (2021). Neurophysiology of slip sensation and grip reaction: insights for hand prosthesis control of slippage. J. Neurophysiol. 126, 477–492. doi: 10.1152/jn.00087.2021

PubMed Abstract | Crossref Full Text | Google Scholar

Zhang, J., Chou, C., Wu, X., Pei, W., and Lan, N. (2022a). Non-Invasive Stable Sensory Feedback for Closed-Loop Control of Hand Prosthesis. 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 2344–2347. Glasgow, Scotland, United Kingdom: IEEE

Google Scholar

Zhang, J., Hao, M., Yang, F., Liang, W., Sun, A., Chou, C.-H., et al. (2022b). Evaluation of multiple perceptual qualities of transcutaneous electrical nerve stimulation for evoked tactile sensation in forearm amputees. J. Neural Eng. 19:26041. doi: 10.1088/1741-2552/ac6062

PubMed Abstract | Crossref Full Text | Google Scholar

Zhang, Z., Zhang, J., Luo, Q., Chou, C.-H., Xie, A., Niu, C. M., et al. (2022). A biorealistic computational model unfolds human-like compliant properties for control of hand prosthesis. IEEE Open J. Eng. Med. Biol. 3, 150–161. doi: 10.1109/OJEMB.2022.3215726

PubMed Abstract | Crossref Full Text | Google Scholar