Claudio Castellini- 1Chair of Medical Robotics, Friedrich-Alexander Universität Erlangen-Nürnberg, Erlangen, Germany
- 2The Adaptive Bio-Interfaces Group, German Aerospace Center (DLR), Institute of Robotics and Mechatronics, Oberpfaffenhofen, Germany
Introduction
Rehabilitation robotics, prosthetics, and assistive robotics are a hard business—especially, controlling a rehabilitation robot or a prosthesis is difficult, not only because the user needs to learn to use the device and, possibly, re-learn to use his own body, but also because designing, building, and testing the related control system is difficult. How can a person with no hands, or with severely reduced mobility, let the device know-how and when to grasp, stand, or help perform a reaching movement? Whenever the user's residual ability permits it, an appealing solution, consisting of gathering signals from (intent detection, ID) and stimulating (somatosensory feedback, SF), the peripheral nervous system (PNS), in order to build a bidirectional human-machine, interfaces fostering a deep, synergistic connexion between user and machine (Beckerle et al., 2018).
Now, one of the major hurdles in this field is the generally bad quality of the signals one is interested in gathering and using. Signals coming from the PNS of an impaired user are, per se, less stable, reliable, and repeatable than those obtained by able-bodied subjects; moreover, signals gathered in controlled conditions, e.g., while the user sits in a laboratory, can differ dramatically from those the same user produces while trying to perform the same actions in real life (Jiang et al., 2012), hampering the translational potential of research. This problem is being tackled by introducing machine learning able to progressively couple with the user over time (Hahne et al., 2015), allowing for direct, continuous, and sustained interaction with him (Nowak et al., 2018) and by shifting the focus from off-line lab testing to early deployment on end-users in the clinics. There is, however, a second major inherent problem: PNS signals are hard to find. Either we try and build a direct connection to the nerves and/or the muscles (invasive approaches) or we listen to their surface manifestation (non-invasive approaches). Invasiveness means better signals at the price of surgery, to various levels of discomfort for the patient; non-invasiveness is user-friendly and won't violate bodily integrity but yields less clear distorted signals. So, we face a trade-off between the quality of the signals and the quality of user life.
As electronics are more and more integrated with the body, however, more acceptable minimally invasive surgery and permanent implants appear, and their safety and reliability grow. So, eventually, tighter physical integration of man and machine could become tolerable and desirable. In a recent perspective paper, (Farina et al., 2021) concur that, at least in prosthetics, where PNS signal processing is the main means to achieve control, osseointegration, targeted muscle reinnervation, and implanted sensors and stimulators are majorly way ahead toward high-performance bionic limbs. One of the messages of the paper is that as technology advances and brings us closer to the biological integration of man and machine, invasive or minimally invasive approaches become more and more appealing, to the point of, eventually, overcoming non-invasive ones and becoming standard.
In this short paper, on the other hand, I argue that PNS non-invasive techniques for both ID and SF will still be indeed preferable in the mid-term. Given the technological advancements we are witnessing in the fields of bodily surface sensing and stimulation, the advantages non-invasive techniques enjoy will still constitute an unsurmountable gap at least in the decade to come. I first talk about non-invasive versus invasive techniques in intent detection (the feed-forward path, section Intent Detection), then in somatosensory feedback (the feedback path, section Somatosensory Feedback), and finally I draw some concluding remarks (section Discussion).
Intent Detection
Control of motorised prostheses via “neural” signals started in the 50s, when two surface electromyography (sEMG) sensors were used to determine the speed of opening/closing of a mechanical one-degree-of-freedom gripper (Fougner et al., 2012). This has remained the clinical standard till the mid-2000s, when multi-fingered prosthetic hands started to appear, calling for a more refined form of control, relying on classification (machine learning) applied to sEMG. Despite the promising perspective of the idea though, still, today there are only two commercial ID systems based upon the classification of sEMG: the Complete Control by Coapt Engineering1 and the MyoPlus by Ottobock2, thus, relying on 6-8 single sensors and basic classification. This is not surprising; in general, given the huge variety of daily-living situations, the “one-shot” pattern matching (data collection/model building/control) is unfit for the task (Nowak et al., 2018). Moreover, PNS surface methods, to different degrees, all suffer from sensor displacement caused by donning and doffing, short- and mid-term changes in the morphology of the body, variations in skin impedance, and sensor lift-off (Merletti et al., 2011). The reduced number of sensors that can be embedded in a prosthetic socket is usually insufficient to provide a proper view of muscular activity in the residual limb, and alternative approaches such as pressure sensing and ultrasound scanning are still in the academic prototyping phase (Castellini et al., 2014).
More invasive approaches, on the other hand, promise to provide more focussed, clearer signals, and, as the technology of man-machine integration advances, their appeal increases (Farina et al., 2021). PNS invasive techniques entail different degrees of invasiveness, from the injection of miniaturised EMG sensors into the muscles (Weir et al., 2009) to osseointegration (Ortiz-Catalan et al., 2014) and targeted muscle reinnervation (Aszmann et al., 2015); depending on the severity of the impairment then, patients might agree to stand surgery.
Nevertheless, besides the absence of surgery, surface ID approaches rely on technology requiring (almost) no direct interfacing to biological tissue. Improving surface sensing means achieving higher resolution, both in time and in space, and pushing on miniaturisation; and to achieve this, no complex biological interfaces are needed. To me, this means that, for a long time to come, the non-invasive approaches will still maintain a technological advantage over invasive ones. High-density wearable sEMG arrays comprising ~120 sensors sampled at 2 kHz have recently appeared (e.g., the MuoviPro by OTBioelettronica3), as well as ultraminiaturised ultrasound scanning devices, whose transducer array fits in the size of a coin (Fournelle et al., 2021). Both systems deliver an unprecedented precision, both in time and space, in the detection of residual muscle activity.
To sum up, high-density surface approaches, coupled with “life-long” interactive machine learning, are not limited to the one-shot schema (Castellini et al., 2015; Beckerle et al., 2018) and shall overcome the problems that are typically associated to non-invasiveness (low density and unreliability in time) at no additional burden for the user. For this reason, I argue, they will still represent the methods of choice in the short- and mid-term.
Somatosensory Feedback
Similar remarks hold, in my opinion, as far as SF is concerned. By SF in this specific framework mechanical or electrical signals are provided to the user's somatosensory system, which relates to the status of the rehabilitation device and/or the environment (Beckerle et al., 2018). SF enables closed-loop human control of rehab- and assistive devices: through repeated, stable and identifiable patterns of SF stimulation, corresponding to actions of the device, the user increases the sense of agency (being in control) and ownership (being a part of the self) over it, leading to embodiment and—at least, this is the current opinion in the community—reciprocal adaptation and optimal control (Hahne et al., 2015). The goal is to provide a rich set of feelings to the user, both in terms of the type of feeling (touch, pressure, temperature, texture, etc.) and of its intensity, responsiveness, and appropriateness. At the same time, the stimulator and control system must be as small, lightweight, and low-power as possible (Došen et al., 2017).
Given these goals and despite the spectacular advancements appearing in the scientific literature in the past years, invasive SF is still far from being clinically applicable. Although experiments are reported, in which direct electrical stimulation of peripheral nerves has enabled users to discriminate several tens of spatially stable sensations, such implants are mostly temporary (remarkable exceptions to this are reported, e.g., in Petrini et al., 2019; Ortiz-Catalan et al., 2020), and involve complex surgical procedures; also, nociception seems to be a non-negligible associated issue (Davis et al., 2016). Non-invasive SF, on the other hand, has recently evolved from bulky sets of a few vibrotactile/mechano-tactile actuators to hundreds of high-density electrical stimulators. Electro-tactile stimulation (ECS), for example, works by injecting small electrical currents across the skin, thereby not affecting muscle contractions, and maintaining compatibility with sensing techniques for intent detection (e.g., sEMG, see Došen et al., 2017). High-density integrated ECS delivers highly differentiated stimuli, both qualitatively and quantitatively, at essentially no additional psychological burden for the user (Dideriksen et al., 2022; Isaković et al., 2022). In my opinion, any advantage delivered by direct connection to the afferent nerves, e.g., cuff or intra-fascicular electrodes [and there is ample evidence for their effectiveness, see for instance (D'Anna et al., 2019; Petrini et al., 2019)], needs to face an inevitable acceptance gap; moreover, here, too, as it was the case for ID, the technologies involved in ECS vs. invasive techniques belong to two different levels of maturity.
Discussion
This work has concentrated on prosthetics; nevertheless, in my opinion, similar remarks apply to rehabilitation robotics—to a lesser extent, indeed, but in an important way. The usage of ID and SF in the management of rehabilitation exoskeletons, exo-suits, and virtual reality environments is still in its infancy, since robotic therapies for, e.g., stroke survivors and patients with SCI are still largely based upon repetitive motions, not initiated by the patients (if not via verbal interaction with a therapist). Here, PNS interfacing is a second-class denizen, although progress in this direction is being made (Lobo-Prat et al., 2014; Sullivan et al., 2017), since, as opposed to the case of prosthetics, patients in rehabilitation are supposed to engage in their therapy only for a limited amount of time and already suffer from neural conditions. For these reasons, it seems to me that ID/SF techniques in rehabilitation should be kept even more non-invasive.
Non-invasive techniques for sensing and stimulation in prosthetics have clear advantages over invasive ones; both in terms of their immediate applicability, but also in perspective, since they do not need direct interfacing with the nervous system. This is just my informed opinion, and this paper is not meant as a survey, and, for sure, does not exhaustively cover the field; so, my statement may be challenged or even overturned in a few years. Still, the technological advancement of non-invasive techniques is faster, and I believe that their advantage over invasive techniques will remain significant for at least one decade; they will still be preferable to invasive ones. Of course, as the technology of invasive techniques progresses, too, there could be a point in the future at which they will be as convenient as non-invasive ones. Mixing invasive and non-invasive techniques could, at that point, be an interesting option to take out the best of both worlds, for instance, coupling ultrasound-based ID and SF are given by implanted stimulators (a thorough review, providing a unified view of such techniques, is found in Shokur et al., 2021). A life-long adaptive form of machine learning will be needed, just as it is today, to reach optimal control of any reha- or assistive device. If it is true that more good data is better, it is also true that data from impaired users change in time, whatever the source of data is, and in that case, a strategy to determine which data to use or to emphasise will be needed. True adaptation to the user should provide this possibility.
Author Contributions
The author confirms being the sole contributor of this work and has approved it for publication.
Funding
This work was partially supported by the DFG (German Research Agency) project Deep-Hand, CA1389/1-2.
Conflict of Interest
The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Footnotes
1. ^https://coaptengineering.com
2. ^https://www.ottobockus.com/prosthetics/upper-limb-prosthetics/solution-overview/myo-plus/myo-plus.html
3. ^https://www.otbioelettronica.it/component/sppagebuilder/?view=page&id=150
References
Aszmann, O. C., Roche, A. D., Salminger, S., Paternostro-Sluga, T., Herceg, M., Sturma, A., et al. (2015). Bionic reconstruction to restore hand function after brachial plexus injury: a case series of three patients. Lancet 385, 2183–2189. doi: 10.1016/S0140-6736(14)61776-1
Beckerle, P., Castellini, C., and Lenggenhager, B. (2018). Robotic interfaces for cognitive psychology and embodiment research: a research roadmap. Wiley Interdiscip. Rev. Cogn. Sci. 10, e1486. doi: 10.1002/wcs.1486
Castellini, C., Artemiadis, P., Wininger, M., Ajoudani, A., Alimusaj, M., Bicchi, A., et al. (2014). Proceedings of the first workshop on Peripheral Machine Interfaces: going beyond traditional surface electromyography. Front. Neurorobotics. 8, 22. doi: 10.3389/fnbot.2014.00022
Castellini, C., Bongers, R. M., Nowak, M., and van der Sluis, C. K. (2015). Upper-limb prosthetic myocontrol: two recommendations. Front. Neurosci. 9, 496. doi: 10.3389/fnins.2015.00496
D'Anna, E., Valle, G., Mazzoni, A., Strauss, I., Iberite, F., Patton, J., et al. (2019). A closed-loop hand prosthesis with simultaneous intraneural tactile and position feedback. Sci. Robot. 4, eaau8892. doi: 10.1126/scirobotics.aau8892
Davis, T. S., Wark, H. A., Hutchinson, D. T., Warren, D. J., O'Neill, K., Scheinblum, T., et al. (2016). Restoring motor control and sensory feedback in people with upper extremity amputations using arrays of 96 microelectrodes implanted in the median and ulnar nerves. J. Neural Eng. 13, 036001. doi: 10.1088/1741-2560/13/3/036001
Dideriksen, J. L., Mercader, I. U., and Dosen, S. (2022). Task-dependent adaptations in closed-loop motor control based on electrotactile feedback. IEEE Trans. Human-Machine Syst. doi: 10.1109/THMS.2021.3134556. [Epub ahead of print].
Došen, S., Marković, M., Štrbac, M., Belić, M., Kojić, V., Bijelić, G., et al. (2017). Multichannel electrotactile feedback with spatial and mixed coding for closed-loop control of grasping force in hand prostheses. IEEE Trans. Neural Syst. Rehabil. Eng. 25, 183–195. doi: 10.1109/TNSRE.2016.2550864
Farina, D., Vujaklija, I., Brånemark, R., Bull, A. M., Dietl, H., Graimann, B., et al. (2021). Toward higher-performance bionic limbs for wider clinical use. Nat. Biomed. Eng. doi: 10.1038/s41551-021-00732-x
Fougner, A., Stavdahl, Ø., Kyberd, P. J., Losier, Y. G., and Parker, P. A. (2012). Control of upper limb prostheses: terminology and proportional myoelectric control - a review. IEEE Trans. Neur. Syst. Rehab. Eng. 20, 663–677. doi: 10.1109/TNSRE.2012.2196711
Fournelle, M., Grün, T., Speicher, D., Weber, S., Yilmaz, M., Schoeb, D., et al. (2021). Portable ultrasound research system for use in automated bladder monitoring with machine-learning-based segmentation. Sensors 21, 6481. doi: 10.3390/s21196481
Hahne, J. M., Dähne, S., Hwang, H.-J., Müller, K.-R., and Parra, L. C. (2015). Concurrent adaptation of human and machine improves simultaneous and proportional myoelectric control. IEEE Trans. Neur. Syst. Rehabil. Eng. 23, 618–627. doi: 10.1109/TNSRE.2015.2401134
Isaković, M., Malesević, J., Kostić, M., Došen, S., and Štrbac, M. (2022). The impact of size and position of reference electrode on the localization of biphasic electrotactile stimulation on the fingertips. IEEE Trans. Haptics. doi: 10.1109/TOH.2022.3141187. [Epub ahead of print].
Jiang, N., Dosen, S., Müller, K.-R., and Farina, D. (2012). Myoelectric control of artificial limbs - is there a need to change focus? IEEE Signal Process. Mag. 29, 148–152. doi: 10.1109/MSP.2012.2203480
Lobo-Prat, J., Kooren, P., Stienen, A., Herder, J., Koopman, B., and Veltink, P. (2014). Non-invasive control interfaces for intention detection in active movement-assistive devices. J. NeuroEng. Rehabil. 11, 168. doi: 10.1186/1743-0003-11-168
Merletti, R., Botter, A., Cescon, C., Minetto, M. A., and Vieira, T. M. (2011). Advances in surface EMG: recent progress in clinical research applications. Crit. Rev. Biomed. Eng. 38, 347–379. doi: 10.1615/CritRevBiomedEng.v38.i4.20
Nowak, M., Castellini, C., and Massironi, C. (2018). Applying Radical Constructivism to machine learning: a pilot study in assistive robotics. Constructivist Found. 13, 250–262. Available online at: https://constructivist.info/13/2/250.nowak
Ortiz-Catalan, M., Håkansson, B., and Brånemark, R. (2014). An osseointegrated human-machine gateway for long-term sensory feedback and motor control of artificial limbs. Sci. Transl. Med. 6, 257re6. doi: 10.1126/scitranslmed.3008933
Ortiz-Catalan, M., Mastinu, E., Sassu, P., Aszmann, O., and Brånemark, R. (2020). Self-contained neuromusculoskeletal arm prostheses. N. Engl. J. Med. 382, 1732–1738. doi: 10.1056/NEJMoa1917537
Petrini, F. M., Valle, G., Strauss, I., Granata, G., Di Iorio, R., D'Anna, E., et al. (2019). Six-Month assessment of a hand prosthesis with intraneural tactile feedback. Ann. Neurol. 85, 137–154. doi: 10.1002/ana.25384
Shokur, S., Mazzoni, A., Schiavone, G., Weber, D. J., and Micera, S. (2021). A modular strategy for next-generation upper-limb sensory-motor neuroprostheses. Med 2, 912–937. doi: 10.1016/j.medj.2021.05.002
Sullivan, J. L., Bhagat, N. A., Yozbatiran, N., Paranjape, R., Losey, C. G., Grossman, R. G., et al. (2017). “Improving robotic stroke rehabilitation by incorporating neural intent detection: Preliminary results from a clinical trial,' in 2017 International Conference on Rehabilitation Robotics (ICORR) (London), 122–127. doi: 10.1109/ICORR.2017.8009233
Keywords: neurorobotics, machine learning, human-robot interaction, intent detection, somatosensory feedback
Citation: Castellini C (2022) Peripheral Nervous System Interfaces: Invasive or Non-invasive? Front. Neurorobot. 16:846866. doi: 10.3389/fnbot.2022.846866
Received: 31 December 2021; Accepted: 25 March 2022;
Published: 29 April 2022.
Edited by:
Florian Röhrbein, Technische Universität Chemnitz, GermanyReviewed by:
Solaiman Shokur, Swiss Federal Institute of Technology Lausanne, SwitzerlandAbdelkader Nasreddine Belkacem, United Arab Emirates University, United Arab Emirates
Copyright © 2022 Castellini. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Claudio Castellini, Y2xhdWRpby5jYXN0ZWxsaW5pJiN4MDAwNDA7ZmF1LmRl