Andrea Soltoggio
Frank van der Velde- 1Computer Science, Loughborough University, Loughborough, UK
- 2CTIT, CPE-BMS, University of Twente, Enschede, Netherlands
Models of adaptation and neural plasticity are often demonstrated in robotic scenarios with heavily pre-processed and regulated information streams to provide learning algorithms with appropriate, well timed, and meaningful data to match the assumptions of learning rules. On the contrary, natural scenarios are often rich of raw, asynchronous, overlapping and uncertain inputs and outputs whose relationships and meaning are progressively acquired, disambiguated, and used for further learning. Therefore, recent research efforts focus on neural embodied systems that rely less on well timed and pre-processed inputs, but rather extract autonomously relationships and features in time and space. The bio-inspired focus does not seek the most effective machine learning method to solve those problems, it rather points toward a better understanding of problem solving mechanisms in neural systems, which can in turn also provide viable solutions to difficult problems.
Realistic models of plasticity must account for delayed rewards (Soltoggio et al., 2013a), noisy and ambiguous data (Soltoggio et al., 2013b), and emerging and novel input features during online and value learning (Krichmar and Röhrbein, 2013). Those factors have indeed been an emerging focus of search (e.g., Sporns and Alexander, 2003; Lungarella and Sporns, 2006; Martius et al., 2013), with a growing number of studies that cannot be reviewed in this short editorial. Such approaches model the progressive acquisition of knowledge by neural systems through experience in environments that may be affected by ambiguities, uncertain signals, delays, or novel features (Pugh et al., 2014; Soltoggio, 2015). This Research Topic in Frontiers in Neurorobotics explored fundamental properties and dynamics of neural learning systems that are naturally immersed in a rich information flow. We are pleased with the contributions collected in this Research Topic, each of which addresses key topics in this emerging and important field of research.
One overarching problem in this field is that of making sense of large amounts of data from sensory systems in order to recognize particular situations and perform basic tasks. Parisi and colleagues took a self-organizing neural approach to action recognition using human pose-motion features. The Growing When Required (GWR) networks manifest a high-level structural plasticity that regulates network complexity in relation to the task (Parisi et al., 2015). Such a bio-inspired approach recorded state-of-the-art performance on a dataset of full-body actions captured with a depth sensor, with competitive results in a public benchmark of domestic daily actions.
Another source of large, noisy and uncertain data is found in robotic tactile sensors. Chou et al. (2015) deployed a specific robot called CARL-SJR with a full-body tactile sensory area. CARL-SJR encourages people to communicate with it through gentle touch, and provides feedback to users by displaying bright colors on its surface. The time-delayed and uncertain nature of the interactions poses challenges to the formation of correct associations between stimuli, rewards and actions. The approach devised by Chou et al. (2015) experiments with a strongly bio-inspired architecture of spiking neurons with neuromodulated plasticity. The model abstracts brain areas such as the primary somatosensory cortex, prefrontal cortex, striatum, and the insular cortex to process noisy data generated directly from CARL-SJR's tactile sensory area. The result is a robust learning mechanism that reliably forms correct associations and preferences for directions without heavily pre-processed inputs.
Uncertainty and large amount of data are also found in collaborative multi-robot scenarios in which multiple robots work alongside humans. Galbraith and colleagues propose a motor babbling approach to learn a complex set of relations and interactions with the 11-degrees-of-freedom RoPro Calliope mobile robot (Galbraith et al., 2015). Motor babbling of its wheels and arm enabled the Calliope to learn how to relate visual and proprioceptive information to achieve hand-eye-body coordination.
Motor control is a problem in which neural plasticity results in high level of adaptation, adjusting neural systems to operate in combination with specific bio-mechanical structures and morphologies. (Burms et al., 2015) demonstrated the utility of modulated Hebbian plasticity in embodied computation for compliant robotics. In such scenarios, control policies are generally unknown due to the partial offload of control policies to morphological computation. Modulated Hebbian plasticity was shown to lead to hybrid controllers that naturally integrate the computations that are performed by the robot's body into a neural network architecture. Those results demonstrate the potential of universal applicability of plasticity rules to complex control problems.
A similar problem was tackled in Dasgupta et al. (2015) in which they used distributed recurrent neural networks with synaptic adaptation to find a range of complex behaviors for walking robots. In particular, their approach demonstrated a remarkable flexibility in designing control systems that can work with multi-legged robots. A Central Pattern Generator is used to feed a self-adaptive reservoir network, which in turn provides motor control through a read-out integration unit. These results contribute to demonstrate the efficacy and continuous advancement of plastic neural models in complex input-output control scenarios.
The overall vision provided by these research papers outlines an increasingly more effective deployment of plastic neural models to tackle complex perception and control problems in which noise, uncertainty and delays pose a challenge to many algorithms. This vision matches the intuition of bioinspired neurorobotics approaches that propose advanced, plastic neural systems as viable models when sensory-motor information flows approach the richness and complexity found in the behavior of biological systems. We foresee a continuous growing attention to this emerging research area, in particular related to the development of more effective, scalable and general neural learning algorithms to effectively tackle rich and uncertain robotic information streams.
Author Contributions
AS devised the structure. Both authors AS and FV formulated the content and wrote the paper.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
Burms, J., Caluwaerts, K., and Dambre, J. (2015). Reward modulated hebbian plasticity as leverage for partially embodied control in compliant robotics. Front. Neurorobot. 9:9. doi: 10.3389/fnbot.2015.00009
Chou, T.-S., Bucci, L. D., and Krichmar, J. L. (2015). Learning touch preferences with a tactile robot using dopamine modulated stdp in a model of insular cortex. Front. Neurorobot. 9:6. doi: 10.3389/fnbot.2015.00006
Dasgupta, S., Goldschmidt, D., Wörgötter, F., and Manoonpong, P. (2015). Distributed recurrent neural forward models with synaptic adaptation and CPG-based control for complex behaviors of walking robots. Front. Neurorobot. 9:10. doi: 10.3389/fnbot.2015.00010
Galbraith, B. V., Guenther, F. H., and Versace, M. (2015). A neural network-based exploratory learning and motor planning system for co-robots. Front. Neurorobot. 9:7. doi: 10.3389/fnbot.2015.00007
Krichmar, J. L., and Röhrbein, F. (2013). Value and reward based learning in neurorobots. Front. Neurorobot. 7:13. doi: 10.3389/fnbot.2013.00013
Lungarella, M., and Sporns, O. (2006). Mapping information flow in sensorimotor networks. PLoS Comput. Biol. 2:e144. doi: 10.1371/journal.pcbi.0020144
Martius, G., Der, R., and Ay, N. (2013). Information driven self-organization of complex robotic behaviors. PLoS ONE 8:e63400. doi: 10.1371/journal.pone.0063400
Parisi, G. I., Weber, C., and Wermter, S. (2015). Self-organizing neural integration of pose-motion features for human action recognition. Front. Neurorobot. 9:3. doi: 10.3389/fnbot.2015.00003
Pugh, J. K., Soltoggio, A. and Stanley, K. O. (2014). “Real-time hebbian learning from autoencoder features for control tasks,” in Artificial Life 14: Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems (ALIFE XIV) (Cambridge, MA: MIT Press), 202–209.
Soltoggio, A. (2015). Short-term plasticity as cause-effect hypothesis testing in distal reward learing. Biol. Cybern. 109, 75–94.
Soltoggio, A., Lemme, A., Reinhart, F. R., and Steil, J. J. (2013a). Rare neural correlations implement robotic conditioning with reward delays and disturbances. Front. Neurorobot. 7:6. doi: 10.3389/fnbot.2013.00006
Soltoggio, A., Reinhart, F. R., Lemme, A., and Steil, J. J. (2013b). “Learning the rules of a game: neural conditioning in human-robot interaction with delayed rewards,” in Proceedings of the Third Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics (Osaka).
Keywords: neural plasticity, neural adaptation, neuro-robotics, cognitive modeling, emobodied cognition
Citation: Soltoggio A and van der Velde F (2015) Editorial: Neural plasticity for rich and uncertain robotic information streams. Front. Neurorobot. 9:12. doi: 10.3389/fnbot.2015.00012
Received: 03 September 2015; Accepted: 08 October 2015;
Published: 27 October 2015.
Edited by:
Florian Röhrbein, Technische Universität München, GermanyReviewed by:
Georg Martius, Institute for Science and Technology Austria, AustriaCopyright © 2015 Soltoggio and van der Velde. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Andrea Soltoggio, YS5zb2x0b2dnaW9AbGJvcm8uYWMudWs=