Event Abstract

Back to Event

Learning a Reachable Space Map in a Gaze Centered Reference Frame

Lorenzo Jamone1*, Matthieu Destephe1, Lorenzo Natale2, Kenji Hashimoto1, Atsuo Takanishi1 and Giulio Sandini2
  • 1 Waseda University, Humanoid Robotics Institute, Japan
  • 2 Italian Institute of Technology, RBCS, Italy

I. INTRODUCTION
Humanoid robots that are supposed to help humans at work and during daily life should be able to reach for objects using the arms or even the whole body, eventually ending up grasping and using such objects. An important issue associated with these behaviors is the definition of the space that the robot can reach, i.e. the reachable space or workspace. In general, if an accurate model of the system is available, analytical or geometric methods can be used to analyze and obtain the robot reachable space. However, to build analytical model of current humanoid robots is becoming a more and more difficult task, due to their increasing complexity. Learning techniques represent an appealing solution if poor analytical knowledge is available, and seem even mandatory as long as humanoids are supposed to become completely autonomous and gain knowledge through their own experience.
In this work we investigate how a humanoid robot can learn a representation of the reachable space from motor experience: a Reachable Space Map. The map provides information about the reachability of a visually detected object (i.e. a 3D point in space). Learning is performed online during the execution of goal directed reaching movements; reaching control is based on kinematic models that are also learned online. Recent studies have tested numerical or machine learning tools in order to build a representation of the robot reachable space [1], [2]. The proposed methods describe the workspace with respect to a cartesian frame (either placed in the world or on the robot) and compute it offline. Three main concepts make our solution innovative with respect to previous works: the use of a gaze centered reference frame to describe the robot workspace, the primary role of action in order to build and represent knowledge, the need for learning to be performed autonomously and online.

II. A REACHABLE SPACE MAP
The position of an object in space can be defined in different ways, the most common one being the cartesian position with respect to a fixed reference frame, either placed somewhere in the environment or on the robot body. Another possible choice is to encode the object position with motor coordinates, as proposed in [3]. Indeed, it has been hypothesized that humans employ a gaze centered frame of reference for reaching control [4], [5], even in the case of whole body reaching [6].
Furthermore, neurophysiological evidences show that human perception of what is reachable or not relies on motor information [7]. This suggests that a map of the reachable space can be described using motor coordinates, and can be learned from motor experience. Before the reaching movement starts the robot head is controlled to fixate the target object (i.e. gaze control), then the object position with respect to the robot is encoded with the head motor configuration: in our implementation, pitch and yaw rotation of the head and eyes vergence (3 DOFs). This information is used both to drive the reaching action and to build the reachable space map.
Here we propose two different solutions to design the Reachable Space Map: the Basic Map and the Enhanced Map. Both maps are implemented using a LWPR neural network [8] which is trained online during the execution of arm reaching movements. After some learning the robot can use the map to modify its body posture in order to make a perceived object reachable (e.g. bending the waist or walking toward the object). We carried on the experiments using the iCub Dynamic Simulator [9]. All the software has been realized using YARP [10]. Preliminary results show that the system is able to learn a representation of the reachable space and use it to improve reaching control.
A. Basic Map
The easier solution is to train the LWPR with the head configuration, $\q_{head} \in \Re^3$, as input and a value $S \in \left\{0,1\right\}$ which indicates the failure/success of the reaching action as output. Every time a fixated object is reached the LWPR is trained with $<\q_{head},S=1>$, because a feasible arm configuration which brings the hand in the fixation point defined by $\q_{head}$ exists. Conversely, if a reaching task has not been accomplished the LWPR is trained with $<\q_{head},S=0>$. Figure 1 shows the map after about 5000 reaching movements.
B. Enhanced Map
A more complete description of the reachable space can be obtained by providing additional information about the location in space the robot is trying to reach for. The new set of data is described by $<\q_{head}, S, \e, \sigma>$, where $\e \in [0;\e_{MAX}]$ is the final error of the reaching controller and $\sigma \in [0;\sigma_{MAX}]$ is a measure of the distance from singular configurations of the arm (here we choose the smaller singular value of the arm Jacobian [11]). A training value $V$ is computed from these data. If the target is reached ($S=1$), then $V=1-\frac{\sigma}{\sigma_{MAX}}$. If the target is not reached ($S=0$), then $V=1+\frac{\e}{\e_{MAX}}$. Therefore $V$ is a continuous value ranging from 0 to 2, where 0 means a reachable point which can be reached with a good arm configuration (far from singularities) and 2 indicates a point that is not reachable and probably lies very far from the robot (large error of the reaching controller). After every reaching action the LWPR is trained with the head configuration $\q_{head}$ as input and the training value $V$ as output. Figure 2 shows the map after about 5000 reaching movements.
C. Waist control
When the map is queried with $\q_{arm}$ it outputs a value $R$ which provides information about the reachability of the fixated point. By moving the waist while keeping the point (or object) in fixation the robot can collect sensory data in the form $<\Delta\q_{waist},\Delta{R}>$ and use them to learn an approximation of the Jacobian for which holds $\dot{R}=J(\q_{head},\q_{waist})\dot{\q}_{waist}$. Then the waist can be actuated using $\dot{\q}_{waist}=J^\dagger(\q_{head},\q_{waist})(R^d-R)$ to cause $R \rightarrow R^d$, i.e. to make a fixated object reachable. The same approach can be extended to other motions, e.g. walking.

Fig. 1. Basic Map: 2D projections of the 3D map for different values of pitch (i.e. head elevation). Circles and crosses indicate respectively training samples of reachable and unreachable points.

Fig. 2. Enhanced Map: 2D projections of the 3D map for different values of pitch (i.e. head elevation). Circles and crosses indicate respectively training samples of reachable and unreachable points.

Figure 1
Image
Figure 2
Image

References

REFERENCES
[1] Y. Guan, K. Yokoi, and X. Zhang, “Numerical methods for reachable space generation of humanoid robots,” The International Journal of Robotics Research, no. 27, pp. 935–950, 2008.
[2] F. Zacharias, C. Borst, and G. Hirzinger, “Capturing robot workspace structure: representing robot capabilities,” in International Conference on Intelligent Robots and Systems (IROS), 2007, pp. 3229–3236.
[3] G. Metta, G. Sandini, and J. Konczak, “A developmental approach to visually-guided reaching in artificial systems,” Neural Networks, vol. 12, no. 10, pp. 1413–1427, 1999.
[4] J. F. Soechting and M. Flanders, “Sensorimotor representations for pointing to targets in three-dimensional space,” Journal of Neurophysiology, vol. 62, pp. 582–594, 1989.
[5] J. McIntyre, F. Stratta, and F. Lacquaniti, “Viewer-centered frame of reference for pointing to memorized targets in three-dimensional space,” Journal of Neurophysiology, vol. 62, pp. 582–594, 1989.
[6] M. Flanders, L. Daghestani, and A. Berthoz, “Reaching beyond reach,” Experimental Brain Research, vol. 126, no. 1, pp. 19–30, 1999.
[7] Y. Coello, A. Bartolo, B. Amiti, H. Devanne, E. Houdayer, and P. Derambure, “Perceiving what is reachable depends on motor representations: Evidence from a transcranial magnetic stimulation study,” 2008. [Online]. Available: http://www.plosone.org/article/info/3Adoi/2F10.1371/2Fjournal.pone.0002862
[8] S. Vijayakumar and S. Schaal, “Locally weighted projection regression : An o(n) algorithm for incremental real time learning in high dimensional space,” in International Conference on Machine Learning (ICML), 2000, pp. 1079–1086.
[9] V. Tikhanoff, P. Fitzpatrick, G. Metta, L. Natale, F. Nori, and A. Cangelosi, “An open source simulator for cognitive robotics research: The prototype of the icub humanoid robot simulator,” in Workshop on Performance Metrics for Intelligent Systems, National Institute of Standards and Technology, Washington DC, August 19-21 2008.
[10] G. Metta, P. Fitzpatrick, and L. Natale, “Yarp: yet another robot platform,” International Journal on Advanced Robotics Systems, March 2006, special Issue on Software Development and Integration in Robotics.
[11] C. A. Klein and B. E. Blaho, “Dexterity measures for the design and control of kinematically redundant manipulators,” International Journal of Robotics Research, no. 6, pp. 72–83, 1987.

Keywords: autonomous online learning, humanoid robot, reachable space generation, sensorimotor learning

Conference: IEEE ICDL-EPIROB 2011, Frankfurt, Germany, 24 Aug - 27 Aug, 2011.

Presentation Type: Poster Presentation

Topic: Embodied cognition

Citation: Jamone L, Destephe M, Natale L, Hashimoto K, Takanishi A and Sandini G (2011). Learning a Reachable Space Map in a Gaze Centered Reference Frame. Front. Comput. Neurosci. Conference Abstract: IEEE ICDL-EPIROB 2011. doi: 10.3389/conf.fncom.2011.52.00021

Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.

The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.

Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.

For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.

Received: 11 Apr 2011; Published Online: 12 Jul 2011.

* Correspondence: Dr. Lorenzo Jamone, Waseda University, Humanoid Robotics Institute, Tokyo, Japan, l.jamone@qmul.ac.uk