- 1Department of Mechanical Engineering, Case Western Reserve University, Cleveland, OH, United States
- 2Department of Mechanical Engineering, National University of Singapore, Singapore, Singapore
- 3Department of Mechanical Engineering, Birla Institute of Technology and Science Pilani, Pilani, India
- 4Department of Electrical and Electronics Engineering, Birla Institute of Technology and Science Pilani, Pilani, India
Legged robots can have a unique role in manipulating objects in dynamic, human-centric, or otherwise inaccessible environments. Although most legged robotics research to date typically focuses on traversing these challenging environments, many legged platform demonstrations have also included “moving an object” as a way of doing tangible work. Legged robots can be designed to manipulate a particular type of object (e.g., a cardboard box, a soccer ball, or a larger piece of furniture), by themselves or collaboratively. The objective of this review is to collect and learn from these examples, to both organize the work done so far in the community and highlight interesting open avenues for future work. This review categorizes existing works into four main manipulation methods: object interactions without grasping, manipulation with walking legs, dedicated non-locomotive arms, and legged teams. Each method has different design and autonomy features, which are illustrated by available examples in the literature. Based on a few simplifying assumptions, we further provide quantitative comparisons for the range of possible relative sizes of the manipulated object with respect to the robot. Taken together, these examples suggest new directions for research in legged robot manipulation, such as multifunctional limbs, terrain modeling, or learning-based control, to support a number of new deployments in challenging indoor/outdoor scenarios in warehouses/construction sites, preserved natural areas, and especially for home robotics.
1 Introduction
Research on legged robots design and locomotion has mainly been fueled by the desire to deploy teleoperated or autonomous systems in otherwise inaccessible terrains. While wheeled and tracked vehicles are broadly used on paved surfaces (currently no more than 7% of Earth’s land surface) and fields for agriculture and forestry (roughly 46.6%) (Hooke et al., 2013), they have poor performance on sandy, rocky, and other unmodified natural terrains (roughly 46.5%). Nearly half of Earth’s ice-free land area has not been modified by humans (Hooke et al., 2013), and is often inhabited by animals that use their legs to survive by walking (Schroer et al., 2004), running (Raibert et al., 2008), climbing (Grieco et al., 1998), jumping (Zhang et al., 2017), and swimming (Song et al., 2016). In addition to accessing natural terrains, legs can also enable improved access to human environments by climbing stairs, changing body height to crawl through confined spaces, or carefully stepping around clutter or hazards. Potential real world applications such as industrial inspection can also rely on legged robots to regularly validate visual, thermal, and acoustic data at waypoints in monitoring and exploration tasks (see recent relevant review Bellicoso et al. (2018)). Furthermore, legged robots can deliver payloads such as medkits in search-and-rescue scenarios to hard-to-reach locations, e.g., after standard transport routes have been compromised by a natural disaster.
Impressive recent advances across different fields of robotics now bring us closer to future legged robots that more actively interact with new environments. Legged robots can learn to traverse challenging or urban environments (Liu et al., 2020; Miki et al., 2022), by adapting their gait, footholds, and body pose, and the next frontier lies beyond passive exploration (such as mapping/inspection/localization). Actively turning a valve (Zhao et al., 2019), retrieving an object (Roennau et al., 2014), pushing away an obstacle (Stüber et al., 2020) or opening a door (Schmid et al., 2008) are some fundamental manipulation tasks that can be the difference between mission success and failure for a legged robot, and can drastically enhance their abilities. Fortunately, decades of manipulation work has developed an ecosystem of platforms capable of both fine dexterous in-hand-manipulation [e.g., robotic Rubik’s cube solver (Yang B. et al., 2020a)] and inexpensive, high-volume pick and place robots (Altuzarra et al., 2011) with ongoing goals to approach human-like manual dexterity (Billard and Kragic, 2019). Manipulation research has primarily focused on stationary arms, but manipulation also occurs on mobile platforms (see relevant reviews on wheeled robots (Thakar et al., 2023) and on RoboCup (Sereinig et al., 2020)). Part of the reason why legged robots have often remained limited to simple interactions with their surroundings is that they are usually deployed in environments that are less predictable, making legged manipulation tasks more challenging. However, some of the most advanced legged robots are overcoming these challenges to perform dynamic manipulation (e.g., catching a ball, or opening a door) during locomotion (Bellicoso et al., 2019; Zimmermann et al., 2021). These works demonstrate the promising potential of legged manipulation platforms as one of the next frontiers for the legged robot community.
In this review, we consider different approaches to moving objects with a legged platform. While “manipulation” encompasses any robotic efforts to change the state of an object (Lynch and Park, 2018), a unique ability for mobile robotic platforms is changing the position of an object. Focusing on moving objects also enables us to compare fundamentals of the relationships between an independent object to be moved and different legged robotic platforms. These concepts apply broadly to applications in factories, hospitals, agriculture, land management, and domestic environments. A wide variety of other potentially useful examples of interactions such as digging in granular media, adjusting a component of a larger structure, pressing buttons, or driving a car are more specialized, but often include some of the same primitive abilities.
The approach selected for manipulation also determines the overall design of the robot and system. If a legged platform is carrying task-specific manipulators, design metrics such as weight, range, strength, and generality are important. If the legs themselves become tools for moving objects, key design trade-offs become impact resistance and sensitivity. Teams of robots can also collaborate to move larger objects together, but each robot then is typically smaller and more dependent on coordination. In all cases, the stability of the platform must be considered to prevent robots from falling over and ensure useful contacts with the environment are leveraged, while the robustness of the grasp must be considered to prevent objects from being dropped or damaged.
While legged robots have made significant strides in manipulation capability in recent years, a proper summary and review of the current progress in legged robot manipulation is missing. Most reviews surrounding legged robots focus on mechanical designs and control strategies that power their impressive legged locomotion ability (Liu et al., 2007; Sayyad et al., 2007; Wu et al., 2009; Silva and Machado, 2012; Zhou and Bi, 2012; Aoi et al., 2017; Suzumori and Faudzi, 2018). However, many recent works in the community have started focusing on legged robot manipulation, and this upcoming field at the interface between legged robot and (mobile) manipulation is becoming popular and active research area, which has not been covered by any other review paper to our knowledge. This work aims to fill this gap by providing an in-depth analysis of the various types of legged robot manipulation. Specifically, our aim is to present a comprehensive summary of the general challenges, design, control strategies, and features of legged robot manipulation, as well as to highlight rising opportunities for future research. We also hope that this work can assist new researchers in this field, or those coming into it from an interdisciplinary background, in selecting and designing new legged robots and approaches for legged manipulation tasks.
The outline of this review is as follows: first, we provide a brief background on general requirements related to legged robot manipulation. We then classify manipulation approaches into four main categories: (a). Object interactions without grasping (Pushing, kicking, and non-prehensile lifting). (b). Manipulation with walking legs (single leg, double leg, and whole body). (c). Manipulation with dedicated non-locomotive arms (e.g., when an arm or trunk is added to the legged platform). (d). Legged teams for manipulation (when a legged robot helps other robots or humans with an object interaction). We propose a new theoretical analysis to relate the size of manipulated objects and the manipulation limits of robots that use legs to grasp (Section 3.2). Through a number of new figures and tables, we summarize the carrying capacity of the different manipulation methods discussed in this work, by comparing the maximum object size vs. maximum robot size. In doing so, we examine the key characteristics of different manipulation methods, with the goal of helping future users select the proper system and control approach based on the required tasks. The examples provided, most of which have been validated on physical platforms. We finally discuss open avenues for future research, including both open/new challenges in this area and future applications for these platforms.
2 General requirements
This section discusses the general requirements/challenges that all types of legged manipulation platforms face. First of all, we note that legged robot manipulation is much more complex than pure locomotion or pure manipulation, even though each of these two parts bring their own challenges. However, since the robot’s body, legs, and manipulated object also affect each other, one of the key new challenges lies in ensuring simultaneous stabilization of both locomotion and manipulation, even during dynamic configuration changes in complex environments.
First, a legged platform’s stance must be stable with respect to the environment during manipulation. In other words, the robot should not be destabilized by manipulation motions or reaction forces. To maintain the static stability of a legged robot, the projected center of mass (along the gravity vector) must fall within its support polygon (i.e., the convex hull constructed from all ground contacts, i.e., leg positions) (McGhee and Frank, 1968). When a legged robot extends its manipulator to reach an object, the weight of the manipulator and the object may move the robot’s CoM out of its support polygon. This can explain why grasping an object is harder for bipedal robots, since their support polygons are relatively small, only formed by two legs, and the mass of the torso and arms can easily shift the CoM outside of such a small area (Kim et al., 2021). Adding more legs can help mitigate this problem, thus fueling increasing research in quadruped- and hexapod-based manipulation (Kalouche et al., 2015; Fang and Gao, 2018; van Dam et al., 2022; Xin et al., 2022; Yao et al., 2022). Conversely, we note that pushing against a large object can actually increase the robot’s stability, if the object can help support some of the robot’s weight.
Second, contact with the object to be manipulated must be reliable and safe, to prevent the robot from dropping or damaging the object. The different classes of legged manipulation approaches described in this work exhibit different types of contact forces. For most types of grasping discussed, the ultimate goal is to lift the object off the ground, implying the need for either form closure (Prattichizzo and Trinkle, 2016) (i.e., kinematically constraining a rigid body by stationary contacts) or force closure (i.e., securing objects by frictional contacts). However, non-grasping (non-prehensile) manipulation is another way of moving objects without the requiring a secure grasp, by instead applying forces to push, impacts to kick, or relying on another (non-grasping) type of attachment mechanism to the object such as hooks/trays (Ruggiero et al., 2018). Performing manipulation tasks with a legged platform also increases the potential vibrations transmitted to the manipulator, which may need to be corrected/accounted for to maintain continuous and secure contact with the object (Ferrolho et al., 2022).
The third and final requirement is related to the feedback effects between the locomotor and manipulator, with the aim to minimize the effects that the manipulator can have on the robot’s stability that can affect manipulation accuracy. In manufacturing, industrial arms are usually installed on a fixed base so that end-effectors’ positions and rotation angles can be easily and accurately calculated by Denavit Hartenberg transformation (Corke, 2007) or Product of Exponentials (Chen et al., 2018). For mobile manipulation, manipulators have been installed on wheeled platforms for pick-and-place tasks; but there, relatively precise kinematic and dynamic models still exist, which can even take into account slip (Liu and Liu, 2009). Although legged robots can generate precise gait patterns without payload, increasing the mass carried by the robot can dramatically decreases locomotion/stability accuracy and reliability. While elasticity (i.e., compliance) in the robot’s legs may enables smoother suspension of loads, these can also further decrease control/motion accuracy (Nishihama et al., 2003). In addition, granular media, like sand or pebbles, are often found in natural terrains and are hard to model due to their complexity and stochastic nature (Nagabandi et al., 2018). Keeping the robot’s body stable on these surfaces is thus even more challenging, because ground penetration varies based on the weight supported by each leg, which changes as objects are moved/lifted/dropped or gait cycles modified. More generally, we note that legged manipulation systems are typically hybrid, underactuated, and non-linear, making the development of controllers for legged-base manipulators complex, especially in the presence of additional manipulation appendages (manipulating with non-locomotive arms) (Sleiman et al., 2021). Potential solutions can involve continuous feedback to correct relative motion errors, coordinate between locomotor and manipulator to improve the overall accuracy (i.e., whole-body control), as well as adaptive controllers that learn and improve over time.
3 Types of manipulation
In this section, we discuss each of the four main categories of legged robot manipulation. For each category, we bring up the fundamental methodology, system design, and control strategy of existing works, as well as the pros and cons of each strategy. All cited papers that were validated with physical robots are also summarized in Figure 1. We further summarize the carrying capacity and control strategy of the different manipulation methods discussed in this section in Figure 2; Supplementary Appendix Table SA3, respectively.
FIGURE 1. Our work categorizes legged platforms for object manipulation into four main groups based on grasp type: Object interactions without grasping, manipulation with the walking legs, dedicated non-locomotive arms, and legged teams. Sixty-one papers have been validated with physical robots. As an overview, this pie chart shows that the number of publications is roughly evenly divided between these types.
FIGURE 2. Carrying capacity of different manipulation methods (maximum object size vs. maximum robot size). The data points plotted are listed in Supplementary Appendix Table SA2. If the point is above the reference line (object size is equal to robot size), the robot can manipulate an object larger than itself. This data was obtained through literature sources. For cases when this was not explicitly provided, we approximated the object size from the published figures or approximated their volume using the density of water. Robot size is determined by the maximum dimensions.
3.1 Object interactions without grasping
3.1.1 Locomoting into an object: Pushing
Conceptually, pushing an object is one of the simplest methods for a legged robot to manipulate its environment. Here, we define pushing as walking such that the robot’s body contacts an object imparting some horizontal force, such that the object slides along the ground. Any mobile robot can move an object by locomoting into it, provided it can impart enough pushing force. To successfully move an object, the pushing force must exceed the resistance force, which can be approximated with Coulomb friction on flat surfaces:
where Fpush is the force applied by the robot, μ is the coefficient of friction, and W is the weight of the object to be moved.
However, the effect of the pushing force on the object depends on the location of the pushing force, the object’s weight distribution, and shape, as well as its contact(s) with the ground. Robots can easily (and inadvertently) push over objects (Inoue et al., 2010), navigating an obstacle course of tall thin objects without pushing them over is an effective demonstration of agility. To push while avoiding knocking over the object, the moment caused by the pushing force must be less than the moment of the object’s weight about the tipping edge. Specifically, for a rigid object,
where h is the height of the point of application of force, and xCoM is the horizontal distance of the object’s center of mass from the tipping edge. Thus, a pushing strategy is most likely to be effective in moving relatively light and wide objects on flat smooth surfaces rather than heavy and tall objects on rough terrains.
The mechanics of pushing helps inform the hardware design of robots for these environments. Because of Newton’s third law, equal and opposite reaction forces will also be applied to the robot during pushing that can cause the robot to similarly slip or tip, which means the configuration of the robot is similarly constrained. As a result, pushing robots tend to be relatively heavy, wide-based, and powerful. In addition, broad contact surfaces may reduce the precision of the contact calculations, but can help provide evenly distributed forces. These design concepts are evident in robot snowplows (Klein et al., 2016); their wide blades provide an even force distribution and extra weight is added to generate higher traction forces. However, wheeled robots are limited to flat terrains. The additional degrees of freedom offered by legged robots can enable locomotion over a larger variety of structured and unstructured challenging environments. A robot might climb into a truck, climb up stairs, walk over rugged terrain, or walk behind an object to begin pushing.
A second advantage of pushing with robotic legs is that the legs typically have more strength and provide more range of motion than the upper appendages alone. The loading capacity of arms is enhanced by actuating the whole body. By erasing the boundary between walking and working spaces, the overall workspace is expanded in both size and dimension. As a result, the forces applied to the object can be much higher and the load on the actuators is better distributed. In Inoue et al. (2002) and Yoshida et al. (2001), a humanoid robot changes its body’s configuration by adjusting its posture and changing its footholds. There, an objective function based on the manipulability [which essentially measures how easily the robot can instantaneously move in any direction, as defined in Lynch and Park (2017)] of both its arms is maximized, making it more dexterous. Yoshida et al. (2002) uses a similar approach to optimize the robot’s body posture to uniformly distribute load across the robot’s actuators, comparable to proper lifting techniques for humans. This also applies to pushing as well to other scenarios that use dedicated non-locomotive arms as discussed in Section 3.3.
Multi-legged robots can lean on and rest some of their legs on an object to push it more stably with their remaining legs. For example, Inoue et al. (2010) utilizes a hexapod robot, which leans against a large block while holding it with two legs to push it along the floor. Leaning while holding establishes three points of contact between the robot and the object, equivalent to secure force closure, as discussed in Section 3.2.2. Furthermore, leaning provides stability to the robot due to the wider resulting base of support. Finally, leaning puts some of the robot’s weight onto the pushed object, increasing and stabilizing the available contact force. Since the robot is supported by the object, the remaining four legs can work in pairs to provide bilateral symmetric pushing forces (i.e., the middle left and right legs push together while the back left and right legs swing and vice versa). These gaits can be optimized for more stable pushing, bypassing much of the complicated friction modelling. With more complex modeling, the leg could even specifically brace against particular footholds to exploit higher effective coefficients of friction.
In cases where the robot makes and breaks contact with the object, model-based control has been often used for its inherent robustness and safety guarantees. Unlike simple open-loop cases, where the robot is assumed to remain rigidly fixed to the object, closed-loop pushing addresses the complex contact dynamics incorporating friction between the robot and the object. Most existing approaches to this problem as well as other examples of non-prehensile manipulation with legged robots, like kicking, rely on model-based and model-free hierarchical approaches. These employ optimization methods such as Model Predictive Control (MPC) or Reinforcement Learning (RL), often using a high-level planner on top of a low-level controller. For example, Rigo et al. (2022) aims to minimize tracking error when tasking a quadruped robot by pushing a large box using its body along a given trajectory on the floor. Similarly; Sombolestan and Nguyen (2022) considers a quadruped robot tasked with pushing a large and heavy box (up to 60% of the robot’s weight) up a slope with no knowledge of the object’s mass, coefficient of friction, or slope of the terrain. In both of these methods, separate mode sequences are established for frictional contact, while a real-time controller is used to connect these contact modes into a feasible plan. This is done through two cascading MPCs where the high-level manipulation MPC optimizes contact forces and location while the low-level “loco-manipulation” MPC regulates interaction forces between the robot and the object during the locomotion task.
A fundamental limitation of pushing is that the object’s friction with the ground must be overcome. One way effective friction can be reduced is if the object can be rolled. The dung beetle robot can use two front legs to walk backward on different terrains while using the middle and hind legs to stabilize and manipulate a large ball (Leung et al., 2018; 2020; Thor et al., 2018; Billeschou et al., 2020). Although only validated in simulation, an interesting example is the dual MPC-based approach described in Yang C. et al. (2020b), where a quadruped robot stands on top of a large ball and rolls it to transport itself and the ball. The other way to avoid losing energy and momentum to friction is to lift the object, as discussed in subsequent sections.
3.1.2 Imparting an impulse: Kicking
Kicking imparts a quick impulse rather than a sustained force on the object. In other words, some of the robot’s momentum is imparted to the object during their (usually brief) contact. This is typically treated differently since there is no attempt to maintain or continually reestablish contact with the object. The object to be manipulated is ideally lightweight to facilitate projectile motion, elastically deformable to minimize any damage due to large impulses, and round to facilitate rolling as well as mitigate any aerodynamic, structural, or contact considerations due to shape. This is why balls made of flexible materials are well-suited and the most common candidates.
The problem of robot kicking using legs has broadly been tackled using two distinct approaches. The first is to develop a robot that can kick a ball a certain large distance accurately while the robot itself remains stationary. This approach deals with the problem of striking a ball but ignores that of locomoting and maintaining stability while kicking. Rober (2021) demonstrated a stationary kicking robot using springs that could kick an American football up to 64 m without breaking itself. However, the speed and point-of-contact during kicking were not precise resulting in a large variance in the ballistic trajectory of the ball and hence, the landing location. Flemmer and Flemmer (2014) constructed a similar kicking robot powered by pneumatic cylinders with an automatic vision-based aiming system using an onboard camera. This robot could consistently kick a ball for an average distance of 45.6 m. Shankhdhar et al. (2022) made a mobile kicking robot by mounting a kicking apparatus onto a holonomic wheeled base. Although this method allows a mobile robot to kick objects, it may not be considered a true kicking legged robot since the kicking “leg” is purely for striking and therefore non-locomotive. More recent attempts go a step further and model the rigid contact dynamics by devising model-based and model-free RL approaches for stationary robots tasked with striking a hockey puck (Chebotar et al., 2017) and playing mini golf (Muratore et al., 2021), respectively.
The second approach is to use walking robots to additionally kick/dribble while locomoting. The most well-known proponents for this research have been the RoboCup Leagues (Stone, 2007). The RoboCup Leagues consist of various leagues primarily centered around robot soccer, but also include search-and-rescue competitions using legged robots. Robot soccer leagues are simulation or hardware competitions divided into three classes: quadruped robots, medium-sized humanoids, and small-sized humanoids. In the quadruped class, only Sony’s fully autonomous robot dogs, AIBO, are allowed, while in the humanoid class, each robot must have two legs, two arms, a head, and a trunk within specific restrictions on the height of the center of mass and size of the feet. This has inspired a large body of work based on optimizing robot vision, robotic learning, localization, and locomotion strategies dedicated to these well-defined problem statements. In most early works such as Cherubini et al. (2010); Friedmann et al. (2008); Behnke and Stückler (2008); Acosta-Calderon et al. (2008), the task of kicking a ball into a goal is broken up into several simpler behaviours such as approaching, controlling, and reaching, with further-distinguished sub-behaviours such as kicking to the front or the side, etc. (which is always carried out using their heads to control and kick the ball). Though only validated in simulation, Jouandeau and Hugel (2014) is one of the first attempts to improve shooting distance using trajectory optimization, while Peng et al. (2017); da Silva et al. (2021); Teixeira et al. (2020) developed an RL based approach to teach humanoid robots to dribble, kick, and shoot.
Quadrupeds can also accurately kick a ball into a goal (Ji et al., 2022). This is an example of a contact-rich task, where MPC is not practical since the dynamics at play are difficult to model accurately (e.g., deformable contacts with the ball). Furthermore, even if an accurate model could be obtained, it would still be computationally expensive and slow, thus unsuitable for such an online application. Thus, in that work, hierarchical RL is used with a high-level trajectory behaviors policy on top of a low-level kicking motion control policy. The low-level control policy swings the leg quickly while keeping the robot stable, and the high-level planning policy approximates the contact between the robot and the ball, the rolling friction, and the interaction of the ball with the world for sending the ball to the goal. Multi-stage training is performed with a rigid ball in simulation and fine-tuned with a soft ball in the real world, toward a final demonstration on hardware.
3.1.3 Non-prehensile lifting
Finally, it is also possible to use legged robots for other types of non-prehensile lifting, although few examples exist yet in the literature. For example, an object and body robot could have mating attachment points (such as a hook and an eye), which allows the robot to lift the object off the ground and carry it without any grasping involved. A robot waiter could have a long flat protrusion to slide under a tray (similar to the way a forklift carries pallets.) A robot might have an electromagnet to collect metals from a field. For more examples, a rigorous review of non-prehensile manipulation and manipulation primitives is provided in Ruggiero et al. (2018) and applied to a wheeled robot in Bertoncelli et al. (2020). The legs of the platform must be able to support the additional weight of the object or objects. If the center of mass of the robot changes, walking gaits may need to be adapted. Legged platforms will induce more oscillation than wheeled robots, which may require them to choose slower speeds or more secure attachments, such as those offered by the grasping approaches described in the next sections.
3.2 Manipulation with walking legs
In robot designs where legs have many degrees of freedom for locomotion, these same degrees of freedom can also be suitable for directly grasping and manipulating objects. We classified manipulation with walking legs into three categories based on the number of legs used (Figure 3): Grasping with a single leg, multi-legged grasping (usually two), and whole-body grasping, in which no legs are reserved for walking on the ground.
FIGURE 3. Main classes of grasping with walking legs (left to right): (A) grasping with a single leg (Heppner et al., 2015), (B) grasping with two legs (Booysen and Reiner, 2015), and (C) grasping using all legs/whole body (Gong et al., 2022). Image by and courtesy of Yixian Qiu.
We analyze the relationship between the size of the manipulated object (relative to the robot) and the manipulation challenges/limitations of these three methods of manipulation. The required coefficient of contact friction μ represents the manipulation difficulty. For a given value of μ, each manipulation method has a different maximum grasp size to ensure force closure. We derive these limits for each manipulation method by assuming the object to be a sphere for generality, thin legs for simplicity, and non-backdrivable actuators with no torque limits as this analysis is limited to spatial considerations for the design parameters of the robot. The object radius is r, the fully extended leg length is l, the distance between the shoulders of two legs (width) is s, and the pincer length is lp. The dimensions of our models are based on LAURON V (Roennau et al., 2014): l = 2 s = 4 lp while the grippers are assumed to be scissor-like pincers. By forming geometric models based on Coulomb friction, we obtain the following constraints for force-closure.
3.2.1 Grasping with a single leg
For grasping with a single leg, the robot’s gripper design is heavily dependent on the design of its legs (Figure 4). Heppner et al. (2015) presented a bio-inspired hexapod, LAURON V, which includes a lightweight gripper on one front leg that can be retracted actively when needed (e.g., during locomotion on rough terrain). This gripper-to-leg design (Figure 4A) is necessary for the robot to protect the gripper from directly supporting the robot’s heavy body. By shifting its CoM backward, LAURON V can stand on its four hind legs, leaving the two front legs free for manipulation. To prevent stability loss when the robot walks with a grasped object, LAURON V stores the object on its back so that the front legs can be freed to support and push the body forward during locomotion. Similarly, Kim and Jun (2014) developed a hexapod for underwater locomotion featuring two front grippers which can be concealed inside the robot’s shins during walking (Figure 4B). Compared with LAURON V, these grippers each possess three DoFs but are limited in size by the shins they are retracted into. Irawan et al. (2016) designed a scissor-like gripper-to-leg for the hexapod Hexaquad, that two scissor tips can be directly converted into leg mode by closing tips together or into gripper mode by splitting them (Figure 4C). While many legged robots only use front legs for manipulation, Hexaquad expands the workspace since all legs are gripper-to-legs that can be transformed to be grippers whenever needed. In other words, Hexaquad does not need to turn around to grab an object that is behind it. To ensure a large support polygon for both front leg and back leg manipulation, the base joints of the middle legs are capable of moving back and forth. For certain applications, designing grippers as interchangeable tools can be essential for improving versatility (Figure 4D). The quadruped TITAN-IX can store tools (including grippers) on its back and then equip them on its end-effectors for demining tasks (Hirose et al., 2005). When retrieving an object, one leg works as a manipulator, and the other three legs keep the body stable. Developing methods for controlling such manipulators is a notable challenge, especially when performing such contact-rich tasks. TITAN-IX implements a human-robot interface for cooperative manipulation using a Leader-Follower Manipulator: As a human operator controls a master gripper through haptic feedback, the robotic gripper tracks the operator’s motion.
FIGURE 4. Designs of individual leg manipulators: (A) Heppner et al. (2015), (B) Kim and Jun (2014), (C) Irawan et al. (2016), (D) Hirose et al. (2005). While the first three designs illustrate integrated gripper-to-leg designs, the last design features grippers as modular tools.
For grasping with a single leg (Figures 3A, 5A), two pincers grip the object with the friction generated due to pressing against it. The friction force component acting towards the gripper must be greater than the normal force component squeezing the object away, i.e., μN cos(θ) > N sin(θ), or μ > tan(θ). Here θ is half of the angle subtended by the pincers. Therefore:
where pincer length lp can be expressed as
FIGURE 5. Orthogonal top views of (A) one-legged and (B) double-legged grasping. Isometric view of (C) whole-body grasping for a quadruped.
3.2.2 Multi-legged grasping
Multiple legs working together can lift larger objects better than a single gripper can, owing to a larger grasp and distributed load. Gong et al. (2023) used two legs on the same side of a 12-DOF hexapod to grasp a simulated munition buried in the sand. However, as more legs are involved in manipulation, fewer are available for locomotion. This proposes a unique challenge to walking while grasping, necessitating research on generating and transitioning to gaits with fewer legs. In Whitman et al. (2017) a hexapod with pincers on its front legs is used for both single-legged and double-legged grasping and transporting of objects. They demonstrated fluent gait transitions while simultaneously walking and lifting up a can of WD40 with one leg, a tall box with its front two legs, or a long box below its body with its middle two legs. Similarly, Tsvetkov and Ramamoorthy (2022) proposed a small-scale quadruped robot with leg-mounted manipulators used for both single- and double-legged grasping. Shaw and Sartoretti (2022) presented rapid online gait transitions for a hexapod robot from an alternating tripod gait with six legs, to a quadrupedal walking gait with four hind legs, using the front two legs for manipulation. When multiple legs are used together for manipulation, the same end-effector designs that make for predictable contact on the ground (soft, durable, rounded shapes) also ensure good object contact—thus the end-effector does not warrant modification.
However, in order to walk while holding large objects, stance stability adjustments are crucial to prevent excessive pitch of the robots. For instance, if a robot picks up a heavy enough object, resulting in a significant shift in its center of mass position, it is important to prevent the robot from toppling over. This problem was first demonstrated in 1997 when Koyachi et al. (1997) presented two hexapod prototypes, Melmantis-1 and Melmantis-2, based on their previous work on six-link “Limb Mechanisms” to both walk and manipulate (Koyachi et al., 1993). Later, in 2002, they remotely controlled Melmantis-1 to handle an object using its two front limbs. There, to maintain balance while raising two legs, the other four legs transformed into a side-by-side symmetric quadruped, expanding the robot’s support polygon (Koyachi et al., 2002). Similarly, Booysen and Reiner (2015) proposed gait transition methods during grasping for a hexapod. Besides shifting the robot’s CoM and expanding its support polygon while lifting an object, they introduced an automatic pitch correction method while walking. If the robot was at risk of toppling, it would stop, lean back (raising the front legs and lowering the hind legs), retract its gripper, or even drop the item if the average pitch angle surpassed a given safety limit. This helped the robot prioritize recovering its own balance first, before grasping the object again and attempting to continue its locomotion. More recent works have focused on removing the need for hexapods to even stop walking when recovering their balance. Deng et al. (2018) proposed a method to automatically adjust the robot’s gait when lifting heavier objects with two legs or smaller objects with a single leg. They estimated the object’s mass by sensor feedback (embedded in the end-effector) and then calculated joint motions according to the desired CoM trajectory. Considering energy efficiency in manipulation, Ding and Yang (2016) optimized the energy used against stability constraints.
Compared with front-leg manipulation, using two opposing middle legs for grasping can ensure a larger support polygon. Takahashi et al. (2000) designed a hexapod with omnidirectional manipulability and mobility. Unlike most hexapods with longitudinally symmetrical left and right legs, their design incorporated six radially symmetrical legs on a hexagonal body, minimizing the overlap between the workspaces of individual legs. Using a set of these radially opposite legs for lifting, stability during walking was greatly improved. Whitman et al. (2017) also observed that the volume of the object that their hexapod could carry using the front/middle two legs was ten times larger than what it could carry with a single leg. Further, given the modular nature of their hexapod, two of them could be attached together by the body, creating a larger dodecapod capable of using more legs for carrying larger payloads underneath the robot.
This is analyzed with an extension to the single leg case. When the object is larger than the limit of the pincers, the robot can grab the object using two front legs, being fully extended for maximum grasp size, i.e., double-legged grasping (Figures 3B, 5B). If the diameter of the object is smaller than the width, i.e., r ≤ 0.5 s, the front legs can always secure the object even with a small μ. Otherwise, if r > 0.5 s, like the single-leg case it must be ensured that μ > tan(θ) where θ is half the angle subtended by the front legs (Figure 5B). It follows that:
where robot width s can be expressed as
3.2.3 Whole-body grasping
For an even larger object, a quadruped or hexapod robot can sacrifice its locomotion capabilities by using all its legs to achieve whole-body grasping (Figures 3C, 5C). We observe that for most effective grasping, the robot design must be radially symmetrical (E.g., the base joints of legs must form a square for a quadruped or a regular hexagon for a hexapod). In this case, θ is dependent on the distance d between the base joints and the robot’s centre. For an n-legged robot, d can be expressed as
where sin(θ) can be expressed by
Our manipulation model analysis is presented in Figure 6. We can infer a strong correlation between our method of analysis and experimental results in real-life examples, helping validate our simplified model and analyses. Generally speaking, by grasping with a single leg the robot can grasp relatively small objects, depending on the pincer length lp. While double-legged grasping has an extended area for bigger objects than a single leg, most robots can only manipulate small-size objects in the double-legged grasping area since the robots usually cannot fully extend legs for big object manipulation due to the lack of motor torque, joint angles and the position shift of CoM. In comparison, whole-body grasping can safely grasp objects roughly the robot’s size. Hexapods can use all legs to grasp larger objects than quadrupeds if the length of the legs and the width of the robot are the same.
FIGURE 6. The manipulation model analysis depicts the required friction coefficient μ vs. object size normalized against robot leg length r/l. The data points plotted are listed in Supplementary Appendix Table SA1 When not provided in literature, respective friction coefficients are approximated depending on the material of the manipulator surface: For metal μ is 0.35 (EngineeringToolBox, 2004) (static coefficient between steel and polystyrene is 0.3–0.35); and for plastic μ is 0.5 (EngineeringToolBox, 2004) (static coefficient between polystyrene and polystyrene is 0.5).
Instead of using some of the legs to grasp an object, the whole robot can behave as a mobile gripper that walks towards an object and grasps it with all legs (i.e., whole-body grasping). A legged robot will not be able to continue walking with the large object after grasping it with all of its legs, but it can still be transported externally, e.g., via a cable from a boat, crane, or flying platform (Figure 7). One of the key advantages there is that, unlike a typical crane end-effector, the robot can walk on the ground to systematically search for the object, and/or reach areas occluded from the crane by overhangs or ledges. The tether can then be used to recover both the object and robot, after the robot has securely grasped it with all legs. In this case, the need for walking ends when the object is found, which simplifies the problem of lifting the found object and thus enables the potential for a relatively smaller robot to transport a relatively larger object. Besides walking and grasping, climbing is another main challenge for whole-body manipulation since the robot must often climb over thean object to then grasp it. Notably, Gong et al. (2022) designed a four-DoF walking robot by adding coxa joints on a seven-link Klann mechanism, which is able to climb onto a rectangular box and then grasp it using all legs. While two (drive) DOFs are responsible for generating periodic gait patterns (Figure 7A), the other two (lift) DoFs are used to raise the front legs separately and climb onto the object (Figure 7B) and then securely press its legs inward onto it (Figure 7C, generating sufficient friction for a secure grasp. Both the robot and the object can be manually retracted by a tether (Figure 7D).
FIGURE 7. Tethered to a boat, crane, or flying platform, the Klann robot presented in Gong et al. (2022) can (A) walk to the object, (B) climb the object (C) grasp the object, and (D) be pulled by the tether while grasping the object.
Given the growing popularity of legged quadrupeds with the team Cerberus from ETH-Zurich winning DARPA’s Subterranean Challenge in 2021, as well as the increasing availability of commercial quadruped dog-scale robots like Spot mini, Ghost, ANYmal, and others, manipulation with walking legs is a promising direction. Even legs may at times be less reliable and precise than industrial manipulators, legged robots can still carry a variety of objects in many relevant scenarios, without requiring an additional expensive manipulator appendages.
3.3 Dedicated non-locomotive arms
More sophisticated tasks may require dexterous end-effectors that are more delicate and multi-functional than legs. In this section, we discuss systems in which a dedicated arm was added to a legged robot that is different from the walking legs. Because they are not used for locomotion, these dedicated appendages can be specialized for fine sensing, grasping, and manipulation, or can be designed to lift heavier objects. However, equipping a legged robot with a dedicated arm will also present some challenges (e.g., stability, and cooperation between arms and legs) as discussed in the following paragraphs.
Here, we will reserve the term “legs” for appendages that are meant to carry a significant portion of the body’s weight during locomotion, and non-locomoting appendages as “arms”. Sometimes these robots have a giraffe-like body plan with a sensorized arm on top for high reach, however, they also include bipeds (Maniatopoulos et al., 2016), centaurs (Schwarz et al., 2017), octopuses (Mazzolai et al., 2012) and many different form factors. Note that any brachiating robots (Lo et al., 2017) (robots that swing like apes) would be a special case here, but to our knowledge at this time, few, if any brachiating robots have been demonstrated for manipulation of the environment and could rather be categorized as adding an inverted gripper for the ground to the legs.
Thus, arguably the most impressive robotic tasks performed by legged robots have used an added arm. Bigdog was first demonstrated to be capable of throwing a 16.5 kg block over 17 feet using a hydraulic manipulator, including the gripper, installed on top of its body (Murphy et al., 2012). In 2016, Boston Dynamics equipped the Spotmini with an electric robotic arm. With this, Spotmini could complete everyday tasks like opening doors or stabilizing an object in the world frame while the robot’s base moved around, showing great cooperation between the legs, body, and arm (BostonDynamics, 2018) (Figure 8A).
FIGURE 8. Legged robots equipped with dedicated non-locomotive arms can perform various useful tasks, such as (A) opening doors (Spot BostonDynamics, 2018), (B) manipulation on structured terrain (ANYmal Ferrolho et al. (2020)), and (C) lifting [CB-i Gams et al. (2015)] or (D) pushing large objects [CENTAURO Polverini et al. (2020)].
Using a legged platform for arms enables adaptations that are not possible with wheeled robots. A legged robot with an arm can dynamically manipulate objects on challenging terrain, including stairs and ladders. For example, Ferrolho et al. (2020) developed a control framework allowing ANYmal-B to complete pick-and-place tasks on various rough terrains (Figure 8B). Ma et al. (2022) later combined a learning-based locomotion policy with a model-based manipulation controller to make ANYmal-C track a desired end-effector position for its arm when trotting on challenging terrains. Moreover, compared to wheeled platforms, legged platforms can significantly improve the arm’s manipulation capabilities in terms of reachability (workspace), payload capacity, and manipulation speed since legged platforms often have more degrees of freedom resulting in finer control of the base’s position (especially height) and orientation. Specifically, as stated in Murphy et al. (2012), the knees and hind legs can assist the arm during lifting and throwing motions. Finally, legged platforms are able to change foot positions for bracing, as discussed in Section 3.1.1.
Another advantage is that giraffe-like platforms can achieve baseline performance with almost independent controllers for locomotion and manipulation, and then advance to more whole body optimization as needed. This is different from standard mobile manipulation with a wheeled platform, where the base platform cannot reconfigure as much during manipulation. Zimmermann et al. (2021) combine Boston Dynamics’ Spotmini with a light-weight robotic arm and a gripper. They propose a straightforward trajectory optimization method based on a simple dynamic model of the platform, where Spotmini is treated as a black box. Experiments demonstrate that the combined platform is capable of performing dynamic grasping tasks using a feedforward controller. Contrary to model-based learning methods, model-free learning methods do not incur an increase in computational cost due to the complexity of the underlying dynamic model. Ma et al. (2022) combine a learning-based policy for locomotion with a model-based controller for manipulation. They train the locomotion policy with known random wrench disturbances, obtained from the dynamic plans of the model-based manipulator controller. Without retraining, the combined controller adapts to a variety of manipulator configurations. In contrast, Yao et al. (2022) use a policy based on reinforcement learning to discover the dynamic representation of different arms for rapid migration, and then feed the estimated dynamic parameters to the low-level model-based locomotion controller. In addition to hierarchical control pipeline for legged manipulators, Fu et al. (2022) train a unified policy for the whole system with reinforcement learning. Their policy demonstrates coordination between the legs and arms as well as dynamic control over them.
Furthermore, an increasing number of works have considered giraffe-like platform as a whole-body optimization problem. Relying on online hierarchical optimization, ANYmal-B can perform reactive human-robot collaboration and body posture optimization to increase the manipulability of its onboard arm (Bellicoso et al., 2019). More recently, Ewen et al. (2021) improved their work from immutable contact force to online optimized force trajectory to ensure dynamic feasibility and stability of the platform while Sleiman et al. (2021) presented the first holistic MPC framework combining dynamic locomotion and manipulation for ANYmal-C. Chiu et al. (2022) extended the whole-body MPC controller to generate collision-free motions during the coordination of locomotion and manipulation. Morlando et al. (2022) proposed a whole body controller to transport a non-prehensile object with a giraffe-like platform.
An onboard arm can be specialized to reach a large volume around the robot, which the legs could not easily reach without drastic morphological changes to the robot. In other words, if the arm is mounted on the top of the body and can rotate fully around a vertical axis, its actual workspace can approach a sphere and completely encompass the body of the robot. Therefore, a specialized arm can be used to tackle complex real-world tasks such as rotating a handwheel, handling articulated objects, and unlatching a door. (Sleiman et al., 2021; Ferrolho et al., 2022; Mittal et al., 2022), which would be much harder to approach using locomotive legs. In addition, an onboard arm can reach the legs themselves, which may prove useful if the legs need to be repaired or untangled.
However, because the legs and body occlude the arm’s workspace, the arm’s footprint on the ground may be smaller than that of the combined legs. Thus, if the goal is primarily to manipulate objects on the floor, legs may be better suited (Heppner et al., 2015). Furthermore, the statically stable workspace of the arm may be a smaller subset of the kinematic workspace. In other words, if a long arm is heavy or holding a heavy object, the configuration of the legs will be crucial in preventing the robot from tipping over. For example, the HyQ robot is equipped with a hydraulic manipulator (HyArm (Rehman et al., 2016)) at the front-middle of the body. The HyArm weighs 12.5 kg and is capable of carrying up to 10 kg of payload. The arm placement dramatically shifts the robot’s CoM and affects its balance. Rehman et al. (2016) proposed a new control framework through optimizing the Ground Reaction Forces (GRFs) of a quadruped and compensating for external/internal disturbances induced by the manipulator. The integrated payload estimator module is utilized to estimate an unknown payload carried by the arm, in order to update the robot’s center of mass (CoM) position and control its stability.
Towards even more human-like behavior, some robots, specifically bipeds and centaurs, have two onboard arms. Different from the double-legged grasping discussed in Section 3.2.2, dedicated dual-arm setups have more degrees of freedom and more flexible end effectors. Dual arms have the advantage of being able to directly mimic human motions, either repetitive motions or human tele-operated motions (Kim et al., 1999; Kulić et al., 2012; Zuher and Romero, 2012; Gams et al., 2022; Zhou et al., 2022). In addition, dual arms can work together for large object manipulation Gams et al. (2015) (Figure 8C) and hand-to-hand manipulation Vahrenkamp et al. (2009). However, as a result of their hefty upper body, centaurs may experience instability when manipulating larger objects. Polverini et al. (2020) presents a control architecture for pushing a heavy object by optimizing the contact force with the environment (Figure 8D). For additional information on manipulation with bipeds and centaurs, we refer the reader to Gams et al. (2022), which studies humanoid robot learning for manipulation capabilities, and Rehman et al. (2018), which provides an overview of centaur-style robots with dual arm manipulator systems. Another way of tackling the stability problem for very large objects is to have multiple robots working together, as described in the next section.
3.4 Legged teams for manipulation
A team of legged robots may work together to manipulate objects that are too large, heavy, or awkwardly shaped for a single robot. Although the work discussed in previous sections has shown that individual robots can manipulate a wide variety of objects in diverse applications, there are some key restrictions in their abilities due to gripper size, ground support based on body/leg dimensions, and actuator torque limits (Figure 2). On the other hand, a team of robots can bring additional support/grasping points to securely hold a large object, as well as distribute the object weight among team members to increase the total manageable payload capacity and/or size (Hara et al., 1999; Hamed et al., 2019; Chang and Tao, 2022). Furthermore, even in cases where a single robot is capable of manipulating an object, the object can be cooperatively relayed from one robot to the next, thereby expanding the overall range and speed of the operation. For example, Hu and Gu (2001) leveraged this principle by using robots to pass a football (Figure 9A). More generally, collaboration is especially useful in cases where multiple general purpose robots are available rather than a large, special-purpose robot.
FIGURE 9. Main classes of multi-Robot manipulation works to date: (A) soccer playing (Hu and Gu, 2001), (B) carrying (Hara et al., 1999), (C) towing (Christensen et al., 2016), (D) pushing (Mataric et al., 1995), (E) lifting (Fujita and Kimura, 1998), and (F) heterogeneous robot teams (Govindaraj et al., 2021). Image by and courtesy of Yixian Qiu.
One way that legged robots can collaboratively manipulate a large and/or heavy object is by lifting and then carrying it together. The team of robots must first move to designated locations around the object, after which they can perform coordinated motions for lifting. Chang and Tao (2022) proposed to detect the object using a vision system that recognizes specific markers on the object to determine its location, after which a team of legged robots can move towards it until they reach appropriate collaborative lifting positions. In this case, the robots used a sideways gait to move, which was synchronized using a fuzzy motion control algorithm that selects the appropriate repeat cycles for the fast and slow pre-planned gaits to cooperatively transport the object toward the destination. This is similar to how humans adjust their posture to avoid injury when lifting/carrying objects together. It is essential for lifting robots to receive feedback about the orientation of the object, which ensures that they do not apply any unwanted redundant forces onto it. To this end, Hara et al. (1999) implemented coordinated lifting while introducing a leader-follower system using two TITAN VIII quadruped robots (Figure 9B). There, the leader was tasked with maintaining a course to a given destination, while the follower(s) adjusted their own path based on feedback of the object’s relative position without explicit communications with the leader. Instead of correcting the follower’s positions based on the state of the object, Inoue et al. (2003) proposed another approach on a small team of HOAP-1 humanoid robots, where only the follower robot(s) corrected their position relative to the leader, which still mainly maintained a path towards the goal. Hawley and Suleiman (2019a) used NAO humanoid robots to demonstrate robot-object-robot constraints (i.e., relative safe distances between robots and object) during cooperative manipulation using a Zero Moment Point controller that maintained the dynamic stability of the system. Once the object was lifted, the entire system (robots + object) could work together as a large multi-legged robot, with the object behaving as its body. McGill and Lee (2011) demonstrated a robotic system composed of two DARwIn-OP humanoids tasked with manipulating and transporting a stretcher system. After picking up the stretcher, the robots walked together as a “virtual quadruped” with careful synchronization between their legs. This synchronization relied on a two-stage motion controller, which consisted of a step controller and a walking controller to harmonize the movement of the robots. In particular, the walking controller guided the legs’ lifting motions by generating foot and torso trajectories and maintaining the robots’ and the object’s stability.
As objects get larger, it can become difficult for robots to lift or carry them on their backs: instead, an easier approach relies on dragging an object along the ground (assuming it is flat enough to do so, and the object can be dragged safely). A team of legged robots can tow such objects by attaching them to their bodies using wires, cables, or rigid rods, similar to how horses pull a wagon or mules a plow. BostonDynamics (2019) famously demonstrated how 10 Spot robots can tow a truck on a shallow uphill slope (approximately 1 degree of inclination). In general, the towing performance and efficiency of a legged robot team depend on a variety of factors such as team size, locomotion gait, and surface interactions. For example, a walking gait generates a relatively constant pulling force as compared to a running gait. That is, walking is a smooth gait in which at least a few legs are always in contact with and pushing/pulling against the ground, while running is an impulsive gait that generates pulling forces in bursts as the legs hit the ground. Christensen et al. (2016) studied and compared the towing performance of walking and running teams of hexapods (Figure 9C). Overall, they showed that walking robots have a better load sharing ability than running robots. As the number of robots increases, their results show that walking robots also outperform running ones in terms of peak pulling force. Similar to robot-ground interactions, robot-robot interactions also affect towing performance. The physical interactions between robots, which depend on the type of object connection used, can be classified into hard (e.g., rigid rods), soft (e.g., deformable objects), and hybrid (e.g., wires, cables), and naturally affect the complexity of the underlying system dynamics and the control problem. The object can be connected to the robots at single or multiple attachment points, where more points offer the system greater controllability during manipulation. Unlike rigid rods, cables can allow the system to change its effective dimensions (footprint). For instance, if the team has to transport a cable-towed object from one place to another in a cluttered environment, it may need to avoid obstacles and travel through narrow spaces. Due to their agile nature, legged robots can quickly reorient and reposition themselves to change the effective footprint of the overall system by letting the cables switch between taut and slack (e.g., approach the object when needed to reduce the effective footprint of the overall convoy). Using this feature of cables, Yang et al. (2022) demonstrated their framework for the navigation and collaborative manipulation of a cable-towed load using a team of Mini Cheetah and two A1s through narrow, cluttered, and diagonal spaces.
Communication between legged robots is another crucial aspect during multi-robot cooperative manipulation. If robots cannot exchange information, each robot can only ever rely on partial knowledge of its local environment (and, more importantly, of other robots’ states and strategies/intents). This often results in robots making independent/uncoordinated decisions, leading to worse collaboration. Mataric et al. (1995) evaluated the performance of a team of two hexapod robots in a box-pushing task with and without cooperative communication (Figure 9D). Experiments demonstrated that the team cannot complete the task at all in the absence of communications. In contrast, by communicating with each other (here, sharing sensor data), transient faults of one of the robots can be largely compensated for by the other, building a form of resilience in the system. However, communications among robots are typically expensive in practice, and limitations usually arise on bandwidth and range in practice. In order to improve communication efficiency, it is critical to share only the most pertinent information, and to limit communication to times where new/update information needs to be propagated among agents. For example, Fujita and Kimura (1998) proposed a framework for event-driven control that can initiate communication only when the task enters a problematic state, significantly improving the execution time of a two-hexapod box-lifting task (Figure 9E). In addition, cooperative communication can allow robots to select and enact distinct synergistic roles, and thus contribute to a specific task to the best of their (potentially heterogeneous) abilities. Hu and Gu (2001) proposed an approach that relies on explicit and implicit communication allowing a team of Sony AIBO robots to play different roles in a football competition, including kicking, dribbling, passing the ball, intercepting, and scoring. There, in order to optimize the performance of the team, robots are able to dynamically swap roles during the game based on the situation at hand.
Finally, teams of heterogeneous robots can take advantage of different of synergistic motion or manipulation capabilities (e.g., legged, wheeled, and flying robots, with dedicated arms or using their legs/body for manipulation). Benefiting from complementary capabilities of individual robots, system efficiency and robustness can be increased by autonomy, parallelization, and functional redundancy. In particular, heterogeneous teams that include legged robots have been proposed for different aspects of planetary exploration missions. Govindaraj et al. (2021) presented a multi-robot system consisting of a rover and a hexapod robot with a movable gantry for load manipulation and 3D printing towards the construction of lunar infrastructure (Figure 9F). They demonstrate the system’s cooperative manipulation and transport capabilities by transporting two objects to different locations simultaneously using Mantis and Veles robots. In addition, and combining the specific stability and speed advantages of wheels and legs over different types of terrains, Cordes et al. (2011, 2012) proposed the use of a wheeled rover to transport a legged robot for enhanced extra-terrestrial mobility in craters exploration. Then, using specialized grippers on its front legs, the legged robot can climb down the crater to collect geological samples. Kiener and Von Stryk (2007) also proposed a method that enables a humanoid robot mounted on a wheeled robot to track a soccer ball and kick it into a goal. The entire process is broken down into subtasks that are assigned to the individual robots by underlying finite state machines. A similar approach was used to command a collection of humanoid robots to clean a table by picking up cans and dropping them into a trash box in Lim et al. (2008).
4 Open avenues for research
4.1 Specific challenges in legged robot manipulation
The current progress of legged robot manipulation poses many theoretical and experimental challenges to be overcome to make robots more efficient and effective. In this section, we highlight and summarize three key challenges that are common to multiple types of legged robot manipulation in the open avenue: utilizing multi-functional appendages, exploiting challenging terrain, and learning-based control for legged manipulation. By providing a detailed discussion of these challenges, we believe that these challenges can be interesting opportunities for new research.
4.1.1 Utilizing multi-functional appendages
A robot that can easily adapt its appendages for use as both legs and arms can make better use of all available degrees of freedom in any unpredictable field scenario. However, finding the best body configuration for a particular application remains an open question, where we cannot solely rely on biological inspiration. For example, while many biological quadrupeds have specialized front limbs for a variety of tasks (swimming, digging, grasping, etc.), adding a fifth limb seems a more difficult task for evolution than it does for robot designers. A variety of possibilities can result from such an adaptation, including improved stability and strength, recovering from limb failure, and better dexterity. Therefore, a comprehensive understanding of how weight, power consumption, and redundancy affect multi-functionality in robots can help inform such designs.
New design principles and strategies will be key in developing such multi-functional limbs. As legs, they must be robust enough to support a heavy body and resist repeated impacts with hard substrates during locomotion. Since legged robots are often envisioned to replace wheeled robots in unstructured outdoor environments, their legs should also survive dirt and water ingress. As arms or manipulators, they must be capable of dexterous manipulation. To achieve this, haptic feedback and/or refined dexterous grippers can be employed. Additionally, this also aids in locomotion as haptic sensing provides information as to the robot’s interaction with the environment (e.g., ground reaction force). However, operating such delicate sensors and grippers in these load-bearing applications remains an open challenge. While current research on walking legs still uses simple grippers without sensor feedback (as discussed in Section 3.2), robust multi-functional appendages will most likely pave the way to a myriad of exciting research and applications, and will create new opportunities for materials research, mechanical design, as well as for advanced data filtering and processing.
4.1.2 Exploiting challenging terrain
Legged robots can be especially valuable in hazardous or unstable environments that are unsuitable for humans, such as search and rescue tasks in disaster relief scenarios, remote sample collection, or habitat construction for space exploration missions. While legged robots exhibit impressive mobility on unstructured and/or inclined terrain, even up to vertical climbing (Hong et al., 2022), manipulation in these environments requires special considerations. If the terrain is inclined, soft or granular, ground reaction forces may cause slippage or sinkage during critical manipulation tasks. Sub-surface support may be insufficient, causing the robot to trip or stumble. While some works have focused on mitigating these perturbations during locomotion in a reactive manner (Sartoretti et al., 2018), the effect of such challenging terrain on manipulation abilities has been considered less often to date.
Terrain models will likely be important. While stabilizing the manipulator using reactive control may improve performance by incorporating appropriate terrain/interaction models into a control framework (e.g., Model Predictive Control), a robot could predict how the terrain will be modified during the manipulation. This will enable the legs to choose configurations, footholds, and timing to maximize success. Planning further ahead, it may be possible to circumvent any issues in advance, leading to improved and more reliable performances during challenging manipulation tasks.
4.1.3 Learning-based control for legged manipulation
Even for pure locomotion, model-based control methods for online planning are burdened by expensive computational costs from the high number of degrees of freedom. To learn more about the control method involved in model-based approaches for legged manipulation, we suggest referring to the survey Chai et al. (2022) for an up-to-date overview. To avoid high computational costs, one of the solutions that the locomotion community has started investigating is learning-based control. In learning-based approaches, heavy computation is mostly focused upon within the training phase, yielding (often time-invariant) policies that can be used at high frequencies. However, the robustness and generalizability of these policies are often lower, as many learning-based methods rely on neural networks for function approximation, which famously obfuscate what was truly learned by the system behind a “black box.” In the long-run, these issues may be mitigated by some clever training techniques, such as privileged or imitation learning (Chen et al., 2020) or by provably-robust learning approaches (Choromanski et al., 2020). In the shorter-term, many approaches have been proposed to embed learned policies within a robust high-level control framework to ensure the system’s safety at run-time, with minimal effect on the overall performance and time complexity.
Learning-based methods have been used effectively and have demonstrated promising results for pure locomotion tasks (Miki et al., 2022), but few approaches, if any, have looked at legged manipulation tasks. We believe that learning a high-dimensional controller for whole-body manipulation is a key long-term objective in this area since current model-based controllers (e.g., MPC) still struggle with this task. In the longer-term, future research will also investigate an end-to-end control pipeline for the entire system, where the input to the neural network is raw sensor data (e.g., from cameras/LiDAR, proprioceptive, inertial, and haptic sensors), and the output is the final control output (e.g., joint angles or desired torques). Even though such an approach will inevitably further reduce explainability, such all-in-one control pipelines are much easier to implement on real robots and most likely represent the next frontier for high-performance, versatile legged robot controllers for advanced legged manipulation tasks.
4.2 New applications for legged robot manipulation
The combination of legged locomotion and the ability to manipulate objects will also open up the possibility for new robotic deployments for manipulation tasks in both industry and daily life.
4.2.1 Challenging environment applications
Combined legged locomotion and manipulation literally and figuratively open new doors for robotic access in challenging environments. While wheeled robots may be sufficient for many tasks where terrains can be initially constructed or modified for wheels (e.g., offices or warehouses), many applications remain out-of-reach of wheeled platforms (and might never be the solution). On the other hand, legged robots can enable manipulation tasks in a wide array of environments and conditions. For example, instead of sending humans into disaster sites after extreme events such as nuclear disasters, earthquakes, or floods, legged robots can be deployed to explore hazardous areas and obtain early situation reports (Siegwart et al., 2015). Legged manipulation robots can further help by manipulating the environment to sanitize or secure areas before humans can enter, through teleoperation, or eventually, through autonomous operation.
Conversely, manipulation can help increase the reachability of legged locomotion, by allowing the robot to modify its surrounding to clear obstacles (Lu et al., 2020). A common impediment to the increasing use of mobile robots is that if their pre-defined path is blocked, the only option often is to find an alternative trajectory, which does not always exist, making some areas unreachable. Legged manipulation robots can move away obstacles or modify their surrounding (e.g., move soil to create a ramp, or to reduce the steepness of an incline) to clear/create a path, showing increased potential for search-and-rescue, domestic cleaning, and elder care applications. In the long run, the fundamental advantage of legged robot manipulation will be in allowing robots to autonomously understand their environment to navigate and manipulate their surroundings, for the purpose of completing a high-level objective with minimal human supervision/action.
4.2.2 Home robotic applications
In certain industrial settings (e.g., warehouses or shop floors), the environment may be simple and flat, or could even be specifically designed for robots. In contrast, home environments are quite complex and constantly changing (e.g., stairs, gardens, and grounds with many obstacles or opportunities for small accidents), for which legged robots are naturally better suited. We believe that home robotics will be one of the most important future applications for legged robot manipulation (Linghan et al., 2022) because they will be able to improve living quality, safety, and emergency response. For example, disabled and elderly people, who have difficulty performing basic daily activities independently, may specifically benefit from a cheaper, on demand legged robotic home assistant. Legged robots can help with laundry and carrying heavy payloads (e.g., moving furniture), and can even help with other everyday tasks such as preparing food and managing medications for people who suffer from cognitive disorders. Finally, a legged robot may even help a person after a home accident, such as an elder person who has fallen down or someone experiencing a stroke, by calling for help from a phone, bringing medical supplies, helping them to stand back up or to reach a safe area in the house, or even by directly performing CPR!
5 Conclusion
This review provides a broad overview of recent work in legged robot manipulation, which can be classified into four main categories: non-prehensile manipulation, manipulation with walking legs, manipulation with a specialized arm, and legged team manipulation. Each of these approaches results in different robot designs and has different advantages. If the object to be moved is small or light, a dedicated non-locomotive arm or leg-mounted gripper can be easily added to an existing robot. Larger objects can be moved if multiple legs or multiple robots work together. Dexterity is also important and requires both precise actuation and sensing. Another consideration is specificity vs. generality, in which some end effectors are useful only for a small range of objects, while other approaches such as pushing can be implemented on any robot. With the manipulation methods mentioned in the review, legged robots are expected to work on various applications in tough terrains by actively interacting with their surroundings instead of passive interaction only. We believe legged manipulation is one of the next frontiers of robotic research. Our hope is that this review provides helpful examples and motivates additional research effort in all these exciting areas.
Author contributions
YG and GSu contributed equally to synthesizing the overall manuscript. YG developed Sections 1, 2, 3.2, and 4. GSu co-performed Sections 1, 2, 4 and developed Section 3.3. AN wrote Section 3.2. AB and RC wrote Section 3.4. JG contributed to the conception and idea of the review. KAD and GSa lead overall content organization, project administration, supervision, and figure planning. All authors contributed to the manuscript revision, read, and approved the submitted version.
Funding
This work was partly supported by the Singapore Ministry of Education Academic Research Fund Tier 1. This work was partly supported by the Office of Naval Research under a Young Investigator Award under Grant N000141912138 and in part by the Strategic Environmental Research and Development Program (SERDP) SEED under Grant MR19-1369.
Acknowledgments
The authors would like to thank Swasti Khurana and Carla Macklin for helping proofread the entire manuscript.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmech.2023.1142421/full#supplementary-material
References
Acosta-Calderon, C. A., Mohan, R. E., Zhou, C., Hu, L., Yue, P. K., and Hu, H. (2008). A modular architecture for humanoid soccer robots with distributed behavior control. Int. J. Humanoid Robotics 5, 397–416. doi:10.1142/s0219843608001522
Altuzarra, O., Şandru, B., Pinto, C., and Petuya, V. (2011). A symmetric parallel schönflies-motion manipulator for pick-and-place operations. Robotica 29, 853–862. doi:10.1017/s0263574711000063
Aoi, S., Manoonpong, P., Ambe, Y., Matsuno, F., and Wörgötter, F. (2017). Adaptive control strategies for interlimb coordination in legged robots: A review. Front. neurorobotics 11, 39. doi:10.3389/fnbot.2017.00039
Behnke, S., and Stückler, J. (2008). Hierarchical reactive control for humanoid soccer robots. Int. J. Humanoid Robotics 5, 375–396. doi:10.1142/s0219843608001510
Bellicoso, C. D., Bjelonic, M., Wellhausen, L., Holtmann, K., Günther, F., Tranzatto, M., et al. (2018). Advances in real-world applications for legged robots. J. Field Robotics 35, 1311–1326. doi:10.1002/rob.21839
Bellicoso, C. D., Krämer, K., Stäuble, M., Sako, D., Jenelten, F., Bjelonic, M., et al. (2019). Alma-articulated locomotion and manipulation for a torque-controllable robot. In 2019 international conference on robotics and automation (ICRA) (IEEE), 8477–8483
Bertoncelli, F., Ruggiero, F., and Sabattini, L. (2020). “Linear time-varying mpc for nonprehensile object manipulation with a nonholonomic mobile robot,” in 2020 IEEE international conference on robotics and automation (ICRA) (IEEE), 11032–11038.
Billard, A., and Kragic, D. (2019). Trends and challenges in robot manipulation. Science 364, eaat8414. doi:10.1126/science.aat8414
Billeschou, P., Bijma, N. N., Larsen, L. B., Gorb, S. N., Larsen, J. C., and Manoonpong, P. (2020). Framework for developing bio-inspired morphologies for walking robots. Appl. Sci. 10, 6986. doi:10.3390/app10196986
Booysen, T., and Reiner, F. (2015). “Gait adaptation of a six legged walker to enable gripping,” in 2015 pattern recognition association of South Africa and robotics and Mechatronics international conference (PRASA-RobMech) (IEEE), 195–200.
BostonDynamics (2018). Hey buddy, can you give me a hand? Available at: https://www.youtube.com/watch?v=fUyU3lKzoio.
BostonDynamics (2019). Mush, spot. mush! Available at: https://www.youtube.com/watch?v=OnWolLQSZic.
Chai, H., Li, Y., Song, R., Zhang, G., Zhang, Q., Liu, S., et al. (2022). A survey of the development of quadruped robots: Joint configuration, dynamic locomotion control method and mobile manipulation approach. Biomim. Intell. Robotics 2, 100029. doi:10.1016/j.birob.2021.100029
Chang, C.-W., and Tao, C.-W. (2022). The design and the development of a biped robot cooperation system. Processes 10, 1350. doi:10.3390/pr10071350
Chebotar, Y., Hausman, K., Zhang, M., Sukhatme, G., Schaal, S., and Levine, S. (2017). “Combining model-based and model-free updates for trajectory-centric reinforcement learning,” in International conference on machine learning, Sydney (PMLR), 703–711.
Chen, D., Zhou, B., Koltun, V., and Krähenbühl, P. (2020). “Learning by cheating,” in Conference on robot learning (PMLR), 66–75.
Chen, G., Kong, L., Li, Q., Wang, H., and Lin, Z. (2018). Complete, minimal and continuous error models for the kinematic calibration of parallel manipulators based on poe formula. Mech. Mach. Theory 121, 844–856. doi:10.1016/j.mechmachtheory.2017.11.003
Cherubini, A., Giannone, F., Iocchi, L., Nardi, D., and Palamara, P. F. (2010). Policy gradient learning for quadruped soccer robots. Robotics Aut. Syst. 58, 872–878. doi:10.1016/j.robot.2010.03.008
Chiu, J.-R., Sleiman, J.-P., Mittal, M., Farshidian, F., and Hutter, M. (2022). “A collision-free mpc for whole-body dynamic locomotion and manipulation,” in 2022 international conference on robotics and automation (ICRA) (IEEE), 4686–4693.
Choromanski, K., Pacchiano, A., Parker-Holder, J., Tang, Y., Jain, D., Yang, Y., et al. (2020). “Provably robust blackbox optimization for reinforcement learning,” in Conference on robot learning (PMLR), 683–696.
Christensen, D. L., Suresh, S. A., Hahm, K., and Cutkosky, M. R. (2016). Let’s all pull together: Principles for sharing large loads in microrobot teams. IEEE Robotics Automation Lett. 1, 1089–1096. doi:10.1109/lra.2016.2530314
Cordes, F., Ahrns, I., Bartsch, S., Birnschein, T., Dettmann, A., Estable, S., et al. (2011). Lunares: Lunar crater exploration with heterogeneous multi robot systems. Intell. Serv. Robot. 4, 61–89. doi:10.1007/s11370-010-0081-4
Cordes, F., Roehr, T. M., Kirchner, F., and Bremen, D. R. I. C. (2012). “Rimres: A modular reconfigurable heterogeneous multi-robot exploration system,” in International symposium on artificial intelligence, robotics and automatoin in space.
Corke, P. I. (2007). A simple and systematic approach to assigning denavit–hartenberg parameters. IEEE Trans. robotics 23, 590–594. doi:10.1109/tro.2007.896765
da Silva, I. J., Perico, D. H., Homem, T. P. D., and da Costa Bianchi, R. A. (2021). Deep reinforcement learning for a humanoid robot soccer player. J. Intelligent Robotic Syst. 102, 69–14. doi:10.1007/s10846-021-01333-1
Deng, H., Xin, G., Zhong, G., and Mistry, M. (2018). Object carrying of hexapod robots with integrated mechanism of leg and arm. Robotics Computer-Integrated Manuf. 54, 145–155. doi:10.1016/j.rcim.2017.11.014
Ding, X., and Yang, F. (2016). Study on hexapod robot manipulation using legs. Robotica 34, 468–481. doi:10.1017/S0263574714001799
EngineeringToolBox (2004). Friction - friction coefficients and calculator. Available at: https://www.engineeringtoolbox.com/friction-coefficients-d_778.html.
Ewen, P., Sleiman, J.-P., Chen, Y., Lu, W.-C., Hutter, M., and Vasudevan, R. (2021). “Generating continuous motion and force plans in real-time for legged mobile manipulation,” in 2021 IEEE international conference on robotics and automation (ICRA) (IEEE), 4933–4939.
Fang, L., and Gao, F. (2018). Type design and behavior control for six legged robots. Chin. J. Mech. Eng. 31, 59–12. doi:10.1186/s10033-018-0259-9
Ferrolho, H., Ivan, V., Merkt, W., Havoutis, I., and Vijayakumar, S. (2022). Roloma: Robust loco-manipulation for quadruped robots with arms. arXiv preprint arXiv:2203.01446.
Ferrolho, H., Merkt, W., Ivan, V., Wolfslag, W., and Vijayakumar, S. (2020). “Optimizing dynamic trajectories for robustness to disturbances using polytopic projections,” in 2020 IEEE/RSJ international conference on intelligent robots and systems (IROS) (IEEE), 7477–7484.
Flemmer, R., and Flemmer, C. (2014). A humanoid robot for research into kicking rugby balls. Tech. rep., Res. Rep., 1–14. available at: http://www. massey. ac. nz/˜ rcflemme/A.
Friedmann, M., Kiener, J., Petters, S., Thomas, D., Von Stryk, O., and Sakamoto, H. (2008). Versatile, high-quality motions and behavior control of a humanoid soccer robot. Int. J. Humanoid Robotics 5, 417–436. doi:10.1142/s0219843608001509
Fu, Z., Cheng, X., and Pathak, D. (2022). “Deep whole-body control: Learning a unified policy for manipulation and locomotion,” in Conference on robot learning (CoRL).
Fujita, T., and Kimura, H. (1998). Tight cooperative working system by multiple robots. IROS, 1405–1410.
Gams, A., Petrič, T., Nemec, B., and Ude, A. (2022). Manipulation learning on humanoid robots. Curr. Robot. Rep. 1–13, 97–109. doi:10.1007/s43154-022-00082-9
Gams, A., Ude, A., and Morimoto, J. (2015). “Accelerating synchronization of movement primitives: Dual-arm discrete-periodic motion of a humanoid robot,” in 2015 IEEE/RSJ international conference on intelligent robots and systems (IROS) (IEEE), 2754–2760.
Gong, Y., Behr, A., Graf, N., Chen, K., Gong, Z., and Daltorio, K. (2022). A walking claw for tethered object retrieval. J. Mech. Robotics, 1–15. doi:10.1115/1.4055812
Gong, Y., Grezmak, J., Zhou, J., Graf, N., Gong, Z., Carmichael, N., et al. (2023). Surf zone exploration with crab-like legged robots. IEEE Int. Conf. Robotics Automation 023.
Govindaraj, S., Brinkmann, W., Colmenero, F., Nieto, I. S., But, A., de Benedetti, M., et al. (2021). “Building a lunar infrastructure with the help of a heterogeneous (semi) autonomous multi-robot-team,” in Proceedings of the 72nd international astronautical congress 2021 (IAC-2021.
Grieco, J., Prieto, M., Armada, M., and Gonzalez de Santos, P. (1998). A six-legged climbing robot for high payloads. In Proceedings of the 1998 IEEE international conference on control applications (cat. No.98CH36104). 1, 446–450. doi:10.1109/CCA.1998.728488
Hamed, K. A., Ma, W.-L., Kamidi, V. R., and Ames, A. D. (2019). Hierarchical feedback control for complex hybrid models of multiagent legged robotic systems. arXiv 2019.
Hara, M., Fukuda, M., Nishibayashi, H., Aiyama, Y., Ota, J., and Arai, T. (1999). “Motion control of cooperative transportation system by quadruped robots based on vibration model in walking,” in Proceedings 1999 IEEE/RSJ international conference on intelligent robots and systems. Human and environment friendly robots with high intelligence and emotional quotients (cat. No. 99CH36289) (IEEE), 3, 1651–1656.
Hawley, L., and Suleiman, W. (2019a). Control framework for cooperative object transportation by two humanoid robots. Robotics Aut. Syst. 115, 1–16. doi:10.1016/j.robot.2019.02.003
Hawley, L., and Suleiman, W. (2019b). Control framework for cooperative object transportation by two humanoid robots. Robotics Aut. Syst. 115, 1–16. doi:10.1016/j.robot.2019.02.003
Heppner, G., Roennau, A., Oberländer, J., Klemm, S., and Dillmann, R. (2015). Laurope-six legged walking robot for planetary exploration participating in the spacebot cup. WS Adv. Space Technol. Robotics Automation 2, 69–76.
Hirose, S., Yokota, S., Torii, A., Ogata, M., Suganuma, S., Takita, K., et al. (2005). “Quadruped walking robot centered demining system-development of titan-ix and its operation,” in Proceedings of the 2005 IEEE international conference on robotics and automation (IEEE), 1284–1290.
Hong, S., Um, Y., Park, J., and Park, H.-W. (2022). Agile and versatile climbing on ferromagnetic surfaces with a quadrupedal robot. Sci. Robotics 7, eadd1017. doi:10.1126/scirobotics.add1017
Hooke, R. L., Martín Duque, J. F., and Pedraza Gilsanz, J. d. (2013). Land transformation by humans: A review. Ene 12, 4–10. doi:10.1130/gsat151a.1
Hu, H., and Gu, D. (2001). Reactive behaviours and agent architecture for sony legged robots to play football. Industrial Robot Int. J. 28, 45–54. doi:10.1108/01439910110365421
Inoue, K., Nishihama, Y., Arai, T., and Mae, Y. (2002). Mobile manipulation of humanoid robots-body and leg control for dual arm manipulation. In Proc. 2002 IEEE Int. Conf. Robotics Automation (Cat. No.02CH37292) 3, 2259–2264. doi:10.1109/ROBOT.2002.1013568
Inoue, K., Ooe, K., and Lee, S. (2010). “Pushing methods for working six-legged robots capable of locomotion and manipulation in three modes,” in 2010 IEEE international conference on robotics and automation, 4742–4748. doi:10.1109/ROBOT.2010.5509220
Inoue, Y., Tohge, T., and Iba, H. (2003). “Cooperative transportation by humanoid robots: Learning to correct positioning,” in HIS, Berlin (Citeseer), 1124–1134.
Irawan, A., Yin, T. Y., Lezaini, W. M. N. W., Razali, A. R., and Yusof, M. S. M. (2016). “Reconfigurable foot-to-gripper leg for underwater bottom operator, hexaquad,” in 2016 IEEE international conference on underwater system Technology: Theory and applications (USYS) (IEEE), 94–99.
Ji, Y., Li, Z., Sun, Y., Peng, X. B., Levine, S., Berseth, G., et al. (2022). Hierarchical reinforcement learning for precise soccer shooting skills using a quadrupedal robot. ArXiv abs/2208.01160.
Jouandeau, N., and Hugel, V. (2014). “Optimization of parametrised kicking motion for humanoid soccer player,” in 2014 IEEE international conference on autonomous robot systems and competitions (ICARSC), 241–246.
Kalouche, S., Rollinson, D., and Choset, H. (2015). “Modularity for maximum mobility and manipulation: Control of a reconfigurable legged robot with series-elastic actuators,” in 2015 IEEE international symposium on safety, security, and rescue robotics (SSRR) (IEEE), 1–8.
Kiener, J., and Von Stryk, O. (2007). “Cooperation of heterogeneous, autonomous robots: A case study of humanoid and wheeled robots,” in 2007 IEEE/RSJ international conference on intelligent robots and systems (IEEE), 959–964.
Kim, J.-Y., and Jun, B.-H. (2014). Design of six-legged walking robot, little crabster for underwater walking and operation. Adv. Robot. 28, 77–89. doi:10.1080/01691864.2013.856832
Kim, K., Spieler, P., Lupu, E.-S., Ramezani, A., and Chung, S.-J. (2021). A bipedal walking robot that can fly, slackline, and skateboard. Sci. Robotics 6, eabf8136. doi:10.1126/scirobotics.abf8136
Kim, M., Kang, S., Lee, S., Chung, W., Cho, K., and Lee, C.-W. (1999). “Development of a humanoid robot centaur-design, human interface, planning and control of its upper-body,” in IEEE SMC’99 conference proceedings. 1999 IEEE international conference on systems, man, and cybernetics (cat. No. 99CH37028) (IEEE), 4, 948–953.
Klein, M. A., Hart, C., and Quinn, R. D. (2016). “Autonomous, precision snow removal in an urban environment, featuring ultra-wideband beacons,” in Proceedings of the 29th international technical meeting of the satellite division of the institute of navigation (ION GNSS+ 2016), 1178–1183.
Koyachi, N., Adachi, H., Izumi, M., and Hirose, T. (2002). “Control of walk and manipulation by a hexapod with integrated limb mechanism: Melmantis-1,” in Proceedings 2002 IEEE international conference on robotics and automation (cat. No. 02CH37292) (IEEE), 4, 3553–3558.
Koyachi, N., Arai, T., Adachi, H., and Itoh, Y. (1993). “Integrated limb mechanism of manipulation and locomotion for dismantling robot-basic concept for control and mechanism,” in Proceedings of 1993 IEEE/tsukuba international workshop on advanced robotics (IEEE), 81–84.
Koyachi, N., Arai, T., Adachi, H., Murakami, A., and Kawai, K. (1997). “Mechanical design of hexapods with integrated limb mechanism: Melmantis-1 and melmantis-2,” in 1997 8th international conference on advanced robotics. Proceedings. ICAR’97 (IEEE), 273–278.
Kulić, D., Ott, C., Lee, D., Ishikawa, J., and Nakamura, Y. (2012). Incremental learning of full body motion primitives and their sequencing through human motion observation. Int. J. Robotics Res. 31, 330–345. doi:10.1177/0278364911426178
Leung, B., Bijma, N., Baird, E., Dacke, M., Gorb, S., and Manoonpong, P. (2020). Rules for the leg coordination of dung beetle ball rolling behaviour. Sci. Rep. 10, 9278–8. doi:10.1038/s41598-020-66248-7
Leung, B., Thor, M., and Manoonpong, P. (2018). “Modular neural control for bio-inspired walking and ball rolling of a dung beetle-like robot,” in Alife 2018: The 2018 conference on artificial life (MIT Press), 335–342.
Lim, H., Kang, Y., Lee, J., Kim, J., and You, B.-J. (2008). “Multiple humanoid cooperative control system for heterogeneous humanoid team,” in RO-MAN 2008-the 17th IEEE international symposium on robot and human interactive communication (IEEE), 231–236.
Linghan, M., Cong, W., Shengguo, C., Qifeng, Z., and Baoqi, Z. (2022). “Dogget: A legged manipulation system in human environments,” in 2022 12th international conference on CYBER Technology in automation, control, and intelligent systems (CYBER) (IEEE), 1130–1135.
Liu, J., Iacoponi, S., Laschi, C., Wen, L., and Calisti, M. (2020). Underwater mobile manipulation: A soft arm on a benthic legged robot. IEEE Robot. Autom. Mag. 27, 12–26. doi:10.1109/MRA.2020.3024001
Liu, J., Tan, M., and Zhao, X. (2007). Legged robots—An overview. Trans. Inst. Meas. Control 29, 185–202. doi:10.1177/0142331207075610
Liu, Y., and Liu, G. (2009). Modeling of tracked mobile manipulators with consideration of track–terrain and vehicle–manipulator interactions. Robotics Aut. Syst. 57, 1065–1074. doi:10.1016/j.robot.2009.07.007
Lo, A. K.-Y., Yang, Y.-H., Lin, T.-C., Chu, C.-W., and Lin, P.-C. (2017). Model-based design and evaluation of a brachiating monkey robot with an active waist. Appl. Sci. 7, 947. doi:10.3390/app7090947
Lu, B., Tam, B., and Kottege, N. (2020). Autonomous obstacle legipulation with a hexapod robot. arXiv preprint arXiv:2011.06227.
Lynch, K. M., and Park, F. C. (2018). Modernrobotics. Available at: https://modernrobotics.northwestern.edu/nu-gm-book-resource/grasping-and-manipulation/.
Ma, Y., Farshidian, F., Miki, T., Lee, J., and Hutter, M. (2022). Combining learning-based locomotion policy with model-based manipulation for legged mobile manipulators. IEEE Robotics Automation Lett. 7, 2377–2384. doi:10.1109/lra.2022.3143567
Maniatopoulos, S., Schillinger, P., Pong, V., Conner, D. C., and Kress-Gazit, H. (2016). “Reactive high-level behavior synthesis for an atlas humanoid robot,” in 2016 IEEE international conference on robotics and automation (ICRA) (IEEE), 4192–4199.
Mataric, M. J., Nilsson, M., and Simsarin, K. T. (1995). “Cooperative multi-robot box-pushing,” in Proceedings 1995 IEEE/RSJ international conference on intelligent robots and systems. Human robot interaction and cooperative robots (IEEE), 3, 556–561.
Mazzolai, B., Margheri, L., Cianchetti, M., Dario, P., and Laschi, C. (2012). Soft-robotic arm inspired by the octopus: Ii. from artificial requirements to innovative technological solutions. Bioinspiration biomimetics 7, 025005. doi:10.1088/1748-3182/7/2/025005
McGhee, R. B., and Frank, A. A. (1968). On the stability properties of quadruped creeping gaits. Math. Biosci. 3, 331–351. doi:10.1016/0025-5564(68)90090-4
McGill, S. G., and Lee, D. D. (2011).Cooperative humanoid stretcher manipulation and locomotion. In 2011 11th IEEE-RAS international conference on humanoid robots. IEEE, 429–433.
Miki, T., Lee, J., Hwangbo, J., Wellhausen, L., Koltun, V., and Hutter, M. (2022). Learning robust perceptive locomotion for quadrupedal robots in the wild. Sci. Robotics 7, eabk2822. doi:10.1126/scirobotics.abk2822
Mittal, M., Hoeller, D., Farshidian, F., Hutter, M., and Garg, A. (2022). “Articulated object interaction in unknown scenes with whole-body mobile manipulation,” in 2022 IEEE/RSJ international conference on intelligent robots and systems (IROS) (IEEE), 1647–1654.
Morlando, V., Selvaggio, M., and Ruggiero, F. (2022). “Nonprehensile object transportation with a legged manipulator,” in 2022 international conference on robotics and automation (ICRA) (IEEE), 6628–6634.
Muratore, F., Gruner, T., Wiese, F., Belousov, B., Gienger, M., and Peters, J. (2021). “Neural posterior domain randomization,” in CoRL.
Murphy, M. P., Stephens, B., Abe, Y., and Rizzi, A. A. (2012). 8387. SPIE, 339–348.High degree-of-freedom dynamic manipulation. Unmanned Syst. Technol. XIV, Baltimore.
Nagabandi, A., Yang, G., Asmar, T., Pandya, R., Kahn, G., Levine, S., et al. (2018). “Learning image-conditioned dynamics models for control of underactuated legged millirobots,” in 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS) (IEEE), 4606–4613.
Nishihama, Y., Inoue, K., Arai, T., and Mae, Y. (2003). “Mobile manipulation of humanoid robots -control method for accurate manipulation,” in Proceedings 2003 IEEE/RSJ international conference on intelligent robots and systems (IROS 2003) (cat. No.03CH37453), 2, 1914–1919. doi:10.1109/IROS.2003.1248924
Peng, X. B., Berseth, G., Yin, K., and Van De Panne, M. (2017). Deeploco: Dynamic locomotion skills using hierarchical deep reinforcement learning. ACM Trans. Graph. (TOG) 36, 1–13. doi:10.1145/3072959.3073602
Polverini, M. P., Laurenzi, A., Hoffman, E. M., Ruscelli, F., and Tsagarakis, N. G. (2020). Multi-contact heavy object pushing with a centaur-type humanoid robot: Planning and control for a real demonstrator. IEEE Robotics Automation Lett. 5, 859–866. doi:10.1109/lra.2020.2965906
Prattichizzo, D., and Trinkle, J. C. (2016). “Grasping,” in Springer handbook of robotics (Springer), 955–988.
Raibert, M., Blankespoor, K., Nelson, G., and Playter, R. (2008). Bigdog, the rough-terrain quadruped robot. IFAC Proc. Vol. 41, 10822–10825. doi:10.3182/20080706-5-kr-1001.01833
Rehman, B. U., Caldwell, D. G., and Semini, C. (2018). “Centaur robots-a survey,” in Human-centric robotics: Proceedings of CLAWAR 2017: 20th international conference on climbing and walking robots and the support technologies for mobile machines (World Scientific), 247–258.
Rehman, B. U., Focchi, M., Lee, J., Dallali, H., Caldwell, D. G., and Semini, C. (2016). Towards a multi-legged mobile manipulator. In 2016 IEEE international conference on robotics and automation (ICRA) (IEEE), 3618–3624.
Rigo, A., Chen, Y., Gupta, S. K., and Nguyen, Q. (2022). Contact optimization for non-prehensile loco-manipulation via hierarchical model predictive control. arXiv preprint arXiv:2210.03442.
Rober, M. (2021). World’s longest field goal-robot vs nfl kicker. Available at: https://www.youtube.com/watch?v=P_6my53IlxY.
Roennau, A., Heppner, G., Nowicki, M., and Dillmann, R. (2014). “Lauron v: A versatile six-legged walking robot with advanced maneuverability,” in 2014 IEEE/ASME international conference on advanced intelligent Mechatronics (IEEE), 82–87.
Ruggiero, F., Lippiello, V., and Siciliano, B. (2018). Nonprehensile dynamic manipulation: A survey. IEEE Robotics Automation Lett. 3, 1711–1718. doi:10.1109/lra.2018.2801939
Sartoretti, G., Shaw, S., Lam, K., Fan, N., Travers, M., and Choset, H. (2018). “Central pattern generator with inertial feedback for stable locomotion and climbing in unstructured terrain,” in 2018 IEEE international conference on robotics and automation (ICRA) (IEEE), 5769–5775.
Sayyad, A., Seth, B., and Seshu, P. (2007). Single-legged hopping robotics research—A review. Robotica 25, 587–613. doi:10.1017/s0263574707003487
Schmid, A. J., Gorges, N., Goger, D., and Worn, H. (2008). “Opening a door with a humanoid robot using multi-sensory tactile feedback,” in 2008 IEEE international conference on robotics and automation (IEEE), 285–291.
Schroer, R., Boggess, M., Bachmann, R., Quinn, R., and Ritzmann, R. (2004). “Comparing cockroach and whegs robot body motions,” in IEEE international conference on robotics and automation, 2004. Proceedings. ICRA ’04. 2004, 4, 3288–3293. doi:10.1109/ROBOT.2004.1308761
Schwarz, M., Rodehutskors, T., Droeschel, D., Beul, M., Schreiber, M., Araslanov, N., et al. (2017). Nimbro rescue: Solving disaster-response tasks with the mobile manipulation robot momaro. J. Field Robotics 34, 400–425. doi:10.1002/rob.21677
Sereinig, M., Werth, W., and Faller, L.-M. (2020). A review of the challenges in mobile manipulation: Systems design and robocup challenges. e i Elektrotechnik und Inf. 137, 297–308. doi:10.1007/s00502-020-00823-8
Shankhdhar, P., Omer, A., Rawat, A. S., Thakur, A. K., Kataria, Y., Shukla, S., et al. (2022). “Dynamics and control of a mobile kicking robot,” in 2022 IEEE India council international subsections conference (INDISCON) (IEEE), 1–10.
Shaw, S., and Sartoretti, G. (2022). Keyframe-based cpg for stable gait design and online transitions in legged robots. IEEE Conf. Decis. Control.
Siegwart, R., Hutter, M., Oettershagen, P., Burri, M., Gilitschenski, I., Galceran, E., et al. (2015). “Legged and flying robots for disaster response,” in World engineering conference and convention (WECC), Kyoto (ETH-Zürich).
Silva, M. F., and Machado, J. T. (2012). A literature review on the optimization of legged robots. J. Vib. Control 18, 1753–1767. doi:10.1177/1077546311403180
Sleiman, J.-P., Farshidian, F., Minniti, M. V., and Hutter, M. (2021). A unified mpc framework for whole-body dynamic locomotion and manipulation. IEEE Robotics Automation Lett. 6, 4688–4695. doi:10.1109/lra.2021.3068908
Sombolestan, M., and Nguyen, Q. (2022). Hierarchical adaptive loco-manipulation control for quadruped robots. arXiv preprint arXiv:2209.13145.
Song, S.-H., Kim, M.-S., Rodrigue, H., Lee, J.-Y., Shim, J.-E., Kim, M.-C., et al. (2016). Turtle mimetic soft robot with two swimming gaits. Bioinspiration biomimetics 11, 036010. doi:10.1088/1748-3190/11/3/036010
Stone, P. (2007). Intelligent autonomous robotics: A robot soccer case study. Synthesis Lect. Artif. Intell. Mach. Learn. 1, 1–155. doi:10.2200/s00090ed1v01y200705aim001
Stüber, J., Zito, C., and Stolkin, R. (2020). Let’s push things forward: A survey on robot pushing. Front. Robotics AI 7, 8. doi:10.3389/frobt.2020.00008
Suzumori, K., and Faudzi, A. A. (2018). Trends in hydraulic actuators and components in legged and tough robots: A review. Adv. Robot. 32, 458–476. doi:10.1080/01691864.2018.1455606
Takahashi, Y., Arai, T., Mae, Y., Inoue, K., and Koyachi, N. (2000). “Development of multi-limb robot with omnidirectional manipulability and mobility,” in Proceedings. 2000 IEEE/RSJ international conference on intelligent robots and systems (IROS 2000) (cat. No.00CH37113), 3. doi:10.1109/IROS.2000.895266
Teixeira, H., Silva, T., Abreu, M., and Reis, L. P. (2020). Humanoid robot kick in motion ability for playing robotic soccer. 2020 IEEE international conference on autonomous robot systems and competitions (ICARSC), 34–39.
Thakar, S., Srinivasan, S., Al-Hussaini, S., Bhatt, P. M., Rajendran, P., Jung Yoon, Y., et al. (2023). A survey of wheeled mobile manipulation: A decision-making perspective. J. Mech. Robotics 15, 020801. doi:10.1115/1.4054611
Thor, M., Strøm-Hansen, T., Larsen, L. B., Kovalev, A., Gorb, S. N., Baird, E., et al. (2018). A dung beetle-inspired robotic model and its distributed sensor-driven control for walking and ball rolling. Artif. Life Robotics 23, 435–443. doi:10.1007/s10015-018-0456-8
Tsvetkov, Y., and Ramamoorthy, S. (2022). “A novel design and evaluation of a dactylus-equipped quadruped robot for mobile manipulation,” in 2022 IEEE/RSJ international conference on intelligent robots and systems, Kyoto (IROS). 1633–1638. doi:10.1109/IROS47612.2022.9982229
Vahrenkamp, N., Berenson, D., Asfour, T., Kuffner, J., and Dillmann, R. (2009). Humanoid motion planning for dual-arm manipulation and re-grasping tasks. In 2009 IEEE/RSJ international conference on intelligent robots and systems (IEEE), 2464–2470.
van Dam, J., Tulbure, A., Minniti, M. V., Abi-Farraj, F., and Hutter, M. (2022). Collision detection and identification for a legged manipulator. arXiv preprint arXiv:2207.14745.
Whitman, J., Su, S., Coros, S., Ansari, A., and Choset, H. (2017). “Generating gaits for simultaneous locomotion and manipulation,” in 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS) (IEEE). 2723–2729.
Wu, Q., Liu, C., Zhang, J., and Chen, Q. (2009). Survey of locomotion control of legged robots inspired by biological concept. Sci. China Ser. F Inf. Sci. 52, 1715–1729. doi:10.1007/s11432-009-0169-7
Xin, G., Zeng, F., and Qin, K. (2022). Loco-manipulation control for arm-mounted quadruped robots: Dynamic and kinematic strategies. Machines 10, 719. doi:10.3390/machines10080719
Yang, B., Lancaster, P. E., Srinivasa, S. S., and Smith, J. R. (2020a). Benchmarking robot manipulation with the rubik’s cube. IEEE Robotics Automation Lett. 5, 2094–2099. doi:10.1109/lra.2020.2969912
Yang, C., Sue, G. N., Li, Z., Yang, L., Shen, H., Chi, Y., et al. (2022). Collaborative navigation and manipulation of a cable-towed load by multiple quadrupedal robots. IEEE Robotics Automation Lett. 7, 10041–10048. doi:10.1109/lra.2022.3191170
Yang, C., Zhang, B., Zeng, J., Agrawal, A., and Sreenath, K. (2020b). “Dynamic legged manipulation of a ball through multi-contact optimization,” in 2020 IEEE/RSJ international conference on intelligent robots and systems. IROS). 7513–7520. doi:10.1109/IROS45743.2020.9341218
Yao, Q., Wan, J., Yang, S., Wang, C., Meng, L., Zhang, Q., et al. (2022). A transferable legged mobile manipulation framework based on disturbance predictive control. arXiv preprint arXiv:2203.03391.
Yoshida, H., Inoue, K., Arai, T., and Mae, Y. (2001). Mobile manipulation of humanoid robots-a method of adjusting leg motion for improvement of arm’s manipulability. In 2001 IEEE/ASME international conference on advanced intelligent Mechatronics. Proceedings (cat. No.01TH8556), 266–271. doi:10.1109/AIM.2001.936465
Yoshida, H., Inoue, K., Arai, T., and Mae, Y. (2002). Mobile manipulation of humanoid robots-optimal posture for generating large force based on statics. In Proceedings 2002 IEEE international conference on robotics and automation (cat. No.02CH37292), 2271–2276. doi:10.1109/ROBOT.2002.1013570
Zhang, Z., Zhao, J., Chen, H., and Chen, D. (2017). A survey of bioinspired jumping robot: Takeoff, air posture adjustment, and landing buffer. Appl. bionics biomechanics 2017, 1–22. doi:10.1155/2017/4780160
Zhao, Y., Gao, F., and Hu, Y. (2019). Novel method for six-legged robots turning valves based on force sensing. Mech. Mach. Theory 133, 64–83. doi:10.1016/j.mechmachtheory.2018.11.010
Zhou, C., Peers, C., Wan, Y., Richardson, R., and Kanoulas, D. (2022). Teleman: Teleoperation for legged robot loco-manipulation using wearable imu-based motion capture. arXiv preprint arXiv:2209.10314
Zhou, X., and Bi, S. (2012). A survey of bio-inspired compliant legged robot designs. Bioinspiration biomimetics 7, 041001. doi:10.1088/1748-3182/7/4/041001
Zimmermann, S., Poranne, R., and Coros, S. (2021). “Go fetch!-dynamic grasps using boston dynamics spot with external robotic arm,” in 2021 IEEE international conference on robotics and automation (ICRA) (IEEE), 4488–4494.
Keywords: multilegged robots, mobile manipulation, legs as grippers and other end-effectors, mechanism design of mobile robots, legged robots for object manipulation: a review
Citation: Gong Y, Sun G, Nair A, Bidwai A, CS R, Grezmak J, Sartoretti G and Daltorio KA (2023) Legged robots for object manipulation: A review. Front. Mech. Eng 9:1142421. doi: 10.3389/fmech.2023.1142421
Received: 11 January 2023; Accepted: 10 April 2023;
Published: 24 April 2023.
Edited by:
Hamid Reza Karimi, Polytechnic University of Milan, ItalyReviewed by:
Guiyang Xin, Dalian University of Technology, ChinaXin Li, China University of Mining and Technology, China
Haiyang Pan, Anhui University of Technology, China
Yihua Wu, Hunan Agricultural University, China
Copyright © 2023 Gong, Sun, Nair, Bidwai, CS, Grezmak, Sartoretti and Daltorio. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Kathryn A. Daltorio, kati@case.edu
†These authors have contributed equally to this work and share first authorship