Zhongkui Wang
Shinichi Hirai
Sadao Kawamura2- 1Research Organization of Science and Technology, Ritsumeikan University, Kusatsu, Japan
- 2Department of Robotics, Ritsumeikan University, Kusatsu, Japan
Despite developments in robotics and automation technologies, several challenges need to be addressed to fulfill the high demand for automating various manufacturing processes in the food industry. In our opinion, these challenges can be classified as: the development of robotic end-effectors to cope with large variations of food products with high practicality and low cost, recognition of food products and materials in 3D scenario, better understanding of fundamental information of food products including food categorization and physical properties from the viewpoint of robotic handling. In this review, we first introduce the challenges in robotic food handling and then highlight the advances in robotic end-effectors, food recognition, and fundamental information of food products related to robotic food handling. Finally, future research directions and opportunities are discussed based on an analysis of the challenges and state-of-the-art developments.
1 Introduction
The food industry is highly diverse and covers many industrial activities, such as production, processing, packaging, distribution, preparation, preservation, and food service. Traditionally, the food industry is considered to be low-tech, but it has become more technology intensive in recent years, as measured by its R&D to sales ratio (Traill and Meulenberg, 2002). Due to an aging society and labor shortages in countries such as Japan, automation in the food industry is in high demand to maintain profit margins. In particular, upon the impact of the COVID-19 pandemic, automation is strongly advocated at all stages in food production systems considering food safety and ensuring food supply (Henry, 2020).
In food factories, some operations such as food production, processing, and packaging require direct contact with food products. For operations of production and processing, dedicated machines, for example rice making machine, dumpling maker machine, and automatic chocolate molding machine, are usually preferable because the operations are constantly required without frequent change and update. In dedicated machines, handling operations of food materials or products can be specified and pick-and-place operations are often not required. On the other hand, packaging food products are generally conducted by pick-and-place operations. Moreover, pick-and-place operations are also required to transfer food products from one dedicated machine to another for connecting different processing operations. To realize pick-and-place operations, robotic systems consisting of robotic manipulator, end-effector, and sensors (e.g., camera), are often used because of their efficiency and adaptability to various food products.
Currently, industrial robotic arms are used as manipulators in the food industry for generating desired motions and carrying payloads in pick-and-place operations (Bader and Rahimifard, 2020). The industrial robotic arms can provide high position accuracy and high motion speed, but often with high cost. For robotic end-effector, the most widely used ones in food factories are suction cups as shown in Figure 1A, which are inexpensive and can be operated simply. Simple operation is an essential requirement for food handling systems because most handling tasks must be completed within a short time period to maintain production efficiency. However, there are many food products or handling operations, such as grasping food materials with moisture and porous surfaces, cannot be performed using suction cups. As a result, these operations are eventually left for human laborers to perform, as examples shown in Figures 1B,C, and the automation of such operations is currently the main task for most enterprisers and researchers in the food industry.
FIGURE 1. Factory scenarios of food handling operations in the food industry: (A) using suction cups to package cucumber, (B) human laborers packaging fried shrimps, (C) human laborers manufacturing Japanese boxed lunches, and (D) examples of prepared food materials in containers.
To automate the pick-and-place operations as shown in Figures 1B,C, gripping-type robotic end-effectors are needed and they are required to adapt to the variations of food products considering the frequent changes and updates of food products. Moreover, in many scenarios, cooked and prepared food products are randomly distributed or stored in containers as shown in Figure 1D instead of aligning on a belt conveyor. This brings challenges for grasping, recognition, and sensing. To address these challenges, many researches have been carried out and numbers of commercialized systems are available. However, there are still many open issues to be challenged to further accelerate the automation in the food industry. Therefore, in this review, we attempt to address these challenges in details and review recent developments and advances regarding these challenges. Finally, we discuss potential opportunities and research directions for improving food handling automation in the food industry.
2 Challenges
There are many challenges in the development of automated systems to be used in the food industry. In this review, we focus on robotic systems to perform typical handling tasks, such as pick-and-place operations. It is a simple operation for human laborers, but it presents many difficulties for a robotic system to achieve an efficiency similar to that of human laborers. Many technologies are required to complete these simple operations, such as the technologies for successfully recognizing food items, effectively handling food products, and robotic end-effectors for handling various food products.
Food products are mostly non-rigid, sometimes fragile, and easily bruised and marked when they come in contact with hard surfaces. In addition, food products are susceptible to bacterial contamination, and their properties are highly affected by environmental conditions, such as temperature, humidity, and pressure (Chua et al., 2003). These characteristics of food products bring many challenges when developing robotic systems to handle them. In this review, we will focus on the following three aspects: robotic end-effector, food recognition, and fundamental information of food products, as indicated by the light blue area in Figure 2. More details will be discussed in the following sections.
FIGURE 2. The structure of the review contents.
2.1 Robotic End-Effectors
To cope with the large variety and variable characteristics of food products, various robotic end-effectors need to be developed. The lack of effective robotic end-effectors is considered one of the main reasons that hinder the rapid introduction of robots into the food industry (Chua et al., 2003). Lien (2013); Fantoni et al. (2014) summarized the challenges of developing robotic grippers for food handling tasks and suggested that robotic grippers must cope with the softness, uneven surfaces, and non-uniform shapes of food products, and fulfill the hygienic requirements. In addition, robotic end-effectors must also adapt to the food surface conditions, such as wet and sticky surfaces. In some scenarios where the spaces or gaps among food products are small, as shown in Figure 1B, robotic end-effectors must have the ability to enter the small spaces for grasping. From contamination considerations, the robotic end-effectors must contain as few mechanical components as possible to avoid dropping into the food product. From the system’s point of view, the robotic end-effectors must have a simple motion and can be operated at a high speed to achieve a proper takt time. Furthermore, the end-effectors should be low cost and designed to meet the hygienic design principles (EHEDG, 2018).
2.2 Food Recognition
Difficulties in food recognition vary significantly depending on the operation scenarios. In food factories, scenarios of food recognition can be divided roughly into two categories: 1) food products aligned or scattered on a food conveyor with no overlap, 2) food products randomly distributed in a food container with overlaps and contacts among food products, as shown in Figure 1B, which is also known as the random bin-picking (RBP) scenario. In the first scenario, food products or materials are separately located on a flat surface. Therefore, they can be recognized by using conventional 2D image processing methods based on color information or pattern-matching techniques. The position and posture of the food product need to be calculated only in a 2D plane. For such a scenario, automated robotic systems can often be found in food factories, for example, the robotic systems for picking pizza and packaging powder based on pattern matching (Connolly, 2007a). On the other hand, it is difficult to perform food recognition in the RBP scenario because the food products may overlap and are located in a 3D space in which the position and posture of the food product must be described. The 3D template matching technique can be used in 3D space to recognize the position and pose of objects with well-defined geometries (Vock et al., 2019), but it is difficult to be applied to food products that have large variations in geometrical parameters. Therefore, food recognition in the 3D or RBP scenario for various food products remains an challenging issue.
2.3 Fundamental Information
To achieve successful handling of food product, an effective handling strategy can be very helpful. For instance, the grasping force needs to be small enough to avoid damage on food product, but it must also be large enough to complete a pick-and-place task without dropping. Moreover, grasping velocity also plays an important role when considering the viscosity of food product and possible impacts upon grasping. To the best of the authors’ knowledge, there are very few researches focusing on investigating optimal handling strategies of food products. In actual applications, these handling strategies are usually pre-determined through trial-and-error experiments. The reason behind this is the lack of fundamental information to properly model the “engineering” properties of food products, such as size, shape, weight, softness, surface condition, friction coefficient, viscoelasticity, rheology, fragility, ease of bruising, and so on. Researches in this area, especially from the viewpoint of robotic handling, have not been carried out frequently. There are specific machines or devices used for measuring these properties for various research purposes, such as food science, nutrition, and mastication. Unfortunately, such data for the purpose of robotic handling are barely available, but they are essentially important for designing end-effectors and investigating grasping strategies. In addition, handling strategy depends on robotic end-effector and food target. To improve versatility of handling strategy, categorizations of robotic end-effectors and food products based on their characterizations are also essential, and such research activities have not been carried out frequently so far.
3 Recent Advances
To tackle the above-mentioned challenges, many researches and commercial robotic systems have been developed in the last few decades. In this section, we review these advances as indicated with the light orange area in Figure 2.
3.1 Advances in Robotic End-Effectors
Many robotic hands and grippers have been proposed and studied so far to handle food products and materials. To better review and address related work, we divided robotic end-effectors into six categories, as shown in Figure 3, based on the position/positions where an end-effector contacts the target food product. We employ this classification approach instead of using grasping principles as nicely reviewed in (Fantoni et al., 2014) because we tend to focus on the forces that the food target may be received and reduce the number of categories.
FIGURE 3. Different types of robotic end-effectors according to their handling positions at (A) top surface, (B) side surface, (C) bottom surface, (D) top and side surfaces, (E) side and bottom surfaces, and (F) top, side, and bottom surfaces. Red star marks indicate contact positions.
3.1.1 Grasping at Top Surface
In the case of grasping food products from the top surface (Figure 3A), conventional grippers can be roughly categorized into four groups: 1) suction cups, 2) grippers using the Bernoulli principle (Petterson et al., 2010) or the Coanda effect (Elango et al., 2012, 2018), 3) devices using adhesion force from a roller (Davis et al., 2007) or by freezing the moist surface (Lien and Gjerstad, 2008), and 4) gripper penetrating inside the food product by needles (Gjerstad et al., 2006). A suction cup with vacuum pressure has the advantages of a simple structure and stable grasp. However, suction cups usually require the surface of the food product to be relatively flat, smooth, and dry. In addition, it may leave a bruise or mark on the product surface when handling raw food products. To overcome these disadvantages, grippers based on the Bernoulli principle or Coanda effect have been proposed, and these grippers do not require direct contact with food products for grasping. However, the grasping becomes unstable, and rotation motion may be generated owing the high-speed air flow. Grippers using adhesion force are usually used for specific food products with sticky properties (e.g., pasta dough) for the roller type and moist surfaces (e.g., fresh fish piece) for the freezer type. The freezer-type gripper also requires a sharp knife mechanism or heat flow to release the food product upon placement (Fantoni et al., 2014).
In recent years, new types of grippers have also been proposed for grasping various objects, including food products. Amend et al. (2016) proposed jamming grippers and successfully tested them for grasping various types of objects. They appeared to be very promising for practical applications in the food industry. Unfortunately, commercialization failed due to challenges such as leaks, difficult actuation and assembly, and materials. Koivikko et al. (2021) developed soft suction grippers with switchable stiffness to achieve both small and large forces. The grippers were tested on fruits, such as mangoes and bananas. Recently, a pneumatically driven needle gripper was also proposed to grasp raw food materials from their top surface (Wang et al., 2021c). However, these grippers are still in their research stages, and commercialization and applications in the food industry are expected only in the future.
3.1.2 Grasping at Side Surfaces
Grippers belonging to this group have been widely studied and are frequently applied in the food industry. When grasping an object at its side surfaces, the object size must be known, and the gripper must provide sufficient stroke for successful grasping. Because the friction force dominates the grasping performance, it is important to ensure a sufficient friction force for stable grasping. This type of grasping has the advantage of better placement accuracy because the object is enclosed inside the gripper, and the posture of the object can be easily adjusted. Many conventional two-fingered or multi-fingered parallel grippers for manufacturing automation have been modified for application in the food industry after solving the food compatibility issues and ensuring that there is no damage to the food products. One good example is the SCHUNK food gripper (SCHUNK, 2021), which provides customized, fully regulation-compliant components and gripper solutions for the food industry. In recent years, along with the rise of soft robotics, many soft robotic grippers have been developed for handling food products. Soft grippers have the advantages of easy adaptation to food variations and because of their soft bodies less risk of damage to the food products. Examples of commercialized soft grippers include the mGrip grippers from Soft Robotics, Inc. (Robotics Inc., Soft, 2021), soft gripper from OnRobot (OnRobot, A/S, 2021), a modular-designed soft gripper from SoftGripping (SoftGripping, 2021), and the soft flexible gripper from Soft Robot Technology Co. Ltd. (SRT., 2021). These grippers are pneumatically driven and fabricated with silicone materials. They are able to handle a wide range of irregular shaped and delicate food products. In addition, Festo (Festo, 2021a) provides a shape adaptive gripper using soft Fin Ray® structure which can passively adapt to the shape of a target object.
In addition to these commercialized end-effectors, many studies have been conducted to design end-effectors for handling challenging food products. Pettersson et al. (2010) developed a soft gripper using the effects of magnetorheological fluid to cope with the ease of bruising and shape variations of food products, such as apples, carrots, strawberries, broccoli, and grapes. Maruyama et al. (2013) proposed a robotic gripper made from an incompressible fluid enclosed in a rubber part to grasp fragile objects, such as potato chips and tofu. Endo and Otomo developed a two-degree-of-freedom multi-fingered gripper for grasping noodles and simmered food by considering an appetizing presentation (Endo and Otomo, 2016). To handle similar chopped and granular food materials, Kuriyama et al. (2019) proposed a pneumatically driven wrapping gripper for automating the topping operation of a Japanese lunch box. Blanes et al. (2014) proposed several pneumatic actuators and mechanisms fabricated by additive manufacturing for food handling. These grippers have promising capabilities for handling specific food materials, but many requirements, such as food compatibility and hygiene, need to be considered before they can be commercialized or used in actual applications.
3.1.3 Grasping From Bottom Surface
Handling food products solely from the bottom surface can often be seen in kitchens when using a spatula to cook foods. However, this approach is not widely adopted in automation systems because of the possible instability or slippage during high-speed translation motion. The only commercialized robotic end-effector using this principle is the SWITL hand developed by FURUKAWA KIKO (Furukawa, 2021). The hand consisted of a Teflon film and a stainless-steel plate. The Teflon rotates around the plate like a belt conveyer and creeps under the food material. It can handle various food materials, such as slices of ham and cheese, semi-liquid-gels, mousses, and mayonnaise sauce or melted cheese (Kusuda, 2011).
3.1.4 Grasping at Top and Side Surfaces
It is natural to combine different principles to improve the grasping capabilities. In this subsection, we review related work on end-effectors that combine the grasping at both the top and side surfaces of an object. However, we do not consider those that combine both principles, but they are operated independently by using a tool changer. In this end-effector group, it is often possible to observe the combination of suction and gripping. A commercialized example is the gripper from RightHand Robotics, Inc., which consists of a suction cup and a three-fingered gripper (RightHand Robotics, 2021). Another commercialized example is the TentacleGripper from Festo, which is structured as a silicone tentacle and two rows of suction cups located on the surface of the tentacle mimicking the octopus leg (Xie et al., 2020; Festo., 2021b). In addition, researchers have proposed other types of grippers that combine suction and gripping principles. Bryan et al. (2019) proposed a soft robotic gripper with three cable-driven fingers, and each finger is equipped with three suction cups. Multi-fingered grippers with suction cups at the fingertips have also been developed by researchers for handling various objects, including food materials (Yamaguchi et al., 2013; Wang et al., 2020). Moreover, soft grippers with enveloping motion have also been proposed to grasp various objects, including fragile food products (Li et al., 2019; Hao et al., 2021). The gripper, upon grasping can form an enclosed space; therefore, the gripper itself works as a suction cup when vacuum pressure is applied (Hao et al., 2021). Both grippers can adapt to the object shape and generate large grasping forces. In addition, grippers integrated with gecko-inspired adhesives have been proposed by researchers to improve grasping performance (Song et al., 2014; Glick et al., 2018). These grippers combine the principles of gripping and adhesion, and require the contact surface to be relatively dry to improve the adhesion performance. Some of the above-mentioned grippers may not be designed specifically for food industry applications, but the approaches used to achieve stable grasping can be extended to handle food products.
3.1.5 Grasping by Side and Bottom Surfaces
There are also robotic end-effectors that combine grasping from the side and bottom surfaces. Ma et al. (2020) proposed a flat-shaped paper gripper for food grasping. This gripper can slide a thin paper sheet under the food product for grasping, and the puller sheet can also make contact with the food product for stabilization purposes. Gafer et al. (2020) developed a quad-spatula gripper for handling food ingredients. This gripper has four cable-driven fingers with a spatula-shaped plate at each fingertip. When grasping, the plates scoop the food ingredients, and the fingers also apply a grasping force. In addition, Wang et al. (2021a) proposed a scooping-binding gripper to handle various food products, especially those with a low height profile and slippery properties. The gripper consists of two thin scooping plates and multiple rubber strings. Upon grasping, the scooping plates are inserted under the bottom surface of the food product and the rubber strings wrap around the side surfaces to stabilize the grasp. However, no commercially available end-effectors were found that use this combination of grasping by the side and bottom surfaces of food products.
3.1.6 Grasping by Top, Side and Bottom Surfaces
The last group of grippers achieves stable grasping by enveloping food products from all surfaces. One commercially available gripper of this type is the meat gripper (AppliedRobotics., 2021). This gripper consists of two L-shaped grip plates to grasp food products and a center plate at the top with a passive spring mechanism to provide a pushing force for stabilization. The gripper is manufactured from lightweight materials approved by the FDA and USDA, and it can be used to handle various food products, such as fresh meat, fish, cheese, bacon, and many other nonuniform products. For research activities, Sam and Nefti proposed a multifunctional gripper for handling various food products (Sam and Nefti, 2010). This gripper was constructed using a suction mechanism and four rigid fingers. The suction mechanism was based on the Bernoulli effect. After the food product was held by the suction mechanism, the four fingers were closed to envelop the food product for stabilization.
3.2 Advances in Food Recognition
3.2.1 2D Recognition
For the scenario of food scattered or aligned on food conveyor, recognition can be performed using 2D image processing or pattern matching approaches, which have been integreated in commercially available robotic systems, such as the ABB FlexPicker Packaing Robot (ABB, 2021). In addition, 2D image processing can be also used in recognizing certain food products with overlapping and occluding (Muhammedali et al., 2004). The approach of 2D image processing assumes the depth information of the food target is known and only the position and orientation in the horizontal plane (usually the food conveyor) need to be recognized. Using only image processing can greatly reduce the complexity of the recognition problem and results in an efficient solution. However, it is difficult to be applied to recognize multiple food products or materials of different categories. For recognizing multiple targets, machine learning based approaches are often adopted. Kawano and Yanai developed a smartphone-based food recognition system (FoodCam) to estimate calories and nutrients in foods and record a user’s eating habits (Kawano and Yanai, 2013; Kawano and Yanai, 2014b; Kawano and Yanai, 2015). It achieved a classification rate of 79.2% for the top five category candidates for a 100-category food dataset (Kawano and Yanai, 2015). Liu et al. (2018) proposed a deep-learning-based food recognition system for dietary assessment on an edge computing service infrastructure and achieved approximately a 90% classification accuracy with three different food datasets. In addition, Ciocca et al. (2014) developed a food recognition system that can track the eaten food and the user’s dietary habits, realize automatic billing procedure based on the recognized foods, and evaluate the leftovers for better estimation of food intake. Even though these approaches are not proposed for robotic handling purposes, the ideas of food segmentation and classification can be extended to applications of robotic handling.
3.2.2 3D Recognition
3D recognition is commonly required in the RBP scenario, as an example shown in Figure 1B. The RBP problem has been widely studied for grasping rigid parts with known CAD models in industrial applications (Liu et al., 2012). Many commercial systems are available to solve the RBP problem. Many robot manufacturers provide such vision systems together with their robotic manipulators, such as the FANUC 3D Vision Sensor (Connolly, 2007b; FANUC., 2021), MELFA-3D Vision from Mitsubishi Electric (MITSUBISHI ELECTRIC, 2021), and KUKA.PerceptionTech (KUKA., 2021). There are also companies that provide specific 3D vision systems for RBP tasks, such as the 3D vision system from Pickit (Pickit., 2021) and the 3D vision sensor “TVS” from Kyoto Robotics (KYOTO ROBOTICS, 2021). However, most of these approaches assume dealing with objects of known and non-deformable shapes (Castaman et al., 2020), and therefore, they have not yet been applied to food products. Machine learning-based approaches have been investigated to recognize food products in the RBP scenario. Joffe et al. (2019) proposed a method using a standard Faster R-CNN architecture with a Resnet 101 feature extractor to recognize chicken and evaluated two pose estimation approaches: the augmented autoencoder and direct regression approach. A suction cup gripper was used to pick the chickens. Nishina and Hasegawa proposed an approach to obtain the optimal grasping points through a deep neural network and successfully applied it to a two-fingered robotic hand and a suction cup gripper (Nishina and Hasegawa, 2020). Moreover, Low et al. (2021) proposed a YOLOv3 based object detection algorithm for recognizing sousage, potato, broccoli, and tomato in a RBP scenario. The accuracy of object classification and the speed of pose estimation were achieve at 67.06% and 92.7 ms, respectively.
3.3 Advances in Fundamental Information
3.3.1 Food Categorization
Food products have large varieties and big variations in shape, size, weight, surface conditions, softness, and other physical properties. To maximize the cost performance of a robotic end-effector, it is essential to investigate how many categories of food products the end-effector can handle. Unfortunately, it is impractical to perform experimental tests on each food product considering the large varieties. Therefore, food categorization needs to be carried out from a viewpoint of robotic handling. Erzincanli and Sharp (1997) proposed a classification system for robotic food handling and categorized food products depending on their shapes, dimensions, surfaces, compiance, temperature, and weight. The food shape was classified into eight groups with standard geometries, such as flat, cylinder, square, ellipse, and so on. The surface and compliance were qualitatively described as smooth, furry, thorny, rigid, semi-rigid, non-rigid, and so on. In addition, Wurdemann et al. (2011) presented a categorization system for classifying food products to assist food ordering process. Key characteristics used for the classification are symmetry, surface condition, hardness, springiness, and resistance to damage. All these characteristics are also defined as qualitative descriptions.
3.3.2 Food Properties
Food properties have been studied for many decades. Among the different properties, elasticity, often indicated by Young’s modulus, has been widely studied for various purposes. Williams et al. (2005) experimentally studied the Young’s modulus of a series of foods, ranging from apple pulp to prune pit, to develop a primate masticatory apparatus. Ogawa et al. (2015) investigated the changes in Young’s modulus and Poisson’s ratio in Japanese radish and carrot roots during boiling to assess food quality. Kadowaki et al. (2013) measured the Young’s modulus of crispy foods at the microscopic level to study food crash. Recently, Sinha and Bhargav studied the effects of experimental parameters, such as the deformation rate, sample shape and size, and moisture content, when measuring the Young’s modulus of potato and sweet potato samples for food quality assessment (Sinha and Bhargav, 2020). In addition to elasticity, viscoelastic or rheological properties were also investigated, and different analytical models were proposed to capture the complex deformation and force behaviors. The linear viscoelasticity of gummy candy, Mozzarella cheese, and cooked ham was characterized by Singh et al. (2006) using broadband viscoelastic spectroscopy (BVS). Sakamoto et al. (2007, 2009) studied the viscoelasticity of Japanese food “Norimaki” using Maxwell and Burger models to realize the optimum design of robotic handling. Rheological models were also investigated by Wang and Hirai (2010, 2012) to simultaneously capture both deformation (especially, residual deformation) and force of Japanese sweets.
Friction, as a surface property, has mostly been investigated for studying the oral mastication sensation. For example, Joyner et al. (2014) proposed a double-ball tribological system to evaluate the friction of acid milk gels with and without the addition of saliva. Chojnicka-Paszun and de Jongh explored the tribological properties of food-relevant aqueous solutions on different surfaces to study the mastication of food products (Chojnicka-Paszun and de Jongh, 2014). A universal mechanical tester “Tribolab” was used to measure the friction force between the polydimethylsiloxane (PDMS) surface, which mimics the oral surface, and intact soft solid foods such as gelatin gels and sausages (Fuhrmann et al., 2020). In addition, a measuring apparatus mimicking a robotic grasping scenario was developed to measure the friction coefficient between a flat stainless plate and food material (Wang et al., 2021b).
Food geometry is another important data for assisting with robotic handling. In particular, geometry is essential to study robotic grasping in a simulation environment or when performing 3D template matching for food recognition. Balcerzak et al. (2015) proposed an approach to construct a geometric model of agri-food products using the Autodesk 3ds Max to predict the behavior of agri-food products subjected to drying, cooling, and heating operations. Goñi et al. (2008) investigated three-dimensional geometric models of lamb, pork, and chicken carcasses through magnetic resonance imaging for food process modeling applications. Research related to food quality inspection also employs the shape information of food materials. Weres et al. (2009) developed software packages to assess quality and classify selected agri-food products, to represent their 3D geometry, and visualize their property changes during thermo-mechanical processing. Ding and Gunasekaran developed an automated food shape inspection system that included a feature extraction stage and a classification stage, and tested it on corn kernels, almonds, and animal-shaped crackers (Ding and Gunasekaran, 1994). In addition, Loebnitz and Grunert conducted a survey to explore the effect of food shape abnormality on purchase intention and how environmental concern and social trust might moderate this intention (Loebnitz and Grunert, 2015).
3.3.3 Food Database
In terms of the food database, we often found those related to nutrient profile (U.S. Department of Agriculture, 2021), food composition (Food Standards Australia and New Zealand, 2021), and food constituents, chemistry, and biology (The Metabolomics Innovation Centre, 2021). Unfortunately, there are no databases directly applicable to robotic handling tasks. Available food databases are mainly developed for recognizing food products from 2D images and conducting calculations of calorie or nutrition for health monitoring purposes. Kawano and Yanai created a series of datasets for Japanese food (Kawano and Yanai, 2021). They first released a dataset (UEC FOOD-100) containing 100 types of food photos with a bounding box for each photo to indicate the location of the food item (Matsuda et al., 2012). The dataset was then extended to UEC FOOD-256 which contains 256 types of food photos (Kawano and Yanai, 2014a). Recently, the authors updated the UEC FOOD-100 dataset to the UEC-FoodPix Complete dataset with manually refined segmentation masks to enable accurate food segmentation (Okamoto and Yanai, 2021). In addition, the authors created a school lunch dataset containing 3940 multiple-dish images with bounding boxes on 21-class labels (Ege and Yanai, 2017). Furthermore, there are also datasets of Food-101 (Bossard et al., 2014) and Google Food-201 (Myers et al., 2015) for the recognition of Western food items, and a dataset for Chinese food identification (Chen et al., 2012). Even though these databases are not created directly for the purpose of robotic handling tasks, they have the potentials to be used for classifying food categories.
4 Future Opportunities
With an aging society and resulting labor shortages, advancements in robotic technologies will lead to the introduction of an increasing number of robotic systems into the food industry to replace human laborers performing simple tasks. There are plenty of opportunities (green area in Figure 2) for researchers and enterprises in the fields of robotic manipulators, robotic end-effectors, computer science, artificial intelligence, and system integration. In this study, we did not review robotic manipulators because industrial robots are commonly used in the food industry (Bader and Rahimifard, 2020), and research on new robotic manipulators specifically for the food industry is scarce. However, this does not mean that there is no need to develop novel robotic manipulators for industrial food applications. In fact, the majority of industrial robotic manipulators are not well suited to the specific needs of industrial food applications (Masey et al., 2010). In particular, the high cost of the current robotic systems presents a financial obstacle for food manufacturers. As summarized by Masey et al. (2010), robotic manipulators designed for the food industry should fulfill the requirements of easy to clean hygienic design, low cost, fast operational pick-and-place speed, safe operation alongside human workers, and easy to reprogram. In recent years, collaborative robots have been employed frequently in various applications, and they can meet the requirements of operating alongside human workers and are easy to program. However, the requirements of low cost and fast operational speed remain to be fulfilled.
Regarding to robotic end-effectors, research has been carried out intensively in recent years, along with the rise of the soft robotics field. However, there are still few commercialized end-effectors, and successful user cases in the food industry are even more limited. To be successful, the end-effectors must not only meet all the hygienic design principles and takt time requirements, but also be low cost and easy to be integrated into the existing robotic systems. Moreover, one end-effector should be able to handle various food products to adapt to rapid changes or updates of the target products in an automation line. Theoretical and experimental investigations need to be carried out to establish relationships between robotic end-effectors and food products. In addition, considering the grasping principle, as summarized in Table 1, only end-effectors grasping at the top and side surfaces are well studied and commercialized, but end-effectors grasping from the bottom surface and all surfaces have been barely studied so far. However, such end-effectors are required for handling various kinds of food products, especially slippery, heavy, and low-profile food products. An increasing number of such end-effectors are expected.
TABLE 1. Summarization of robotic end-effectors for food handling.
For food recognition, the RBP scenario should be further investigated for various food products and materials. Attention need to be paid not only for classification and recognition, but also for the determination of the grasping position and orientation. In the bin-picking scenario, these positions and orientations are in three-dimensional space, and the accuracy of the recognition and determination may significantly affect the grasping performance. To improve the recognition accuracy, one potential approach is to use deep learning methods that learn representational features from the dataset during the training process and demonstrate stronger ability than traditional methods (Zhou et al., 2019). The databases mentioned in Section 3.3.3 may be used for developing deep learning models, but new databases for bin-picking scenarios are also needed to ensure satisfactory performance.
Regarding food categorization, it would be better to have a categorization system including quantitative descriptions of features such as surface condition, hardness, and resistance to damage. Relationships between robotic end-effectors and food categories need to be established to help the selection of proper robotic end-effector. For the physical properties and database of food products, it would be helpful to have a database containing food property information related to robotic handling, such as the properties of viscoelasticity or rheology, friction, geometry, and weight, similar to the food version of the Yale-CMU-Berkeley dataset (Calli et al., 2015, 2017). The properties should be presented in a way that facilitates robotic handling tasks. Such a database can lay a foundation for the development of robotic systems for food handling and can also be used to study and analyze robotic systems in a simulation environment. It may also inspire researchers to develop property-measuring devices and establish evaluation standards for various robotic end-effectors.
Finally, from a system’s point of view, a compact system with portability is preferable to cope with the high-mix low-volume production in the food industry. In addition, system engineers are usually not available in food factories; therefore, the robotic system used in such factories must have an easy-to-use interface, and complex system maintenance should be avoided or done automatically. Along with the development of the Fourth Industrial Revolution (4IR or Industry 4.0), an increasing number of industrial and home devices are connected through the technologies of the Internet of Things (IoT) or cyber physical systems (CPS) (Khan et al., 2018). Therefore, it is also a good opportunity to apply such technologies to the robotic systems in the food industry to extend their capabilities of machine-to-machine communication, self-monitoring, and automatic system updating.
5 Conclusion
The food industry has a very long history, but it is still a labor-intensive industry. There are many benefits to fully automate food preparation and processing operations, but difficulties are deep-rooted, and new technologies need to be integrated together to make a step forward. In this review, we investigated a large number of research articles and commercial systems related to the robotic handling of food products. We first summarized the basic challenges faced in the food industry for introducing robotic systems, and then elaborated on the advances in different aspects, such as the robotic end-effector, food recognition, and fundamental information of food categorization, property, and database, which are essentially important for developing robotic systems. Finally, we suggest future directions that are potentially promising to tackle these challenges and eventually help the process of automation in the food industry. The purpose of this review is to encourage researchers and enterprises in this field to further advance the existing technologies, develop new technologies, and put them into practice for automating various operations in the food industry.
Author Contributions
ZW, SH, and SK contributed to the idea and structure of the review. ZW wrote the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.
Funding
This work was supported by the Cabinet Office (CAO), Cross-ministerial Strategic Innovation Promotion Program (SIP), “An intelligent knowledge processing infrastructure, integrating physical and virtual domains” (funding agency: NEDO), and in part by JSPS KAKENHI Grant Number 20K04406.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors, and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
ABB (2021). Irb 360 Flexpicker Available at: https://new.abb.com/products/robotics/industrial-robots/irb-360. (accessed Dec01, 2021).
Amend, J., Cheng, N., Fakhouri, S., and Culley, B. (2016). Soft Robotics Commercialization: Jamming Grippers from Research to Product. Soft Robotics 3, 213–222. doi:10.1089/soro.2016.0021
AppliedRobotics (2021). Meat Gripper Available at: https://www.appliedrobotics.com/products/automation/grippers/meat-gripper/(accessed Sept08, 2021).
Bader, F., and Rahimifard, S. (2020). A Methodology for the Selection of Industrial Robots in Food Handling. Innovative Food Sci. Emerging Tech. 64, 102379. doi:10.1016/j.ifset.2020.102379
Balcerzak, K., Weres, J., Gorna, K., and Idziaszek, P. (2015). Modeling of Agri-Food Products on the Basis of Solid Geometry with Examples in Autodesk 3ds max and Finite Element Mesh Generation. J. Res. Appl. Agric. Eng. 60, 5–8.
Blanes, C., Mellado, M., and Beltran, P. (2014). Novel Additive Manufacturing Pneumatic Actuators and Mechanisms for Food Handling Grippers. Actuators 3, 205–225. doi:10.3390/act3030205
Bossard, L., Guillaumin, M., and Van Gool, L. (2014). “Food-101 - Mining Discriminative Components with Random Forests,” in Proc. European Conference on Computer Vision (Springer), 446–461. doi:10.1007/978-3-319-10599-4_29
Bryan, P., Kumar, S., and Sahin, F. (2019). “Design of a Soft Robotic Gripper for Improved Grasping with Suction Cups,” in Proc. 2019 IEEE International Conference on Systems, Man and Cybernetics (IEEE), 2405–2410. doi:10.1109/smc.2019.8914479
Calli, B., Singh, A., Bruce, J., Walsman, A., Konolige, K., and Srinivasa, S. (2017). Yale-cmu-berkeley Dataset for Robotic Manipulation Research. Int. J. Robotics Res. 36, 261–268. doi:10.1177/0278364917700714
Calli, B., Walsman, A., Singh, A., Srinivasa, S., Abbeel, P., and Dollar, A. M. (2015). Benchmarking in Manipulation Research: the Ycb Object and Model Set and Benchmarking Protocols. IEEE Robotics Automation Mag. 22, 36–52. doi:10.1109/MRA.2015.2448951
Castaman, N., Cenzato, A., Tonello, S., Menegatti, E., Ghidoni, S., and Pretto, A. (2020). “Model-free Bin-Picking: Food Processing and Parcel Processing Use Cases,” in Proc. 2nd Italian Conference on Robotics and Intelligent Machines (Italian Institute of Robotics and Intelligent Machines).
Chen, M., Yang, Y., Ho, C., Wang, S., Liu, S., Yeh, C., et al. (2012). “Automatic Chinese Food Identification and Quantity Estimation,” in Proc. SIGGRAPH Asia. doi:10.1145/2407746.2407775
Chojnicka-Paszun, A., and de Jongh, H. H. J. (2014). Friction Properties of Oral Surface Analogs and Thier Interaction with Polysaccharide/mcc Particle Dispersions. Food Res. Int. 62, 1020–1028. doi:10.1016/j.foodres.2014.05.028
Chua, P. Y., Ilschner, T., and Caldwell, D. G. (2003). Robotic Manipulation of Food Products - a Review. Ind. Robot: Int. J. 30, 345–354. doi:10.1108/01439910310479612
Ciocca, G., Napoletano, P., and Schettini, R. (2014). “Food Recognition and Leftover Estimation for Daily Diet Monitoring,” in Proc. International Conference on Image Analysis and Processing (Springer), 334–341.
Connolly, C. (2007b). A New Integrated Robot Vision System from Fanuc Robotics. Ind. Robot: Int. J. 34, 103–106. doi:10.1108/01439910710727423
Connolly, C. (2007a). Abb High-Speed Picking Robots Establish Themselves in Food Packaging. Ind. Robot: Int. J. 34, 281–284. doi:10.1108/01439910710749591
Davis, S., Casson, J. W., Masey, R. J. M., King, M., Gray, J. O., and Caldwell, D. G. (2007). Robot Prototyping in the Design of Food Processing Machinery. Ind. Robot: Int. J. 34, 135–141. doi:10.1108/01439910710727487
Ding, K., and Gunasekaran, S. (1994). Shape Feature Extraction and Classification of Food Material Using Computer Vision. Trans. ASAE 37, 1537–1545. doi:10.13031/2013.28238
Ege, T., and Yanai, K. (2017). “Estimating Food Calories for Multiple-Dish Food Photos,” in Proc. 4th IAPR Asian Conference on Pattern Recognition (IEEE), 646–651. doi:10.1109/acpr.2017.145
EHEDG (2018). “Hygienic Design Principles,” in EHEDG Guidelines (European Hygienic Engineering & Design Group) (Doc), 8.
Elango, N., Lim, W. H., Manickam, R., Chong, C. H., and Ramesh, S. (2018). “Design and Development of a Robot Gripper for Food Industries Using Coanda Effect,” in Proceedings of the 2018 IEEE 4th International Symposium on Robotics and Manufacturing Automation.
Elango, N., Onubogu, N. O., and Ragunathan, S. (2012). Experimental Study of Non-contact Robot Gripper for Food Industries. Appl. Mech. Mater. 232, 392–397. doi:10.4028/www.scientific.net/AMM.232.392
Endo, G., and Otomo, N. (2016). “Development of a Food Handling Gripper Considering an Appetizing Presentation,” in Proceedings of 2013 IEEE International Conference on Robotics and Automation (IEEE), 4901–4906. doi:10.1109/icra.2016.7487695
Erzincanli, F., and Sharp, J. M. (1997). A Classification System for Robotic Food Handling. Food Control 8, 191–197. doi:10.1016/s0956-7135(97)00048-0
Fantoni, G., Santochi, M., Dini, G., Tracht, K., Scholz-Reiter, B., Fleischer, J., et al. (2014). Grasping Devices and Methods in Automated Production Processes. CIRP Ann. - Manufacturing Tech. 63, 679–701. doi:10.1016/j.cirp.2014.05.006
FANUC (2021). Bin Picking Robot Available at: https://www.fanuc.co.jp/en/product/robot/function/baradumi.html (accessed Sept10, 2021).
Festo. (2021a). Multichoicegripper-one Gripper for the Widest Variety of Shapes. Available at: https://www.festo.com/net/SupportPortal/Files/333986/Festo_MultiChoiceGripper_en.pdf [Accessed Sept 01, 2021].
Festo (2021b). Tentaclegripper–gripping Modelled on an octopus Tentacle Available at: https://www.festo.com/group/en/cms/12745.htm (Accessed Sept 03 2021).
Food Standards Australia and New Zealand (2021). Australian Food Composition Database Available at: https://www.foodstandards.gov.au/science/monitoringnutrients/afcd/pages/default.aspx (accessed Sept 15, 2021).
Fuhrmann, P. L., Aguayo-Mendoza, M., Jansen, B., Stieger, M., and Scholten, E. (2020). Characterisation of Friction Behaviour of Intact Soft Solid Foods and Food Boli. Food Hydrocolloids 100, 105441. doi:10.1016/j.foodhyd.2019.105441
Furukawa (2021). Switl Available at: https://www.furukawakikou.co.jp/switl/(accessed Sept 03 2021).
Gafer, A., Heymans, D., Prattichizzo, D., and Salvietti, G. (2020). “The Quad-Spatula Gripper: a Novel Soft-Rigid Gripper for Food Handling,” in Proc. 2020 3rd IEEE International Conference on Soft Robotics (IEEE), 39–45. doi:10.1109/robosoft48309.2020.9115968
Gjerstad, T. B., Lien, T. K., and Buljo, J. O. (2006). “Handle of Non-rigid Products Using a Compact Needle Gripper,” in Proceedings of the 39th CIRP International Seminar on Manufacturing Systems, 145–151.
Glick, P., Suresh, S. A., Cutkosky, M., Tolley, M. T., and Parness, A. (2018). A Soft Robotic Gripper with Gecko-Inspired Adhesive. IEEE Robotics Automation Lett. 3, 903–910. doi:10.1109/LRA.2018.2792688
Goñi, S. M., Purlis, E., and Salvadori, V. O. (2008). Geometry Modelling of Food Materials from Magnetic Resonance Imaging. J. Food Eng. 88, 561–567. doi:10.1016/j.jfoodeng.2008.03.020
Hao, Y., Biswas, S., Hawkes, E., Wang, T., Zhu, M., Wen, L., et al. (2021). A Multimodal, Enveloping Soft Gripper: Shape Conformation, Bioinspired Adhesion, and Expansion-Driven Suction. IEEE Trans. Robotics 37, 350–362. doi:10.1109/TRO.2020.3021427
Henry, R. (2020). Innovations in Agriculture and Food Supply in Response to the Covid-19 Pandemic. Mol. Plant 13, 1095–1097. doi:10.1016/j.molp.2020.07.011
Joffe, B., Walker, T., Gourdon, R., and Ahlin, K. (2019). Pose Estimation and Bin Picking for Deformable Products. Int. Fed. Automatic Control. 52, 361–366. doi:10.1016/j.ifacol.2019.12.566
Joyner, H. S., Pernell, C. W., and Daubert, C. R. (2014). Impact of Formulation and Saliva on Acid Milk Gel Friction Behavior. J. Food Sci. 79, E867–E880. doi:10.1111/1750-3841.12439
Kadowaki, R., Inou, N., and Kimura, H. (2013). “Measurement of Microscopic Young’s Modulus of Crispy Foods,” in Proc. 2013 4th International Conference on Food Engineering and Biotechnology (IACSIT Press), 79–83.
Kawano, Y., and Yanai, K. (2014a). “Automatic Expansion of a Food Image Dataset Leveraging Existing Categories with Domain Adaptation,” in Proc. ECCV Workshop on on Transferring and Adapting Source Knowledge in Computer Vision (Springer).
Kawano, Y., and Yanai, K. (2021). Food Available at: http://foodcam.mobi/dataset.html (accessed Sept 15, 2021).
Kawano, Y., and Yanai, K. (2015). Foodcam: A Real-Time Food Recognition System on a Smartphone. Multimedia Tools Appl. 74, 5263–5287. doi:10.1007/s11042-014-2000-8
Kawano, Y., and Yanai, K. (2014b). “Foodcam: A Real-Time mobile Food Recognition System Employing fisher Vector,” in Proc. International Conference on Multimedia Modeling (Springer), 369–373. doi:10.1007/978-3-319-04117-9_38
Kawano, Y., and Yanai, K. (2013). “Real-time mobile Food Recognition System,” in Proc. 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops (IEEE). doi:10.1109/cvprw.2013.5
Khan, Z. H., Khalid, A., and Iqbal, J. (2018). Towards Realizing Robotic Potential in Future Intelligent Food Manufacturing Systems. Innovative Food Sci. Emerging Tech. 48, 11–24. doi:10.1016/j.ifset.2018.05.011
Koivikko, A., Drotlef, D., Sitti, M., and Sariola, V. (2021). Magnetically Switchable Soft Suction Grippers. Extreme Mech. Lett. 44. doi:10.1016/j.eml.2021.101263
KUKA (2021). Kuka.perceptiontech Available at: https://www.kuka.com/en-de/products/robot-systems/software/application-software/kuka_perceptiontech (accessed Sept 10, 2021).
Kuriyama, Y., Okino, Y., Wang, Z., and Hirai, S. (2019). “A Wrapping Gripper for Packaging Chopped and Granular Food Materials,” in Proc. 2nd IEEE International Conference on Soft Robotics (IEEE), 114–119. doi:10.1109/robosoft.2019.8722745
Kusuda, Y. (2011). Robots at Fooma japan: a Food Machinery and Technology Exhibition. Ind. Robot: Int. J. 38, 572–576. doi:10.1108/01439911111179084
KYOTO ROBOTICS (2021). Kyoto Robotics Corporation - home Available at: https://orange-zebra-602c81b2419bea51.znlc.jp/en/index.html (accessed Sept 10, 2021).
Li, S., Stampfli, J. J., Xu, H. J., Malkin, E., Diaz, E. V., Rus, D., et al. (2019). “A Vacuum-Driven Origami “magic-ball” Soft Gripper,” in Proc. 2019 International Conference on Robotics and Automation (IEEE), 7401–7408. doi:10.1109/icra.2019.8794068
Lien, T. K., and Gjerstad, T. B. (2008). A New Reversible thermal Flow Gripper for Non-rigid Products. Trans. North Am. Manufacturing Res. Inst. SME 36, 565–572. doi:10.1108/01439910710727487
Lien, T. K. (2013). “Gripper Technologies for Food Industry Robots,” in Robotics And Automation In the Food Industry (Sawston: Woodhead Publishing Limited), 143–170. doi:10.1533/9780857095763.1.143
Liu, C., Cao, Y., Luo, Y., Chen, G., Vokkarane, V., Ma, Y., et al. (2018). A New Deep Learning-Based Food Recognition System for Dietary Assessment on an Edge Computing Service Infrastructure. IEEE Trans. Serv. Comput. 11, 249–261. doi:10.1109/TSC.2017.2662008
Liu, M., Tuzel, O., Veeraraghavan, A., Taguchi, Y., Marks, T. K., and Chellappa, R. (2012). Fast Objet Localization and Pose Estimation in Heavy Clutter for Robotic Bin Picking. Int. J. Robotics Res. 31, 951–973. doi:10.1177/0278364911436018
Loebnitz, N., and Grunert, K. G. (2015). The Effect of Food Shape Abnormality on purchase Intentions in china. Food Qual. Preference 40, 24–30. doi:10.1016/j.foodqual.2014.08.005
Low, J. H., Khin, P. M., Han, Q. Q., Yao, H., Teoh, Y. S., Zeng, Y., et al. (2021). “Sensorized Reconfigurable Soft Robotic Gripper System for Automated Food Handling,” in IEEE/ASME Transactions On Mechatronics, Early Access. doi:10.1109/TMECH.2021.3110277
Ma, S., Du, L., Tsuchiya, E., and Fuchimi, M. (2020). “Paper-made Grippers for Soft Food Grasping,” in Proc. 2020 17th International Conference on Ubiquitous Robotics (IEEE), 362–367. doi:10.1109/ur49135.2020.9144853
Maruyama, R., Watanabe, T., and Uchida, M. (2013). “Delicate Grasping by Robotic Gripper with Incompressible Fluid-Based Deformable Fingertips,” in Proceedings of 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE), 5469–5474. doi:10.1109/iros.2013.6697148
Masey, R. J. M., Gray, J. O., Dodd, T. J., and Caldwell, D. G. (2010). Guidelines for the Design of Low-Cost Robots for the Food Industry. Ind. Robot: Int. J. 37, 509–517. doi:10.1108/01439911011081650
Matsuda, Y., Hoashi, H., and Yanai, K. (2012). “Recognition of Multiple-Food Images by Detecting Candidate Regions,” in Proc. 2012 IEEE International Conference on Multimedia and Expo (IEEE), 25–30. doi:10.1109/icme.2012.157
MITSUBISHI ELECTRIC, . (2021). Melfa-3d Vision. Available at: https://www.mitsubishielectric.com/fa/products/rbt/robot/pmerit/option/vision/index.html [accessed Sept. 10, 2021].
Muhammedali, B., Abdullah, M. Z., and Azemi, M. N. M. (2004). “Food Handling and Packaging Using Computer Vision and Robot,” in Proc. Internation Conference on Computer Grasphics, Imaging and Visualization (IEEE).
Myers, A., Johnston, N., Rathod, V., Korattikara, A., Gorban, A., Silberman, N., et al. (2015). “Im2calories: towards an Automated mobile Vision Food Diary,” in Proc. IEEE International Conference on Computer Vision (IEEE), 1233–1241. doi:10.1109/iccv.2015.146
Nishina, Y., and Hasegawa, T. (2020). Model-less Grasping Points Estimation for Bin-Picking of Non-rigid Objects and Irregular-Shaped Objects. Omron Technics 52, 1–8.
Ogawa, Y., Matsuura, M., and Yamamoto, N. (2015). Young’s Modulus and Poisson’s Ratio Changes in Japanese Radish and Carrot Root Tissues during Boiling. Int. J. Food Properties 18, 1006–1013. doi:10.1080/10942912.2013.879388
Okamoto, K., and Yanai, K. (2021). “Uec-foodpix Complete: A Large-Scale Food Image Segmentation Dataset,” in Proc. ICPR Workshop on Multimedia Assisted Dietary Management (Springer), 647–659. doi:10.1007/978-3-030-68821-9_51
OnRobot, A/S (2021). Soft Gripper - Flexible Food-Grade Robot Gripper, Available at: https://onrobot.com/en/products/soft-gripper (accessed 2021 Sept 01).
Petterson, A., Ohlsson, T., Caldwell, D. G., Davis, S., Gray, J., and Dodd, T. J. (2010). A Bernoulli Principle Gripper for Handling of Planar and 3d (Food) Products. Ind. Robot: Int. J. 37, 518–526. doi:10.1108/01439911011081669
Pettersson, A., Davis, S., Gray, J. O., Dodd, T. J., and Ohlsson, T. (2010). Design of a Magnetorheological Robot Gripper for Handling of Delicate Food Products with Varying Shapes. J. Food Eng. 98, 332–338. doi:10.1016/j.jfoodeng.2009.11.020
Pickit (2021). 3d Vision Applications Available at: https://www.pickit3d.com/en/3d-vision-applications (accessed Sept 10, 2021).
Robotics Inc., Soft (2021). Food-grade Automation Solutions, Available at: https://www.softroboticsinc.com/industries/food-and-beverage/(accessed Sept 10, 2021).
RightHand Robotics (2021). Piece-picking Solutions for Predictable Order Fulfillment Available at: https://www.righthandrobotics.com/(accessed Sept 10, 2021).
Sakamoto, N., Higashimori, M., Tsuji, T., and Kaneko, M. (2009). An Optimum Design of Robotic Food Handling by Using Burger Model. Intell. Serv. Robotics 2, 53–60. doi:10.1007/s11370-008-0032-5
Sakamoto, N., Higashimori, M., Tsuji, T., and Kaneko, M. (2007). “An Optimum Design of Robotic Hand for Handling a Visco-Elastic Object Based on maxwell Model,” in Proc. 2007 IEEE International Conference on Robotics and Automation (IEEE), 1219–1225. doi:10.1109/robot.2007.363151
Sam, R., and Nefti, S. (2010). “Design and Feasibility Tests of Multi-Functional Gripper for Handling Variable Shape of Food Products,” in Proc. 2010 IEEE International Conference on Systems, Man and Cybernetics (IEEE), 1267–1272. doi:10.1109/icsmc.2010.5642431
SCHUNK (2021). Food and Packaging, Available at: https://schunk.com/jp_en/solutions/industry-solutions/list/food-and-packaging/.(accessed Sept. 01 2021).
Singh, A. P., Lakes, R. S., and Gunasekaran, S. (2006). Viscoelastic Characterization of Selected Foods over an Extended Frequency Range. Rheologica Acta 46, 131–142. doi:10.1007/s00397-006-0107-1
Sinha, A., and Bhargav, A. (2020). Young’s Modulus Estimation in Food Samples: Effect of Experimental Parameters. Mech. Industry 21, 404. doi:10.1051/meca/2020025
SoftGripping (2021). The Modular Design System for Flexible Gripping, Available at: https://soft-gripping.com/(accessed Sept. 01 2021).
Song, S., Majidi, C., and Sitti, M. (2014). “Geckogripper: a Soft, Inflatable Robotic Gripper Using Gecko-Inspired Elastomer Micro-fiber Adhesives,” in Proc. 2014 IEEE/RSJ International Conference on Robots and System (IEEE), 4624–4629. doi:10.1109/iros.2014.6943218
SRT (2021). Intelligent Manufacturing Solutions, Available at: https://www.softrobottech.com/web/en/(accessed Sept. 01 2021).
The Metabolomics Innovation Centre (2021). Foodb. Available at: https://foodb.ca/(accessed Sept 15, 2021).
Traill, W. B., and Meulenberg, M. (2002). Innovation in the Food Industry. Agribusiness 18, 1–21. doi:10.1002/agr.10002
U.S. Department of Agriculture (2021). Fooddata central Available at: https://fdc.nal.usda.gov/(accessed Sept 15, 2021).
Vock, R., Dieckmann, A., Ochmann, S., and Klein, R. (2019). Fast Template Matching and Pose Estimation in 3d point Clouds. Comput. Graphics 79, 36–45. doi:10.1016/j.cag.2018.12.007
VUOTOTECNICA (2021). Suction Cups Based on Bernoulli’s Theorem Available at: https://www.vuototecnica.net/products.php?cat=1 (accessed Sept 23, 2021).
Wang, Z., Furata, H., Hirai, S., and Kawamura, S. (2021a). A Scooping-Binding Robotic Gripper for Handling Various Food Products. Front. Robotics AI 8, 640805. doi:10.3389/frobt.2021.640805
Wang, Z., and Hirai, S. (2012). Finite Element Modeling and Physical Property Estimation of Rheological Food Objects. J. Food Res. 1, 48–67. doi:10.5539/jfr.v1n1p48
Wang, Z., and Hirai, S. (2010). “Modeling and Parameter Estimation of Rheological Objects for Simultaneous Reproduction of Force and Deformation,” in Proc. 1st International Conference on Applied Bionics and Biomechanics.
Wang, Z., Inoue, S., Hashimoto, Y., and Kawamura, S. (2021b). Measuring Viscoelasticity and Friction of Tempuras for Robotic Handling. J. Food Eng. 310. doi:10.1016/j.jfoodeng.2021.110707
Wang, Z., Makiyama, Y., and Hirai, S. (2021c). A Soft Needle Gripper Capable of Grasping and Piercing for Handling Food Materials. J. Robotics Mechatronics 33, 935–943. doi:10.20965/jrm.2021.p0935
Wang, Z., Or, K., and Hirai, S. (2020). A Dual-Mode Soft Gripper for Food Packaging. Robotics Autonomous Syst. 125, 103427. doi:10.1016/j.robot.2020.103427
Weres, J., Nowakowski, K., Koszela, K., Rogacki, P., and Jarysz, M. (2009). “Image Analysis and 3d Geometry Modelling in Investigating Agri-Food Product Properties,” in Proc. EFITA Conference ’09 (EFITA), 67–72.
Williams, S. H., Wright, B. W., Truong, V. D., Daubert, C. R., and Vinyard, C. J. (2005). Mechanical Properties of Foods Used in Experimental Studies of Primate Masticatory Function. Am. J. Primatology 67, 329–346. doi:10.1002/ajp.20189
Wurdemann, H. A., Aminzadeh, V., Dai, J. S., Reed, J., and Purnell, G. (2011). Category-based Food Ordering Processes. Trends Food Sci. Tech. 22, 14–20. doi:10.1016/j.tifs.2010.10.003
Xie, Z., Domel, A. G., An, N., Green, C., Gong, Z., Wang, T., et al. (2020). Octopus Arm-Inspired Tapered Soft Actuators with Suckers for Improved Grasping. Soft Robotics 7, 639–648. doi:10.1089/soro.2019.0082
Yamaguchi, K., Hirata, Y., and Kosuge, K. (2013). “Development of Robot Hand with Suction Mechanism for Robust and Dexterous Grasping,” in Proc. 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE), 5500–5505. doi:10.1109/iros.2013.6697153
Keywords: food property, robotic end-effector, food recognition, food database, food handling, automation
Citation: Wang Z, Hirai S and Kawamura S (2022) Challenges and Opportunities in Robotic Food Handling: A Review. Front. Robot. AI 8:789107. doi: 10.3389/frobt.2021.789107
Received: 04 October 2021; Accepted: 22 December 2021;
Published: 13 January 2022.
Edited by:
David Navarro-Alarcon, Hong Kong Polytechnic University, Hong Kong SAR, ChinaReviewed by:
Jihong Zhu, Delft University of Technology, NetherlandsChaoyang Song, Southern University of Science and Technology, China
Juan Antonio Corrales Ramon, Sigma Clermont, France
Copyright © 2022 Wang, Hirai and Kawamura. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Zhongkui Wang, wangzk@fc.ritsumei.ac.jp