Citation: Zhu, D.; Seki, H.; Tsuji, T.;
Hiramitsu, T. Tableware Tidying-Up
Robot System for Self-Service
Restaurant–Detection and
Manipulation of Leftover Food and
Tableware-. Sensors 2022, 22, 7006.
https://doi.org/10.3390/s22187006
Academic Editors: Shuai Li,
Dechao Chen, Vasilios N. Katsikis,
Predrag Stanimirovi´c, Dunhui Xiao
and Mohammed Aquil Mirza
Received: 3 August 2022
Accepted: 13 September 2022
Published: 15 September 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Article
Tableware Tidying-Up Robot System for Self-Service
Restaurant–Detection and Manipulation of Leftover Food
and Tableware-
Deheng Zhu
1,
* , Hiroaki Seki
2
, Tokuo Tsuji
2
and Tatsuhiro Hiramitsu
2
1
Division of Mechanical Science and Engineering, Kanazawa University,
Kakuma, Kanazawa 920-1192, Ishikawa, Japan
2
Institute of Science and Engineering, Kanazawa University, Kakuma, Kanazawa 920-1192, Ishikawa, Japan
* Correspondence: zhudeheng2013@yahoo.co.jp
Abstract:
In this study, an automated tableware tidying-up robot system was developed to tidy up
tableware in a self-service restaurant with a large amount of tableware. This study focused on sorting
and collecting tableware placed on trays detected by an RGB-D camera. Leftover food was also
treated with this robot system. The RGB-D camera efficiently detected the position and height of the
tableware and whether there was leftover food or not by image processing. A parallel arm and robot
hand mechanism was designed to realize the advantages of a low cost and high processing speed.
Two types of rotation mechanisms were designed to realize the function of throwing away leftover
food. The effectiveness of the camera detection system was verified through the experiments of
tableware and leftover food detection. The effectiveness of the prototype robot and the rotation assist
mechanism was verified through the experiments of grasping tableware, throwing away leftover food
by two types of rotating mechanisms, collecting multiple tableware, and the sorting of overlapping
tableware with multiple robots.
Keywords:
parallel arm; leftover food treatment; tableware; tidying-up robot; RGB-D camera; self-
service restaurant; air cylinder
1. Introduction
In recent years, the labor shortage has become a major challenge with the advent of an
aging society. Robots are expected to be introduced to automate human tasks. The chain
in the meal industry includes food production, food provision, table cleaning, tableware
sorting, and cleaning. Some studies designed a two-handed robot for cooking, expanding
the motion sparseness, optimizing the mission, and shortening the cooking time [
1
,
2
]. Some
studies focusing on food provision used multiple robots designed with a ringed robot by
planning the robot’s running trajectory [
3
,
4
]. Other studies reported on the collection of
tableware from a table by using a single motor to operate multiple fingers simultaneously
and attaching a camera to grasp the overall condition of the tableware [
5
,
6
]. Some research
on tableware sorting were also primarily conducted for homes with some tableware, for
which the length of time does not matter. A six-axis serial robot with a hand with many
sensors was developed to trace the tableware to grasp the state of the tableware and then
put it into the washing machine [
7
]. In this study, it was reported to take 1 min 42 s to clean
up one piece of tableware, which is time-consuming. High-speed processing is necessary
to tidy up tableware, especially in a self-service restaurant environment where the quantity
of tableware is large and the work is monotonous. Concerning these issues, this study
focused on following the characteristics of tidying up tableware in a self-service restaurant,
“After a person puts the tableware on the tray the amount of tableware is large, the type
of tableware is limited, and the time to tidy up the tableware needs to be shortened”. The
contribution of this study is that we proposed a method to detect overlapping tableware
Sensors 2022, 22, 7006. https://doi.org/10.3390/s22187006 https://www.mdpi.com/journal/sensors
