Abstract:In recent years, the technology of artificial intelligence (AI) and robots is rapidly spreading to countries around the world. More and more scholars and industry experts have proposed AI deep learning models and methods to solve human life problems and improve work efficiency. Modern people’s lives are very busy, which led us to investigate whether the demand for Bento buffet cafeterias has gradually increased in Taiwan. However, when eating at a buffet in a cafeteria, people often encounter two problems. The first problem is that customers need to queue up to check out after they have selected and filled their dishes from the buffet. However, it always takes too much time waiting, especially at lunch or dinner time. The second problem is sometimes customers question the charges calculated by cafeteria staff, claiming they are too expensive at the checkout counter. Therefore, it is necessary to develop an AI-enabled checkout system. The AI-enabled self-checkout system will help the Bento buffet cafeterias reduce long lineups without the need to add additional workers. In this paper, we used computer vision and deep-learning technology to design and implement an AI-enabled checkout system for Bento buffet cafeterias. The prototype contains an angle steel shelf, a Kinect camera, a light source, and a desktop computer. Six baseline convolutional neural networks were applied for comparison on food recognition. In our experiments, there were 22 different food categories in a Bento buffet cafeteria employed. Experimental results show that the inception_v4 model can achieve the highest average validation accuracy of 99.11% on food recognition, but it requires the most training and recognition time. AlexNet model achieves a 94.5% accuracy and requires the least training time and recognition time. We propose a hierarchical approach with two stages to achieve good performance in both the recognition accuracy rate and the required training and recognition time. The approach is designed to perform the first step of identification and the second step of recognizing similar food images, respectively. Experimental results show that the proposed approach can achieve a 96.3% accuracy rate on our test dataset and required very little recognition time for input images. In addition, food volumes could be estimated using the depth images captured by the Kinect camera, and a framework of visual checkout system was successfully built.

An Intelligent Master–slave Collaborative Robot System for Cafeteria Service

A Human-Robot Collaboration System for Object Handover

An Improved Supervoxel Clustering Algorithm of 3D Point Clouds for the Localization of Industrial Robots

CrowdBot: an Open-Environment Robot Management System for On-Campus Services

Design of an Intelligent Service Robot based on Deep Learning

Design and implementation of an autonomous service robot based on cyber physical modeling systems

Intelligent Mobile Robot Controller Design for Hotel Room Service with Deep Learning Arm-Based Elevator Manipulator

Autonomous dishwasher loading from cluttered trays using pre‐trained deep neural networks

A Novel Continuous Learning and Collaborative Decision Making Mechanism for Real-Time Cooperation of Humanoid Service Robots

A cyber-physical robotic mobile fulfillment system in smart manufacturing: The simulation aspect

Multi-scale deep learning and clustering-based tabletop object instance segmentation for robot manipulation

A Deep Learning-Based Autonomous RobotManipulator for Sorting Application

Robotic picking in dense clutter via domain invariant learning from synthetic dense cluttered rendering

Dynamic Open-Vocabulary 3D Scene Graphs for Long-term Language-Guided Mobile Manipulation

A Food Package Recognition and Sorting System Based on Structured Light and Deep Learning

Robot Skill Learning and Generalization Based on Human-robot Collaboration and Computer Vision

A Framework of Visual Checkout System Using Convolutional Neural Networks for Bento Buffet

Robot-Assisted Feeding: Generalizing Skewering Strategies across Food Items on a Realistic Plate

RGB-D-Based Pose Estimation of Workpieces with Semantic Segmentation and Point Cloud Registration

A pushing-grasping collaborative method based on deep Q-network algorithm in dual viewpoints

Cloud robot: semantic map building for intelligent service task