Abstract:In recent years, the technology of artificial intelligence (AI) and robots is rapidly spreading to countries around the world. More and more scholars and industry experts have proposed AI deep learning models and methods to solve human life problems and improve work efficiency. Modern people’s lives are very busy, which led us to investigate whether the demand for Bento buffet cafeterias has gradually increased in Taiwan. However, when eating at a buffet in a cafeteria, people often encounter two problems. The first problem is that customers need to queue up to check out after they have selected and filled their dishes from the buffet. However, it always takes too much time waiting, especially at lunch or dinner time. The second problem is sometimes customers question the charges calculated by cafeteria staff, claiming they are too expensive at the checkout counter. Therefore, it is necessary to develop an AI-enabled checkout system. The AI-enabled self-checkout system will help the Bento buffet cafeterias reduce long lineups without the need to add additional workers. In this paper, we used computer vision and deep-learning technology to design and implement an AI-enabled checkout system for Bento buffet cafeterias. The prototype contains an angle steel shelf, a Kinect camera, a light source, and a desktop computer. Six baseline convolutional neural networks were applied for comparison on food recognition. In our experiments, there were 22 different food categories in a Bento buffet cafeteria employed. Experimental results show that the inception_v4 model can achieve the highest average validation accuracy of 99.11% on food recognition, but it requires the most training and recognition time. AlexNet model achieves a 94.5% accuracy and requires the least training time and recognition time. We propose a hierarchical approach with two stages to achieve good performance in both the recognition accuracy rate and the required training and recognition time. The approach is designed to perform the first step of identification and the second step of recognizing similar food images, respectively. Experimental results show that the proposed approach can achieve a 96.3% accuracy rate on our test dataset and required very little recognition time for input images. In addition, food volumes could be estimated using the depth images captured by the Kinect camera, and a framework of visual checkout system was successfully built.

Self-Supervised Multi-Category Counting Networks for Automatic Check-Out

Data Priming Network for Automatic Check-Out

Multi-branch Progressive Embedding Network for Crowd Counting

Object Detection for Vision-Aided Inventory Counting

Prototype Learning for Automatic Check-Out

Split-Check: Boosting Product Recognition Via Instance-Level Retrieval

Uniformity in Heterogeneity: Diving Deep into Count Interval Partition for Crowd Counting

Smart retail SKUs checkout using improved residual network

A Region-Based Deep Learning Approach to Automated Retail Checkout

Return of Small-Scale Crowd Counting via Fast and Accurate Semi-Supervised Least Squares Model

SCD: A Stacked Carton Dataset for Detection and Segmentation

Shelf Auditing Based on Image Classification Using Semi-Supervised Deep Learning to Increase On-Shelf Availability in Grocery Stores

Computer Vision-Based Quantity Detection of Goods in Vending Cabinets

A Framework of Visual Checkout System Using Convolutional Neural Networks for Bento Buffet

Product Recognition for Unmanned Vending Machines

Scale-Aware Network with Regional and Semantic Attentions for Crowd Counting under Cluttered Background

Enhanced Self-Checkout System for Retail Based on Improved YOLOv10

RPC: A Large-Scale Retail Product Checkout Dataset

ARC: A Vision-based Automatic Retail Checkout System

Dilated-Scale-Aware Attention ConvNet For Multi-Class Object Counting

Attend to Count: Crowd Counting with Adaptive Capacity Multi-Scale CNNs.