Abstract:Instance segmentation of foods is an important technology to ensure the food success rate of meal-assisting robotics. However, due to foods have strong intraclass variability, interclass similarity, and complex physical properties, which leads to more challenges in recognition, localization, and contour acquisition of foods. To address the above issues, this paper proposed a novel method for instance segmentation of foods. Specifically, in backbone network, deformable convolution was introduced to enhance the ability of YOLOv8 architecture to capture finer-grained spatial information, and efficient multiscale attention based on cross-spatial learning was introduced to improve sensitivity and expressiveness of multiscale inputs. In neck network, classical convolution and C2f modules were replaced by lightweight convolution GSConv and improved VoV-GSCSP aggregation module, respectively, to improve inference speed of models. We abbreviated it as the DEG-YOLOv8n-seg model. The proposed method was compared with baseline model and several state-of-the-art (SOTA) segmentation models on datasets, respectively. The results show that the DEG-YOLOv8n-seg model has higher accuracy, faster speed, and stronger robustness. Specifically, the DEG-YOLOv8n-seg model can achieve 84.6% Box_mAP@0.5 and 84.1% Mask_mAP@0.5 accuracy at 55.2 FPS and 11.1 GFLOPs. The importance of adopting data augmentation and the effectiveness of introducing deformable convolution, EMA, and VoV-GSCSP were verified by ablation experiments. Finally, the DEG-YOLOv8n-seg model was applied to experiments of food instance segmentation for meal-assisting robots. The results show that the DEG-YOLOv8n-seg can achieve better instance segmentation of foods. This work can promote the development of intelligent meal-assisting robotics technology and can provide theoretical foundations for other tasks of the computer vision field with some reference value.

A Chinese Dish Detector with Modified YOLO v3

A Lightweight SE-YOLOv3 Network for Multi-Scale Object Detection in Remote Sensing Imagery.

Chinese Dish Segmentation Based on Local Variation Driven Superpixel Grouping and Region Analysis

Food Image Recognition and Food Safety Detection Method Based on Deep Learning

Mixed Dish Recognition with Contextual Relation and Domain Alignment

Multi-Task Learning for Food Identification and Analysis with Deep Convolutional Neural Networks

Detection of Small Foreign Objects in Pu-erh Sun-Dried Green Tea: an Enhanced YOLOv8 Neural Network Model Based on Deep Learning

Enhanced Mask R-CNN for Chinese Food Image Detection

Deep-Learning-Assisted Multi-Dish Food Recognition Application for Dietary Intake Reporting

Where and What to Eat: Simultaneous Restaurant and Dish Recognition from Food Image

ChineseFoodNet: A large-scale Image Dataset for Chinese Food Recognition

Fine-grained recognition of Chinese food image based on DenseNet with attention mechanism

Real-time and accurate model of instance segmentation of foods

Recognition of Chinese food using convolutional neural network

DeepFood: Deep Learning-Based Food Image Recognition for Computer-Aided Dietary Assessment

YOLO-GD: A Deep Learning-Based Object Detection Algorithm for Empty-Dish Recycling Robots

Sushi Dish - Object detection and classification from real images

Convolutional Neural Network for Object Detection in Garlic Root Cutting Equipment

Mixed Dish Recognition Through Multi-Label Learning.

Dietlens-Eout: Large Scale Restaurant Food Photo Recognition

Automatic Chinese Food recognition based on a stacking fusion model