Abstract:Models based on convolutional neural networks (CNN) and transformers have steadily been improved. They also have been applied in various computer vision downstream tasks. However, in object detection tasks, accurately localizing and classifying almost infinite categories of foods in images remains challenging. To address these problems, we first segmented the food as the region-of-interest (ROI) by using the segment-anything model (SAM) and masked the rest of the region except ROI as black pixels. This process simplified the problems into a single classification for which annotation and training were much simpler than object detection. The images in which only the ROI was preserved were fed as inputs to fine-tune various off-the-shelf models that encoded their own inductive biases. Among them, Data-efficient image Transformers (DeiTs) had the best classification performance. Nonetheless, when foods' shapes and textures were similar, the contextual features of the ROI-only images were not enough for accurate classification. Therefore, we introduced a novel type of combined architecture, RveRNet, which consisted of ROI, extra-ROI, and integration modules that allowed it to account for both the ROI's and global contexts. The RveRNet's F1 score was 10% better than other individual models when classifying ambiguous food images. If the RveRNet's modules were DeiT with the knowledge distillation from the CNN, performed the best. We investigated how architectures can be made robust against input noise caused by permutation and translocation. The results indicated that there was a trade-off between how much the CNN teacher's knowledge could be distilled to DeiT and DeiT's innate strength. Code is publicly available at: <a class="link-external link-https" href="https://github.com/Seonwhee-Genome/RveRNet" rel="external noopener nofollow">this https URL</a>.

Food Det: Detecting Foods in Refrigerator with Supervised Transformer Network

High Precision Food Detection Method Based on Deep Object Detection Network

A Transformer-Based Object Detector with Coarse-Fine Crossing Representations

Fruits and vegetables image recognition in intelligent refrigerator based on convolutional neural network

Fruit Detection from Digital Images Using CenterNet

Research on Food Recognition of Smart Refrigerator Based on SSD Target Detection Algorithm

Food recognition via an efficient neural network with transformer grouping

A Transformer-Based Mask R-CNN for Tomato Detection and Segmentation

Research on the Classification Method of Refrigerator Ingredients Based on Inception Structure

Multi-source Data Fusion Using Deep Learning for Smart Refrigerators

Automatic food recognition based on efficientnet and ResNet

A Lightweight Hybrid Model with Location-Preserving ViT for Efficient Food Recognition

Food Image Recognition with Convolutional Neural Networks.

Enhancing Fruit and Vegetable Detection in Unconstrained Environment with a Novel Dataset

Real-time and Accurate Meal Detection for Meal-Assisting Robots

Research on RFID Localization Algorithm of Refrigerator Foods Based on Improved DenseNet Model

Food Image Recognition and Food Safety Detection Method Based on Deep Learning

Fine grained food image recognition based on swin transformer

Automatic Classification of Refrigerator Using Doubly Convolutional Neural Network with Jointly Optimized Classification Loss and Similarity Loss

Food Image Classification Based on Residual Network.

Knowledge distillation to effectively attain both region-of-interest and global semantics from an image where multiple objects appear