Abstract:Ingredient prediction has received more and more attention with the help of image processing for its diverse real-world applications, such as nutrition intake management and cafeteria self-checkout system. Existing approaches mainly focus on multi-task food category-ingredient joint learning to improve final recognition by introducing task relevance, while seldom pay attention to making good use of inherent characteristics of ingredients independently. Actually, there are two issues for ingredient prediction. First, compared with fine-grained food recognition, ingredient prediction needs to extract more comprehensive features of the same ingredient and more detailed features of various ingredients from different regions of the food image. Because it can help understand various food compositions and distinguish the differences within ingredient features. Second, the ingredient distributions are extremely unbalanced. Existing loss functions can not simultaneously solve the imbalance between positive-negative samples belonging to each ingredient and significant differences among all classes. To solve these problems, we propose a novel framework named Class-Adaptive Context Learning Network (CACLNet) for ingredient prediction. In order to extract more comprehensive and detailed features, we introduce Ingredient Context Learning (ICL) to reduce the negative impact of complex background in food images and construct internal spatial connections among ingredient regions of food objects in a self-supervised manner, which can strengthen the contacts of the same ingredients through region interactions. In order to solve the imbalance of different classes among ingredients, we propose one novel Class-Adaptive Asymmetric Loss (CAAL) to focus on various ingredient classes adaptively. Besides, considering that the over-suppression of negative samples will over-fit positive samples of those rare ingredients, CAAL alleviates this continuous suppression according to the imbalanced ratios based on gradients while maintaining the contribution of positive samples by lesser suppression. Extensive evaluation on two popular benchmark datasets (Vireo Food-172, UEC Food-100) demonstrates our proposed method achieves the state-of-the-art performance. Further qualitative analysis and visualization show the effectiveness of our method. Code and models are available at https://123.57.42.89/codes/CACLNet/index.html.

Foodnet: Multi-Scale and Label Dependency Learning-Based Multi-Task Network for Food and Ingredient Recognition

Region-Level Attention Network for Food and Ingredient Joint Recognition

Food Ingredients Recognition through Multi-label Learning

MVANet: Multi-Task Guided Multi-View Attention Network for Chinese Food Recognition

A Study of Multi-Task and Region-Wise Deep Learning for Food Ingredient Recognition.

Food and Ingredient Joint Learning for Fine-Grained Recognition

Learning Using Privileged Information for Food Recognition

ChineseFoodNet: A large-scale Image Dataset for Chinese Food Recognition

Fine-grained recognition of Chinese food image based on DenseNet with attention mechanism

Fine-grained food image classification and recipe extraction using a customized deep neural network and NLP

Food Image Recognition via Multi-scale Jigsaw and Reconstruction Network

Automatic food recognition based on efficientnet and ResNet

Ingredient-Guided Cascaded Multi-Attention Network for Food Recognition

Multi-Task Learning for Food Identification and Analysis with Deep Convolutional Neural Networks

A fine-grained recognition technique for identifying Chinese food images

DeepFood: Automatic Multi-Class Classification of Food Ingredients Using Deep Learning

Convolution-Enhanced Bi-Branch Adaptive Transformer With Cross-Task Interaction for Food Category and Ingredient Recognition

Image Augmentation-Based Food Recognition with Convolutional Neural Networks

Recognizing Multiple Ingredients in Food Images Using a Single-Ingredient Classification Model

Ingredient Prediction via Context Learning Network With Class-Adaptive Asymmetric Loss

Attention-Based Convolutional Neural Network for Ingredients Identification