Abstract:Ingredient prediction has received more and more attention with the help of image processing for its diverse real-world applications, such as nutrition intake management and cafeteria self-checkout system. Existing approaches mainly focus on multi-task food category-ingredient joint learning to improve final recognition by introducing task relevance, while seldom pay attention to making good use of inherent characteristics of ingredients independently. Actually, there are two issues for ingredient prediction. First, compared with fine-grained food recognition, ingredient prediction needs to extract more comprehensive features of the same ingredient and more detailed features of various ingredients from different regions of the food image. Because it can help understand various food compositions and distinguish the differences within ingredient features. Second, the ingredient distributions are extremely unbalanced. Existing loss functions can not simultaneously solve the imbalance between positive-negative samples belonging to each ingredient and significant differences among all classes. To solve these problems, we propose a novel framework named Class-Adaptive Context Learning Network (CACLNet) for ingredient prediction. In order to extract more comprehensive and detailed features, we introduce Ingredient Context Learning (ICL) to reduce the negative impact of complex background in food images and construct internal spatial connections among ingredient regions of food objects in a self-supervised manner, which can strengthen the contacts of the same ingredients through region interactions. In order to solve the imbalance of different classes among ingredients, we propose one novel Class-Adaptive Asymmetric Loss (CAAL) to focus on various ingredient classes adaptively. Besides, considering that the over-suppression of negative samples will over-fit positive samples of those rare ingredients, CAAL alleviates this continuous suppression according to the imbalanced ratios based on gradients while maintaining the contribution of positive samples by lesser suppression. Extensive evaluation on two popular benchmark datasets (Vireo Food-172, UEC Food-100) demonstrates our proposed method achieves the state-of-the-art performance. Further qualitative analysis and visualization show the effectiveness of our method. Code and models are available at https://123.57.42.89/codes/CACLNet/index.html.

Dynamic Mixup for Multi-Label Long-Tailed Food Ingredient Recognition

Mix from Failure: Confusion-Pairing Mixup for Long-Tailed Recognition

Robust Deep Neural Network for Learning in Noisy Multi-Label Food Images

Harnessing Hard Mixed Samples with Decoupled Regularizer

A Study of Multi-Task and Region-Wise Deep Learning for Food Ingredient Recognition.

Recognizing Multiple Ingredients in Food Images Using a Single-Ingredient Classification Model

Food Ingredients Recognition through Multi-label Learning

Decoupled Mixup for Data-efficient Learning

Enhanced Long-Tailed Recognition with Contrastive CutMix Augmentation

Text-Guided Mixup Towards Long-Tailed Image Categorization

SnapMix: Semantically Proportional Mixing for Augmenting Fine-grained Data

Ingredient Prediction via Context Learning Network With Class-Adaptive Asymmetric Loss

Long-Tailed Continual Learning For Visual Food Recognition

Sequential learning for ingredient recognition from images

Mixed Dish Recognition Through Multi-Label Learning.

SMMix: Self-Motivated Image Mixing for Vision Transformers

SUMix: Mixup with Semantic and Uncertain Information

ConfidentMix: Confidence-Guided Mixup for Learning With Noisy Labels

Un-mix: Rethinking Image Mixtures for Unsupervised Visual Representation Learning

Mixed Mutual Transfer for Long-Tailed Image Classification

Learning Using Privileged Information for Food Recognition