Abstract:Food computing brings various perspectives to computer vision like vision-based food analysis for nutrition and health. As a fundamental task in food computing, food detection needs Zero-Shot Detection (ZSD) on novel unseen food objects to support real-world scenarios, such as intelligent kitchens and smart restaurants. Therefore, we first benchmark the task of Zero-Shot Food Detection (ZSFD) by introducing FOWA dataset with rich attribute annotations. Unlike ZSD, fine-grained problems in ZSFD like inter-class similarity make synthesized features inseparable. The complexity of food semantic attributes further makes it more difficult for current ZSD methods to distinguish various food categories. To address these problems, we propose a novel framework ZSFDet to tackle fine-grained problems by exploiting the interaction between complex attributes. Specifically, we model the correlation between food categories and attributes in ZSFDet by multi-source graphs to provide prior knowledge for distinguishing fine-grained features. Within ZSFDet, Knowledge-Enhanced Feature Synthesizer (KEFS) learns knowledge representation from multiple sources (e.g., ingredients correlation from knowledge graph) via the multi-source graph fusion. Conditioned on the fusion of semantic knowledge representation, the region feature diffusion model in KEFS can generate fine-grained features for training the effective zero-shot detector. Extensive evaluations demonstrate the superior performance of our method ZSFDet on FOWA and the widely-used food dataset UECFOOD-256, with significant improvements by 1.8% and 3.7% ZSD mAP compared with the strong baseline RRFS. Further experiments on PASCAL VOC and MS COCO prove that enhancement of the semantic knowledge can also improve the performance on general ZSD. Code and dataset are available at

ESE-GAN: Zero-Shot Food Image Classification Based on Low Dimensional Embedding of Visual Features

GENERATING MANIFOLD-ALIGNED SEMANTIC FEATURE FOR ZERO-SHOT LEARNING

Contrastive Visual Feature Filtering for Generalized Zero-Shot Learning

Cluster-based Contrastive Disentangling for Generalized Zero-Shot Learning

Zero-shot image classification via Visual–Semantic Feature Decoupling

Visual Data Synthesis Via GAN for Zero-Shot Video Classification

Zero-VAE-GAN: Generating Unseen Features for Generalized and Transductive Zero-Shot Learning

VGSE: Visually-Grounded Semantic Embeddings for Zero-Shot Learning

Weakly Supervised Classification Model for Zero‐shot Semantic Segmentation

Zero-Shot Image Classification with Rectified Embedding Vectors Using a Caption Generator

Learning discriminative visual semantic embedding for zero-shot recognition

Zero-Shot Recognition Based on Semantic Embeddings and Deep Clustering

Synthesizing Knowledge-enhanced Features for Real-world Zero-shot Food Detection

Data Driven Recurrent Generative Adversarial Network for Generalized Zero Shot Image Classification

Towards Effective Deep Embedding for Zero-Shot Learning

Learning discriminative and representative feature with cascade GAN for generalized zero-shot learning

Zero-Shot Learning via Discriminative Dual Semantic Auto-Encoder

Fine-Grained Feature Generation for Generalized Zero-Shot Video Classification

Zero-Shot Learning on Semantic Class Prototype Graph

Zero-Shot Ingredient Recognition by Multi-Relational Graph Convolutional Network

Semi-Supervised Low-Rank Semantics Grouping for Zero-Shot Learning