Intracellular localization of glutamine synthetase in Chlorogloeopsis fritschii

A. Sallal

DOI: https://doi.org/10.1007/BF00144457

1994-07-01

Abstract:

What problem does this paper attempt to address?

Transductive Multi-View Embedding For Zero-Shot Recognition And Annotation

Yanwei Fu,Timothy M. Hospedales,Tao Xiang,Zhen-Yong Fu,Shaogang Gong

DOI: https://doi.org/10.1007/978-3-319-10605-2_38

2014-01-01

Abstract:Most existing zero-shot learning approaches exploit transfer learning via an intermediate-level semantic representation such as visual attributes or semantic word vectors. Such a semantic representation is shared between an annotated auxiliary dataset and a target dataset with no annotation. A projection from a low-level feature space to the semantic space is learned from the auxiliary dataset and is applied without adaptation to the target dataset. In this paper we identify an inherent limitation with this approach. That is, due to having disjoint and potentially unrelated classes, the projection functions learned from the auxiliary dataset/domain are biased when applied directly to the target dataset/domain. We call this problem the projection domain shift problem and propose a novel framework, transductive multi-view embedding, to solve it. It is 'transductive' in that unlabelled target data points are explored for projection adaptation, and 'multi-view' in that both low-level feature (view) and multiple semantic representations (views) are embedded to rectify the projection shift. We demonstrate through extensive experiments that our framework (1) rectifies the projection shift between the auxiliary and target domains, (2) exploits the complementarity of multiple semantic representations, (3) achieves state-of-the-art recognition results on image and video benchmark datasets, and (4) enables novel cross-view annotation tasks.
Transductive Multi-class and Multi-label Zero-shot Learning

Yanwei Fu,Yongxin Yang,Timothy M. Hospedales,Tao Xiang,Shaogang Gong

DOI: https://doi.org/10.48550/arxiv.1503.07884

2015-01-01

Abstract:Recently, zero-shot learning (ZSL) has received increasing interest. The key idea underpinning existing ZSL approaches is to exploit knowledge transfer via an intermediate-level semantic representation which is assumed to be shared between the auxiliary and target datasets, and is used to bridge between these domains for knowledge transfer. The semantic representation used in existing approaches varies from visual attributes to semantic word vectors and semantic relatedness. However, the overall pipeline is similar: a projection mapping low-level features to the semantic representation is learned from the auxiliary dataset by either classification or regression models and applied directly to map each instance into the same semantic representation space where a zero-shot classifier is used to recognise the unseen target class instances with a single known 'prototype' of each target class. In this paper we discuss two related lines of work improving the conventional approach: exploiting transductive learning ZSL, and generalising ZSL to the multi-label case.
Transductive Multi-label Zero-shot Learning.

Yanwei Fu,Yongxin Yang,Tim Hospedales,Tao Xiang,Shaogang Gong

DOI: https://doi.org/10.5244/c.28.7

2014-01-01

Abstract:Zero-shot learning has received increasing interest as a means to alleviate the often prohibitive expense of annotating training data for large scale recognition problems. These methods have achieved great success via learning intermediate semantic representations in the form of attributes and more recently, semantic word vectors. However, they have thus far been constrained to the single-label case, in contrast to the growing popularity and importance of more realistic multi-label data. In this paper, for the first time, we investigate and formalise a general framework for multi-label zero-shot learning, addressing the unique challenge therein: how to exploit multi-label correlation at test time with no training data for those classes? In particular, we propose (1) a multi-output deep regression model to project an image into a semantic word space, which explicitly exploits the correlations in the intermediate semantic layer of word vectors; (2) a novel zero-shot learning algorithm for multi-label data that exploits the unique compositionality property of semantic word vector representations; and (3) a transductive learning strategy to enable the regression model learned from seen classes to generalise well to unseen classes. Our zero-shot learning experiments on a number of standard multi-label datasets demonstrate that our method outperforms a variety of baselines.
Transductive Unbiased Embedding for Zero-Shot Learning

Jie Song,Chengchao Shen,Yezhou Yang,Yang Liu,Mingli Song

DOI: https://doi.org/10.1109/cvpr.2018.00113

2018-01-01

Abstract:Most existing Zero-Shot Learning (ZSL) methods have the strong bias problem, in which instances of unseen (target) classes tend to be categorized as one of the seen (source) classes. So they yield poor performance after being deployed in the generalized ZSL settings. In this paper, we propose a straightforward yet effective method named Quasi-Fully Supervised Learning (QFSL) to alleviate the bias problem. Our method follows the way of transductive learning, which assumes that both the labeled source images and unlabeled target images are available for training. In the semantic embedding space, the labeled source images are mapped to several fixed points specified by the source categories, and the unlabeled target images are forced to be mapped to other points specified by the target categories. Experiments conducted on AwA2, CUB and SUN datasets demonstrate that our method outperforms existing state-of-the-art approaches by a huge margin of 9.3 ~ 24.5% following generalized ZSL settings, and by a large margin of 0.2 ~ 16.2% following conventional ZSL settings.
Dual Collaborative Visual-Semantic Mapping for Multi-Label Zero-Shot Image Recognition

Yunqing Hu,Xuan Jin,Xi Chen,Yin Zhang

DOI: https://doi.org/10.1109/icassp49357.2023.10095788

2023-01-01

Abstract:Multi-label zero-shot learning (ML-ZSL), with the difficulty of both multi-label learning and zero-shot learning, aims to recognize various unseen objects that are not observed during training. Previous methods mainly use a single directional visual-semantic mapping to associate the visual and semantic embedding space, which is not sufficient to adequately realize knowledge transfer from seen to unseen classes. In this paper, we propose a novel dual collaborative visual-semantic mapping framework, constructing abundant connection relationships by exploring two aspects of mapping streams, i.e., the visual-to-semantic (V2S) mapping and the semantic-to-visual (S2V) mapping. Through the collaborative learning of these two effective mappings, our method achieves state-of-the-art performance on the MS-COCO and PASCAL-VOC, two benchmarks for ML-ZSL.
Domain Adaptation Meets Zero-Shot Learning: an Annotation-Efficient Approach to Multi-Modality Medical Image Segmentation

Cheng Bian,Chenglang Yuan,Kai Ma,Shuang Yu,Dong Wei,Yefeng Zheng

DOI: https://doi.org/10.1109/tmi.2021.3131245

IF: 10.6

2022-01-01

IEEE Transactions on Medical Imaging

Abstract:Due to the lack of properly annotated medical data, exploring the generalization capability of the deep model is becoming a public concern. Zero-shot learning (ZSL) has emerged in recent years to equip the deep model with the ability to recognize unseen classes. However, existing studies mainly focus on natural images, which utilize linguistic models to extract auxiliary information for ZSL. It is impractical to apply the natural image ZSL solutions directly to medical images, since the medical terminology is very domain-specific, and it is not easy to acquire linguistic models for the medical terminology. In this work, we propose a new paradigm of ZSL specifically for medical images utilizing cross-modality information. We make three main contributions with the proposed paradigm. First, we extract the prior knowledge about the segmentation targets, called relation prototypes, from the prior model and then propose a cross-modality adaptation module to inherit the prototypes to the zero-shot model. Second, we propose a relation prototype awareness module to make the zero-shot model aware of information contained in the prototypes. Last but not least, we develop an inheritance attention module to recalibrate the relation prototypes to enhance the inheritance process. The proposed framework is evaluated on two public cross-modality datasets including a cardiac dataset and an abdominal dataset. Extensive experiments show that the proposed framework significantly outperforms the state of the arts.
Matrix Tri-Factorization With Manifold Regularizations For Zero-Shot Learning

Xing Xu,Fumin Shen,Yang Yang,Dongxiang Zhang,Heng Tao Shen,Jingkuan Song

DOI: https://doi.org/10.1109/CVPR.2017.217

2017-01-01

Abstract:Zero-shot learning (ZSL) aims to recognize objects of unseen classes with available training data from another set of seen classes. Existing solutions are focused on exploring knowledge transfer via an intermediate semantic embedding (e.g., attributes) shared between seen and unseen classes. In this paper, we propose a novel projection framework based on matrix tri-factorization with manifold regularizations. Specifically, we learn the semantic embedding projection by decomposing the visual feature matrix under the guidance of semantic embedding and class label matrices. By additionally introducing manifold regularizations on visual data and semantic embeddings, the learned projection can effectively capture the geometrical manifold structure residing in both visual and semantic spaces. To avoid the projection domain shift problem, we devise an effective prediction scheme by exploiting the test-time manifold structure. Extensive experiments on four benchmark datasets show that our approach significantly outperforms the state-of-the-arts, yielding an average improvement ratio by 7.4% and 31.9% for the recognition and retrieval task, respectively.
Transferrable Feature and Projection Learning with Class Hierarchy for Zero-Shot Learning.

Li Aoxue,Lu Zhiwu,Guan Jiechao,Xiang Tao,Wang Liwei,Wen Ji-Rong

DOI: https://doi.org/10.1007/s11263-020-01342-x

IF: 13.369

2020-01-01

International Journal of Computer Vision

Abstract:Zero-shot learning (ZSL) aims to transfer knowledge from seen classes to unseen ones so that the latter can be recognised without any training samples. This is made possible by learning a projection function between a feature space and a semantic space (e.g. attribute space). Considering the seen and unseen classes as two domains, a big domain gap often exists which challenges ZSL. In this work, we propose a novel inductive ZSL model that leverages superclasses as the bridge between seen and unseen classes to narrow the domain gap. Specifically, we first build a class hierarchy of multiple superclass layers and a single class layer, where the superclasses are automatically generated by data-driven clustering over the semantic representations of all seen and unseen class names. We then exploit the superclasses from the class hierarchy to tackle the domain gap challenge in two aspects: deep feature learning and projection function learning. First, to narrow the domain gap in the feature space, we define a recurrent neural network over superclasses and then plug it into a convolutional neural network for enforcing the superclass hierarchy. Second, to further learn a transferrable projection function for ZSL, a novel projection function learning method is proposed by exploiting the superclasses to align the two domains. Importantly, our transferrable feature and projection learning methods can be easily extended to a closely related task—few-shot learning (FSL). Extensive experiments show that the proposed model outperforms the state-of-the-art alternatives in both ZSL and FSL tasks.
Transformer-Based Approach Via Contrastive Learning for Zero-Shot Detection.

Wei Liu,Hui Chen,Yongqiang Ma,Jianji Wang,Nanning Zheng

DOI: https://doi.org/10.1142/s0129065723500351

IF: 6.325

2023-01-01

International Journal of Neural Systems

Abstract:Zero-shot detection (ZSD) aims to locate and classify unseen objects in pictures or videos by semantic auxiliary information without additional training examples. Most of the existing ZSD methods are based on two-stage models, which achieve the detection of unseen classes by aligning object region proposals with semantic embeddings. However, these methods have several limitations, including poor region proposals for unseen classes, lack of consideration of semantic representations of unseen classes or their inter-class correlations, and domain bias towards seen classes, which can degrade overall performance. To address these issues, the Trans-ZSD framework is proposed, which is a transformer-based multi-scale contextual detection framework that explicitly exploits inter-class correlations between seen and unseen classes and optimizes feature distribution to learn discriminative features. Trans-ZSD is a single-stage approach that skips proposal generation and performs detection directly, allowing the encoding of long-term dependencies at multiple scales to learn contextual features while requiring fewer inductive biases. Trans-ZSD also introduces a foreground-background separation branch to alleviate the confusion of unseen classes and backgrounds, contrastive learning to learn inter-class uniqueness and reduce misclassification between similar classes, and explicit inter-class commonality learning to facilitate generalization between related classes. Trans-ZSD addresses the domain bias problem in end-to-end generalized zero-shot detection (GZSD) models by using balance loss to maximize response consistency between seen and unseen predictions, ensuring that the model does not bias towards seen classes. The Trans-ZSD framework is evaluated on the PASCAL VOC and MS COCO datasets, demonstrating significant improvements over existing ZSD models.
Transductive discriminative dictionary learning approach for zero-shot classification

Zhong Ji,Tao Sun,Yun-Long Yu

DOI: https://doi.org/10.13328/j.cnki.jos.005338

2017-01-01

Journal of Software

Abstract:Zero-Shot classification aims at recognizing instances from unseen categories that have no training instances in the training stage. To address this task, most existing approaches resort to class semantic information to transfer knowledge from the seen classes to the unseen ones. In this paper, a transductive dictionary learning approach is proposed to facilitate the task in two steps. A discriminative dictionary learning model is first proposed for constructing the relations between the visual modality and the class semantic modality with the labeled seen instances. Then a transductive modified model is used to alleviate the domain shift issue caused by the disjointness between the seen classes and the unseen classes. Experimental results on three benchmark datasets (Aw A, CUB and SUN) demonstrate the effectiveness and superiority of the proposed approach.
Manifold Regularized Cross-Modal Embedding for Zero-Shot Learning

Zhong Ji,Yunlong Yu,Yanwei Pang,Jichang Guo,Zhongfei Zhang

DOI: https://doi.org/10.1016/j.ins.2016.10.025

IF: 8.1

2017-01-01

Information Sciences

Abstract:Zero-Shot Learning (ZSL) aims at classifying previously unseen class samples and has gained its popularity in applications where samples of some categories are scarce for training. The basic idea to address this issue is transferring knowledge from the seen classes to the unseen classes through mapping the visual feature to an embedding space spanned by class semantic information. The class semantic information can be obtained from human-labeled attributes or text corpus in an unsupervised fashion. Therefore, the embedding function from visual space to the embedding space is extremely important. However, the existing embedding approaches to ZSL mainly focus on aligning pairwise semantic consistency from heterogeneous spaces but ignore the intrinsic structure of the locally homogeneous isomorph. In order to preserve the locally visual structure in the embedding process, this paper proposes a Manifold regularized Cross-Modal Embedding (MCME) approach for ZSL by formulating the manifold constraint for intrinsic structure of the visual features as well as aligning pairwise consistency. The linear, closed-form solution makes MCME efficient to compute. Furthermore, rather than applying the embedding function learned from the seen classes directly, we also propose a new domain adaptation strategy to overcome the domain-shift problem during the knowledge transfer process. The MCME with the domain adaptation method is called MCME-DA. Extensive experiments on the benchmark datasets of AwA and CUB validate the superiority and promise of MCME and MCME-DA.
Transductive Zero-Shot Learning with a Self-Training Dictionary Approach

Yunlong Yu,Zhong Ji,Xi Li,Jichang Guo,Zhongfei Zhang,Haibin Ling,Fei Wu

DOI: https://doi.org/10.1109/tcyb.2017.2751741

IF: 11.8

2018-01-01

IEEE Transactions on Cybernetics

Abstract:As an important and challenging problem in computer vision, zero-shot learning (ZSL) aims at automatically recognizing the instances from unseen object classes without training data. To address this problem, ZSL is usually carried out in the following two aspects: 1) capturing the domain distribution connections between seen classes data and unseen classes data and 2) modeling the semantic interactions between the image feature space and the label embedding space. Motivated by these observations, we propose a bidirectional mapping-based semantic relationship modeling scheme that seeks for cross-modal knowledge transfer by simultaneously projecting the image features and label embeddings into a common latent space. Namely, we have a bidirectional connection relationship that takes place from the image feature space to the latent space as well as from the label embedding space to the latent space. To deal with the domain shift problem, we further present a transductive learning approach that formulates the class prediction problem in an iterative refining process, where the object classification capacity is progressively reinforced through bootstrapping-based model updating over highly reliable instances. Experimental results on four benchmark datasets (animal with attribute, Caltech-UCSD Bird2011, aPascal-aYahoo, and SUN) demonstrate the effectiveness of the proposed approach against the state-of-the-art approaches.
Manifold Embedding for Zero-Shot Recognition

Zhong Ji,Xuejie Yu,Yunlong Yu,Yuqing He

DOI: https://doi.org/10.1016/j.cogsys.2018.12.003

IF: 4.541

2019-01-01

Cognitive Systems Research

Abstract:Zero-Shot Recognition (ZSR) has gained its popularity recently owing to its promising characteristic that extends the classifiers to the unseen classes. It is typically addressed by resorting to a class semantic space to transfer the knowledge from the seen classes to unseen ones. Therefore, constructing the effective interactions between the visual space and the class semantic space is the key for ZSR. In this paper, under the assumption that the distribution of the semantic categories in the semantic space has an intrinsic manifold structure, we propose two manifold embedding-based ZSR approaches to capture the intrinsic structures of both the visual space and the class semantic space, i.e., ME-ZSR and MCCA-ZSR. Specifically, ME-ZSR builds embedding from visual space to semantic space, while MCCA-ZSR explores to embed both visual and semantic features into a common space. The linear, closed-form solutions make both methods efficient to optimize. Extensive experiments on three popular datasets AwA, CUB and NAB validate the effectiveness of both methods.
Multi-level Fusion of Multi-modal Semantic Embeddings for Zero Shot Learning

Yuhan Liu,Zhe Kong,Neng Gao,Chenyang Tu,Xin Eric Wang,Yifei Zhang

DOI: https://doi.org/10.1145/3536221.3556575

2022-11-07

Abstract:Zero shot learning aims to recognize objects whose instances may not be covered by the training data. To generalize knowledge from seen classes to the novel ones, semantic space is built to embed knowledge from various views into multi-modal semantic embeddings. Existing semantic embeddings neglect the relationships between classes which are essential to transfer knowledge between classes. Moreover, existing zero shot learning models ignore the complementarity between semantic embeddings from different modalities. To tackle these problems, in this work, we resort to graph theory to explicitly model the interdependence between classes and then obtain new modal semantic embeddings. Furthermore, we pioneer to propose a multi-level fusion model to effectively combine knowledge encoded in multi-modal semantic embeddings together. By the virtue of subsequent fusion block, the results of multi-level fusion can be furtherly enriched and fused. Experiments show that our model could achieve promising results on various datasets. Ablation study suggests that our method is well suited for zero shot learning.

Computer Science
Meta-Transfer Networks for Zero-Shot Learning

Yunlong Yu,Zhongfei Zhang,Jungong Han

2019-01-01

Abstract:Zero-Shot Learning (ZSL) aims at recognizing unseen categories using some class semantics of the categories. The existing studies mostly leverage the seen categories to learn a visual-semantic interaction model to infer the unseen categories. However, the disjointness between the seen and unseen categories cannot ensure that the models trained on the seen categories generalize well to the unseen categories. In this work, we propose an episode-based approach to accumulate experiences on addressing disjoint-ness issue by mimicking extensive classiﬁcation scenarios where training classes and test classes are disjoint. In each episode, a visual-semantic interaction model is ﬁrst trained on a subset of seen categories as a learner that provides an initial prediction for the rest disjoint seen categories and then a meta-learner ﬁne-tunes the learner by minimizing the differences between the prediction and the ground-truth labels in a pre-deﬁned space. By training extensive episodes on the seen categories, the model is trained to be an expert in predicting the mimetic unseen categories, which will generalize well to the real unseen categories. Extensive experiments on four datasets under both the traditional ZSL and generalized ZSL tasks show that our framework out-performs the state-of-the-art approaches by large margins.
Multi-view graph representation with similarity diffusion for general zero-shot learning

Beibei Yu,Cheng Xie,Peng Tang,Haoran Duan

DOI: https://doi.org/10.1016/j.neunet.2023.06.045

Abstract:Zero-shot learning (ZSL) aims to predict unseen classes without using samples of these classes in model training. The ZSL has been widely used in many knowledge-based models and applications to predict various parameters, including categories, subjects, and anomalies, in different domains. Nonetheless, most existing ZSL methods require the pre-defined semantics or attributes of particular data environments. Therefore, these methods are difficult to be applied to general data environments, such as ImageNet and other real-world datasets and applications. Recent research has tried to use open knowledge to enhance the ZSL methods to adapt it to an open data environment. However, the performance of these methods is relatively low, namely the accuracy is normally below 10%, which is due to the inadequate semantics that can be used from open knowledge. Moreover, the latest methods suffer from a significant "semantic gap" problem between the generated features of unseen classes and the real features of seen classes. To this end, this paper proposes a multi-view graph representation with a similarity diffusion model, applying the ZSL tasks to general data environments. This model applies a multi-view graph to enhance the semantics fully and proposes an innovative diffusion method to augment the graph representation. In addition, a feature diffusion method is proposed to augment the multi-view graph representation and bridge the semantic gap to realize zero-shot predicting. The results of numerous experiments in general data environments and on benchmark datasets show that the proposed method can achieve new state-of-the-art results in the field of general zero-shot learning. Furthermore, seven ablation studies analyze the effects of the settings and different modules of the proposed method on its performance in detail and prove the effectiveness of each module.
Zero-Shot Recognition Using Dual Visual-Semantic Mapping Paths.

Yanan Li,Donghui Wang,Huanhang Hu,Yuetan Lin,Yueting Zhuang

DOI: https://doi.org/10.1109/cvpr.2017.553

2017-01-01

Computer Vision and Pattern Recognition

Abstract:Zero-shot recognition aims to accurately recognize objects of unseen classes by using a shared visual-semantic mapping between the image feature space and the semantic embedding space. This mapping is learned on training data of seen classes and is expected to have transfer ability to unseen classes. In this paper, we tackle this problem by exploiting the intrinsic relationship between the semantic space manifold and the transfer ability of visual-semantic mapping. We formalize their connection and cast zero-shot recognition as a joint optimization problem. Motivated by this, we propose a novel framework for zero-shot recognition, which contains dual visual-semantic mapping paths. Our analysis shows this framework can not only apply prior semantic knowledge to infer underlying semantic manifold in the image feature space, but also generate optimized semantic embedding space, which can enhance the transfer ability of the visual-semantic mapping to unseen classes. The proposed method is evaluated for zero-shot recognition on four benchmark datasets, achieving outstanding results.
Towards Effective Deep Embedding for Zero-Shot Learning

Lei Zhang,Peng Wang,Lingqiao Liu,Chunhua Shen,Wei Wei,Yanning Zhang,Anton van den Hengel

DOI: https://doi.org/10.1109/tcsvt.2020.2984666

IF: 5.859

2020-09-01

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:Zero-shot learning (ZSL) can be formulated as a cross-domain matching problem: after being projected into a joint embedding space, a visual sample will match against all candidate class-level semantic descriptions and be assigned to the nearest class. In this process, the embedding space underpins the success of such matching and is crucial for ZSL. In this paper, we conduct an in-depth study on the construction of embedding space for ZSL and posit that an ideal embedding space should satisfy two criteria: intra-class compactness and inter-class separability. While the former encourages the embeddings of visual samples of one class to distribute tightly close to the semantic description embedding of this class, the latter requires embeddings from different classes to be well separated from each other. Towards this goal, we present a simple but effective two-branch network to simultaneously map semantic descriptions and visual samples into a joint space, on which visual embeddings are forced to regress to their class-level semantic embeddings and the embeddings crossing classes are required to be distinguishable by a trainable classifier. Furthermore, we extend our method to a transductive setting to better handle the model bias problem in ZSL (i.e., samples from unseen classes tend to be categorized into seen classes) with minimal extra supervision. Specifically, we propose a pseudo labeling strategy to progressively incorporate the testing samples into the training process and thus balance the model between seen and unseen classes. Experimental results on five standard ZSL datasets show the superior performance of the proposed method and its transductive extension.

engineering, electrical & electronic
OntoZSL: Ontology-enhanced Zero-shot Learning

Yuxia Geng,Jiaoyan Chen,Zhuo Chen,Jeff Z. Pan,Zhiquan Ye,Zonggang Yuan,Yantao Jia,Huajun Chen

DOI: https://doi.org/10.1145/3442381.3450042

2021-01-01

Abstract:Zero-shot Learning (ZSL), which aims to predict for those classes that have never appeared in the training data, has arisen hot research interests. The key of implementing ZSL is to leverage the prior knowledge of classes which builds the semantic relationship between classes and enables the transfer of the learned models (e.g., features) from training classes (i.e., seen classes) to unseen classes. However, the priors adopted by the existing methods are relatively limited with incomplete semantics. In this paper, we explore richer and more competitive prior knowledge to model the inter-class relationship for ZSL via ontology-based knowledge representation and semantic embedding. Meanwhile, to address the data imbalance between seen classes and unseen classes, we developed a generative ZSL framework with Generative Adversarial Networks (GANs). Our main findings include: (i) an ontology-enhanced ZSL framework that can be applied to different domains, such as image classification (IMGC) and knowledge graph completion (KGC); (ii) a comprehensive evaluation with multiple zero-shot datasets from different domains, where our method often achieves better performance than the state-of-the-art models. In particular, on four representative ZSL baselines of IMGC, the ontology-based class semantics outperform the previous priors e.g., the word embeddings of classes by an average of 12.4 accuracy points in the standard ZSL across two example datasets (see Figure 4).
Epsilon: Exploring Comprehensive Visual-Semantic Projection for Multi-Label Zero-Shot Learning

Ziming Liu,Jingcai Guo,Song Guo,Xiaocheng Lu

2024-08-25

Abstract:This paper investigates a challenging problem of zero-shot learning in the multi-label scenario (MLZSL), wherein the model is trained to recognize multiple unseen classes within a sample (e.g., an image) based on seen classes and auxiliary knowledge, e.g., semantic information. Existing methods usually resort to analyzing the relationship of various seen classes residing in a sample from the dimension of spatial or semantic characteristics and transferring the learned model to unseen ones. However, they neglect the integrity of local and global features. Although the use of the attention structure will accurately locate local features, especially objects, it will significantly lose its integrity, and the relationship between classes will also be affected. Rough processing of global features will also directly affect comprehensiveness. This neglect will make the model lose its grasp of the main components of the image. Relying only on the local existence of seen classes during the inference stage introduces unavoidable bias. In this paper, we propose a novel and comprehensive visual-semantic framework for MLZSL, dubbed Epsilon, to fully make use of such properties and enable a more accurate and robust visual-semantic projection. In terms of spatial information, we achieve effective refinement by group aggregating image features into several semantic prompts. It can aggregate semantic information rather than class information, preserving the correlation between semantics. In terms of global semantics, we use global forward propagation to collect as much information as possible to ensure that semantics are not omitted. Experiments on large-scale MLZSL benchmark datasets NUS-Wide and Open-Images-v4 demonstrate that the proposed Epsilon outperforms other state-of-the-art methods with large margins.

Computer Vision and Pattern Recognition

Intracellular localization of glutamine synthetase in Chlorogloeopsis fritschii

Transductive Multi-View Embedding For Zero-Shot Recognition And Annotation

Transductive Multi-class and Multi-label Zero-shot Learning

Transductive Multi-label Zero-shot Learning.

Transductive Unbiased Embedding for Zero-Shot Learning

Dual Collaborative Visual-Semantic Mapping for Multi-Label Zero-Shot Image Recognition

Domain Adaptation Meets Zero-Shot Learning: an Annotation-Efficient Approach to Multi-Modality Medical Image Segmentation

Matrix Tri-Factorization With Manifold Regularizations For Zero-Shot Learning

Transferrable Feature and Projection Learning with Class Hierarchy for Zero-Shot Learning.

Transformer-Based Approach Via Contrastive Learning for Zero-Shot Detection.

Transductive discriminative dictionary learning approach for zero-shot classification

Manifold Regularized Cross-Modal Embedding for Zero-Shot Learning

Transductive Zero-Shot Learning with a Self-Training Dictionary Approach

Manifold Embedding for Zero-Shot Recognition

Multi-level Fusion of Multi-modal Semantic Embeddings for Zero Shot Learning

Meta-Transfer Networks for Zero-Shot Learning

Multi-view graph representation with similarity diffusion for general zero-shot learning

Zero-Shot Recognition Using Dual Visual-Semantic Mapping Paths.

Towards Effective Deep Embedding for Zero-Shot Learning

OntoZSL: Ontology-enhanced Zero-shot Learning

Epsilon: Exploring Comprehensive Visual-Semantic Projection for Multi-Label Zero-Shot Learning