Abstract:Deep learning well demonstrates its potential in learning latent feature representations. Recent years have witnessed an increasing enthusiasm for regularizing deep neural networks by incorporating various side information, such as user-provided labels or pair-wise constraints. However, the effectiveness and parameter sensitivity of such algorithms have been major obstacles for putting them into practice. The major contribution of our work is the exposition of a novel supervised deep learning algorithm, which distinguishes from two unique traits. First, it regularizes the network construction by utilizing similarity or dissimilarity constraints between data pairs, rather than sample-specific annotations. Such kind of side information is more flexible and greatly mitigates the workload of annotators. Secondly, unlike prior works, our proposed algorithm decouples the supervision information and intrinsic data structure. We design two heterogeneous networks, each of which encodes either supervision or unsupervised data structure respectively. Specifically, we term the supervision-oriented network as "auxiliary network" since it is principally used for facilitating the parameter learning of the other one and will be removed when handling out-of-sample data. The two networks are complementary to each other and bridged by enforcing the correlation of their parameters. We name the proposed algorithm SUpervision-Guided AutoencodeR (SUGAR). Comparing prior works on unsupervised deep networks and supervised learning, SUGAR better balances numerical tractability and the flexible utilization of supervision information. The classification performance on MNIST digits and eight benchmark datasets demonstrates that SUGAR can effectively improve the performance by using the auxiliary networks, on both shallow and deep architectures. Particularly, when multiple SUGARS are stacked, the performance is significantly boosted. On the selected benchmarks, ours achieve up to 11.35% relative accuracy improvement compared to the state-of-the-art models.

Siamese Labels Auxiliary Learning

Dual Enhancement for Multi-Label Learning with Missing Labels

MLAAN: Scaling Supervised Local Learning with Multilaminar Leap Augmented Auxiliary Network

Dynamic Auxiliary Soft Labels for Decoupled Learning

Semi-Supervised Dual Relation Learning for Multi-Label Classification

Semi-Supervised Learning using Siamese Networks

Non-Intrusive Adaptive Load Identification Based on Siamese Network

Learning from Noisy Labels with Decoupled Meta Label Purifier

Deep Metric Learning-Based Semi-Supervised Regression with Alternate Learning

Deep dual incomplete multi-view multi-label classification via label semantic-guided contrastive learning

Siamese Prototypical Contrastive Learning

An Intelligent Multi-View Active Learning Method Based on a Double-Branch Network

Supervised Deep Learning with Auxiliary Networks

A meta-framework for multi-label active learning based on deep reinforcement learning

Rethinking Curriculum Learning with Incremental Labels and Adaptive Compensation

Free Lunches in Auxiliary Learning: Exploiting Auxiliary Labels with Negligibly Extra Inference Cost

M3LA: A Novel Approach Based on Encoder-Decoder with Attention Framework for Multi-modal Multi-label Learning

Deep Modality Assistance Co-Training Network for Semi-Supervised Multi-Label Semantic Decoding

Module-Aware Optimization for Auxiliary Learning

LaSSL: Label-Guided Self-Training for Semi-supervised Learning

Hierarchical Auxiliary Learning