Abstract:The resurgence of unsupervised learning can be attributed to the remarkable progress of self-supervised learning, which includes generative $(\mathcal{G})$ and discriminative $(\mathcal{D})$ models. In computer vision, the mainstream self-supervised learning algorithms are $\mathcal{D}$ models. However, designing a $\mathcal{D}$ model could be over-complicated; also, some studies hinted that a $\mathcal{D}$ model might not be as general and interpretable as a $\mathcal{G}$ model. In this paper, we switch from $\mathcal{D}$ models to $\mathcal{G}$ models using the classical auto-encoder $(AE)$ . Note that a vanilla $\mathcal{G}$ model was far less efficient than a $\mathcal{D}$ model in self-supervised computer vision tasks, as it wastes model capability on overfitting semantic-agnostic high-frequency details. Inspired by perceptual learning that could use cross-view learning to perceive concepts and semantics 1 1 Following [26], we refer to semantics as visual concepts, e.g., a semantic-ware model indicates the model can perceive visual concepts, and the learned features are efficient in object recognition, detection, etc., we propose a novel $AE$ that could learn semantic-aware representation via cross-view image reconstruction. We use one view of an image as the input and another view of the same image as the reconstruction target. This kind of $AE$ has rarely been studied before, and the optimization is very difficult. To enhance learning ability and find a feasible solution, we propose a semantic aligner that uses geometric transformation knowledge to align the hidden code of $AE$ to help optimization. These techniques significantly improve the representation learning ability of $AE$ and make selfsupervised learning with $\mathcal{G}$ models possible. Extensive experiments on many large-scale benchmarks (e.g., ImageNet, COCO 2017, and SYSU-30k) demonstrate the effectiveness of our methods. Code is available at https://github.com/wanggrun/Semantic-Aware-AE.

Representation Learning With Smooth Autoencoder

Representation Learning Via a Semi-Supervised Stacked Distance Autoencoder for Image Classification

Functional Autoencoder for Smoothing and Representation Learning

Discriminative Representation Learning with Supervised Auto-encoder

Diversified feature representation via deep auto-encoder ensemble through multiple activation functions

Supervised Representation Learning: Transfer Learning with Deep Autoencoders

Semi-supervised Auto-Encoder Based on Manifold Learning

Representation learning via an integrated autoencoder for unsupervised domain adaptation

Semantic-Aware Auto-Encoders for Self-supervised Representation Learning

Reconstruction of Hidden Representation for Robust Feature Extraction

Representation learning via serial robust autoencoder for domain adaptation

Smooth Representation Learning of Infrared and Visible Images

Single Sample Face Recognition Via Learning Deep Supervised Autoencoders

Autoencoder Network for Hyperspectral Unmixing with Adaptive Abundance Smoothing

Supervised Representation Learning with Double Encoding-Layer Autoencoder for Transfer Learning.

Learning Smooth Representations with Generalized Softmax for Unsupervised Domain Adaptation

Target-Embedding Autoencoders for Supervised Representation Learning

Supervised Extreme Learning Machine-Based Auto-Encoder for Discriminative Feature Learning

Semi-Supervised Autoencoder : A Joint Approach of Representation and Classification

ABAE: Auxiliary Balanced AutoEncoder for class-imbalanced semi-supervised learning