Abstract:Unsupervised representation learning aims at finding methods that learn representations from data without annotation-based signals. Abstaining from annotations not only leads to economic benefits but may - and to some extent already does - result in advantages regarding the representation's structure, robustness, and generalizability to different tasks. In the long run, unsupervised methods are expected to surpass their supervised counterparts due to the reduction of human intervention and the inherently more general setup that does not bias the optimization towards an objective originating from specific annotation-based signals. While major advantages of unsupervised representation learning have been recently observed in natural language processing, supervised methods still dominate in vision domains for most tasks. In this dissertation, we contribute to the field of unsupervised (visual) representation learning from three perspectives: (i) Learning representations: We design unsupervised, backpropagation-free Convolutional Self-Organizing Neural Networks (CSNNs) that utilize self-organization- and Hebbian-based learning rules to learn convolutional kernels and masks to achieve deeper backpropagation-free models. (ii) Evaluating representations: We build upon the widely used (non-)linear evaluation protocol to define pretext- and target-objective-independent metrics for measuring and investigating the objective function mismatch between various unsupervised pretext tasks and target tasks. (iii) Transferring representations: We contribute CARLANE, the first 3-way sim-to-real domain adaptation benchmark for 2D lane detection, and a method based on prototypical self-supervised learning. Finally, we contribute a content-consistent unpaired image-to-image translation method that utilizes masks, global and local discriminators, and similarity sampling to mitigate content inconsistencies.

Explaining Representation Learning with Perceptual Components

Graphical Perception of Saliency-based Model Explanations

Identifying Interpretable Subspaces in Image Representations

Contextual Explainable Video Representation: Human Perception-based Understanding

Learning attributions grounded in existing facts for robust visual explanation

Explaining Explainability: Understanding Concept Activation Vectors

Explaining with Counter Visual Attributes and Examples

Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspaces

CEIR: Concept-based Explainable Image Representation Learning

Visually Explainable Recommendation

Towards Unsupervised Visual Reasoning: Do Off-The-Shelf Features Know How to Reason?

Explaining Deep Convolutional Neural Networks via Latent Visual-Semantic Filter Attention

Explainability of Deep Learning models for Urban Space perception

Improving Explainability of Disentangled Representations using Multipath-Attribution Mappings

Visualizing Global Explanations of Point Cloud DNNs

Understanding the Dependence of Perception Model Competency on Regions in an Image

Interpretable Network Visualizations: A Human-in-the-Loop Approach for Post-hoc Explainability of CNN-based Image Classification

Towards Unsupervised Representation Learning: Learning, Evaluating and Transferring Visual Representations

Where and What? Examining Interpretable Disentangled Representations

Leveraging saliency priors and explanations for enhanced consistent interpretability

Learning by Reconstruction Produces Uninformative Features For Perception