Abstract:The training data distribution is often biased towards objects in certain orientations and illumination conditions. While humans have a remarkable capability of recognizing objects in out-of-distribution (OoD) orientations and illuminations, Deep Neural Networks (DNNs) severely suffer in this case, even when large amounts of training examples are available. Neurons that are invariant to orientations and illuminations have been proposed as a neural mechanism that could facilitate OoD generalization, but it is unclear how to encourage the emergence of such invariant neurons. In this paper, we investigate three different approaches that lead to the emergence of invariant neurons and substantially improve DNNs in recognizing objects in OoD orientations and illuminations. Namely, these approaches are (i) training much longer after convergence of the in-distribution (InD) validation accuracy, i.e., late-stopping, (ii) tuning the momentum parameter of the batch normalization layers, and (iii) enforcing invariance of the neural activity in an intermediate layer to orientation and illumination conditions. Each of these approaches substantially improves the DNN's OoD accuracy (more than 20% in some cases). We report results in four datasets: two datasets are modified from the MNIST and iLab datasets, and the other two are novel (one of 3D rendered cars and another of objects taken from various controlled orientations and illumination conditions). These datasets allow to study the effects of different amounts of bias and are challenging as DNNs perform poorly in OoD conditions. Finally, we demonstrate that even though the three approaches focus on different aspects of DNNs, they all tend to lead to the same underlying neural mechanism to enable OoD accuracy gains - individual neurons in the intermediate layers become invariant to OoD orientations and illuminations. We anticipate this study to be a basis for further improvement of deep neural networks' OoD generalization performance, which is highly demanded to achieve safe and fair AI applications.

The mr-MDA: An Invariant to Shifting, Scaling, and Rotating Variance for 3D Object Recognition using Diffractive Deep Neural Network

3D-SSD: Learning Hierarchical Features from RGB-D Images for Amodal 3D Object Detection

MRF3Net: Infrared Small Target Detection Using Multi-Receptive Field Perception and Effective Feature Fusion

MT-SSD: Single-Stage 3D Object Detector Based on Magnification Transformation

Scale-, shift- and rotation-invariant diffractive optical networks

Misalignment resilient diffractive optical networks

Three approaches to facilitate invariant neurons and generalization to out-of-distribution orientations and illuminations

Robust 3D Face Recognition by Local Shape Difference Boosting

Class-specific Differential Detection in Diffractive Optical Neural Networks Improves Inference Accuracy

Multi-Scale Direction-Aware Network for Infrared Small Target Detection

Efficient Design Optimization for Diffractive Deep Neural Networks

DEVIANT: Depth EquiVarIAnt NeTwork for Monocular 3D Object Detection

Advances and Progress of Diffractive Deep Neural Networks

All-Optical Machine Learning Using Diffractive Deep Neural Networks

Designing robust diffractive neural networks with improved transverse shift tolerance

3D Object Recognition By Corresponding and Quantizing Neural 3D Scene Representations

Recognizing multi-view objects with occlusions using a deep architecture

Time-lapse image classification using a diffractive neural network

Dual Attention MobDenseNet(DAMDNet) for Robust 3D Face Alignment

DAN: Deep-Attention Network for 3D Shape Recognition

M&M3D: Multi-Dataset Training and Efficient Network for Multi-view 3D Object Detection