Abstract:Contrastive learning (CL) methods achieve great success by learning the invariant representation from various transformations. However, rotation transformations are considered harmful to CL and are rarely used, which results in failure when the objects show unseen orientations. This article proposes a representation focus shift network (RefosNet), which adds the rotation transformations to CL methods to improve the robustness of representation. First, the RefosNet constructs the rotation-equivariant mapping between the features of the original image and the rotated ones. Then, the RefosNet learns semantic-invariant representations (SIRs) based on explicitly decoupling the rotation-invariant features and the rotation-equivariant features. Moreover, an adaptive gradient passivation strategy is introduced to gradually shift the representation focus to invariant representations. This strategy can prevent catastrophic forgetting of the rotation equivariance, which is beneficial to the generalization of representations in both seen and unseen orientations. We adapt the baseline methods (i.e.“, SimCLR” and “momentum contrast (MoCo) v2”) to work with RefosNet to verify the performance. Extensive experimental results show that our method achieves significant improvements on the task of recognition. On ObjectNet-13 with unseen orientations, RefosNet gains 7.12% in terms of classification accuracy compared with SimCLR. On datasets in seen orientation, the performance improves by 5.5% on ImageNet-100, 7.29% on STL10, and 1.93% on CIFAR10. In addition, RefosNet has strong generalization on Place205, PASCAL VOC, and Caltech 101. Our method has also achieved satisfactory results in image retrieval tasks.

Oriented Response Networks

Orientation Convolutional Networks for Image Recognition

OACNNs: Orientation adaptive convolutional neural networks

Direction-Sensitivity Features Ensemble Network for Rotation-Invariant Face Detection.

Orientation-Aware Deep Neural Network for Real Image Super-Resolution.

Adaptive Rotated Convolution for Rotated Object Detection

Rotated Feature Network for Multiorientation Object Detection of Remote-Sensing Images

RRL:Regional Rotation Layer in Convolutional Neural Networks

Rotation equivariant vector field networks

Effective Rotate: Learning Rotation-Robust Prototype for Aerial Object Detection

Rotation Invariant Local Binary Convolution Neural Networks.

RiDOP: A Rotation-Invariant Detector with Simple Oriented Proposals in Remote Sensing Images

Orientation robust object detection in aerial images using deep convolutional neural network

Oriented R-CNN and Beyond

CRRCNN: Cascade Rotational RCNN for Dense Arbitrary-Oriented Object Detection

Structurally re-parameterized rotation detector for arbitrary-oriented objects in high-resolution remote sensing images

Oriented R-CNN for Object Detection

Deep Rotation Equivariant Network

RA 2 DC-Net:A residual augment-convolutions and adaptive deformable convolution for points-based anchor-free orientation detection network in remote sensing images

Robust and Rotation-Equivariant Contrastive Learning

Learning RoI Transformer for Oriented Object Detection in Aerial Images