Abstract:In this paper we show how Group Equivariant Convolutional Neural Networks use subsampling to learn to break equivariance to their symmetries. We focus on 2D rotations and reflections and investigate the impact of broken equivariance on network performance. We show that a change in the input dimension of a network as small as a single pixel can be enough for commonly used architectures to become approximately equivariant, rather than exactly. We investigate the impact of networks not being exactly equivariant and find that approximately equivariant networks generalise significantly worse to unseen symmetries compared to their exactly equivariant counterparts. However, when the symmetries in the training data are not identical to the symmetries of the network, we find that approximately equivariant networks are able to relax their own equivariant constraints, causing them to match or outperform exactly equivariant networks on common benchmark datasets.
What problem does this paper attempt to address?
The paper primarily explores how group equivariant convolutional neural networks (GCNNs) break the original rotational and translational equivariance during subsampling operations and investigates the impact of this equivariance disruption on network performance. Specifically, the paper attempts to address the following issues:
1. **How subsampling layers affect equivariance**: The paper demonstrates how the MaxPool subsampling layer disrupts the network's equivariance by changing the input dimensions and proves that even a single pixel change is sufficient to transform the network from exact equivariance to approximate equivariance.
2. **Performance differences of approximately equivariant networks**: The study finds that approximately equivariant networks perform significantly worse than exactly equivariant networks when handling unseen symmetric transformations. However, in cases where the symmetry in the training data is inconsistent with the network's symmetry, approximately equivariant networks can relax their equivariance constraints, thereby achieving or exceeding the performance of exactly equivariant networks in common benchmarks.
3. **Measuring the accuracy of equivariance**: The paper proposes a method to measure the network's equivariance and invariance errors and experimentally verifies that when the network is trained to distinguish rotated inputs, its equivariance is broken.
4. **Performance on different datasets**: By conducting experiments on various datasets (such as MNIST, RotMNIST, Flowers102, PatchCamelyon, Cifar10, Cifar100, and ImageNet), the paper showcases the performance differences between exactly equivariant networks and approximately equivariant networks. The results indicate that exactly equivariant networks perform better on datasets with limited rotations; whereas, in cases where the training data already includes rotational transformations, approximately equivariant networks can learn invariance, thus performing comparably or even better on these datasets.
In summary, the paper aims to reveal the impact of subsampling operations on the equivariance of GCNNs and explores how adjusting input dimensions can ensure the network's exact equivariance, thereby enhancing the network's robustness and generalization ability in handling various symmetric transformations.