Abstract:Abstract The ability to handle large scale variations is crucial for many real-world visual tasks. A straightforward approach for handling scale in a deep network is to process an image at several scales simultaneously in a set of scale channels . Scale invariance can then, in principle, be achieved by using weight sharing between the scale channels together with max or average pooling over the outputs from the scale channels. The ability of such scale-channel networks to generalise to scales not present in the training set over significant scale ranges has, however, not previously been explored. In this paper, we present a systematic study of this methodology by implementing different types of scale-channel networks and evaluating their ability to generalise to previously unseen scales. We develop a formalism for analysing the covariance and invariance properties of scale-channel networks, including exploring their relations to scale-space theory, and exploring how different design choices, unique to scaling transformations, affect the overall performance of scale-channel networks. We first show that two previously proposed scale-channel network designs, in one case, generalise no better than a standard CNN to scales not present in the training set, and in the second case, have limited scale generalisation ability . We explain theoretically and demonstrate experimentally why generalisation fails or is limited in these cases. We then propose a new type of foveated scale-channel architecture , where the scale channels process increasingly larger parts of the image with decreasing resolution. This new type of scale-channel network is shown to generalise extremely well, provided sufficient image resolution and the absence of boundary effects. Our proposed FovMax and FovAvg networks perform almost identically over a scale range of 8, also when training on single-scale training data , and do also give improved performance when learning from data sets with large scale variations in the small sample regime.

Where-What Network 5: Dealing with Scales for Objects in Complex Backgrounds

Skull-closed Autonomous Development : WWN-7 Dealing with Scales

Synapse Maintenance in the Where-What Networks

Where-What Network With Cuda: General Object Recognition And Location In Complex Backgrounds

Skull-closed Autonomous Development: WWN-6 Using Natural Video.

ScaleFormer: Revisiting the Transformer-based Backbones from a Scale-wise Perspective for Medical Image Segmentation.

Deep Dual-Stream Network with Scale Context Selection Attention Module for Semantic Segmentation

Types, Locations, and Scales from Cluttered Natural Video and Actions

Skull-closed Autonomous Development: Object-Wise Incremental Learning

WWN-9: Cross-domain synaptic maintenance and its application to object groups recognition

Cross-Domain and Within-Domain Synaptic Maintenance for Autonomous Development of Visual Areas

WW-Nets: Dual Neural Networks for Object Detection

Hierarchical Gate Network for Fine-Grained Visual Recognition.

ScaleNet: Searching for the Model to Scale.

Skull-closed autonomous development

Hierarchical Multi-Scale Network for Cross-Scale Visual Defect Detection

Scale-Invariant Scale-Channel Networks: Deep Networks That Generalise to Previously Unseen Scales

SSN: Scale Selection Network for Multi-Scale Object Detection in Remote Sensing Images

ScaleNet: Scale Invariance Learning in Directed Graphs

GCWNet: A Global Context-Weaving Network for Object Detection in Remote Sensing Images

Exploring the ability of CNNs to generalise to previously unseen scales over wide scale ranges