Abstract:Recent optical flow estimators usually employ deep models designed for image classification as the encoders for feature extraction and matching. However, those encoders developed for image classification may be sub-optimal for flow estimation. In contrast, the decoder design of optical flow estimators often requires meticulous design for flow estimation. The disconnect between the encoder and decoder could negatively affect optical flow estimation. To address this issue, we propose a neural architecture search method, FlowNAS, to automatically find the more suitable and stronger encoder architecture for existing flow decoders. We first design a suitable search space, including various convolutional operators, and construct a weight-sharing super-network for efficiently evaluating the candidate architectures. To better train the super-network, we present a Feature Alignment Distillation module that utilizes a well-trained flow estimator to guide the training of the super-network. Finally, a resource-constrained evolutionary algorithm is exploited to determine an optimal architecture (i.e., sub-network). Experimental results show that FlowNAS can be easily incorporated into existing flow estimators and achieves state-of-the-art performance with the trade-off between accuracy and efficiency. Furthermore, the encoder architecture discovered by FlowNAS with the weights inherited from the super-network achieves 4.67% F1-all error on KITTI, an 8.4% reduction of RAFT baseline, surpassing state-of-the-art handcrafted GMA and AGFlow models, while reducing the model complexity and latency. The source code and trained models will be released at https://github.com/VDIGPKU/FlowNAS .

Secrets in Computing Optical Flow by Convolutional Networks

FlowNet: Learning Optical Flow with Convolutional Networks

Optical Flow Estimation Using a Non-Local Convolutional Network.

A Dense Optical Flow Registration Algorithm Based on Deep Learning

Dense Motion Estimation of Particle Images Via a Convolutional Neural Network

Deep Networks for Image Motion Estimation

Optical Flow as Spatial-Temporal Attention Learners

SKFlow: Learning Optical Flow with Super Kernels

Perceptual Loss for Convolutional Neural Network Based Optical Flow Estimation

ReFlowNet: Revisiting Coarse-to-fine Learning of Optical Flow

Traditional and modern strategies for optical flow: an investigation

Deep Networks for Image-to-Image Prediction

Learning By Analogy: Reliable Supervision From Transformations For Unsupervised Optical Flow Estimation

Cubes3D: Neural Network based Optical Flow in Omnidirectional Image Scenes

TransFlow: Transformer as Flow Learner

Unsupervised Learning for Optical Flow Estimation Using Pyramid Convolution LSTM

NccFlow: Unsupervised Learning of Optical Flow With Non-occlusion from Geometry

Optical Flow Field Generation Method Based on Video Compression Information

FastFlowNet: A Lightweight Network for Fast Optical Flow Estimation

FlowNAS: Neural Architecture Search for Optical Flow Estimation