Abstract:Recent optical flow estimators usually employ deep models designed for image classification as the encoders for feature extraction and matching. However, those encoders developed for image classification may be sub-optimal for flow estimation. In contrast, the decoder design of optical flow estimators often requires meticulous design for flow estimation. The disconnect between the encoder and decoder could negatively affect optical flow estimation. To address this issue, we propose a neural architecture search method, FlowNAS, to automatically find the more suitable and stronger encoder architecture for existing flow decoders. We first design a suitable search space, including various convolutional operators, and construct a weight-sharing super-network for efficiently evaluating the candidate architectures. To better train the super-network, we present a Feature Alignment Distillation module that utilizes a well-trained flow estimator to guide the training of the super-network. Finally, a resource-constrained evolutionary algorithm is exploited to determine an optimal architecture (i.e., sub-network). Experimental results show that FlowNAS can be easily incorporated into existing flow estimators and achieves state-of-the-art performance with the trade-off between accuracy and efficiency. Furthermore, the encoder architecture discovered by FlowNAS with the weights inherited from the super-network achieves 4.67% F1-all error on KITTI, an 8.4% reduction of RAFT baseline, surpassing state-of-the-art handcrafted GMA and AGFlow models, while reducing the model complexity and latency. The source code and trained models will be released at https://github.com/VDIGPKU/FlowNAS .

An FPGA-Optimized Architecture of Real-time Farneback Optical Flow

A FPGA Implementation of Farneback Optical Flow by High-Level Synthesis

Real-time FPGA Implementation of an Optic Flow Algorithm Suitable for Background Oriented Schlieren Technique

FPGA Implementation of Non-Parametric Stereo Matching Algorithm

Real-Time Efficient FPGA Implementation of the Multi-Scale Lucas-Kanade and Horn-Schunck Optical Flow Algorithms for a 4K Video Stream

High speed video signal acquisition and processing system based on FPGA technique

NeuFlow: Real-time, High-accuracy Optical Flow Estimation on Robots Using Edge Devices

NeuFlow v2: High-Efficiency Optical Flow Estimation on Edge Devices

FPGA implementation of full HD real-time depth estimation

FPGA-Based Feature Extraction and Tracking Accelerator for Real-Time Visual SLAM

A Fast and Accurate Optical Flow Camera for Resource-Constrained Edge Applications

RAPIDFlow: Recurrent Adaptable Pyramids with Iterative Decoding for Efficient Optical Flow Estimation

High-performance architecture for real-time high-definition short-wave infrared streaming video processing and its field programmable gate array prototype

FPGA Implementation of Feature Detection Algorithm Based on High Level Synthesis

Parallel Implementation of Dense Optical Flow Computation on Many-Core Processor.

FastFlowNet: A Lightweight Network for Fast Optical Flow Estimation

Neuromorphic Optical Flow and Real-time Implementation with Event Cameras

FlowNAS: Neural Architecture Search for Optical Flow Estimation

Pflow: An end-to-end heterogeneous acceleration framework for CNN inference on FPGAs

Rethinking RAFT for Efficient Optical Flow

fpgaHART: A toolflow for throughput-oriented acceleration of 3D CNNs for HAR onto FPGAs