Abstract:Optical flow estimation is a fundamental task in the field of autonomous driving. Event cameras are capable of responding to log-brightness changes in microseconds. Its characteristic of producing responses only to the changing region is particularly suitable for optical flow estimation. In contrast to the super low-latency response speed of event cameras, existing datasets collected via event cameras, however, only provide limited frame rate optical flow ground truth, (e.g., at 10Hz), greatly restricting the potential of event-driven optical flow. To address this challenge, we put forward a high-frame-rate, low-latency event representation Unified Voxel Grid, sequentially fed into the network bin by bin. We then propose EVA-Flow, an EVent-based Anytime Flow estimation network to produce high-frame-rate event optical flow with only low-frame-rate optical flow ground truth for supervision. The key component of our EVA-Flow is the stacked Spatiotemporal Motion Refinement (SMR) module, which predicts temporally dense optical flow and enhances the accuracy via spatial-temporal motion refinement. The time-dense feature warping utilized in the SMR module provides implicit supervision for the intermediate optical flow. Additionally, we introduce the Rectified Flow Warp Loss (RFWL) for the unsupervised evaluation of intermediate optical flow in the absence of ground truth. This is, to the best of our knowledge, the first work focusing on anytime optical flow estimation via event cameras. A comprehensive variety of experiments on MVSEC, DESC, and our EVA-FlowSet demonstrates that EVA-Flow achieves competitive performance, super-low-latency (5ms), fastest inference (9.2ms), time-dense motion estimation (200Hz), and strong generalization. Our code will be available at <a class="link-external link-https" href="https://github.com/Yaozhuwa/EVA-Flow" rel="external noopener nofollow">this https URL</a>.

Efficient Meshflow and Optical Flow Estimation from Event Cameras

Towards Anytime Optical Flow Estimation with Event Cameras

Unsupervised Learning of Scene Flow Estimation Fusing with Local Rigidity.

A Dense Optical Flow Registration Algorithm Based on Deep Learning

BlinkFlow: A Dataset to Push the Limits of Event-based Optical Flow Estimation

EV-FlowNet: Self-Supervised Optical Flow Estimation for Event-based Cameras

Learning Optical Flow from Event Camera with Rendered Dataset

ResFlow: Fine-tuning Residual Optical Flow for Event-based High Temporal Resolution Motion Estimation

Dense Continuous-Time Optical Flow from Event Cameras

EV-MGRFlowNet: Motion-Guided Recurrent Network for Unsupervised Event-Based Optical Flow With Hybrid Motion-Compensation Loss

EMatch: A Unified Framework for Event-based Optical Flow and Stereo Matching

Spatio-Temporal Recurrent Networks for Event-Based Optical Flow Estimation

Event-based Optical Flow Via Transforming into Motion-dependent View

RPEFlow: Multimodal Fusion of RGB-PointCloud-Event for Joint Optical Flow and Scene Flow Estimation

MemFlow: Optical Flow Estimation and Prediction with Memory

Dense Continuous-Time Optical Flow from Events and Frames

Lightweight Event-based Optical Flow Estimation via Iterative Deblurring

Fusion-FlowNet: Energy-Efficient Optical Flow Estimation using Sensor Fusion and Deep Fused Spiking-Analog Network Architectures

Motion-prior Contrast Maximization for Dense Continuous-Time Motion Estimation

Event-based Temporally Dense Optical Flow Estimation with Sequential Learning

TMA: Temporal Motion Aggregation for Event-based Optical Flow