Abstract:Optical flow estimation is a fundamental task in the field of autonomous driving. Event cameras are capable of responding to log-brightness changes in microseconds. Its characteristic of producing responses only to the changing region is particularly suitable for optical flow estimation. In contrast to the super low-latency response speed of event cameras, existing datasets collected via event cameras, however, only provide limited frame rate optical flow ground truth, (e.g., at 10Hz), greatly restricting the potential of event-driven optical flow. To address this challenge, we put forward a high-frame-rate, low-latency event representation Unified Voxel Grid, sequentially fed into the network bin by bin. We then propose EVA-Flow, an EVent-based Anytime Flow estimation network to produce high-frame-rate event optical flow with only low-frame-rate optical flow ground truth for supervision. The key component of our EVA-Flow is the stacked Spatiotemporal Motion Refinement (SMR) module, which predicts temporally dense optical flow and enhances the accuracy via spatial-temporal motion refinement. The time-dense feature warping utilized in the SMR module provides implicit supervision for the intermediate optical flow. Additionally, we introduce the Rectified Flow Warp Loss (RFWL) for the unsupervised evaluation of intermediate optical flow in the absence of ground truth. This is, to the best of our knowledge, the first work focusing on anytime optical flow estimation via event cameras. A comprehensive variety of experiments on MVSEC, DESC, and our EVA-FlowSet demonstrates that EVA-Flow achieves competitive performance, super-low-latency (5ms), fastest inference (9.2ms), time-dense motion estimation (200Hz), and strong generalization. Our code will be available at <a class="link-external link-https" href="https://github.com/Yaozhuwa/EVA-Flow" rel="external noopener nofollow">this https URL</a>.

Optical Flow Prediction in Auto Driving from Single Image Via Conditional Variational Auto-Encoder

Optical Flow Matters: an Empirical Comparative Study on Fusing Monocular Extracted Modalities for Better Steering

Learning By Analogy: Reliable Supervision From Transformations For Unsupervised Optical Flow Estimation

An Unsupervised Optical Flow Estimation For LiDAR Image Sequences

VectorFlow: Combining Images and Vectors for Traffic Occupancy and Flow Prediction

VectorFlow: Combining Images and Vectors for Traffic Occupancy and Flow Prediction

Unsupervised Learning Optical Flow in Multi-frame Dynamic Environment Using Temporal Dynamic Modeling

Rethink Predicting the Optical Flow with the Kinetics Perspective

Real-time high speed motion prediction using fast aperture-robust event-driven visual flow

End-to-End Interactive Prediction and Planning with Optical Flow Distillation for Autonomous Driving

Towards Anytime Optical Flow Estimation with Event Cameras

Learning to Estimate Optical Flow Using Dual-Frequency Paradigm

Freespace Optical Flow Modeling for Automated Driving

Vehicle Trajectory Prediction Using Intention-based Conditional Variational Autoencoder

CSFlow: Learning Optical Flow via Cross Strip Correlation for Autonomous Driving

Unsupervised video forecasting with flow parsing mechanism of human visual system

2D LiDAR Map Prediction via Estimating Motion Flow with GRU

Optical Flow augmented Semantic Segmentation networks for Automated Driving

Beyond Learning: Back to Geometric Essence of Visual Odometry via Fusion-Based Paradigm

Learning Generalized Visual Odometry Using Position-Aware Optical Flow and Geometric Bundle Adjustment