Abstract:Optical flow estimation is a basic task in self-driving and robotics systems, which enables to temporally interpret traffic scenes. Autonomous vehicles clearly benefit from the ultra-wide Field of View (FoV) offered by 360° panoramic sensors. However, due to the unique imaging process of panoramic cameras, models designed for pinhole images do not directly generalize satisfactorily to 360° panoramic images. In this paper, we put forward a novel network framework——PANO FLOW, to learn optical flow for panoramic images. To overcome the distortions introduced by equirectangular projection in panoramic transformation, we design a Flow Distortion Augmentation (FDA) method, which contains radial flow distortion (FDA-R) or equirectangular flow distortion (FDA-E). We further look into the definition and properties of cyclic optical flow for panoramic videos, and hereby propose a Cyclic Flow Estimation (CFE) method by leveraging the cyclicity of spherical images to infer 360° optical flow and converting large displacement to relatively small displacement. PanoFlow is applicable to any existing flow estimation method and benefits from the progress of narrow-FoV flow estimation. In addition, we create and release a synthetic panoramic dataset FlowScape based on CARLA to facilitate training and quantitative analysis. PanoFlow achieves state-of-the-art performance on the public OmniFlowNet and the fresh established FlowScape benchmarks. Our proposed approach reduces the End-Point-Error (EPE) on FlowScape by 27.3%. On OmniFlowNet, PanoFlow achieves an EPE of 3.17 pixels, a 55.5% error reduction from the best published result (7.12 pixels). We also qualitatively validate our method via an outdoor collection vehicle and a public real-world OmniPhotos dataset, indicating strong potential and robustness for real-world navigation applications. Code and d- taset are publicly available at PanoFlow.

Learning to Estimate Optical Flow Using Dual-Frequency Paradigm

Unsupervised Learning of Scene Flow Estimation Fusing with Local Rigidity.

RFRFlow: Recurrent feature refinement network for optical flow estimation

STFlow: Self-Taught Optical Flow Estimation Using Pseudo Labels

FlowNAS: Neural Architecture Search for Optical Flow Estimation

Optical Flow Estimation Using Dual Self-Attention Pyramid Networks

Unsupervised Optical Flow Estimation Based on Improved Feature Pyramid

MRDFlow: Unsupervised Optical Flow Estimation Network With Multi-Scale Recurrent Decoder

Skin the sheep not only once: Reusing Various Depth Datasets to Drive the Learning of Optical Flow

HMAFlow: Learning More Accurate Optical Flow via Hierarchical Motion Field Alignment

Unsupervised Learning Optical Flow in Multi-frame Dynamic Environment Using Temporal Dynamic Modeling

PanoFlow: Learning 360° Optical Flow for Surrounding Temporal Understanding

FlowNet: Learning Optical Flow with Convolutional Networks

SciFlow: Empowering Lightweight Optical Flow Models with Self-Cleaning Iterations

Optical Flow as Spatial-Temporal Attention Learners

SKFlow: Learning Optical Flow with Super Kernels

FastFlowNet: A Lightweight Network for Fast Optical Flow Estimation

Self-Attention-Based Multiscale Feature Learning Optical Flow with Occlusion Feature Map Prediction

NeuFlow: Real-time, High-accuracy Optical Flow Estimation on Robots Using Edge Devices

NeuFlow v2: High-Efficiency Optical Flow Estimation on Edge Devices