Abstract:Optical flow has made great progress in clean scenes, while suffers degradation under adverse weather due to the violation of the brightness constancy and gradient continuity assumptions of optical flow. Typically, existing methods mainly adopt domain adaptation to transfer motion knowledge from clean to degraded domain through one-stage adaptation. However, this direct adaptation is ineffective, since there exists a large gap due to adverse weather and scene style between clean and real degraded domains. Moreover, even within the degraded domain itself, static weather (e.g., fog) and dynamic weather (e.g., rain) have different impacts on optical flow. To address above issues, we explore synthetic degraded domain as an intermediate bridge between clean and real degraded domains, and propose a cumulative homogeneous-heterogeneous adaptation framework for real adverse weather optical flow. Specifically, for clean-degraded transfer, our key insight is that static weather possesses the depth-association homogeneous feature which does not change the intrinsic motion of the scene, while dynamic weather additionally introduces the heterogeneous feature which results in a significant boundary discrepancy in warp errors between clean and degraded domains. For synthetic-real transfer, we figure out that cost volume correlation shares a similar statistical histogram between synthetic and real degraded domains, benefiting to holistically aligning the homogeneous correlation distribution for synthetic-real knowledge distillation. Under this unified framework, the proposed method can progressively and explicitly transfer knowledge from clean scenes to real adverse weather. In addition, we further collect a real adverse weather dataset with manually annotated optical flow labels and perform extensive experiments to verify the superiority of the proposed method.

Self-adaptive optical flow estimation in multiharsh outdoor scenes

Unsupervised Learning of Scene Flow Estimation Fusing with Local Rigidity.

An Unsupervised Optical Flow Estimation For LiDAR Image Sequences

Self-Attention-Based Multiscale Feature Learning Optical Flow with Occlusion Feature Map Prediction

Unsupervised Hierarchical Domain Adaptation for Adverse Weather Optical Flow

Adverse Weather Optical Flow: Cumulative Homogeneous-Heterogeneous Adaptation

Unsupervised Learning Optical Flow in Multi-frame Dynamic Environment Using Temporal Dynamic Modeling

FocusFlow: Boosting Key-Points Optical Flow Estimation for Autonomous Driving

FlowDepth: Decoupling Optical Flow for Self-Supervised Monocular Depth Estimation

CSFlow: Learning Optical Flow via Cross Strip Correlation for Autonomous Driving

HMAFlow: Learning More Accurate Optical Flow via Hierarchical Motion Field Alignment

STFlow: Self-Taught Optical Flow Estimation Using Pseudo Labels

Learning to Estimate Optical Flow Using Dual-Frequency Paradigm

SciFlow: Empowering Lightweight Optical Flow Models with Self-Cleaning Iterations

An Efficient Optical Flow Based Motion Detection Method for Non-stationary Scenes

StreamFlow: Streamlined Multi-Frame Optical Flow Estimation for Video Sequences

Optical Flow as Spatial-Temporal Attention Learners

RFRFlow: Recurrent feature refinement network for optical flow estimation

Optical Flow Estimation Via Motion Feature Recovery

Freespace Optical Flow Modeling for Automated Driving

Optical Flow Computation Based On Bayesian Selecting Scheme