Abstract:For visual estimation of optical flow, a crucial function for many vision tasks, unsupervised learning, using the supervision of view synthesis has emerged as a promising alternative to supervised methods, since ground-truth flow is not readily available in many cases. However, unsupervised learning is likely to be unstable when pixel tracking is lost due to occlusion and motion blur, or the pixel matching is impaired due to variation in image content and spatial structure over time. In natural environments, dynamic occlusion or object variation is a relatively slow temporal process spanning several frames. We, therefore, explore the optical flow estimation from multiple-frame sequences of dynamic scenes, whereas most of the existing unsupervised approaches are based on temporal static models. We handle the unsupervised optical flow estimation with a temporal dynamic model by introducing a spatial-temporal dual recurrent block based on the predictive coding structure, which feeds the previous high-level motion prior to the current optical flow estimator. Assuming temporal smoothness of optical flow, we use motion priors of the adjacent frames to provide more reliable supervision of the occluded regions. To grasp the essence of challenging scenes, we simulate various scenarios across long sequences, including dynamic occlusion, content variation, and spatial variation, and adopt self-supervised distillation to make the model understand the object's motion patterns in a prolonged dynamic environment. Experiments on KITTI 2012, KITTI 2015, Sintel Clean, and Sintel Final datasets demonstrate the effectiveness of our methods on unsupervised optical flow estimation. The proposal achieves state-of-the-art performance with advantages in memory overhead.

Unsupervised Dynamic Texture Segmentation Using Appearance and Motion

Automatic Dynamic Texture Segmentation Using Local Descriptors and Optical Flow

Unsupervised Dynamic Texture Segmentation Using Local Spatiotemporal Descriptors

An Improved Local Descriptor and Threshold Learning for Unsupervised Dynamic Texture Segmentation

Motionobject segmentation algorithm based on spatial-temporal information

Detection and Segmentation of Moving Objects Using Temporal and Spatial Cues

Dynamic Texture Recognition by Spatio-Temporal Multiresolution Histograms

Spatio-Temporal Segmentation with Depth-Inferred Videos of Static Scenes

Spatio-Temporal Video Segmentation of Static Scenes and Its Applications

Fast unsupervised texture segmentation using Texel similarity map

Unsupervised Learning Optical Flow in Multi-frame Dynamic Environment Using Temporal Dynamic Modeling

Dynamic texture synthesis using a spatial temporal descriptor

Texture Segmentation Using Kernel-based Techniques

Unsupervised Texture Segmentation Via Applying Geodesic Active Regions To Gaborian Feature Space

Unsupervised Texture Segmentation Based on Geodesic Active Regions

3D-TexSeg: Unsupervised Segmentation of 3D Texture using Mutual Transformer Learning

Contour Based Automatic Scene Segmentation in Image Sequences

Dynamic texture recognition with video set based collaborative representation

Human Segmentation with Dynamic LiDAR Data

A Novel Method of Dynamic Textures Analysis and Synthesis.

A Pixel-Level Segmentation-Synthesis Framework for Dynamic Texture Video Compression