Abstract:Natural images often contain multiple shadow regions, and existing video shadow detection methods tend to fail in fully identifying all shadow regions, since they mainly learned temporal features at single-scale and single memory. In this work, we develop a novel convolutional neural network (CNN) to learn motion-guided multi-scale memory features to obtain multi-scale temporal information based on multiple network memories for boosting video shadow detection. To do so, our network first constructs three memories (i.e., a global memory, a local memory, and a motion memory) to combine spatial context and object motion for detecting shadows. Based on these three memories, we then devise a multi-scale motion-guided long-short transformer (MMLT) module to learn multi-scale temporal and motion memory features for predicting a shadow detection map of the input video frame. Our MMLT module includes a dense-scale long transformer (DLT), a dense-scale short transformer (DST), and a dense-scale motion transformer (DMT) to read three memories for learning multi-scale transformer features. Our DLT, DST, and DMT consist of a set of memory-read pooling attention (MPA) blocks and densely connect these output features of multiple MPA blocks to learn multi-scale transformer features since the scales of these output features are varied. By doing so, we can more accurately identify multiple shadow regions with different sizes from the input video. Moreover, we devise a self-supervised pretext task to pre-training the feature encoder for enhancing the downstream video shadow detection. Experimental results on three benchmark datasets show that our video shadow detection network quantitatively and qualitatively outperforms 26 state-of-the-art methods.

Shadow Detection with Multi-Feature Fusion and MMSE Optimization

Moving shadow detection using fusion of multiple features

A Moving Shadow Elimination Method Based on Fusion of Multi-Feature

Moving Shadow Detection by Integrating Multiple Features

Adaptive Shadow Detection Based on GMM and MRF

A fuzzy integral method for shadow detection of moving object

An Effective Moving Cast Shadow Detection Framework for Surveillance Video Analysis

An Algorithm of Motion Detection Based on the Multi-Feature Background Model

Shadow Detection Based on Mixed Feature and HSI Color Space

Fake Shadow Detection Based on SIFT Features Matching

A Multi-Feature Fusion-Based Change Detection Method for Remote Sensing Images

Shadow Extraction Method Based on Multi-Information Fusion and Discrete Wavelet Transform

Shadow Elimination Algorithm of Combination of RGB Color Feature and Texture Feature

Moving Object Detection and Shadow Removing under Changing Illumination Condition

Moving cast shadow detection using joint color and texture features with neighboring information

Learning Motion-guided Multi-scale Memory Features for Video Shadow Detection

Moving Cast Shadow Detection Using Joint Color and Texture Features Based on Direction and Distance

Moving Cast Shadow Elimination Based on Color and Gradient Features

Multi-Feature Fusion Based Object Detecting and Tracking

Shadow Detection for Color Remotely Sensed Images Based on Multi-Feature Integration

Moving Object Shadow Detection Based on Regional Color Clustering