Abstract:Aggregating temporal features from other frames is verified to be very effective for video object detection to overcome the challenges in still images, such as occlusion, motion blur, and rare pose. Currently, proposal-level feature aggregation dominates this direction. However, there are two main problems for the holistic proposal-level feature aggregation. First, the object proposals generated by the region proposal network ignore the useful context information around the object which is proved to be helpful for object classification. Second, the traditional proposal-level feature aggregation regards the proposal as a whole without considering the important object structure information, which makes the similarity comparison between two proposals less effective when occlusion or pose misalignment occurs on proposal objects. To deal with these problems, we propose the Context and Structure Mining Network to better aggregate features for video object detection. In our method, we first encode the spatial-temporal context information into object features in a global manner, which can benefit the object classification. In addition, the holistic proposal is divided into several patches to capture the structure information of the object, and cross patch matching is conducted to alleviate the pose misalignment between objects in target and support proposals. Moreover, an importance weight is learned for each target proposal patch to indicate how informative this patch is for the final feature aggregation, by which the occluded patches can be neglected. This enables the aggregation module to leverage the most important and informative patches to obtain the final feature aggregation. The proposed framework outperforms all the latest state-of-the-art methods on the ImageNet VID dataset with a large margin. This project is publicly available https://github.com/LiangHann/Context-and-Structure-Mining-Network-for-Video-Object-Detection .

Spatial-Temporal Feature Aggregation Network for Video Object Detection

Multilevel Spatial-Temporal Feature Aggregation for Video Object Detection

Object Detection in Video with Spatial-temporal Context Aggregation.

A Channel-Wise Spatial-Temporal Aggregation Network for Action Recognition

Class-aware Feature Aggregation Network for Video Object Detection

Adaptive Feature Aggregation for Video Object Detection

Temporal-adaptive sparse feature aggregation for video object detection

Non-dense Feature Aggregation for Video Object Detection

Exploiting Better Feature Aggregation for Video Object Detection

Guided Sampling Based Feature Aggregation for Video Object Detection.

Video Object Detection by Aggregating Features Across Adjacent Frames

Context and Structure Mining Network for Video Object Detection

Relation-Guided Multi-stage Feature Aggregation Network for Video Object Detection.

Global-local Feature Aggregation for Event-based Object Detection on EventKITTI

Adaptive Scale and Spatial Aggregation for Real-Time Object Detection

SSGA-Net: Stepwise Spatial Global-local Aggregation Networks for for Autonomous Driving

Dualfeat: Dual Feature Aggregation for Video Object Detection.

Proposal Distillation of Multi-Modal Feature Aggregation Network for Video Object Detection

DFA: Dynamic Feature Aggregation for Efficient Video Object Detection

Spatio-Temporal Motion Aggregation Network for Video Action Detection

Real-Time and Accurate Object Detection in Compressed Video by Long Short-term Feature Aggregation