Abstract:Aggregating temporal features from other frames is verified to be very effective for video object detection to overcome the challenges in still images, such as occlusion, motion blur, and rare pose. Currently, proposal-level feature aggregation dominates this direction. However, there are two main problems for the holistic proposal-level feature aggregation. First, the object proposals generated by the region proposal network ignore the useful context information around the object which is proved to be helpful for object classification. Second, the traditional proposal-level feature aggregation regards the proposal as a whole without considering the important object structure information, which makes the similarity comparison between two proposals less effective when occlusion or pose misalignment occurs on proposal objects. To deal with these problems, we propose the Context and Structure Mining Network to better aggregate features for video object detection. In our method, we first encode the spatial-temporal context information into object features in a global manner, which can benefit the object classification. In addition, the holistic proposal is divided into several patches to capture the structure information of the object, and cross patch matching is conducted to alleviate the pose misalignment between objects in target and support proposals. Moreover, an importance weight is learned for each target proposal patch to indicate how informative this patch is for the final feature aggregation, by which the occluded patches can be neglected. This enables the aggregation module to leverage the most important and informative patches to obtain the final feature aggregation. The proposed framework outperforms all the latest state-of-the-art methods on the ImageNet VID dataset with a large margin. This project is publicly available https://github.com/LiangHann/Context-and-Structure-Mining-Network-for-Video-Object-Detection .

Category Aggregation Among Region Proposals for Object Detection.

Multilevel Spatial-Temporal Feature Aggregation for Video Object Detection

Object Detection in Video with Spatial-temporal Context Aggregation.

Box Aggregation For Proposal Decimation: Last Mile Of Object Detection

Exploiting Better Feature Aggregation for Video Object Detection

Region Proposal Ranking Via Fusion Feature for Object Detection

Category-Aware Spatial Constraint for Weakly Supervised Detection

Class-aware Feature Aggregation Network for Video Object Detection

An Improved Approach for Object Proposals Generation

Training-free Boost for Open-Vocabulary Object Detection with Confidence Aggregation

S-CNN: Subcategory-Aware Convolutional Networks for Object Detection

Multi-level Proposal Relations Aggregation for Video Object Detection

Implicit Negative Sub-Categorization and Sink Diversion for Object Detection

Adaptive Scale and Spatial Aggregation for Real-Time Object Detection

Semi-Autonomous Learning Algorithm for Remote Image Object Detection Based on Aggregation Area Instance Refinement

Vehicle Object Detection Method Based on Candidate Region Aggregation.

Class-specific Object Proposals Re-Ranking for Object Detection in Automatic Driving

A Simple Approach and Benchmark for 21, 000-Category Object Detection.

Context and Structure Mining Network for Video Object Detection

Proposal Distillation of Multi-Modal Feature Aggregation Network for Video Object Detection

Improving Object Detection with Region Similarity Learning