Abstract:Multispectral person detection aims at automatically localizing humans in images that consist of multiple spectral bands. Usually, the visual-optical (VIS) and the thermal infrared (IR) spectra are combined to achieve higher robustness for person detection especially in insufficiently illuminated scenes. This paper focuses on analyzing existing detection approaches for their generalization ability. Generalization is a key feature for machine learning based detection algorithms that are supposed to perform well across different datasets. Inspired by recent literature regarding person detection in the VIS spectrum, we perform a cross-validation study to empirically determine the most promising dataset to train a well-generalizing detector. Therefore, we pick one reference Deep Convolutional Neural Network (DCNN) architecture and three different multispectral datasets. The Region Proposal Network (RPN) originally introduced for object detection within the popular Faster R-CNN is chosen as a reference DCNN. The reason is that a stand-alone RPN is able to serve as a competitive detector for two-class problems such as person detection. Furthermore, current state-of-the-art approaches initially apply an RPN followed by individual classifiers. The three considered datasets are the KAIST Multispectral Pedestrian Benchmark including recently published improved annotations for training and testing, the Tokyo Multi-spectral Semantic Segmentation dataset, and the OSU Color-Thermal dataset including recently released annotations. The experimental results show that the KAIST Multispectral Pedestrian Benchmark with its improved annotations provides the best basis to train a DCNN with good generalization ability compared to the other two multispectral datasets. On average, this detection model achieves a log-average Miss Rate (MR) of 29.74 % evaluated on the reasonable test subsets of the three datasets.

PU Learning for RPN Generalization

RINet: Efficient 3D Lidar-Based Place Recognition Using Rotation Invariant Neural Network

DeepRING: Learning Roto-translation Invariant Representation for LiDAR Based Place Recognition.

SC-RPN: A Strong Correlation Learning Framework for Region Proposal.

C-RPNs: Promoting object detection in real world via a cascade structure of Region Proposal Networks.

Split-PU: Hardness-aware Training Strategy for Positive-Unlabeled Learning

Revisiting Faster R-Cnn: A Deeper Look At Region Proposal Network

Improved region proposal network for enhanced few-shot object detection

Label-efficient object detection via region proposal network pre-training

Enhancing representation learning by exploiting effective receptive fields for object detection

Crpn: distinguish novel categories via class-relevant region proposal network for few-shot object detection

Deep Regionlets: Blended Representation and Deep Learning for Generic Object Detection

Learning Long-term Visual Dynamics with Region Proposal Interaction Networks

Generalization ability of region proposal networks for multispectral person detection

Hierarchical Region Proposal Refinement Network for Weakly Supervised Object Detection

Learning Region Features for Object Detection.

Object Detection in Remote Sensing Imagery Based on Prototype Learning Network With Proposal Relation

Robust Tracking Using Region Proposal Networks.

Delving deep into the imbalance of positive proposals in two-stage object detection

Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks