Progress in small object detection for remote sensing images
Yuan Xiang,Cheng Gong,Li Ge,Dai Wei,Yin Wenxin,Feng Yingchao,Yao Xiwen,Huang Zhongling,Sun Xian,Han Junwei
DOI: https://doi.org/10.11834/jig.221202
2023-01-01
Journal of Image and Graphics
Abstract:独特的拍摄视角和多变的成像高度使得遥感影像中包含大量尺寸极其有限的目标，如何准确有效地检测这些小目标对于构建智能的遥感图像解译系统至关重要。本文聚焦于遥感场景，对基于深度学习的小目标检测进行全面调研。首先，根据小目标的内在特质梳理了遥感影像小目标检测的3个主要挑战，包括特征表示瓶颈、前背景混淆以及回归分支敏感。其次，通过深入调研相关文献，全面回顾了基于深度学习的遥感影像小目标检测算法。选取3种代表性的遥感影像小目标检测任务，即光学遥感图像小目标检测、SAR图像小目标检测和红外图像小目标检测，系统性总结了3个领域内的代表性方法，并根据每种算法使用的技术思路进行分类阐述。再次，总结了遥感影像小目标检测常用的公开数据集，包括光学遥感图像、SAR图像及红外图像3种数据类型，借助于3种领域的代表性数据集SODA-A（small object detection datasets）、AIR-SARShip和NUAA-SIRST（Nanjing University of Aeronautics and Astronautics，single-frame infrared small target），进一步对主流的遥感影像目标检测算法在面对小目标时的性能表现进行横向对比及深入评估。最后，对遥感影像小目标检测的应用现状进行总结，并展望了遥感场景下小目标检测的发展趋势。;Remote sensing images are often captured from multiview and multiple altitudes, thereby comprising a mass of objects with limited sizes which significantly challenge current detection methods that can achieve outstanding performance on natural images. Moreover, how to precisely detect these small objects plays a crucial role in developing an intelligent interpretation system for remote sensing images. Focusing on the remote sensing images, this paper conducts a comprehensive survey for deep learning-based small object detection(SOD) can be reviewed and analyzed literately, including 1) features-represent bottlenecks, 2) background confusion, and 3) branching-regressed sensitivity. Specially, one of the major bottlenecks is for objective's representation. It refers that the down-sample operations in the prevailing feature extractors can suppress the signals of small objects unavoidably, and the following detection is impaired further in terms of the weak representations. The detection of size-limited instances is also interference of the confusion between the objects and backgrounds and the sensitivity of regression branch. For the former, the representations of small objects tend to be contaminated in related to feature extraction-contextual factors, which may erase the discriminative information that plays a significant role in head network. And the sensitivity of regression branch in small object detection is derived from the low tolerance for bounding box perturbation, in which a slight deviation of a predicted box will cause a drastic drop on the intersection-over-union(IoU), which is generally adopted to evaluate the accuracy of localization. Furthermore, we review and analyze the literature of small object detection for remote sensing images in the deep-learning era. In detail, by systematically reviewing corresponding methods of three small object detection tasks, i. e., SOD for optical remote sensing images, SOD for synthetic aperture radar(SAR) images and SOD for infrared images, an understandable taxonomy of the reviewed algorithms for each task is given. Specifically, we rigorously split the representative methods into several categories according to the major techniques used. In addition to the algorithm survey, considering the deep learning-based methods are hungry for data and to provide a comprehensive survey about small object detection, we also retrospect several publicly available datasets which are commonly used in these three SOD tasks. For each concrete field, we list the prevailing benchmarks in accordance with the published papers, and a brief introduction and some example images about these datasets are illustrated:small size. What is more, other related features about small object detection are highlighted as well, such as image resolution, data source, the number of images and annotated instances, and some proper statistics of each task, etc. Additionally, to better investigate the performance of generic detection methods on small objects, we analyze an in-depth evaluation and comparison of main-stream detection algorithms and several SOD methods for remote sensing images, namely SODA-A (small object detection datasets), AIR-SARShip and NUAA-SIRST (Nanjing University of Aeronautics and Astronautics, single-frame infrared small target). Afterwards, current situation in applications of small object detection for remote sensing images are analyzed, including SOD-based intelligent transportation system and scene-related understanding. Such harbor-targeted recognition is based on SAR image analysis, the precision-guided weapons based on the detection and recognition techniques of infrared images, and the tracking of moving targets at sea on top of multimodal remote sensing data. In the end, to enlighten the further research of small object detection in remote sensing images, we discuss four promising directions in the future. Concretely, it is required that an efficient backbone network can avoid the information loss of small objects while capturing the discriminative features to optimize the down-stream tasks about small objects. Large-scale benchmarks with well annotated small instances play an irreplaceable role linked to small object detection in remote sensing images further. Moreover, a multimodal remote sensing data-collaborated SOD algorithm is also preferred. A proper evaluation metric can not only guide the training and inference of small object detection methods under some specific scenes, but also rich its domain-related development.