Compositional Oil Spill Detection Based on Object Detector and Adapted Segment Anything Model from SAR Images

Wenhui Wu,Man Sing Wong,Xinyu Yu,Guoqiang Shi,Coco Yin Tung Kwok,Kang Zou
2024-01-15
Abstract:Semantic segmentation-based methods have attracted extensive attention in oil spill detection from SAR images. However, the existing approaches require a large number of finely annotated segmentation samples in the training stage. To alleviate this issue, we propose a composite oil spill detection framework, SAM-OIL, comprising an object detector (e.g., YOLOv8), an adapted Segment Anything Model (SAM), and an Ordered Mask Fusion (OMF) module. SAM-OIL is the first application of the powerful SAM in oil spill detection. Specifically, the SAM-OIL strategy uses YOLOv8 to obtain the categories and bounding boxes of oil spill-related objects, then inputs bounding boxes into the adapted SAM to retrieve category-agnostic masks, and finally adopts the Ordered Mask Fusion (OMF) module to fuse the masks and categories. The adapted SAM, combining a frozen SAM with a learnable Adapter module, can enhance SAM's ability to segment ambiguous objects. The OMF module, a parameter-free method, can effectively resolve pixel category conflicts within SAM. Experimental results demonstrate that SAM-OIL surpasses existing semantic segmentation-based oil spill detection methods, achieving mIoU of 69.52%. The results also indicated that both OMF and Adapter modules can effectively improve the accuracy in SAM-OIL.
Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
The problem that this paper attempts to solve is that when detecting oil spills in Synthetic Aperture Radar (SAR) images, existing methods require a large number of finely - annotated segmentation samples for training, which leads to high time consumption and resource intensity in dataset construction. To alleviate this problem, the paper proposes a composite oil spill detection framework - SAM - OIL, which combines an object detector (such as YOLOv8), an Adapted Segment Anything Model (SAM), and an Ordered Mask Fusion (OMF) module. Specifically, SAM - OIL works through the following steps: 1. Use YOLOv8 to obtain the classes and bounding boxes of objects related to oil spills. 2. Input these bounding boxes into the adapted SAM to retrieve class - independent masks. 3. Finally, use the OMF module to fuse these masks and classes to solve the pixel - class conflict problem. In addition, the adapted SAM combines a frozen SAM and a learnable adapter module, which can enhance the ability of SAM to segment ambiguous objects. The OMF module is a parameter - free method and can effectively solve the pixel - class conflict within SAM. Experimental results show that SAM - OIL outperforms existing semantic - segmentation - based methods in oil spill detection and achieves an mIoU of 69.52%. The results also show that both the OMF and adapter modules can effectively improve the accuracy of SAM - OIL.