A feature fusion method on landslide identification in remote sensing with Segment Anything Model

Chuan Yang,Yueqin Zhu,Jiantong Zhang,Xiaoqiang Wei,Haomeng Zhu,Zhehui Zhu
DOI: https://doi.org/10.1007/s10346-024-02390-x
IF: 6.153
2024-11-01
Landslides
Abstract:Utilizing remote sensing and deep learning methods can greatly enhance efficiency and accuracy in landslide identification. However, the presence of vegetation cover and human activities pose challenges to identify landslides using optical images alone. In addition, the limited availability of landslide samples affects the generalization and adaptability of identification models. This study introduces a novel feature fusion model called SAMLS, which is adapted from the vision foundation model Segment Anything Model (SAM). The SAMLS is designed with two-branch encoders, namely the RGB branch and the digital elevation model (DEM) branch, which retains the strong feature extraction abilities from the original SAM encoder but utilizes adapter approach to finetune on landslide datasets. The SAMLS utilizes a cross-branch attention mechanism to dynamically fuse terrain features from DEM branch with optical features from RGB branch based on different types of landslides. The SAMLS also follows the prompt learning approach for an automatic landslide identification process. Experiment results on the Bijie dataset demonstrate the effectiveness of SAMLS in identifying visually distinct landslides with an improvement of 0.820 and 0.831 to 0.843 in terms of F1-score compared with Mask R-CNN and Mask2Former. Furthermore, experiment results on the Zhejiang dataset indicate that SAMLS is capable to identify both visually distinct and occluded landslides. The combination of RGB and color relief image map (CRIM) data achieves the best F1-score of 0.592, primarily due to its enhanced visibility of terrain characteristics of landslides in CRIM images. The proposed method holds promise for applications in landslide mapping and risk management in the future.
geosciences, multidisciplinary,engineering, geological
What problem does this paper attempt to address?