LEFormer: A Hybrid CNN-Transformer Architecture for Accurate Lake Extraction from Remote Sensing Imagery

Ben Chen,Xuechao Zou,Yu Zhang,Jiayu Li,Kai Li,Junliang Xing,Pin Tao

2024-01-08

Abstract:Lake extraction from remote sensing images is challenging due to the complex lake shapes and inherent data noises. Existing methods suffer from blurred segmentation boundaries and poor foreground modeling. This paper proposes a hybrid CNN-Transformer architecture, called LEFormer, for accurate lake extraction. LEFormer contains three main modules: CNN encoder, Transformer encoder, and cross-encoder fusion. The CNN encoder effectively recovers local spatial information and improves fine-scale details. Simultaneously, the Transformer encoder captures long-range dependencies between sequences of any length, allowing them to obtain global features and context information. The cross-encoder fusion module integrates the local and global features to improve mask prediction. Experimental results show that LEFormer consistently achieves state-of-the-art performance and efficiency on the Surface Water and the Qinghai-Tibet Plateau Lake datasets. Specifically, LEFormer achieves 90.86% and 97.42% mIoU on two datasets with a parameter count of 3.61M, respectively, while being 20 minor than the previous best lake extraction method. The source code is available at <a class="link-external link-https" href="https://github.com/BastianChen/LEFormer" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper aims to address the problem of accurately extracting lakes from remote sensing images. Specifically, existing methods have limitations in segmenting blurred boundaries and poor foreground modeling, making lake extraction challenging. To solve these issues, the authors propose a hybrid CNN-Transformer architecture called LEFormer. This architecture combines Convolutional Neural Networks (CNN) and Transformers to capture both local and global features and improve mask prediction accuracy through a cross-encoding fusion module.

The main objectives include:
1. **Improving segmentation accuracy**: By integrating the advantages of CNNs and Transformers, LEFormer can effectively recover local spatial information and capture long-range dependencies, thereby enhancing segmentation accuracy.
2. **Reducing computational cost**: Compared to existing methods, LEFormer significantly reduces the number of parameters and floating-point operations (FLOPs) while maintaining or even improving performance.
3. **Enhancing model efficiency**: By designing a lightweight Transformer encoder, the computational demand is reduced while maintaining high accuracy.

Experimental results show that LEFormer outperforms the current state-of-the-art methods on two benchmark datasets (Surface Water and Qinghai-Tibet Plateau Lake), particularly achieving significant optimization in terms of the number of parameters and FLOPs.