AI Foundation Models in Remote Sensing: A Survey

Siqi Lu,Junlin Guo,James R Zimmer-Dauphinee,Jordan M Nieusma,Xiao Wang,Parker VanValkenburgh,Steven A Wernke,Yuankai Huo
2024-08-07
Abstract:Artificial Intelligence (AI) technologies have profoundly transformed the field of remote sensing, revolutionizing data collection, processing, and analysis. Traditionally reliant on manual interpretation and task-specific models, remote sensing has been significantly enhanced by the advent of foundation models--large-scale, pre-trained AI models capable of performing a wide array of tasks with unprecedented accuracy and efficiency. This paper provides a comprehensive survey of foundation models in the remote sensing domain, covering models released between June 2021 and June 2024. We categorize these models based on their applications in computer vision and domain-specific tasks, offering insights into their architectures, pre-training datasets, and methodologies. Through detailed performance comparisons, we highlight emerging trends and the significant advancements achieved by these foundation models. Additionally, we discuss the technical challenges, practical implications, and future research directions, addressing the need for high-quality data, computational resources, and improved model generalization. Our research also finds that pre-training methods, particularly self-supervised learning techniques like contrastive learning and masked autoencoders, significantly enhance the performance and robustness of foundation models in remote sensing tasks such as scene classification, object detection, and other applications. This survey aims to serve as a resource for researchers and practitioners by providing a panorama of advances and promising pathways for continued development and application of foundation models in remote sensing.
Computer Vision and Pattern Recognition,Machine Learning
What problem does this paper attempt to address?
### What problem does this paper attempt to solve? This paper aims to comprehensively investigate and evaluate large pre-trained foundation models used in the field of remote sensing. Specifically, the paper focuses on the following points: 1. **Background and Methods**: - The paper first introduces the background and development of foundation models, explaining how these models significantly enhance the performance and efficiency of remote sensing tasks through large-scale pre-training data and complex architectures. - It discusses in detail the application of foundation models in computer vision tasks and specific domain tasks, including scene classification, semantic segmentation, object detection, change detection, etc. 2. **Model Classification and Analysis**: - The foundation models are classified according to their application domains, covering 51 vision models released from June 2021 to June 2024. - A detailed analysis and comparison of each model's architecture, pre-training datasets, pre-training methods, and performance are provided. 3. **Technical Challenges and Future Directions**: - The paper discusses the challenges faced by foundation models in the field of remote sensing, such as the need for high-quality data, limitations of computational resources, and the improvement of model generalization capabilities. - Future research directions are proposed, emphasizing the importance of self-supervised learning techniques (such as contrastive learning and masked autoencoders) in improving model performance and robustness. 4. **Practical Applications and Impact**: - The paper explores the practical applications of foundation models in areas such as environmental monitoring, agriculture, urban planning, and disaster management, demonstrating how these models significantly improve the performance of various downstream tasks. - It highlights the advantages of foundation models in handling large-scale, multi-modal, and multi-temporal remote sensing data. ### Summary By systematically reviewing and analyzing foundation models in the field of remote sensing, this paper aims to provide researchers and practitioners with a comprehensive resource to help them understand the latest developments and potential application directions of these models. The paper not only summarizes the strengths and weaknesses of existing models but also points out possible paths for future research, providing important references for advancing remote sensing technology.