Abstract:Crop classification is an important task in remote sensing with many applications, such as estimating yields, detecting crop diseases and pests, and ensuring food security. In this study, we combined knowledge from remote sensing, machine learning, and agriculture to investigate the application of transfer learning with a transformer model for variable length satellite image time series (SITS). The objective was to produce a map of agricultural land, reduce required interventions, and limit in-field visits. Specifically, we aimed to provide reliable agricultural land class predictions in a timely manner and quantify the necessary amount of reference parcels to achieve these outcomes. Our dataset consisted of Sentinel-2 satellite imagery and reference crop labels for Slovenia spanning over years 2019, 2020, and 2021. We evaluated adaptability through fine-tuning in a real-world scenario of early crop classification with limited up-to-date reference data. The base model trained on a different year achieved an average F1 score of 82.5% for the target year without having a reference from the target year. To increase accuracy with a new model trained from scratch, an average of 48,000 samples are required in the target year. Using transfer learning, the pre-trained models can be efficiently adapted to an unknown year, requiring less than 0.3% (1500) samples from the dataset. Building on this, we show that transfer learning can outperform the baseline in the context of early classification with only 9% of the data after 210 days in the year.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to apply transfer learning for early crop classification in satellite image time series (SITS) over many years. Specifically, the research aims to process satellite image time - series data of different lengths by using the Transformer model to achieve the following goals: 1. **Generate agricultural land maps**: Generate detailed distribution maps of agricultural land by early identification of crop types. 2. **Reduce the need for intervention**: Reduce the need for field visits by providing timely and reliable crop - classification predictions. 3. **Quantify the number of reference plots**: Determine the minimum number of reference plots required to achieve the above - mentioned goals. ### Research background - **Importance of crop classification**: Crop classification has many applications in remote sensing, such as estimating yield, detecting crop diseases and pests, and ensuring food security. - **Limitations of existing methods**: Most existing studies use data from a single year for training and testing, which limits the generalization ability of the model. Changes in meteorological conditions can affect the development of vegetation, resulting in different time - series characteristics of the same crop in different years. - **Advantages of transfer learning**: Through transfer learning, historical data can be used to pre - train the model and then fine - tune it in the target year, thereby improving the performance of the model, especially when the reference data is limited. ### Research methods - **Dataset**: The research used Sentinel - 2 satellite images and reference crop labels in Slovenia from 2019 to 2021. - **Model architecture**: The SITS - BERT model based on Transformer was adopted, which can process time - series data of different lengths and learn general spectral - temporal features using a self - supervised pre - training scheme. - **Experimental design**: - **Temporal generalization**: Evaluate the generalization ability of the model between different years. - **Under - sampling**: Study the impact of a limited sample size on the performance of the model. - **Transfer learning**: Use the pre - trained model and limited data in the target year for fine - tuning to optimize the performance of the model. - **Early classification**: Combine the above experiments to evaluate the impact of limited time information on the performance of the model, simulating the situation where new observation data gradually increases in practical applications. ### Main results - **Temporal generalization**: The model performs well in the training years and can also maintain high performance in unseen years. - **Under - sampling**: As the number of samples increases, the performance of the model gradually improves. When the number of samples reaches 48,000, the performance of the model trained from scratch exceeds that of the baseline model. - **Transfer learning**: After fine - tuning the pre - trained model in the target year, less than 0.3% of the samples are required to achieve good performance. - **Early classification**: In practical applications, the model can provide reliable crop - classification predictions early in the growing season, and only 9% of the data is required to exceed the performance of the baseline model after 210 days. ### Conclusion This research demonstrates the effectiveness and advantages of transfer learning in early crop classification, especially when the reference data is limited. By using historical data to pre - train the model and then fine - tuning it in the target year, the performance and generalization ability of the model can be significantly improved. This method provides strong support for crop monitoring and management in practical applications.

Multi-Year Time Series Transfer Learning: Application of Early Crop Classification

Cropnet: Deep Spatial-Temporal-Spectral Feature Learning Network for Crop Classification from Time-Series Multi-Spectral Images

Transfer learning approach based on satellite image time series for the crop classification problem

A Multi-Level Fine-Grained Crop Classification Method Based on Multi-Expert Knowledge Distill

Temporally transferable crop mapping with temporal encoding and deep learning augmentations

Deep Learning-Based Crop Mapping in the Cloudy Season Using One-Shot Hyperspectral Satellite Imagery.

Early Crop Classification via Multi-Modal Satellite Data Fusion and Temporal Attention

Towards Global Crop Maps with Transfer Learning

Multimodal Crop Type Classification Fusing Multi-Spectral Satellite Time Series with Farmers Crop Rotations and Local Crop Distribution

Deep learning based multi-temporal crop classification

Crop Classification of Satellite Imagery Using Synthetic Multitemporal and Multispectral Images in Convolutional Neural Networks

End-to-End Learned Early Classification of Time Series for In-Season Crop Type Mapping

Crop Type Mapping from Optical and Radar Time Series Using Attention-Based Deep Learning

A CNN-Transformer Hybrid Approach for Crop Classification Using Multitemporal Multisensor Images

DeepCropMapping: A multi-temporal deep learning approach with improved spatial generalizability for dynamic corn and soybean mapping

Evaluation of Data Sufficiency for Interannual Knowledge Transfer of Crop Type Classification Models

Boosting crop classification by hierarchically fusing satellite, rotational, and contextual data

Towards more efficient agricultural practices via transformer-based crop type classification

Efficient Multi-temporal and In-season Crop Mapping with Landsat Analysis Ready Data via Long Short-term Memory Networks

Transferability of Machine Learning Models for Crop Classification in Remote Sensing Imagery Using a New Test Methodology: A Study on Phenological, Temporal, and Spatial Influences