Generalization Enhancement Strategies to Enable Cross-year Cropland Mapping with Convolutional Neural Networks Trained Using Historical Samples

Sam Khallaghi,Rahebe Abedi,Hanan Abou Ali,Mary Dziedzorm Asipunu,Ismail Alatise,Nguyen Ha,Boka Luo,Cat Mai,Lei Song,Amos Wussah,Sitian Xiong,Qi Zhang,Lyndon D. Estes

2024-08-15

Abstract:The accuracy of mapping agricultural fields across large areas is steadily improving with high-resolution satellite imagery and deep learning (DL) models, even in regions where fields are small and geometrically irregular. However, developing effective DL models often requires large, expensive label datasets, typically available only for specific years or locations. This limits the ability to create annual maps essential for agricultural monitoring, as domain shifts occur between years and regions due to changes in farming practices and environmental conditions. The challenge is to design a model flexible enough to account for these shifts without needing yearly labels. While domain adaptation techniques or semi-supervised training are common solutions, we explored enhancing the model's generalization power. Our results indicate that a holistic approach is essential, combining methods to improve generalization. Specifically, using an area-based loss function, such as Tversky-focal loss (TFL), significantly improved predictions across multiple years. The use of different augmentation techniques helped to encode different types of invariance, particularly photometric augmentations encoded invariance to brightness changes, though they increased false positives. The combination of photometric augmentation, TFL loss, and MC-dropout produced the best results, although dropout alone led to more false negatives in subsequent year predictions. Additionally, the choice of input normalization had a significant impact, with the best results obtained when statistics were calculated either locally or across the entire dataset over all bands (lab and gab). We developed a workflow that enabled a U-Net model to generate effective multi-year crop maps over large areas. Our code, available at: <a class="link-external link-https" href="https://github.com/agroimpacts/cnn-generalization-enhancement" rel="external noopener nofollow">this https URL</a>, will be regularly updated with improvements.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### What problems does this paper attempt to solve?

This paper aims to solve the **domain shift** problem encountered when mapping crops between different years, especially when using Convolutional Neural Network (CNN) models for large - scale farmland mapping. Specifically, the main challenges faced by the researchers include:

1. **High cost of data annotation**: Developing effective deep - learning models usually requires large and expensive annotated datasets, and these datasets are often only for specific years or geographical locations.

2. **Difficulty in generating annual maps**: Due to changes in agricultural practices and environmental conditions, the domain shift phenomenon between different years is significant, which limits the ability to generate reliable farmland maps every year.

3. **Insufficient generalization ability of the model**: Existing deep - learning models perform poorly when dealing with cross - year data, especially when facing small - scale, irregularly - shaped and dynamically - changing farmlands in small - holder areas.

To address these challenges, the authors propose a method to enhance the generalization ability of the model, enabling it to accurately map crops over multiple years without new annotated data each year. Specific methods include:

- **Using the Tversky - focal loss function (TFL)**: Adjust the loss function to improve the model's adaptability to data from different years.

- **Image enhancement techniques**: Apply multiple photometric enhancement techniques to encode different types of invariance, especially invariance to brightness changes.

- **Input normalization**: Select an appropriate normalization method, such as local or global statistical calculation, to optimize model performance.

- **MC - dropout**: Combine the Monte Carlo dropout method to provide prediction uncertainty estimates during the inference stage while enhancing the robustness and generalization ability of the model.

Finally, the researchers developed a workflow that enables the U - Net model to generate effective multi - year crop maps on a large scale, thus providing a more cost - effective solution for agricultural monitoring.