Abstract:Multi-temporal deep learning approaches have exhibited excellent classification performance in large-scale crop mapping. These approaches efficiently and automatically transform remote sensing time series into highdimensional feature representations to identify crop types. The lack of interpretation, however, is regarded as a major drawback of these high-performance approaches. Interpreting deep learning approaches in multitemporal crop mapping is critical for verifying their reliability. This study aims to quantify the impact of multi-temporal information in input time series on classification performance and develop a multi-perspective interpretation pipeline for deep learning models. The pipeline involves three interpretation approaches: evaluating input feature importance, analyzing hidden features, and monitoring temporal changes in model's soft output. An experiment is conducted to classify corn and soybean in the U.S corn belt in 2018. The study area consists of three sites each encompassing millions of pixel-level samples at 30 m resolution. The Landsat Analysis Ready Data are used as the input remote sensing time series and Cropland Data Layer is used as the ground reference. Attention-based Long Short-Term Memory (AtLSTM) and Transformer models are built as multitemporal deep learning models, and compared to Random Forest (RF). Complete time series input in the correct order achieves a higher overall accuracy of 97.8% than using single-window or out-of-order inputs, indicating multi-temporal information facilitates crop classification. An assessment of the input feature importance demonstrates that the AtLSTM, Transformer, and RF models all consider the period from weeks 11 to 20 (earlyJuly to late-August) as a key growth period and the shortwave infrared band as the critical band for corn and soybean discrimination. Hidden feature analysis suggests that the AtLSTM model accumulates the useful information over the growth period, while the Transformer model extracts the temporal dependencies that contribute important information to high-level feature learning. The learned features contain more effective and refined information than the raw input features and thus are better suited for crop classification. The soft output analysis in the in-season classification scenario demonstrates that increased length of input time series improves the model's confidence in the classification results. The further comparison of input feature importance in different sites and years demonstrates the applicability of the interpretation approach at larger spatiotemporal extents with heterogeneous landscapes and interannual variability. This study provides a multi-perspective evaluation to identify key features in multi-spectral and multi-temporal remote sensing data, and yields a practical approach to integrate agronomy knowledge in deep learning-based crop mapping.

Evaluation of Feature Selection and Feature Extraction Techniques on Multi-Temporal Landsat-8 Images for Crop Classification

Cropnet: Deep Spatial-Temporal-Spectral Feature Learning Network for Crop Classification from Time-Series Multi-Spectral Images

Deep One-Class Crop Extraction Framework for Multi-Modal Remote Sensing Imagery.

Field-Level Crop Classification Using an Optimal Dataset of Multi-temporal Sentinel-1 and Polarimetric RADARSAT-2 SAR Data with Machine Learning Algorithms

Using of Multi-Source and Multi-Temporal Remote Sensing Data Improves Crop-Type Mapping in the Subtropical Agriculture Region

Crop Classification Method Based on Optimal Feature Selection and Hybrid CNN-RF Networks for Multi-Temporal Remote Sensing Imagery

Crop type classification with combined spectral, texture, and radar features of time-series Sentinel-1 and Sentinel-2 data

PAMICRM: Improving Precision Agriculture Through Multimodal Image Analysis for Crop Water Requirement Estimation Using Multidomain Remote Sensing Data Samples

Leveraging Hyperspectral Remote Sensing Imaging for Agricultural Crop Classification Using Coot Bird Optimization With Entropy-Based Feature Fusion Model

Crop type discrimination using Geo-Stat Endmember Extraction and machine learning algorithms

Evaluation of Feature Selection Methods for Object-Based Land Cover Mapping of Unmanned Aerial Vehicle Imagery Using Random Forest and Support Vector Machine Classifiers

Extraction of Crop Planting Structure in County Based on Multi-temporal Images of Sentinel-2

Machine learning based object-level crop classification of PlanetScope data at South India Basin

PCA-based Feature Reduction for Hyperspectral Remote Sensing Image Classification

Performance Evaluation of Best Feature Subsets for Crop Yield Prediction Using Machine Learning Algorithms

Advancing crop classification in smallholder agriculture: A multifaceted approach combining frequency-domain image co-registration, transformer-based parcel segmentation, and Bi-LSTM for crop classification

Towards Interpreting Multi-Temporal Deep Learning Models in Crop Mapping

Crop classification using aerial images by analyzing an ensemble of DCNNs under multi-filter & multi-scale framework

Dipper throated optimization with deep convolutional neural network-based crop classification for remote sensing image analysis

Discriminating different landuse types by using multitemporal NDXI in a rice planting area

Crop Classification for Agricultural Applications in Hyperspectral Remote Sensing Images