Abstract:In both plant breeding and crop management, interpretability plays a crucial role in instilling trust in AI-driven approaches and enabling the provision of actionable insights. The primary objective of this research is to explore and evaluate the potential contributions of deep learning network architectures that employ stacked LSTM for end-of-season maize grain yield prediction. A secondary aim is to expand the capabilities of these networks by adapting them to better accommodate and leverage the multi-modality properties of remote sensing data. In this study, a multi-modal deep learning architecture that assimilates inputs from heterogeneous data streams, including high-resolution hyperspectral imagery, LiDAR point clouds, and environmental data, is proposed to forecast maize crop yields. The architecture includes attention mechanisms that assign varying levels of importance to different modalities and temporal features that, reflect the dynamics of plant growth and environmental interactions. The interpretability of the attention weights is investigated in multi-modal networks that seek to both improve predictions and attribute crop yield outcomes to genetic and environmental variables. This approach also contributes to increased interpretability of the model's predictions. The temporal attention weight distributions highlighted relevant factors and critical growth stages that contribute to the predictions. The results of this study affirm that the attention weights are consistent with recognized biological growth stages, thereby substantiating the network's capability to learn biologically interpretable features. Accuracies of the model's predictions of yield ranged from 0.82-0.93 R2 ref in this genetics-focused study, further highlighting the potential of attention-based models. Further, this research facilitates understanding of how multi-modality remote sensing aligns with the physiological stages of maize. The proposed architecture shows promise in improving predictions and offering interpretable insights into the factors affecting maize crop yields, while demonstrating the impact of data collection by different modalities through the growing season. By identifying relevant factors and critical growth stages, the model's attention weights provide valuable information that can be used in both plant breeding and crop management. The consistency of attention weights with biological growth stages reinforces the potential of deep learning networks in agricultural applications, particularly in leveraging remote sensing data for yield prediction. To the best of our knowledge, this is the first study that investigates the use of hyperspectral and LiDAR UAV time series data for explaining/interpreting plant growth stages within deep learning networks and forecasting plot-level maize grain yield using late fusion modalities with attention mechanisms.

Deep Time Series Attention Models for Crop Yield Prediction and Insights

Crop yield prediction integrating genotype and weather variables using deep learning

TFSF: Topological and Feature Space Fusion with Spatio-Temporal Modeling for Crop Yield Prediction.

Deep Hierarchical Temporal Data Fusion Improves Yield Estimation under Extreme Climate Stress

A Temporal–Geospatial Deep Learning Framework for Crop Yield Prediction

County-Level Soybean Yield Prediction Using Deep CNN-LSTM Model

3D-ResNet-BiLSTM Model: A Deep Learning Model for County-Level Soybean Yield Prediction with Time-Series Sentinel-1, Sentinel-2 Imagery, and Daymet Data

Estimating Soybean Yields Using Causal Inference and Deep Learning Approaches With Satellite Remote Sensing Data

Integrating multi-modal remote sensing, deep learning, and attention mechanisms for yield prediction in plant breeding experiments

Comparison of Attention Mechanism-Based Deep Learning and Transfer Strategies for Wheat Yield Estimation Using Multisource Temporal Drone Imagery

Daily DeepCropNet: A Hierarchical Deep Learning Approach with Daily Time Series of Vegetation Indices and Climatic Variables for Corn Yield Estimation

Deep ensemble model with hybrid intelligence technique for crop yield prediction

A Deep Learning Approach to Conflating Heterogeneous Geospatial Data for Corn Yield Estimation: A Case Study of the US Corn Belt at the County Level.

Simultaneous Prediction of Wheat Yield and Grain Protein Content Using Multitask Deep Learning from Time-Series Proximal Sensing

Simultaneous corn and soybean yield prediction from remote sensing data using deep transfer learning

DeepCropNet: a Deep Spatial-Temporal Learning Framework for County-Level Corn Yield Estimation

Prediction of crop yield in India using machine learning and hybrid deep learning models

Explainability of Sub-Field Level Crop Yield Prediction using Remote Sensing

A GNN-RNN Approach for Harnessing Geospatial and Temporal Information: Application to Crop Yield Prediction

Is deeper always better? Evaluating deep learning models for yield forecasting with small data

Crop Yield Prediction Using Deep Reinforcement Learning Model for Sustainable Agrarian Applications