Abstract:Process-based models are widely used to predict the agroecosystem dynamics, but such modeled results often contain considerable uncertainty due to the imperfect model structure, biased model parameters, and inaccurate or inaccessible model inputs. Data assimilation (DA) techniques are widely adopted to reduce prediction uncertainty by calibrating model parameters or dynamically updating the model state variables using observations. However, high computational cost, difficulties in mitigating model structural error, and low flexibility in framework development hinder its applications in large-scale agroecosystem predictions. In this study, we addressed these challenges by proposing a novel DA framework that integrates a Knowledge-Guided Machine Learning (KGML)-based surrogate with tensorized ensemble Kalman filter (EnKF) and parallelized particle swarm optimization (PSO) to effectively assimilate historical and in-season multi-source remote sensing data. Specifically, we incorporate knowledge from a process-based model, ecosys , into a Gated Recurrent Unit (GRU)-based hierarchical neural network. The hierarchical architecture of KGML-DA mimics key processes of ecosys and builds a causal relationship between target variables. Using carbon budget quantification in the US Corn-Belt as a context, we evaluated KGML-DA's performance in predicting key processes of the carbon cycle at three agricultural sites (US-Ne1, US-Ne2, US-Ne3), along with county-level (627 counties) and 30-m pixel-level (Champaign County, IL) grain yield. The site experiments show that updating the upstream variable, e.g., gross primary production (GPP), improved the prediction of downstream variables such as ecosystem respiration, net ecosystem exchange, biomass, and leaf area index (LAI), with RMSE reductions ranging from 9.2% to 30.5% for corn and 4.8% to 24.6% for soybean. Uncertainty in downstream variables was automatically constrained after correcting the upstream variables, demonstrating the effectiveness of the causality linkages in the hierarchical surrogate. We found joint use of in-season GPP and evapotranspiration (ET) products along with historical GPP and surveyed yields achieved the best prediction for county-level yields, while assimilating in-season LAI observations benefitted the prediction in extreme years. Uncertainty and error analysis of regional yield estimation demonstrated that KGML-DA could reduce prediction error by 26.5% for corn and 36.2% for soybean. Remarkably, the GPU-based tensor operation design makes this DA framework more than 7000 times faster than the PB model with a High-Performance Computing system, indicating the high potential of the proposed framework for in-season, high-resolution agroecosystem predictions.

Knowledge Guided Representation Learning and Causal Structure Learning in Soil Science

Causal Modeling of Soil Processes for Improved Generalization

Spatial-Temporal Causality Modeling for Industrial Processes with a Knowledge-Data Guided Reinforcement Learning

Towards Human-like Perception: Learning Structural Causal Model in Heterogeneous Graph

Domain Adaptation for Sustainable Soil Management using Causal and Contrastive Constraint Minimization

Active Learning for Efficient Soil Monitoring in Large Terrain with Heterogeneous Sensor Network

Deep Learning Optimizes Data-Driven Representation of Soil Organic Carbon in Earth System Model over the Conterminous United States

Estimating Soybean Yields Using Causal Inference and Deep Learning Approaches With Satellite Remote Sensing Data

A framework for modelling soil structure dynamics induced by biological activity

Causal machine learning for sustainable agroecosystems

Towards Robust Trajectory Representations: Isolating Environmental Confounders with Causal Learning

Causal inference for process understanding in Earth sciences

Deep Causal Learning: Representation, Discovery and Inference

Dynamic Structure Learning through Graph Neural Network for Forecasting Soil Moisture in Precision Agriculture

The Landscape of Causal Discovery Data: Grounding Causal Discovery in Real-World Applications

Spatially heterogeneous mechanisms underlying soil carbon sequestration as revealed via big data-driven Earth system modelling and deep learning

Toward More Realistic Projections of Soil Carbon Dynamics by Earth System Models

Multi-Sensor Soil Probe and Machine Learning Modeling for Predicting Soil Properties

Tracing State Structure for Ecological Processes in Soil Including Greenhouse Gas Exchange with Lower Atmosphere

A flexible and efficient knowledge-guided machine learning data assimilation (KGML-DA) framework for agroecosystem prediction in the US Midwest

PROcess-guided Deep Learning and DAta-driven Modelling (PRODA) Uncovers Key Mechanisms Underlying Global Soil Carbon Storage