MAGIC: Modular Auto-encoder for Generalisable Model Inversion with Bias Corrections

Yihang She,Clement Atzberger,Andrew Blake,Adriano Gualandi,Srinivasan Keshav

2024-05-29

Abstract:Scientists often model physical processes to understand the natural world and uncover the causation behind observations. Due to unavoidable simplification, discrepancies often arise between model predictions and actual observations, in the form of systematic biases, whose impact varies with model completeness. Classical model inversion methods such as Bayesian inference or regressive neural networks tend either to overlook biases or make assumptions about their nature during data preprocessing, potentially leading to implausible results. Inspired by recent work in inverse graphics, we replace the decoder stage of a standard autoencoder with a physical model followed by a bias-correction layer. This generalisable approach simultaneously inverts the model and corrects its biases in an end-to-end manner without making strong assumptions about the nature of the biases. We demonstrate the effectiveness of our approach using two physical models from disparate domains: a complex radiative transfer model from remote sensing; and a volcanic deformation model from geodesy. Our method matches or surpasses results from classical approaches without requiring biases to be explicitly filtered out, suggesting an effective pathway for understanding the causation of various physical processes.

Machine Learning

What problem does this paper attempt to address?

The problem addressed in this paper is how to correct model biases and extract physically meaningful causal variables during the process of physical model inversion, especially when dealing with systemic biases in incomplete models.

Traditional inversion methods such as Bayesian inference and neural network regression may overlook or make assumptions about biases during data preprocessing, leading to errors in the inversion results. Inspired by inverse graphics, the paper proposes a modularized autoencoder method called MAGIC (Model-Aware Geometrically Invariant Convolutional) by replacing the decoder stage of a standard autoencoder with a physics model followed by a bias correction layer. This method is able to invert the model and correct biases simultaneously in an end-to-end manner without making strong assumptions about the nature of biases.

The paper validates the proposed method using two physics models from different domains: a complex radiative transfer model in remote sensing and a volcano deformation model in geology. Experimental results show that this method achieves performance comparable to or beyond traditional methods without the need for explicit bias filtering, providing an effective approach to understanding the causal relationships in various physical processes.

The paper also discusses the balance between the incompleteness of physics models, the complexity of bias correction, and the interpretability of causal variables. It also explains how to make complex physics models differentiable and stable during training using large language models (LLMs) to improve the generalizability of the method.

In summary, the MAGIC method aims to address bias issues in physical model inversion and provides a universal approach for a more accurate understanding of causal relationships in natural phenomena.