Abstract:Derivatives of differential equation solutions are commonly for parameter estimation, fitting neural differential equations, and as model diagnostics. However, with a litany of choices and a Cartesian product of potential methods, it can be difficult for practitioners to understand which method is likely to be the most effective on their particular application. In this manuscript we investigate the performance characteristics of Discrete Local Sensitivity Analysis implemented via Automatic Differentiation (DSAAD) against continuous adjoint sensitivity analysis. Non-stiff and stiff biological and pharmacometric models, including a PDE discretization, are used to quantify the performance of sensitivity analysis methods. Our benchmarks show that on small stiff and non-stiff systems of ODEs (approximately < 100 parameters+ODEs), forward-mode DSAAD is more efficient than both reverse-mode and continuous forward/adjoint sensitivity analysis. The scalability of continuous adjoint methods is shown to be more efficient than discrete adjoints and forward methods after crossing this size range. These comparative studies demonstrate a trade-off between memory usage and performance in the continuous adjoint methods that should be considered when choosing the technique, while numerically unstable backsolve techniques from the machine learning literature are demonstrated as unsuitable for most scientific models. The performance of adjoint methods is shown to be heavily tied to the reverse-mode AD method used for the vector-Jacobian product calculations, with tape-based AD methods shown to be 2 orders of magnitude slower on nonlinear partial differential equations than static AD techniques. In addition, these results demonstrate the out-of-the-box applicability of DSAAD to differential-algebraic equations, delay differential equations, and hybrid differential equation systems where the event timing and effects are dependent on model parameters, showcasing an ease of implementation advantage for DSAAD approaches. Together, these benchmarks provide a guide to help practitioners to quickly identify the best mixture of continuous sensitivities and automatic differentiation for their applications.

Derivative-Based Global Sensitivity Analysis for Models with High-Dimensional Inputs and Functional Outputs

Global sensitivity analysis of computer models with functional inputs

Derivative based global sensitivity analysis and its entropic link

Data-driven Global Sensitivity Analysis of Variable Groups for Understanding Complex Physical Interactions in Engineering Design

A screening approach for non-parametric global sensitivity analysis

Active Learning for Derivative-Based Global Sensitivity Analysis with Gaussian Processes

EXTREME LEARNING MACHINES FOR VARIANCE-BASED GLOBAL SENSITIVITY ANALYSIS

Surrogate-assisted global sensitivity analysis: an overview

Efficient global sensitivity analysis method for dynamic models in high dimensions

Derivative-based global sensitivity measures and their link with Sobol sensitivity indices

A Comparison of Automatic Differentiation and Continuous Sensitivity Analysis for Derivatives of Differential Equation Solutions

Mixed-Variable Global Sensitivity Analysis For Knowledge Discovery And Efficient Combinatorial Materials Design

An Improved Copula‐Based Framework for Efficient Global Sensitivity Analysis

Hyper-Differential Sensitivity Analysis for Inverse Problems Constrained by Partial Differential Equations

Variance-based sensitivity analysis for time-dependent processes

A-optimal designs for non-parametric symmetrical global sensitivity analysis

Analytical Global Sensitivity Analysis with Gaussian Processes

Generalized sensitivity functions for size-structured population models

Exploration of methods for computing sensitivities in ODE models at dynamic and steady states

Global sensitivity analysis using derivative-based sparse Poincaré chaos expansions

On one dimensional weighted Poincare inequalities for Global Sensitivity Analysis