Abstract:Derivatives of differential equation solutions are commonly for parameter estimation, fitting neural differential equations, and as model diagnostics. However, with a litany of choices and a Cartesian product of potential methods, it can be difficult for practitioners to understand which method is likely to be the most effective on their particular application. In this manuscript we investigate the performance characteristics of Discrete Local Sensitivity Analysis implemented via Automatic Differentiation (DSAAD) against continuous adjoint sensitivity analysis. Non-stiff and stiff biological and pharmacometric models, including a PDE discretization, are used to quantify the performance of sensitivity analysis methods. Our benchmarks show that on small stiff and non-stiff systems of ODEs (approximately < 100 parameters+ODEs), forward-mode DSAAD is more efficient than both reverse-mode and continuous forward/adjoint sensitivity analysis. The scalability of continuous adjoint methods is shown to be more efficient than discrete adjoints and forward methods after crossing this size range. These comparative studies demonstrate a trade-off between memory usage and performance in the continuous adjoint methods that should be considered when choosing the technique, while numerically unstable backsolve techniques from the machine learning literature are demonstrated as unsuitable for most scientific models. The performance of adjoint methods is shown to be heavily tied to the reverse-mode AD method used for the vector-Jacobian product calculations, with tape-based AD methods shown to be 2 orders of magnitude slower on nonlinear partial differential equations than static AD techniques. In addition, these results demonstrate the out-of-the-box applicability of DSAAD to differential-algebraic equations, delay differential equations, and hybrid differential equation systems where the event timing and effects are dependent on model parameters, showcasing an ease of implementation advantage for DSAAD approaches. Together, these benchmarks provide a guide to help practitioners to quickly identify the best mixture of continuous sensitivities and automatic differentiation for their applications.

State-space based discretize-then-differentiate adjoint sensitivity method for transient responses of non-viscously damped systems

A comparative study of design sensitivity analysis based on adjoint variable method for transient response of non-viscously damped systems

Design sensitivity analysis for transient responses of viscoelastically damped systems using model order reduction techniques

Design sensitivity analysis of dynamic response of nonviscously damped systems

The Application of Sensitivity Method in Vehicle Dynamics Analysis

A Comparison of Automatic Differentiation and Continuous Sensitivity Analysis for Derivatives of Differential Equation Solutions

Active Vibration Control Analysis of Spatial Structures by State-Derivative Feedback Algorithm

Transient Response Sensitivity Analysis of Localized Nonlinear Structure Using Direct Differentiation Method

A New Time Integration Method Based on State Formulations for Dynamic Analysis of Nonviscously Damped Systems

Comparison of numerical differentiation based damping sensitivity method

Application of Automatic Differentiation in Power System Trajectory Sensitivity Analysis

Assessment of continuous and discrete adjoint method for sensitivity analysis in two-phase flow simulations

Efficient and Accurate Method for Calculating the Stochastic Seismic Response of A Nonproportionally Damped Structure

A time-domain method for free vibration responses of an equivalent viscous damped system based on a complex damping model

A novel reduced basis method for adjoint sensitivity analysis of dynamic topology optimization

An improved time-marching formulation based on an explicit time integration method for dynamic analysis of non-viscous damping systems

An Early-Stop Adjoint Transient Sensitivity Analysis Method for Objective Functions Associated with Many Time Points

Extended Dissipative Sliding-Mode Control for Discrete-Time Piecewise Nonhomogeneous Markov Jump Nonlinear Systems

Analytical Sensitivity Analysis of Dynamic Problems with Direct Differentiation of Generalized-α Time Integration

Exploration of methods for computing sensitivities in ODE models at dynamic and steady states

An Efficient Time Step Control Method in Transient Simulation for DAE System.