Abstract:Where dual-numbers forward-mode automatic differentiation (AD) pairs each scalar value with its tangent value, dual-numbers reverse-mode AD attempts to achieve reverse AD using a similarly simple idea: by pairing each scalar value with a backpropagator function. Its correctness and efficiency on higher-order input languages have been analysed by Brunel, Mazza and Pagani, but this analysis used a custom operational semantics for which it is unclear whether it can be implemented efficiently. We take inspiration from their use of linear factoring to optimise dual-numbers reverse-mode AD to an algorithm that has the correct complexity and enjoys an efficient implementation in a standard functional language with support for mutable arrays, such as Haskell. Aside from the linear factoring ingredient, our optimisation steps consist of well-known ideas from the functional programming community. We demonstrate the use of our technique by providing a practical implementation that differentiates most of Haskell98. Where previous work on dual numbers reverse AD has required sequentialisation to construct the reverse pass, we demonstrate that we can apply our technique to task-parallel source programs and generate a task-parallel derivative computation.

Dual Numbers and Operational Umbral Methods

Operational Umbral Calculus

Riordan arrays and applications via the classical umbral calculus

Dual Hybrid Numbers and Their Hybrid Matrix Representations

Parallel Dual-Numbers Reverse AD

A new approach to Leonardo number sequences with the dual vector and dual angle representation

A Characterization of Umbral Calculus Inspired by Fractional Sums

Sheffer-Dunkl Sequences Via Umbral Calculus in the Dunkl Context

Umbral Calculus and Cancellative Semigroup Algebras

Unveiling new perspectives of hypergeometric functions using umbral techniques

Calculus on Dual Real Numbers

Multi-indice B-series

A note on dual third order Jacobsthal vectors

A dual number formulation to efficiently compute higher order directional derivatives

Integral equations, fractional calculus and shift operator

Calculation of machine precision second order derivatives using dual-complex numbers

Deriving two dualities simultaneously from a family of identities for multiple harmonic sums

Duality in stochastic processes from the viewpoint of basis expansions

Dual Core-Nilpotent Decomposition and Dual Binary Relation

Dual $n_1$-Appell-like Systems in Infinite-Dimensional Analysis