Abstract:We illustrate a technique to calculate the gradient of scalar functions that are defined on any arbitrary matrix subspace. It generalizes our earlier work titled "What is the gradient of a scalar function of a symmetric matrix ?"( Indian Journal of Pure and Applied Mathematics (2022), https://doi.org/10.1007/s13226-022-00313-x ), in which we considered the special case of the subspace of symmetric matrices. Extant methods to calculate the gradient in such cases have an inherent flaw which leads to spurious results that populate several publications, as well as respected textbooks and handbooks on matrix calculus. We examine these sources and results in a rigorous and concrete mathematical setting of a finite-dimensional inner-product space and discover the inherent flaw and also a remedy. We demonstrate two ways to calculate the derivative/gradient and second derivative for scalar functions of matrices defined over an arbitrary matrix subspace; the first method is by considering any (differentiable) extension to the space of square matrices and projection of its gradient onto the given subspace. The second method utilizes an ordered basis and computes each component of the gradient through evaluation of the directional derivative. All the ideas presented are illustrated by non-trivial examples, namely, considering the subspace of circulant and Toeplitz matrices and presenting the results of gradient-descent with both the spurious and correct gradients. Moreover, our bibliography makes it clear that a rigorous approach to matrix calculus is not common in practice, and our presentation of matrix calculus in the language of inner-product spaces will be significant and meaningful for applied mathematicians, engineers and researchers working in inter-disciplinary fields to avoid the conceptual pitfalls that exist.

On the gradient rearrangement of functions

Symmetric gradient Sobolev spaces endowed with rearrangement-invariant norms

Gradient flows in asymmetric metric spaces and applications

Convergence Analysis of Gradient Algorithms on Riemannian Manifolds Without Curvature Constraints and Application to Riemannian Mass

Unilateral gradient flow of the Ambrosio-Tortorelli functional by minimizing movements

On De Giorgi's Conjecture of Nonlocal approximations for free-discontinuity problems: The symmetric gradient case

Anisotropic symmetrization, convex bodies, and isoperimetric inequalities

A relation between Γ-convergence of functionals and their associated gradient flows

Rearrangement and polarization

Gradient higher integrability for singular parabolic double-phase systems

Souplet-Zhang and Hamilton type gradient estimates for nonlinear elliptic equations on smooth metric measure spaces

Symmetries in an overdetermined problem for the Green's function

What is the gradient of a scalar function defined on a subspace of square matrices ?

A Symmetry problem for some quasi-linear equations in Euclidean space

A Relation Between Г-convergence of Functionals and Their Associated Gradient Flows

A slope generalization of Attouch theorem

On the role of semismoothness in nonsmooth numerical analysis: Theory

Gradient descent for unbounded convex functions on Hadamard manifolds and its applications to scaling problems

Spectral properties of the gradient operator with nonconstant coefficients

On the equivalence between SOR-type methods for linear systems and discrete gradient methods for gradient systems

Beltrami operators, non--symmetric elliptic equations and quantitative Jacobian bounds