Abstract:In numerous robotics and mechanical engineering applications, among others, data is often constrained on smooth manifolds due to the presence of rotational degrees of freedom. Common datadriven and learning-based methods such as neural ordinary differential equations (ODEs), however, typically fail to satisfy these manifold constraints and perform poorly for these applications. To address this shortcoming, in this paper we study a class of neural ordinary differential equations that, by design, leave a given manifold invariant, and characterize their properties by leveraging the controllability properties of control affine systems. In particular, using a result due to Agrachev and Caponigro on approximating diffeomorphisms with flows of feedback control systems, we show that any map that can be represented as the flow of a manifold-constrained dynamical system can also be approximated using the flow of manifold-constrained neural ODE, whenever a certain controllability condition is satisfied. Additionally, we show that this universal approximation property holds when the neural ODE has limited width in each layer, thus leveraging the depth of network instead for approximation. We verify our theoretical findings using numerical experiments on PyTorch for the manifolds S2 and the 3-dimensional orthogonal group SO(3), which are model manifolds for mechanical systems such as spacecrafts and satellites. We also compare the performance of the manifold invariant neural ODE with classical neural ODEs that ignore the manifold invariant properties and show the superiority of our approach in terms of accuracy and sample complexity.

Homotopy methods for zero finding from a learning/control Liapunov function viewpoint

A Control Liapunov Function Approach to Generalized and Regularized Descent Methods for Zero Finding.

Iterative learning control of inhomogeneous distributed parameter systems - frequency domain design and analysis.

Convergence analysis of inexact proximal point algorithms on Hadamard manifolds

A Control-Theoretic Approach to the Design of Zero Finding Numerical Methods

Proximal Point Algorithms on Hadamard Manifolds: Linear Convergence and Finite Termination

Real-time robust and adaptive solutions to zero finding problems with uncertainty

Immersion and Invariance Stabilization of Nonlinear Systems: A Horizontal Contraction Approach

A Numerical Study of Basins of Attraction of Zero-Finding Neural Nets Designed Using Control Theory

Globally convergent homotopies for discrete-time optimal control

Functorial Manifold Learning

Learning on Manifolds: Universal Approximations Properties using Geometric Controllability Conditions for Neural ODEs

An attempt to apply the homotopy method to the domain of machine learning

Tight bounds for the learning of homotopy à la Niyogi, Smale, and Weinberger for subsets of Euclidean spaces and of Riemannian manifolds

Approximately Harmonic Projection: Theoretical Analysis and an Algorithm

An Inverse Mapping With Manifold Alignment For Zero-Shot Learning

Learning the Solution Manifold in Optimization and Its Application in Motion Planning

Homotopic policy iteration-based learning design for unknown linear continuous-time systems

Data-driven approximation of control invariant set for linear system based on convex piecewise linear fitting

Deep neural network approximations for the stable manifolds of the Hamilton-Jacobi-Bellman equations

VISCOCITY SOLUTIONS AND APPROXIMATE ALGORITHM ANALYSIS OF NONLINEAR H∞ CONTROL