Abstract:In neural networks, it is often desirable to work with various representations of the same space. For example, 3D rotations can be represented with quaternions or Euler angles. In this paper, we advance a definition of a continuous representation, which can be helpful for training deep neural networks. We relate this to topological concepts such as homeomorphism and embedding. We then investigate what are continuous and discontinuous representations for 2D, 3D, and n-dimensional rotations. We demonstrate that for 3D rotations, all representations are discontinuous in the real Euclidean spaces of four or fewer dimensions. Thus, widely used representations such as quaternions and Euler angles are discontinuous and difficult for neural networks to learn. We show that the 3D rotations have continuous representations in 5D and 6D, which are more suitable for learning. We also present continuous representations for the general case of the n-dimensional rotation group SO(n). While our main focus is on rotations, we also show that our constructions apply to other groups such as the orthogonal group and similarity transforms. We finally present empirical results, which show that our continuous rotation representations outperform discontinuous ones for several practical problems in graphics and vision, including a simple autoencoder sanity test, a rotation estimator for 3D point clouds, and an inverse kinematics solver for 3D human poses.

Configuration of Continuous Piecewise-Linear Neural Networks

A special kind of neural networks: continuous piecewise linear functions

A Topological Approach to the Existence of Solutions for Nonlinear Differential Equations with Piecewise Constant Argument

Continuous Volumetric Convolution Network with Self-Learning Kernels for Point Clouds

Approximation And Inverse Control Of Nonlinear System Using Standard Continuous Piecewise Linear Neural Networks

Identification Algorithm for Standard Continuous Piecewise Linear Neural Network

Continuous piecewise linear identification with moderate number of subregions

Some Theoretical Issues on Continuous Process Neural Networks

A New Neural Network Model And Its Application

Nonlinear system identification with continuous piecewise linear neural network

Standard neural network model and its application

A discrete-time neural network for optimization problems with hybrid constraints.

Piecewise Linear Neural Networks and Deep Learning

Training neural networks for solving 1-D optimal piecewise linear approximation

Continuous Neural Decoding Method Based on General Regression Neural Network.

A Process Neural Network for Continuous Process Approximation and Its Application

ReLU neural network approximation to piecewise constant functions

Linearly Constrained Global Optimization Via Piecewise-Linear Approximation

On the Continuity of Rotation Representations in Neural Networks

Continuous Piecewise Linear Programming Via Concave Optimization and Genetic Algorithm