Abstract:Recent years have witnessed a hot wave of deep neural networks in various domains; however, it is not yet well understood theoretically. A theoretical characterization of deep neural networks should point out their approximation ability and complexity, i.e., showing which architecture and size are sufficient to handle the concerned tasks. This work takes one step on this direction by theoretically studying the approximation and complexity of deep neural networks to invariant functions. We first prove that the invariant functions can be universally approximated by deep neural networks. Then we show that a broad range of invariant functions can be asymptotically approximated by various types of neural network models that includes the complex-valued neural networks, convolutional neural networks, and Bayesian neural networks using a polynomial number of parameters or optimization iterations. We also provide a feasible application that connects the parameter estimation and forecasting of high-resolution signals with our theoretical conclusions. The empirical results obtained on simulation experiments demonstrate the effectiveness of our method.

Approximation of fuzzy functions by regular fuzzy neural networks

A neurofuzzy network structure for nonlinear system modeling

Fuzzy RBF Neural Network Based Function Approximation

Function approximation ability of fuzzy neural networks

Approximation capability of the convolution methods for fuzzy numbers

Characterization of Degree of Approximation for Neural Networks with One Hidden Layer

Rough Approximation In Fuzzy Decision Systems

Optimal Method of Neuro-Fuzzy Network and Its Application Based on Variable Precision Rough Set

Approximation Capability of a Novel Neural Network Model for Dynamic Systems

Optimal Rates of Approximation by Shallow ReLU Neural Networks and Applications to Nonparametric Regression

Generalized fuzzy rough sets based on fuzzy coverings

Fuzzy Approximation Operators Based on Coverings

Approximations of Continuous Functionals by Neural Networks with Application to Dynamic Systems

Integral transform and its application to neural network approximation

Design Methods of General Fuzzy Systems As Function Approximators

Lower Estimation of Approximation Rate for Neural Networks

Rough Fuzzy Set Approximations in Fuzzy Formal Contexts

Generalized Fuzzy Rough Approximation Operators Based on Fuzzy Coverings.

Duality for Best Approximation in Fuzzy Quasi-normed Spaces

On the Approximation and Complexity of Deep Neural Networks to Invariant Functions

Fuzzy approximation network perturbation systems and its application to risk analysis in transportation capacity