Neural network approach to computing matrix inversion

Luo Fa-Long,Bao Zheng

DOI: https://doi.org/10.1016/0096-3003(92)90040-8

IF: 4.397

1992-01-01

Applied Mathematics and Computation

Abstract:A neural network for matrix inversion is proposed. We show both analytically and by simulations that this network is guaranteed to be stable and to provide results arbitrarily close to the accurate...

What problem does this paper attempt to address?

A Novel Recurrent Neural Network-Based Ultra-Fast, Robust, and Scalable Solver for Inverting a "Time-Varying Matrix"

Vahid Tavakkoli,Jean Chamberlain Chedjou,Kyandoghere Kyamakya

DOI: https://doi.org/10.3390/s19184002

2019-09-16

Abstract:The concept presented in this paper is based on previous dynamical methods to realize a time-varying matrix inversion. It is essentially a set of coupled ordinary differential equations (ODEs) which does indeed constitute a recurrent neural network (RNN) model. The coupled ODEs constitute a universal modeling framework for realizing a matrix inversion provided the matrix is invertible. The proposed model does converge to the inverted matrix if the matrix is invertible, otherwise it converges to an approximated inverse. Although various methods exist to solve a matrix inversion in various areas of science and engineering, most of them do assume that either the time-varying matrix inversion is free of noise or they involve a denoising module before starting the matrix inversion computation. However, in the practice, the noise presence issue is a very serious problem. Also, the denoising process is computationally expensive and can lead to a violation of the real-time property of the system. Hence, the search for a new 'matrix inversion' solving method inherently integrating noise-cancelling is highly demanded. In this paper, a new combined/extended method for time-varying matrix inversion is proposed and investigated. The proposed method is extending both the gradient neural network (GNN) and the Zhang neural network (ZNN) concepts. Our new model has proven that it has exponential stability according to Lyapunov theory. Furthermore, when compared to the other previous related methods (namely GNN, ZNN, Chen neural network, and integration-enhanced Zhang neural network or IEZNN) it has a much better theoretical convergence speed. To finish, all named models (the new one versus the old ones) are compared through practical examples and both their respective convergence and error rates are measured. It is shown/observed that the novel/proposed method has a better practical convergence rate when compared to the other models. Regarding the amount of noise, it is proven that there is a very good approximation of the matrix inverse even in the presence of noise.
From Zhang Neural Network to Newton Iteration for Matrix Inversion

Yunong Zhang,Weimu Ma,Binghuang Cai

DOI: https://doi.org/10.1109/tcsi.2008.2007065

2009-07-01

Abstract:Different from gradient-based neural networks, a special kind of recurrent neural network (RNN) has recently been proposed by Zhang et al. for online matrix inversion. Such an RNN is designed based on a matrix-valued error function instead of a scalar-valued error function. In addition, it was depicted in an implicit dynamics instead of an explicit dynamics. In this paper, we develop and investigate a discrete-time model of Zhang neural network (termed as such and abbreviated as ZNN for presentation convenience), which is depicted by a system of difference equations. Comparing with Newton iteration for matrix inversion, we find that the discrete-time ZNN model incorporates Newton iteration as its special case. Noticing this relation, we perform numerical comparisons on different situations of using ZNN and Newton iteration for matrix inversion. Different kinds of activation functions and different step-size values are examined for superior convergence and better stability of ZNN. Numerical examples demonstrate the efficacy of both ZNN and Newton iteration for online matrix inversion.
Global Exponential Convergence and Stability of Gradient-Based Neural Network for Online Matrix Inversion

Yunong Zhang,Yanyan Shi,Ke Chen,Chaoli Wang

DOI: https://doi.org/10.1016/j.amc.2009.06.048

IF: 4.397

2009-01-01

Applied Mathematics and Computation

Abstract:Wang proposed a gradient-based neural network (GNN) to solve online matrix-inverses. Global asymptotical convergence was shown for such a neural network when applied to inverting nonsingular matrices. As compared to the previously-presented asymptotical convergence, this paper investigates more desirable properties of the gradient-based neural network; e.g., global exponential convergence for nonsingular matrix inversion, and global stability even for the singular-matrix case. Illustrative simulation results further demonstrate the theoretical analysis of gradient-based neural network for online matrix inversion.
Nonlinearity Activated Noise-Tolerant Zeroing Neural Network for Real-Time Varying Matrix Inversion

Wenhui Duan,Long Jin,Bin Hu,Huiyan Lu,Mei Liu,Kene Li,Lin Xiao,Chenfu Yi

DOI: https://doi.org/10.23919/chicc.2018.8483798

2018-01-01

Abstract:Real-time varying matrix inversion is widely used in the fields of science and engineering, e.g., image processing, signal processing and robot technology, etc. In this paper, a nonlinearity activated noise-tolerant zeroing neural network (NANTZNN) is constructed and employed to the time-dependent matrix inversion in the noisy environment. Compared with the gradient approach related neural network (GNN) and the existing noise-tolerant zeroing neural network (NTZNN), the proposed NANTZNN model is activated by specially-constructed nonlinear activation functions, and thus possesses the better convergence performance. Additionally, theoretical analyses are provided to guarantee the convergence of the proposed model. Finally, simulations are conducted to demonstrate the efficiency and superiority of the NANTZNN model for time-dependent matrix inversion, as compared with the NTZNN model.
The Link Between Newton Iteration for Matrix Inversion and Zhang Neural Network (ZNN)

Yunong Zhang,Weimu Ma,Chenfu Yi

DOI: https://doi.org/10.1109/icit.2008.4608578

2008-01-01

Abstract:Different from gradient-based neural networks, a special kind of recurrent neural network has recently been proposed by Zhang et al for online matrix inversion. Such a neural network is designed based on a matrix-valued error function instead of a scalar-valued norm-based error function. In this paper, we develop and investigate a discrete-time model of Zhang neural network (termed as such and abbreviated to ZNN for presentation convenience), which is depicted by a system of difference equations. Compared with Newton iteration for matrix inversion, we find that the discrete-time ZNN model incorporates Newton iteration as one of its special cases. Noticing this relation, we perform numerical comparisons on different situations of using Zhang neural network and Newton iteration for the matrix inversion. Different kinds of activation functions and different step-size values are examined as well for the superior convergence and better stability of ZNN model. Numerical examples demonstrate the effectiveness of both ZNN model and Newton iteration for constant matrix inversion.
MATLAB Simulation of Gradient-Based Neural Network for Online Matrix Inversion

Yunong Zhang,Ke Chen,Weimu Ma,Xiao-Dong Li

DOI: https://doi.org/10.1007/978-3-540-74205-0_12

2007-01-01

Abstract:This paper investigates the simulation of a gradient-based recurrent neural network for online solution of the matrix-inverse problem. Several important techniques are employed as follows to simulate such a neural system. 1) Kronecker product of matrices is introduced to transform a matrix-differential-equation (MDE) to a vector-differential-equation (VDE); i.e., finally, a standard ordinary-differential-equation (ODE) is obtained. 2) MATLAB routine "ode45" is introduced to solve the transformed initial-value ODE problem. 3) In addition to various implementation errors, different kinds of activation functions are simulated to show the characteristics of such a neural network. Simulation results substantiate the theoretical analysis and efficacy of the gradient-based neural network for online constant matrix inversion.
Neural networks based approach for computing eigenvectors and eigenvalues of symmetric matrix

Zhang Yi,Yan Fu,Hua Jin Tang

DOI: https://doi.org/10.1016/S0898-1221(04)90110-1

IF: 3.218

2004-01-01

Computers & Mathematics with Applications

Abstract:Efficient computation of eigenvectors and eigenvalues of a matrix is an important problem in engineering, especially for computing eigenvectors corresponding to largest or smallest eigenvalues of a matrix. This paper proposes a neural network based approach to compute eigenvectors corresponding to the largest or smallest eigenvalues of any real symmetric matrix. The proposed network model is described by differential equations, which is a class of continuous time recurrent neural network model. It has parallel processing ability in an asynchronous manner and can achieve high computing performance. This paper provides a clear mathematical understanding of the network dynamic behaviors relating to the computation of eigenvectors and eigenvalues. Computer simulation results show the computational capability of the network model.
Neural Network Approach to Computing Outer Inverses Based on the Full Rank Representation

Predrag S. Stanimirovic,Ivan S. Zivkovic,Yimin Wei

DOI: https://doi.org/10.1016/j.laa.2016.03.035

IF: 1.307

2016-01-01

Linear Algebra and its Applications

Abstract:The first intention of this paper is to emphasize equivalence between two well known general representations of outer inverses with prescribed range and null space of a given matrix. Two dynamical state equations, corresponding to particular expressions which are included in these representations, are defined within the second objective. In this way, two gradient based neural networks, initiated by introduced dynamical equations, are exploited in generating the class of outer inverses. The results corresponding to most common generalized inverses are obtained in particular cases. Numerical results are presented.
An Adaptive Gradient Neural Network to Solve Dynamic Linear Matrix Equations

Shan Liao,Jiayong Liu,Yimeng Qi,Haoen Huang,Rongfeng Zheng,Xiuchun Xiao

DOI: https://doi.org/10.1109/tsmc.2021.3129855

2021-01-01

IEEE Transactions on Systems, Man, and Cybernetics: Systems

Abstract:In this article, the existing approaches, including numerical algorithms as well as neural networks to solve dynamic linear matrix equations, have been presented and reviewed. Specifically, the conventional gradient recurrent neural networks (CGRNNs) and the conventional zeroing neural networks (CZNNs) are successively provided to solve the dynamic problems and linear matrix equations, both of which manifest inherent limitations during the solving procedures. To remedy the drawbacks on convergence time, nonzero residual error, and large computational load of the traditional models, an adaptive gradient recurrent neural network (AGRNN) to solve dynamic linear matrix equations is proposed. This proposed inversion-free model possesses rapid convergence rate and accurate calculated solutions. Moreover, theoretical analyses guarantee the advantages of the AGRNN compared with the CGRNN and the CZNN to solve dynamic linear matrix equations. Finally, three numerical experiments, and applications to a PUMA 560 robot motion planning and a mobile subject localization based on angle-of-arrival technique are implemented to testify the advantages of the AGRNN.
Performance Analysis of Gradient Neural Network Exploited for Online Time-Varying Matrix Inversion

Yunong Zhang,Ke Chen,Hong-Zhou Tan

DOI: https://doi.org/10.1109/tac.2009.2023779

IF: 6.549

2009-01-01

IEEE Transactions on Automatic Control

Abstract:This technical note presents theoretical analysis and simulation results on the performance of a classic gradient neural network (GNN), which was designed originally for constant matrix inversion but is now exploited for time-varying matrix inversion. Compared to the constant matrix-inversion case, the gradient neural network inverting a time-varying matrix could only approximately approach its time-varying theoretical inverse, instead of converging exactly. In other words, the steady-state error between the GNN solution and the theoretical/exact inverse does not vanish to zero. In this technical note, the upper bound of such an error is estimated firstly. The global exponential convergence rate is then analyzed for such a Hopfield-type neural network when approaching the bound error. Computer-simulation results finally substantiate the performance analysis of this gradient neural network exploited to invert online time-varying matrices.
A parallel computing method based on zeroing neural networks for time-varying complex-valued matrix Moore-Penrose inversion

Xiuchun Xiao,Chengze Jiang,Huiyan Lu,Long Jin,Dazhao Liu,Haoen Huang,Yi Pan

DOI: https://doi.org/10.1016/j.ins.2020.03.043

IF: 8.1

2020-01-01

Information Sciences

Abstract:•The complex-valued zeroing neural network (CVZNN) model, activated by the complex-valued saturation-allowed activation functions, is proposed for solving the dynamic complex-valued matrix Moore-Penrose inverse.•The convergence performance of the proposed CVZNN model for solving the dynamic complex-valued matrix Moore-Penrose inverse problem is analyzed by using Lyapunov stability theory.•The robustness of the CVZNN model under noise interference, i.e., the time-varying bounded noises and constant noises, is investigated, separately.•Simulation results and theoretical analysis verify the validity and superiority of the proposed CVZNN model for solving the dynamic complex-valued Moore-Penrose inverse problem.
An Efficient Second‐order Neural Network Model for Computing the Moore–Penrose Inverse of Matrices

Lin Li,Jianhao Hu

DOI: https://doi.org/10.1049/sil2.12156

IF: 1.819

2022-01-01

IET Signal Processing

Abstract:The computation of the Moore-Penrose inverse is widely encountered in science and engineering. Due to the parallel-processing nature and strong-learning ability, the neural network has become a promising approach to solving the Moore-Penrose inverse recently. However, almost all the existing neural networks for matrix inversion are based on the gradient-descent (GD) method, whose main drawbacks are slow convergence and sensitivity to learning parameters. Moreover, there is no unified neural network to compute the Moore-Penrose inverse for both the full-rank matrix and rank-deficient matrix. In this paper, an efficient second-order neural network model with the improved Newton's method is proposed to obtain the accurate Moore-Penrose inverse of an arbitrary matrix by one epoch without any learning parameter. Compared with the GD-based neural networks for Moore-Penrose inverse computation, the proposed model converges faster and has lower complexity. Furthermore, through in-depth derivation, the neural network for computing the Moore-Penrose inverse is well interpretable. Numerical studies and application to the random matrix inversion in multiple-input multiple-output detection are provided to validate the efficiency of the proposed model for solving the Moore-Penrose inverse.
Integration-Enhanced Zhang Neural Network for Real-Time-Varying Matrix Inversion in the Presence of Various Kinds of Noises

Long Jin,Yunong Zhang,Shuai Li

DOI: https://doi.org/10.1109/tnnls.2015.2497715

IF: 14.255

2015-01-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:Matrix inversion often arises in the fields of science and engineering. Many models for matrix inversion usually assume that the solving process is free of noises or that the denoising has been conducted before the computation. However, time is precious for the real-time-varying matrix inversion in practice, and any preprocessing for noise reduction may consume extra time, possibly violating the requirement of real-time computation. Therefore, a new model for time-varying matrix inversion that is able to handle simultaneously the noises is urgently needed. In this paper, an integration-enhanced Zhang neural network (IEZNN) model is first proposed and investigated for real-time-varying matrix inversion. Then, the conventional ZNN model and the gradient neural network model are presented and employed for comparison. In addition, theoretical analyses show that the proposed IEZNN model has the global exponential convergence property. Moreover, in the presence of various kinds of noises, the proposed IEZNN model is proven to have an improved performance. That is, the proposed IEZNN model converges to the theoretical solution of the time-varying matrix inversion problem no matter how large the matrix-form constant noise is, and the residual errors of the proposed IEZNN model can be arbitrarily small for time-varying noises and random noises. Finally, three illustrative simulation examples, including an application to the inverse kinematic motion planning of a robot manipulator, are provided and analyzed to substantiate the efficacy and superiority of the proposed IEZNN model for real-time-varying matrix inversion.
A Poisson’s Equation Solver Based on Neural Network Precondtioned CG Method

Tianchen Shao,Tao Shan,Maokun Li,Fan Yang,Shenheng Xu

DOI: https://doi.org/10.1109/aces-china56081.2022.10064917

2022-01-01

Abstract:In this study, we investigate the feasibility of utilizing deep learning technique to construct preconditioners for iterative matrix solvers. A neural network (NN) is proposed to simulate the optimum-preconditioner’s mapping properties, and participates in NN-precondtioned conjugate gradient (NNPCG) method. Training and testing sets are generated by finite difference method (FDM). Numerical examples demonstrate that compared to conjugate gradient (CG) method, NN-PCG significantly improves convergence performance on solving 2-D Possion’s equation.
More Than Newton Iterations Generalized from Zhang Neural Network for Constant Matrix Inversion Aided with Line-Search Algorithm

Yunong Zhang,Dongsheng Guo,Chenfu Yi,Lingfeng Li,Zhende Ke

DOI: https://doi.org/10.1109/icca.2010.5524442

2010-01-01

Abstract:Since 12 March 2001, Zhang et al have proposed a special class of recurrent neural networks for online time-varying problems solving, especially for matrix inversion. For possible hardware (e.g., digital-circuit) realization, such Zhang neural networks (ZNN) could also be reformulated in the discrete-time form, which incorporates Newton iteration as a special case. In this paper, for constant matrix inversion, we generalize and investigate more discrete-time ZNN models (which could also be termed as ZNN iterations) by using multiple-point backward-difference formulas. For fast convergence to the theoretical inverse, a line-search algorithm is employed to obtain an appropriate step-size value (in each iteration). Computer-simulation results demonstrate the efficacy of the presented new discrete-time ZNN models aided with a line-search algorithm, as compared to Newton iteration.
RNN Models for Dynamic Matrix Inversion: A Control-Theoretical Perspective

Long Jin,Shuai Li,Bin Hu

DOI: https://doi.org/10.1109/tii.2017.2717079

IF: 12.3

2018-01-01

IEEE Transactions on Industrial Informatics

Abstract:In this paper, the existing recurrent neural network (RNN) models for solving zero-finding (e.g., matrix inversion) with time-varying parameters are revisited from the perspective of control and unified into a control-theoretical framework. Then, limitations on the activated functions of existing RNN models are pointed out and remedied with the aid of control-theoretical techniques. In addition, gradient-based RNNs, as the classical method for zero-finding, have been remolded to solve dynamic problems in manners free of errors and matrix inversions. Finally, computer simulations are conducted and analyzed to illustrate the efficacy and superiority of the modified RNN models designed from the perspective of control. The main contribution of this paper lies in the removal of the convex restriction and the elimination of the matrix inversion in existing RNN models for the dynamic matrix inversion. This work provides a systematic approach on exploiting control techniques to design RNN models for robustly and accurately solving algebraic equations.
Matlab Simulink Modeling and Simulation of Zhang Neural Network for Online Time-Varying Matrix Inversion

Yunong Zhang,Xiaojiao Guo,Weimu Ma,Ke Chen,Binghuang Cai

DOI: https://doi.org/10.1109/icnsc.2008.4525454

2008-01-01

Abstract:Recently, a special kind of recurrent neural networks (RNN) with implicit dynamics has been proposed by Zhang et al for online time-varying problems solving (such as time-varying matrix inversion). Such a neural-dynamic system is elegantly designed by defining a matrix-valued error function rather than the usual scalar-valued norm-based error function. Its computational error can be made decrease to zero globally and exponentially. For the final purpose of field programmable gate array (FPGA) and application-specific integrated circuit (ASIC) realization, we investigate in this paper the MATLAB Simulink modeling and simulative verification of such a Zhang neural network (ZNN). By using click-and-drag mouse operations, it is easier to model and simulate in comparison with MATLAB coding. Both convergence and robustness properties of such a ZNN model are analyzed, which substantiate the effectiveness of Zhang neural network on inverting the time-varying matrices.
Zhang Neural Network Without Using Time-Derivative Information for Constant and Time-Varying Matrix Inversion

Yunong Zhang,Zenghai Chen,Ke Chen,Binghuang Cai

DOI: https://doi.org/10.1109/ijcnn.2008.4633780

2008-01-01

Abstract:To obtain the inverses of time-varying matrices in real time, a special kind of recurrent neural networks has recently been proposed by Zhang et al. It is proved that such a Zhang neural network (ZNN) could globally exponentially converge to the exact inverse of a given time-varying matrix. To find out the effect of time-derivative term on global convergence as well as for easier hardware-implementation purposes, the ZNN model without exploiting time-derivative information is investigated in this paper for inverting online matrices. Theoretical results of both constant matrix inversion case and time-varying matrix inversion case are presented for comparative and illustrative purposes. In order to substantiate the presented theoretical results, computer-simulation results are shown, which demonstrate the importance of time derivative term of given matrices on the exact convergence of ZNN model to time-varying matrix inverses.
Constant matrix inversion method of discrete-time ZNN based on forward-difference and central-difference

Yunong ZHANG,Weibing LI,Dongsheng GUO,Zhijun ZHANG,Zhanwei HOU

DOI: https://doi.org/10.3969/j.issn.0253-2778.2013.04.001

2013-01-01

Journal of University of Science and Technology of China

Abstract:A special class of recurrent neural networks(ZNN),different from the conventional gradient-based neural network,were proposed in 2001 for solving time-varying problems(e.g.time-varying matrix inversion).For possible digital-circuit realization,such ZNN models need to be discretized.Based on the previous work,a method depicted by a series of discrete-time ZNN(DTZNN) models was proposed for matrix inversion by exploiting multiple-point forward-difference and central-difference formulas.Mathematical analysis shows that Newton iteration is actually a special case of DTZNN models.In order to verify the efficacy of the DTZNN models,these models are applied for constant matrix inversion.In addition,a line-search algorithm is employed to guarantee the convergence of such DTZNN models.Results show that the discrete-time ZNN models based on difference formulas and aided with line-search algorithm are effective on constant matrix inversion and have superior convergent performance.
Real-Time Neural Computation of the Eigenvectors of A Matrix

FL LUO,YD LI

DOI: https://doi.org/10.1109/tencon.1993.320129

1993-01-01

Abstract:This paper proposes a neural network approach for computing in real-time the eigenvectors corresponding to all the eigenvalues of a matrix. We show that the proposed neural network is guaranteed to be stable and to provide the desired eigenvectors with arbitrarily small error during an elapsed time of only a few characteristic time constants of the network.<>

Neural network approach to computing matrix inversion

A Novel Recurrent Neural Network-Based Ultra-Fast, Robust, and Scalable Solver for Inverting a "Time-Varying Matrix"

From Zhang Neural Network to Newton Iteration for Matrix Inversion

Global Exponential Convergence and Stability of Gradient-Based Neural Network for Online Matrix Inversion

Nonlinearity Activated Noise-Tolerant Zeroing Neural Network for Real-Time Varying Matrix Inversion

The Link Between Newton Iteration for Matrix Inversion and Zhang Neural Network (ZNN)

MATLAB Simulation of Gradient-Based Neural Network for Online Matrix Inversion

Neural networks based approach for computing eigenvectors and eigenvalues of symmetric matrix

Neural Network Approach to Computing Outer Inverses Based on the Full Rank Representation

An Adaptive Gradient Neural Network to Solve Dynamic Linear Matrix Equations

Performance Analysis of Gradient Neural Network Exploited for Online Time-Varying Matrix Inversion

A parallel computing method based on zeroing neural networks for time-varying complex-valued matrix Moore-Penrose inversion

An Efficient Second‐order Neural Network Model for Computing the Moore–Penrose Inverse of Matrices

Integration-Enhanced Zhang Neural Network for Real-Time-Varying Matrix Inversion in the Presence of Various Kinds of Noises

A Poisson’s Equation Solver Based on Neural Network Precondtioned CG Method

More Than Newton Iterations Generalized from Zhang Neural Network for Constant Matrix Inversion Aided with Line-Search Algorithm

RNN Models for Dynamic Matrix Inversion: A Control-Theoretical Perspective

Matlab Simulink Modeling and Simulation of Zhang Neural Network for Online Time-Varying Matrix Inversion

Zhang Neural Network Without Using Time-Derivative Information for Constant and Time-Varying Matrix Inversion

Constant matrix inversion method of discrete-time ZNN based on forward-difference and central-difference

Real-Time Neural Computation of the Eigenvectors of A Matrix