Abstract:This paper establishes several convergence results about flows of the dynamical system LCA (Locally Competitive Algorithm) to the mixed $\ell_2$-$\ell_1$ minimization problem LASSO and the constrained version, called CLASSO here, where the parameters are required to be non-negative. (C)LASSO problems are closely related to various important applications including efficient coding, image recognition and image reconstruction. That the solution of (C)LASSO can be determined by LCA allows the former to be solved in novel ways such as through a physical realization of analog circuits or on non-von Neumann computers. As discussed in the paper, previous works that show convergence of LCA to LASSO are incomplete, and do not consider CLASSO. The main contributions of this paper are a particular generalization of LaSalle's invariance principle and its application to rigorously establish LCA's convergence to (C)LASSO.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is whether the LCA (Locally Competitive Algorithm) flow can converge to the solution of the (C)LASSO problem. Specifically:
- **LASSO problem**: Minimize the linear regression problem with regularization, that is, solve the following optimization problem:
\[
\arg \min_{a \in \mathbb{R}^N} \frac{1}{2} \| s - \Phi a \|_2^2 + \lambda \| a \|_1
\]
where \( s \in \mathbb{R}^M \), \(\Phi \in \mathbb{R}^{M \times N}\), and \(\lambda > 0\).
- **CLASSO problem**: The constrained version of the LASSO problem, which requires the components of the parameter vector \( a \) to be non - negative, that is, solve the following optimization problem:
\[
\arg \min_{a \in \mathbb{R}^N, a \geq 0} \frac{1}{2} \| s - \Phi a \|_2^2 + \lambda \| a \|_1
\]
The paper points out that although previous literature has discussed the convergence of the LCA flow to the LASSO problem, these results are incomplete, especially without considering the CLASSO problem. Therefore, the main contributions of this paper are:
1. **Generalize the LaSalle invariance principle**: By generalizing the LaSalle invariance principle, it is strictly proved that the LCA flow can converge to the solution of the (C)LASSO problem under certain conditions.
2. **Fill the theoretical gap**: Solve the theoretical deficiencies in previous literature regarding the convergence of the LCA flow, especially when dealing with the non - negative - constrained CLASSO problem.
Through these contributions, the paper provides a solid theoretical basis for understanding and applying the LCA algorithm, especially in fields such as image processing, sparse coding, and compressed sensing.