Abstract:The subdifferential of convex functions of the singular spectrum of real matrices has been widely studied in matrix analysis, optimization and automatic control theory. Convex optimization over spaces of tensors is now gaining much interest due to its potential applications in signal processing, statistics and engineering. The goal of this paper is to present an extension of the approach by Lewis \cite{lewis1995convex} for the analysis of the subdifferential of certain convex functions of the spectrum of symmetric tensors. We give a complete characterization of the subdifferential of Schatten-type tensor norms for symmetric tensors. Some partial results in this direction are also given for Orthogonally Decomposable tensors.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is about the calculation and characterization of the sub - differentials of spectral functions of symmetric tensors and orthogonally decomposable tensors (odeco tensors). Specifically, the authors aim to extend the existing results in matrix analysis to the tensor setting, especially to provide a complete characterization of the sub - differentials of the Schatten norms of symmetric tensors and odeco tensors. In addition, the paper also explores the potential applications of these results in fields such as signal processing, statistics, and engineering.
### Background Introduction
Tensors can be regarded as a high - order generalization of vectors and matrices and have been widely used in fields such as statistics, signal processing, and automatic control. Many natural and useful linear algebraic quantities, such as rank or singular value decomposition, become very difficult to calculate or generalize in the tensor setting. However, for symmetric tensors, there are effective processing methods, which are particularly effective in recent statistics/machine - learning problems, such as clustering, estimation in hidden Markov chains, etc.
### Research Motivation
In machine learning, it is usually necessary to solve a least - squares problem with a penalty term, in the following form:
\[ \min_{X \in \mathbb{R}^{n_1 \times n_2}} \|y - A(X)\|+\lambda p(X), \]
where \( p \) is a penalty term that promotes low - rank sparsity, such as the nuclear norm, and \( A \) is a linear operator. In the tensor setting, the form of the problem becomes:
\[ \min_{X \in \mathbb{R}^{n_1 \times \cdots \times n_D}} \|y - A(X)\|+\lambda p(X), \]
where \( D>2 \), and \( p \) is a generalization of the tensor nuclear norm or some Schatten - type norm.
### Main Contributions
1. **Sub - differential of Symmetric Tensors**: The paper provides a complete characterization of the sub - differentials of the Schatten norms of symmetric tensors.
2. **Sub - differential of odeco Tensors**: The paper describes a subset of the sub - differentials of the Schatten norms of odeco tensors.
3. **Theoretical Tools**: The main tool in the paper is the tensor generalization of the Von Neumann trace inequality, a result recently proven by one of the authors.
### Application Prospects
These results may find applications in the field of compressed sensing. The future work plan is to extend these results to the non - symmetric setting and further study their applications in other fields.
### Conclusion
Through the study of the sub - differentials of the spectral functions of symmetric tensors and odeco tensors, the paper provides an important theoretical basis for the calculation of tensor norms and is expected to promote the development of related technologies in multiple fields.