Abstract:Every simple Hermitian Lie group has a unique family of spherical representations induced from a maximal parabolic subgroup whose unipotent radical is a Heisenberg group. For most Hermitian groups, this family contains a complementary series, and at its endpoint sits a proper unitarizable subrepresentation. We show that this subrepresentation is next-to-minimal in the sense that its associated variety is a next-to-minimal nilpotent coadjoint orbit. Moreover, for the Hermitian groups $\operatorname{SO}_0(2,n)$ and $E_{6(-14)}$ we study some branching problems of these next-to-minimal representations.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the study of the sub - minimal representations in the Heisenberg parabolic - induced representations of Hermitian Lie groups and their branching laws. Specifically:
1. **Definition and Properties of Sub - Minimal Representations**:
- The paper first defines the concept of sub - minimal representations, that is, their associated varieties are the closures of a sub - minimal nilpotent co - adjoint orbit. These representations have an important position in Lie group representation theory, but compared with minimal representations, they have been less studied.
- The author proves that in some Hermitian Lie groups, sub - minimal representations appear at the end of the complementary series, and these representations are unitary and irreducible sub - representations.
2. **Construction of Sub - Minimal Representations**:
- The author constructs sub - minimal representations by studying Heisenberg parabolically - induced representations. These representations can be realized by induced representations on the maximal parabolic subgroups of Hermitian Lie groups.
- Specifically, for the Lie groups \( \text{SO}_0(2,n) \) and \( E_6(-14) \), the author studies in detail the properties of these sub - minimal representations.
3. **Branching Laws**:
- The paper further studies the restriction problem of these sub - minimal representations on some symmetric subgroups. In particular, the author considers the case of restricting these representations to \( \text{SL}(2,\mathbb{R}) \times M \), where \( M \) is a subgroup of the Hermitian Lie group.
- For \( \text{SO}_0(2,n) \), the author gives the decomposition formula of the sub - minimal representation on \( \text{SL}(2,\mathbb{R}) \times \text{SL}(2,\mathbb{R}) \times \text{SO}(n - 2) \).
- For \( E_6(-14) \), the author also gives a similar decomposition formula, but the representations involved are more complex.
### Main Results
1. **Construction of Sub - Minimal Representations**:
- The author proves that in the Hermitian Lie group \( G \), there exists a sub - minimal representation \( \pi_{\text{ntm}} \), which is an irreducible, unitary sub - representation and is spherical.
- The \( K \)-type decomposition of this representation is also given in the paper.
2. **Branching Laws**:
- For \( \text{SO}_0(2,n) \), the decomposition formula of the sub - minimal representation \( \pi_{\text{ntm}} \) restricted to \( \text{SL}(2,\mathbb{R}) \times \text{SL}(2,\mathbb{R}) \times \text{SO}(n - 2) \) is:
\[
\pi_{\text{ntm}} \big|_{\text{SL}(2,\mathbb{R}) \times \text{SL}(2,\mathbb{R}) \times \text{SO}(n - 2)} \simeq \bigoplus_{k = 0}^{\infty} \left( \int_{\mathbb{R}_{\geq 0}} \tau_{\mu, \text{par}(\mu)} \boxtimes \tau_{\mu, \text{par}(\mu)} \, d\mu \oplus \bigoplus_{2 \leq \ell \leq k, \ell \equiv k \mod 2} \tau_{\ell} \boxtimes \tau_{\ell} \right) \boxtimes \eta_k^{\text{SO}(n - 2)}
\]
- For \( E_6(-14) \), the sub - minimal representation \( \pi_{\text{ntm}} \) (the text seems to be incomplete here for \( E_6(-14) \) part)