Abstract:The orbits of the orthogonal and symplectic groups on the flag variety are in bijection, respectively, with the involutions and fixed-point-free involutions in the symmetric group $S_n$. Wyser and Yong have described polynomial representatives for the cohomology classes of the closures of these orbits, which we denote as $\hat{\mathfrak{S}}_y$ (to be called involution Schubert polynomials) and $\hat{\mathfrak{S}}^{\tt FPF}_y$ (to be called fixed-point-free involution Schubert polynomials). Our main results are explicit formulas decomposing the product of $\hat{\mathfrak{S}}_y$ (respectively, $\hat{\mathfrak{S}}^{\tt FPF}_y$) with any $y$-invariant linear polynomial as a linear combination of other involution Schubert polynomials. These identities serve as analogues of Lascoux and Schützenberger's transition formula for Schubert polynomials, and lead to a self-contained algebraic proof of the nontrivial equivalence of several definitions of $\hat{\mathfrak{S}}_y$ and $\hat{\mathfrak{S}}^{\tt FPF}_y$ appearing in the literature. Our formulas also imply combinatorial identities about involution words, certain variations of reduced words for involutions in $S_n$. We construct operators on involution words based on the Little map to prove these identities bijectively. The proofs of our main theorems depend on some new technical results, extending work of Incitti, about covering relations in the Bruhat order of $S_n$ restricted to involutions.

On Involutions in Symmetric Groups and a Conjecture of Lusztig

Positivity conjectures for Kazhdan–Lusztig theory on twisted involutions: The finite case

Volume conjecture for $SU(n)$-invariants

On Involutions inWeyl Groups

On the Cuspidal Cohomology of Arithmetic Groups

Integral U-Deformed Involution Modules

Wall-crossing in genus zero Landau-Ginzburg theory

An Inversion Formula for -Relative Kazhdan–Lusztig Polynomials

An Inversion Formula for [image omitted]-Relative Kazhdan-Lusztig Polynomials

The geometry of twisted conjugacy classes in wreath products

Mullineux involution and twisted affine Lie algebras

Involutions in Weyl Groups

Transition formulas for involution Schubert polynomials

An Inversion Formula For (J, (J)Over-Cap)-Relative Kazhdan-Lusztig Polynomials

Involution words: counting problems and connections to Schubert calculus for symmetric orbit closures

Lusztig sheaves and integrable highest weight modules in symmetrizable case

Conjectures P1-P15 for hyperbolic Coxeter groups of rank 3

Lefschetz numbers of symplectic involutions on arithmetic groups

Simply-laced Coxeter groups and groups generated by symplectic transvections

Denominators in Lusztig's asymptotic Hecke algebra via the Plancherel formula

The correspondence between the canonical and semicanonical bases