Abstract:The orbits of the orthogonal and symplectic groups on the flag variety are in bijection, respectively, with the involutions and fixed-point-free involutions in the symmetric group $S_n$. Wyser and Yong have described polynomial representatives for the cohomology classes of the closures of these orbits, which we denote as $\hat{\mathfrak{S}}_y$ (to be called involution Schubert polynomials) and $\hat{\mathfrak{S}}^{\tt FPF}_y$ (to be called fixed-point-free involution Schubert polynomials). Our main results are explicit formulas decomposing the product of $\hat{\mathfrak{S}}_y$ (respectively, $\hat{\mathfrak{S}}^{\tt FPF}_y$) with any $y$-invariant linear polynomial as a linear combination of other involution Schubert polynomials. These identities serve as analogues of Lascoux and Schützenberger's transition formula for Schubert polynomials, and lead to a self-contained algebraic proof of the nontrivial equivalence of several definitions of $\hat{\mathfrak{S}}_y$ and $\hat{\mathfrak{S}}^{\tt FPF}_y$ appearing in the literature. Our formulas also imply combinatorial identities about involution words, certain variations of reduced words for involutions in $S_n$. We construct operators on involution words based on the Little map to prove these identities bijectively. The proofs of our main theorems depend on some new technical results, extending work of Incitti, about covering relations in the Bruhat order of $S_n$ restricted to involutions.

Double orthodontia formulas and Lascoux positivity

A Molev-Sagan type formula for double Schubert polynomials

Lascoux expansion of the product of a Lascoux and a stable Grothendieck

Tableau formulas for skew Schubert polynomials

A Determinantal Formula for Orthosymplectic Schur Functions

Positivity conjectures for Kazhdan–Lusztig theory on twisted involutions: The finite case

Schur Polynomials through Lindström Gessel Viennot Lemma

On A Generalization of Motohashi's Formula

Skew symplectic and orthogonal characters through lattice paths

The Gerby Gopakumar-Mariño-Vafa Formula

Partial $γ$-Positivity for Quasi-Stirling Permutations of Multisets

Transition formulas for involution Schubert polynomials

On positivity of Roger--Yang skein algebras

On Łojasiewicz Inequalities and the Effective Putinar's Positivstellensatz

Demazure crystals and the Schur positivity of Catalan functions

Polynomial bases: positivity and Schur multiplication

On some properties of symplectic Grothendieck polynomials

Lattice Paths and Branched Continued Fractions: An Infinite Sequence of Generalizations of the Stieltjes–Rogers and Thron–Rogers Polynomials, with Coefficientwise Hankel-Total Positivity

Combinatorial formulas for shifted dual stable Grothendieck polynomials

Deformed Kazhdan-Lusztig elements and Macdonald polynomials