Abstract:The higher-spin Kitaev magnets, in which the Kitaev interaction and off-diagonal exchange couplings are overwhelmingly large, have emerged as a fertile avenue to explore exotic phases and unusual excitations. In this work, we study the quantum phase diagram of the spin-1 Kitaev-$\Gamma$ model on the honeycomb lattice using density-matrix renormalization group. It harbours six distinct phases and the intriguing findings are three magnetically ordered phases in which both time-reversal symmetry and lattice symmetry albeit of different sort are broken spontaneously. The chiral spin state originates from the order-by-disorder effect and exhibits an almost saturated scalar spin chirality at the quantum level. Depending on the relative strength of the two interactions, it also features columnar-like or plaquette-like dimer pattern as a consequence of the translational symmetry breaking. In parallel, the nematic ferromagnets are situated at ferromagnetic Kitaev side and possess small but finite ferromagnetic ordering. The lattice-rotational symmetry breaking enforces nonequivalent bond energy along one of the three bonds. Although the intrinsic difference between the two nematic ferromagnets remains elusive, the discontinuities in the von Neumann entropy, hexagonal plaquette operator, and Wilson loop operator convincingly suggest that they are separated via a first-order phase transition.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper mainly explores the quantum phase diagram of the spin - 1 Kitaev - Γ model on the honeycomb lattice. Specifically, the authors study how the relative strength changes between the Kitaev interaction and the Γ interaction affect the ground - state properties of the system through the density matrix renormalization group (DMRG) method. The following are the specific problems that this paper attempts to solve: 1. **Explore new magnetic ordered states**: - The study found that there are six different phases in the spin - 1 Kitaev - Γ model, three of which are magnetic ordered phases. These magnetic ordered phases spontaneously break time - reversal symmetry and lattice symmetry, showing unique physical properties. - One particularly interesting phase is the chiral spin state, which originates from the order - by - disorder effect and shows an almost saturated scalar spin chirality. 2. **Understand the properties of nematic ferromagnets**: - The paper also reveals the existence of two nematic ferromagnets. These phases are near the ferromagnetic Kitaev limit, have finite ferromagnetic order, and due to the breaking of lattice rotation symmetry, the bond energies in different directions are not equivalent. - Through the changes of von Neumann entropy, hexagonal plaquette operator and Wilson loop operator, the authors determine the first - order quantum phase transition from Kitaev quantum spin liquid to nematic ferromagnet. 3. **Draw a complete quantum phase diagram**: - The authors systematically change the ratio of Kitaev and Γ interactions to draw a complete quantum phase diagram. This helps to understand the novel phases in higher - spin Kitaev magnets, even though there are no corresponding actual materials at present. 4. **Verify the theoretical model**: - By comparing with the classical phase diagram and comparing the numerical simulation results of different cluster sizes, the validity of the theoretical model is verified. For example, by calculating von Neumann entropy and nematic order parameter, the existence and characteristics of nematic ferromagnets are confirmed. In summary, this paper aims to deeply understand the complex quantum phase behavior in the spin - 1 Kitaev - Γ model, especially the formation mechanisms and physical properties of the chiral spin state and nematic ferromagnets. This research not only expands the understanding of the Kitaev model, but also provides theoretical guidance for finding and designing new quantum materials. ### Involved formulas 1. **Hamiltonian of the spin - 1 Kitaev - Γ model**: \[ H=\sum_{\langle ij \rangle \| \gamma}\left(K S_{i}^{\gamma} S_{j}^{\gamma}+\Gamma\left(S_{i}^{\alpha} S_{j}^{\beta}+S_{i}^{\beta} S_{j}^{\alpha}\right)\right) \] where $S_{i}^{\gamma}$ is the $\gamma$ - component of the spin operator, $\langle ij \rangle \| \gamma$ represents the $\gamma$ - type nearest - neighbor bond connecting $i$ and $j$, and $\alpha$ and $\beta$ are the remaining two bond types. 2. **Hexagonal plaquette operator**: \[ \hat{W}_{p}=e^{i\pi(S_{x}^{1}+S_{y}^{2}+S_{z}^{3}+S_{x}^{4}+S_{y}^{5}+S_{z}^{6})} \] 3. **Wilson loop operator**: \[ \hat{W}_{l}=\prod_{j = 1}^{2L_{y}}e^{i\pi S_{y}^{j}} \] 4. **Static structure factor (SSF)**: \[

Chiral spin state and nematic ferromagnet in the spin-1 Kitaev-$Γ$ model

Unveiling the phase diagram of a bond-alternating spin-1/2 <i>K</i>- chain

Unveiling the phase diagram of a bond-alternating spin-$\frac12$ $K$-$\Gamma$ chain

The Heart of Entanglement: Chiral, Nematic, and Incommensurate Phases in the Kitaev-Gamma Ladder in a Field

Unveiling the Phase Diagram of a Bond-Alternating Spin- 12 K−Γ Chain

Ground state phase diagram and the exotic phases in the spin-1/2 square lattice J1-J2-Jx model

Spontaneous dimerization, spin-nematic order, and deconfined quantum critical point in a spin-1 Kitaev chain with tunable single-ion anisotropy

Quantum phase transition between spin liquid and spin nematics in spin-1 Kitaev honeycomb model

Eight-color chiral spin liquid in the $S=1$ bilinear-biquadratic model with Kitaev interactions

Emergent magnetic order in the antiferromagnetic Kitaev model with a [111] field

Left-left-right-right magnetic order in spin-1/2 Kitaev-Heisenberg chain

Kitaev-Heisenberg model on the star lattice: From chiral Majorana fermions to chiral triplons

Multinode Quantum Spin Liquids on the Honeycomb Lattice

Chiral spin liquid in a generalized Kitaev honeycomb model with $\mathbb{Z}_4$ 1-form symmetry

Effective Model for Rare-earth Kitaev Materials and Its Classical Monte Carlo Simulation

Chiral spin liquid and quantum phase diagram of spin-$1/2$ $J_1$-$J_2$-$J_χ$ model on the square lattice

Complex orders and chirality in the classical Kitaev-$Γ$ model

Nematic correlations and nematic Berezinskii-Kosterlitz-Thouless transition in spin-1 kagomé lattice antiferromagnets

Topological phases of higher Chern numbers in Kitaev-Heisenberg ferromagnet with further-neighbor interactions

Classical spin order near the antiferromagnetic Kitaev point in the spin- 1 2 Kitaev-Gamma chain

Nematic chiral spin liquid in a Kitaev magnet under external magnetic field