Abstract:The celebrated Kitaev honeycomb model provides an analytically tractable example with an exact quantum spin liquid ground state. While in real materials, other types of interactions besides the Kitaev coupling (K) are present, such as Heisenberg (J) and symmetric off-diagonal (Γ) terms, these interactions can also be generalized to a triangular lattice. Here, we carry out a comprehensive study of the J-K-Γ model on a triangular lattice covering the whole parameter region, using a combination of exact diagonalization, classical Monte Carlo, and analytic methods, with an emphasis on the effects of the Γ term. In the HK limit (Γ=0), we find five quantum phases which are quite similar to their classical counterparts. Among them, stripe-A and dual Néel phases are robust against the introduction of the Γ term, in particular, stripe A extends to the region connecting K=−1 and K=1 for Γ<0. Though the 120∘ Néel phase also extends to a finite Γ, its region is largely reduced compared to the previous classical result. Interestingly, the ferromagnetic (dubbed FM-A) and stripe-B phases are unstable in response to an infinitesimal Γ interaction. Moreover, we find five additional phases for Γ≠0 which are elaborated by both the quantum and classical numerical methods. Part of the parameter space previously identified as the 120∘ Néel phase in the classical study is found to give way to the modulated stripe phase. Depending on the sign of the Γ term, the FM-A phase transits into FM-B (Γ>0) and FM-C (Γ<0) phases with different spin orientations. Similarly, the stripe-B phase transits into stripe-C (Γ>0) and stripe-A (Γ<0) phases. Around the positive Γ point, due to the interplay of the Heisenberg, Kitaev, and Γ interactions, we find a possible quantum spin liquid in a noticeable region with a continuum in spin excitations.

Phase diagram of ferromagnetic XY model with nematic coupling on a triangular lattice

Phase Diagram of Xy Model with Nematic Coupling on the Simple Cubic Lattice

Study on the Ising-like phase transitions in square lattice fully frustrated XY models

Quantum Phase Transitions in the Triangular Coupled-Top Model

Second-order Phase Transition in the Fully Frustratedxymodel with Next-Nearest-neighbor Coupling

Phase Transitions in Frustrated Xy Model on A Square Lattice

Phase diagram of generalized XY model using tensor renormalization group

Phase diagram of the quantum Ising model on a triangular lattice under external field

Phase Transitions of XY Model in Ultra-Thin Magnetic Film with a Triangular Latt Ice

Ferromagnetic Quantum Critical Behavior in Heavy-Fermion Compounds Ceti1-Xnixge3

Is There a Phase Transition in the Isotropic Heisenberg Antiferromagnet on the Triangular Lattice?

Phase diagrams of antiferromagnetic $XY$ model on a triangular lattice with higher-order interactions

Critical properties of XY model on two-layer Villain-ferromagnetic lattice

Phase Diagram and Topological Phases in the Triangular Lattice Kitaev-Hubbard Model

Magnetic and charge orders on the triangular lattice: Extended Hubbard model with intersite Ising-like magnetic interactions in the atomic limit

XY model on a four-layer triangular lattice with competition between ferromagnetic and antiferromagnetic layers

Phase Diagram of the Anisotropic Xy Model

Monte Carlo study on low-temperature phase diagrams of the $J_1$-$J_2$ classical $XY$ kagome antiferromagnet

Comprehensive study of the global phase diagram of the J − K − Γ model on a triangular lattice

Monte Carlo Approach to Phase Transitions in Ferroelectromagnets