Abstract:The possible existence of a quantum spin liquid (QSL) phase, an exotic state of matter with long-range quantum entanglement and fractionalized excitations, in $\alpha$-RuCl$_3$ has sparked widespread interests in exploring QSLs in various Kitaev models under magnetic fields. Recently, a $K$-$J$-$\Gamma$-$\Gamma'$ model has been proposed to accurately describe the compound by fitting the measured thermodynamic data, where $J$ is Heisenberg interaction, and $\Gamma$, $\Gamma'$ are off-diagonal exchanges on top of the dominant Kitaev coupling $K$. Based on this effective model, an intermediate QSL phase in presence of an out-of-plane fields along the $[1 1 1]$ direction, between the low-field zigzag order and high-field polarized phase, has been predicted. By combining density matrix renormalization group (DMRG), exponential tensor renormalization group (XTRG), and variational Monte Carlo (VMC) calculations, we address the nature of this QSL phase in the honeycomb $K$-$J$-$\Gamma$-$\Gamma'$ model under the $[1 1 1]$-direction field. Our DMRG calculations find the algebraic-like decay of spin correlation function, the finite spin scalar chiral order, and lattice nematic order. Together with the XTRG results of power-law specific heat at low temperature, our findings naturally suggest a gapless nematic chiral spin liquid. On the other hand, our VMC study finds a gapped nematic chiral spin liquid with the variational energy very close to that obtained in DMRG. As VMC finds a very small gap that is beyond the current resolution of both ground-state DMRG and finite-temperature XTRG calculations on finite-width cylinders, we resort the full clarification for the nature of the intermediate QSL to future studies. Lastly, we discuss the implications of our results to the recent experiment on $\alpha$-RuCl$_3$ and the QSL-like phase in a similar $K$-$\Gamma$-$\Gamma'$ model.

Muon Spin Relaxation Study of Spin Dynamics on a Kitaev Honeycomb Material H_3LiIr_2O_6

Muon Spin Relaxation Study of Spin Dynamics on a Kitaev honeycomb material H$_3$LiIr$_2$O$_6$

Muon spin relaxation study of spin dynamics on a Kitaev honeycomb material H 3 LiIr 2 O 6

Muon Spin Relaxation Study of Spin Dynamics in Quantum Spin Liquid Candidate H$_3$LiIr$_2$O$_6$

Multinode Quantum Spin Liquids on the Honeycomb Lattice

Signatures of a Gapless Quantum Spin Liquid in the Kitaev Material Na_3Co_2-xZn_xSbO_6

Gapless Quantum Spin Liquid in a Honeycomb Γ Magnet

Possible Kitaev Quantum Spin Liquid State in 2D Materials with S=3/2

Low-energy Spin Dynamics of the Quantum Spin Liquid Candidate NaYbSe2

Signatures of a gapless quantum spin liquid in the Kitaev material Na$_3$Co$_{2-x}$Zn$_x$SbO$_6$

Quasi-one-dimensional Spin Dynamics in a Molecular Spin Liquid System

Dynamics of vacancy-induced modes in the non-Abelian Kitaev spin liquid

Possible Gapless Quantum Spin Liquid Behavior in the Triangular-Lattice Ising Antiferromagnet PrMgAl11O19

Nematic chiral spin liquid in a Kitaev magnet under external magnetic field

Static and dynamical magnetic properties of the extended Kitaev-Heisenberg model with spin vacancies

Magnetic Raman Continuum in Single-Crystalline H3LiIr2O6

Possible quantum paraelectric state in Kitaev spin liquid candidate H 3 LiIr 2 O 6

Fluctuating magnetic droplets immersed in a sea of quantum spin liquid

Emergent photons and fractionalized excitations in a quantum spin liquid

Imaging quantum interference in a monolayer Kitaev quantum spin liquid candidate