Abstract:The possible existence of a quantum spin liquid (QSL) phase, an exotic state of matter with long-range quantum entanglement and fractionalized excitations, in $\alpha$-RuCl$_3$ has sparked widespread interests in exploring QSLs in various Kitaev models under magnetic fields. Recently, a $K$-$J$-$\Gamma$-$\Gamma'$ model has been proposed to accurately describe the compound by fitting the measured thermodynamic data, where $J$ is Heisenberg interaction, and $\Gamma$, $\Gamma'$ are off-diagonal exchanges on top of the dominant Kitaev coupling $K$. Based on this effective model, an intermediate QSL phase in presence of an out-of-plane fields along the $[1 1 1]$ direction, between the low-field zigzag order and high-field polarized phase, has been predicted. By combining density matrix renormalization group (DMRG), exponential tensor renormalization group (XTRG), and variational Monte Carlo (VMC) calculations, we address the nature of this QSL phase in the honeycomb $K$-$J$-$\Gamma$-$\Gamma'$ model under the $[1 1 1]$-direction field. Our DMRG calculations find the algebraic-like decay of spin correlation function, the finite spin scalar chiral order, and lattice nematic order. Together with the XTRG results of power-law specific heat at low temperature, our findings naturally suggest a gapless nematic chiral spin liquid. On the other hand, our VMC study finds a gapped nematic chiral spin liquid with the variational energy very close to that obtained in DMRG. As VMC finds a very small gap that is beyond the current resolution of both ground-state DMRG and finite-temperature XTRG calculations on finite-width cylinders, we resort the full clarification for the nature of the intermediate QSL to future studies. Lastly, we discuss the implications of our results to the recent experiment on $\alpha$-RuCl$_3$ and the QSL-like phase in a similar $K$-$\Gamma$-$\Gamma'$ model.

Critical Spin Liquid versus Valence Bond Glass in Triangular Lattice Organic $κ$-(ET)$_2$Cu$_2$(CN)$_3$

Novel Dipole-Lattice coupling in the Quantum-Spin-Liquid Material $κ$-(BEDT-TTF)$_2$Cu$_2$(CN)$_3$

Gapped magnetic ground state in quantum-spin-liquid candidate $κ$-(BEDT-TTF)$_2$-Cu$_2$(CN)$_3$

Quantum criticality in a metallic spin liquid

Spin-Peierls instability of the U(1) Dirac spin liquid

Dynamics of K2Ni2(SO4)3 governed by proximity to a 3D spin liquid model

Dynamics of K$_2$Ni$_2$(SO$_4$)$_3$ governed by proximity to a 3D spin liquid model

Quasi-one-dimensional Spin Dynamics in a Molecular Spin Liquid System

A possible quantum spin liquid candidate with Cu-Kagome lattice

Nematic chiral spin liquid in a Kitaev magnet under external magnetic field

Chiral Spin Liquid and Quantum Phase Transition in the Triangular Lattice Hofstadter-Hubbard Model

Spin excitations of a proximate Kitaev quantum spin liquid realized in Cu$_2$IrO$_3$

Evidence of quantum spin liquid state in a Cu$^{2+}$-based $S = 1/2$ triangular lattice antiferromagnet

Combinatorial exploration of quantum spin liquid candidates in the herbertsmithite material family

Possible Kitaev Quantum Spin Liquid State in 2D Materials with S=3/2

Quantum Orders and Spin Liquids in Cs$_2$CuCl$_4$

Magnetic ground states in kagome YCu$_3$(OH)$_6$[(Cl$_x$Br$_{1-x}$)$_{3-y}$(OH)$_{y}$]

Emerging quantum critical phase in a cluster spin-glass

Spin liquid and ferroelectricity close to a quantum critical point in PbCuTe$_2$O$_6$

Emergent Symmetry in Quantum Phase Transitions: From Deconfined Quantum Critical Point to Gapless Quantum Spin Liquid

Spin Liquid Landscapes in the Kagome Lattice: A Variational Monte Carlo Study of the Chiral Heisenberg Model and Experimental Consequences