Abstract:Fluctuations and disorder effects are substantially enhanced in reduced dimensionalities. While they are mostly considered as the foe for long-range orders, fluctuations and disorders can also stimulate the emergence of novel phases of matter, for example, vestigial orders. Taking 2D magnetism as a platform, existing efforts have been focused on maintaining 2D long-range magnetic orders by suppressing fluctuations, whereas the other side, exploiting fluctuations for realizing new 2D magnetic phases, remains as an uncharted territory. Here, using a combination of NV spin relaxometry, optical spectroscopy, and Monte Carlo simulations, we report, in an XY-type honeycomb magnet NiPS3, the phase transition from the zigzag AFM order in 3D bulk to a new Z3 vestigial Potts-nematicity in 2D few layers. Spin fluctuations are shown to significantly enhance over the GHz-THz range as the layer number of NiPS3 reduces, using the NV spin relaxometry and the optical Raman quasi-elastic scattering. As a result, the Raman signatures of the zigzag AFM for bulk NiPS3, a zone-folded phonon at ~30cm-1 from the broken translational symmetry (PBTS) and a degeneracy lift of two phonons at ~180cm-1 for the broken 3-fold rotational symmetry (PBRS), evolve into the disappearance of PBTS and the survival of PBRS in few-layer NiPS3, with a critical thickness of ~10nm. The optical linear dichroism microscopy images all three nematic domain states in a single few-layer NiPS3 flake. The large-scale Monte Carlo simulations for bilayer NiPS3 model confirms the absence of long-range zigzag AFM order but the formation of the Z3 vestigial Potts-nematic phase, corroborating with the experimental finding. Our results demonstrate the positivity of strong fluctuations in creating new phases of matter after destroying more conventional ones, and offer an unprecedented pathway for developing novel 2D phases.

2D Frustrated Magnets: Reentrant Spin Glass and Large Coercive Field Observed in a Spin Integer Dimerized Honeycomb Lattice (Adv. Funct. Mater. 1/2021)

Large-scale Dynamical Simulations of the Three-Dimensionalxyspin Glass

Spin‐Lattice Coupled Metamagnetism in Frustrated van der Waals Magnet CrOCl (Small 33/2023)

Chemical Design of Spin Frustration to Realize Topological Spin Glasses

Realization of a Spin Glass in a two-dimensional van der Waals material

Topological magnetic-spin textures in two-dimensional van der Waals Cr2Ge2Te6

Intrinsic 2D-XY Ferromagnetism in a Van Der Waals Monolayer

Spiral-spin-liquid behaviors and persistent reciprocal kagomé structure in frustrated van der Waals magnets and beyond

Intrinsic Van Der Waals Magnetic Materials from Bulk to the 2D Limit: New Frontiers of Spintronics

Partial Geometric Frustration in Inorganic Supramolecular Spin Systems with One-Dimensional Trigonally Aligned Magnetic Chains ∞1(mcl4)2− (M = Fe2+, Co2+)

Glassy Spin Dynamics in Geometrically Frustrated Buckled Colloidal Crystals

Frustrated magnets in the limit of infinite dimensions: dynamics and disorder-free glass transition

High-temperature ferromagnetism and ferroelasticity in ultraflexible atomically thin square-shaped lattices

Magnetic Frustration Driven by Itinerancy in Spinel CoV2O4

Frustrated honeycomb-lattice bilayer quantum antiferromagnet in a magnetic field: Unconventional phase transitions in a two-dimensional isotropic Heisenberg model

Proximate Quantum Spin Liquid on Designer Lattice

Designing topological and correlated 2D magnetic states via superatomic lattice constructions of zirconium dichloride

Quantum mechanically driven structural-spin glass in two dimensions at finite temperature

Dimensionality crossover to 2D vestigial nematicity from 3D zigzag antiferromagnetism in an XY-type honeycomb van der Waals magnet

Dirac magnons in a honeycomb lattice quantum XY magnet CoTiO3

Topological frustration of artificial spin ice