$\mathbb Z_2$-Nontrivial Moir\'e Minibands and Interaction-Driven Quantum Anomalous Hall Insulators in Topological Insulator Based Moir\'e Heterostructures

Kaijie Yang,Zian Xu,Yanjie Feng,Frank Schindler,Yuanfeng Xu,Zhen Bi,B. Andrei Bernevig,Peizhe Tang,Chao-Xing Liu

2023-01-01

Abstract: We studied electronic band structure and topological property of a topological insulator thin film under a moir\'e superlattice potential to search for two-dimensional (2D) $\mathbb Z_2$ non-trivial isolated mini-bands. To model this system, we assume the Fermi energy inside the bulk band gap and thus consider an effective model Hamiltonian with only two surface states that are located at the top and bottom surfaces and strongly hybridized with each other. The moir\'e potential is generated by another layer of 2D insulating materials on top of topological insulator films. In this model, the lowest conduction (highest valence) mini-bands can be $\mathbb Z_2$ non-trivial when the minima (maxima) of the moir\'e potential approximately forms a hexagonal lattice with six-fold rotation symmetry. For the nontrivial conduction mini-band cases, the two lowest Kramers' pairs of conduction mini-bands both have nontrivial $\mathbb Z_2$ invariant in presence of inversion, while applying external gate voltages to break inversion leads to only the lowest Kramers' pair of mini-bands to be topologically non-trivial. The Coulomb interaction can drive the lowest conduction Kramers' mini-bands into the quantum anomalous Hall state when they are half-filled, which is further stabilized by breaking inversion symmetry. We propose the monolayer Sb$_{2}$ on top of Sb$_2$Te$_3$ thin films to realize our model based on results from the first principles calculations.