TeethDreamer: 3D Teeth Reconstruction from Five Intra-oral Photographs
Chenfan Xu,Zhentao Liu,Yuan Liu,Yulong Dou,Jiamin Wu,Jiepeng Wang,Minjiao Wang,Dinggang Shen,Zhiming Cui
2024-07-16
Abstract:Orthodontic treatment usually requires regular face-to-face examinations to monitor dental conditions of the patients. When in-person diagnosis is not feasible, an alternative is to utilize five intra-oral photographs for remote dental monitoring. However, it lacks of 3D information, and how to reconstruct 3D dental models from such sparse view photographs is a challenging problem. In this study, we propose a 3D teeth reconstruction framework, named TeethDreamer, aiming to restore the shape and position of the upper and lower teeth. Given five intra-oral photographs, our approach first leverages a large diffusion model's prior knowledge to generate novel multi-view images with known poses to address sparse inputs and then reconstructs high-quality 3D teeth models by neural surface reconstruction. To ensure the 3D consistency across generated views, we integrate a 3D-aware feature attention mechanism in the reverse diffusion process. Moreover, a geometry-aware normal loss is incorporated into the teeth reconstruction process to enhance geometry accuracy. Extensive experiments demonstrate the superiority of our method over current state-of-the-arts, giving the potential to monitor orthodontic treatment remotely. Our code is available at <a class="link-external link-https" href="https://github.com/ShanghaiTech-IMPACT/TeethDreamer" rel="external noopener nofollow">this https URL</a>
Computer Vision and Pattern Recognition