Abstract:Due to the inherent limitations of a single viewpoint, reconstructing 3D human meshes from a single image has long been a challenging task. While deep learning networks enable us to approximate the shape of unseen sides, capturing the texture details of the non-visible side remains difficult with just one image. Traditional methods utilize Generative Adversarial Networks (GANs) to predict the normal maps of the non-visible side, thereby inferring detailed textures and wrinkles on the model's surface. However, we have identified challenges with existing normal prediction networks when dealing with complex scenes, such as a lack of focus on local features and insufficient modeling of spatial relationships.To address these challenges, we introduce EMAR—Enhanced Multi-scale Attention-Driven Single-Image 3D Human Reconstruction. This approach incorporates a novel Enhanced Multi-Scale Attention (EMSA) mechanism, which excels at capturing intricate features and global relationships in complex scenes. EMSA surpasses traditional single-scale attention mechanisms by adaptively adjusting the weights between features, enabling the network to more effectively leverage information across various scales. Furthermore, we have improved the feature fusion method to better integrate representations from different scales. This enhanced feature fusion allows the network to more comprehensively understand both fine details and global structures within the image. Finally, we have designed a hybrid loss function tailored to the introduced attention mechanism and feature fusion method, optimizing the network's training process and enhancing the quality of reconstruction results. Our network demonstrates significant improvements in performance for 3D human model reconstruction. Experimental results show that our method exhibits greater robustness to challenging poses compared to traditional single-scale approaches.

MPF-GAN: an enhanced architecture for 3D face reconstruction

High-fidelity 3D Face Reconstruction with Multi-Scale Details

Advanced 3D Face Reconstruction from Single 2D Images Using Enhanced Adversarial Neural Networks and Graph Neural Networks

Beyond 3DMM Space: Towards Fine-Grained 3D Face Reconstruction

GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction

Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction

FaceLift: Single Image to 3D Head with View Generation and GS-LRM

FaceScape: 3D Facial Dataset and Benchmark for Single-View 3D Face Reconstruction

MMFace: A Multi-Metric Regression Network for Unconstrained Face Reconstruction

Texture-Shape Optimized GAT for 3D Face Reconstruction.

3D face reconstruction and dense alignment with a new generated dataset

Learning 3-D Face Shape from Diverse Sources with Cross-Domain Face Synthesis

Pixel-Face: A Large-Scale, High-Resolution Benchmark for 3D Face Reconstruction

3D3M: 3D Modulated Morphable Model for Monocular Face Reconstruction

Beyond 3DMM: Learning to Capture High-fidelity 3D Face Shape

3d Face Reconstruction With Texture Details From A Single Image Based On Gan

Transformer-Based 3D Face Reconstruction with End-to-End Shape-Preserved Domain Transfer

Extreme 3D Face Reconstruction: Seeing Through Occlusions

Enhanced Multi-Scale Attention-Driven 3D Human Reconstruction from Single Image

AvatarMe++: Facial Shape and BRDF Inference With Photorealistic Rendering-Aware GANs

CFR-GAN: A Generative Model for Craniofacial Reconstruction