Abstract:Due to the inherent limitations of a single viewpoint, reconstructing 3D human meshes from a single image has long been a challenging task. While deep learning networks enable us to approximate the shape of unseen sides, capturing the texture details of the non-visible side remains difficult with just one image. Traditional methods utilize Generative Adversarial Networks (GANs) to predict the normal maps of the non-visible side, thereby inferring detailed textures and wrinkles on the model's surface. However, we have identified challenges with existing normal prediction networks when dealing with complex scenes, such as a lack of focus on local features and insufficient modeling of spatial relationships.To address these challenges, we introduce EMAR—Enhanced Multi-scale Attention-Driven Single-Image 3D Human Reconstruction. This approach incorporates a novel Enhanced Multi-Scale Attention (EMSA) mechanism, which excels at capturing intricate features and global relationships in complex scenes. EMSA surpasses traditional single-scale attention mechanisms by adaptively adjusting the weights between features, enabling the network to more effectively leverage information across various scales. Furthermore, we have improved the feature fusion method to better integrate representations from different scales. This enhanced feature fusion allows the network to more comprehensively understand both fine details and global structures within the image. Finally, we have designed a hybrid loss function tailored to the introduced attention mechanism and feature fusion method, optimizing the network's training process and enhancing the quality of reconstruction results. Our network demonstrates significant improvements in performance for 3D human model reconstruction. Experimental results show that our method exhibits greater robustness to challenging poses compared to traditional single-scale approaches.

Feature Sharing Attention 3D Face Reconstruction with Unsupervised Learning from In-the-Wild Photo Collection

SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction

3D Face Reconstruction Based on ResNet Feature Extraction and CBAM

Self-supervised single-image 3D face reconstruction method based on attention mechanism and attribute refinement

Accurate 3D Face Reconstruction With Weakly-Supervised Learning: From Single Image to Image Set

3D Face Reconstruction From A Single Image Assisted by 2D Face Images in the Wild

3D Face Reconstruction and Semantic Annotation from Single Depth Image

Weakly-Supervised Multi-Face 3D Reconstruction

3D face-model reconstruction from a single image: A feature aggregation approach using hierarchical transformer with weak supervision

3D Face Reconstruction Using A Spectral-Based Graph Convolution Encoder

Unsupervised Style-based Explicit 3D Face Reconstruction from Single Image

3D3M: 3D Modulated Morphable Model for Monocular Face Reconstruction

Research on 3D Face Reconstruction Algorithm Based on ResNet and Transformer

Self-supervised Re-renderable Facial Albedo Reconstruction from Single Image

Enhanced Multi-Scale Attention-Driven 3D Human Reconstruction from Single Image

3D Face Reconstruction Based on A Single Image: A Review

Multi-view self-supervised learning for 3D facial texture reconstruction from single image

Dense 3D Facial Reconstruction from a Single Depth Image in Unconstrained Environment

Learning Semantic Representations via Joint 3D Face Reconstruction and Facial Attribute Estimation

FaceScape: 3D Facial Dataset and Benchmark for Single-View 3D Face Reconstruction

A Self-Supervised Bootstrap Method for Single-Image 3D Face Reconstruction