Abstract:Due to the inherent limitations of a single viewpoint, reconstructing 3D human meshes from a single image has long been a challenging task. While deep learning networks enable us to approximate the shape of unseen sides, capturing the texture details of the non-visible side remains difficult with just one image. Traditional methods utilize Generative Adversarial Networks (GANs) to predict the normal maps of the non-visible side, thereby inferring detailed textures and wrinkles on the model's surface. However, we have identified challenges with existing normal prediction networks when dealing with complex scenes, such as a lack of focus on local features and insufficient modeling of spatial relationships.To address these challenges, we introduce EMAR—Enhanced Multi-scale Attention-Driven Single-Image 3D Human Reconstruction. This approach incorporates a novel Enhanced Multi-Scale Attention (EMSA) mechanism, which excels at capturing intricate features and global relationships in complex scenes. EMSA surpasses traditional single-scale attention mechanisms by adaptively adjusting the weights between features, enabling the network to more effectively leverage information across various scales. Furthermore, we have improved the feature fusion method to better integrate representations from different scales. This enhanced feature fusion allows the network to more comprehensively understand both fine details and global structures within the image. Finally, we have designed a hybrid loss function tailored to the introduced attention mechanism and feature fusion method, optimizing the network's training process and enhancing the quality of reconstruction results. Our network demonstrates significant improvements in performance for 3D human model reconstruction. Experimental results show that our method exhibits greater robustness to challenging poses compared to traditional single-scale approaches.

Human attention‐inspired volume reconstruction method on serial section electron microscopy images

Fast and Lightweight Network Improves Serial Brain Section Stitching

Skeleton-based image registration of serial electron microscopy sections.

A novel registration method for long-serial section images of EM with a serial split technique based on unsupervised optical flow network

Neuronal Morphological Model-Driven Image Registration for Serial Electron Microscopy Sections

UTR: Unsupervised Learning of Thickness-Insensitive Representations for Electron Microscope Image

Semi-Automatic Neuron Tracing in Electron Microscope Images: Preliminary Results

3D Reconstruction of Synapses with Deep Learning Based on EM Images

Morphology-Retained Non-Linear Image Registration of Serial Electron Microscopy Sections.

Microscopic neural image registration based on the structure of mitochondria

Fast and Accurate Electron Microscopy Image Registration with 3D Convolution

High-Speed Automated Reconstruction of Drosophila Larval Brain from Volumetric EM Data

Dual Networks for High-Precision and High-Speed Registration of Brain Electron Microscopy Images

Correction of Image Distortion in Large-Field Ssem Stitching by an Unsupervised Intermediate-Space Solving Network.

Registering large volume serial-section electron microscopy image sets for neural circuit reconstruction using FFT signal whitening

Automatically Segmenting and Reconstructing Neurons in SEM Images

A modular hierarchical approach to 3D electron microscopy image segmentation.

DEEP CONTEXTUAL RESIDUAL NETWORK FOR ELECTRON MICROSCOPY IMAGE SEGMENTATION IN CONNECTOMICS

Petascale pipeline for precise alignment of images from serial section electron microscopy

The 3D Instance Segmentation Network for Synapse Reconstruction from Serial Electron Microscopy Images

Enhanced Multi-Scale Attention-Driven 3D Human Reconstruction from Single Image