Robust shape normalization of 3D articulated volumetric models

WangChao,LiuYu-Shen,LiuMin,YongJun-Hai,PaulJean-Claude

IF: 3.652

2012-01-01

Computer-Aided Design

Abstract:3D shape normalization is a common task in various computer graphics and pattern recognition applications. It aims to normalize different objects into a canonical coordinate frame with respect to r...

What problem does this paper attempt to address?

Robust Shape Normalization of 3D Articulated Volumetric Models

Chao Wang,Yu-Shen Liu,Min Liu,Jun-Hai Yong,Jean-Claude Paul

DOI: https://doi.org/10.1016/j.cad.2012.07.006

2012-01-01

Abstract:3D shape normalization is a common task in various computer graphics and pattern recognition applications. It aims to normalize different objects into a canonical coordinate frame with respect to rigid transformations containing translation, rotation and scaling in order to guarantee a unique representation. However, the conventional normalization approaches do not perform well when dealing with 3D articulated objects. To address this issue, we introduce a new method for normalizing a 3D articulated object in the volumetric form. We use techniques from robust statistics to guide the classical normalization computation. The key idea is to estimate the initial normalization by using implicit shape representation, which produces a novel articulation insensitive weight function to reduce the influence of articulated deformation. We also propose and prove the articulation insensitivity of implicit shape representation. The final solution is found by means of iteratively reweighted least squares. Our method is robust to articulated deformation without any explicit shape decomposition. The experimental results and some applications are presented for demonstrating the effectiveness of our method.
Robust Shape Normalization Based on Implicit Representations

Tingting Jiang,Carlo Tomasi

DOI: https://doi.org/10.1109/icpr.2008.4761202

2008-01-01

Abstract:We introduce a new shape normalization method based on implicit shape representations. The proposed method is robust with respect to deformations and invariant to similarity transformations (translation, isotropic scaling and rotation). The new method has been tested and compared to the classical shape normalization method and previous work in terms of aligning groups of shapes with deformations.
Radius-normal Histogram and Hybrid Strategy for 3D Shape Retrieval

X Pan,Y Zhang,SY Zhang,XZ Ye

DOI: https://doi.org/10.1109/smi.2005.36

2008-01-01

Journal of Image and Graphics

Abstract:Recent development in computer hardware and modeling technologies has led to a fast increasing of 3D models. To help user find expected models accurately and efficiently, technologies on content-based 3D model retrievals have become one of the most challenging research topic recently. In this paper, radius-normal histogram(RNH) is proposed to describe shape contents and used for shape retrievals. The RNH shape descriptor first uses a series of concentric spheres to capture the point distribution information of the given model. Then for points in each concentric sphere, a radian normal angle is computed to extract the local geometry features. Finally, the radius-normal histogram is constructed by using the extracted shape signatures. The proposed shape representation remains invariant under rotations. It can be generated from the given 3D model efficiently and easily as well. Performance comparisons for the shape benchmark database have proven that the proposed algorithm can achieve better retrieving performance than other similar histogram-based shape representations. To further improve the retrieving performance, different shape features are combined based on dynamic weight selections and a hierarchical architecture is used to speed up the matching process.
A Comparative Study of Three Shape Normalization Algorithms

Wen-Liang Zhang,Tao Zhang,Jing-Yan Song

DOI: https://doi.org/10.1109/icwapr.2007.4421635

2007-01-01

Abstract:Shape is one of the fundamental visual features in pattern recognition and target tracking. And shape normalization is a very important pre-processing step in image understanding. In general, there are four basic forms of planar shape distortions caused by changes in viewer's location: translation, rotation, scaling and skewing. A good shape descriptor should be invariant to these distortions. In this paper, we study and compare three shape normalization algorithms: J.G. Leu's shape compacting algorithm and its two modified versions: Wang's image ellipse algorithm and Liang J.J.'s principal axis algorithm. Experiments on a set of images show that all of these algorithms have some drawbacks and we give some advices for modification.
A new 3D shape descriptor based on rotation

Pan Jeng-Shyang,Luo Hao,Lu Zhe-Ming,Chang Jung-Chou Harry

DOI: https://doi.org/10.1109/ISDA.2006.253850

2006-01-01

Abstract:Recent developments in techniques for modeling, digitizing and visualizing 3D shapes have led to an explosion in the number of available 3D models on the Internet and in domain-specific databases. However, how to describe 3D shape is a difficult problem. A new 3D shape description method based on rotation is proposed in this paper. The first step is to sample points on the surface and compute normal vectors. Secondly, we do Gauss mapping and rotate model, at the same time, rotate the Gauss sphere. The third step is to count normal distributions on the sphere surface. Finally, the Euclidean distance is computed, thus a statistic histogram is constructed Experimental results show the effectiveness of this method.
Detail-Preserving Shape Unfolding

Bin Liu,Weiming Wang,Jun Zhou,Bo Li,Xiuping Liu

DOI: https://doi.org/10.3390/s21041187

IF: 3.9

2021-02-08

Sensors

Abstract:Canonical extrinsic representations for non-rigid shapes with different poses are preferable in many computer graphics applications, such as shape correspondence and retrieval. The main reason for this is that they give a pose invariant signature for those jobs, which significantly decreases the difficulty caused by various poses. Existing methods based on multidimentional scaling (MDS) always result in significant geometric distortions. In this paper, we present a novel shape unfolding algorithm, which deforms any given 3D shape into a canonical pose that is invariant to non-rigid transformations. The proposed method can effectively preserve the local structure of a given 3D model with the regularization of local rigid transform energy based on the shape deformation technique, and largely reduce geometric distortion. Our algorithm is quite simple and only needs to solve two linear systems during alternate iteration processes. The computational efficiency of our method can be improved with parallel computation and the robustness is guaranteed with a cascade strategy. Experimental results demonstrate the enhanced efficacy of our algorithm compared with the state-of-the-art methods on 3D shape unfolding.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
3D Body Shapes Estimation from Dressed-Human Silhouettes

SongDan,TongRuofeng,ChangJian,YangXiaosong,TangMin,ZhangJian Jun

IF: 2.5

2016-01-01

Computer Graphics Forum

Abstract:Estimation of 3D body shapes from dressed-human photos is an important but challenging problem in virtual fitting. We propose a novel automatic framework to efficiently estimate 3D body shapes unde...
Non-rigid 3D Shape Retrieval Based on Automatic Anti-articulating

WANG Xulei,ZHA Hongbin

DOI: https://doi.org/10.13209/j.0479-8023.2013.029

2013-01-01

Abstract:The authors propose a new non-rigid 3D shape retrieval method based on automatic anti-articulating.First,feature points are extracted on the 3D shape,and then are used in shape partition and skeleton extraction.Then,the mesh editing technique is employed to automatically regularize the articulation,in order to generate a rigid 3D shape without deformation,which is used as the isometry-invariant canonical form.Finally,shape descriptors are constructed on the canonical form,and Earth Mover’s distance is employed to measure the intrinsic similarities between non-rigid objects.The method introduces less geometric distortion and requires much less computational cost.Experimental results demonstrate a significant performance improvement in retrieving non-rigid 3D shapes.
Weakly Supervised 3D Human Pose and Shape Reconstruction with Normalizing Flows

Andrei Zanfir,Eduard Gabriel Bazavan,Hongyi Xu,Bill Freeman,Rahul Sukthankar,Cristian Sminchisescu

DOI: https://doi.org/10.48550/arXiv.2003.10350

2020-08-22

Abstract:Monocular 3D human pose and shape estimation is challenging due to the many degrees of freedom of the human body and thedifficulty to acquire training data for large-scale supervised learning in complex visual scenes. In this paper we present practical semi-supervised and self-supervised models that support training and good generalization in real-world images and video. Our formulation is based on kinematic latent normalizing flow representations and dynamics, as well as differentiable, semantic body part alignment loss functions that support self-supervised learning. In extensive experiments using 3D motion capture datasets like CMU, Human3.6M, 3DPW, or AMASS, as well as image repositories like COCO, we show that the proposed methods outperform the state of the art, supporting the practical construction of an accurate family of models based on large-scale training with diverse and incompletely labeled image and video data.

Computer Vision and Pattern Recognition
3D VR Sketch Guided 3D Shape Prototyping and Exploration

Ling Luo,Pinaki Nath Chowdhury,Tao Xiang,Yi-Zhe Song,Yulia Gryaditskaya

2024-01-10

Abstract:3D shape modeling is labor-intensive, time-consuming, and requires years of expertise. To facilitate 3D shape modeling, we propose a 3D shape generation network that takes a 3D VR sketch as a condition. We assume that sketches are created by novices without art training and aim to reconstruct geometrically realistic 3D shapes of a given category. To handle potential sketch ambiguity, our method creates multiple 3D shapes that align with the original sketch's structure. We carefully design our method, training the model step-by-step and leveraging multi-modal 3D shape representation to support training with limited training data. To guarantee the realism of generated 3D shapes we leverage the normalizing flow that models the distribution of the latent space of 3D shapes. To encourage the fidelity of the generated 3D shapes to an input sketch, we propose a dedicated loss that we deploy at different stages of the training process. The code is available at <a class="link-external link-https" href="https://github.com/Rowl1ng/3Dsketch2shape" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition
ARTIC3D: Learning Robust Articulated 3D Shapes from Noisy Web Image Collections

Chun-Han Yao,Amit Raj,Wei-Chih Hung,Yuanzhen Li,Michael Rubinstein,Ming-Hsuan Yang,Varun Jampani

DOI: https://doi.org/10.48550/arXiv.2306.04619

2023-06-08

Abstract:Estimating 3D articulated shapes like animal bodies from monocular images is inherently challenging due to the ambiguities of camera viewpoint, pose, texture, lighting, etc. We propose ARTIC3D, a self-supervised framework to reconstruct per-instance 3D shapes from a sparse image collection in-the-wild. Specifically, ARTIC3D is built upon a skeleton-based surface representation and is further guided by 2D diffusion priors from Stable Diffusion. First, we enhance the input images with occlusions/truncation via 2D diffusion to obtain cleaner mask estimates and semantic features. Second, we perform diffusion-guided 3D optimization to estimate shape and texture that are of high-fidelity and faithful to input images. We also propose a novel technique to calculate more stable image-level gradients via diffusion models compared to existing alternatives. Finally, we produce realistic animations by fine-tuning the rendered shape and texture under rigid part transformations. Extensive evaluations on multiple existing datasets as well as newly introduced noisy web image collections with occlusions and truncation demonstrate that ARTIC3D outputs are more robust to noisy images, higher quality in terms of shape and texture details, and more realistic when animated. Project page: <a class="link-external link-https" href="https://chhankyao.github.io/artic3d/" rel="external noopener nofollow">this https URL</a>

Computer Vision and Pattern Recognition
Robust Feature Classification and Editing

Yu-Kun Lai,Qian-Yi Zhou,Shi-Min Hu,Johannes Wallner,Helmut Pottmann,Yu-kun Lai,Qian-yi Zhou,Shi-min Hu

DOI: https://doi.org/10.1109/tvcg.2007.19

IF: 5.2

2007-01-01

IEEE Transactions on Visualization and Computer Graphics

Abstract:Sharp edges, ridges, valleys, and prongs are critical for the appearance and an accurate representation of a 3D model. In this paper, we propose a novel approach that deals with the global shape of features in a robust way. Based on a remeshing algorithm which delivers an isotropic mesh in a feature-sensitive metric, features are recognized on multiple scales via integral invariants of local neighborhoods. Morphological and smoothing operations are then used for feature region extraction and classification into basic types such as ridges, valleys, and prongs. The resulting representation of feature regions is further used for feature-specific editing operations.

computer science, software engineering
Robust Monocular 3D Car Shape Estimation from 2D Landmarks.

Yanan Miao,Xiaoming Tao,Jianhua Lu

DOI: https://doi.org/10.1109/tcsvt.2016.2628207

IF: 5.859

2016-01-01

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:Estimating 3D shape of an object from 2D observations in a monocular image is fundamentally an inverse problem due to the ambiguity of the projection from 3D to 2D and becomes more challenging when there are undesirable outliers in the observations. In this paper, we develop a robust model to estimate 3D shape from 2D landmarks with an unknown camera pose. The 3D shape of the object is assumed as a linear combination of a group of prior shape bases. At the same time, we explicitly model the outliers as sparse noises to handle severely contaminated observations. The objective function is nonconvex and nonsmooth constrained on Stiefel manifold, where the coupling of underdetermined shape representation coefficients and camera pose makes it more difficult to solve. We first propose a numerical algorithm based on alternating direction method of multipliers for the no-outlier case. We set the orthogonality constraints into the smooth subproblem, which admits a closed-form solution, and the other subproblems are all well known and can be easily solved. We then extend this algorithm to the proposed robust model. The proposed algorithms can achieve convergence rapidly. The experimental results on both synthetic data and real data show that the proposed method outperforms the other methods.
Robust Shape Fitting for 3D Scene Abstraction

Florian Kluger,Eric Brachmann,Michael Ying Yang,Bodo Rosenhahn

2024-03-16

Abstract:Humans perceive and construct the world as an arrangement of simple parametric models. In particular, we can often describe man-made environments using volumetric primitives such as cuboids or cylinders. Inferring these primitives is important for attaining high-level, abstract scene descriptions. Previous approaches for primitive-based abstraction estimate shape parameters directly and are only able to reproduce simple objects. In contrast, we propose a robust estimator for primitive fitting, which meaningfully abstracts complex real-world environments using cuboids. A RANSAC estimator guided by a neural network fits these primitives to a depth map. We condition the network on previously detected parts of the scene, parsing it one-by-one. To obtain cuboids from single RGB images, we additionally optimise a depth estimation CNN end-to-end. Naively minimising point-to-primitive distances leads to large or spurious cuboids occluding parts of the scene. We thus propose an improved occlusion-aware distance metric correctly handling opaque scenes. Furthermore, we present a neural network based cuboid solver which provides more parsimonious scene abstractions while also reducing inference time. The proposed algorithm does not require labour-intensive labels, such as cuboid annotations, for training. Results on the NYU Depth v2 dataset demonstrate that the proposed algorithm successfully abstracts cluttered real-world 3D scene layouts.

Computer Vision and Pattern Recognition
Unsupervised Shape and Pose Disentanglement for 3D Meshes

Keyang Zhou,Bharat Lal Bhatnagar,Gerard Pons-Moll

DOI: https://doi.org/10.48550/arXiv.2007.11341

2020-07-22

Computer Vision and Pattern Recognition

Abstract:Parametric models of humans, faces, hands and animals have been widely used for a range of tasks such as image-based reconstruction, shape correspondence estimation, and animation. Their key strength is the ability to factor surface variations into shape and pose dependent components. Learning such models requires lots of expert knowledge and hand-defined object-specific constraints, making the learning approach unscalable to novel objects. In this paper, we present a simple yet effective approach to learn disentangled shape and pose representations in an unsupervised setting. We use a combination of self-consistency and cross-consistency constraints to learn pose and shape space from registered meshes. We additionally incorporate as-rigid-as-possible deformation(ARAP) into the training loop to avoid degenerate solutions. We demonstrate the usefulness of learned representations through a number of tasks including pose transfer and shape retrieval. The experiments on datasets of 3D humans, faces, hands and animals demonstrate the generality of our approach. Code is made available at https://virtualhumans.mpi-inf.mpg.de/unsup_shape_pose/.
Discriminative Sketch-Based 3d Model Retrieval Via Robust Shape Matching

Tianjia Shao,Weiwei Xu,Kangkang Yin,Jingdong Wang,Kun Zhou,Baining Guo

DOI: https://doi.org/10.1111/j.1467-8659.2011.02050.x

IF: 2.5

2011-01-01

Computer Graphics Forum

Abstract:We propose a sketch-based 3D shape retrieval system that is substantially more discriminative and robust than existing systems, especially for complex models. The power of our system comes from a combination of a contour-based 2D shape representation and a robust sampling-based shape matching scheme. They are defined over discriminative local features and applicable for partial sketches; robust to noise and distortions in hand drawings; and consistent when strokes are added progressively. Our robust shape matching, however, requires dense sampling and registration and incurs a high computational cost. We thus devise critical acceleration methods to achieve interactive performance: precomputing kNN graphs that record transformations between neighboring contour images and enable fast online shape alignment; pruning sampling and shape registration strategically and hierarchically; and parallelizing shape matching on multi-core platforms or GPUs. We demonstrate the effectiveness of our system through various experiments, comparisons, and user studies.
Multiresolution Mean Shift Clustering Algorithm for Shape Interpolation

Hung-Kuo Chu,Tong-Yee Lee

DOI: https://doi.org/10.1109/tvcg.2009.40

IF: 5.2

2009-01-01

IEEE Transactions on Visualization and Computer Graphics

Abstract:In this paper, we solve the problem of 3D shape interpolation with significant pose variation. For an ideal 3D shape interpolation, especially the articulated model, the shape should follow the movement of the underlying articulated structure and be transformed in a way that is as rigid as possible. Given input shapes with compatible connectivity, we propose a novel multiresolution mean shift (MMS) clustering algorithm to automatically extract their near-rigid components. Then, by building the hierarchical relationship among extracted components, we compute a common articulated structure for these input shapes. With the aid of this articulated structure, we solve the shape interpolation by combining 1) a global pose interpolation of near-rigid components from the source shape to the target shape with 2) a local gradient field interpolation for each pair of components, followed by solving a Poisson equation in order to reconstruct an interpolated shape. As a result, an aesthetically pleasing shape interpolation can be generated, with even the poses of shapes varying significantly. In contrast to a recent state-of-the-art work (Kilian et al., 2007), the proposed approach can achieve comparable or even better results and have better computational efficiency as well.
Self-supervised Learning of Implicit Shape Representation with Dense Correspondence for Deformable Objects

Baowen Zhang,Jiahe Li,Xiaoming Deng,Yinda Zhang,Cuixia Ma,Hongan Wang

2023-12-25

Abstract:Learning 3D shape representation with dense correspondence for deformable objects is a fundamental problem in computer vision. Existing approaches often need additional annotations of specific semantic domain, e.g., skeleton poses for human bodies or animals, which require extra annotation effort and suffer from error accumulation, and they are limited to specific domain. In this paper, we propose a novel self-supervised approach to learn neural implicit shape representation for deformable objects, which can represent shapes with a template shape and dense correspondence in 3D. Our method does not require the priors of skeleton and skinning weight, and only requires a collection of shapes represented in signed distance fields. To handle the large deformation, we constrain the learned template shape in the same latent space with the training shapes, design a new formulation of local rigid constraint that enforces rigid transformation in local region and addresses local reflection issue, and present a new hierarchical rigid constraint to reduce the ambiguity due to the joint learning of template shape and correspondences. Extensive experiments show that our model can represent shapes with large deformations. We also show that our shape representation can support two typical applications, such as texture transfer and shape editing, with competitive performance. The code and models are available at <a class="link-external link-https" href="https://iscas3dv.github.io/deformshape" rel="external noopener nofollow">this https URL</a>

Computer Vision and Pattern Recognition
PlankAssembly: Robust 3D Reconstruction from Three Orthographic Views with Learnt Shape Programs

Wentao Hu,Jia Zheng,Zixin Zhang,Xiaojun Yuan,Jian Yin,Zihan Zhou

DOI: https://doi.org/10.48550/arXiv.2308.05744

2023-08-11

Abstract:In this paper, we develop a new method to automatically convert 2D line drawings from three orthographic views into 3D CAD models. Existing methods for this problem reconstruct 3D models by back-projecting the 2D observations into 3D space while maintaining explicit correspondence between the input and output. Such methods are sensitive to errors and noises in the input, thus often fail in practice where the input drawings created by human designers are imperfect. To overcome this difficulty, we leverage the attention mechanism in a Transformer-based sequence generation model to learn flexible mappings between the input and output. Further, we design shape programs which are suitable for generating the objects of interest to boost the reconstruction accuracy and facilitate CAD modeling applications. Experiments on a new benchmark dataset show that our method significantly outperforms existing ones when the inputs are noisy or incomplete.

Computer Vision and Pattern Recognition,Graphics

Robust shape normalization of 3D articulated volumetric models

Robust Shape Normalization of 3D Articulated Volumetric Models

Robust Shape Normalization Based on Implicit Representations

Radius-normal Histogram and Hybrid Strategy for 3D Shape Retrieval

A Comparative Study of Three Shape Normalization Algorithms

A new 3D shape descriptor based on rotation

Detail-Preserving Shape Unfolding

3D Body Shapes Estimation from Dressed-Human Silhouettes

Non-rigid 3D Shape Retrieval Based on Automatic Anti-articulating

Weakly Supervised 3D Human Pose and Shape Reconstruction with Normalizing Flows

3D VR Sketch Guided 3D Shape Prototyping and Exploration

ARTIC3D: Learning Robust Articulated 3D Shapes from Noisy Web Image Collections

Robust Feature Classification and Editing

Robust Monocular 3D Car Shape Estimation from 2D Landmarks.

Robust Shape Fitting for 3D Scene Abstraction

Unsupervised Shape and Pose Disentanglement for 3D Meshes

Discriminative Sketch-Based 3d Model Retrieval Via Robust Shape Matching

Multiresolution Mean Shift Clustering Algorithm for Shape Interpolation

Self-supervised Learning of Implicit Shape Representation with Dense Correspondence for Deformable Objects

PlankAssembly: Robust 3D Reconstruction from Three Orthographic Views with Learnt Shape Programs