Spectral 3D mesh segmentation with a novel single segmentation field

WangHao,LuTong,AuOscar Kin-Chung,TaiChiew-Lan

2014-01-01

Abstract:We present an automatic mesh segmentation framework that achieves 3D segmentation in two stages, hierarchical spectral analysis and isoline-based boundary detection. During the hierarchical spectra...

What problem does this paper attempt to address?

Spectral 3D Mesh Segmentation with a Novel Single Segmentation Field.

Hao Wang,Tong Lu,Oscar Kin-Chung Au,Chiew-Lan Tai

DOI: https://doi.org/10.1016/j.gmod.2014.04.009

IF: 1.094

2014-01-01

Graphical Models

Abstract:We present an automatic mesh segmentation framework that achieves 3D segmentation in two stages, hierarchical spectral analysis and isoline-based boundary detection. During the hierarchical spectral analysis stage, a novel segmentation field is defined to capture a concavity-aware decomposition of eigenvectors from a concavity-aware Laplacian. Specifically, a sufficient number of eigenvectors is first adaptively selected and simultaneously partitioned into sub-eigenvectors through spectral clustering. Next, on the sub-eigenvectors level, we evaluate the confidence of identifying a spectral-sensitive mesh boundary for each sub-eigenvector by two joint measures, namely, inner variations and part oscillations. The selection and combination of sub-eigenvectors are thereby formulated as an optimization problem to generate a single segmentation field. In the isoline-based boundary detection stage, the segmentation boundaries are recognized by a divide-merge algorithm and a cut score, which respectively filters and measures desirable isolines from the concise single segmentation field. Experimental results on the Princeton Segmentation Benchmark and a number of other complex meshes demonstrate the effectiveness of the proposed method, which is comparable to recent state-of-the-art algorithms.
3D mesh segmentation using a two-stage merging strategy

Xiang Pan,Xiuzi Ye,Sanyuan Zhang

DOI: https://doi.org/10.1109/CIT.2004.1357281

2004-01-01

Abstract:3D mesh segmentation has been widely used in fields such as computer vision. Existing algorithms have the problems of over- and under-merging when performing 3D mesh segmentation. In this paper, we propose a robust and efficient segmentation algorithm using a so-called two-stage merging strategy. We first define a flatness measure for mesh faces, and partition the given mesh model into different patches based on the flatness measure. In order to handle the over- and under-merging problems, we first perform a local merging operation and then a global merging operation. Experiments show that our algorithm is effective, and works well even for models without obvious part boundaries at their joints.
Mesh Segmentation with Concavity-Aware Fields

Oscar Kin-Chung Au,Youyi Zheng,Menglin Chen,Pengfei Xu,Chiew-Lan Tai

DOI: https://doi.org/10.1109/TVCG.2011.131

IF: 5.2

2012-01-01

IEEE Transactions on Visualization and Computer Graphics

Abstract:This paper presents a simple and efficient automatic mesh segmentation algorithm that solely exploits the shape concavity information. The method locates concave creases and seams using a set of concavity-sensitive scalar fields. These fields are computed by solving a Laplacian system with a novel concavity-sensitive weighting scheme. Isolines sampled from the concavity-aware fields naturally gather at concave seams, serving as good cutting boundary candidates. In addition, the fields provide sufficient information allowing efficient evaluation of the candidate cuts. We perform a summarization of all field gradient magnitudes to define a score for each isoline and employ a score-based greedy algorithm to select the best cuts. Extensive experiments and quantitative analysis have shown that the quality of our segmentations are better than or comparable with existing state-of-the-art more complex approaches.
A Spectral Segmentation Method for Large Meshes

Xiaohan Bao,Weihua Tong,Falai Chen

DOI: https://doi.org/10.1007/s40304-021-00265-4

2022-01-01

Communications in Mathematics and Statistics

Abstract:Mesh segmentation is a fundamental and critical task in mesh processing, and it has been studied extensively in computer graphics and geometric modeling communities. However, current methods are not well suited for segmenting large meshes which are now common in many applications. This paper proposes a new spectral segmentation method specifically designed for large meshes inspired by multi-resolution representations. Building on edge collapse operators and progressive mesh representations, we first devise a feature-aware simplification algorithm that can generate a coarse mesh which keeps the same topology as the input mesh and preserves as many features of the input mesh as possible. Then, using the spectral segmentation method proposed in Tong et al. (IEEE Trans Vis Comput Graph 26(4):1807–1820, 2020), we perform partition on the coarse mesh to obtain a coarse segmentation which mimics closely the desired segmentation of the input mesh. By reversing the simplification process through vertex split operators, we present a fast algorithm which maps the coarse segmentation to the input mesh and therefore obtain an initial segmentation of the input mesh. Finally, to smooth some jaggy boundaries between adjacent parts of the initial segmentation or align with the desired boundaries, we propose an efficient method to evolve those boundaries driven by geodesic curvature flows. As demonstrated by experimental results on a variety of large meshes, our method outperforms the state-of-the-art segmentation method in terms of not only speed but also usability.
A novel segmentation algorithm for complex 3D mesh model in computer vision.

Zixian Zhang,Yixiong Feng,Ichiro Hagiwara

DOI: https://doi.org/10.1109/BICTA.2010.5645240

2010-01-01

Abstract:Segmentation is a critical and necessary procedure for computer vision, texture mapping and reverse engineering, which aims to digitally partition scanned point cloud or polygon mesh into subset which belongs to some kind of algebraic or NURBS surface. In this paper, a novel segmentation algorithm on triangulated boundary mesh is proposed based on boundary extraction and feature identification to extract algebraic subset, facilitating subsequent CAD model reconstruction to a great extent. Normal vector and principle curvatures are estimated firstly to identify sharp edge and strips with large curvature (fillet), which can provide a fundamental partition, allowing an accurate identification of patch features, utilizing least-square fitting and statistical error distribution. The outstanding merit of proposed method lies on the accurately hierarchical boundary identification by sharp edge and fillet independently, followed by convenient feature extraction and rather smooth subset boundary. Some complex scanned models are processed on a PC with 2.7GHZ CPU and IG memory, with acceptable running time and satisfactory result, demonstrating the implementation and reliability of proposed method. © 2010 IEEE.
User-controllable Mesh Segmentation Using Shape Harmonic Signature

Yongwei Miao,Jieqing Feng,Jinrong Wang,Xiaogang Jin

DOI: https://doi.org/10.1016/j.pnsc.2008.06.024

2009-01-01

Abstract:Due to different shape modeling applications, partitioning a given complex 3D mesh model into some patches or meaningful subparts is one of the fundamental problems in digital geometry processing. By using the high-dimensional mean-shift clustering scheme in shape signature space, a new method is proposed which can generate user-specified segmentation results automatically for different applications. The shape signature is composed of mesh geometric attributes and its spectral harmonics. The latter one can reflect mesh frequency spectrum information. The low frequency components are essential for semantics-oriented segmentation, while the high frequency components are important for purely geometry-oriented segmentation. The effects of the proposed method are demonstrated by several examples.
Dot Scissor: A Single-Click Interface for Mesh Segmentation.

Youyi Zheng,Chiew-Lan Tai,Oscar Kin-Chung Au

DOI: https://doi.org/10.1109/tvcg.2011.140

IF: 5.2

2011-01-01

IEEE Transactions on Visualization and Computer Graphics

Abstract:This paper presents a very easy-to-use interactive tool, which we call dot scissor, for mesh segmentation. The user's effort is reduced to placing only a single click where a cut is desired. Such a simple interface is made possible by a directional search strategy supported by a concavity-aware harmonic field and a robust voting scheme that selects the best isoline as the cut. With a concavity-aware weighting scheme, the harmonic fields gather dense isolines along concave regions which are natural boundaries of semantic components. The voting scheme relies on an isoline-face scoring mechanism that considers both shape geometry and user intent. We show by extensive experiments and quantitative analysis that our tool advances the state-of-the-art segmentation methods in both simplicity of use and segmentation quality.
Spectral Mesh Segmentation Via L0 Gradient Minimization.

Weihua Tong,Xiankang Yang,Maodong Pan,Falai Chen

DOI: https://doi.org/10.1109/tvcg.2018.2882212

IF: 5.2

2019-01-01

IEEE Transactions on Visualization and Computer Graphics

Abstract:Mesh segmentation is a process of partitioning a mesh model into meaningful parts - a fundamental problem in various disciplines. This paper introduces a novel mesh segmentation method inspired by sparsity pursuit. Based on the local geometric and topological information of a given mesh, we build a Laplacian matrix whose Fiedler vector is used to characterize the uniformity among elements of the same segment. By analyzing the Fiedler vector, we reformulate the mesh segmentation problem as a l0 gradient minimization problem. To solve this problem efficiently, we adopt a coarse-to-fine strategy. A fast heuristic algorithm is first devised to find a rational coarse segmentation, and then an optimization algorithm based on the alternating direction method of multiplier (ADMM) is proposed to refine the segment boundaries within their local regions. To extract the inherent hierarchical structure of the given mesh, our method performs segmentation in a recursive way. Experimental results demonstrate that the presented method outperforms the state-of-the-art segmentation methods when evaluated on the Princeton Segmentation Benchmark, the LIFL/LIRIS Segmentation Benchmark and a number of other complex meshes.
3D Image Segmentation Based on Feature-Sensitive and Adaptive Tetrahedral Meshes.

Ming Xu,Zeyun Yu

DOI: https://doi.org/10.1109/icip.2016.7532478

2016-01-01

Abstract:In this paper, a tetrahedral mesh-based approach is investigated for 3D image segmentation on a given image volume. We present a series of algorithms to generate high quality, feature-sensitive, and adaptive meshes to partition a 3D volume, where the 3D Canny edge detector is utilized to preserve important feature boundaries in the generated tetrahedral mesh. Each cluster of voxels within a tetrahedron is called a supervoxel, which is treated as a node in a graph and weighted edges are formed between adjacent tetrahedra (or supervoxels). A graph cut method is then applied on the weighted graph to partition the tetrahedral mesh, resulting in a segmentation of the original image volume. Thanks to the significantly reduced number of elements (supervoxels) as compared to that of voxels in a 3D image volume, the supervoxel-based segmentation method has tremendously improved the segmentation speed. In addition, the segmented image feature is naturally composed of high-quality, adaptive tetrahedral mesh that can be readily used for finite element analysis without further post-generation of tetrahedral mesh.
MeshSegmenter: Zero-Shot Mesh Semantic Segmentation via Texture Synthesis

Ziming Zhong,Yanxu Xu,Jing Li,Jiale Xu,Zhengxin Li,Chaohui Yu,Shenghua Gao

2024-07-25

Abstract:We present MeshSegmenter, a simple yet effective framework designed for zero-shot 3D semantic segmentation. This model successfully extends the powerful capabilities of 2D segmentation models to 3D meshes, delivering accurate 3D segmentation across diverse meshes and segment descriptions. Specifically, our model leverages the Segment Anything Model (SAM) model to segment the target regions from images rendered from the 3D shape. In light of the importance of the texture for segmentation, we also leverage the pretrained stable diffusion model to generate images with textures from 3D shape, and leverage SAM to segment the target regions from images with textures. Textures supplement the shape for segmentation and facilitate accurate 3D segmentation even in geometrically non-prominent areas, such as segmenting a car door within a car mesh. To achieve the 3D segments, we render 2D images from different views and conduct segmentation for both textured and untextured images. Lastly, we develop a multi-view revoting scheme that integrates 2D segmentation results and confidence scores from various views onto the 3D mesh, ensuring the 3D consistency of segmentation results and eliminating inaccuracies from specific perspectives. Through these innovations, MeshSegmenter offers stable and reliable 3D segmentation results both quantitatively and qualitatively, highlighting its potential as a transformative tool in the field of 3D zero-shot segmentation. The code is available at \url{<a class="link-external link-https" href="https://github.com/zimingzhong/MeshSegmenter" rel="external noopener nofollow">this https URL</a>}.

Computer Vision and Pattern Recognition
Spatio-Temporal Video Segmentation of Static Scenes and Its Applications

Hanqing Jiang,Guofeng Zhang,Huiyan Wang,Hujun Bao

DOI: https://doi.org/10.1109/tmm.2014.2368273

IF: 7.3

2015-01-01

IEEE Transactions on Multimedia

Abstract:Extracting spatio-temporally consistent segments from a video sequence is a challenging problem due to the complexity of color, motion and occlusions. Most existing spatio-temporal segmentation approaches have inherent difficulties in handling large displacement with significant occlusions. This paper presents a novel framework for spatio-temporal segmentation. With the estimated depth data beforehand by a multi-view stereo technique, we project the pixels to other frames for collecting the boundary and segmentation statistics in a video, and incorporate them into the segmentation energy for spatio-temporal optimization. In order to effectively solve this problem, we introduce an iterative optimization scheme by first initializing segmentation maps for each frame independently, and then link the correspondences among different frames and iteratively refine them with the collected statistics, so that a set of spatio-temporally consistent volume segments are finally achieved. The effectiveness and usefulness of our automatic framework are demonstrated via its applications for 3D reconstruction, video editing and semantic segmentation on a variety of challenging video examples.
3D Object Segmentation Using Cross-Window Point Transformer with Latent Semantic Boundary Guidance

Qide Wang,Daxin Liu,Zhenyu Liu,Jiatong Xu,Jianrong Tan

DOI: https://doi.org/10.1109/tmm.2023.3342697

IF: 7.3

2024-01-01

IEEE Transactions on Multimedia

Abstract:Accurate 3D object segmentation in point clouds is a basis for industrial robot applications, such as robot manipulation and digital twin, which require an understanding of the 3D environment. However, the unstructured and disordered nature of point clouds makes it challenging, especially for the incomplete 3D data under a single view in the real-world scenario. To this end, this paper proposes a novel 3D object segmentation framework (3DT-Seg) based on Cross-Window Point Transformer (CP-Former). CP-Former captures the long-range dependencies between local windows and latent semantic boundaries to enhance the point-wise features extracted from irregular point clouds via a bidirectional cross-attention mechanism. In addition, a contrastive learning loss and an adaptive dual aggregation strategy are introduced on semantic transition regions during the semantic supervising and instance clustering process, respectively. In this way, the latent boundary information is further utilized to improve the overall segmentation performance. Experiments on the popular benchmark (SI3DS) dataset show the state-of-the-art performance of the proposed approach in terms of semantic and instance segmentation. Furthermore, a real-world point cloud dataset (IP-Cloud) for the robotic grasping task is presented to fully validate the effectiveness of our method in practice, where it also achieves remarkable performance.
Robust Mesh Segmentation Using Feature-Aware Region Fusion

Lulu Wu,Yu Hou,Junli Xu,Yong Zhao

DOI: https://doi.org/10.3390/s23010416

IF: 3.9

2022-12-31

Sensors

Abstract:This paper introduces a simple but powerful segmentation algorithm for 3D meshes. Our algorithm consists of two stages: over-segmentation and region fusion. In the first stage, adaptive space partition is applied to perform over-segmentation, which is very efficient. In the second stage, we define a new intra-region difference, inter-region difference, and fusion condition with the help of various shape features and propose an iterative region fusion method. As the region fusion process is feature aware, our algorithm can deal with complex 3D meshes robustly. Massive qualitative and quantitative experiments also validate the advantages of the proposed algorithm.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Low-rank 3D mesh segmentation and labeling with structure guiding

LiuXiuping,ZhangJie,LiuRisheng,WangJun,CaoJunjie

IF: 1.821

2015-01-01

Computers & Graphics

Abstract:Semantic mesh segmentation and labeling is a fundamental problem in graphics. Conventional data-driven approaches usually employ a tedious offline pre-training process. Moreover, the number and esp...
Mesh Segmentation by Local Depth

Jinjie Lin,Yubin Yang,Tong Lu,Guangnan He,Jiabin Ruan

DOI: https://doi.org/10.1109/iccms.2010.288

2010-01-01

Abstract:In this paper, we propose a novel segmentation algorithm based on the local depth, a new measure for capturing the geometrical feature of 3D boundary points. Compared to the curvature feature, the local depth has two advantages: 1) The local depth is in itself defined on a discrete space, while the curvature is defined on a smooth space. Thus, the local depth is more suitable for capturing the geometrical feature of partial boundary on unsmooth 3D surface. 2) The local depth considers a wider range of geometry information than the curvature does, hence it avoids discovering some false boundary, and is able to find more boundaries that the curvature. Based on the local depth, we then introduce the Fast Marching Watersheds algorithm to segment a mesh into representative regions.
Saliency Regions for 3D Mesh Abstraction

Yu-Bin Yang,Tong Lu,Jin-Jie Lin

DOI: https://doi.org/10.1007/978-3-642-10467-1_25

2009-01-01

Abstract:This paper proposes a 3D mesh abstraction framework that is capable of segmenting a 3D mesh into specific parts called "saliency regions" based on a saliency computation process, by which the intrinsic pattern of the 3D mesh is promisingly able to be identified. It is then potential to extract effective feature representations from saliency regions for the purpose of 3D mesh retrieval. The process of searching saliency regions is a combination of vertex saliency computation and face saliency computation. Afterwards, a simple selection strategy is adopted to determine the final partition of saliency regions. Experimental results are also provided to show the effectiveness of the proposed framework.
Feature Sensitive Mesh Segmentation

Yu-Kun Lai,Qian-Yi Zhou,Shi-Min Hu,Ralph R. Martin

DOI: https://doi.org/10.1145/1128888.1128891

2006-01-01

Abstract:Segmenting meshes into natural regions is useful for model understanding and many practical applications. In this paper, we present a novel, automatic algorithm for segmenting meshes into meaningful pieces. Our approach is a clustering-based top-down hierarchical segmentation algorithm. We extend recent work on feature sensitive isotropic remeshing to generate a mesh hierarchy especially suitable for segmentation of large models with regions at multiple scales. Using integral invariants for estimation of local characteristics, our method is robust and efficient. Moreover, statistical quantities can be incorporated, allowing our approach to segment regions with different geometric characteristics or textures.
OmniSeg3D: Omniversal 3D Segmentation via Hierarchical Contrastive Learning

Haiyang Ying,Yixuan Yin,Jinzhi Zhang,Fan Wang,Tao Yu,Ruqi Huang,Lu Fang

2023-11-20

Abstract:Towards holistic understanding of 3D scenes, a general 3D segmentation method is needed that can segment diverse objects without restrictions on object quantity or categories, while also reflecting the inherent hierarchical structure. To achieve this, we propose OmniSeg3D, an omniversal segmentation method aims for segmenting anything in 3D all at once. The key insight is to lift multi-view inconsistent 2D segmentations into a consistent 3D feature field through a hierarchical contrastive learning framework, which is accomplished by two steps. Firstly, we design a novel hierarchical representation based on category-agnostic 2D segmentations to model the multi-level relationship among pixels. Secondly, image features rendered from the 3D feature field are clustered at different levels, which can be further drawn closer or pushed apart according to the hierarchical relationship between different levels. In tackling the challenges posed by inconsistent 2D segmentations, this framework yields a global consistent 3D feature field, which further enables hierarchical segmentation, multi-object selection, and global discretization. Extensive experiments demonstrate the effectiveness of our method on high-quality 3D segmentation and accurate hierarchical structure understanding. A graphical user interface further facilitates flexible interaction for omniversal 3D segmentation.

Computer Vision and Pattern Recognition
SAM3D: Segment Anything in 3D Scenes

Yunhan Yang,Xiaoyang Wu,Tong He,Hengshuang Zhao,Xihui Liu

2023-06-07

Abstract:In this work, we propose SAM3D, a novel framework that is able to predict masks in 3D point clouds by leveraging the Segment-Anything Model (SAM) in RGB images without further training or finetuning. For a point cloud of a 3D scene with posed RGB images, we first predict segmentation masks of RGB images with SAM, and then project the 2D masks into the 3D points. Later, we merge the 3D masks iteratively with a bottom-up merging approach. At each step, we merge the point cloud masks of two adjacent frames with the bidirectional merging approach. In this way, the 3D masks predicted from different frames are gradually merged into the 3D masks of the whole 3D scene. Finally, we can optionally ensemble the result from our SAM3D with the over-segmentation results based on the geometric information of the 3D scenes. Our approach is experimented with ScanNet dataset and qualitative results demonstrate that our SAM3D achieves reasonable and fine-grained 3D segmentation results without any training or finetuning of SAM.

Computer Vision and Pattern Recognition
Open-set Hierarchical Semantic Segmentation for 3D Scene

Diwen Wan,Jiaxiang Tang,Jingbo Wang,Xiaokang Chen,Lingyun Gan,Gang Zeng

DOI: https://doi.org/10.1109/icme57554.2024.10687661

2024-01-01

Abstract:The Segment-Anything Model (SAM) shows exceptional zero-shot capabilities for 2D images. Developing a similar model for 3D, however, is challenging due to limited datasets. In this paper, we introduce a zero-shot algorithm to segment a 3D scene into elements at various levels of detail, and further organize the results in a hierarchical tree structure. We propose a tree quality metric to evaluate the algorithm’s performance. Notably, our algorithm eliminates the need for 3D annotations. It uses robust 2D models to generate a 2D segmentation tree for each rendered image. Then, using graph neural networks, it aggregates these 2D trees to form a unified 3D segmentation tree. Extensive experiments on the PartNet dataset and complex 3D scenes validate the algorithm’s effectiveness. We release the source code at https://github.com/dnvtmf/OTS.

Spectral 3D mesh segmentation with a novel single segmentation field

Spectral 3D Mesh Segmentation with a Novel Single Segmentation Field.

3D mesh segmentation using a two-stage merging strategy

Mesh Segmentation with Concavity-Aware Fields

A Spectral Segmentation Method for Large Meshes

A novel segmentation algorithm for complex 3D mesh model in computer vision.

User-controllable Mesh Segmentation Using Shape Harmonic Signature

Dot Scissor: A Single-Click Interface for Mesh Segmentation.

Spectral Mesh Segmentation Via L0 Gradient Minimization.

3D Image Segmentation Based on Feature-Sensitive and Adaptive Tetrahedral Meshes.

MeshSegmenter: Zero-Shot Mesh Semantic Segmentation via Texture Synthesis

Spatio-Temporal Video Segmentation of Static Scenes and Its Applications

3D Object Segmentation Using Cross-Window Point Transformer with Latent Semantic Boundary Guidance

Robust Mesh Segmentation Using Feature-Aware Region Fusion

Low-rank 3D mesh segmentation and labeling with structure guiding

Mesh Segmentation by Local Depth

Saliency Regions for 3D Mesh Abstraction

Feature Sensitive Mesh Segmentation

OmniSeg3D: Omniversal 3D Segmentation via Hierarchical Contrastive Learning

SAM3D: Segment Anything in 3D Scenes

Open-set Hierarchical Semantic Segmentation for 3D Scene