Isolated mitral valve prolapse: chordal architecture as an anatomic basis in older patients.

J. M. van der Bel-Kahn,D. Duren,A. Becker

DOI: https://doi.org/10.1016/S0735-1097(85)80345-4

IF: 24

1985-06-01

Journal of the American College of Cardiology

Abstract:

What problem does this paper attempt to address?

Point Contrastive Prediction with Semantic Clustering for Self-Supervised Learning on Point Cloud Videos

Xiaoxiao Sheng,Zhiqiang Shen,Gang Xiao,Longguang Wang,Yulan Guo,Hehe Fan

DOI: https://doi.org/10.1109/iccv51070.2023.01514

2023-01-01

Abstract:We propose a unified point cloud video self-supervised learning framework for object-centric and scene-centric data. Previous methods commonly conduct representation learning at the clip or frame level and cannot well capture fine-grained semantics. Instead of contrasting the representations of clips or frames, in this paper, we propose a unified self-supervised framework by conducting contrastive learning at the point level. Moreover, we introduce a new pretext task by achieving semantic alignment of superpoints, which further facilitates the representations to capture semantic cues at multiple scales. In addition, due to the high redundancy in the temporal dimension of dynamic point clouds, directly conducting contrastive learning at the point level usually leads to massive undesired negatives and insufficient modeling of positive representations. To remedy this, we propose a selection strategy to retain proper negatives and make use of high-similarity samples from other instances as positive supplements. Extensive experiments show that our method outperforms supervised counterparts on a wide range of downstream tasks and demonstrates the superior transferability of the learned representations.
DCCN: A dual-cross contrastive neural network for 3D point cloud representation learning

Xiaopeng Wu,Guangsi Shi,Zexing Zhao,Mingjie Li,Xiaojun Gao,Xiaoli Yan

DOI: https://doi.org/10.1016/j.eswa.2024.123564

IF: 8.5

2024-03-02

Expert Systems with Applications

Abstract:The proliferation of depth cameras and LiDAR sensors in actual industrial environments has fueled the pursuit of an effective and efficient 3D point cloud model that enables us to perceive and interact with the physical world. However, the intrinsic complexity of 3D semantic information poses significant challenges to model design, including spatial rotation invariance and irregular point cloud structure, which fundamentally impact the representation and behavior of 3D point cloud systems. Existing have either heavily relied on labeling information in a supervised learning setting or failed to effectively capture the inherent patterns of the 3D point clouds within a self-supervised learning framework, leading to poor performance in specific downstream tasks. To address these limitations, this paper introduces a self-supervised framework, D ual- C ross C ontrastive Neural N etwork (DCCN) for 3D point cloud self-supervised representation learning. DCCN leverages cross-view, cross-network, and domain-specific knowledge distillation to enhance the extraction of hidden features from point clouds and fully exploit the capabilities of the encoder. Our DCCN employs a pseudo-Siamese network consisting of an online network and a target network, facilitating knowledge interaction and distillation. The method extracts internal states from augmented 3D point cloud by learning cross-view relationships and optimizes model parameters through intra-modal cross-network learning. We incorporate a momentum-updating mechanism without shared weights in the Siamese network architecture to distill knowledge and enhance the role differentiation the online and target networks. Experimental results demonstrate that our approach outperforms a range of supervised and self-supervised learning methods across a series of downstream tasks consisting of four tasks in three representative datasets. Ablation studies validate the component-wise effectiveness of cross-view, cross-network, and moment-updating learning objectives in achieving superior point cloud representation. The overall findings establish our method, DCCN, as an effective solution for 3D point cloud representation learning in real-world applications.

computer science, artificial intelligence,engineering, electrical & electronic,operations research & management science
Self-Supervised Intra-Modal and Cross-Modal Contrastive Learning for Point Cloud Understanding

Yue Wu,Jiaming Liu,Maoguo Gong,Peiran Gong,Xiaolong Fan,A. K. Qin,Qiguang Miao,Wenping Ma

DOI: https://doi.org/10.1109/tmm.2023.3284591

IF: 7.3

2023-01-01

IEEE Transactions on Multimedia

Abstract:Learning effective representations from unlabeled data is a challenging task for point cloud understanding. As the human visual system can map concepts learned from 2D images to the 3D world, and inspired by recent multimodal research, we introduce data from point cloud modality and image modality for joint learning. Based on the properties of point clouds and images, we propose CrossNet, a comprehensive intra- and cross-modal contrastive learning method that learns 3D point cloud representations. The proposed method achieves 3D-3D and 3D-2D correspondences of objectives by maximizing the consistency of point clouds and their augmented versions, and with the corresponding rendered images in invariant space. We further distinguish the rendered images into RGB and grayscale images to extract color and geometric features, respectively. These training objectives combine feature correspondences between modalities to combine rich learning signals from point clouds and images. Our CrossNet is simple: we add a feature extraction module and a projection head module to the point cloud and image branches, respectively, to train the backbone network in a self-supervised manner. After the network is pretrained, only the point cloud feature extraction module is required for fine-tuning and directly predicting results for downstream tasks. Our experiments on multiple benchmarks demonstrate improved point cloud classification and segmentation results, and the learned representations can be generalized across domains.

computer science, information systems,telecommunications, software engineering
Rethinking Local-to-global Representation Learning for Rotation-Invariant Point Cloud Analysis

Zhaoxuan Wang,Yunlong Yu,Xianzhi Li

DOI: https://doi.org/10.1016/j.patcog.2024.110624

IF: 8

2024-01-01

Pattern Recognition

Abstract:Point cloud analysis has drawn much attention in recent years, whereas most existing point-based deep networks ignore the rotation-invariant property of the encoded features, which leads to poor performance given 3D shapes with arbitrary rotation. In this paper, we propose a novel rotation-invariant method that embeds both distinctive local and global rotation-invariant information. Specifically, we design a two-branch network that separately extracts purely local and global rotation-invariant features. In the global branch, we leverage canonical transformation to extract global representations, while in the local branch, we utilize hand-crafted geometric features (e.g., relative distances and angles) to embed local representations. To fuse the features from distinct branches, we introduce an attention-based fusion module to adaptively integrate the local-to-global representation by considering the geometry contexts of each point. Particularly, different from existing rotation-invariant works, we further introduce a self-attention unit into the global branch for embedding non-local information and also insert multiple fusion modules into the local branch to emphasize the global features. Extensive experiments on standard benchmarks show that our method achieves consistent and competitive performance on various downstream tasks, and also the best performance on the shape classification task on the ModelNet40 dataset with a 0.8% accuracy gain, compared to state-of-the-art methods. The code and pre-trained models are available at https://github.com/CentauriStar/Rotation-Invariant-Point-Cloud-Analysis.
EPContrast: Effective Point-level Contrastive Learning for Large-scale Point Cloud Understanding

Zhiyi Pan,Guoqing Liu,Wei Gao,Thomas H. Li

2024-10-23

Abstract:The acquisition of inductive bias through point-level contrastive learning holds paramount significance in point cloud pre-training. However, the square growth in computational requirements with the scale of the point cloud poses a substantial impediment to the practical deployment and execution. To address this challenge, this paper proposes an Effective Point-level Contrastive Learning method for large-scale point cloud understanding dubbed \textbf{EPContrast}, which consists of AGContrast and ChannelContrast. In practice, AGContrast constructs positive and negative pairs based on asymmetric granularity embedding, while ChannelContrast imposes contrastive supervision between channel feature maps. EPContrast offers point-level contrastive loss while concurrently mitigating the computational resource burden. The efficacy of EPContrast is substantiated through comprehensive validation on S3DIS and ScanNetV2, encompassing tasks such as semantic segmentation, instance segmentation, and object detection. In addition, rich ablation experiments demonstrate remarkable bias induction capabilities under label-efficient and one-epoch training settings.

Computer Vision and Pattern Recognition
Unsupervised contrastive learning with simple transformation for 3D point cloud data

Jincen Jiang,Xuequan Lu,Wanli Ouyang,Meili Wang

DOI: https://doi.org/10.1007/s00371-023-02921-y

2023-08-02

Abstract:Though a number of point cloud learning methods have been proposed to handle unordered points, most of them are supervised and require labels for training. By contrast, unsupervised learning of point cloud data has received much less attention to date. In this paper, we propose a simple yet effective approach for unsupervised point cloud learning. In particular, we identify a very useful transformation which generates a good contrastive version of an original point cloud. They make up a pair. After going through a shared encoder and a shared head network, the consistency between the output representations are maximized with introducing two variants of contrastive losses to respectively facilitate downstream classification and segmentation. To demonstrate the efficacy of our method, we conduct experiments on three downstream tasks which are 3D object classification (on ModelNet40 and ModelNet10), shape part segmentation (on ShapeNet Part dataset) as well as scene segmentation (on S3DIS). Comprehensive results show that our unsupervised contrastive representation learning enables impressive outcomes in object classification and semantic segmentation. It generally outperforms current unsupervised methods, and even achieves comparable performance to supervised methods.

computer science, software engineering
Cross-Modal Self-Supervised Learning with Effective Contrastive Units for LiDAR Point Clouds

Mu Cai,Chenxu Luo,Yong Jae Lee,Xiaodong Yang

2024-09-11

Abstract:3D perception in LiDAR point clouds is crucial for a self-driving vehicle to properly act in 3D environment. However, manually labeling point clouds is hard and costly. There has been a growing interest in self-supervised pre-training of 3D perception models. Following the success of contrastive learning in images, current methods mostly conduct contrastive pre-training on point clouds only. Yet an autonomous driving vehicle is typically supplied with multiple sensors including cameras and LiDAR. In this context, we systematically study single modality, cross-modality, and multi-modality for contrastive learning of point clouds, and show that cross-modality wins over other alternatives. In addition, considering the huge difference between the training sources in 2D images and 3D point clouds, it remains unclear how to design more effective contrastive units for LiDAR. We therefore propose the instance-aware and similarity-balanced contrastive units that are tailored for self-driving point clouds. Extensive experiments reveal that our approach achieves remarkable performance gains over various point cloud models across the downstream perception tasks of LiDAR based 3D object detection and 3D semantic segmentation on the four popular benchmarks including Waymo Open Dataset, nuScenes, SemanticKITTI and ONCE.

Computer Vision and Pattern Recognition
Hyperbolic Image-and-Pointcloud Contrastive Learning for 3D Classification

Naiwen Hu,Haozhe Cheng,Yifan Xie,Pengcheng Shi,Jihua Zhu

2024-09-24

Abstract:3D contrastive representation learning has exhibited remarkable efficacy across various downstream tasks. However, existing contrastive learning paradigms based on cosine similarity fail to deeply explore the potential intra-modal hierarchical and cross-modal semantic correlations about multi-modal data in Euclidean space. In response, we seek solutions in hyperbolic space and propose a hyperbolic image-and-pointcloud contrastive learning method (HyperIPC). For the intra-modal branch, we rely on the intrinsic geometric structure to explore the hyperbolic embedding representation of point cloud to capture invariant features. For the cross-modal branch, we leverage images to guide the point cloud in establishing strong semantic hierarchical correlations. Empirical experiments underscore the outstanding classification performance of HyperIPC. Notably, HyperIPC enhances object classification results by 2.8% and few-shot classification outcomes by 5.9% on ScanObjectNN compared to the baseline. Furthermore, ablation studies and confirmatory testing validate the rationality of HyperIPC's parameter settings and the effectiveness of its submodules.

Computer Vision and Pattern Recognition
Cross-Modal Information-Guided Network Using Contrastive Learning for Point Cloud Registration

Yifan Xie,Jihua Zhu,Shiqi Li,Pengcheng Shi

DOI: https://doi.org/10.1109/lra.2023.3331625

IF: 5.2

2024-01-01

IEEE Robotics and Automation Letters

Abstract:The majority of point cloud registration methods currently rely on extracting features from points. However, these methods are limited by their dependence on information obtained from a single modality of points, which can result in deficiencies such as inadequate perception of global features and a lack of texture information. Actually, humans can employ visual information learned from 2D images to comprehend the 3D world. Based on this fact, we present a novel Cross-Modal Information-Guided Network (CMIGNet), which obtains global shape perception through cross-modal information to achieve precise and robust point cloud registration. Specifically, we first incorporate the projected images from the point clouds and fuse the cross-modal features using the attention mechanism. Furthermore, we employ two contrastive learning strategies, namely overlapping contrastive learning and cross-modal contrastive learning. The former focuses on features in overlapping regions, while the latter emphasizes the correspondences between 2D and 3D features. Finally, we propose a mask prediction module to identify keypoints in the point clouds. Extensive experiments on several benchmark datasets demonstrate that our network achieves superior registration performance.

robotics
CrossVideo: Self-supervised Cross-modal Contrastive Learning for Point Cloud Video Understanding

Yunze Liu,Changxi Chen,Zifan Wang,Li Yi

2024-01-17

Abstract:This paper introduces a novel approach named CrossVideo, which aims to enhance self-supervised cross-modal contrastive learning in the field of point cloud video understanding. Traditional supervised learning methods encounter limitations due to data scarcity and challenges in label acquisition. To address these issues, we propose a self-supervised learning method that leverages the cross-modal relationship between point cloud videos and image videos to acquire meaningful feature representations. Intra-modal and cross-modal contrastive learning techniques are employed to facilitate effective comprehension of point cloud video. We also propose a multi-level contrastive approach for both modalities. Through extensive experiments, we demonstrate that our method significantly surpasses previous state-of-the-art approaches, and we conduct comprehensive ablation studies to validate the effectiveness of our proposed designs.

Computer Vision and Pattern Recognition
SimIPU: Simple 2D Image and 3D Point Cloud Unsupervised Pre-Training for Spatial-Aware Visual Representations

Zhenyu Li,Zehui Chen,Ang Li,Liangji Fang,Qinhong Jiang,Xianming Liu,Junjun Jiang,Bolei Zhou,Hang Zhao

DOI: https://doi.org/10.1609/aaai.v36i2.20040

2022-01-01

Abstract:Pre-training has become a standard paradigm in many computer vision tasks. However, most of the methods are generally designed on the RGB image domain. Due to the discrepancy between the two-dimensional image plane and the three-dimensional space, such pre-trained models fail to perceive spatial information and serve as sub-optimal solutions for 3D-related tasks. To bridge this gap, we aim to learn a spatial-aware visual representation that can describe the three-dimensional space and is more suitable and effective for these tasks. To leverage point clouds, which are much more superior in providing spatial information compared to images, we propose a simple yet effective 2D Image and 3D Point cloud Unsupervised pre-training strategy, called SimIPU. Specifically, we develop a multi-modal contrastive learning framework that consists of an intra-modal spatial perception module to learn a spatial-aware representation from point clouds and an inter-modal feature interaction module to transfer the capability of perceiving spatial information from the point cloud encoder to the image encoder, respectively. Positive pairs for contrastive losses are established by the matching algorithm and the projection matrix. The whole framework is trained in an unsupervised end-to-end fashion. To the best of our knowledge, this is the first study to explore contrastive learning pre-training strategies for outdoor multi-modal datasets, containing paired camera images and LIDAR point clouds.
SimC3D: A Simple Contrastive 3D Pretraining Framework Using RGB Images

Jiahua Dong,Tong Wu,Rui Qian,Jiaqi Wang

2024-12-07

Abstract:The 3D contrastive learning paradigm has demonstrated remarkable performance in downstream tasks through pretraining on point cloud data. Recent advances involve additional 2D image priors associated with 3D point clouds for further improvement. Nonetheless, these existing frameworks are constrained by the restricted range of available point cloud datasets, primarily due to the high costs of obtaining point cloud data. To this end, we propose SimC3D, a simple but effective 3D contrastive learning framework, for the first time, pretraining 3D backbones from pure RGB image data. SimC3D performs contrastive 3D pretraining with three appealing properties. (1) Pure image data: SimC3D simplifies the dependency of costly 3D point clouds and pretrains 3D backbones using solely RBG images. By employing depth estimation and suitable data processing, the monocular synthesized point cloud shows great potential for 3D pretraining. (2) Simple framework: Traditional multi-modal frameworks facilitate 3D pretraining with 2D priors by utilizing an additional 2D backbone, thereby increasing computational expense. In this paper, we empirically demonstrate that the primary benefit of the 2D modality stems from the incorporation of locality information. Inspired by this insightful observation, SimC3D directly employs 2D positional embeddings as a stronger contrastive objective, eliminating the necessity for 2D backbones and leading to considerable performance improvements. (3) Strong performance: SimC3D outperforms previous approaches that leverage ground-truth point cloud data for pretraining in various downstream tasks. Furthermore, the performance of SimC3D can be further enhanced by combining multiple image datasets, showcasing its significant potential for scalability. The code will be available at <a class="link-external link-https" href="https://github.com/Dongjiahua/SimC3D" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition
P4Contrast: Contrastive Learning with Pairs of Point-Pixel Pairs for RGB-D Scene Understanding

Yunze Liu,Li Yi,Shanghang Zhang,Qingnan Fan,Thomas A. Funkhouser,Hao Dong

2020-01-01

Abstract:Self-supervised representation learning is a critical problem in computer vision, as it provides a way to pretrain feature extractors on large unlabeled datasets that can be used as an initialization for more efficient and effective training on downstream tasks. A promising approach is to use contrastive learning to learn a latent space where features are close for similar data samples and far apart for dissimilar ones. This approach has demonstrated tremendous success for pretraining both image and point cloud feature extractors, but it has been barely investigated for multi-modal RGB-D scans, especially with the goal of facilitating high-level scene understanding. To solve this problem, we propose contrasting "pairs of point-pixel pairs", where positives include pairs of RGB-D points in correspondence, and negatives include pairs where one of the two modalities has been disturbed and/or the two RGB-D points are not in correspondence. This provides extra flexibility in making hard negatives and helps networks to learn features from both modalities, not just the more discriminating one of the two. Experiments show that this proposed approach yields better performance on three large-scale RGB-D scene understanding benchmarks (ScanNet, SUN RGB-D, and 3RScan) than previous pretraining approaches.
3-D Object Detection With Balanced Prediction Based on Contrastive Point Loss

Jiaxun Tong,Kaiqi Liu,Xia Bai,Wei Li

DOI: https://doi.org/10.1109/jsen.2023.3347575

IF: 4.3

2024-02-16

IEEE Sensors Journal

Abstract:Object detection on point clouds is widely used in autonomous driving technology. Recent studies have demonstrated that good feature representation is the key to 3-D object detection, especially for point-based methods. Meanwhile, contrastive learning has shown its effectiveness in learning the general visual representation of images as a pretraining paradigm. Motivated by this, contrastive learning is extended to the task of object detection for better distinguishing different types of objects. In this article, a simple and effective single-stage detector, named 3-D object detection with balanced prediction based on contrastive point loss (BP-CPL), is proposed with a contrastive point loss (CPL) and a rebalanced branch (RBB). Through the CPL, similar representations among points of the same category and discriminative information among points of different categories can be learned. To keep the contrastive points consistent, a point filter is proposed. In addition, compared to the original point-wise feature, the self-supervised learned embedding is more robust to few-sample categories. Therefore, an RBB combined with the origin classification branch in a cumulative learning manner is proposed to rebalance the prediction results during the training phase. Extensive experiments show the effectiveness of the proposed method, especially for few-sample objects. The code will be available at https://github.com/Tongjiaxun/BP-CPL.

engineering, electrical & electronic,instruments & instrumentation,physics, applied
A point cloud self-learning network based on contrastive learning for classification and segmentation

Haoran Zhou,Wenju Wang,Gang Chen,Xiaolin Wang

DOI: https://doi.org/10.1007/s00371-023-03248-4

IF: 2.835

2024-01-24

The Visual Computer

Abstract:In the field of point cloud representation learning, many self-supervised learning methods aim to address the issue of conventional supervised learning methods relying heavily on labeled data. Particularly in recent years, contrastive learning-based methods have gained an increasing popularity. However, most of the current contrastive learning methods solely rely on conventional random augmentation, limiting the effectiveness of representation learning. Moreover, to prevent model collapse, they construct positive and negative sample pairs or explicit clustering centers, which adds complexity to data preprocessing operations. To address these challenges effectively and achieve accurate point cloud classification and segmentation, we propose PointSL, a self-learning network for point clouds based on contrastive learning. PointSL incorporates a learnable point cloud augmentation (LPA) module, which transforms samples with high precision, significantly improving the augmentation effect. To further enhance feature discrimination, PointSL introduces a self-learning process along a refined feature predictor (FFP). This innovative approach leverages the attention mechanism to facilitate mutual feature prediction between pairs of point clouds, thereby continuously improving discriminant performance. Additionally, the network constructed a simple yet effective self-adaptive loss function that optimizes the entire network through gradient feedback. For pretraining, it is beneficial to obtain encoders with a better generalization and a higher accuracy. We evaluate PointSL on benchmark datasets such as ModelNet40, Sydney Urban Objects and ShapeNetPart. Experimental results demonstrate that PointSL outperforms state-of-the-art self-supervised methods and supervised counterparts, achieving exceptional performance in classification and segmentation tasks. Notably, on the Sydney Urban Objects and ModelNet40 datasets, PointSL achieves OA and AA metrics of 80.6%, 69.9%, 94.2% and 91.4%, respectively. On the ShapeNetPart dataset, PointSL achieves Inst.mIoU and Cls.mIoU metrics of 86.3% and 85.1%, respectively.

computer science, software engineering
SegContrast: 3D Point Cloud Feature Representation Learning Through Self-Supervised Segment Discrimination

Lucas Nunes,Rodrigo Marcuzzi,Xieyuanli Chen,Jens Behley,Cyrill Stachniss

DOI: https://doi.org/10.1109/lra.2022.3142440

IF: 5.2

2022-04-01

IEEE Robotics and Automation Letters

Abstract:Semantic scene interpretation is essential for autonomous systems to operate in complex scenarios. While deep learning-based methods excel at this task, they rely on vast amounts of labeled data that is tedious to generate and might not cover all relevant classes sufficiently. Self-supervised representation learning has the prospect of reducing the amount of required labeled data by learning descriptive representations from unlabeled data. In this letter, we address the problem of representation learning for 3D point cloud data in the context of autonomous driving. We propose a new contrastive learning approach that aims at learning the structural context of the scene. Our approach extracts class-agnostic segments over the point cloud and applies the contrastive loss over these segments to discriminate between similar and dissimilar structures. We apply our method on data recorded with a 3D LiDAR. We show that our method achieves competitive performance and can learn a more descriptive feature representation than other state-of-the-art self-supervised contrastive point cloud methods.

robotics
Point-GCC: Universal Self-supervised 3D Scene Pre-training via Geometry-Color Contrast

Guofan Fan,Zekun Qi,Wenkai Shi,Kaisheng Ma

2023-06-01

Abstract:Geometry and color information provided by the point clouds are both crucial for 3D scene understanding. Two pieces of information characterize the different aspects of point clouds, but existing methods lack an elaborate design for the discrimination and relevance. Hence we explore a 3D self-supervised paradigm that can better utilize the relations of point cloud information. Specifically, we propose a universal 3D scene pre-training framework via Geometry-Color Contrast (Point-GCC), which aligns geometry and color information using a Siamese network. To take care of actual application tasks, we design (i) hierarchical supervision with point-level contrast and reconstruct and object-level contrast based on the novel deep clustering module to close the gap between pre-training and downstream tasks; (ii) architecture-agnostic backbone to adapt for various downstream models. Benefiting from the object-level representation associated with downstream tasks, Point-GCC can directly evaluate model performance and the result demonstrates the effectiveness of our methods. Transfer learning results on a wide range of tasks also show consistent improvements across all datasets. e.g., new state-of-the-art object detection results on SUN RGB-D and S3DIS datasets. Codes will be released at <a class="link-external link-https" href="https://github.com/Asterisci/Point-GCC" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition,Machine Learning
Prototype-Voxel Contrastive Learning for LiDAR Point Cloud Panoptic Segmentation

Minzhe Liu,Qiang Zhou,Hengshuang Zhao,Jianing Li,Yuan Du,Kurt Keutzer,Li Du,Shanghang Zhang

DOI: https://doi.org/10.1109/icra46639.2022.9811638

2022-01-01

Abstract:LiDAR point cloud panoptic segmentation, including both semantic and instance segmentation, plays a critical role in meticulous scene understanding for autonomous driving. Existing 3D voxelized approaches either utilize 3D sparse convolution that only focuses on local scene understanding, or add extra and time-consuming PointNet branch to capture global feature structures. To address these limitations, we propose an end-to-end Prototype-Voxel Contrastive Learning (PVCL) framework for learning stable and discriminative semantic representations, which includes voxel-level and prototype-level contrastive learning (CL). The voxel-level CL decreases intra-class distance and increases inter-class distance among sample representations, while the prototype-level CL further reduces the dependence of CL on negative sampling and avoids the influence of outliers from the same class, enabling PVCL to be more effective for outdoor point cloud panoptic segmentation. Extensive experiments are conducted on the public point cloud panoptic segmentation datasets, Semantic-KITTI and nuScenes, where evaluations and ablation studies demonstrate PVCL achieves superior performance compared with the state-of-the-art. Our approach ranks the top on the public leaderboard of Semantic-KITTI at the time of submission, and surpasses the published 2nd rank, EfficientLPS, by 1.7% in PQ.
Bringing Masked Autoencoders Explicit Contrastive Properties for Point Cloud Self-Supervised Learning

Bin Ren,Guofeng Mei,Danda Pani Paudel,Weijie Wang,Yawei Li,Mengyuan Liu,Rita Cucchiara,Luc Van Gool,Nicu Sebe

2024-07-08

Abstract:Contrastive learning (CL) for Vision Transformers (ViTs) in image domains has achieved performance comparable to CL for traditional convolutional backbones. However, in 3D point cloud pretraining with ViTs, masked autoencoder (MAE) modeling remains dominant. This raises the question: Can we take the best of both worlds? To answer this question, we first empirically validate that integrating MAE-based point cloud pre-training with the standard contrastive learning paradigm, even with meticulous design, can lead to a decrease in performance. To address this limitation, we reintroduce CL into the MAE-based point cloud pre-training paradigm by leveraging the inherent contrastive properties of MAE. Specifically, rather than relying on extensive data augmentation as commonly used in the image domain, we randomly mask the input tokens twice to generate contrastive input pairs. Subsequently, a weight-sharing encoder and two identically structured decoders are utilized to perform masked token reconstruction. Additionally, we propose that for an input token masked by both masks simultaneously, the reconstructed features should be as similar as possible. This naturally establishes an explicit contrastive constraint within the generative MAE-based pre-training paradigm, resulting in our proposed method, Point-CMAE. Consequently, Point-CMAE effectively enhances the representation quality and transfer performance compared to its MAE counterpart. Experimental evaluations across various downstream applications, including classification, part segmentation, and few-shot learning, demonstrate the efficacy of our framework in surpassing state-of-the-art techniques under standard ViTs and single-modal settings. The source code and trained models are available at: <a class="link-external link-https" href="https://github.com/Amazingren/Point-CMAE" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition

Isolated mitral valve prolapse: chordal architecture as an anatomic basis in older patients.

Point Contrastive Prediction with Semantic Clustering for Self-Supervised Learning on Point Cloud Videos

DCCN: A dual-cross contrastive neural network for 3D point cloud representation learning

Self-Supervised Intra-Modal and Cross-Modal Contrastive Learning for Point Cloud Understanding

Rethinking Local-to-global Representation Learning for Rotation-Invariant Point Cloud Analysis

EPContrast: Effective Point-level Contrastive Learning for Large-scale Point Cloud Understanding

Unsupervised contrastive learning with simple transformation for 3D point cloud data

Cross-Modal Self-Supervised Learning with Effective Contrastive Units for LiDAR Point Clouds

Hyperbolic Image-and-Pointcloud Contrastive Learning for 3D Classification

Cross-Modal Information-Guided Network Using Contrastive Learning for Point Cloud Registration

CrossVideo: Self-supervised Cross-modal Contrastive Learning for Point Cloud Video Understanding

SimIPU: Simple 2D Image and 3D Point Cloud Unsupervised Pre-Training for Spatial-Aware Visual Representations

SimC3D: A Simple Contrastive 3D Pretraining Framework Using RGB Images

P4Contrast: Contrastive Learning with Pairs of Point-Pixel Pairs for RGB-D Scene Understanding

3-D Object Detection With Balanced Prediction Based on Contrastive Point Loss

A point cloud self-learning network based on contrastive learning for classification and segmentation

SegContrast: 3D Point Cloud Feature Representation Learning Through Self-Supervised Segment Discrimination

Point-GCC: Universal Self-supervised 3D Scene Pre-training via Geometry-Color Contrast

Prototype-Voxel Contrastive Learning for LiDAR Point Cloud Panoptic Segmentation

Bringing Masked Autoencoders Explicit Contrastive Properties for Point Cloud Self-Supervised Learning