Abstract:Code search aims to retrieve semantically relevant code snippets for a given natural language query. Recently, many approaches employing contrastive learning have shown promising results on code representation learning and greatly improved the performance of code search. However, there is still a lot of room for improvement in using contrastive learning for code search. In this paper, we propose CoCoSoDa to effectively utilize contrastive learning for code search via two key factors in contrastive learning: data augmentation and negative samples. Specifically, soft data augmentation is to dynamically masking or replacing some tokens with their types for input sequences to generate positive samples. Momentum mechanism is used to generate large and consistent representations of negative samples in a mini-batch through maintaining a queue and a momentum encoder. In addition, multimodal contrastive learning is used to pull together representations of code-query pairs and push apart the unpaired code snippets and queries. We conduct extensive experiments to evaluate the effectiveness of our approach on a large-scale dataset with six programming languages. Experimental results show that: (1) CoCoSoDa outperforms 14 baselines and especially exceeds CodeBERT, GraphCodeBERT, and UniXcoder by 13.3%, 10.5%, and 5.9% on average MRR scores, respectively. (2) The ablation studies show the effectiveness of each component of our approach. (3) We adapt our techniques to several different pre-trained models such as RoBERTa, CodeBERT, and GraphCodeBERT and observe a significant boost in their performance in code search. (4) Our model performs robustly under different hyper-parameters. Furthermore, we perform qualitative and quantitative analyses to explore reasons behind the good performance of our model.

DaCo: Domain-Agnostic Contrastive Learning for Visual Place Recognition

Explicit Feature Disentanglement for Visual Place Recognition Across Appearance Changes

Adversarial Feature Disentanglement for Place Recognition Across Changing Appearance.

Category Contrast for Unsupervised Domain Adaptation in Visual Tasks

Domain Adaptation for Remote Sensing Image Semantic Segmentation: An Integrated Approach of Contrastive Learning and Adversarial Learning

Domain-Invariant Similarity Activation Map Contrastive Learning for Retrieval-Based Long-Term Visual Localization

Domain Contrast for Domain Adaptive Object Detection

Towards Domain-Agnostic Contrastive Learning

DASGIL: Domain Adaptation for Semantic and Geometric-Aware Image-Based Localization.

CCL: Continual Contrastive Learning for LiDAR Place Recognition

Contrastive Learning with Consistent Representations

C^2DA: Contrastive and Context-aware Domain Adaptive Semantic Segmentation

Attention-Aware Age-Agnostic Visual Place Recognition

Self-Supervised Visual Representations Learning by Contrastive Mask Prediction

Contrast and Clustering: Learning Neighborhood Pair Representation for Source-free Domain Adaptation

Domain Adaptation for Multi-label Remote Sensing Image Annotation with Contrastive Pseudo-Label Generation

Cross-modal & Cross-domain Learning for Unsupervised LiDAR Semantic Segmentation

CoCoSoDa: Effective Contrastive Learning for Code Search

CroCo: Cross-Modal Contrastive learning for localization of Earth Observation data

Domain adaptation method for extracting buildings from remote sensing images using dual contrastive learning

Locality-aware Cross-modal Correspondence Learning for Dense Audio-Visual Events Localization