Abstract:Code search plays a crucial role in software development, enabling developers to retrieve and reuse code using natural language queries. While the performance of code search models improves with an increase in high-quality data, obtaining such data can be challenging and expensive. Recently, large language models (LLMs) such as ChatGPT have made remarkable progress in both natural and programming language understanding and generation, offering user-friendly interaction via simple prompts. Inspired by these advancements, we propose a novel approach ChatDANCE, which utilizes high-quality and diverse augmented data generated by a large language model and leverages a filtering mechanism to eliminate low-quality augmentations. Specifically, we first propose a set of ChatGPT prompting rules that are specifically designed for source code and queries. Then, we leverage ChatGPT to rewrite code and queries based on the according prompts and then propose a filtering mechanism which trains a cross-encoder from the backbone model UniXcoder to filter out code and query pairs with low matching scores. Finally, we re-train the backbone model using the obtained high-quality augmented data. Experimental results show that ChatDANCE achieves state-of-the-art performance, improving the best baseline by 13.2% (R@1) and 7% (MRR). Surprisingly, we find that this augment-filter-retrain strategy enables the backbone model (UniXcoder) to self-grow. Moreover, extensive experiments show the effectiveness of each component and ChatDANCE has stable performance under different hyperparameter settings. In addition, we conduct qualitative and quantitative analyses to investigate why ChatDANCE works well and find that it learns a more uniform distribution of representations and effectively aligns the code and query spaces.

ZeroShotDataAug: Generating and Augmenting Training Data with ChatGPT

AugGPT: Leveraging ChatGPT for Text Data Augmentation

ChatGPT as Data Augmentation for Compositional Generalization: A Case Study in Open Intent Detection

ChatGPT for Zero-shot Dialogue State Tracking: A Solution or an Opportunity?

You Augment Me: Exploring ChatGPT-based Data Augmentation for Semantic Code Search

Insufficient Data Can Also Rock！Learning to Converse Using Smaller Data with Augmentation

From Big to Small Without Losing It All: Text Augmentation with ChatGPT for Efficient Sentiment Analysis

Contextual Data Augmentation for Task-Oriented Dialog Systems

Improving Text Classification with Large Language Model-Based Data Augmentation

Training Conversational Agents with Generative Conversational Networks

Improving Training Dataset Balance with ChatGPT Prompt Engineering

Improving Low-resource Question Answering by Augmenting Question Information

Educational data augmentation in physics education research using ChatGPT

Knowledge-Grounded Conversational Data Augmentation with Generative Conversational Networks

Data Augmentation using Generative-AI

Unraveling ChatGPT: A Critical Analysis of AI-Generated Goal-Oriented Dialogues and Annotations

A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capability

Data Augmentation for Retrieval- and Generation-Based Dialog Systems

Decoding ChatGPT: A Taxonomy of Existing Research, Current Challenges, and Possible Future Directions

ChatGPT: Revolutionizing User Interactions with Advanced Natural Language Processing

Augmenting AI’s Prospects: ChatGPT Future Insights