Abstract:Copy-Paste is a simple and effective data augmentation strategy for instance segmentation. By randomly pasting object instances onto new background images, it creates new training data for free and significantly boosts the segmentation performance, especially for rare object categories. Although diverse, high-quality object instances used in Copy-Paste result in more performance gain, previous works utilize object instances either from human-annotated instance segmentation datasets or rendered from 3D object models, and both approaches are too expensive to scale up to obtain good diversity. In this paper, we revisit Copy-Paste at scale with the power of newly emerged zero-shot recognition models (e.g., CLIP) and text2image models (e.g., StableDiffusion). We demonstrate for the first time that using a text2image model to generate images or zero-shot recognition model to filter noisily crawled images for different object categories is a feasible way to make Copy-Paste truly scalable. To make such success happen, we design a data acquisition and processing framework, dubbed “X-Paste", upon which a systematic study is conducted. On the LVIS dataset, X-Paste provides impressive improvements over the strong baseline CenterNet2 with Swin-L as the backbone. Specifically, it archives +2.6 box AP and +2.1 mask AP gains on all classes and even more significant gains with +6.8 box AP, +6.5 mask AP on long-tail classes. Our code and models are available at https://github.com/yoctta/XPaste.

Instance Segmentation Challenge Track Technical Report, VIPriors Workshop at ICCV 2021: Task-Specific Copy-Paste Data Augmentation Method for Instance Segmentation

X-Paste: Revisiting Scalable Copy-Paste for Instance Segmentation Using CLIP and StableDiffusion

Instance Segmentation under Occlusions via Location-aware Copy-Paste Data Augmentation

SDI-Paste: Synthetic Dynamic Instance Copy-Paste for Video Instance Segmentation

Image- and Instance-Level Data Augmentation for Occluded Instance Segmentation

3rd Place Scheme on Instance Segmentation Track of ICCV 2021 VIPriors Challenges

Task-Specific Data Augmentation and Inference Processing for VIPriors Instance Segmentation Challenge

Augment Before Copy-Paste: Data and Memory Efficiency-Oriented Instance Segmentation Framework for Sport-scenes

InstaBoost: Boosting Instance Segmentation Via Probability Map Guided Copy-Pasting

Data-Efficient Instance Segmentation with a Single GPU

Continuous Copy-Paste for One-stage Multi-object Tracking and Segmentation

A Multi-Stage Adaptive Copy-Paste Data Augmentation Algorithm Based on Model Training Preferences

The Second Place Solution for ICCV2021 VIPriors Instance Segmentation Challenge

Evaluating the Efficacy of Cut-and-Paste Data Augmentation in Semantic Segmentation for Satellite Imagery

Augmentation for small object detection

TumorCP: A Simple but Effective Object-Level Data Augmentation for Tumor Segmentation

The Second-place Solution for CVPR VISION 23 Challenge Track 1 -- Data Effificient Defect Detection

Spatially Exclusive Pasting: A General Data Augmentation for the Polyp Segmentation

Enrich the content of the image Using Context-Aware Copy Paste

2nd Place Solution to ECCV 2020 VIPriors Object Detection Challenge

ECAP: Extensive Cut-and-Paste Augmentation for Unsupervised Domain Adaptive Semantic Segmentation