Single-image driven 3d viewpoint training data augmentation for effective wine label recognition

Yueh-Cheng Huang,Hsin-Yi Chen,Cheng-Jui Hung,Jen-Hui Chuang,Jenq-Neng Hwang

2024-04-13

Abstract:Confronting the critical challenge of insufficient training data in the field of complex image recognition, this paper introduces a novel 3D viewpoint augmentation technique specifically tailored for wine label recognition. This method enhances deep learning model performance by generating visually realistic training samples from a single real-world wine label image, overcoming the challenges posed by the intricate combinations of text and logos. Classical Generative Adversarial Network (GAN) methods fall short in synthesizing such intricate content combination. Our proposed solution leverages time-tested computer vision and image processing strategies to expand our training dataset, thereby broadening the range of training samples for deep learning applications. This innovative approach to data augmentation circumvents the constraints of limited training resources. Using the augmented training images through batch-all triplet metric learning on a Vision Transformer (ViT) architecture, we can get the most discriminative embedding features for every wine label, enabling us to perform one-shot recognition of existing wine labels in the training classes or future newly collected wine labels unavailable in the training. Experimental results show a significant increase in recognition accuracy over conventional 2D data augmentation techniques.

Computer Vision and Pattern Recognition,Machine Learning

What problem does this paper attempt to address?

The paper attempts to address the issue of insufficient training data in the task of wine label recognition. Specifically, the main challenges faced in wine label recognition include:

1. **Limited training data**: In practical applications, it is very difficult to obtain a large amount of high-quality and representative training data, especially in tasks like wine label recognition that require handling complex images.
2. **Limitations of traditional data augmentation methods**: Traditional 2D data augmentation methods (such as rotation, cropping, color transformation, etc.) cannot generate sufficiently realistic training samples, particularly for images of wine labels that contain complex text and patterns.
3. **Handling perspective changes**: Wine labels are usually attached to cylindrical wine bottles, and images of wine labels taken from different angles will have significant perspective changes, making it difficult for the model to generalize to unseen perspectives.

To address these issues, the paper proposes a 3D perspective data augmentation technique based on a single image, which generates visually realistic training samples to improve the performance of deep learning models. This method not only generates diverse training data but also improves recognition accuracy during the testing phase by generating front-view images.