An automatic inspection system for the detection of tire surface defects and their severity classification through a two-stage multimodal deep learning approach

Thomas Mignot,François Ponchon,Alexandre Derville,Stefan Duffner,Christophe Garcia

DOI: https://doi.org/10.1007/s10845-024-02378-3

IF: 8.3

2024-05-23

Journal of Intelligent Manufacturing

Abstract:In the tire manufacturing field, the pursuit of uncompromised product quality stands as a cornerstone. This paper introduces an innovative multimodal approach aimed at automating the tire quality control process through the use of deep learning on data obtained from stereo-photometric cameras meticulously integrated into a purpose-built, sophisticated tire acquisition system capable of comprehensive data capture across all tire zones. The defects sought exhibit significant variations in size (ranging from a few millimeters to several tens of centimeters) and type (including abnormal stains during processing, marks resulting from demolding issues, foreign particles, air bubbles, deformations, etc.). Our proposed methodology comprises two distinct stages: an initial instance segmentation phase for defect detection and localization, followed by a classification stage based on severity levels, integrating features extracted from the detection network of the first stage alongside tire metadata. Experimental validation demonstrates that the proposed approach achieves automation objectives, attaining satisfactory results in terms of defect detection and classification according to severity, with a F1 score between 0.7 and 0.89 depending on the tire zone. In addition, this study presents a novel method applicable to all tire areas, addressing a wide variety of defects within the domain.

engineering, manufacturing,computer science, artificial intelligence

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve

This paper aims to address the issue of tire surface defect detection and severity classification in the tire manufacturing process. Specifically, the paper proposes an innovative multimodal deep learning method for the automated tire quality control process. The main objectives include:

1. **Real-time processing of high-resolution photometric stereo images**: Achieving technological innovation in the tire industry from defect detection to severity classification.
2. **Defining fine-tuning strategies and training processes**: Capable of handling quality control for all areas inside and outside the tire.
3. **Designing a two-stage architecture**:
- A detection network based on MASK-RCNN, with two additional IoU prediction branches.
- A multimodal deep learning model combining Gated Recurrent Units (GRU) and attention mechanisms for classifying large images and their metadata.

Through these methods, the paper aims to improve the accuracy and efficiency of tire surface defect detection and classification, reduce the workload of manual inspection, and enhance production efficiency. Experimental results show that the method achieves satisfactory results in defect detection and classification tasks in different tire areas, with F1 scores ranging from 0.7 to 0.89. Additionally, the study proposes a new method applicable to all tire areas, covering various types of defects.