Abstract:Visual recognition of materials and their states is essential for understanding the physical world, from identifying wet regions on surfaces or stains on fabrics to detecting infected areas on plants or minerals in rocks. Collecting data that captures this vast variability is complex due to the scattered and gradual nature of material states. Manually annotating real-world images is constrained by cost and precision, while synthetic data, although accurate and inexpensive, lacks real-world diversity. This work aims to bridge this gap by infusing patterns automatically extracted from real-world images into synthetic data. Hence, patterns collected from natural images are used to generate and map materials into synthetic scenes. This unsupervised approach captures the complexity of the real world while maintaining the precision and scalability of synthetic data. We also present the first comprehensive benchmark for zero-shot material state segmentation, utilizing real-world images across a diverse range of domains, including food, soils, construction, plants, liquids, and more, each appears in various states such as wet, dry, infected, cooked, burned, and many others. The annotation includes partial similarity between regions with similar but not identical materials and hard segmentation of only identical material states. This benchmark eluded top foundation models, exposing the limitations of existing data collection methods. Meanwhile, nets trained on the infused data performed significantly better on this and related tasks. The dataset, code, and trained model are available. We also share 300,000 extracted textures and SVBRDF/PBR materials to facilitate future datasets generation.

An Improved Dataset Augmentation Approach for Deep Learning-Based XCT Images Segmentation in Layered Composite Fabric

Improved 3D image segmentation for X-ray tomographic data of biaxial warp-knitted composites

Automated processing of X-ray computed tomography images via panoptic segmentation for modeling woven composite textiles

Improved XCT image automatic segmentation for quantitative characterization of the meso-morphological features in the damaged braided composite fabric

CT Image Segmentation Method of Composite Material Based on Improved Watershed Algorithm and U-Net Neural Network Model

Segmentation of computed tomography images and high-precision reconstruction of rubber composite structure based on deep learning

Automatic reconstruction of closely packed fabric composite RVEs using yarn-level micro-CT images processed by convolutional neural networks (CNNs) and based on physical characteristics

Effective Transfer of Pretrained Large Visual Model for Fabric Defect Segmentation via Specifc Knowledge Injection

A Machine Vision-based Fabric Defect Detection Solution for Textile Production Industry Using Object Detection

Variational and Deep Learning Segmentation of Very-Low-Contrast X-ray Computed Tomography Images of Carbon/Epoxy Woven Composites

A Deep Learning Labeling Method for Material Microstructure Image Segmentation

Synthetic Data Generation for Automatic Segmentation of X-ray Computed Tomography Reconstructions of Complex Microstructures

Descriptive modeling of textiles using FE simulations and deep learning

Synthetic-to-real Composite Semantic Segmentation in Additive Manufacturing

Instance Segmentation of Fibers from Low Resolution CT Scans via 3D Deep Embedding Learning

Fabric surface characterization: assessment of deep learning-based texture representations using a challenging dataset

An Efficient Finite Element Mesh Generation Methodology Based on Μct Images of Multi-Layer Woven Composites

Generating 3D digital material twins for woven ceramic‐matrix composites from μCT images

Learning Zero-Shot Material States Segmentation, by Implanting Natural Image Patterns in Synthetic Data

Image segmentation of exfoliated two-dimensional materials by generative adversarial network-based data augmentation

Knitted fabric structure recognition based on deep learning