Abstract:As industries transition into the Industry 4.0 paradigm, the relevance and interest in concepts like Digital Twin (DT) are at an all-time high. DTs offer direct avenues for industries to make more accurate predictions, rational decisions, and informed plans, ultimately reducing costs, increasing performance and productivity. Adequate operation of DTs in the context of smart manufacturing relies on an evolving data-set relating to the real-life object or process, and a means of dynamically updating the computational model to better conform to the data. This reliance on data is made more explicit when physics-based computational models are not available or difficult to obtain in practice, as it's the case in most modern manufacturing scenarios. For data-based model surrogates to "adequately" represent the underlying physics, the number of training data points must keep pace with the number of degrees of freedom in the model, which can be on the order of thousands. However, in niche industrial scenarios like the one in manufacturing applications, the availability of data is limited (on the order of a few hundred data points, at best), mainly because a manual measuring process typically must take place for a few of the relevant quantities, e.g., level of wear of a tool. In other words, notwithstanding the popular notion of big-data, there is still a stark shortage of ground-truth data when examining, for instance, a complex system's path to failure. In this work we present a framework to alleviate this problem via modern machine learning tools, where we show a robust, efficient and reliable pathway to augment the available data to train the data-based computational models. Small sample size data is a key limitation in performance in machine learning, in particular with very high dimensional data. Current efforts for synthetic data generation typically involve either Generative Adversarial Networks (GANs) or Variational AutoEncoders (VAEs). These, however, are are tightly related to image processing and synthesis, and are generally not suited for sensor data generation, which is the type of data that manufacturing applications produce. Additionally, GAN models are susceptible to mode collapse, training instability, and high computational costs when used for high dimensional data creation. Alternatively, the encoding of VAEs greatly reduces dimensional complexity of data and can effectively regularize the latent space, but often produces poor representational synthetic samples. Our proposed method thus incorporates the learned latent space from an AutoEncoder (AE) architecture into the training of the generation network in a GAN. The advantages of such scheme are twofold: \textbf{(\textit{i})} the latent space representation created by the AE reduces the complexity of the distribution the generator must learn, allowing for quicker discriminator convergence, and \textbf{(\textit{ii})} the structure in the sensor data is better captured in the transition from the original space to the latent space. Through time statistics (up to the fifth moment), ARIMA coefficients and Fourier series coefficients, we compare the synthetic data from our proposed AE+GAN model with the original sensor data. We also show that the performance of our proposed method is at least comparable with that of the Riemannian Hamiltonian VAE, which is a recently published data augmentation framework specifically designed to handle very small high dimensional data sets.

Multimodal Synthetic Dataset Balancing: a Framework for Realistic and Balanced Training Data Generation in Industrial Settings

Ensemble Data Augmentation for Imbalanced Fault Diagnosis.

Augmented Industrial Data-Driven Modeling under the Curse of Dimensionality

Data Augmentation Classifier for Imbalanced Fault Classification

Deep Multimodal Data Fusion

Incomplete Multimodal Industrial Anomaly Detection via Cross-Modal Distillation

Training on Synthetic Data Beats Real Data in Multimodal Relation Extraction

DAGGER: Data AuGmentation GEneRative Framework for Time-Series Data in Data-Driven Smart Manufacturing Systems

Gaussian Mixture Model and Double-Weighted Deep Neural Networks for Data Augmentation Soft Sensing

Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects

Multimodal Industrial Anomaly Detection via Hybrid Fusion

Combining Synthetic Images and Deep Active Learning: Data-Efficient Training of an Industrial Object Detection Model

Fake it to make it: Using synthetic data to remedy the data shortage in joint multimodal speech-and-gesture synthesis

Distribution Bias Aware Collaborative Generative Adversarial Network for Imbalanced Deep Learning in Industrial IoT

SSE: Multimodal Semantic Data Selection and Enrichment for Industrial-scale Data Assimilation

Effective Techniques for Multimodal Data Fusion: A Comparative Analysis

Cross-Modal Data Augmentation for Tasks of Different Modalities

Siamese Neural Network and Multimodal Data Fusion Approach for Small-Sample Learning in Industrial Soft Sensor Modeling

Multimodal Fusion Balancing Through Game-Theoretic Regularization

Scalable Modular Synthetic Data Generation for Advancing Aerial Autonomy

A Generative Adversarial Networks Based Methodology for Imbalanced Multidimensional Time-Series Augmentation of Complex Electromechanical Systems