Abstract:Due to the incapability of one sensory measurement to provide enough information for condition monitoring of some complex engineered industrial mechanisms and also for overcoming the misleading noise of a single sensor, multiple sensors are installed to improve the condition monitoring of some industrial equipment. Therefore, an efficient data fusion strategy is demanded. In this research, we presented a Denoising Multi-Modal Autoencoder with a unique training strategy based on contrastive learning paradigm, both being utilized for the first time in the machine health monitoring realm. The presented approach, which leverages the merits of both supervised and unsupervised learning, not only achieves excellent performance in fusing multiple modalities (or views) of data into an enriched common representation but also takes data fusion to the next level wherein one of the views can be omitted during inference time with very slight performance reduction, or even without any reduction at all. The presented methodology enables multi-modal fault diagnosis systems to perform more robustly in case of sensor failure occurrence, and one can also intentionally omit one of the sensors (the more expensive one) in order to build a more cost-effective condition monitoring system without sacrificing performance for practical purposes. The effectiveness of the presented methodology is examined on a real-world private multi-modal dataset gathered under non-laboratory conditions from a complex engineered mechanism, an inline four-stroke spark-ignition engine, aiming for spark plug fault diagnosis. This dataset, which contains the accelerometer and acoustic signals as two modalities, has a very slight amount of fault, and achieving good performance on such a dataset promises that the presented method can perform well on other equipment as well.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve the problems in the fault diagnosis of internal combustion engine spark plugs, where single - sensor measurement cannot provide sufficient information and is easily misled by single - mode noise. Specifically, the paper proposes a Denoising Multi - Modal Autoencoder based on the contrastive learning paradigm for fusing vibration signals and acoustic signals to achieve more robust spark plug fault diagnosis. #### Main problems and solutions 1. **Insufficient information from a single sensor**: - The information provided by a single sensor is not sufficient to comprehensively monitor the state of complex industrial equipment. - Solution: Use multiple sensors to collect data of different modalities (such as vibration signals and acoustic signals), and improve the performance of state monitoring through effective data fusion strategies. 2. **Misleading by single - mode noise**: - The noise in a single modality may mislead the fault diagnosis results. - Solution: Through multi - modal data fusion, reduce the influence of single - mode noise and improve the accuracy of diagnosis. 3. **Sensor failure or cost problems**: - In practical applications, some sensors may fail or be too expensive. - Solution: The proposed model can omit the data of one modality at the inference stage without significantly degrading performance, thus improving the robustness and cost - effectiveness of the system. #### Specific application scenarios This research specifically focuses on the spark plug gap enlargement fault in four - stroke in - line four - cylinder spark - ignition engines. Spark plug gap enlargement is one of the common faults of spark plugs, which can lead to problems such as ignition system damage and engine performance degradation. By fusing vibration signals and acoustic signals, this method can more accurately identify this fault and has been verified under actual operating conditions. #### Innovation points 1. **Introducing the contrastive learning paradigm for the first time**: - Using the contrastive learning paradigm for multi - modal data fusion, which is the first application in the field of machine health monitoring. 2. **Supporting the case of missing modalities**: - The proposed method can omit the data of one modality during testing without significantly affecting performance, which is especially important in cases of sensor failure or deliberately omitting expensive sensors. 3. **More realistic fault simulation**: - Simulating more minor spark plug gap enlargement fault situations, which is more in line with the actual situation and increases the practical value of the research. In conclusion, this paper solves the problems of insufficient information from a single sensor and noise misguidance by proposing an innovative multi - modal data fusion method, and shows its superiority in practical applications.

Contrastive Multi-Modal Representation Learning for Spark Plug Fault Diagnosis

Multi-sensor heterogeneous data fusion method for rotor system diagnosis based on multi-mode residual network and discriminant correlation analysis

Multi-view graph convolutional networks based on multi-source information fusion for mechanical fault diagnosis

Deep Coupling Autoencoder for Fault Diagnosis with Multimodal Sensory Data

A novel approach of multisensory fusion to collaborative fault diagnosis in maintenance

A multi-modal joint attention network for vibro-acoustic fusion diagnosis of engines

Multi-Modal Learning-Based Equipment Fault Prediction in the Internet of Things.

A convolutional multisensor fusion fault diagnosis framework based on multidimensional distance matrix for rotating machinery

Multimodal fusion fault diagnosis method under noise interference

Hybrid multi-mode machine learning-based fault diagnosis strategies with application to aircraft gas turbine engines

Cross-modal Fusion Convolutional Neural Networks with Online Soft Label Training Strategy for Mechanical Fault Diagnosis

An Expert Condition Monitoring System via Fusion of Signal Processing for Vibration of Industrial Rotating Machinery with Unseen Operational Conditions

Aero-engine prognosis strategy based on multi-scale feature fusion and multi-task parallel learning

Deep Convolutional Autoencoder for Assessment of Anomalies in Multi-stream Sensor Data

Multi-sensor data fusion-enabled lightweight convolutional double regularization contrast transformer for aerospace bearing small samples fault diagnosis

A novel digital twin approach based on deep multimodal information fusion for aero-engine fault diagnosis

An Ensemble Convolutional Neural Networks for Bearing Fault Diagnosis Using Multi-Sensor Data

Semisupervised fault diagnosis of aeroengine based on denoising autoencoder and deep belief network

A novel self-supervised representation learning framework based on time-frequency alignment and interaction for mechanical fault diagnosis

A Lightweight and Efficient Multimodal Feature Fusion Network for Bearing Fault Diagnosis in Industrial Applications

Rolling Bearing Fault Diagnosis Method Based on Multi-sensor Two-stage Fusion