Abstract:When the machinery device operates abnormally, it is not sufficient for fault detection only via extracting fault features from a single sensor due to the latent fault information may be scattered across multiple sensors. Multi-sensory fusion techniques with deep learning framework have attracted increasing attention from researchers due to the exploiting and integration of fault information between multiple sensors. Nevertheless, there are two remaining shortcomings in most existing multi-sensory fusion technologies. (1) Most existing fusion methods merely concentrate on conducting multi-sensory information fusion from time-domain or frequency-domain to achieve fault diagnosis, which are often unsatisfactory in the face of strong noise environments. (2) The collaborative fusion between several vibration sensors is generally considered in the past works, whereas the complementary information fusion between multi-sensory vibro-acoustic heterogeneous data are rarely studied. To address these deficiencies, this paper proposes a novel coarse-to-fine dual-scale time-frequency attention fusion network (CDTFAFN) for machinery fault diagnosis, which not only adequately considers the complementary information fusion of vibro-acoustic signal, but also has robust feature learning capabilities in a noisy scenario. Firstly, the signal-to-image encoding unit (SIEU) containing the improved constant-Q non-stationary Gabor transform (ICQ-NSGT) is introduced to convert the collected raw vibro-acoustic heterogeneous signal into time-frequency representation (TFR) and achieve the coarse-grained feature fusion. Secondly, the time-frequency attention feature fusion unit (TFA-FFU) is designed to concurrently learn the fine-grained features at two scales from the low-level fused features which are meaningful for fault diagnosis. Finally, the coarse-to-fine features are sequentially concatenated and fed into softmax classifier to preferably promote the network learning performance and automatically implement fault classification. The performance of the proposed approach is validated against those state-of-the-art results on two groups of multi-sensory vibro-acoustic data in different experimental platforms. Experiment results show that the proposed method with the diagnosis accuracy of 99 % above outperforms other several representative fusion technologies (i.e., 2MNet, MFF-GBFD, MSCNN-BiLSTM, MFAN-VAF, 1D-CNN-VAF, MI-CNN-TFT and TFFN-VAF) in the raw noise-free addition scenario. Moreover, the average testing accuracy of the proposed method can still reach 97 % above in the noisy scenarios with Gaussian white noises, which shows its competitive superiority and strong robustness against noises in machinery fault diagnosis. According to the five ensemble macro-average performance evaluation metrics (i.e., accuracy, precision, sensitivity, specificity and F1-score) and the receiver operator characteristic (ROC) analysis, our findings also emphasize the superiority of applying our method for machinery fault diagnosis under the colored noises compared with other fusion technologies (i.e., 2MNet, MFF-GBFD, MSCNN-BiLSTM, MFAN-VAF, 1D-CNN-VAF, MI-CNN-TFT and TFFN-VAF) reported in this paper.

A Deep Coupled Network for Health State Assessment of Cutting Tools Based on Fusion of Multisensory Signals

Fault Diagnosis Method Based on Improved Deep Boltzmann Machines

Health State Identification of Circulating Seawater Pump-Unit in Nuclear Power Plant Based on Multi-Virtual Vibration Source Fusion in the Presence of Strong Data Imbalance

A hybrid network capturing multisource feature correlations for tool remaining useful life prediction

Deep Coupling Autoencoder for Fault Diagnosis with Multimodal Sensory Data

A Multi-State Diagnosis and Prognosis Framework with Feature Learning for Tool Condition Monitoring

CDTFAFN: A Novel Coarse-to-fine Dual-Scale Time-Frequency Attention Fusion Network for Machinery Vibro-Acoustic Fault Diagnosis

Intelligent Health Monitoring of Machine Tools Using a Bayesian Multibranch Neural Network

Multimodal Correlation-Aware Fusion Framework for Enhanced Machinery Health Prognosis With Unlabeled and Low-Quality Data Exploitation

Deep Multisource Parallel Bilinear-Fusion Network for Remaining Useful Life Prediction of Machinery

Multi-Sensor Data Fusion for Remaining Useful Life Prediction of Machining Tools by IABC-BPNN in Dry Milling Operations

Research on a Real-Time Monitoring Method for the Wear State of a Tool Based on a Convolutional Bidirectional LSTM Model

A novel approach to tool condition monitoring based on multi-sensor data fusion imaging and an attention mechanism

Tool Wear Monitoring with Vibration Signals Based on Short-Time Fourier Transform and Deep Convolutional Neural Network in Milling

Coupled Hidden Markov Fusion of Multichannel Fast Spectral Coherence Features for Intelligent Fault Diagnosis of Rolling Element Bearings

Learning to Monitor Machine Health with Convolutional Bi-Directional LSTM Networks

Estimation of Tool Wear and Surface Roughness Development Using Deep Learning and Sensors Fusion

Multi-frequency-band deep CNN model for tool wear prediction

Dimensionality reduce-based for remaining useful life prediction of machining tools with multisensor fusion

An acoustic imaging recognition based cutting tools wear state prediction method

Joint-Module Health Status Recognition for an Unmanned Platform: A Time–Frequency Representation and Extraction Network-Based Approach