Abstract:Distributed acoustic sensing (DAS) is an emerging seismic acquisition technique with great practical potential. However, various types of noise seriously corrupt DAS signals, making it difficult to recover signals, particularly in low signal-to-noise ratio (S/N) regions. Existing deep-learning methods address this challenge by augmenting data sets or strengthening the complex architecture, which can cause overdenoising and a computational power burden. Hence, the heterogeneous knowledge distillation (HKD) method is developed to more efficiently address the signal reconstruction under low S/N. HKD uses ResNet 20 as the teacher and student (T-S) model. It uses residual learning and skip connections to facilitate feature representation at deeper levels. The main contribution is the training of the T-S framework with different noise levels. The teacher model that is trained using slightly noisy data serves as a powerful feature extractor to capture more accurate signal features because high-quality data are easy to recover. By minimizing the difference between the outputs of T-S models, the student model that is trained using severely noisy data can distill the absent signal features from the teacher to improve its own signal recovery, which enables heterogeneous feature distillation. Furthermore, simultaneous positive and negative learning (P&NL) is developed to extract more useful features from the teacher, enabling the T-S framework to learn from the predicted signal and noise during training. Consequently, a new loss function that combines student denoising loss and HKD loss weighted by P&NL is developed to alleviate signal leakage. The experimental results demonstrate that the HKD achieves distinct and consistent signal recovery without increasing computational costs.

Improved Student Model Training for Acoustic Event Detection Models.

Semi-supervised Acoustic Event Detection based on tri-training

Joint framework with deep feature distillation and adaptive focal loss for weakly supervised audio tagging and acoustic event detection

Modelling of Sound Events with Hidden Imbalances Based on Clustering and Separate Sub-Dictionary Learning

Dual Knowledge Distillation for Efficient Sound Event Detection

Transferring Voice Knowledge for Acoustic Event Detection: An Empirical Study

Exploiting the Ground-Truth: An Adversarial Imitation Based Knowledge Distillation Approach for Event Detection

Anomaly sound detection of industrial devices by using teacher-student incremental continual learning

Domestic sound event detection by shift consistency mean-teacher training and adversarial domain adaptation

Knowledge Distillation from Multiple Foundation Models for End-to-End Speech Recognition

Multi-Scale Convolutional Recurrent Neural Network with Ensemble Method for Weakly Labeled Sound Event Detection

A Four-Stage Data Augmentation Approach to ResNet-Conformer Based Acoustic Modeling for Sound Event Localization and Detection

CED: Consistent ensemble distillation for audio tagging

Efficient signal-to-noise ratio enhancement model for severely contaminated distributed acoustic sensing seismic data based on heterogeneous knowledge distillation

Maximizing Audio Event Detection Model Performance on Small Datasets Through Knowledge Transfer, Data Augmentation, And Pretraining: An Ablation Study

Sub-band Knowledge Distillation Framework for Speech Enhancement

Acoustic Model Ensembling Using Effective Data Augmentation for CHiME-5 Challenge.

Integrated Multi-Level Knowledge Distillation for Enhanced Speaker Verification

Ensemble Knowledge Distillation of Self-Supervised Speech Models

Improved Knowledge Distillation via Adversarial Collaboration

FMSG-JLESS Submission for DCASE 2024 Task4 on Sound Event Detection with Heterogeneous Training Dataset and Potentially Missing Labels