Abstract:In this study, we propose a novel deep neural network (DNN) architecture for speech enhancement (SE) via a multiobjective learning and ensembling (MOLE) framework to achieve a compact and lowlatency design, while maintaining good performance in quality evaluations. MOLE follows the boosting concept when combining weak models into a strong classifier and consists of two compact DNNs. The first, called the multiobjective learning DNN (MOL-DNN), takes multiple features, such as log-power spectra (LPS), mel-frequency cepstral coefficients (MFCCs) and Gammatone frequency cepstral coefficients (GFCCs) to predict a multiobjective set that includes clean speech feature, dynamic noise feature, and ideal ratio mask (IRM). The second, called the multiobjective ensembling DNN (MOE-DNN), takes the learned features from MOL-DNN as inputs and separately predicts clean LPS and IRM, clean MFCC and IRM, and clean GFCC and IRM using three sets of weak regression functions. Finally, a postprocessing operation can be applied to the estimated clean features by leveraging the multiple targets learned from both the MOL-DNN and the MOE-DNN. On speech corrupted by 15 noise types not seen in model training the SE results show that the MOLE approach, which features a small model size and low run-time latency, can achieve consistent improvements over both DNN- and long short-term memory (LSTM)-based techniques in terms of all the objective metrics evaluated in this study for all three cases (the input contexts contain 1-frame, 4-frame and 7-frame instances). The 1-frame MOLE-based SE system outperforms the DNN-based SE system with a 7-frame input expansion at a 3-frame delay and also achieves better performance than the LSTM-based SE system with 4-frame, no delay expansion by including only 3 previous frames, and with 170 times less processing latency.

Learning FOFE Based FNN-LMs with Noise Contrastive Estimation and Part-of-speech Features

The Fixed-Size Ordinally-Forgetting Encoding Method for Neural Network Language Models.

A Fixed-Size Encoding Method for Variable-Length Sequences with its Application to Neural Network Language Models

Learning neural trans-dimensional random field language models with noise-contrastive estimation

Improved training of neural trans-dimensional random field language models with dynamic noise-contrastive estimation.

CLLMFS: A Contrastive Learning enhanced Large Language Model Framework for Few-Shot Named Entity Recognition

Fine-Grained Classification with Noisy Labels

FE-RNN: A fuzzy embedded recurrent neural network for improving interpretability of underlying neural network

Empirically Combining Unnormalized NNLM and Back-off N -Gram for Fast N -Best Rescoring in Speech Recognition

Towards harnessing feature embedding for robust learning with noisy labels

Optimal Training of Feedforward Neural Networks Using Teaching-Learning-Based Optimization with Modified Learning Phases

A Multiobjective Learning and Ensembling Approach to High-Performance Speech Enhancement with Compact Neural Network Architectures

Learning Regularized Noise Contrastive Estimation for Robust Network Embedding.

LoRAMoE: Alleviating World Knowledge Forgetting in Large Language Models via MoE-Style Plugin

LoRAMoE: Alleviate World Knowledge Forgetting in Large Language Models via MoE-Style Plugin

Compact Feedforward Sequential Memory Networks for Large Vocabulary Continuous Speech Recognition

LoRAMoE: Alleviating World Knowledge Forgetting in Large Language Models Via MoE-Style Plugin.

LeMoLE: LLM-Enhanced Mixture of Linear Experts for Time Series Forecasting

CogniFNN: A Fuzzy Neural Network Framework for Cognitive Word Embedding Evaluation

Incorporating Class-based Language Model for Named Entity Recognition in Factorized Neural Transducer

Noisy-Labeled NER with Confidence Estimation