Abstract:In this paper, we propose a deep learning based system for the task of deepfake audio detection. In particular, the draw input audio is first transformed into various spectrograms using three transformation methods of Short-time Fourier Transform (STFT), Constant-Q Transform (CQT), Wavelet Transform (WT) combined with different auditory-based filters of Mel, Gammatone, linear filters (LF), and discrete cosine transform (DCT). Given the spectrograms, we evaluate a wide range of classification models based on three deep learning approaches. The first approach is to train directly the spectrograms using our proposed baseline models of CNN-based model (CNN-baseline), RNN-based model (RNN-baseline), C-RNN model (C-RNN baseline). Meanwhile, the second approach is transfer learning from computer vision models such as ResNet-18, MobileNet-V3, EfficientNet-B0, DenseNet-121, SuffleNet-V2, Swint, Convnext-Tiny, GoogLeNet, MNASsnet, RegNet. In the third approach, we leverage the state-of-the-art audio pre-trained models of Whisper, Seamless, Speechbrain, and Pyannote to extract audio embeddings from the input spectrograms. Then, the audio embeddings are explored by a Multilayer perceptron (MLP) model to detect the fake or real audio samples. Finally, high-performance deep learning models from these approaches are fused to achieve the best performance. We evaluated our proposed models on ASVspoof 2019 benchmark dataset. Our best ensemble model achieved an Equal Error Rate (EER) of 0.03, which is highly competitive to top-performing systems in the ASVspoofing 2019 challenge. Experimental results also highlight the potential of selective spectrograms and deep learning approaches to enhance the task of audio deepfake detection.

Dynamic Ensemble Teacher-Student Distillation Framework for Light-weight Fake Audio Detection

End-to-end Spoofing Speech Detection and Knowledge Distillation under Noisy Conditions

Learning From Yourself: A Self-Distillation Method for Fake Speech Detection

FTDKD: Frequency-Time Domain Knowledge Distillation for Low-Quality Compressed Audio Deepfake Detection

Fully Automated End-to-End Fake Audio Detection.

Generalized Fake Audio Detection via Deep Stable Learning

Lightweight Voice Spoofing Detection Using Improved One-Class Learning and Knowledge Distillation

Frequency-mix Knowledge Distillation for Fake Speech Detection

Genuine-Focused Learning using Mask AutoEncoder for Generalized Fake Audio Detection

A lightweight feature extraction technique for deepfake audio detection

Continual Learning for Fake Audio Detection

Deepfake Audio Detection Using Spectrogram-based Feature and Ensemble of Deep Learning Models

FADA: Fast Diffusion Avatar Synthesis with Mixed-Supervised Multi-CFG Distillation

An Adversarial Feature Distillation Method for Audio Classification

Advancing Continual Learning for Robust Deepfake Audio Classification

A Lightweight Framework for Online Voice Activity Detection in the Wild.

Speaker Recognition-Assisted Robust Audio Deepfake Detection

Data Augmentation Using McAdams-Coefficient-Based Speaker Anonymization for Fake Audio Detection

Towards Robust Audio Deepfake Detection: A Evolving Benchmark for Continual Learning

Audio Deepfake Detection Based on a Combination of F0 Information and Real Plus Imaginary Spectrogram Features

Mixture of Experts Fusion for Fake Audio Detection Using Frozen wav2vec 2.0