Abstract:Audio content synthesis has stepped into a new era and brought a great threat to daily life since the development of deep learning techniques. The ASVSpoof Challenge and the ADD Challenge have been launched to motivate the development of Deepfake audio detection algorithms. Currently, the detection models, which consist of front-end feature extractors and back-end classifiers, utilize the physical features mainly, rather than the perceptual features that relate to natural emotions or breathiness. Therefore, we provide a comprehensive study on 16 physical and perceptual features and evaluate their effectiveness in both Track 1 and Track 2 of the ADD Challenge. Based on results, PLP, as a perceptual feature, outperforms the rest of the features in Track 1, while CQCC has the best performance in Track 2. Our experiments demonstrate the significance of perceptual features in detecting Deepfake audios. We also seek to explore the underlying characteristics of features that can distinguish Deepfake audio from a real one. We perform statistical analysis on each feature to show its distribution differences on real and synthesized audios. This paper will provide a potential direction in selecting appropriate feature extraction methods for the future implementation of detection models.

Perceptual Similarity Between Audio Clips and Feature Selection for Its Measurement

Audio retrieval based on perceptual similarity

$$\Hbox {u}^2\hbox {f}^2\hbox {S}^2$$ U 2 F 2 S 2 : Uncovering Feature-level Similarities for Unsupervised Feature Selection.

U^2F^2S^2 : Uncovering Feature-level Similarities for Unsupervised Feature Selection

Audio Similarity Measure by Graph Modeling and Matching

Effective Audio Fingerprint Retrieval Based on the Spectral Sub-Band Centroid Feature

Audio Retrieval with Fast Relevance Feedback Based on Constrained Fuzzy Clustering and Stored Index Table.

Semi-supervised Minimum Redundancy Maximum Relevance Feature Selection for Audio Classification

Semi-supervised Feature Selection for Audio Classification Based on Constraint Compensated Laplacian Score

Daubechies Wavelets Based Robust Audio Fingerprinting for Content-Based Audio Retrieval

Understanding Visual-Auditory Correlation from Heterogeneous Features for Cross-Media Retrieval

Audio Retrieval: Based On Unsupervised Learning Approach

Multidimensional Similarity Fusion for Speech Quality Assessment

Two-Stage Method for Specific Audio Retrieval.

Bridging the gap between visual and auditory feature spaces for cross-media retrieval

Audio Retrieval Based on Dynamic Range of Sub-band Energy Feature

Towards a perceptual distance metric for auditory stimuli

Using Earth Mover'S Distance for Audio Clip Retrieval

Open-Vocabulary Retrieval of Spoken Content with Shorter/Longer Queries Considering Word/Subword-based Acoustic Feature Similarity.

Exploring composite acoustic features for efficient music similarity query.

A Comparative Study on Physical and Perceptual Features for Deepfake Audio Detection