Abstract:Speech emotion recognition (SER) processes speech signals to detect and characterize expressed perceived emotions. Many SER application systems often acquire and transmit speech data collected at the client-side to remote cloud platforms for inference and decision making. However, speech data carry rich information not only about emotions conveyed in vocal expressions, but also other sensitive demographic traits such as gender, age and language background. Consequently, it is desirable for SER systems to have the ability to classify emotion constructs while preventing unintended/improper inferences of sensitive and demographic information. Federated learning (FL) is a distributed machine learning paradigm that coordinates clients to train a model collaboratively without sharing their local data. This training approach appears secure and can improve privacy for SER. However, recent works have demonstrated that FL approaches are still vulnerable to various privacy attacks like reconstruction attacks and membership inference attacks. Although most of these have focused on computer vision applications, such information leakages exist in the SER systems trained using the FL technique. To assess the information leakage of SER systems trained using FL, we propose an attribute inference attack framework that infers sensitive attribute information of the clients from shared gradients or model parameters, corresponding to the FedSGD and the FedAvg training algorithms, respectively. As a use case, we empirically evaluate our approach for predicting the client's gender information using three SER benchmark datasets: IEMOCAP, CREMA-D, and MSP-Improv. We show that the attribute inference attack is achievable for SER systems trained using FL. We further identify that most information leakage possibly comes from the first layer in the SER model.

Gradient-Level Differential Privacy Against Attribute Inference Attack for Speech Emotion Recognition

Attribute Inference Attack of Speech Emotion Recognition in Federated Learning Settings

Investigating Effective Speaker Property Privacy Protection in Federated Learning for Speech Emotion Recognition

Shield Against Gradient Leakage Attacks: Adaptive Privacy-Preserving Federated Learning

A Two-Stage Differential Privacy Scheme for Federated Learning Based on Edge Intelligence

Hierarchical federated learning with global differential privacy

Adaptive Differential Privacy in Federated Learning: A Priority-Based Approach

Does Differential Privacy Really Protect Federated Learning from Gradient Leakage Attacks?

Efficient federated learning privacy preservation method with heterogeneous differential privacy

Noise-Aware Algorithm for Heterogeneous Differentially Private Federated Learning

A Fine-Grained Differentially Private Federated Learning Against Leakage from Gradients

Discriminative Feature Learning-Based Federated Lightweight Distillation Against Multiple Attacks

Performance Analysis and Optimization in Privacy-Preserving Federated Learning

AWDP-FL: An Adaptive Differential Privacy Federated Learning Framework

Enhancing Federated Learning with Adaptive Differential Privacy and Priority-Based Aggregation

Privacy-Preserving Federated Learning with Differentially Private Hyperdimensional Computing

Federated Learning with Differential Privacy for End-to-End Speech Recognition

A Communication-Efficient, Privacy-Preserving Federated Learning Algorithm Based on Two-Stage Gradient Pruning and Differentiated Differential Privacy

Towards Efficient and Privacy-Preserving Federated Deep Learning

A New Federated Learning Framework Against Gradient Inversion Attacks