Abstract:In the realm of multimedia data analysis, the extensive use of image datasets has escalated concerns over privacy protection within such data. Current research predominantly focuses on privacy protection either in data sharing or upon the release of trained machine learning models. Our study pioneers a comprehensive privacy protection framework that safeguards image data privacy concurrently during data sharing and model publication. We propose an interactive image privacy protection framework that utilizes generative machine learning models to modify image information at the attribute level and employs machine unlearning algorithms for the privacy preservation of model parameters. This user-interactive framework allows for adjustments in privacy protection intensity based on user feedback on generated images, striking a balance between maximal privacy safeguarding and maintaining model performance. Within this framework, we instantiate two modules: a differential privacy diffusion model for protecting attribute information in images and a feature unlearning algorithm for efficient updates of the trained model on the revised image dataset. Our approach demonstrated superiority over existing methods on facial datasets across various attribute classifications.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the issue of image data privacy protection in multimedia data analysis. Specifically, current research mainly focuses on privacy protection during the data - sharing or post - training machine - learning model release stages. However, there is a lack of a comprehensive framework that can effectively protect privacy in both of these stages simultaneously. This paper proposes an interactive image privacy - protection framework based on diffusion models and machine - unlearning techniques, aiming to solve the following problems: 1. **Privacy protection during the data - sharing stage**: During the data - sharing process, how to effectively protect sensitive attribute information in images, such as gender, race, age, etc., to prevent this information from being illegally exploited or leaked. 2. **Privacy protection during the model - release stage**: During the post - training model - release process, how to ensure that the model parameters do not contain any sensitive information, especially when a user requests the deletion of their personal data, being able to efficiently remove this information from the model. 3. **Balancing privacy protection and model performance**: While strengthening privacy protection, how to maintain the performance of the model and the availability of data, avoiding affecting the accuracy and practicality of the model due to over - protection. To achieve the above goals, the paper proposes a user - interactive image privacy - protection framework, which includes two main modules: - **Differential - privacy diffusion model (Diffusion - DP)**: Generate new images through the diffusion model and add noise during the generation process to protect sensitive attribute information in the images. This method not only enhances the strength of privacy protection but also maintains the quality and availability of the images. - **Advanced machine - unlearning module (AMU)**: When it is necessary to remove specific sensitive information from the model, use machine - unlearning techniques to quickly adjust the model parameters without the need to retrain the entire model. This significantly reduces the consumption of resources and time while ensuring high - performance of the model. Through the combination of these two modules, the paper provides a comprehensive solution that can effectively protect users' privacy during both the data - sharing and model - release stages while maintaining the practicality and performance of the model. Experimental results show that this method outperforms existing methods on facial datasets and has achieved significant improvements in multiple attribute - classification tasks.

Enhancing User-Centric Privacy Protection: An Interactive Framework through Diffusion Models and Machine Unlearning

Unlearnable Examples for Diffusion Models: Protect Data from Unauthorized Exploitation

Diff-Privacy: Diffusion-based Face Privacy Protection

PAC Privacy Preserving Diffusion Models

Generation of Face Privacy-Protected Images Based on the Diffusion Model

A Personal Privacy Preserving Framework

Data Forensics in Diffusion Models: A Systematic Analysis of Membership Privacy

A Framework to Preserve User Privacy for Machine Learning as a Service

Human Action Image Generation with Differential Privacy

Protective Perturbations against Unauthorized Data Usage in Diffusion-based Image Generation

Privacy Data Diffusion Modeling and Preserving in Online Social Network

Revealing the Unseen: Guiding Personalized Diffusion Models to Expose Training Data

Enhancing Privacy Protection for Online Learning Resource Recommendation with Machine Unlearning

Differentially Private Fine-Tuning of Diffusion Models

DiffProtect: Generate Adversarial Examples with Diffusion Models for Facial Privacy Protection

Exploring Privacy and Fairness Risks in Sharing Diffusion Models: An Adversarial Perspective

Latent Diffusion Models for Attribute-Preserving Image Anonymization

Hiding among Your Neighbors: Face Image Privacy Protection with Differential Private K-Anonymity

A Novel Generation Method for Diverse Privacy Image Based on Machine Learning

Can Protective Perturbation Safeguard Personal Data from Being Exploited by Stable Diffusion?