Abstract:Deep learning (DL) models are used in a variety of real-world applications but are often vulnerable to privacy attacks. Nevertheless, this DL model is attacked by membership inference attacks, model inversion attacks, reconstruction attacks, model extraction attacks, gradient leakage attacks, correlation attacks, and white box attacks (inference attacks). In order to mitigate this issue, various existing research has attempted to design an effective privacy mechanism. However, the existing schemes failed to obtain higher security in DL because of several limitations like computational complexity, lower efficiency, difficult-to-select parameters, cumulative privacy loss, etc. Recently, the auto-encoder-based deep learning model has become more popular due to its great ability, and its variants have achieved notable success in various fields such as medicine, healthcare and NLP (Natural Language Programming). However, the privacy of the auto-encoder model is affected because of the vulnerable attacks. Thus, to avoid this issue, the proposed study prefers the differential privacy (DP) method for securing the deep auto-encoder model. DP is a privacy-preserving technique that can be used to protect deep learning models from the aforementioned attacks. In this paper, a Differential Privacy-Improved Stochastic Gradient Descent (DP-ISGD) algorithm is proposed to improve the privacy and utility of the Deep Autoencoder method by adding Gaussian noise to the gradients before the clipping process. Thus, the convergence speed and accuracy of the proposed algorithm are enhanced. The experimentation is conducted in the Python platform, and metrics like convergence, accuracy and TPVD (Total Parameters Value Difference) are evaluated to measure the performance of the proposed study. The comparative analysis is performed for the no privacy, privacy with SGD, privacy with batch gradient descent (BGD) and mini-batch gradient descent (MBGD) models. The proposed approach is evaluated against six datasets, Pima Indians Diabetes (PID), Adult, MNIST, CIFAR-10, MovieLens 20 M and CD-FSL, with improved accuracy results of 98.6%, 98.6%, 98.3%, 98.14%, 97.92% and 98.17% for each dataset at the epsilon ( ) value of 0.2. The comparison analysis showed that the proposed algorithm achieves better accuracy than other privacy protection methods. Thus, the significant findings in the proposed work state that the proposed privacy model is suitable for several applications, including medical fields, algorithm development, education and awareness, by affording strong privacy guarantees.

Adaptive differential privacy preserving based on multi‐objective optimization in deep neural networks

DP-EPSO: Differential privacy protection algorithm based on differential evolution and particle swarm optimization

Adaptive Differential Privacy Mechanism Based on Entropy Theory for Preserving Deep Neural Networks

An Adaptive and Fast Convergent Approach to Differentially Private Deep Learning

Differentially Private Deep Learning With Dynamic Privacy Budget Allocation and Adaptive Optimization

Fine-Tuning Language Models with Differential Privacy through Adaptive Noise Allocation

An Improved Privacy-Preserving Stochastic Gradient Descent Algorithm

Differentially Private Adaptive Optimization with Delayed Preconditioners

A(DP)$^2$SGD: Asynchronous Decentralized Parallel Stochastic Gradient Descent with Differential Privacy

A(DP)$^2$2SGD: Asynchronous Decentralized Parallel Stochastic Gradient Descent with Differential Privacy

Inverse optimization strategy for improved differential privacy in deep auto encoder

Differentially Private Robust ADMM for Distributed Machine Learning.

AMOUE: Adaptive modified optimized unary encoding method for local differential privacy data preservation

Differential Privacy Meets Neural Network Pruning

Deep Learning with Gaussian Differential Privacy

Discriminative Adversarial Privacy: Balancing Accuracy and Membership Privacy in Neural Networks

GReDP: A More Robust Approach for Differential Private Training with Gradient-Preserving Noise Reduction

Differential privacy may have a potential optimization effect on some swarm intelligence algorithms besides privacy-preserving

Deep Learning with Differential Privacy

Differentially Private Graph Neural Networks for Graph Classification and Its Adaptive Optimization

DPAdapter: Improving Differentially Private Deep Learning through Noise Tolerance Pre-training