Machine Learning with Privacy by Knowledge Aggregation and Transfer

Nicolas Papernot, Martín Abadi, U Erlingsson, Ian Goodfellow, Kunal Talwar

2016-01-01

Abstract:Machine learning relies on the availability of high-quality training data and—whether by its inherent nature, or by accident—this data will sometimes contain private information. When the model is to be published or made publicly accessible and the training data is not, it is important that the details of the sensitive training data cannot be inadvertently revealed by the model. This abstract presents a generally applicable approach to providing strong privacy guarantees for machine learning training data. The approach is based on combining, in a black-box fashion, multiple machine learning models trained with disjoint sensitive datasets, such as data for different users. Because they rely directly on sensitive data, these models are used only as “teachers” for a “student” machine learning model. However, when training the student, the teachers transfer only the labels upon which they all generally agree, via a noisy aggregation mechanism. The student has privacy properties that can be understood both intuitively (since no single teacher dictates the student’s training) and formally, in terms of differential privacy. These properties address “glass-box” attacks of the kind that arise if an adversary not only queries the student but also inspects its internal workings. The approach imposes only weak assumptions on how the teachers are trained. It applies to powerful, deep models, possibly with many layers and parameters. Our experiments demonstrate that the approach applies to real-world machine learning tasks, at a reasonable cost in accuracy, privacy, and software complexity.