arXiv:1812.00910v1 [stat.ML] 3 Dec 2018
Comprehensive Privacy Analysis of Deep Learning
Stand-alone and Federa te d Learning under Passive and Active White-box Inference Attacks
Milad Nasr
University of Massachusetts Amher st
Reza Shokri
National University of Sing apore
Amir Houmansad r
University of Massachusetts Amher st
Abstract—Deep neural networks are susceptible to various
inference attacks as they remember information about their
training data. We perform a comprehensive analysis of white-box
privacy inference attacks on deep learning models. We measure
the privacy leakage by leveraging the final model parameters
as well as the parameter updates during the training and fine-
tuning processes. We design the attacks in the stand-alone and
federated settings, with respect to passive and active inference
attackers, and assuming different adversary prior knowledge.
We evaluate our novel white-box membership inference attacks
against deep learning algorithms to measure their training data
membership leakage. We show that a straightforward extension
of the known black-box attacks to the white-box setting (through
analyzing the outputs of activation functions) is in eff ective. We
therefore design new algorithms tailored t o the white-b ox setting
by exploiting the privacy vulnerabilities of the stochastic gradient
descent algorithm, widely used to t rain deep neural networks. We
show that even well- generalized models are significantly suscep-
tible to white-box membership inference attacks, by an alyzin g
state-of-the-art pre-trained and publicly availabl e models for t he
CIFAR dataset. We also show how ad versarial participants of a
federated learning setting can run active membership inference
attacks against other participants, even when the global model
achieves high prediction accuracies.
I. INTRODUCTION
Machine learn ing models based on deep neural networks
have been shown to have significantly high generalization
accuracies for various learning tasks, from image and spe e ch
recogn ition to generating realistic-loo king data. This success
has led to many applications and services that use de ep
learning algorithms on large-dimension (and potentially sen si-
tive) user data, including user speeches, images, and medical,
financial, social, and lo cation data points. The crucial question
we ask in this paper is the following: How much is the privacy
risks of deep learning for individuals whose data is used as
part of the training set? In other words, how much is the
informa tion leakage of deep learning a lgorithms about the ir
individual training data samples?
We define privacy-sensitive leaka ge of a model about a
set of target trained data record(s) as the information that
an adversary can infer about the data records, that could not
have been inferred from similar models n ot using the target
data records. This distinguishes between the inf ormation that
we can learn f rom the model about the population and the
informa tion that it leaks about particular samples from the
population that were in its training set. The former indicates
utility gain, and the later reflects privacy loss. We design
inference attacks to quantify such privacy leakage.
The inference attacks fall into two fundamental an d related
categories: tracing (a.k.a. member ship inference) attacks, and
reconstruction attacks [
1]. In a reconstruction attack, the
attacker’s ob je c tive is to infe r attributes of the records in the
training set [
2], [3]. In a tracing attack, however, the attacker’s
objective is to learn if a particular individual’s data was
included in the training dataset [4]–[6]. This is a decisional
problem, and its accuracy directly demonstrates the privacy-
sensitive leakage of the model about its training data . We
choose this attack as the basis for our privacy analysis.
Recent works [6], [7] presen t membership inference at-
tacks in a black-box setting, where the attacker can only
observe the model predictions. The results of these works
show that the distribution of the trainin g data as well as the
generalizability of the model significantly contribute to the
membersh ip lea kage. Particularly, they show that overfitted
models are more susceptible to mem bership inference attacks
than generalized models. Such black-box attacks, however,
might not be effective against deep neura l networks that
generalize well (having a large set of parame te rs). Besides,
deep learning algorithms are used in a variety of settings
where parameters are visible to curious adversaries, e.g., in
federated learning where multiple data hold ers collaborate to
train a global model by sharing their pa rameter updates with
each other through a central parameter aggregator.
Our contributions. In this paper, we present a comprehensive
framework for the privacy analysis of deep neural networks,
using white-box membership inference attacks. We go beyond
membersh ip inference attacks against fully-trained models.
We take all major scenarios where deep learning is u sed
for training and fine-tuning or updating models, with one
or multiple collaborative data holders, when attacker only
passively observes the model updates or actively influences
the target model to extract more information, and for attackers
with different types of prior knowledge. Desp ite differences
in knowledge, observation, and actio ns of the ad versary, the
objective is the same: membership inference.
A simple extension of existing black-box membership infer-
ence attacks to the white-box setting is to use the same attack
on all of the activation fun c tions of the model. Our empirical
evaluations show that this will not result in accuracies higher
than a black-box attacker. This is b ecause the activation
functions in the model tend to generalize mu ch faster than
the output layer. The fir st layers extract very simple features
1
剩余14页未读，继续阅读
评论0