Privacy in Machine Learning (NeurIPS 2019 Workshop)

Differential privacy has seen remarkable success as a rigorous and practical formalization of data privacy in the past decade. This privacy definition and its divergence based relaxations, however, have several acknowledged weaknesses, either in handling composition of private algorithms or in analyzing important primitives like privacy amplification by subsampling. Inspired by the hypothesis testing formulation of privacy, this paper proposes a new relaxation, which we term ❝f-differential privacy❞ (f-DP). This notion of privacy has a number of appealing properties and, in particular, avoids difficulties associated with divergence based relaxations. First, f-DP preserves the hypothesis testing interpretation. In addition, f-DP allows for lossless reasoning about composition in an algebraic fashion. Moreover, we provide a powerful technique to import existing results proven for original DP to f-DP and, as an application, obtain a simple subsampling theorem for f-DP. In addition to the above findings, we introduce a canonical single-parameter family of privacy notions within the f-DP class that is referred to as ❝Gaussian differential privacy❞ (GDP), defined based on testing two shifted Gaussians. GDP is focal among the f-DP class because of a central limit theorem we prove. More precisely, the privacy guarantees of any hypothesis testing based definition of privacy (including original DP) converges to GDP in the limit under composition. The CLT also yields a computationally inexpensive tool for analyzing the exact composition of private algorithms. Taken together, this collection of attractive properties render f-DP a mathematically coherent, analytically tractable, and versatile framework for private data analysis. Finally, we demonstrate the use of the tools we develop by giving an improved privacy analysis of noisy stochastic gradient descent.

Recently, there has been a wealth of effort devoted to the design of secure protocols for machine learning tasks. Much of this is aimed at enabling secure prediction from highly-accurate Deep Neural Networks (DNNs). However, as DNNs are trained on data, a key question is how such models can be also trained securely. The few prior works on secure DNN training have focused either on designing custom protocols for existing training algorithms or on developing tailored training algorithms and then applying generic secure protocols. In this work, we investigate the advantages of designing training algorithms alongside a novel secure protocol, incorporating optimizations on both fronts. We present QUOTIENT, a new method for discretized training of DNNs, along with a customized secure two-party protocol for it. QUOTIENT incorporates key components of state-of-the-art DNN training such as layer normalization and adaptive gradient methods, and improves upon the state-of-the-art in DNN training in two-party computation. Compared to prior work, we obtain an improvement of 50X in WAN time and 6% in absolute accuracy.

Data collected about individuals is regularly used to make decisions that impact those same individuals. We consider settings where sensitive personal data is used to decide who will receive resources or benefits. While it is well known that there is a tradeoff between protecting privacy and the accuracy of decisions, in this talk, I will describe our recent work on a first-of-its-kind empirical study into the impact of formally private mechanisms (based on differential privacy) on fair and equitable decision-making.

Accepted Papers

Clément Canonne, Gautam Kamath, Audra McMillan, Jonathan Ullman and Lydia Zakynthinou
Private Identity Testing for High-Dimensional Distributions [arxiv]

In this work we present novel differentially private identity (goodness-of-fit) testers for natural and widely studied classes of multivariate product distributions: Gaussians in ℝ^d with known covariance and product distributions over {±1}^d. Our testers have improved sample complexity compared to those derived from previous techniques, and are the first testers whose sample complexity matches the order-optimal minimax sample complexity of O(d^1/2/α²) in many parameter regimes. We construct two types of testers, exhibiting tradeoffs between sample complexity and computational complexity. Finally, we provide a two-way reduction between testing a subclass of multivariate product distributions and testing univariate distributions, and thereby obtain upper and lower bounds for testing this subclass of product distributions.

Kwang-Sung Jun and Francesco Orabona
Parameter-Free Locally Differentially Private Stochastic Subgradient Descent [arxiv]

We consider the problem of minimizing a convex risk with stochastic subgradients guaranteeing ϵ-locally differentially private (ϵ-LDP). While it has been shown that stochastic optimization is possible with ϵ-LDP via the standard SGD (Song et al., 2013), its convergence rate largely depends on the learning rate, which must be tuned via repeated runs. Further, tuning is detrimental to privacy loss since it significantly increases the number of gradient requests. In this work, we propose BANCO (Betting Algorithm for Noisy COins), the first ϵ-LDP SGD algorithm that essentially matches the convergence rate of the tuned SGD without any learning rate parameter, reducing privacy loss and saving privacy budget.

Seth Neel, Zhiwei Steven Wu, Aaron Roth and Giuseppe Vietri
Differentially Private Objective Perturbation: Beyond Smoothness and Convexity [arxiv]

One of the most effective algorithms for differentially private learning and optimization is objective perturbation. This technique augments a given optimization problem (e.g. deriving from an ERM problem) with a random linear term, and then exactly solves it. However, to date, analyses of this approach crucially rely on the convexity and smoothness of the objective function. We give two algorithms that extend this approach substantially. The first algorithm requires nothing except boundedness of the loss function, and operates over a discrete domain. Its privacy and accuracy guarantees hold even without assuming convexity. The second algorithm operates over a continuous domain and requires only that the loss function be bounded and Lipschitz in its continuous parameter. Its privacy analysis does not even require convexity. Its accuracy analysis does require convexity, but does not require second order conditions like smoothness. We complement our theoretical results with an empirical evaluation of the non-convex case, in which we use an integer program solver as our optimization oracle. We find that for the problem of learning linear classifiers, directly optimizing for 0/1 loss using our approach can out-perform the more standard approach of privately optimizing a convex-surrogate loss function on the Adult dataset.

Di Wang, Huanyu Zhang, Marco Gaboardi and Jinhui Xu
Estimating Smooth GLM in Non-interactive Local Differential Privacy Model with Public Unlabeled Data [pdf]

In this paper, we study the problem of estimating smooth Generalized Linear Models (GLM) in the Non-interactive Local Differential Privacy (NLDP) model. Different from its classical setting, our model allows the server to process some additional public but unlabeled data. By using Stein’s lemma and its variants, we first show that there is an (ϵ,δ)-NLDP algorithm for GLM (under some mild assumptions), if each data record is i.i.d sampled from some sub-Gaussian distribution with bounded ℓ₁-norm. Then with high probability, the sample complexity of public and private data, for the algorithm to achieve an α estimation error (in ℓ_∞-norm), is O(p²α⁻²) and O(p²α⁻²ϵ⁻²) respectively if α is not too small (i.e., α ≥ Ω(1/√p)), where p is the dimensionality of the data. This is a significant improvement over the previously known quasi-polynomial (in α) or exponential (in p) complexity of GLM with no public data. We demonstrate the effectiveness of our algorithms through experiments on both synthetic and real world datasets.

Liwei Song, Reza Shokri and Prateek Mittal
Privacy vs Robustness (against Adversarial Examples) in Machine Learning [pdf]

Research into challenges of machine learning models typically considers the security domain and the privacy domain separately. It is thus unclear whether the defenses in one domain will have any unexpected impact on the other domain. In this paper, we combine two domains together by investigating the interplay between membership inference and robustness against adversarial examples in machine learning. By performing membership inference attacks against both robust models and natural (undefended) models, we find that the adversarial defense methods, although increase the model robustness against adversarial examples, also make the model more vulnerable to membership inference attacks, indicating a potential conflict between privacy and robustness in machine learning.

Jonathan Lebensold, William Hamilton, Borja Balle and Doina Precup
Actor Critic with Differentially Private Critic [arxiv]

Reinforcement learning algorithms are known to be sample inefficient, and often performance on one task can be substantially improved by leveraging information (e.g., via pre-training) on other related tasks. In this work, we propose a technique to achieve such knowledge transfer in cases where agent trajectories contain sensitive or private information, such as in the healthcare domain. Our approach leverages a differentially private policy evaluation algorithm to initialize an actor-critic model and improve the effectiveness of learning in downstream tasks. We empirically show this technique increases sample efficiency in resource-constrained control problems while preserving the privacy of trajectories collected in an upstream task.

Samyadeep Basu, Rauf Izmailov and Chris Mesterharm
Membership Model Inversion Attacks for Deep Networks [arxiv]

With the increasing adoption of AI, inherent security and privacy vulnerabilities for machine learning systems are being discovered. One such vulnerability makes it possible for an adversary to obtain private information about the types of instances used to train the targeted machine learning model. This so-called model inversion attack is based on sequential leveraging of classification scores towards obtaining high confidence representations for various classes. However, for deep networks, such procedures usually lead to unrecognizable representations that are useless for the adversary. In this paper, we introduce a more realistic definition of model inversion, where the adversary is aware of the general purpose of the attacked model (for instance, whether it is an OCR system or a facial recognition system), and the goal is to find realistic class representations within the corresponding lower-dimensional manifold (of, respectively, general symbols or general faces). To that end, we leverage properties of generative adversarial networks for constructing a connected lower-dimensional manifold, and demonstrate the efficiency of our model inversion attack that is carried out within that manifold.

Gautam Kamath, Janardhan Kulkarni, Zhiwei Steven Wu and Huanyu Zhang
Privately Learning Markov Random Fields [pdf]

We consider the problem of learning Markov Random Fields, particularly the special case of the Ising model, under the constraint of differential privacy. This includes both structure learning, where we try to estimate the underlying graph structure of the model, as well as the harder goal of parameter learning, in which we additionally estimate the parameter on each clique. We provide algorithms and lower bounds for both problems under a variety of privacy models. While the non-private sample complexity bounds for these two problems are both logarithmic in the dimension, we show that this is not the case under differential privacy. In particular, we investigate the sample complexity bounds for both problems under the constraints of pure, approximate, and concentrated differential privacy. We show that only structure learning under approximate differential privacy has logarithmic sample complexity in the dimension, while a change in either the learning goal or the privacy notion would necessitate a polynomial dependence. This suggests that if we are operating on very high-dimensional data, the aforementioned setting is the only one which is tractable.

Lovedeep Gondara and Ke Wang
Differentially Private Survival Function Estimation [arxiv]

Survival function estimation is used in many disciplines, but it is most common in medical analytics in the form of the Kaplan-Meier estimator. Sensitive data (patient records) is used in the estimation without any explicit control on the information leakage, which is a significant privacy concern. We propose a first differentially private estimator of the survival function and show that it can be easily extended to provide differentially private confidence intervals and test statistics without spending any extra privacy budget. We further provide extensions for differentially private estimation of the competing risk cumulative incidence function. Using nine real-life clinical datasets, we provide empirical evidence that our proposed method provides good utility while simultaneously providing strong privacy guarantees.

Ang Li, Jiayi Guo, Huanrui Yang and Yiran Chen
DeepObfuscator: Adversarial Training Framework for Privacy-Preserving Image Classification [arxiv]

Deep learning has been widely utilized in many computer vision applications and achieved remarkable commercial success. However, running deep learning models on mobile devices is generally challenging due to limitation of the available computing resources. It is common to let the users send their service requests to cloud servers that run the large-scale deep learning models to process. Sending the data associated with the service requests to the cloud, however, impose risks on the user data privacy. Some prior arts proposed sending the features extracted from raw data (e.g., images) to the cloud. Unfortunately, these extracted features can still be exploited by attackers to recover raw images and to infer embedded private attributes (e.g., age, gender, etc.). In this paper, we propose an adversarial training framework DeepObfuscator that can prevent extracted features from being utilized to reconstruct raw images and infer private attributes, while retaining the useful information for the intended cloud service (i.e., image classification). DeepObfuscator includes a learnable encoder, namely, obfuscator that is designed to hide privacy-related sensitive information from the features by performing our proposed adversarial training algorithm. The proposed algorithm is designed by simulating the game between an attacker who makes efforts to reconstruct raw images and infer private attributes from the extracted features and a defender who aims to protect user privacy. Our experiments on CelebA dataset show that the quality of the reconstructed images from the obfuscated features of the raw image is dramatically decreased from 0.9458 to 0.3175 in terms of multi-scale structural similarity (MS-SSIM). The person in the reconstructed image, hence, becomes hardly to be re-identified. The classification accuracy of the inferred private attributes that can be achieved by the attacker drops down to a random-guessing level, e.g., the accuracy of gender is reduced from 97.36% to 58.85%. As a comparison, the accuracy of the intended classification tasks performed via the cloud service drops by only 2%.

Fatemehsadat Mireshghallah, Mohammadkazem Taram, Prakash Ramrakhyani, Dean Tullsen and Hadi Esmaeilzadeh
Shredder: Learning Noise Distributions to Protect Inference Privacy [arxiv]

Sheer amount of computation in deep neural networks has pushed their execution to the cloud. This de facto cloud-hosted inference, however, raises serious privacy concerns as private data is communicated and stored in remote servers. The data could be mishandled by cloud providers, used for unsolicited analytics, or simply compromised through network and system security vulnerability. To that end, this paper devises SHREDDER that reduces the information content of the communicated data without diminishing the cloud's ability to maintain acceptably high accuracy. To that end, SHREDDER learns two sets of noise distributions whose samples, named multiplicative and additive noise tensors, are applied to the communicated data while maintaining the inference accuracy. The key idea is that SHREDDER learns these noise distributions offline without altering the topology or the weights of the pre-trained network. SHREDDER repeatedly learns sample noise tensors from the distributions by casting the tensors as a set of trainable parameters while keeping the weights constant. Since the key idea is learning the noise, we are able to devise a loss function that strikes a balance between accuracy and information degradation. To this end, we use self-supervision to train the noise tensors to achieve an intermediate representation of the data that contains less private information. Experimentation with real-world deep neural networks shows that, compared to the original execution, SHREDDER reduces the mutual information between the input and the communicated data by 66.90%, and yields a misclassification rate of 94.5% over private labels, significantly reducing adversary's ability to infer private data, while sacrificing only 1.74% loss in accuracy without any knowledge about the private labels.

Jinshuo Dong, Aaron Roth and Weijie Su
Gaussian Differential Privacy (contributed talk) [arxiv]

Andres Munoz, Umar Syed, Sergei Vassilvitskii and Ellen Vitercik
Private Linear Programming Without Constraint Violations [pdf]

We show how to solve linear programs whose constraints depend on private data. Existing techniques allow constraint violations whose magnitude is bounded interms of the differential privacy parameters ϵ and δ. In many applications, however, the constraints cannot be violated under any circumstances. We demonstrate that straightforward applications of common differential privacy tools, such as Laplace noise and the exponential mechanism, are inadequate for guaranteeing that the constraints are satisfied. We then present a new differentially private mechanism that takes as input a linear program and releases a solution that satisfies the constraints and differs from the optimal solution by only a small amount. Empirically, we show that alternative mechanisms do violate constraints in practice.

Hafiz Imtiaz, Jafar Mohammadi and Anand D. Sarwate
Correlation-Assisted Distributed Differentially Private Estimation [arxiv]

Many applications of machine learning, such as human health research, involve processing private or sensitive information. Privacy concerns may impose significant hurdles to collaboration in scenarios where there are multiple sites holding data and the goal is to estimate properties jointly across all datasets. Differentially private decentralized algorithms can provide strong privacy guarantees. However, the accuracy of the joint estimates may be poor when the datasets at each site are small. This paper proposes a new framework, Correlation Assisted Private Estimation (CAPE), for designing privacy-preserving decentralized algorithms with better accuracy guarantees in an honest-but-curious model. CAPE can be used in conjunction with the functional mechanism for statistical and machine learning optimization problems. A tighter characterization of the functional mechanism is provided that allows CAPE to achieve the same performance as a centralized algorithm in the decentralized setting using all datasets. Empirical results on regression and neural network problems for both synthetic and real datasets show that differentially private methods can be competitive with non-private algorithms in many scenarios of interest.

Naoise Holohan, Stefano Braghin, Pol Mac Aonghusa and Killian Levacher
Diffprivlib: The IBM Differential Privacy Library [arxiv]

Since its conception in 2006, differential privacy has emerged as the de-facto standard in data privacy, owing to its robust mathematical guarantees, generalised applicability and rich body of literature. Over the years, researchers have studied differential privacy and its applicability to an ever-widening field of topics. Mechanisms have been created to optimise the process of achieving differential privacy, for various data types and scenarios. Until this work however, all previous work on differential privacy has been conducted on a ad-hoc basis, without a single, unifying codebase to implement results. In this work, we present the IBM Differential Privacy Library, a general purpose, open source library for investigating, experimenting and developing differential privacy applications in the Python programming language. The library includes a host of mechanisms, the building blocks of differential privacy, alongside a number of applications to machine learning and other data analytics tasks. Simplicity and accessibility has been prioritised in developing the library, making it suitable to a wide audience of users, from those using the library for their first investigations in data privacy, to the privacy experts looking to contribute their own models and mechanisms for others to use.

Antti Koskela, Joonas Jälkö and Antti Honkela
Computing Exact Guarantees for Differential Privacy [arxiv]

Differentially private (DP) machine learning has recently become popular. The privacy loss of DP algorithms is commonly reported using (ε,δ)-DP. In this paper, we propose a numerical accountant for evaluating the privacy loss for algorithms with continuous one dimensional output. This accountant can be applied to the subsampled multidimensional Gaussian mechanism which underlies the popular DP stochastic gradient descent. The proposed method is based on a numerical approximation of an integral formula which gives the exact (ε,δ)-values. The approximation is carried out by discretising the integral and by evaluating discrete convolutions using the fast Fourier transform algorithm. We give both theoretical error bounds and numerical error estimates for the approximation. Experimental comparisons with state-of-the-art techniques demonstrate significant improvements in bound tightness and/or computation time. Python code for the method can be found in Github.

Joonas Jälkö, Antti Honkela and Samuel Kaski
Privacy-Preserving Data Sharing via Probabilistic Modelling [pdf]

Differential privacy allows quantifying privacy loss from computations using sensitive personal data. This loss grows with the number of accesses to the data, making it hard to open the use of such data while respecting privacy. Instead of accessing the data multiple times, we propose a method of fitting a probabilistic model on the data using privacy preserving modelling techniques. From this probabilistic model we sample a new synthetic dataset that may be subjected to unlimited amount of future analysis, without affecting the privacy guarantees. We demonstrate empirically that similar statistical discoveries can be made from the synthetic as the original data. We expect the method to have broad use in sharing anonymized versions of key data sets for research.

Nitin Agrawal, Ali Shahin Shamsabadi, Matthew Kusner and Adria Gascon
QUOTIENT: Two-Party Secure Neural Network Training and Prediction (contributed talk) [arxiv]

Dingfan Chen, Ning Yu, Yang Zhang and Mario Fritz
GAN-Leaks: A Taxonomy of Membership Inference Attacks against GANs [pdf]

In recent years, deep learning has achieved overwhelming success, spanning from discriminative models to generative models. In particular, generative adversarial networks (GANs) have facilitated a new level of performance in a myriad of areas, ranging from media manipulation to sanitized dataset generation. Despite the great success, the potential risks of privacy breach caused by GANs have not been analyzed systematically. In this paper, we focus on membership inference attack against GANs that reveals information about the training data used for victim models. Specifically, we present the first taxonomy of membership inference attacks, comprising not only existing attacks but also our novel ones. In addition, we propose the first generic attack model that can be instantiated in a large range of settings according to the adversary’s knowledge about the victim models. We complement the systematic analysis of attack performance by a comprehensive experimental study, that investigates the effectiveness of various attacks w.r.t. model type and training configurations, over three diverse application scenarios (i.e., images, medical data, and location data).

Si Kai Lee, Luigi Gresele, Mijung Park and Krikamol Muandet
Private Causal Inference using Propensity Scores [arxiv]

The use of inverse probability weighting (IPW) methods to estimate the causal effect of treatments from observational studies is widespread in econometrics, medicine and social sciences. Although these studies often involve sensitive information, thus far there has been no work on privacy-preserving IPW methods. We address this by providing a novel framework for privacy-preserving IPW (PP-IPW) methods. We include a theoretical analysis of the effects of our proposed privatisation procedure on the estimated average treatment effect, and evaluate our PP-IPW framework on synthetic, semi-synthetic and real datasets. The empirical results are consistent with our theoretical findings.

Kareem Amin, Matthew Joseph and Jieming Mao
Pan-Private Uniformity Testing (contributed talk) [arxiv]

A centrally differentially private algorithm maps raw data to differentially private outputs. In contrast, a locally differentially private algorithm may only access data through public interaction with data holders, and this interaction must be a differentially private function of the data. We study the intermediate model of pan-privacy. Unlike a locally private algorithm, a pan-private algorithm receives data in the clear. Unlike a centrally private algorithm, the algorithm receives data one element at a time and must maintain a differentially private internal state while processing this stream. First, we show that pan-privacy against multiple intrusions on the internal state is equivalent to sequentially interactive local privacy. Next, we contextualize pan-privacy against a single intrusion by analyzing the sample complexity of uniformity testing over domain [k]. Focusing on the dependence on k, centrally private uniformity testing has sample complexity Θ(√k), while noninteractive locally private uniformity testing has sample complexity Θ(k). We show that the sample complexity of pan-private uniformity testing is Θ(k^2/3). By a new Ω(k) lower bound for the sequentially interactive setting, we also separate pan-private from sequentially interactive locally private and multi-intrusion pan-private uniformity testing.

Ios Kotsogiannis, Yuchao Tao, Xi He, Ashwin Machanavajjhala, Michael Hay and Gerome Miklau
PrivateSQL: A Differentially Private SQL Query Engine [pdf]

Differential privacy is considered a de facto standard for private data analysis. However, the definition and much of the supporting literature applies to flat tables. While there exist variants of the definition and specialized algorithms for specific types of relational data (e.g. graphs), there isn’t a general privacy definition for multi-relational schemas with constraints, and no system that permits accurate differentially private answering of SQL queries while imposing a fixed privacy budget across all queries posed by the analyst. This work presents PrivateSQL, a first-of-its-kind end-to-end differentially private relational database system. PrivateSQL allows an analyst to query data stored in a standard database management system using a rich class of SQL counting queries. PrivateSQL adopts a novel generalization of differential privacy to multi-relational data that takes into account constraints in the schema like foreign keys, and allows the data owner to flexibly specify entities in the schema that need privacy. PrivateSQL ensures a fixed privacy loss across all the queries posed by the analyst by answering queries on private synopses generated from several views over the base relation that are tuned to have low error on a representative query workload. We experimentally evaluate PrivateSQL on a real-world dataset and a work-load of more than 3,600 queries. We show that for 50% of the queries PrivateSQL offers at least 1,000x better error rates than solutions adapted from prior work.

Chao Jin, Ahmad Qaisar Ahmad Al Badawi, Balagopal Unnikrishnan, Jie Lin, Fook Mun Chan, James Brown, J. Peter Campbell, Michael F. Chiang, Jayashree Kalpathy-Cramer, Vijay Chandrasekhar, Pavitra Krishnaswamy and Khin Mi Mi Aung
CareNets: Efficient Homomorphic CNN for High Resolution Images [pdf]

Deep learning as a service paradigms are increasingly employed for image-based applications spanning surveillance, healthcare, biometrics, and e-commerce. Typically, trained convolutional neural networks (CNNs) are hosted on cloud infrastructure, and applied for inference on input images. There is interest in approaches to enhance data privacy and security in such settings. Fully homomorphic encryption (FHE) can address this need as it caters to computations on encrypted data, but poses intensive computational burden. Prior works have proposed approaches to alleviate this burden for 32×32 images, but practical applications require at least 10X higher resolution. Here, we present CareNets: Compact and Resource Efficient CNN for homomorphic inference on encrypted high-resolution images. Our approach is based on a novel compact packing scheme that packs CNN inputs, weights and activations densely into HE ciphertexts; and integrates them into the CNN computation flow. We implement CareNets using a GPU-accelerated FHE library for CNN inference on encrypted retinal images of size 96×96 and 256×256. Our results show that CareNets achieves over 32.78X speedup, 45X improvement in memory efficiency, and 5851X reduction in transferred message size while maintaining accuracy within 3% of the non-encrypted CNN baselines.

Alexandra Schofield, Gregory Yauney and David Mimno
Combatting The Challenges of Local Privacy for Distributional Semantics with Compression [pdf]

Traditional methods for adding locally private noise to bag-of-words features overwhelm the true signal in the text data, removing the properties of sparsity and non-negativity often relied upon by distributional semantic models. We argue the formulation of limited-precision local privacy, which guarantees privacy between documents of less than a user-specified maximum distance, is a more appropriate framework for bag-of-words features. To reduce the number of features to which we must add random noise, we also compress word features before adding noise, then decompress those features before model inference. We test randomized methods of aggregation as well as methods informed by distributional properties of words. Applying LDA and LSA to synthetic and real data, we show that these approaches produce distributional models closer to those in the original data.

Vitaly Feldman, Tomer Koren and Kunal Talwar
Private Stochastic Convex Optimization: Optimal Rates in Linear Time (contributed talk) [pdf]

We study differentially private (DP) algorithms for stochastic convex optimization: the problem of minimizing the population loss given i.i.d. samples from a distribution over convex loss functions. A recent work of Bassily et al. (2019) has established the optimal bound on the excess population loss achievable given n samples. Unfortunately, their algorithm achieving this bound is relatively inefficient: it requires O(min{n^3/2,n^5/2/d}) gradient computations, where d is the dimension of the optimization problem. We describe two new techniques for deriving DP convex optimization algorithms both achieving the optimal bound on excess loss and using O(min{n,n²/d}) gradient computations. In particular, the algorithms match the running time of the optimal non-private algorithms. The first approach relies on the use of variable batch sizes and is analyzed using the privacy amplification by iteration technique of Feldman et al. (2018). The second approach is based on a general reduction to the problem of localizing an approximately optimal solution with differential privacy. Such localization, in turn, can be achieved using existing (non-private) uniformly stable optimization algorithms. As in the earlier work, our algorithms require a mild smoothness assumption. We also give a linear-time algorithm achieving the optimal bound on the excess loss for the strongly convex case, as well as a faster algorithm for the non-smooth case.

Amrita Roy Chowdhury, Chenghong Wang, Xi He, Ashwin Machanavajjhala and Somesh Jha
Cryptϵ: Crypto-Assisted Differential Privacy on Untrusted Servers [pdf]

In this work, we propose, Cryptϵ, a system and programming framework that (1) achieves the accuracy guarantees and algorithmic expressibility of the central model (2) without any trusted data collector like in the local model. Cryptϵ achieves the “best of both worlds” by employing two non-colluding untrusted servers that run DP programs on encrypted data from the data owners. Although straightforward implementations of DP programs using secure computation tools can achieve the above goal theoretically, in practice they are beset with many challenges such as poor performance and tricky security proofs. To this end, Cryptϵ allows data analysts to author logical DP programs that are automatically translated to secure protocols that work on encrypted data. These protocols ensure that the untrusted servers learn nothing more than the noisy outputs, thereby guaranteeing ϵ-DP for all Cryptϵ programs. Cryptϵ supports a rich class of DP programs that can be expressed via a small set of transformation and measurement operators followed by arbitrary post-processing. Further, we propose performance optimizations leveraging the fact that the output is noisy. We demonstrate Cryptϵ’s feasibility for practical DP analysis with extensive empirical evaluations on real datasets.

Jiaming Xu and Dana Yang
Optimal Query Complexity of Private Sequential Learning [arxiv]

Motivated by privacy concerns in many practical applications such as Federated Learning, we study a stylized private sequential learning problem: a learner tries to estimate an unknown scalar value, by sequentially querying an external database and receiving binary responses; meanwhile, a third-party adversary observes the learner's queries but not the responses. The learner's goal is to design a querying strategy with the minimum number of queries (optimal query complexity) so that she can accurately estimate the true value, while the adversary even with the complete knowledge of her querying strategy cannot. Prior work has obtained both upper and lower bounds on the optimal query complexity, however, these upper and lower bounds have a large gap in general. In this paper, we construct new querying strategies and prove almost matching upper and lower bounds, providing a complete characterization of the optimal query complexity as a function of the estimation accuracy and the desired levels of privacy.

Benjamin Spector, Andrew Tomkins and Ravi Kumar
Preventing Adversarial Use of Datasets through Fair Core-set Construction [arxiv]

We propose improving the privacy properties of a dataset by publishing only a strategically chosen "core-set" of the data containing a subset of the instances. The core-set allows strong performance on primary tasks, but forces poor performance on unwanted tasks. We give methods for both linear models and neural networks and demonstrate their efficacy on data.

Nhathai Phan, My Thai, Devu Shila and Ruoming Jin
Differentially Private Lifelong Learning [pdf]

In this paper, we aim to develop a novel mechanism to preserve differential privacy (DP) in lifelong learning (L2M) for deep neural networks. Our key idea is to employ functional perturbation approaches in an original algorithm to preserve DP in both learning new tasks and memorizing acquired tasks in the past. Theoretical analysis shows that our mechanism significantly tighten the privacy loss, by avoiding the privacy budget accumulated in the continual learning and memorizing processes. Thorough evaluations show the effectiveness of our mechanism in preserving DP in L2M. Our study opens a new research avenue by uncovering the trade-off among privacy loss, model utility, and computational efficiency in L2M.

Alessandro Epasto, Hossein Esfandiari, Vahab Mirrokni, Andreas Munoz Medina, Umar Syed and Sergei Vassilvitskii
Anonymizing List Data [pdf]

We provide novel approximation algorithms for anonymizing list data. List data is ubiquitous in ML applications, for instance in graph adjacency lists or in sparse matrix representations. We introduce a novel notion of anonymity in list datasets, and present polynomial time approximation algorithm for anonymizing such datasets. We show how this notion anonymity allows us to improve over current approximation results for k-anonymization.

Mimansa Jaiswal and Emily Mower Provost
Privacy Enhanced Multimodal Neural Representations for Emotion Recognition [arxiv]

Many mobile applications and virtual conversational agents now aim to recognize and adapt to emotions. To enable this, data are transmitted from users' devices and stored on central servers. Yet, these data contain sensitive information that could be used by mobile applications without user's consent or, maliciously, by an eavesdropping adversary. In this work, we show how multimodal representations trained for a primary task, here emotion recognition, can unintentionally leak demographic information, which could override a selected opt-out option by the user. We analyze how this leakage differs in representations obtained from textual, acoustic, and multimodal data. We use an adversarial learning paradigm to unlearn the private information present in a representation and investigate the effect of varying the strength of the adversarial component on the primary task and on the privacy metric, defined here as the inability of an attacker to predict specific demographic information. We evaluate this paradigm on multiple datasets and show that we can improve the privacy metric while not significantly impacting the performance on the primary task. To the best of our knowledge, this is the first work to analyze how the privacy metric differs across modalities and how multiple privacy concerns can be tackled while still maintaining performance on emotion recognition.

Mrinank Sharma, Michael Hutchinson, Siddharth Swaroop, Antti Honkela and Richard Turner
Differentially Private Federated Variational Inference [arxiv]

In many real-world applications of machine learning, data are distributed across many clients and cannot leave the devices they are stored on. Furthermore, each client's data, computational resources and communication constraints may be very different. This setting is known as federated learning, in which privacy is a key concern. Differential privacy is commonly used to provide mathematical privacy guarantees. This work, to the best of our knowledge, is the first to consider federated, differentially private, Bayesian learning. We build on Partitioned Variational Inference (PVI) which was recently developed to support approximate Bayesian inference in the federated setting. We modify the client-side optimisation of PVI to provide an (ϵ, δ)-DP guarantee. We show that it is possible to learn moderately private logistic regression models in the federated setting that achieve similar performance to models trained non-privately on centralised data.

Hassan Takabi, Robert Podschwadt, Jeff Druce, Curt Wu and Kevin Procopio
Privacy preserving Neural Network Inference on Encrypted Data with GPUs [arxiv]

Machine Learning as a Service (MLaaS) has become a growing trend in recent years and several such services are currently offered. MLaaS is essentially a set of services that provides machine learning tools and capabilities as part of cloud computing services. In these settings, the cloud has pre-trained models that are deployed and large computing capacity whereas the clients can use these models to make predictions without having to worry about maintaining the models and the service. However, the main concern with MLaaS is the privacy of the client's data. Although there have been several proposed approaches in the literature to run machine learning models on encrypted data, the performance is still far from being satisfactory for practical use. In this paper, we aim to accelerate the performance of running machine learning on encrypted data using combination of Fully Homomorphic Encryption (FHE), Convolutional Neural Networks (CNNs) and Graphics Processing Units (GPUs). We use a number of optimization techniques, and efficient GPU-based implementation to achieve high performance. We evaluate a CNN whose architecture is similar to AlexNet to classify homomorphically encrypted samples from the Cars Overhead With Context (COWC) dataset. To the best of our knowledge, it is the first time such a complex network and large dataset is evaluated on encrypted data. Our approach achieved reasonable classification accuracy of 95% for the COWC dataset. In terms of performance, our results show that we could achieve several thousands times speed up when we implement GPU-accelerated FHE operations on encrypted floating point numbers.

Casey Meehan and Kamalika Chaudhuri
Location Trace Privacy Under Conditional Priors [arxiv]

Providing meaningful privacy to users of location based services is particularly challenging when multiple locations are revealed in a short period of time. This is primarily due to the tremendous degree of dependence that can be anticipated between points. We propose a Rényi differentially private framework for bounding expected privacy loss for conditionally dependent data. Additionally, we demonstrate an algorithm for achieving this privacy under Gaussian process conditional priors. This framework both exemplifies why conditionally dependent data is so challenging to protect and offers a strategy for preserving privacy to within a fixed radius for every user location in a trace.

Zhengli Zhao, Nicolas Papernot, Sameer Singh, Neoklis Polyzotis and Augustus Odena
Improving Differentially Private Models via Active Learning [arxiv]

Broad adoption of machine learning techniques has increased privacy concerns for models trained on sensitive data such as medical records. Existing techniques for training differentially private (DP) models give rigorous privacy guarantees, but applying these techniques to neural networks can severely degrade model performance. This performance reduction is an obstacle to deploying private models in the real world. In this work, we improve the performance of DP models by fine-tuning them through active learning on public data. We introduce two new techniques - DIVERSEPUBLIC and NEARPRIVATE - for doing this fine-tuning in a privacy-aware way. For the MNIST and SVHN datasets, these techniques improve state-of-the-art accuracy for DP models while retaining privacy guarantees.

Hsiang Hsu, Shahab Asoodeh and Flavio Calmon
Discovering Information-Leaking Samples and Features [pdf]

Discovering samples or features which leak information about correlated private data is a key challenge in designing context-aware privacy mechanisms. In this paper, we propose a framework to discover information-leaking samples and features based on an information-theoretic quantity known as the information density. We provide an estimator, TIDE, to approximate the thresholded information density with a provable sample complexity. Our framework is then validated on two real-world datasets providing evidence that the TIDE can potentially be used as a building block to design privacy mechanisms targeting only those information-leaking samples and features.

Martine De Cock, Rafael Dowsley, Anderson Nascimento, Davis Railsback, Jianwei Shen and Ariel Todoki
Fast Secure Logistic Regression for High Dimensional Gene Data [pdf]

When collaboratively training Machine Learning (ML) models with Secure Multiparty Computation (SMC), the price paid for keeping the data of the parties private, is an increase in computational cost and runtime. A careful choice of ML techniques, algorithmic and implementation optimizations are a necessity to enable practical secure ML over distributed datasets. Such optimizations can be tailored to the kind of data and ML problem at hand. To the best of our knowledge, we present the fastest existing SMC implementation for training logistic regression models on high dimensional data. For our largest dataset, we train a model that requires over 7 billion secure multiplications; the training completes in about 2 hours in a local area network.

Giuseppe Vietri, Grace Tian, Mark Bun, Thomas Steinke and Steven Wu
New Oracle-Efficient Algorithms for Private Synthetic Data Release [pdf]

We present three new algorithms for constructing differentially private synthetic data—a sanitized version of a sensitive dataset that approximately preserves the answers to a large collection of statistical queries. All three algorithms are oracle-efficient in the sense that they are computationally efficient given access to an optimization oracle, which can implemented using many existing (non-private) sophisticated optimization tools such as integer program solvers. While the accuracy of the synthetic data is contingent on the oracle’s optimization performance, the algorithms satisfy differential privacy even in the worst case. For all three algorithms, we provide theoretical results as well as preliminary empirical evaluation, which shows that the algorithms can efficiently and accurately answer a large collection of queries on the Adult dataset.

Abraham Flaxman
Empirical quantification of privacy loss with examples relevant to the 2020 US Census [pdf]

The 2020 US Census will use differential privacy for disclosure avoidance, employing a new algorithm called TopDown to guarantee privacy loss of at most ϵ. However, it is possible that there is some slack in the bound (which is proven using the sequential composition theorem), and in practice, the privacy loss will be substantially less than ϵ. In this paper, I develop an empirical measure of privacy loss, and apply it to three example algorithms, inspired by some aspects of TopDown, to better understand how the empirical privacy loss might compare to the theoretical guarantee. My results suggest that (1) it is possible to quantify privacy loss empirically in a reasonable amount of time, at least for counting algorithms like TopDown; and (2) it is likely that the empirical privacy loss of hierarchical counting algorithms like TopDown is substantially lower than the privacy bound derived from the serial composition theorem.

Shadi Rahimian, Tribhuvanesh Orekondy and Mario Fritz
Differential Privacy Defenses and Sampling Attacks for Membership Inference [pdf]

Machine learning models are commonly trained on sensitive and personal data such as pictures, medical records, financial records, etc. A serious breach of the privacy of this training set occurs when an adversary is able to decide whether or not a specific data point in her possession was used to train a model. We protect the model by adding noise to the gradients and as an alternative method, add noise only to the logits to protect the output of the model and evaluate the effect of these two differentially-private techniques against membership inference attacks on 5 diverse datasets. While all previous membership inference attacks rely on access to the posterior probabilities, we present the first attack which only relies on the predicted class label - yet shows high success rate.

Julius Adebayo, Hal Abelson and Danny Weitzner
Tensions Between Differential Privacy and Local Explanations for Deep Neural Networks [pdf]

As deep learning models are beginning to be applied in high stakes and consequential settings, there has been increased interest in interpretability methods for gaining insight into these models. Similarly, privacy is also of paramount importance in these settings. Here, we examine tradeoffs that occur between feature attribution maps, an increasingly popular medium to provide interpretations, and model privacy. Attribution maps reveal information about the features of the input that have the most relevance to the output of a model. On the other hand, differential privacy (DP) seeks to protect against learning information about individual inputs. Consequently, both requirements seem to be at odds. More distressingly, Shokri et al.[1] recently showed that one can infer training set membership using attribution maps. Towards this end, we consider attribution maps derived from models trained to be differentially private. Empirically, we find that there is a trade-off between privacy and interpretability. Attribution maps that seek to reconstruct the input are not sensitive to the privacy level of the model, and consequently reveal the input. On the other hand, attributions that show sensitivity to model privacy degrade sharply in visual coherence.

Amos Beimel, Aleksandra Korolova, Kobbi Nissim, Or Sheffet and Uri Stemmer
The Power of Synergy in Differential Privacy: Combining a Small Curator with Local Randomizers [pdf]

Motivated by the desire to bridge the utility gap between local and trusted curator models of differential privacy for practical applications, we initiate the theoretical study of a hybrid model introduced by “Blender” [Avent et al., USENIX Security ’17], in which differentially private protocols of n agents that work in the local-model are assisted by a differentially private curator that has access to the data of m additional users. We focus on the regime where m ≪ n and study the new capabilities of this (m,n)-hybrid model. We show that, despite the fact that the hybrid model adds no significant new capabilities for the basic task of simple hypothesis-testing, there are many other tasks (under a wide range of parameters) that can be solved in the hybrid model yet cannot be solved either by the curator or by the local-users separately. Moreover, we exhibit additional tasks where at least one round of interaction between the curator and the local-users is necessary – namely, no hybrid model protocol without such interaction can solve these tasks. Taken together, our results show that the combination of the local model with a small curator can become part of a promising toolkit for designing and implementing differential privacy.

8:10	Opening
8:15	Invited talk: Brendan McMahan — Privacy for Federated Learning, and Federated Learning for Privacy
More details coming soon
9:05	Gaussian Differential Privacy (contributed talk) Jinshuo Dong, Aaron Roth and Weijie Su
Differential privacy has seen remarkable success as a rigorous and practical formalization of data privacy in the past decade. This privacy definition and its divergence based relaxations, however, have several acknowledged weaknesses, either in handling composition of private algorithms or in analyzing important primitives like privacy amplification by subsampling. Inspired by the hypothesis testing formulation of privacy, this paper proposes a new relaxation, which we term ❝f-differential privacy❞ (f-DP). This notion of privacy has a number of appealing properties and, in particular, avoids difficulties associated with divergence based relaxations. First, f-DP preserves the hypothesis testing interpretation. In addition, f-DP allows for lossless reasoning about composition in an algebraic fashion. Moreover, we provide a powerful technique to import existing results proven for original DP to f-DP and, as an application, obtain a simple subsampling theorem for f-DP. In addition to the above findings, we introduce a canonical single-parameter family of privacy notions within the f-DP class that is referred to as ❝Gaussian differential privacy❞ (GDP), defined based on testing two shifted Gaussians. GDP is focal among the f-DP class because of a central limit theorem we prove. More precisely, the privacy guarantees of any hypothesis testing based definition of privacy (including original DP) converges to GDP in the limit under composition. The CLT also yields a computationally inexpensive tool for analyzing the exact composition of private algorithms. Taken together, this collection of attractive properties render f-DP a mathematically coherent, analytically tractable, and versatile framework for private data analysis. Finally, we demonstrate the use of the tools we develop by giving an improved privacy analysis of noisy stochastic gradient descent.
9:25	QUOTIENT: Two-Party Secure Neural Network Training & Prediction (contributed talk) Nitin Agrawal, Ali Shahin Shamsabadi, Matthew Kusner and Adria Gascon
Recently, there has been a wealth of effort devoted to the design of secure protocols for machine learning tasks. Much of this is aimed at enabling secure prediction from highly-accurate Deep Neural Networks (DNNs). However, as DNNs are trained on data, a key question is how such models can be also trained securely. The few prior works on secure DNN training have focused either on designing custom protocols for existing training algorithms or on developing tailored training algorithms and then applying generic secure protocols. In this work, we investigate the advantages of designing training algorithms alongside a novel secure protocol, incorporating optimizations on both fronts. We present QUOTIENT, a new method for discretized training of DNNs, along with a customized secure two-party protocol for it. QUOTIENT incorporates key components of state-of-the-art DNN training such as layer normalization and adaptive gradient methods, and improves upon the state-of-the-art in DNN training in two-party computation. Compared to prior work, we obtain an improvement of 50X in WAN time and 6% in absolute accuracy.
9:45	Coffee break
10:30	Invited talk: Ashwin Machanavajjhala — Fair Decision Making using Privacy-Protected Data
Data collected about individuals is regularly used to make decisions that impact those same individuals. We consider settings where sensitive personal data is used to decide who will receive resources or benefits. While it is well known that there is a tradeoff between protecting privacy and the accuracy of decisions, in this talk, I will describe our recent work on a first-of-its-kind empirical study into the impact of formally private mechanisms (based on differential privacy) on fair and equitable decision-making.
11:20	Spotlight talks
[Jonathan Lebensold, William Hamilton, Borja Balle and Doina Precup] Actor Critic with Differentially Private Critic (#08) [Andres Munoz, Umar Syed, Sergei Vassilvitskii and Ellen Vitercik] Private linear programming without constraint violations (#17) [Ios Kotsogiannis, Yuchao Tao, Xi He, Ashwin Machanavajjhala, Michael Hay and Gerome Miklau] PrivateSQL: A Differentially Private SQL Query Engine (#27) [Amrita Roy Chowdhury, Chenghong Wang, Xi He, Ashwin Machanavajjhala and Somesh Jha] Crypt$\epsilon$: Crypto-Assisted Differential Privacy on Untrusted Servers (#31) [Jiaming Xu and Dana Yang] Optimal Query Complexity of Private Sequential Learning (#32) [Hsiang Hsu, Shahab Asoodeh and Flavio Calmon] Discovering Information-Leaking Samples and Features (#43) [Martine De Cock, Rafael Dowsley, Anderson Nascimento, Davis Railsback, Jianwei Shen and Ariel Todoki] Fast Secure Logistic Regression for High Dimensional Gene Data (#44) [Giuseppe Vietri, Grace Tian, Mark Bun, Thomas Steinke and Steven Wu] New Oracle-Efficient Algorithms for Private Synthetic Data Release (#45)
11:30	Poster session
12:30	Lunch break
14:00	Invited talk: Lalitha Sankar — Fair Universal Representations via Generative Models and Model Auditing Guarantees
There is a growing demand for ML methods that limit inappropriate use of protected information to avoid both disparate treatment and disparate impact. In this talk, we present Generative Adversarial rePresentations (GAP) as a data-driven framework that leverages recent advancements in adversarial learning to allow a data holder to learn universal representations that decouple a set of sensitive attributes from the rest of the dataset while allowing learning multiple downstream tasks. We will briefly highlight the theoretical and practical results of GAP. In the second half of the talk we will focus on model auditing. Privacy concerns have led to the development of privacy-preserving approaches for learning models from sensitive data. Yet, in practice, models (even those learned with privacy guarantees) can inadvertently memorize unique training examples or leak sensitive features. To identify such privacy violations, existing model auditing techniques use finite adversaries defined as machine learning models with (a) access to some finite side information (e.g., a small auditing dataset), and (b) finite capacity (e.g., a fixed neural network architecture). In the second half of the talk, we present requirements under which an unsuccessful attempt to identify privacy violations by a finite adversary implies that no stronger adversary can succeed at such a task. We will do so via parameters that quantify the capabilities of the finite adversary, including the size of the neural network employed by such an adversary and the amount of side information it has access to as well as the regularity of the (perhaps privacy-guaranteeing) audited model.
14:50	Pan-Private Uniformity Testing (contributed talk) Kareem Amin, Matthew Joseph and Jieming Mao
A centrally differentially private algorithm maps raw data to differentially private outputs. In contrast, a locally differentially private algorithm may only access data through public interaction with data holders, and this interaction must be a differentially private function of the data. We study the intermediate model of pan-privacy. Unlike a locally private algorithm, a pan-private algorithm receives data in the clear. Unlike a centrally private algorithm, the algorithm receives data one element at a time and must maintain a differentially private internal state while processing this stream. First, we show that pan-privacy against multiple intrusions on the internal state is equivalent to sequentially interactive local privacy. Next, we contextualize pan-privacy against a single intrusion by analyzing the sample complexity of uniformity testing over domain [k]. Focusing on the dependence on k, centrally private uniformity testing has sample complexity Θ(√k), while noninteractive locally private uniformity testing has sample complexity Θ(k). We show that the sample complexity of pan-private uniformity testing is Θ(k^2/3). By a new Ω(k) lower bound for the sequentially interactive setting, we also separate pan-private from sequentially interactive locally private and multi-intrusion pan-private uniformity testing.
15:10	Private Stochastic Convex Optimization: Optimal Rates in Linear Time (contributed talk) Vitaly Feldman, Tomer Koren and Kunal Talwar
We study differentially private (DP) algorithms for stochastic convex optimization: the problem of minimizing the population loss given i.i.d. samples from a distribution over convex loss functions. A recent work of Bassily et al. (2019) has established the optimal bound on the excess population loss achievable given n samples. Unfortunately, their algorithm achieving this bound is relatively inefficient: it requires O(min{n^3/2,n^5/2/d}) gradient computations, where d is the dimension of the optimization problem. We describe two new techniques for deriving DP convex optimization algorithms both achieving the optimal bound on excess loss and using O(min{n,n²/d}) gradient computations. In particular, the algorithms match the running time of the optimal non-private algorithms. The first approach relies on the use of variable batch sizes and is analyzed using the privacy amplification by iteration technique of Feldman et al. (2018). The second approach is based on a general reduction to the problem of localizing an approximately optimal solution with differential privacy. Such localization, in turn, can be achieved using existing (non-private) uniformly stable optimization algorithms. As in the earlier work, our algorithms require a mild smoothness assumption. We also give a linear-time algorithm achieving the optimal bound on the excess loss for the strongly convex case, as well as a faster algorithm for the non-smooth case.
15:30	Coffee break
16:15	Invited talk: Philip Leclerc — Formal Privacy At Scale: The 2020 Decennial Census TopDown Disclosure Limitation Algorithm
To control vulnerabilities to reconstruction-abetted re-identification attacks that leverage massive external data stores and cheap computation, the U.S. Census Bureau has elected to adopt a formally private approach to disclosure limitation in the 2020 Decennial Census of Population and Housing. To this end, a team of disclosure limitation specialists have worked over the past 3 years to design and implement the U.S. Census Bureau TopDown Disclosure Limitation Algorithm (TDA). This formally private algorithm generates Persons and Households micro-data, which will then be tabulated to produce the final set of demographic statistics published as a result of the 2020 Census enumeration. In this talk, I outline the main features of TDA, describe the current iteration of the process used to help policy makers decide how to set and allocate privacy-loss budget, and outline known issues with - and intended fixes for - the current implementation of TDA.
17:05	Panel Discussion
17:55	Closing

8:10	1st block [View recording]
Watch the opening remarks, Brendan McMahan's invited talk, and the first two contributed talks.
10:30	2nd block [View recording]
Watch Ashwin Machanavajjhala's invited talk.
14:00	3rd block [View recording]
Watch Lalitha Sankar's invited talk and two more contributed talks.
16:15	4th block [View recording]
Watch Philip Leclerc's invited talk and the panel discussion.

Privacy in Machine Learning

Scope

Call For Papers & Important Dates

Instructions

Related Workshops

Invited Speakers

Schedule

View on SlidesLive

Accepted Papers

Organization

Workshop organizers

Program Committee

Accessibility