Introduction to Data Privacy
Data privacy comes with a cost. There are security techniques that protect sensitive user data, like customer addresses, from attackers who may attempt to extract them from AI models — but they often make those models less accurate.
A New Framework for Data Privacy
MIT researchers recently developed a framework, based on a new privacy metric called PAC Privacy, that could maintain the performance of an AI model while ensuring sensitive data, such as medical images or financial records, remain safe from attackers. Now, they’ve taken this work a step further by making their technique more computationally efficient, improving the tradeoff between accuracy and privacy, and creating a formal template that can be used to privatize virtually any algorithm without needing access to that algorithm’s inner workings.
How PAC Privacy Works
The team utilized their new version of PAC Privacy to privatize several classic algorithms for data analysis and machine-learning tasks. They also demonstrated that more “stable” algorithms are easier to privatize with their method. A stable algorithm’s predictions remain consistent even when its training data are slightly modified. Greater stability helps an algorithm make more accurate predictions on previously unseen data.
Benefits of PAC Privacy
The researchers say the increased efficiency of the new PAC Privacy framework, and the four-step template one can follow to implement it, would make the technique easier to deploy in real-world situations. “We tend to consider robustness and privacy as unrelated to, or perhaps even in conflict with, constructing a high-performance algorithm. First, we make a working algorithm, then we make it robust, and then private. We’ve shown that is not always the right framing. If you make your algorithm perform better in a variety of settings, you can essentially get privacy for free,” says Mayuri Sridhar, an MIT graduate student and lead author of a paper on this privacy framework.
Estimating Noise
To protect sensitive data that were used to train an AI model, engineers often add noise, or generic randomness, to the model so it becomes harder for an adversary to guess the original training data. This noise reduces a model’s accuracy, so the less noise one can add, the better. PAC Privacy automatically estimates the smallest amount of noise one needs to add to an algorithm to achieve a desired level of privacy.
Improvements to PAC Privacy
The original PAC Privacy algorithm runs a user’s AI model many times on different samples of a dataset. It measures the variance as well as correlations among these many outputs and uses this information to estimate how much noise needs to be added to protect the data. This new variant of PAC Privacy works the same way but does not need to represent the entire matrix of data correlations across the outputs; it just needs the output variances. “Because the thing you are estimating is much, much smaller than the entire covariance matrix, you can do it much, much faster,” Sridhar explains.
Privacy and Stability
As she studied PAC Privacy, Sridhar hypothesized that more stable algorithms would be easier to privatize with this technique. She used the more efficient variant of PAC Privacy to test this theory on several classical algorithms. Algorithms that are more stable have less variance in their outputs when their training data change slightly. PAC Privacy breaks a dataset into chunks, runs the algorithm on each chunk of data, and measures the variance among outputs. The greater the variance, the more noise must be added to privatize the algorithm.
Conclusion
The team showed that these privacy guarantees remained strong despite the algorithm they tested, and that the new variant of PAC Privacy required an order of magnitude fewer trials to estimate the noise. They also tested the method in attack simulations, demonstrating that its privacy guarantees could withstand state-of-the-art attacks. The researchers want to explore how algorithms could be co-designed with PAC Privacy, so the algorithm is more stable, secure, and robust from the beginning.
FAQs
- Q: What is PAC Privacy?
A: PAC Privacy is a new privacy metric that can maintain the performance of an AI model while ensuring sensitive data remain safe from attackers. - Q: How does PAC Privacy work?
A: PAC Privacy adds noise to an AI model to protect sensitive data, and it automatically estimates the smallest amount of noise needed to achieve a desired level of privacy. - Q: What are the benefits of PAC Privacy?
A: The benefits of PAC Privacy include improved tradeoff between accuracy and privacy, and a formal template that can be used to privatize virtually any algorithm without needing access to that algorithm’s inner workings. - Q: Is PAC Privacy efficient?
A: Yes, the new variant of PAC Privacy is more computationally efficient and requires an order of magnitude fewer trials to estimate the noise. - Q: Can PAC Privacy be used in real-world situations?
A: Yes, the increased efficiency of the new PAC Privacy framework and the four-step template make the technique easier to deploy in real-world situations.
