PEPR '25: 2025 USENIX Conference on Privacy Engineering Practice and Respect

Nitin Agrawal and Laura Book, Snap Inc.


Machine learning models, in particular classification models, are used across a wide spectrum of products and applications. These models may be susceptible to attacks like model inversion and attribute inference attacks that could allow for reconstruction of the training data and re-indentification of the data subjects. However, not all models are attackable: well generalized ones specifically are less prone to memorize their training data, and privacy preserving techniques can be used to help ensure training is generalized rather than memorized. However, a key challenge at an industrial scale lies in identifying the attackability of a model as well as calibrating the need for privacy mitigations. Academic literature has established an order relationship between attacks, demonstrating that membership inference attacks are a precursor to the reconstruction and re-identification of training data. In this talk we'll discuss a mechanism to repurpose those attacks into a practical quantifiable metric for ML model attackability measurement. This could be critical in ensuring model privacy and ongoing monitoring of the model in the model deployment lifecycle.


https://www.usenix.org/conference/pepr25/presentation/agrawal