Privacy attacks in machine learning

Feb 28, 2024 5:37 PM
Main page

Diving into the world of machine learning, we explore a critical aspect: privacy attacks. For ML experts interested in this subject, we recommend this awesome repository containing over 200 papers on privacy attacks in machine learningLet's delve deeper into this intriguing topic. This is merely an overview of potential attacks.

Four Types of Attacks

Maria Rigaki and Sebastian Garcia in their Survey of Privacy Attacks in Machine Learning classify attacks by four types: membership inference reconstruction property inference, and model extraction.

Membership inference attacks which are the most common type of attack, aim to determine whether an input sample (e.g., individual) was used as part of the training set.

Reconstruction attacks aim to recreate one or more training samples and/or their respective training labels. It is worth noting that some reconstruction attacks also use publicly available data to find sensitive attributes of targeted individuals.

Property inference attacks aim to extract information that was learned from the model unintentionally and that is not related to the training task. For instance, a trained model performs gender classification and can be used to infer whether people in the training dataset wear glasses or not; however, this information was not an encoded attribute or a label of the dataset.

Model extraction attacks aim to fully reconstruct the attacked model and substitute it. This model can be used later to perform other attacks, for instance membership inference attacks, more efficiently with fewer queries.

Illustrative example of an ML attack

In 2018, Reddit users discovered a funny “glitch” in Google Translate. When you translated from a rare language to English and input dummy data, the translator returned unexpected data.


Very often Google Translate would return parts from the Bible or the Koran, but why? It is because these two books have parallel translations in almost all languages, and Google thus used them to train their translation service.

I hope you now know more about attacks in machine learning and can identify them yourself.


As we've discussed various forms of privacy attacks in machine learning, it's equally paramount to delve into potential solutions and mitigation strategies. The landscape of ML privacy protection is vast and continuously evolving. Here, we explore a set of approaches aimed at fortifying machine learning models against privacy attacks.

Differential Privacy

A leading solution in this space is the implementation of Differential Privacy (DP). Differential privacy aims to provide a way to maximize the accuracy of queries from statistical databases while minimizing the chances of identifying its entries. In machine learning, DP can be injected at various stages of model training to ensure the privacy of the training data. This is achieved by adding noise to the data or the model's parameters, thus making it difficult for attackers to pinpoint the presence of a specific individual's data in the training dataset. Tools such as Google's TensorFlow Privacy library offer practical ways to implement differential privacy in machine learning models.

Federated Learning

Another promising approach is Federated Learning (FL), which brings the model to the data, instead of the traditional method of bringing data to the centralized model. In federated learning, a model is trained across multiple decentralized devices or servers holding local data samples, without exchanging them. This process not only helps improve privacy but also reduces the central point of attack risk. However, federated learning models can still be susceptible to inference attacks, necessitating additional protective measures such as secure aggregation protocols.

Homomorphic Encryption

For scenarios demanding the use of cloud computing resources, Homomorphic Encryption (HE) provides a powerful solution. HE allows data to be encrypted in such a way that computations can be performed on the encrypted data, without ever requiring decryption. The results of such computations remain encrypted and can only be interpreted by the data owner. This approach is particularly useful in preserving privacy when outsourcing ML model training or inference tasks to third-party services.

Access Control and Anonymization

Implementing stringent Access Control measures to training datasets and model APIs can significantly reduce the risk of unauthorized access and subsequent attacks. This includes techniques like authentication, role-based access control, and auditing of access logs. Alongside, Data Anonymization techniques can help obscure the identity of individuals in training datasets, although they must be carefully balanced against the potential loss of data utility.

Continuous Monitoring and Anomaly Detection

Finally, continuous monitoring of model performance and usage patterns can help identify potential privacy attacks in real-time. Anomaly Detection systems can be configured to alert administrators upon detecting unusual query patterns that may indicate an attack, such as an excessively high rate of queries from a single source.


In conclusion, protecting machine learning models from privacy attacks requires a multifaceted approach, combining advanced cryptographic techniques with traditional data protection practices. As machine learning continues to mature and find applications across various domains, the importance of ensuring privacy and ethics in AI will only escalate. Equip yourself with knowledge, tools, and best practices in ML privacy to defend against evolving threats in this dynamic field.