cispa_all_3490.pdf (1.92 MB)

Quantifying and Mitigating Privacy Risks of Contrastive Learning

Download (1.92 MB)
conference contribution
posted on 2023-11-29, 18:17 authored by xinlei.he, Yang ZhangYang Zhang
Data is the key factor to drive the development of machine learning (ML) during the past decade. However, high-quality data, in particular labeled data, is often hard and expensive to collect. To leverage large-scale unlabeled data, self-supervised learning, represented by contrastive learning, is introduced. The objective of contrastive learning is to map different views derived from a training sample (e.g., through data augmentation) closer in their representation space, while different views derived from different samples more distant. In this way, a contrastive model learns to generate informative representations for data samples, which are then used to perform downstream ML tasks. Recent research has shown that machine learning models are vulnerable to various privacy attacks. However, most of the current efforts concentrate on models trained with supervised learning. Meanwhile, data samples' informative representations learned with contrastive learning may cause severe privacy risks as well. In this paper, we perform the first privacy analysis of contrastive learning through the lens of membership inference and attribute inference. Our experimental results show that contrastive models trained on image datasets are less vulnerable to membership inference attacks but more vulnerable to attribute inference attacks compared to supervised models. The former is due to the fact that contrastive models are less prone to overfitting, while the latter is caused by contrastive models' capability of representing data samples expressively. To remedy this situation, we propose the first privacy-preserving contrastive learning mechanism, Talos, relying on adversarial training. Empirical results show that Talos can successfully mitigate attribute inference risks for contrastive models while maintaining their membership privacy and model utility.


Preferred Citation

Xinlei He and Yang Zhang. Quantifying and Mitigating Privacy Risks of Contrastive Learning. In: ACM Conference on Computer and Communications Security (CCS). 2021.

Primary Research Area

  • Trustworthy Information Processing

Name of Conference

ACM Conference on Computer and Communications Security (CCS)

Legacy Posted Date


Open Access Type

  • Unknown


@inproceedings{cispa_all_3490, title = "Quantifying and Mitigating Privacy Risks of Contrastive Learning", author = "He, Xinlei and Zhang, Yang", booktitle="{ACM Conference on Computer and Communications Security (CCS)}", year="2021", }

Usage metrics


    No categories selected


    Ref. manager