CISPA
Browse
cispa_all_3639.pdf (1.79 MB)

SSLGuard: A Watermarking Scheme for Self-supervised Learning Pre-trained Encoders

Download (1.79 MB)
conference contribution
posted on 2023-11-29, 18:20 authored by Tianshuo Cong, xinlei.he, Yang ZhangYang Zhang
Self-supervised learning is an emerging machine learning (ML) paradigm. Compared to supervised learning that leverages high-quality labeled datasets to achieve good performance, self-supervised learning relies on unlabeled datasets to pre-train powerful encoders which can then be treated as feature extractors for various downstream tasks. The huge amount of data and computational resources consumption makes the encoders themselves become a valuable intellectual property of the model owner. Recent research has shown that the ML model's copyright is threatened by model stealing attacks, which aims to train a surrogate model to mimic the behavior of a given model. We empirically show that pre-trained encoders are highly vulnerable to model stealing attacks. However, most of the current efforts of copyright protection algorithms such as fingerprinting and watermarking concentrate on classifiers. Meanwhile, the intrinsic challenges of pre-trained encoder's copyright protection remain largely unstudied. We fill the gap by proposing SSLGuard, the first watermarking algorithm for pre-trained encoders. Given a clean pre-trained encoder, SSLGuard embeds a watermark into it and outputs a watermarked version. The shadow training technique is also applied to preserve the watermark under potential model stealing attacks. Our extensive evaluation shows that SSLGuard is effective in watermark injection and verification, and is robust against model stealing and other watermark removal attacks such as pruning and finetuning.

History

Preferred Citation

Tianshuo Cong, Xinlei He and Yang Zhang. SSLGuard: A Watermarking Scheme for Self-supervised Learning Pre-trained Encoders. In: ACM Conference on Computer and Communications Security (CCS). 2022.

Primary Research Area

  • Trustworthy Information Processing

Name of Conference

ACM Conference on Computer and Communications Security (CCS)

Legacy Posted Date

2022-05-02

Open Access Type

  • Unknown

BibTeX

@inproceedings{cispa_all_3639, title = "SSLGuard: A Watermarking Scheme for Self-supervised Learning Pre-trained Encoders", author = "Cong, Tianshuo and He, Xinlei and Zhang, Yang", booktitle="{ACM Conference on Computer and Communications Security (CCS)}", year="2022", }

Usage metrics

    Categories

    No categories selected

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC