CISPA
Browse
sec24summer-prepub-617-zhang-boyang.pdf (539.68 kB)

SecurityNet: Assessing Machine Learning Vulnerabilities on Public Models

Download (539.68 kB)
conference contribution
posted on 2024-03-19, 13:09 authored by Boyang ZhangBoyang Zhang, Zheng Li, Ziqing Yang, Xinlei He, M Backes, Mario FritzMario Fritz, Yang Zhang
While advanced machine learning (ML) models are deployed in numerous real-world applications, previous works demonstrate these models have security and privacy vulnerabilities. Various empirical research has been done in this field. However, most of the experiments are performed on target ML models trained by the security researchers themselves. Due to the high computational resource requirement for training advanced models with complex architectures, researchers generally choose to train a few target models using relatively simple architectures on typical experiment datasets. We argue that to understand ML models' vulnerabilities comprehensively, experiments should be performed on a large set of models trained with various purposes (not just the purpose of evaluating ML attacks and defenses). To this end, we propose using publicly available models with weights from the Internet (public models) for evaluating attacks and defenses on ML models. We establish a database, namely SecurityNet, containing 910 annotated image classification models. We then analyze the effectiveness of several representative attacks/defenses, including model stealing attacks, membership inference attacks, and backdoor detection on these public models. Our evaluation empirically shows the performance of these attacks/defenses can vary significantly on public models compared to self-trained models. We share SecurityNet with the research community and advocate researchers to perform experiments on public models to better demonstrate their proposed methods' effectiveness in the future.

History

Primary Research Area

  • Trustworthy Information Processing

Name of Conference

Usenix Security Symposium (USENIX-Security)

BibTeX

@conference{Zhang:Li:Yang:He:Backes:Fritz:Zhang:2024, title = "SecurityNet: Assessing Machine Learning Vulnerabilities on Public Models", author = "Zhang, Boyang" AND "Li, Zheng" AND "Yang, Ziqing" AND "He, Xinlei" AND "Backes, M" AND "Fritz, Mario" AND "Zhang, Yang", year = 2024, month = 8 }

Usage metrics

    Categories

    No categories selected

    Licence

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC