CISPA
Browse
96_codelmsec_benchmark_systematic.pdf (8.98 MB)

CodeLMSec Benchmark: Systematically Evaluating and Finding Security Vulnerabilities in Black-Box Code Language Models

Download (8.98 MB)
conference contribution
posted on 2024-04-03, 11:21 authored by Hossein Hajipour, Keno Hassler, Thorsten Holz, Lea Schönherr, Mario Fritz
Large language models (LLMs) for automatic code generation have achieved breakthroughs in several programming tasks. Their advances in competition-level programming problems have made them an essential pillar of AI-assisted pair programming, and tools such as GitHub Copilot have emerged as part of the daily programming workflow used by millions of developers. The training data for these models is usually collected from the Internet (e.g., from open-source repositories) and is likely to contain faults and security vulnerabilities. This unsanitized training data can cause the language models to learn these vulnerabilities and propagate them during the code generation procedure. While these models have been extensively assessed for their ability to produce functionally correct programs, there remains a lack of comprehensive investigations and benchmarks addressing the security aspects of these models. In this work, we propose a method to systematically study the security issues of code language models to assess their susceptibility to generating vulnerable code. To this end, we introduce the first approach to automatically find generated code that contains vulnerabilities in black-box code generation models. To achieve this, we present an approach to approximate inversion of the black-box code generation models based on few-shot prompting. We evaluate the effectiveness of our approach by examining code language models in generating high-risk security weaknesses. Furthermore, we establish a collection of diverse non-secure prompts for various vulnerability scenarios using our method. This dataset forms a benchmark for evaluating and comparing the security weaknesses in code language models.

History

Primary Research Area

  • Trustworthy Information Processing

Name of Conference

IEEE Conference on Secure and Trustworthy Machine Learning (SaTML)

Journal

2nd IEEE Conference on Secure and Trustworthy Machine Learning

Pages

26

Open Access Type

  • Gold

BibTeX

@conference{Hajipour:Hassler:Holz:Schönherr:Fritz:2024, title = "CodeLMSec Benchmark: Systematically Evaluating and Finding Security Vulnerabilities in Black-Box Code Language Models", author = "Hajipour, Hossein" AND "Hassler, Keno" AND "Holz, Thorsten" AND "Schönherr, Lea" AND "Fritz, Mario", year = 2024, month = 3, journal = "2nd IEEE Conference on Secure and Trustworthy Machine Learning" }

Usage metrics

    Categories

    No categories selected

    Licence

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC