CISPA
Browse
cispa_all_4038.pdf (1.04 MB)

Revisiting Neural Program Smoothing for Fuzzing

Download (1.04 MB)
conference contribution
posted on 2024-03-05, 12:19 authored by Nicolae, Maria-Irina, Max EiseleMax Eisele, Andreas ZellerAndreas Zeller
Testing with randomly generated inputs (fuzzing) has gained significant traction due to its capacity to expose program vulnerabilities automatically. Fuzz testing campaigns generate large amounts of data, making them ideal for the application of machine learning (ML). Neural program smoothing, a specific family of ML-guided fuzzers, aims to use a neural network as a smooth approximation of the program target for new test case generation. In this paper, we conduct the most extensive evaluation of neural program smoothing (NPS) fuzzers against standard gray-box fuzzers (>11 CPU years and >5.5 GPU years), and make the following contributions: (1) We find that the original performance claims for NPS fuzzers do not hold; a gap we relate to fundamental, implementation, and experimental limitations of prior works. (2) We contribute the first in-depth analysis of the contribution of machine learning and gradient-based mutations in NPS . (3) We implement Neuzz++, which shows that addressing the practical limitations of NPS fuzzers improves performance, but standard gray-box fuzzers almost always surpass NPS-based fuzzers. (4) As a consequence, we propose new guidelines targeted at benchmarking fuzzing based on machine learning, and present a platform, MLFuzz, with GPU access for easy and reproducible evaluation of ML -based fuzzers. Neuzz++, MLFuzz, and all our data are public.

History

Preferred Citation

Maria-Irina Nicolae, Max Eisele, Andreas Zeller. Revisiting Neural Program Smoothing for Fuzzing. In: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE ’23). 2023.

Primary Research Area

  • Threat Detection and Defenses

Name of Conference

Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE ’23)

Legacy Posted Date

2023-09-29

Open Access Type

  • Unknown

BibTeX

@inproceedings{cispa_all_4038, author = {Maria-Irina Nicolae AND Max Eisele AND Andreas Zeller}, title = {Revisiting Neural Program Smoothing for Fuzzing}, booktitle = {Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE ’23)}, year = {2023} }

Usage metrics

    Categories

    No categories selected

    Licence

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC