CISPA
Browse
cispa_all_3263.pdf (2.72 MB)

Revisiting the Relationship Between Fault Detection,Test Adequacy Criteria, and Test Set Size

Download (2.72 MB)
conference contribution
posted on 2023-11-29, 18:14 authored by Yiqun T. Chen, Rahul Gopinath, Anita Tadakamalla, Michael D. Ernst, Reid Holmes, Gordon Fraser, Paul Ammann
The research community has long recognized a complex interrelationship between test set size, test adequacy criteria, and test effectiveness in terms of fault detection. However, there is substantial confusion about the role and importance of controlling for test set size when assessing and comparing test adequacy criteria. This paper makes the following contributions: (1) A review of contradictory analyses of the relationship between fault detection, test suite size, and test adequacy criteria. Specifically, this paper addresses the supposed contradiction of prior work and explains why test suite size is neither a confounding variable, as previously suggested,nor an independent variable that should be experimentally manipulated. (2) An explication and discussion of the experimental design and sampling strategies of prior work, together with a discussion of conceptual and statistical problems, and specific guidelines for future work. (3) A methodology for comparing test-adequacy criteria on an equal basis, which accounts for test suite size by treating it as a covariate. (4) An empirical evaluation that compares the effectiveness of coverage-based and mutation-based testing to one another and random testing. Additionally, this paper proposes probabilistic coupling, a methodology for approximating the representativeness of a set of test goals for a given set of real faults

History

Preferred Citation

Yiqun Chen, Rahul Gopinath, Anita Tadakamalla, Michael Ernst, Reid Holmes, Gordon Fraser and Paul Ammann. Revisiting the Relationship Between Fault Detection,Test Adequacy Criteria, and Test Set Size. In: Automated Software Engineering Conference (ASE). 2020.

Primary Research Area

  • Secure Connected and Mobile Systems

Name of Conference

Automated Software Engineering Conference (ASE)

Legacy Posted Date

2020-12-01

Open Access Type

  • Green

BibTeX

@inproceedings{cispa_all_3263, title = "Revisiting the Relationship Between Fault Detection,Test Adequacy Criteria, and Test Set Size", author = "Chen, Yiqun T. and Gopinath, Rahul and Tadakamalla, Anita and Ernst, Michael D. and Holmes, Reid and Fraser, Gordon and Ammann, Paul", booktitle="{Automated Software Engineering Conference (ASE)}", year="2020", }

Usage metrics

    Categories

    No categories selected

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC