CISPA
Browse
cispa_all_3727.pdf (515.81 kB)

Label-Descriptive Patterns and their Application to Characterizing Classification Errors

Download (515.81 kB)
conference contribution
posted on 2023-11-29, 18:21 authored by M. Hedderich, Jonas Fischer, Dietrich Klakow, Jilles VreekenJilles Vreeken
State-of-the-art deep learning methods achieve human-like performance on many tasks, but make errors nevertheless. Characterizing these errors in easily interpretable terms gives insight into whether a classifier is prone to making systematic errors, but also gives a way to act and improve the classifier. We propose to discover those feature- value combinations (ie. patterns) that strongly correlate with correct resp. erroneous predictions to obtain a global and interpretable description for arbitrary classifiers. We show this is an instance of the more general label description problem, which we formulate in terms of the Minimum De- scription Length principle. To discover a good pattern set, we develop the efficient PREMISE al- gorithm. Through an extensive set of experiments we show it performs very well in practice on both synthetic and real-world data. Unlike existing solutions, it ably recovers ground truth patterns, even on highly imbalanced data over many fea- tures. Through two case studies on Visual Ques- tion Answering and Named Entity Recognition, we confirm that PREMISE gives clear and action- able insight into the systematic errors made by modern NLP classifiers.

History

Preferred Citation

M. Hedderich, Jonas Fischer, Dietrich Klakow and Jilles Vreeken. Label-Descriptive Patterns and their Application to Characterizing Classification Errors. In: International Conference on Machine Learning (ICML). 2022.

Primary Research Area

  • Trustworthy Information Processing

Name of Conference

International Conference on Machine Learning (ICML)

Legacy Posted Date

2022-07-15

Open Access Type

  • Unknown

BibTeX

@inproceedings{cispa_all_3727, title = "Label-Descriptive Patterns and their Application to Characterizing Classification Errors", author = "Hedderich, M. and Fischer, Jonas and Klakow, Dietrich and Vreeken, Jilles", booktitle="{International Conference on Machine Learning (ICML)}", year="2022", }

Usage metrics

    Categories

    No categories selected

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC