posted on 2025-04-28, 13:32authored byKhanh Nguyen, Raouf Kerkouche, Mario Fritz, Dimosthenis Karatzas
Document Visual Question Answering (DocVQA) has introduced a new paradigm
for end-to-end document understanding, and quickly became one of the standard
benchmarks for multimodal LLMs. Automating document processing workflows,
driven by DocVQA models, presents significant potential for many business sectors. However, documents tend to contain highly sensitive information, raising
concerns about privacy risks associated with training such DocVQA models. One
significant privacy vulnerability, exploited by the membership inference attack, is
the possibility for an adversary to determine if a particular record was part of the
model’s training data. In this paper, we introduce two novel membership inference
attacks tailored specifically to DocVQA models. These attacks are designed for
two different adversarial scenarios: a white-box setting, where the attacker has full
access to the model architecture and parameters, and a black-box setting, where
only the model’s outputs are available. Notably, our attacks assume the adversary lacks access to auxiliary datasets, which is more realistic in practice but also
more challenging. Our unsupervised methods outperform existing state-of-the-art
membership inference attacks across a variety of DocVQA models and datasets,
demonstrating their effectiveness and highlighting the privacy risks in this domain.
History
Primary Research Area
Trustworthy Information Processing
Name of Conference
International Conference on Learning Representations (ICLR)
CISPA Affiliation
Yes
Journal
CoRR
Volume
abs/2502.03692
Open Access Type
Unknown
BibTeX
@conference{Nguyen:Kerkouche:Fritz:Karatzas:2025,
title = "DocMIA: Document-Level Membership Inference Attacks against DocVQA Models.",
author = "Nguyen, Khanh" AND "Kerkouche, Raouf" AND "Fritz, Mario" AND "Karatzas, Dimosthenis",
year = 2025,
month = 2,
journal = "CoRR"
}