cispa_all_3051.pdf (2.51 MB)

Automatically Detecting Bystanders in Photos to Reduce Privacy Risks

Download (2.51 MB)
conference contribution
posted on 2023-11-29, 18:12 authored by Rakibul Hasan, David Crandall, Mario FritzMario Fritz, Apu Kapadia
Photographs taken in public places often contain bystanders~-- people who are not the main subject of a photo. These photos, when shared online, can reach a large number of viewers and potentially undermine the bystanders' privacy. Furthermore, recent developments in computer vision and machine learning can be used by online platforms to identify and track individuals. To combat this problem, researchers have proposed technical solutions that require bystanders to be proactive and use specific devices and/or applications to broadcast their privacy policy and identifying information while being located in an image. We explore the prospect of a different approach~-- identifying bystanders solely based on the visual information present in an image. Through an online user study, we catalog the rationale humans use to classify subjects and bystanders in an image, and systematically validate a set of intuitive concepts (such as intentionally posing for a photo) that can be used to automatically identify bystanders. Using image data, we infer those concepts and then use them to train several classifier models. We extensively evaluate the models and compare them with human raters. On our training data set, which features a 10-fold cross validation, our best model achieves a mean detection accuracy of 93% for images when human raters have 100% agreement on the class label and 80% when the agreement is only 67%. We validate this model on a completely different test data set and achieve similar results, demonstrating that our model generalizes well.


Preferred Citation

Rakibul Hasan, David Crandall, Mario Fritz and Apu Kapadia. Automatically Detecting Bystanders in Photos to Reduce Privacy Risks. In: IEEE Symposium on Security and Privacy (S&P). 2020.

Primary Research Area

  • Trustworthy Information Processing

Secondary Research Area

  • Empirical and Behavioral Security

Name of Conference

IEEE Symposium on Security and Privacy (S&P)

Legacy Posted Date


Open Access Type

  • Green


@inproceedings{cispa_all_3051, title = "Automatically Detecting Bystanders in Photos to Reduce Privacy Risks", author = "Hasan, Rakibul and Crandall, David and Fritz, Mario and Kapadia, Apu", booktitle="{IEEE Symposium on Security and Privacy (S&P)}", year="2020", }

Usage metrics


    No categories selected


    Ref. manager