cispa_all_3224.pdf (230.67 kB)

Black Widow: Blackbox Data-driven Web Scanning

Download (230.67 kB)
conference contribution
posted on 2023-11-29, 18:15 authored by Benjamin Eriksson, Giancarlo PellegrinoGiancarlo Pellegrino, Andrei Sabelfeld
Modern web applications are an integral part of our digital lives. As we put more trust in web applications, the need for security increases. At the same time, detecting vulnerabilities in web applications has become increasingly hard, due to the complexity, dynamism, and reliance on third-party components. Blackbox vulnerability scanning is especially challenging because (i) for deep penetration of web applications scanners need to exercise such browsing behavior as user interaction and asynchrony, and (ii) for detection of nontrivial injection attacks, such as stored cross-site scripting (XSS), scanners need to discover inter-page data dependencies. This paper illuminates key challenges for crawling and scanning the modern web. Based on these challenges we identify three core pillars for deep crawling and scanning: navigation modeling, traversing, and tracking inter-state dependencies. While prior efforts are largely limited to the separate pillars, we suggest an approach that leverages all three. We develop Black Widow, a blackbox data-driven approach to web crawling and scanning. We demonstrate the effectiveness of the crawling by code coverage improvements ranging from 63% to 280% compared to other crawlers across all applications. Further, we demonstrate the effectiveness of the web vulnerability scanning by featuring no false positives and finding more cross-site scripting vulnerabilities than previous methods. In older applications, used in previous research, we find vulnerabilities that the other methods miss. We also find new vulnerabilities in production software, including HotCRP, osCommerce, PrestaShop and WordPress.


Preferred Citation

Benjamin Eriksson, Giancarlo Pellegrino and Andrei Sabelfeld. Black Widow: Blackbox Data-driven Web Scanning. In: IEEE Symposium on Security and Privacy (S&P). 2021.

Primary Research Area

  • Empirical and Behavioral Security

Name of Conference

IEEE Symposium on Security and Privacy (S&P)

Legacy Posted Date


Open Access Type

  • Unknown


@inproceedings{cispa_all_3224, title = "Black Widow: Blackbox Data-driven Web Scanning", author = "Eriksson, Benjamin and Pellegrino, Giancarlo and Sabelfeld, Andrei", booktitle="{IEEE Symposium on Security and Privacy (S&P)}", year="2021", }

Usage metrics


    No categories selected


    Ref. manager