CISPA
Browse
cispa_all_3603.pdf (953.25 kB)

On the Existence of Universal Lottery Tickets

Download (953.25 kB)
conference contribution
posted on 2023-11-29, 18:20 authored by Rebekka BurkholzRebekka Burkholz, Nilanjana Laha, Rajarshi Mukherjee, Alkis Gotovos
The lottery ticket hypothesis conjectures the existence of sparse subnetworks of large randomly initialized deep neural networks that can be successfully trained in isolation. Recent work has experimentally observed that some of these tickets can be practically reused across a variety of tasks, hinting at some form of universality. We formalize this concept and theoretically prove that not only do such universal tickets exist but they also do not require further training. Our proofs introduce a couple of technical innovations related to pruning for strong lottery tickets, including extensions of subset sum results and a strategy to leverage higher amounts of depth. Our explicit sparse constructions of universal function families might be of independent interest, as they highlight representational benefits induced by univariate convolutional architectures.

History

Preferred Citation

Rebekka Burkholz, Nilanjana Laha, Rajarshi Mukherjee and Alkis Gotovos. On the Existence of Universal Lottery Tickets. In: International Conference on Learning Representations (ICLR). 2022.

Primary Research Area

  • Trustworthy Information Processing

Name of Conference

International Conference on Learning Representations (ICLR)

Legacy Posted Date

2022-04-06

Open Access Type

  • Green

BibTeX

@inproceedings{cispa_all_3603, title = "On the Existence of Universal Lottery Tickets", author = "Burkholz, Rebekka and Laha, Nilanjana and Mukherjee, Rajarshi and Gotovos, Alkis", booktitle="{International Conference on Learning Representations (ICLR)}", year="2022", }

Usage metrics

    Categories

    No categories selected

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC