Ilya Sutskever: Neural Network Generalization as Compression
Simons Institute for the Theory of Computing · Talk · 57 min · 2023 · YouTube title: An Observation on Generalization
Why it's here
Cited in discussions of learning as compression and of why neural networks generalize.
What it covers
Ilya Sutskever gives a Simons Institute talk on why neural networks generalize, framing learning as finding short descriptions, like compression. It was part of the Large Language Models and Transformers program.