Building a Bigram Language Model from Scratch

HN recommends it as a long but worthwhile way to demystify how language models work technically.

Andrej Karpathy builds a bigram character-level language model from scratch, introducing torch.Tensor and the framework of training, sampling and loss evaluation. It is the first step toward a GPT-style Transformer in his makemore series.