The Hundred-Page Language Models Book
Synopsis
Andriy Burkov applies the format of his earlier Hundred-Page Machine Learning Book to language models, compressing the route from first principles to modern large language models into about 160 pages. A fast review of machine learning basics, covering vectors, gradient descent and neural network training, leads into language modeling fundamentals: bag of words, word embeddings, byte-pair encoding, count-based models and evaluation by perplexity. Successive chapters implement a recurrent neural network language model and then a Transformer from scratch in PyTorch, before the longest chapter turns to large language models, including instruction finetuning, low-rank adaptation (LoRA), prompt engineering and getting consistent outputs from a model.
Each idea is presented three ways, in mathematical notation, in diagrams and in Python code, with companion notebooks published on GitHub. The result reads like a dense course reader rather than a narrative, and it rewards readers who run the code as they go. Readers come away able to explain how next-token prediction, attention and finetuning work and to train small models themselves, which makes later work with production LLMs less opaque. Burkov sells the book on a read-first, buy-later basis, posting chapters free on the book's website, and word2vec creator Tomáš Mikolov wrote the foreword.
Scorecard
Editorial score
How it scored
Editorial score: 71.8 / 1005 weighted criteria
- 18.0/30
Influence30%
- 17.5/25
Authority25%
- 16.0/20
Practical value20%
- 12.8/15
Currency15%
- 7.5/10
Readability10%
- 71.8/100
Editorial score
How the rank was computed
Rank No. 162 of 500 comes from a composite of 58.7: observable signals 26.4 of 55 points plus editorial review 32.3 of 45 points.
All 11 inputs
| Input | Points |
|---|---|
| Observable signals | 26.4/55 |
| Reach (Amazon ratings volume) | 7.5/16.5 |
| Consensus across best-of lists | 8.3/13.8 |
| Awards and bestseller evidence | 0.0/8.3 |
| Reader rating, sample-adjusted | 6.7/8.3 |
| Amazon sales rank | 1.1/5.5 |
| Recency | 2.8/2.8 |
| Editorial review | 32.3/45 |
| Author authority | 7.9/11.3 |
| Influence | 8.1/13.5 |
| Practical value | 7.2/9.0 |
| Readability | 3.4/4.5 |
| Currency | 5.7/6.8 |
Each criterion is scored from 1 to 10 and weighted, and the points add up to the editorial score. The rank combines observable signals (55 percent) with this score (45 percent). Read the methodology.
Verdict
A compact, code-driven path from count-based language models to transformers and finetuned LLMs, ideal for engineers who want the mechanics.
Strengths
- Builds each architecture from scratch in PyTorch
- Math, diagrams and code side by side
- Chapters free to read before buying
Keep in mind
The compression is relentless: readers without working Python and basic linear algebra will find the pace steep, and there is little historical or narrative context.
Key takeaways
- Language modeling reduces to predicting the next token, and that single objective connects simple count-based models to billion-parameter transformers.
- Tokenization schemes such as byte-pair encoding and learned word embeddings turn raw text into numbers, and these choices shape what a model can learn.
- Self-attention lets a Transformer weigh every earlier token when predicting the next one, which is why it displaced recurrent networks for language tasks.
- Instruction finetuning and low-rank adaptation let practitioners adapt a pretrained model to new tasks without the cost of full retraining.
Notable facts
- The book carries a foreword by Tomáš Mikolov, creator of word2vec and FastText. Source (opens in a new tab)
- It is sold on a read-first, buy-later model, with every chapter free to read on the book's website. Source (opens in a new tab)
About the author
Questions
What is The Hundred-Page Language Models Book about?
Do I need machine learning experience to read The Hundred-Page Language Models Book?
Is The Hundred-Page Language Models Book worth reading in 2026?
Related books
- What Is ChatGPT Doing ... and Why Does It Work? Stephen Wolfram
- Vibe Coding Gene Kim and Steve Yegge
- LLM Engineer's Handbook Paul Iusztin and Maxime Labonne
- These Strange New Minds Christopher Summerfield
- Generative AI Design Patterns Valliappa Lakshmanan and Hannes Hapke
- Reinforcement Learning from Human Feedback Nathan Lambert
Sources
As an Amazon Associate, Best AI Books earns from qualifying purchases. Best AI Books is sponsored by Iternal.ai, whose founder and CEO, John Byron Hanby IV, wrote The AI Strategy Blueprint, our #1 ranked book. How we rank books.






