No. 115 of 500 No. 12 in Generative AI & LLMs

What Is ChatGPT Doing ... and Why Does It Work?

By Stephen Wolfram

Published
March 9, 2023
Print length
102 pages
Reading level
Intermediate
ISBN-13
978-1579550813
Publisher
Wolfram Media Inc.
Written for
Curious professionals and students who want an intuitive, math-light account of how ChatGPT-style models work.

Synopsis

Stephen Wolfram expanded this short book from an essay he published in February 2023, less than three months after ChatGPT's release made large language models a public topic. He starts from the core mechanism: the model repeatedly estimates the probability of each possible next token and picks one, with a temperature setting that adds controlled randomness. He then explains why such probabilities cannot simply be counted from text and must come from a model, and walks through neural networks, training by gradient descent, embeddings that place words in a geometric meaning space, and the transformer architecture with its attention mechanism. Many illustrations use the smaller GPT-2 model.

The second half asks why such a simple procedure produces fluent language. Wolfram suggests that human language has more regular structure than commonly assumed, a semantic grammar that the network captures implicitly, and he connects this to his idea of computational irreducibility, arguing that some computations cannot be shortcut and that neural networks therefore have limits. A final section proposes pairing ChatGPT with Wolfram|Alpha for exact computation and curated facts. Readers gain a concrete mental model of next-token generation in roughly a hundred pages, along with a sense of the open scientific questions it raises.

Scorecard

Editorial score

How it scored

Editorial score: 73.0 / 1005 weighted criteria

  • Influence30%

    24.0/30
  • Authority25%

    20.0/25
  • Practical value20%

    13.0/20
  • Currency15%

    9.0/15
  • Readability10%

    7.0/10
  • Editorial score

    73.0/100

How the rank was computed

Rank No. 115 of 500 comes from a composite of 62.0: observable signals 29.0 of 55 points plus editorial review 33.0 of 45 points.

All 11 inputs
InputPoints
Observable signals29.0/55
Reach (Amazon ratings volume)11.5/16.5
Consensus across best-of lists5.5/13.8
Awards and bestseller evidence5.0/8.3
Reader rating, sample-adjusted4.1/8.3
Amazon sales rank1.2/5.5
Recency1.7/2.8
Editorial review33.0/45
Author authority9.0/11.3
Influence10.8/13.5
Practical value5.9/9.0
Readability3.2/4.5
Currency4.1/6.8

Each criterion is scored from 1 to 10 and weighted, and the points add up to the editorial score. The rank combines observable signals (55 percent) with this score (45 percent). Read the methodology.

Verdict

A compact, clear account of how language models generate text from a leading computational scientist, though its closing section promotes Wolfram|Alpha.

Strengths

  • Short, clear explanation of next-token prediction
  • Concrete visual examples built on real models
  • Connects mechanics to deeper questions about language

Keep in mind

Examples rely on GPT-2 and early ChatGPT, the closing Wolfram|Alpha section reads as promotional, and newcomers may find the neural-network passages dense.

Key takeaways

  1. A language model writes by repeatedly choosing a plausible next token from a probability distribution, building text one token at a time.
  2. A temperature setting that sometimes picks less likely tokens makes output more varied and natural than always choosing the top prediction.
  3. Embeddings represent words and passages as points in a high-dimensional space where related meanings sit close together, which helps the network generalize.
  4. Computational irreducibility implies that neural networks cannot shortcut every computation, so exact tools such as Wolfram|Alpha can complement them.

Notable facts

  • The book expands an essay of the same title that Wolfram published on February 14, 2023, on his Stephen Wolfram Writings site. Source (opens in a new tab)

About the author

Stephen Wolfram

Stephen Wolfram is the founder and CEO of Wolfram Research, which he started in 1987, and the creator of Mathematica, the Wolfram Language and the Wolfram|Alpha answer engine. He earned a PhD in particle physics from Caltech at 20, received a MacArthur Fellowship in 1981 as its youngest recipient to that date, and wrote A New Kind of Science (2002).

Questions

What does Wolfram's What Is ChatGPT Doing explain?
Stephen Wolfram explains, in about a hundred pages, how ChatGPT produces text by repeatedly predicting a likely next token, and how neural networks, training, embeddings and transformers make that possible. He then considers why this works so well, suggesting that language follows deeper regularities than commonly assumed, and closes by proposing that ChatGPT be paired with Wolfram|Alpha for precise computation.
Do I need a technical background to read Wolfram's ChatGPT book?
No formal background is required, since Wolfram avoids equations and relies on pictures and plain explanation. Readers with no exposure to neural networks may still find the middle chapters on training, embeddings and transformers dense, while readers who already know machine learning may find it introductory. Comfort with basic ideas of probability and graphs makes it noticeably easier.
Is What Is ChatGPT Doing still accurate in 2026?
Its explanation of next-token prediction, embeddings and transformers remains accurate for today's language models. It does not cover developments after early 2023, such as reasoning models trained to work through problems step by step, multimodal systems, tool-using agents or newer training methods, and many examples use GPT-2. Read it for the core mechanism, then turn to newer sources for current capabilities.

Sources

As an Amazon Associate, Best AI Books earns from qualifying purchases. Best AI Books is sponsored by Iternal.ai, whose founder and CEO, John Byron Hanby IV, wrote The AI Strategy Blueprint, our #1 ranked book. How we rank books.