No. 334 of 500 No. 35 in Generative AI & LLMs

An Illustrated Guide to AI Agents

Concepts and Code for Building Agents with LLMs, Tools, and Memory

By Maarten Grootendorst and Jay Alammar

Print length
448 pages
Publisher
O'Reilly Media
Reading level
Intermediate
ISBN-13
979-8341662698
Published
October 6, 2026
Written for
Python developers, ML engineers and technical leads who want to understand how AI agents work internally.

Synopsis

Maarten Grootendorst and Jay Alammar, authors of Hands-On Large Language Models, apply their diagram-led teaching style to AI agents: systems in which language models plan, call tools, remember context and act with some independence. The book's first part dissects a single agent. It covers how LLMs work internally, how reasoning models use extra computation at inference time, how memory and retrieval are engineered, how agents call tools through function calling and the Model Context Protocol, how planning and reflection loops such as ReAct operate, and how to evaluate agents with benchmarks, LLM judges and trajectory analysis.

The second part turns to specialized designs: multi-agent collaboration, multimodal understanding, and coding agents of the kind measured by SWE-bench. Throughout, readers build a small agent called TinyAgent from scratch using only LLM calls, adding memory, tools and autonomy one component at a time in companion notebooks that run on a free Google Colab GPU. The 300-plus custom figures make internal mechanisms visible without heavy mathematics. Engineers who already use agent frameworks gain a clear picture of what those frameworks do underneath, and technical leads gain a shared vocabulary for design reviews. Because the field moves quickly, specific models and protocols will date faster than the underlying concepts.

Scorecard

Editorial score

How it scored

Editorial score: 75.8 / 1005 weighted criteria

  • Influence30%

    18.0/30
  • Authority25%

    20.0/25
  • Practical value20%

    15.0/20
  • Currency15%

    14.3/15
  • Readability10%

    8.5/10
  • Editorial score

    75.8/100

How the rank was computed

Rank No. 334 of 500 comes from a composite of 51.9: observable signals 17.8 of 55 points plus editorial review 34.1 of 45 points.

All 11 inputs
InputPoints
Observable signals17.8/55
Reach (Amazon ratings volume)4.2/16.5
Consensus across best-of lists2.8/13.8
Awards and bestseller evidence0.0/8.3
Reader rating, sample-adjusted5.7/8.3
Amazon sales rank2.3/5.5
Recency2.8/2.8
Editorial review34.1/45
Author authority9.0/11.3
Influence8.1/13.5
Practical value6.8/9.0
Readability3.8/4.5
Currency6.4/6.8

Each criterion is scored from 1 to 10 and weighted, and the points add up to the editorial score. The rank combines observable signals (55 percent) with this score (45 percent). Read the methodology.

Verdict

A heavily illustrated, code-backed tour of the full AI agent stack, best for engineers who want to understand what frameworks hide.

Strengths

  • More than 300 custom explanatory illustrations
  • Builds one agent incrementally across chapters
  • Free Colab notebooks for every chapter

Keep in mind

Readers seeking production deployment guidance or tutorials for a specific agent framework will need other resources, and model-specific details will age quickly.

Key takeaways

  1. An agent is an LLM wrapped with memory, tools and planning; understanding each component separately makes agent behavior and failures easier to diagnose.
  2. Reasoning models improve results by spending more computation at inference time, which changes cost and latency trade-offs for agent design.
  3. Memory is an engineering choice: agentic retrieval and context engineering decide what an agent knows at each step.
  4. Evaluating agents requires examining full action trajectories and repeated-run reliability, not just final answers, because the path an agent takes matters as much as its result.

Notable facts

  • The book carries endorsements from Chip Huyen, author of AI Engineering, MIT EECS assistant professor Omar Khattab, and Meta FAIR research scientist Ofir Press, coauthor of SWE-bench. Source (opens in a new tab)
  • The official code repository for the authors' previous O'Reilly book, Hands-On Large Language Models, has more than 30,000 GitHub stars. Source (opens in a new tab)

About the author

Maarten Grootendorst and Jay Alammar

Maarten Grootendorst is a clinical data scientist at IKNL in the Netherlands, applying NLP and LLMs to cancer research, and creator of the open-source packages BERTopic, KeyBERT and PolyFuzz. Co-author Jay Alammar is Director and Engineering Fellow at Cohere and author of the blog post The Illustrated Transformer. Together they wrote Hands-On Large Language Models (O'Reilly, 2024).

Questions

What is An Illustrated Guide to AI Agents about?
It explains how AI agents are built, using hundreds of diagrams and companion code. Part one covers the anatomy of a single agent: language models, reasoning models, memory, tool use, planning and reflection, and evaluation. Part two covers multi-agent systems, multimodal understanding and coding agents. Readers build a small agent called TinyAgent from scratch, adding capabilities chapter by chapter in Python notebooks.
Do I need to read Hands-On Large Language Models first?
No. The agents book includes its own chapters on how large language models and reasoning models work, so it stands alone. Readers who already know the earlier book will recognize the illustration style and can move faster through those opening chapters. Working knowledge of Python is the main prerequisite, and the companion notebooks run on a free Google Colab GPU or a local setup.
Who should read An Illustrated Guide to AI Agents?
It suits software engineers and data scientists who build with agent frameworks and want to understand what happens inside them, plus technical leads and students who need a structured overview of the agent stack. Visual learners benefit most from its diagram-first approach. Readers looking for a nontechnical business overview of agents, or a deployment manual for a specific framework, should look for a different kind of book.

Sources

As an Amazon Associate, Best AI Books earns from qualifying purchases. Best AI Books is sponsored by Iternal.ai, whose founder and CEO, John Byron Hanby IV, wrote The AI Strategy Blueprint, our #1 ranked book. How we rank books.