No. 8 of 500 No. 2 in Generative AI & LLMs

AI Engineering

Building Applications with Foundation Models

By Chip Huyen

Print length
532 pages
Publisher
O'Reilly Media
Reading level
Intermediate
ISBN-13
978-1098166304
Published
January 7, 2025
Written for
Software engineers, ML engineers and technical product managers building applications on top of foundation models.

Synopsis

Chip Huyen defines AI engineering as the work of building applications on top of models someone else has trained, and she argues that this differs from classic machine learning engineering because teams now start from a capable model and spend most of their effort adapting, evaluating and serving it. The book's ten chapters follow that workflow: how foundation models are trained and sampled, how to evaluate open-ended outputs, prompt engineering and its defenses, retrieval-augmented generation and agents, finetuning, dataset engineering, inference optimization, and finally the architecture and user-feedback loops of a production system.

Evaluation is the spine of the book. Huyen gives two full chapters to measuring model quality, including the strengths and failure modes of using one model to judge another, and she returns to it whenever she weighs a technique such as finetuning against cheaper prompting or retrieval. The approach is conceptual rather than code-heavy: Huyen explains why each method works, what it costs and when it fails, drawing on research papers and her own production experience. Readers come away with a decision framework for choosing among prompting, retrieval and finetuning, a vocabulary for discussing latency and cost trade-offs, and a clearer sense of what separates a demo from a dependable product. Because the frameworks are tool-agnostic, the book has aged more slowly than most titles on the subject.

Scorecard

Editorial score

How it scored

Editorial score: 84.2 / 1005 weighted criteria

  • Influence30%

    25.5/30
  • Authority25%

    21.3/25
  • Practical value20%

    17.6/20
  • Currency15%

    12.8/15
  • Readability10%

    7.0/10
  • Editorial score

    84.2/100

How the rank was computed

Rank No. 8 of 500 comes from a composite of 78.3: observable signals 40.4 of 55 points plus editorial review 37.9 of 45 points.

All 11 inputs
InputPoints
Observable signals40.4/55
Reach (Amazon ratings volume)11.1/16.5
Consensus across best-of lists13.8/13.8
Awards and bestseller evidence3.3/8.3
Reader rating, sample-adjusted6.8/8.3
Amazon sales rank2.6/5.5
Recency2.8/2.8
Editorial review37.9/45
Author authority9.6/11.3
Influence11.5/13.5
Practical value7.9/9.0
Readability3.2/4.5
Currency5.7/6.8

Each criterion is scored from 1 to 10 and weighted, and the points add up to the editorial score. The rank combines observable signals (55 percent) with this score (45 percent). Read the methodology.

Verdict

The most complete systematic guide to building products on foundation models, with unusually strong chapters on evaluation.

Strengths

  • Rigorous, extended treatment of evaluating open-ended outputs
  • Tool-agnostic frameworks that outlast specific APIs
  • Grounded in research papers and production experience

Keep in mind

Readers wanting code walkthroughs will be disappointed: the author states it is not a tutorial, and its density can slow non-engineers.

Key takeaways

  1. Evaluation is the hardest and most important part of AI engineering, because open-ended outputs rarely have one correct answer that a test can confirm.
  2. Using a model as a judge scales evaluation cheaply, but judges carry their own biases and must themselves be validated against human ratings.
  3. Try prompting first, add retrieval when the model lacks information, and finetune mainly when the problem is the model's behavior or format.
  4. Latency and cost are design constraints from day one; inference optimization techniques such as quantization, batching and caching decide whether a product is viable.

Notable facts

  • The author's books page states that AI Engineering has been the most-read book on the O'Reilly platform since its release. Source (opens in a new tab)
  • The book's official GitHub repository lists translations into more than a dozen languages, including Japanese, Korean, Chinese, Spanish, French, German and Portuguese. Source (opens in a new tab)

About the author

Chip Huyen

Chip Huyen is a computer scientist and writer who taught a machine learning systems course at Stanford, which became her book Designing Machine Learning Systems (O'Reilly, 2022). She worked on ML tooling at NVIDIA as a core developer of NeMo, worked at Snorkel AI and Netflix, and founded and sold an AI infrastructure company. She has also published four books in Vietnamese.

Questions

Is AI Engineering worth reading in 2026?
Yes, for anyone building on foundation models. Its core frameworks for evaluation, choosing between prompting, retrieval and finetuning, and controlling latency and cost do not depend on any particular model or vendor, so they still apply. Specific model names and benchmark numbers from late 2024 have dated, and agent tooling has moved quickly, so pair it with current documentation for those parts.
Do I need machine learning experience to read AI Engineering?
No formal ML background is required, but the book assumes comfort with software engineering and basic technical vocabulary. Huyen explains how foundation models are trained and sampled before building on those ideas. Readers with no technical background may find the chapters on finetuning memory bottlenecks and inference optimization heavy going, while engineers new to AI will find them the most useful part.
How is AI Engineering different from Designing Machine Learning Systems?
Designing Machine Learning Systems, Huyen's 2022 book, covers the classic workflow of training your own models on your own data and running them in production. AI Engineering starts from a pretrained foundation model and concentrates on adapting it through prompting, retrieval, agents and finetuning, then evaluating and serving the result. The two books complement each other, and teams running both kinds of system benefit from reading both.

Sources

As an Amazon Associate, Best AI Books earns from qualifying purchases. Best AI Books is sponsored by Iternal.ai, whose founder and CEO, John Byron Hanby IV, wrote The AI Strategy Blueprint, our #1 ranked book. How we rank books.