No. 123 of 500 No. 15 in Machine Learning

Gaussian Processes for Machine Learning

By Carl Edward Rasmussen and Christopher K. I. Williams

Print length
272 pages
Publisher
The MIT Press
Reading level
Advanced
ISBN-13
978-0262182539
Published
November 23, 2005
Written for
Graduate students, researchers and data scientists building probabilistic models, Bayesian optimization or uncertainty-aware regression systems

Synopsis

Carl Edward Rasmussen and Christopher Williams wrote this monograph to give Gaussian processes a unified treatment within machine learning. Instead of fitting a fixed set of parameters, a Gaussian process places a probability distribution directly over functions, so every prediction arrives with a calibrated measure of uncertainty. The authors develop the idea first for regression, where the mathematics is exact, and then for classification, where approximations such as the Laplace method and expectation propagation become necessary. A substantial chapter catalogs covariance, or kernel, functions and explains how the choice of kernel encodes assumptions about smoothness, periodicity and scale.

Later chapters treat model selection through the marginal likelihood and cross-validation, relate Gaussian processes to support vector machines, splines, regularization networks and neural networks, and survey theoretical results including learning curves and PAC-Bayesian bounds. A chapter on approximations for large datasets addresses the method's cubic scaling. Appendices supply the matrix algebra and probability required, and the authors publish the full text and accompanying code online. Readers gain the conceptual foundation behind Bayesian optimization, surrogate modeling and modern probabilistic programming libraries, provided they are comfortable with linear algebra and probability.

Scorecard

Editorial score

How it scored

Editorial score: 81.3 / 1005 weighted criteria

  • Influence30%

    27.0/30
  • Authority25%

    23.8/25
  • Practical value20%

    14.0/20
  • Currency15%

    10.5/15
  • Readability10%

    6.0/10
  • Editorial score

    81.3/100

How the rank was computed

Rank No. 123 of 500 comes from a composite of 61.4: observable signals 24.8 of 55 points plus editorial review 36.6 of 45 points.

All 11 inputs
InputPoints
Observable signals24.8/55
Reach (Amazon ratings volume)6.8/16.5
Consensus across best-of lists5.5/13.8
Awards and bestseller evidence5.0/8.3
Reader rating, sample-adjusted6.8/8.3
Amazon sales rank0.7/5.5
Recency0.0/2.8
Editorial review36.6/45
Author authority10.7/11.3
Influence12.2/13.5
Practical value6.3/9.0
Readability2.7/4.5
Currency4.7/6.8

Each criterion is scored from 1 to 10 and weighted, and the points add up to the editorial score. The rank combines observable signals (55 percent) with this score (45 percent). Read the methodology.

Verdict

The standard reference on Gaussian processes: rigorous, free online, and still the text specialists consult first for probabilistic regression and classification.

Strengths

  • Unified, self-contained treatment of theory and algorithms
  • Full text and code freely available
  • Clear links to SVMs, splines and neural networks

Keep in mind

Assumes fluency in linear algebra and probability, and predates modern scalable variational methods and deep kernel learning developed after 2006.

Key takeaways

  1. A Gaussian process defines a prior over functions, so predictions come with principled error bars rather than single point estimates.
  2. The covariance function carries the model's assumptions; choosing and combining kernels is where most practical modeling decisions are made.
  3. Maximizing the marginal likelihood lets the data set kernel hyperparameters, automatically balancing fit against complexity in a Bayesian Occam's razor.
  4. Exact inference scales cubically with data size, so large problems require sparse or approximate methods the book compares in detail.

Notable facts

About the author

Carl Edward Rasmussen and Christopher K. I. Williams

Carl Edward Rasmussen is Professor of Machine Learning in the University of Cambridge's Department of Engineering, where he heads the Computational and Biological Learning Lab; he earned his PhD under Geoffrey Hinton at Toronto. Christopher K. I. Williams holds a personal chair in machine learning at the University of Edinburgh's School of Informatics and was elected a Fellow of the Royal Society of Edinburgh in 2021.

Questions

What is Gaussian Processes for Machine Learning about?
Rasmussen and Williams's book is a systematic treatment of Gaussian processes, a Bayesian method that models a distribution over functions. It covers regression and classification, covariance functions, hyperparameter learning through the marginal likelihood, links to support vector machines and splines, theoretical results, and approximation methods for large datasets. Appendices provide the needed mathematical background, and the authors supply code and datasets online.
Is Gaussian Processes for Machine Learning still relevant in 2026?
Yes. Gaussian processes remain the usual engine of Bayesian optimization for hyperparameter tuning and of surrogate modeling in engineering and science. The 2006 text does not cover later scalable techniques such as sparse variational inference or deep kernel learning, so practitioners should pair it with recent papers or library documentation, but its derivations remain the standard starting point.
Can I read Gaussian Processes for Machine Learning for free?
Yes. With permission from the MIT Press, the authors host the complete book as downloadable chapters on gaussianprocess.org, alongside links to software and datasets, and the MIT Press sells a print edition. Readers should be comfortable with linear algebra, multivariate Gaussians and basic probability, although Appendix A summarizes the key mathematical results.

Sources

As an Amazon Associate, Best AI Books earns from qualifying purchases. Best AI Books is sponsored by Iternal.ai, whose founder and CEO, John Byron Hanby IV, wrote The AI Strategy Blueprint, our #1 ranked book. How we rank books.