Kafka
Real-Time Data and Stream Processing at Scale
Synopsis
Kafka: The Definitive Guide comes from engineers at Confluent and LinkedIn who worked on Apache Kafka, the distributed event-streaming platform that moves logs, metrics, user activity and transactions between systems in real time. The 2017 first edition, by Kafka co-creator Neha Narkhede with Gwen Shapira and Todd Palino, explains publish-subscribe messaging, then walks through installing brokers, writing producers and consumers, and the internals behind Kafka's guarantees: replication, request handling, the controller and the storage layer. Chapters on reliable delivery, data pipelines with Kafka Connect, cross-cluster mirroring, administration, monitoring and stream processing with Kafka Streams complete the book.
The authors favor explanation over recipes, showing how configuration choices affect durability, ordering and throughput so readers can reason about trade-offs instead of copying settings. A 2021 second edition, by Shapira, Palino, Rajini Sivaram and Krit Petty, added chapters on the AdminClient API, transactions, newer security features and tooling changes. For AI and machine learning teams, Kafka often carries the real-time events that feed feature pipelines and online models, so the book doubles as an introduction to streaming data infrastructure. Neither edition reflects Kafka 4.0, released in March 2025, which removed ZooKeeper entirely in favor of KRaft metadata management.
Scorecard
Editorial score
How it scored
Editorial score: 74.5 / 1005 weighted criteria
- 22.5/30
Influence30%
- 22.5/25
Authority25%
- 15.0/20
Practical value20%
- 7.5/15
Currency15%
- 7.0/10
Readability10%
- 74.5/100
Editorial score
How the rank was computed
Rank No. 305 of 500 comes from a composite of 52.9: observable signals 19.3 of 55 points plus editorial review 33.6 of 45 points.
All 11 inputs
| Input | Points |
|---|---|
| Observable signals | 19.3/55 |
| Reach (Amazon ratings volume) | 9.1/16.5 |
| Consensus across best-of lists | 2.8/13.8 |
| Awards and bestseller evidence | 0.0/8.3 |
| Reader rating, sample-adjusted | 6.6/8.3 |
| Amazon sales rank | 0.8/5.5 |
| Recency | 0.0/2.8 |
| Editorial review | 33.6/45 |
| Author authority | 10.1/11.3 |
| Influence | 10.1/13.5 |
| Practical value | 6.8/9.0 |
| Readability | 3.2/4.5 |
| Currency | 3.4/6.8 |
Each criterion is scored from 1 to 10 and weighted, and the points add up to the editorial score. The rank combines observable signals (55 percent) with this score (45 percent). Read the methodology.
Verdict
A thorough practitioner guide to Apache Kafka by engineers who helped build it, strong on internals and reliability but predating ZooKeeper's removal.
Strengths
- Written by engineers who built Kafka
- Deep explanation of internals and reliability
- Covers both development and production operations
Keep in mind
Both editions predate Kafka 4.0's removal of ZooKeeper, so cluster setup and administration chapters need supplementing with current documentation.
Key takeaways
- Kafka's durability and ordering guarantees depend on configuration choices, such as replication factor, acknowledgments and in-sync replicas, that teams must set deliberately.
- Producers and consumers each carry responsibility for reliable delivery; consumer offset commits determine whether messages are processed once, twice or not at all.
- Kafka Connect and MirrorMaker let organizations move data between systems and clusters without writing custom integration code for every source and destination.
- Monitoring broker, producer and consumer metrics, including consumer lag, is essential to running Kafka clusters reliably in production.
Notable facts
- Confluent distributes Kafka: The Definitive Guide as a free e-book download. Source (opens in a new tab)
- I Programmer reviewer Kay Ewbank rated the first edition 4.5 out of 5 and called it a very good book for anyone who needs to learn Kafka. Source (opens in a new tab)
About the author
Questions
What is Kafka: The Definitive Guide about?
Should I read the first or second edition of Kafka: The Definitive Guide?
Is Kafka: The Definitive Guide relevant to AI and machine learning teams?
Related books
Sources
As an Amazon Associate, Best AI Books earns from qualifying purchases. Best AI Books is sponsored by Iternal.ai, whose founder and CEO, John Byron Hanby IV, wrote The AI Strategy Blueprint, our #1 ranked book. How we rank books.






