A user edits their display name and hits save. The page reloads with the old name. They save again: old name. Thirty seconds later, without touching anything, the new name shows up in their activity feed. Twice.

The bug report reads like nonsense until you look at the infrastructure: writes go to the primary, reads come from a replica, and replication is asynchronous. The user wrote to one machine and read from another that hadn’t caught up. Nothing was down, nothing errored, every component did exactly what it was configured to do, and the system as a whole lied to the user about their own action.

Most of us file this under “replication lag” and move on. I think it deserves more attention, because that little bug is the CAP theorem showing up in a plain CRUD app. Not the conference-slide version; the real one. Knowing exactly what broke, and what the theorem does and doesn’t say about it, turns a slogan into a tool.

What CAP Actually Says

“Consistency, availability, partition tolerance: pick two” compresses a real theorem, conjectured by Eric Brewer in 2000 and proven by Seth Gilbert and Nancy Lynch in 2002, past the point of usefulness. Martin Kleppmann, author of Designing Data-Intensive Applications, spells out the precise definitions in his pointedly titled essay “Please stop calling databases CP or AP” (2015):

That last point kills the “pick two” framing. Partitions aren’t optional, so the real content of the theorem is narrower: when a partition happens, you choose between linearizability and availability. Brewer conceded as much in his 2012 retrospective, “CAP Twelve Years Later”: “the ‘2 of 3’ formulation was always misleading because it tended to oversimplify the tensions among properties.”

Linearizability: The Guarantee You Think You Have

Kleppmann’s informal definition: “If operation B started after operation A successfully completed, then operation B must see the system in the same state as it was on completion of operation A, or a newer state.”

His illustration is the best one I know. Alice and Bob are in the same room during the 2014 World Cup final. Alice refreshes her phone, sees the final score and tells Bob. Bob refreshes his, the request lands on a lagging replica, and his phone says the game is still on. Bob’s query started after Alice’s had finished (he heard her say so, through a channel outside the database), so a linearizable system would have to show him a result at least as fresh as hers. It didn’t.

The display-name bug is the same story with Alice and Bob as one person. That special case has its own name, read-your-own-writes: the user’s second request started after their first one completed, so they expected to see its effect. The replica served them the past.

Why don’t databases just give everyone linearizability? Cost. It “requires a lot of coordination”: the system has to behave as though there’s a single copy of the data, no matter how many replicas and caches exist. Kleppmann’s most humbling example: “Even the CPU in your computer doesn’t provide linearizable access to your local RAM!” You need explicit memory barriers to get it. If linearizability is expensive inside one machine, expecting it for free across a replicated database was always optimistic.

Your Stack Is Eventually Consistent, NoSQL or Not

You don’t opt into eventual consistency by picking a trendy database. You get it the moment any copy of your data can be read while another copy is newer:

The fixes for read-your-own-writes are mechanical. Route a user’s reads to the primary for a short window after they write (a “recently wrote” flag in the session). Or record the primary’s log position after the write and only serve that user from a replica that has caught up. Or, simplest and most ignored, render the UI from the write response instead of fetching again. Each is about an hour of work. The expensive part is noticing which reads actually need the guarantee.

Which brings me to the one idea I’d keep from this whole article: consistency is per operation, not per database. A user reading their own profile edit needs read-your-own-writes. A user reading someone else’s profile can live with a few seconds of staleness and never notice. Same table, same query, different requirement.

“So Is My Database CP or AP?” Wrong Question

With the theorem in hand, the natural next move is to classify your stack: Postgres is CP, Cassandra is AP, done. Kleppmann’s essay exists to demolish that taxonomy, and his counterexamples are the databases you actually run:

His conclusion: “The fact that we haven’t been able to classify even one datastore as unambiguously ‘AP’ or ‘CP’ should be telling us something: those are simply not the right labels to describe systems.”

The model behind the theorem is narrow too: a single read-write register, one kind of fault, and nothing about latency. That last gap is what Daniel Abadi’s PACELC adds: even when there’s no partition (the “Else”), you trade Latency against Consistency. It matches daily life better: most asynchronous replication exists because cross-region round trips are slow, not because anyone planned for partitions. Kleppmann isn’t impressed by PACELC either (“I don’t find that any more enlightening than CAP”), and I see why: it’s still buckets. But the latency half is the trade-off you’ll actually make every week.

So is CAP worthless? No. As Kleppmann puts it, in 2000 “it was meant to start a discussion about trade-offs in distributed data systems, and it did that very well.” As a one-line vaccine against believing you can have perfect consistency and perfect availability over an unreliable network, it still works. What fails is the classification scheme people built on top of it. The Jepsen project’s map of consistency models shows the real landscape: not two buckets but a dozen-plus models (linearizable, sequential, causal, read-your-writes, monotonic reads, eventual), each a different contract with a different price.

When NOT to Pay for Strong Consistency

Strong consistency is the safe default for correctness, which is exactly why you should know where it’s the wrong purchase:

The mirror list, where you must have it, is short and non-negotiable: anything the user just wrote and will immediately look at; anything that gates a decision (inventory before a sale, balance before a withdrawal); anything where two clients seeing different values triggers different side effects.

The Audit

  1. Draw the copies. For one critical entity (a user, an order), list every place a copy lives: primary, replicas, caches, CDN, search index, client state. Every arrow between them has a staleness bound. Write it down. “Unknown” is a finding.
  2. Find the read-your-own-writes paths. For every flow where a user writes and then immediately reads: does that read hit a replica or a cache? If it does, you have the bug from the opening, whether or not anyone has reported it yet.
  3. Classify your reads, not your database. Per endpoint: linearizable, read-your-writes, or eventual is fine. You’ll find the strong bucket is smaller than you feared, and currently unenforced.
  4. Check how your framework routes reads. If it sends every read to replicas indiscriminately, pin reads to the primary after writes for the flows in the second bucket. A few hours of work, and a whole class of bugs disappears.
  5. Retire “CP/AP” from your design docs. Name the actual guarantee per operation. Precision is free here and saves entire meetings.

Next in the series: observability, because none of this is debuggable if you can’t tell which replica served the stale read.

References