← System Design · Lesson 6 of 11
Consistency and CAP
Partitions happen; the only real choice is what you give up when they do.
The idea
CAP is usually stated as "pick two of consistency, availability, partition tolerance", and that framing is misleading enough to be worth correcting.
Partition tolerance is not optional. Networks fail. If you run on more than one machine, you will have partitions, and a system that cannot tolerate them simply breaks. So the real choice is between the other two, during a partition:
- CP — refuse to answer rather than risk a wrong answer. The minority side of the partition returns errors. Correct for money, inventory, and bookings.
- AP — answer from whichever side you can reach, accepting that the two sides may disagree until they reconcile. Correct for feeds, likes, and analytics.
PACELC completes the picture, and is the more useful framing in practice: during a Partition, choose Availability or Consistency; Else — the 99.9% of the time when there is no partition — choose Latency or Consistency. That second half is where the everyday cost lives. Strong consistency across regions means every write pays a cross-region round trip, partition or no partition.
The most useful move in an interview is to stop treating consistency as one global setting. It is per-operation. The same system can require strong consistency for "charge this card" and eventual consistency for "increment the view counter", and saying so is a stronger answer than picking a side.
Finally, "eventual" is a promise with no deadline attached. It is worth asking how long convergence actually takes, because for most systems the honest answer is milliseconds — and a millisecond of staleness is acceptable in far more places than the phrase suggests.
Walkthrough
When to reach for it
- Strong consistency for money, inventory, seat booking, and anything where a double-spend is unacceptable.
- Eventual consistency for counters, feeds, recommendations, and analytics.
- Read-your-writes wherever a user would otherwise think their action failed.
- Causal consistency for messaging and comment threads, where order carries meaning.
Rather than the obvious alternative
One consistency level for the whole system
The most common mistake. Consistency is chosen per operation; forcing the strongest level everywhere pays cross-region latency to protect a view counter.
Distributed transactions (two-phase commit)
They give strong consistency across services and hold locks across the network, which makes availability worse. A saga with compensating actions is usually the better trade.
Key terms
- CAP theorem
- Under a network partition a distributed system must choose between consistency and availability.
- Eventual consistency
- Replicas converge given enough time without new writes.
- Strong consistency
- Every read observes the most recent write.
- Quorum
- Requiring a majority of replicas to acknowledge a read or write.
How to spot it
- The interviewer asks what happens when the network splits.
- The domain involves money, inventory, or a limited resource.
- A user might see their own action fail to appear.
- The system spans regions, which makes the latency half of PACELC the real question.
Where it goes wrong
Reciting "pick two"
Partition tolerance is not a choice for a distributed system. Framing it as CP-or-AP during a partition shows you understand what the theorem actually says.
Treating eventual consistency as a defect
It is the correct choice for most data. The question is whether the convergence window is acceptable for that specific operation.
Ignoring the "else" in PACELC
Most of a system's life has no partition. The latency cost of strong consistency is paid constantly; the partition benefit is claimed rarely.
Assuming a single database is immune
A primary with async replicas is already an eventually consistent system from the reader's point of view.