Design Desk

Core Concepts

CAP Theorem

Why a network partition forces a choice between a wrong answer and no answer, and why that choice is usually made per operation, not once for the whole system.

Core ConceptsAdvanced5 min readPrerequisites: Availability

After this lesson, you should be able to:

  • State the three properties CAP theorem trades off, and explain why partition tolerance isn't really optional.
  • Explain why the CP-versus-AP choice is made per operation, not once for an entire system.
  • Distinguish what a CP system does during a partition (refuse) from what an AP system does (serve possibly-stale data).
  • Recognize CAP as a choice that recurs continuously in asynchronous replication, not only during rare network splits.

At a Glance

  • CAP only bites during a network partition. Outside of one, a system can be both consistent and available.
  • Partition tolerance isn't optional for a real distributed system, so the actual choice is CP vs AP, not "pick any two."
  • CP vs AP is usually a per-operation decision inside one system, not a single label for the whole thing.

01 The Problem

A checkout service keeps its order database replicated across two regions, us-east and eu-west, so a regional outage doesn't take the whole service down. Then the network link between the regions drops. Both regions are still up. They just can't talk to each other.

A customer in us-east tries to place an order. The us-east replica has two options. It can refuse the request until the link comes back, which is correct but means checkout is down for every customer routed to us-east, possibly for minutes. Or it can accept the order on its own, without confirming eu-west agrees, which keeps checkout up but risks a conflict: the same inventory item sold twice, or two orders assigned the same order number, discovered only once the link heals and the two regions compare notes.

Neither option is free. Refuse, and you've traded correctness for an outage. Accept, and you've traded an outage for a risk of a wrong answer. There's no third option where you get both, not during the partition. This is CAP.

02 What It Is

CAP names the three properties in tension: Consistency, every read reflects the most recent acknowledged write, or the system tells you it can't guarantee that. Availability, every request gets a non-error response, though not necessarily one that reflects the latest write. Partition tolerance, the system keeps functioning even when network messages between nodes are dropped or delayed.

Partition tolerance isn't really a choice. Networks fail: cables get cut, routers misbehave, regions lose connectivity to each other. Any system that runs on more than one machine has to decide what happens when that occurs, because it will occur. That's not an architectural preference, it's a fact about running on a network. Which means the theorem isn't actually offering a pick of three. It's offering a pick of two, and one of the two is always partition tolerance. The real decision is what to give up when a partition happens: consistency (CP) or availability (AP).

In Plain Terms

Two business partners run a small catering company together. One is out at a client site with no phone signal, the other is minding the office line. A new customer calls wanting the last open Saturday in June: the partner on the phone can book it immediately, risking that her partner already promised that Saturday to someone else that morning, or she can ask the customer to wait until she can reach her partner, risking that the customer just calls a different caterer instead. Say yes and maybe double-book, or wait and maybe lose the job. That's the whole tension, no computers required.

03 How It Works

Try it: partition the network, then accept a couple of orders in Region A (AP), watch the two counts diverge while Region B has no idea. Heal the partition and see the conflict surface, it doesn't resolve itself, there's no automatic reconciliation, only a Reset that starts a fresh run. Click Reset, then partition again and refuse instead (CP): heal it and the two regions still agree, because nothing was ever accepted without both sides.

Region A (us-east)

0

orders

In sync with Region B

Region B (eu-west)

0

orders

In sync with Region A

Both regions in sync. No partition yet.

04 Trade-offs

 CP (Consistency + Partition tolerance)AP (Availability + Partition tolerance)
During a partitionRefuses requests it can't guarantee are correctKeeps serving, using whatever data it currently has
What the user experiencesAn error or a timeout, but never a wrong answerA response, but possibly stale or later found to conflict with another replica's
RecoveryResumes once the partition heals and replicas can confirm agreementNeeds a conflict-resolution strategy for whatever diverged while split
FitsPayments, inventory counts, anything where a wrong answer is worse than no answerSocial feeds, view counters, anything where a slightly stale answer beats no answer

Watch out

It's tempting to read CAP as "pick any two of Consistency, Availability, Partition tolerance." You can't. Partition tolerance isn't a feature you opt into, it's a fact about running on a network that drops and delays messages. Every real distributed system has to tolerate partitions whether it wants to or not. So the theorem doesn't actually offer three choices. It offers one: when a partition happens, do you stay consistent (CP) or stay available (AP).

05 Where You'll Actually See This

The payment gateway case study leans CP for the same reason payments always do: a duplicate or a missed charge is worse than a request that briefly fails and gets retried, which is exactly what the idempotency-key mechanism is built to support on the client side of that guarantee. Availability covers what happens when a node fails outright. CAP is the same failure-domain question one axis over: what happens when the nodes are still up but can't talk to each other.

Database replication makes this concrete outside of a partition, too, not just during one. Reading from an asynchronous replica is choosing availability over consistency on every single read, the same trade CAP forces during a network split, just made continuously as a normal part of how the system runs.

Disaster recovery is this same choice made on purpose, in advance, instead of forced by an unplanned partition. Failing over to a standby region that's a few seconds behind is choosing availability over waiting for perfect consistency with a region that might not come back for hours, the same AP trade, just scheduled by the team instead of triggered by a network failure at 3am.

How to reason about it

When an interviewer asks whether your system is CP or AP, resist answering with one label for the whole architecture. Pick the specific operation they probably mean (a write, a read, a particular endpoint), say what happens to it during a partition, and name the trade-off you're accepting. A precise answer sounds like "the cart write is AP, the checkout write is CP, and here's why," not "we're AP."

06 Quick Check

Amazon's shopping cart stays available and lets you add items even during a network partition between data centers, but the actual payment charge at checkout gets rejected rather than risk double-charging. What does this tell you about CP vs AP?