Architecture Patterns
Circuit Breaker
A downstream dependency going slow can take down a perfectly healthy caller too, and the fix is knowing when to stop calling entirely, not just when to retry.
Architecture PatternsIntermediate8 min read
After this lesson, you should be able to:
- Explain what a circuit breaker does once tripped open: fail fast instead of waiting out a timeout on a call likely to fail anyway.
- Diagnose why a single successful half-open test call isn't enough evidence a downstream has actually recovered.
- Distinguish a circuit breaker's reactive behavior from rate limiting's proactive policy, even though both cap traffic to somewhere.
- Identify the caller-capacity benefit: threads freed immediately instead of held hostage by a downstream's timeout.
At a Glance
- A downstream dependency that starts timing out doesn't just slow the caller down, it can take the caller down too: every thread waiting on a 30-second timeout is a thread not serving any other request.
- A circuit breaker tracks failures and, past a threshold, stops calling the downstream entirely for a while, failing fast instead of waiting out a timeout that's very likely to happen anyway.
- The hard part isn't tripping open, that's a threshold check. It's the half-open state: deciding when it's safe to try again without immediately getting hit by the same flood of failures that tripped it in the first place.
01 The Problem
A checkout service calls a third-party shipping-rate API synchronously before completing an order, normally a 200ms round trip. The shipping API's infrastructure has a bad day and starts timing out, 30 seconds per call instead of failing fast. Checkout's thread pool has 50 threads, plenty under normal load. Once every call to the shipping API takes 30 seconds to fail, each checkout request holds a thread hostage for up to 30 seconds waiting on a dependency that's overwhelmingly likely to time out anyway. Within a couple of minutes, all 50 threads are stuck waiting, and checkout stops accepting any new request at all, not just ones that need a shipping rate, because there's no thread left to accept them. Checkout's own code never broke. It went down because of a dependency it calls.
Retrying those calls doesn't help, it's the same math with extra steps: each retry still waits up to 30 seconds before failing, consuming a thread twice as long. What actually needs to happen is checkout giving up on calling the shipping API at all, for a while, and failing those specific requests immediately instead of waiting 30 seconds to learn what everyone already knows is coming.
A circuit breaker does exactly that. It tracks failures to a downstream, and once they cross a threshold, it trips, and every call fails immediately, no network round trip, no 30-second wait, for as long as the circuit stays open. The hard part isn't tripping. It's knowing when to try again.
02 What It Is
A circuit breaker sits in front of a call to a downstream dependency and tracks its recent success and failure rate. In its normal, closed state, calls pass through as usual. Once failures cross a threshold, the circuit trips to open: every call fails immediately, without attempting the network call at all, for a configured cooldown period. After that cooldown, the circuit moves to half-open, allowing a small number of test calls through to check whether the downstream has recovered, then closing again if they succeed or tripping back open if they don't.
Tripping to open is a threshold check, easy to get right. The half-open state is the actual design problem. Close the circuit too eagerly, and the full flood of callers hits the downstream again the instant it opens, possibly before it's actually recovered, tripping it right back open, or worse, taking down a downstream that was in the middle of recovering. Stay open too conservatively, and callers keep failing fast long after the downstream would have handled them fine. Getting the half-open probe right, how many test calls, how often, what counts as success, is what separates a circuit breaker that actually protects the system from one that just adds a state machine on top of the same problem.
In Plain Terms
A restaurant stops seating new tables the moment the kitchen falls badly behind, rather than seating everyone and letting them wait an hour for food while servers stand around unable to do anything else. That's the easy call. The hard call is when to start seating again: let everyone back in the instant the kitchen clears one order, and the room floods before the kitchen's actually caught up, backing it right up again. Wait too long, and tables sit empty while the kitchen's been ready for twenty minutes.
03 How It Works
Rendering diagram…
04 Trade-offs
| No circuit breaker | Circuit breaker | |
|---|---|---|
| Failing downstream | Every call still waits out the full timeout before failing | Fails immediately once open, no wait on a call that's overwhelmingly likely to fail anyway |
| Caller's own capacity | Threads pile up waiting on timeouts until the caller itself runs out of capacity | Threads are freed immediately instead of held hostage by a downstream's timeout |
| Recovery | The moment the downstream recovers, all callers resume normally | Needs a half-open probe to detect recovery; a bad probe can delay closing or reopen the circuit prematurely |
| Complexity | Nothing extra to configure beyond a timeout | A threshold, a cooldown, and a probe strategy, all of which can be tuned wrong |
Watch out
It's tempting to treat a circuit breaker and a retry with backoff as the same idea, both react to a struggling downstream. They're not. Retrying with backoff still calls the downstream, just less aggressively, the right response when it's temporarily slow but working. A circuit breaker stops calling entirely for a while, the right response when the downstream is actually down and every call is wasted latency. Conflating the two means either hammering a downstream that's already down, or giving up on one that only needed a slower retry.
05 Where You'll Actually See This
The payment gateway case study's call to the card processor is exactly the kind of external dependency a circuit breaker belongs in front of: a processor outage shouldn't take down the gateway's own capacity to answer merchants at all, even the ones whose requests don't touch the processor at that exact moment. The case study as written doesn't include one, worth naming as a real gap between the scaffold and what a production version would need.
Message queues' retry-with-backoff is a different answer to a similar-looking problem, worth keeping distinct rather than treating as the same mechanism. Backoff still calls the downstream, just with growing gaps between attempts, which works when the downstream is slow but functioning. A circuit breaker stops calling altogether for a while, which is what you actually want once the downstream is genuinely down and every attempt, backed off or not, is wasted. A consumer calling a struggling downstream can reasonably use both: a circuit breaker deciding whether to call at all, backoff governing the pace of retries while the circuit's still closed.
Rate limiting caps how much load you send somewhere on purpose, a policy decision made in advance. A circuit breaker reacts to a downstream that's already failing, a response to what's actually happening right now. One is proactive, about your own traffic. The other is reactive, about someone else's health.
A tripped circuit is exactly the kind of event metrics, logs & traces needs to explain as one story, not three separate ones. The open state is a metric, the trip is a log line, and a trace running through a request that hit the open circuit shows the fast-fail directly instead of the 30-second timeout the caller would otherwise have waited out. Without a trace tying it together, you're left guessing whether a latency alarm and a "circuit opened" log line fifteen minutes apart are the same incident or a coincidence.
A circuit tripping open is also a reasonable candidate to page someone over, which alerting & on-call covers directly. It's worth being precise about what that page actually means, though: real requests are failing against a specific dependency right now, which is different from the whole service's SLO being in danger. Paging whoever owns that dependency is usually the right call. Treating every trip as a full incident is not.
Worth being just as precise about health checks & readiness probes, which looks like the same idea from a distance but isn't. A health check is external and periodic, something outside the process polling it on a timer, whether or not a real request is happening. A circuit breaker is internal and request-triggered, only reacting because actual calls actually failed. A service can be failing every health check and never trip a single breaker, and just as easily pass every health check while a breaker in front of one specific dependency stays wide open.
None of this is worth much if the breaker's threshold, cooldown, and half-open probe have only ever been checked by reading the configuration. Chaos engineering is how a team finds out whether a breaker that looks correct on paper actually trips against a real failure, on a schedule they chose, instead of during the first live outage that happens to exercise it.
How to reason about it
When asked how you'd protect a service from a failing dependency, don't just say "add a circuit breaker." Say what happens in each of its three states, and specifically say how you'd decide a half-open probe succeeded, because that's the part that actually determines whether the circuit protects the system or just flaps open and closed under load. A breaker that closes on one successful test call behaves very differently from one that requires ten consecutive successes.
06 Quick Check
A circuit breaker trips open on a failing payment processor. After the cooldown, it enters half-open and lets exactly one test call through, which succeeds, and immediately closes the circuit, sending full traffic back to the processor. The processor immediately falls over again. What's the most likely problem with this design?