01 What it is

Scalability is a system's ability to handle more load by adding resources, not by making the existing resources work harder. There are two ways to add resources: vertical, a bigger machine with more CPU, RAM, or disk, and horizontal, more machines doing the same job, coordinated by something in front of them. That distinction is the entire subject. Vertical scaling has a ceiling, there's a biggest machine you can buy. Horizontal scaling doesn't, at the cost of coordinating state across machines that don't share memory.

A system scales when doubling the machines roughly doubles the throughput. A system where doubling the machines doesn't move throughput, because everything still serializes through one database connection, one lock, or one queue, isn't scalable, no matter how much hardware you throw at it.

02 Why it matters

Without horizontal scale, you hit a ceiling on the single biggest machine you can rent, vertical scaling has diminishing returns and a hard stop, and the only lever left is to slow down. The failure isn't gradual. Once queueing overtakes the resource, latency doesn't creep, it stacks, and every request slows down while it does, timeouts cascade, and p99 collapses into p50-gone. The design decision that actually causes this is rarely "we didn't buy enough hardware." It's usually "we designed with state that assumed a single instance": an in-memory session, a local file cache, a singleton in-process queue. That's the part that makes horizontal scaling impossible to retrofit without a rearchitecture.

In Plain Terms

A single toll booth can only process cars one at a time, so on a busy morning the line backs up no matter how fast the operator works: that's a ceiling you can't out-hustle your way past. Open four more identical booths and five times as many cars can get through at once, cars just pick whichever booth is free. But that only works if every booth can actually complete the transaction on its own. If four of the booths take your money but still have to radio one specific booth to confirm your ticket before raising the gate, you've built five booths and kept the same one-booth bottleneck.

03 How it works

Rendering diagram…

The mechanism is statelessness, not the instance count. Three instances that each cache a different answer in local memory aren't interchangeable, and adding a fourth doesn't fix that.

04 Trade-offs

	Vertical scaling	Horizontal scaling
Ceiling	Hard stop at the biggest machine you can buy	No inherent ceiling. Keep adding instances.
Code changes required	Usually none. Same single process, more resources under it.	Requires stateless instances; anything held in local memory has to move to a shared store first.
Failure blast radius	The one bigger machine is still a single point of failure	Losing one instance loses roughly 1/N of capacity, not all of it
Cost curve	Gets expensive fast. Bigger machines cost more than linearly.	Roughly linear. N machines cost about N times one machine.

05 Where you'll actually see this

The URL shortener case study's app server is stateless by design: it holds nothing that Redis or Postgres doesn't already have. That's precisely what lets the app tier in the High-Level Design diagram scale horizontally, adding another instance is a capacity change, not a rewrite.

That diagram assumes something in front of those instances actually routes requests to them, without ever drawing it. Load balancers is that missing piece: the mechanism that turns "stateless instances" into instances anyone can actually reach.

Worth being explicit about what horizontal scaling actually buys you: more throughput, not necessarily lower latency. Ten instances handle ten times the concurrent requests, but a single request routed to any one of them still takes as long as it takes, sometimes slightly longer once you add the coordination cost of a shared store or a load balancer hop. Scaling out fixes "too many requests at once." It doesn't fix "this one request is slow."

How to reason about it

When you say a system "scales horizontally," be ready to say what happens at the one thing every instance still has to touch, a database, a lock, a shared cache, because that's usually where the real ceiling is. Adding instances helps exactly as much as that shared dependency lets it and not a request more. If you can't name what that dependency is, you haven't actually checked whether the system scales, you've just counted servers.

06 Quick Check