Web-scale
Design a URL Shortener
A read-heavy redirect service, and why the code-generation scheme you pick decides whether collisions are even possible.
A URL shortener takes a long URL and returns a short code that redirects back to it. The two endpoints are POST /shorten (write) and GET /:code (read), and the read path dominates traffic by a wide margin. Everything below follows from that one asymmetry.
01 Requirements
Functional
- Given a long URL, mint a short code and store the mapping (
POST /shorten). - Given a short code, redirect the caller to the original long URL in one hop (
GET /:code). - Support an optional custom alias when the caller asks for one and the slot is free.
- A code that never existed, or has expired, returns a stable 404, not a slow timeout.
Non-functional
- Read-heavy by orders of magnitude, so redirect latency matters far more than shorten latency.
- Redirects need to pick 301 vs 302 deliberately: a 301 gets cached by the browser and you lose click analytics on repeat visits, a 302 doesn't.
- Two concurrent writes must never mint the same code for two different URLs.
- On a cache outage, degrade to slower reads through the database rather than serving a wrong or stale mapping.
02 Estimation
Plain napkin math, same as the payments case study.
Say the service serves 1,000 redirects/second on average against a 20:1 read:write ratio, so roughly 50 shortens/second. Over a few years that's on the order of 1 billion stored URLs. At around 500 bytes a row (long URL, code, metadata) that's roughly 500GB in Postgres: unremarkable, durable storage is not the constraint here.
Traffic to short links follows a Pareto-ish curve: a small slice of codes account for most reads. Caching the hottest 10 million codes at roughly 150 bytes an entry is about 1.5GB, comfortably in memory, and covers the bulk of read traffic without Postgres seeing most of it.
One more number worth checking before picking an encoding: a 6-character base62 code space is 62^6, about 56 billion combinations. Against 1 billion stored URLs, that's a lot of headroom, which is exactly what makes the encoding choice in the Compare section below a real option instead of a forced move.
03 High-Level Design
Rendering diagram…
The app server is the only thing that talks to Postgres and Redis directly. A client only ever sees POST /shorten and GET /:code.
04 Deep Dive
Given the 20:1 read:write ratio from the estimation above, the hit path is the one that runs almost every time a redirect happens. If it doesn't stay in Redis, every one of those reads falls through to Postgres, and the database is now doing the job the cache was sized for. The miss path exists for the long tail: codes nobody has clicked recently, which is most codes, just not most requests.
05 Compare
The single trade-off that decides how codes get minted: predictable and collision-free, or unpredictable and collision-prone.
| Base62 counter | Hash + collision check | |
|---|---|---|
| Code generation | Auto-increment ID encoded in base62 | Hash of the long URL, truncated |
| Collision risk | None: IDs are unique by construction | Possible: requires a uniqueness check |
| Code predictability | Sequential, guessable | Non-sequential, harder to guess |
| Custom aliases | Awkward to support | Natural fit alongside hashed codes |
A base62 counter never needs a uniqueness check because it can't collide. A hash-based code can, which means the write path either has to check-then-insert (a race window) or lean on a unique constraint and retry on conflict. That's a real cost for a property (unpredictable codes) most URL shorteners don't actually need.
06 Quick Check
Why does a base62-encoded auto-increment counter avoid collisions entirely?