Design Desk

Observability & SRE

SLIs, SLOs & Error Budgets

An SLO is a deliberate choice of how much unreliability is acceptable, not a target to always beat, and spending the budget is supposed to change what the team does next, not just how the team feels about last month.

Observability & SREAdvanced7 min readPrerequisites: Availability, Metrics, Logs & Traces

After this lesson, you should be able to:

  • Distinguish an SLI, what's measured, an SLO, the target chosen for it, and an error budget, what's left before that target is broken.
  • Explain why an SLO is a deliberate ceiling on acceptable failure, not a target to always beat, and why comfortably exceeding one for a long stretch usually means it was set too low.
  • Diagnose why arguing an incident's severity with no agreed number turns into an unresolvable debate, and how a chosen SLO ends that.
  • Recognize the actual mechanism an error budget buys: once it's spent, the trade shifts from shipping features to restoring stability, until it resets.

At a Glance

  • An SLI is the specific thing you measure, an SLO is the target you chose for it, and an error budget is just that target flipped around: the amount of failure you're allowed before you've broken the number you promised.
  • An SLO isn't a target to always beat. Comfortably exceeding one for months usually means the number was set too low, not that the team is doing great.
  • The actual mechanism an error budget buys: once it's spent, the trade is supposed to shift from shipping features to restoring stability, until the budget resets. Without that mechanism, it's just a number on a dashboard nobody acts on.

01 The Problem

A notifications service, push notifications for order updates, has a rough month: a six-minute outage, a separate forty-minute one, and a slow rollout that left a chunk of users without any notification at all for about two hours. In the postmortem, half the team says this is clearly bad and needs a rewrite. The other half says notifications have always been best-effort, a delayed or missing push notification doesn't lose the company money, and this is a normal month, not an incident. Nobody can settle it, because there's no number anyone agreed to beforehand. There's nothing to check the actual month against. Every argument in the room is a vibe.

The team eventually picks a number just to end the argument: 99.99% uptime, the same target the payment service uses. Hitting it means rearchitecting the notification pipeline, a queue with real delivery guarantees, retries with backoff, a dead-letter queue, months of engineering time. It ships three months later. Nobody outside the team notices, because a push notification arriving four minutes late was never something users were measuring or complaining about in the first place. Real engineering time bought reliability nobody asked for, on a service that was already failing softly enough not to matter.

Neither problem was really about uptime. The first was arguing without a number. The second was picking a number without checking whether it was the right one.

02 What It Is

An SLI, service level indicator, is the specific thing you measure: percentage of requests that succeed, percentage served under 200ms, percentage of push notifications delivered within five minutes. It's usually one specific metric, chosen because it represents what a user actually experiences, not because it was the easiest number already sitting on a dashboard.

An SLO, service level objective, is the target you set for that SLI over some window: 99.9% of requests succeed, measured over a rolling 30 days. The error budget is the same number flipped around: 100% minus the SLO. A 99.9% SLO has a 0.1% error budget, roughly 43 minutes a month you're allowed to be down, or slow, or wrong, before you've broken the number you promised.

The part that's easy to miss: the SLO is a deliberate choice, not a ceiling you're trying to beat by the widest margin possible. Every nine of additional reliability costs real engineering effort, and past a certain point buys nothing anyone downstream needed. A team running at 99.995% against a 99.9% SLO for two straight quarters isn't necessarily doing great. They might be spending real effort on reliability nobody's using, effort that could have gone toward something users actually wanted. The number is supposed to be chosen to match what users or the business actually need, and then held to, not perpetually exceeded.

The error budget's other job is deciding what happens once it's gone. If a service has already spent its month's allowed downtime by the fifteenth, the agreed response isn't "keep shipping features and hope." It's freeze new feature work and put engineering time into stability until the budget resets. Without that second half, an SLO is just a number on a dashboard. With it, it's an actual decision rule for when the team's priorities are supposed to change.

In Plain Terms

A football team gets three timeouts a half. The goal was never to finish the half having used zero of them, an unused timeout in the last two minutes when the clock is running out and you desperately need to stop it is a wasted resource, not a badge of discipline. The skill is spending them when they matter and knowing, once all three are gone, that the way you play has to change: no more stopping the clock, every remaining decision has to account for a resource that's no longer there. The timeout count isn't a scoreboard of virtue. It's a budget, and the moment it hits zero, the actual strategy is supposed to shift.

03 How It Works

Rendering diagram…

The SLI is measured over the window. Landing above the SLO means budget is left and work continues as normal. Landing below it means the budget is spent, and the agreed response is to stop shipping features and spend engineering time on stability instead, until the next window resets it.

04 Trade-offs

 No SLO, reliability by vibesSLO with an error budget
Deciding if an incident mattersArgued case by case, no agreed number to check againstChecked against a number chosen in advance, the argument happens once, over the number, not every incident
How much reliability is enoughImplicitly 'as much as possible,' no stopping pointExplicitly capped, chasing more once the SLO is met spends effort nobody asked for
What happens once things break too muchAd hoc, whoever's loudest in the room decides whether to slow downThe budget being spent is itself the signal to freeze features and fix reliability first
Cost of over-deliveringInvisible, nobody notices reliability work that exceeded what was neededVisible, a service always far under its budget usually means the target was set too high

Watch out

It's tempting to treat 99.95% measured over 30 days as evidence things are fine. A single very bad day, an hour of total outage, can hide inside an otherwise-good 30-day window and still land above the SLO on paper. The rolling average smooths out exactly the kind of spike that hurt users the most on the day it happened. A team celebrating "we're within budget this month" might have had one genuinely bad afternoon that already burned through most of the budget, then coasted on three quiet weeks to look fine on the dashboard by the time anyone checked it.

05 Where You'll Actually See This

Availability already names the flip side of this in passing: "three nines" is 99.9%, and the unused portion is an error budget. This article is what that one sentence actually looks like as a practice, a deliberately chosen number, not a target to always beat, and a real behavioral consequence, freezing feature work, once the budget's gone, not just a percentage sitting on a slide.

Metrics, logs & traces is where an SLI's raw number actually comes from. An SLI is usually one specific metric, request success rate, latency under some threshold, the same kind of aggregated number that article covers, chosen specifically because it represents what a user experiences rather than whatever was easiest to already have a dashboard for.

Neither case study names an SLO explicitly, worth naming rather than glossing over. The payment gateway case study is exactly the kind of service that would need a tight one in practice, a failed payment path has real, immediate cost. The URL shortener's redirect path could reasonably run a much looser one, a missed redirect is a bad moment for one user, not lost money, and chasing payment-grade reliability there would be the same mistake the notifications team made.

Choosing the number is only half of it. Alerting & on-call covers the other half, what actually happens the moment this specific number starts moving the wrong way, and why a good alert here is usually built on how fast the budget's being spent, not just on whether it's already gone.

How to reason about it

When asked how you'd decide if a system is reliable enough, don't answer with "as available as possible." Name the actual number you'd target and where it came from, ideally tied to what users or the business actually need, not to what's technically achievable. Then say what happens once the error budget for that number is spent: reliability work has to take priority over new features until it resets. That's the actual mechanism an SLO buys, not just a health score on a dashboard.

06 Quick Check

A team is comfortably beating their 99.9% uptime SLO, running at 99.98% for the past two quarters. An engineer proposes spending the next quarter on infrastructure work to push toward 99.99%. What's the first question worth asking before approving that work?