Lesson 1 of 7 · Core Primitives
SRE Fundamentals & Basics: What Every SRE Must Know
In almost every SRE interview, the first question is: "Explain SLI, SLO, SLA, Error Budgets, and
Toil." Master the concepts below and test them in the live calculator.
PIZZA DELIVERY ANALOGY:
• SLI (Indicator — The Measurement): Using a stopwatch to measure how many pizzas
actually arrived in under 30 minutes (e.g., 99.2% arrived on time this month).
• SLO (Objective — Our Internal Goal): The kitchen team's internal goal that 99.0%
of pizzas must arrive under 30 minutes.
• SLA (Agreement — The Customer Contract): The legal promise printed on the box: "If
fewer than 95% arrive on time, you get a $10 refund."
| Term |
Exact Meaning |
Real Production Example (Checkout API) |
Who Owns It |
| SLI (Indicator) |
The actual metric measuring user-facing service health. |
Percentage of HTTP 200 OK responses faster than 300ms at the load
balancer. |
SRE + Backend Eng |
| SLO (Objective) |
The target threshold we want our SLI to stay above. |
99.9% of requests succeed within 300ms over a rolling 30-day window. |
Product + SRE |
| SLA (Agreement) |
External business contract with financial credits if breached. |
99.5% monthly uptime; 10% customer invoice credit if breached. |
Legal / Business / Execs |
| Error Budget |
100% − SLO%. Allowed unreliability before feature freeze. |
0.1% (43m 12s per month or 1,000 failed requests per 1 Million). |
Shared across Engineering |
| Toil |
Manual, repetitive operational work with no lasting value. |
Manually restarting stuck pods, clearing logs by hand, approving routine tickets. |
Capped at <50% of SRE time |
SIMPLE FORMULA: Error Budget = 100% − SLO%.
If your monthly SLO is 99.9%, your Error Budget is 0.1% (43 minutes
12 seconds of allowed downtime per 30 days). While budget remains, teams ship features fast; if it
hits zero, feature deploys freeze to fix stability.
Interactive SLO, Downtime & Burn Rate Calculator
30-Day Allowed Downtime
43 min 12 sec
Allowed Failed Requests
1,000 requests
Burn Rate Policy
0.40x (HEALTHY)
The 4 Golden Signals (What SREs Monitor)
Defined by Google SRE—if you can only monitor 4 metrics on a service,
monitor these:
- 1. Latency: How long it takes to respond to a request (e.g.,
p99 < 250ms). Always measure slow error responses too!
- 2. Traffic: How much demand is hitting your system (e.g.,
1,200 HTTP requests/sec).
- 3. Errors: The rate of requests that fail—either explicitly (`HTTP 500`) or
implicitly (`HTTP 200` with wrong content).
- 4. Saturation: How "full" your constrained resources are (CPU at 90%, Memory at 95%,
or DB connection pool full).
What is "Toil" (And the 50% Rule)?
Toil is work that is manual, repetitive, automatable,
tactical, and scales linearly as traffic grows.
- Common Examples: Manually restarting stuck containers, running ad-hoc SQL exports,
copying files between servers, or approving routine IAM tickets.
- The 50% Rule: An SRE must spend less than 50% of their time on
toil/on-call tickets, and at least 50% writing software & automation so yesterday's
toil never repeats tomorrow.
HOW PRODUCTION SYSTEMS SURVIVE FAILURES: Hardware crashes, cloud zones go dark, and networks
drop packets every day. SREs design systems with Zero Single Points of Failure (SPOF) so one
broken server never takes down the website.
| Architecture Term |
Plain-English Meaning |
Real Production Example |
| SPOF (Single Point of Failure) |
Any single server or database that takes down the whole site if it dies. |
Running only 1 database server with no standby replica. |
| RTO (Recovery Time Objective) |
How much time can the site stay down after a disaster before business is severely
hurt? |
RTO = 15 minutes (We must failover to our backup region within 15 mins). |
| RPO (Recovery Point Objective) |
How much data (in time) can we afford to lose if the primary database explodes?
|
RPO = 5 minutes (Database backups/replication must be at most 5 mins behind). |
| Active-Passive Failover |
Primary server handles traffic; standby server sits idle waiting to take over if Primary fails. |
PostgreSQL Primary in Zone A + Warm Standby Replica in Zone B. |
| Active-Active Multi-Zone |
Multiple servers across different datacenters all serve live user traffic simultaneously. |
3 Kubernetes nodes across 3 Cloud Availability Zones behind a Load Balancer. |
1. Rate Limiting (HTTP 429)
Caps how many requests one user or bot can send per second so a sudden traffic spike or DDoS attack
cannot crash the backend.
2. Circuit Breaker
If a downstream service (like Payment API) is down, stop calling it for 30 seconds and fail fast instead
of hanging all user threads.
3. Exponential Backoff + Jitter
When retrying a failed request, wait 1s, then 2s, then 4s plus a random millisecond delay (jitter) so
retrying clients don't DDoS the recovering server.
INTERVIEW Q1 "Why shouldn't we aim for 100% reliability?"
How to answer: "Aiming for 100% uptime is exponentially expensive and
slows down new feature releases to zero. Plus, users' own Wi-Fi and ISP networks fail more often than
99.9%. Setting a realistic target like 99.9% gives us an Error Budget so we can innovate rapidly while
keeping users happy."
INTERVIEW Q2 "Why should your internal SLO be stricter than
your customer SLA?"
How to answer: "If our customer SLA is 99.5%, we set our internal SLO at
99.9%. That gives the SRE team an early warning buffer (safety margin) to detect and fix degradation
before we ever breach the legal customer contract."