1Context and scope
An online shop shows product pages. Almost every request reads a product; a few update stock or prices. The database is the source of truth, and a Redis cache keeps popular products in memory so most reads never touch it.
2Goals and non-goals
Goals
- 2,000 requests a second: 90% reads, 10% writes.
- Success at least 99%, p99 latency at most 300 ms, Apdex at least 0.90.
- Keep working when the cache is flushed (a deploy, a restart, a failover).
Non-goals
- Search and recommendations: other services own them.
- Read-after-write consistency on product pages: a few seconds of staleness is acceptable.
- Running in several regions.
3Capacity estimate
Back-of-the-envelope numbers from the template. The simulation uses the same ones, so you can check them against a run.
| Quantity | Value | How |
|---|---|---|
| Reads | 1,800/s | 2,000 × 90% |
| Reads that reach Postgres | 180/s | misses at a 90% hit rate |
| Queries on Postgres | ≈ 380/s | misses + 200 writes/s |
| Postgres CPU capacity | ≈ 1,330 queries/s | 2 vCPU ÷ 1.5 ms of CPU per query |
| After a flush | ≈ 2,000 queries/s | every read misses: 1.5× what the CPU can do |
4The design
Each component in the template, in the order a request meets it, and why it is there.
- Load balancer
Spreads requests over the API replicas and is where traffic enters.
- API (3 replicas)
Stateless application code. It checks the cache first and reads Postgres only on a miss (cache-aside).
- Redis cache
Holds hot products in memory and answers in about a millisecond. It is a copy, not the truth: it can be emptied at any time.
- Postgres
The source of truth for products and stock. Every write goes here; reads only on a cache miss.
5Key concepts
Cache-aside
The application owns the cache: read the cache; on a miss read the database and fill the cache; on a write update the database and invalidate the cached copy. Simple and common, but the database must survive the misses.
Hit rate is a load multiplier
At 90% the database sees 1 read in 10; at 95%, 1 in 20; at 0% (cold), all of them. A small change in hit rate is a large change in database load, which is why a cold cache is dangerous.
Cache stampede (thundering herd)
When the cache empties, every request misses at once and all of them hit the database together. The database was sized for the misses, not for everything, so CPU pins at 100% and it can crash, which makes recovery even slower.
Warm-up
After a flush the hit rate climbs back over time as reads refill the cache. The warm-up time constant in the cache settings controls how long the database is exposed.
6Alternatives considered
- More cache memory raises the hit rate but costs money and does nothing for a cold start.
- Read replicas absorb the miss storm but can return slightly stale data.
- Single-flight locking (one request reloads a key, the rest wait) protects the database but adds latency to those waiting.
7Failure modes and fixes
What goes wrong in the simulation, why, and the change that fixes it. Each one shows up as a finding in the lab; ask “what broke?” to see it, and for the fix when you want it.
At 20 s the cache is flushed and Postgres crashes on CPU.
WhyAbout 2,000 queries a second arrive at a database that can do about 1,330. CPU stays at 100% for 8 s and the crash rule takes it down; requests fail until it restarts and the cache warms.
FixGive Postgres headroom for a cold cache (more vCPU or a read replica), warm the cache before traffic returns, or cap the refill rate so only one request per key reloads it.
8Try it in the lab
Open the template, then type these into the copilot one at a time. The first few are also offered as buttons when the template opens.
flush the cacheHit rate drops to 0%, Postgres goes red, success falls.
add a read replicaThe flush no longer crashes Postgres; compare before and after.
give the database 8 vcpuMore CPU headroom for the miss storm; cost goes up.
cache hit rate 95%Half as many reads reach Postgres in normal running.
stress testHow much traffic it takes before it breaks, and what breaks first.
9Check yourself
- Why does a 5-point change in hit rate (90% to 95%) halve the database read load?
- What three things could you do so a cache flush does not take the database down?
- Why is a database crash during a stampede worse than the stampede itself?