Why does a cold cache overload a database?

A cache in front of a database does not make the database faster. It makes the database see fewer requests. With a 90% hit rate, nine reads in ten are answered from memory and one in ten reaches the database, so a system taking 2,500 reads a second sends the database about 250. Over months, the database gets sized for 250: a small instance, a 40-connection pool, enough CPU for that load and a little headroom.

A flush, a deploy that restarts the cache, or a new cache node with nothing in it sets the hit rate to zero. The database now receives all 2,500 reads a second, ten times what it was sized for. Its connection pool fills in well under a second. Queries queue behind the pool, response times climb from milliseconds to seconds, callers hit their timeouts and retry, and the retries add to the queue. The cache refills only as fast as the database answers, so the warm-up that should take seconds takes much longer, or never completes while the overload lasts.

This is why the incident looks like a database failure when nothing about the database changed. The cache was carrying load the database could never have carried alone, and the flush handed it back.