Caching

The cheapest performance win there is, and the easiest way to serve stale data.

The idea

Caching keeps a copy of expensive-to-produce data somewhere cheap to reach. It is usually the highest-leverage change available: the numbers in the table above differ by four orders of magnitude, and a cache turns the bottom row into the top one.

The reason it is not free is that a cache is a second copy of the truth, and the moment there are two copies they can disagree. Every hard caching question is some version of "how wrong are we willing to be, and for how long?"

Invalidation is where that question is answered. Three strategies, in increasing order of difficulty:

TTL — let entries expire after a fixed time. Simple, predictable, and bounded staleness. Correct for the large majority of cases.
Explicit invalidation — delete the entry when the underlying data changes. Fresher, but every write path now has to remember, and one that forgets is a bug you find months later.
Versioned keys — put a version in the key so new data writes to a new key and old entries age out on their own. No invalidation logic at all, at the cost of holding both copies briefly.

What not to cache matters as much: data that must be exactly right at read time, data cheap to compute anyway, and anything user-specific being stored in a shared cache — which is how one user is served another user's page.

Finally, know the failure mode. A cache stampede is what happens when a popular key expires and a thousand requests all miss simultaneously and all hit the database at once. The fixes are to stagger expiry with jitter, or to let one request repopulate while the others serve the stale value.

Walkthrough

No walkthrough for this topic yet — the guided exercises below run the same way.

When to reach for it

Reads vastly outnumber writes and the same data is requested repeatedly.
Producing the value is expensive — a join, an aggregation, an external API call.
Slightly stale data is acceptable, which is true far more often than people admit.
You need to absorb traffic spikes without scaling the database behind it.

Rather than the obvious alternative

A read replica

Gives fresh data and scales reads without staleness, at the cost of a full database rather than a key-value store. Prefer a replica when correctness at read time matters; prefer a cache when latency does.

A database index

Often the real fix. A query taking 500 ms because it lacks an index does not need a cache in front of it; it needs the index. Cache after optimising, not instead.

Materialised views

Precomputed in the database itself, so there is no second system and no invalidation code. Slower to update, and the right answer for expensive aggregations that change rarely.

Should this be cached, and where?

A cache is a correctness risk you take on purpose. This walks the decision the way it is actually made — starting with whether to take it at all.

Is the read expensive enough, and repeated enough, to be worth stale data?

Caching trades freshness for speed. If you are not buying much speed, you are paying the freshness for nothing — and every cache is a second copy of the truth that can disagree with the first.

Key terms

Cache-aside: The application checks the cache, and on a miss loads from the store and populates it.
Write-through: Writes go to cache and store together.
Eviction policy: The rule deciding what leaves a full cache — LRU, LFU, FIFO.
Cache stampede: Many requests miss the same expired key at once and all hit the origin.

How to spot it

The same expensive result is requested many times.
A read-heavy workload with a clear hot set — the top posts, the popular products.
The interviewer asks how to reduce database load or tail latency.
Data has a natural freshness tolerance measured in seconds or minutes.

Where it goes wrong

No invalidation strategy at all

A cache with no TTL and no invalidation serves the first value it ever saw, forever. Decide the staleness budget before adding the cache.

Caching user-specific data in a shared key

The classic security incident: one user is served another user's page. Scope the key by user, or do not cache it.

Cache stampede on expiry

A thousand simultaneous misses on a hot key can take down the database the cache was protecting. Add jitter to TTLs, or serve stale while one request refreshes.

Treating the cache as durable

It is not a database. The system must work, slower, with the cache entirely empty — that is what happens after every restart.

Build it

You have read how it works. This is where you write one — across several files, one rule at a time, run against tests in your browser.