Caching - The Complete Guide

A STANDALONE REFERENCE · EVERYTHING ABOUT HOW CACHES ACTUALLY BEHAVE

Reading from RAM takes nanoseconds. Reading from a database, over a network, often through a slow query - that's milliseconds to whole seconds. That gap, five to six orders of magnitude wide, is the entire reason caching exists: it's cheaper to keep a copy close by than to re-earn the same answer from far away every single time. A cache is just a small, fast copy of data that's normally slow or expensive to fetch. That idea is simple. Everything else on this page is what happens once real traffic, real failures, and real time get involved - how a cache decides what to keep, when to throw something away, and everything that can quietly go wrong in between.

1. How a Cache Actually Works

The first request is always the slow one. Every request after that is the fast one (looping)

A cache is just a small, fast shelf sitting in front of a slower one
First time you ask for something, it isn't there yet - you pay the full slow price, then a copy gets kept
Every time after that, the copy answers instead - much faster
Found it in the cache = a "hit." Didn't = a "miss," and off to the slow source you go

2. Where Caches Actually Live

Every layer between a user and a disk is caching something

Browser - your device keeps its own local copy
CDN / edge - a copy sitting geographically close to you
App-local cache - fastest, but only that one server instance can see it
Distributed cache (Redis, Memcached) - one shared copy every app server can reach
Database's own buffer pool - even the database caches its hottest pages in memory

🧭 Closer to the user = faster, but smaller and more scattered. Closer to disk = slower, but one shared, complete copy.

3. Reading From a Cache: Two Ways

Who's responsible for going to fetch the real answer?

Cache-Aside (Lazy Loading)

Your app checks the cache itself, directly
Not there? Your app fetches it and saves a copy
The most common approach - simple, explicit

Read-Through

The cache itself does the fetching, quietly
Your app only ever talks to the cache
Simpler app code, one more moving part to run

🧊 Simplest way to remember it: Cache-Aside - you check the fridge, and if it's empty, you walk to the store. Read-Through - you call a stocked pantry service, and it walks to the store for you.

4. Writing Through a Cache: Three Ways

The real question: where does a write actually go, and in what order? (looping)

Write-Through

Every write updates the cache and the database, together, right away. Safest - a little slower, since you're doing two things at once.

Write-Back

The write hits the cache first - fast! The database catches up later, on its own. Quick, but risky if the cache disappears before it catches up.

Write-Around

The write skips the cache entirely and goes straight to the database. Good for data you write once and rarely read again soon after.

5. TTL - Giving Every Cached Item a Timer

TTL just means "Time To Live" - a countdown clock stuck to each cached item (looping)

Short TTL

Data stays fresh - it doesn't sit around wrong for long
But you ask the real, slow source much more often

Long TTL

Far less load on the real, slow source
But people might see stale, outdated data for longer

⏲️ One extra trick - sliding TTL: instead of a fixed expiry, reset the timer every time the item gets used. Popular items almost never expire; forgotten ones quietly do.

6. Eviction Policies - What Gets Thrown Out When It's Full

A cache has limited shelf space. The eviction policy is just the rule for choosing what leaves (looping)

LRU

Least Recently Used. Throws out whatever hasn't been touched in the longest time. Shown above - the most common default.

LFU

Least Frequently Used. Throws out whatever has been asked for the fewest times, ever - total count, not recency.

FIFO

First In, First Out. Throws out whichever item arrived first, no matter how often it's actually being used.

7. Cache Invalidation - The Hardest Part

"There are only two hard problems in computer science: naming things, and knowing when a cached copy has gone stale." This is that second one (looping)

Wait it out (TTL): simplest option - but the copy stays wrong until the timer runs out
Write-invalidate: the instant the real data changes, delete or update the cached copy too - fast, but you must remember to do it everywhere data can change
Event-driven (pub/sub): broadcast a "this changed!" message to every cache holding a copy - works even with many separate caches, needs real messaging infrastructure

8. Real-World Architecture - Who Actually Builds This

Nobody hand-rolls a cache from scratch. Here's what's actually running in production

Redis

Memcached

CDN Edge
(Cloudflare, Akamai, Fastly)

Varnish / Nginx

In-Process
(Caffeine, Guava)

Redis - does far more than key-value: sorted sets, lists, pub/sub for broadcasting invalidations. Can persist to disk (RDB snapshots, AOF logs), so a restart doesn't mean starting fully cold
Memcached - simpler by design: pure key-value, multi-threaded, no persistence. The lighter choice when you don't need Redis's extra data structures
CDN edge networks (Cloudflare, Akamai, Fastly, CloudFront) - cache static and semi-static content geographically close to users, driven almost entirely by the Cache-Control and ETag headers your origin sends
Varnish / Nginx - sit in front of your app as a reverse proxy, caching whole HTTP responses. Extremely fast, HTTP-aware, doesn't know or care what's inside the response
In-process (Caffeine, Guava Cache, Node's lru-cache) - lives inside your app's own memory, no network hop at all - the fastest option, but only that one process can see it

9. Failure Scenarios & Edge Cases

What actually breaks a cache in production - on purpose, so you recognize it later

Cache stampede / "thundering herd": one popular key expires, a pile of identical requests all miss in the same instant, all hit the database together (above)
Locking / coalescing, stale-while-revalidate, and jittered expiry are the three real mitigations - combine at least two, don't rely on just one

The subtler one - a slow reader can silently undo a fast writer's correct invalidation (looping)

The classic cache-aside race: a slow reader can read old data, then finish writing it into the cache after a faster writer already invalidated that same entry - leaving the cache wrong until the TTL finally saves it
Cold-cache thundering herd: right after a deploy or restart, the entire cache is empty at once - the stampede problem above, except every key, not just the hot one
Cache poisoning: accidentally caching an error response, or someone else's personalized data under a shared key - now everyone gets served the mistake
Multi-node inconsistency: a distributed cache spread across several nodes can briefly disagree with itself during a rebalance, exactly like topic 12 in the main series

10. Scale & Numbers

Rough math - the numbers that tell you whether a decision above actually matters yet

📉 Why hit ratio isn't a vanity metric

Say your database comfortably handles 2,000 queries per second before it struggles. Real traffic is 50,000 requests per second - 25x too much for the database alone. Put a cache in front with a 96% hit ratio, and only the 4% that miss ever reach the database: 4% of 50,000 = 2,000 QPS - exactly what it can handle. The cache isn't a nice-to-have here. It's the only reason the database survives.

💾 Sizing the cache itself

Say you're caching user profile objects, roughly 2KB each, for your 10 million most active users. That's 10,000,000 × 2KB ≈ 20GB of raw data. Real caches carry overhead too - pointers, metadata, expiry timestamps - typically another 20-40% on top. Budget closer to 25-28GB of actual memory for this.

⚡ What a hit ratio actually buys you in latency

A cache hit from memory: roughly 1ms. A miss that falls through to the database: roughly 75ms. At a 96% hit ratio, average latency is (0.96 × 1ms) + (0.04 × 75ms) ≈ 3.96ms - about 19x faster than a flat 75ms, even though 1 in 25 requests is still slow. This is exactly why "do you have a cache" is the wrong question - "what's your hit ratio" is the right one.

11. Trade-offs at a Glance

Every decision above, on one page, for whenever you actually need to choose

Decision	Option	What you get	What it costs you
TTL Length	Short	Always close to fresh	More trips to the real source
TTL Length	Long	Far less load on the source	Data can be visibly stale
Eviction Policy	LRU	Keeps what's currently "hot," adapts fast	A brief burst can evict genuinely popular items
	LFU	Protects long-term favorites	Slow to adapt when popularity actually shifts
	FIFO	Dead simple to implement	Ignores usage entirely - can evict something hot
Invalidation	TTL only	Zero extra code, "just works"	Guaranteed staleness for part of every TTL window
	Write-invalidate	Near-instant correctness	Must be triggered at every single write path
	Event-driven	Correct across many caches at once	Real messaging infrastructure to build and run
Stampede Protection	None	Nothing to build	A real, occasional risk of an outage
	Locking	The database sees exactly one request	Everyone else waits a little
	Stale-while-revalidate	Nobody ever waits	Briefly serves an answer you know is outdated
	Jittered expiry	Simple, spreads out the risk	Reduces the herd, doesn't fully rule it out

12. Interview Framing

The questions that actually get asked, and the sharpest honest answer to each

Q: How would you handle a cache stampede on your most popular endpoint?

Protect the hot key specifically: locking or request coalescing so only one request refetches while the rest wait, stale-while-revalidate so nobody has to wait at all, and jittered TTL so keys don't all expire in the same instant. Combine at least two of these - don't rely on just one.

Q: What's the actual difference between Redis and Memcached?

Memcached is simpler - pure key-value, multi-threaded, no persistence. Redis does much more: rich data structures, pub/sub, optional persistence. A fast disposable cache favors Memcached; needing atomic counters, sorted sets, or the cache doubling as pub/sub infrastructure for invalidation favors Redis.

Q: Walk me through what happens if your cache goes down entirely.

Every request that would've hit cache now hits the database directly - the same shape as a stampede, except permanent instead of momentary. Never treat a cache as load-bearing for correctness: have a circuit breaker or graceful-degradation path, and make sure the database can survive reduced service rather than fall over completely.

Q: How do you keep a cache consistent with the source of truth?

TTL, write-invalidate, or event-driven - chosen by how much staleness is tolerable. The detail people miss: even write-invalidate has a race (section 9) - a slow reader can overwrite a fast writer's invalidation with stale data. The safer pattern is invalidating the cache before writing the database, or using a versioned write so a stale write can be detected and rejected.

Q: When would caching make things worse, not better?

When data is written far more than it's read - all the invalidation cost, none of the benefit. When data must be immediately, strongly consistent - a live account balance. And when data is highly personalized, one-shot content that's never re-read - a cache that's only ever used once has the full cost and none of the payoff.

13. Putting It All Together - A Decision Guide

Three real situations, and how the choices above actually combine

📄 Caching a product page that rarely changes

Longer TTL (minutes, not seconds) + LRU eviction + write-invalidate the instant an editor updates it + stale-while-revalidate as a safety net for the rare cases invalidation is missed somewhere.

👤 Caching a logged-in user's session

Short-to-medium TTL, sliding - so an active session never suddenly expires mid-use + LRU eviction + write-through, since a session change (like logging out) has to be immediate, not eventually consistent.

🏠 Caching a home page every single visitor hits

Very short TTL is actually fine here - traffic is so high the cache refills itself constantly + jittered expiry + stampede protection is not optional - this is the single hottest key in the whole system, and the classic place a thundering herd actually happens.

14. Quick Reference - The Whole Thing, One Place

For whenever you just need the short version

Choosing a TTL:

✔ Freshness matters most → shorter

✔ Source load matters most → longer

✔ Item usage varies a lot → sliding

Choosing eviction:

✔ Traffic patterns shift often → LRU

✔ Some items are reliably popular → LFU

✔ You just need something simple → FIFO

Choosing invalidation:

✔ Staleness for a while is fine → TTL only

✔ Correctness matters a lot → write-invalidate

✔ Many separate caches to update → event-driven

Protecting against stampede:

✔ One hot key, everyone hits it → locking

✔ Latency must never spike → stale-while-revalidate

✔ Cheap insurance, low effort → jittered expiry

🏁 Caching is simple to start and easy to get subtly wrong. Every decision above is a trade-off - the goal isn't to avoid them, it's to make every single one on purpose.