Caching
Caching strategies, cache invalidation patterns, and when to use different caching layers.
What is Caching?
Caching stores frequently accessed data in a fast-access storage layer to reduce the load on slower backend systems. Think of it as a shortcut — instead of computing or fetching data every time, you retrieve it from a nearby, fast store.
Request → Cache Hit? ──Yes──→ Return cached data (fast!)
│
No
│
▼
Fetch from DB → Store in Cache → Return data
The 80/20 Rule
In most applications, 20% of the data serves 80% of the requests. Caching that 20% can dramatically improve performance.
Caching Layers
From closest to the user to farthest:
- Browser Cache — Static assets, API responses
- CDN — Geographically distributed static content
- Application Cache — Redis/Memcached for computed results
- Database Cache — Query cache, buffer pool
Cache Strategies
Read Strategies
Write Strategies
Cache Invalidation
The Hardest Problem
Cache invalidation is notoriously difficult. Stale data can cause subtle bugs that are hard to diagnose.
Common Strategies
| Strategy | How it Works | Best For |
|---|---|---|
| TTL (Time-To-Live) | Data expires after set time | Mostly-static data |
| Event-driven | Invalidate on write/update events | Real-time systems |
| Version-based | Append version to cache key | API responses |
Cache Eviction Policies
When the cache is full, which items to remove?
- LRU (Least Recently Used) — Most common, removes oldest-accessed item
- LFU (Least Frequently Used) — Removes least-accessed item
- FIFO (First In, First Out) — Removes oldest item regardless of access
- Random — Surprisingly effective in some workloads
Default Choice
When in doubt, use LRU. It works well for most real-world access patterns and is the default in Redis.
Redis vs Memcached
| Feature | Redis | Memcached |
|---|---|---|
| Data structures | Rich (strings, lists, sets, hashes) | Simple key-value |
| Persistence | Yes (RDB, AOF) | No |
| Replication | Yes | No |
| Memory efficiency | Lower | Higher |
| Best for | Feature-rich caching, pub/sub | Simple, high-throughput caching |
Cache Thundering Herd
When a popular cache key expires, many requests simultaneously hit the database:
Thundering Herd Problem
If 1000 requests arrive for an expired key, all 1000 will query the database simultaneously, potentially causing a cascade failure.
Solutions:
- Locking: Only one request fetches from DB, others wait
- Stale-while-revalidate: Serve stale data while refreshing in background
- Pre-warming: Refresh cache before TTL expires