---
name: caching
description: This skill should be used when the user asks about a "caching strategy", "cache invalidation", "what to cache", "read-through vs write-through vs write-back", "cache eviction" (LRU/LFU/TTL), "Redis vs Memcached", "stale reads", or hits "thundering herd", "cache stampede", "cache penetration", or "hot key" problems, or is choosing a Redis `maxmemory` eviction policy (allkeys-lru, volatile-lru, noeviction) vs TTL expiry for sessions or other cached state. Use it whenever a design is read-heavy or a datastore is overloaded by reads, even if the user doesn't say "cache".
title: caching
canonical_url: https://skilld.dev/gh/proyecto26/system-design-skills/caching
last_updated: 2026-10-29T10:01:24.000Z
---

> Index of every page on this site: [llms.txt](https://skilld.dev/llms.txt)

> **Skill from skilld.dev.** Follow the user's requested action.
> If the user asked to fork this Skill, follow the fork workflow below. Do not execute the Skill instructions while copying it.
> Otherwise, follow the instructions below for this session. You do not need to install anything.
>
> Supporting files, fetch one when the Skill refers to it: [references/deep-dive.md](https://skilld.dev/api/skills-raw/proyecto26/system-design-skills/caching/references/deep-dive.md), [references/providers/aws.md](https://skilld.dev/api/skills-raw/proyecto26/system-design-skills/caching/references/providers/aws.md), [references/providers/azure.md](https://skilld.dev/api/skills-raw/proyecto26/system-design-skills/caching/references/providers/azure.md), [references/providers/gcp.md](https://skilld.dev/api/skills-raw/proyecto26/system-design-skills/caching/references/providers/gcp.md), [references/providers/generic.md](https://skilld.dev/api/skills-raw/proyecto26/system-design-skills/caching/references/providers/generic.md).
>
> If the user asked to install this Skill, run `npx skilld install proyecto26/system-design-skills/caching`. Install writes the Skill files into the project, so every session loads them.
>
> ## Fork workflow
>
> A fork creates an editable local Skill with its original author and licence. The request authorizes copying and local installation.
> 1. Check `./skills/caching`, the project lockfile, and selected Agent targets together. If the local directory or installed Skill exists, stop. Never overwrite an existing directory or Agent target.
> 2. Read [source metadata](https://skilld.dev/api/v1/skills/proyecto26/system-design-skills/caching) once. Use sourceUrl, sourceCommit, skillPath, sourceGone, and license. If the source is gone or its path is missing, stop. If license is null, read licence files at the source commit.
> 3. Fetch only the source commit into a temporary Git repository. Do not clone full history. Derive repository_url from sourceUrl, including repository renames. If sourceCommit is absent, resolve the sourceUrl ref once. Set source_commit to that actual commit. Run these commands in one shell call:
>
> ```sh
> git init --quiet "$temporary_dir"
> git -C "$temporary_dir" fetch --quiet --depth=1 "$repository_url" "$source_commit"
> git -C "$temporary_dir" checkout --quiet --detach FETCH_HEAD
> ```
>
> Read applicable licence declarations and notices at that commit. If copying is not permitted, report the restriction and stop.
> 4. Inspect source entries together, then copy the directory containing skillPath into `./skills/caching`. Use the user's path if selected. Keep the original SKILL.md, relative links, scripts, binary assets, and executable modes. Exclude .git metadata. Reject symlinks and paths outside the Skill directory. After checking entries, use cp -a where available. A regular source directory needs no custom copy script. Do not save this page wrapper as SKILL.md.
> Preserve author credit, notices, and applicable licence files from repository or parent directories. Add PROVENANCE.md with the Skill page, source URL, actual commit, original path, and licence. Retain any existing PROVENANCE.md and record new provenance separately. Batch source inspection, copying, and provenance work where practical.
> 5. In the project root, run `skilld install ./skills/caching --mode copy --plain`. If skilld is unavailable, use `npx skilld install ./skills/caching --mode copy --plain`. This known command needs no help lookup. Install does not support --json. Use detected Agent targets, or add --agent for the targets the user selected. Install the local path, never the upstream selector. If installation fails, preserve the local copy and report the exact failure.
> 6. Confirm the local lockfile source and installed Agent copies once. Report the local path, actual commit, and Agent targets. After edits, reinstall the same local path. Upstream updates must not replace it. Do not publish or push unless the user asks.

# Caching

Put a copy of hot data closer to the reader so most requests skip the slow path.
Caching is the highest-leverage move for read-heavy systems — and the easiest to
get subtly wrong, because a cache adds a second source of truth that can serve
stale or wrong data, and can *amplify* an outage when it misbehaves.

## When to reach for this
Reads dominate (a high read:write ratio from `back-of-the-envelope`); the same
data is read repeatedly; the datastore is the read bottleneck; or recomputation
is expensive. A cache buys read latency and offloads the origin.

## When NOT to
Write-heavy or read-once data (low hit rate — pure overhead). Data that must be
exactly current with zero staleness (a cache is a stale copy by nature; when
strict freshness is required, go to the source or use `consistency-coordination`). Don't
add a cache before a number shows reads are the problem (YAGNI) — it's a new
failure mode and a second thing to operate.

## Clarify first
- **Read:write ratio and hit rate** — is the working set cacheable? (→ `back-of-the-envelope`, 80/20.)
- **Staleness tolerance** — seconds? minutes? must reads see their own writes?
- **Working-set size** — does the hot set fit in RAM across cache nodes?
- **Consistency on write** — can the cache briefly disagree with the store?
- **Eviction trigger** — what's the access pattern (recency? frequency? time-bound?).

## The options

**Where to cache** (often layered): client/browser → CDN edge (→ `content-delivery`)
→ application/in-process → distributed cache (Redis/Memcached) → database buffer
pool. This skill focuses on the application and distributed layers.

**Read strategy**
- **Cache-aside (lazy):** app checks cache, on miss reads the store and populates.
  Use when reads are unpredictable; the default for most systems.
- **Read-through:** the cache library fetches from the store on miss. Use to keep
  app code simple and caching policy centralized.

**Write strategy**
- **Write-through:** write cache and store synchronously. Use when reads right
  after writes must be fresh and slower writes are acceptable.
- **Write-back (write-behind):** write cache now, flush to store async. Use for
  write-heavy/bursty paths that tolerate a small loss window.
- **Write-around:** write only the store; let the cache fill on read. Use when
  written data is rarely re-read soon (avoids cache churn).

**Eviction policy**
- **LRU** for recency-skewed access (most common); **LFU** for stable popularity;
  **TTL** to bound staleness; **FIFO** rarely. Match the policy to the pattern.

## Trade-offs

| Option | What it solves | What it worsens | Change it when |
|---|---|---|---|
| Cache-aside | Simple, resilient (cache down ⇒ just slower) | First read per key is a miss; risk of stale after writes | Misses are too costly → read-through + warming |
| Read-through | Centralized, clean app code | Couples app to cache lib; cold-start misses | Custom per-key load logic is needed |
| Write-through | Fresh reads after write | Slower writes; writes cached data that may never be read | Writes dominate and aren't re-read → write-around/back |
| Write-back | Fast, absorbs write bursts | Data loss window on crash; complex | Durability of recent writes is required |
| Write-around | No churn from write-only data | Recently written keys miss on first read | That data IS read right after write → write-through |
| TTL eviction | Bounds staleness automatically | Mass expiry can stampede the origin | Add jitter / soft-TTL refresh |

## Behavior under stress
A cache that misbehaves doesn't just stop helping — it can take down the origin.

- **Thundering herd / stampede:** a hot key expires (or the cache restarts) and
  thousands of concurrent misses hit the store at once. *Mitigate:* per-key locks
  / request coalescing (single-flight), early/probabilistic refresh, TTL jitter.
- **Cache penetration:** requests for keys that don't exist bypass the cache every
  time (often malicious). *Mitigate:* cache the negative result (short TTL), or a
  Bloom filter in front.
- **Hot key:** one key (a celebrity, a viral item) exceeds a single node's
  throughput. *Mitigate:* replicate the key across nodes, add a local/L1 tier, or
  shard the value.
- **Eviction storm / cold cache:** after a flush or deploy, hit rate craters and
  the origin sees full load. *Mitigate:* warm critical keys; ramp traffic.
- **Stale-after-write:** the store changed but the cache didn't. *Mitigate:*
  invalidate on write, or write-through, or short TTL — pick per staleness budget.

**Monitor:** hit rate, p99 latency, eviction rate, key distribution (hot spots),
and origin QPS during cache restarts.

## How to apply
1. **Clarify the inputs** — confirm the read:write ratio, staleness budget, and
   hot-set size (see *Clarify first*). If no number yet shows reads are the
   bottleneck, stop — a cache is not needed yet (→ `back-of-the-envelope`).
2. **Pick the strategies from the trade-off table** — choose a read strategy
   (cache-aside is the default), a write strategy keyed to the staleness budget,
   and an eviction policy matched to the access pattern.
3. **Set the key knobs** — define the key naming scheme, TTL (with jitter), and the
   per-key-family invalidation event. Decide negative-caching and single-flight up
   front, not after the first incident.
4. **Stress-test the choice** — walk each failure in *Behavior under stress*
   (stampede, penetration, hot key, cold cache, stale-after-write) and confirm a
   mitigation is in place for the ones the traffic profile can trigger.
5. **Size it with numbers** — fit the hot set in RAM across nodes, sanity-check the
   target hit rate (90%+), and confirm the node count covers peak QPS (→ *Numbers
   that matter*).
6. **Pick a provider** — default to the generic recipe; only open a provider file
   if the user named a cloud (see *Choosing a provider*).
7. **Show the trade-off in the answer** — give a `Decision | Solves | Worsens |
   Change it when…` table with one row per choice you made here (e.g. the
   option, its store, its failure policy); the last column is a concrete trigger
   (a number or observable signal). Then name what breaks first.

## Dos and don'ts
**Do**
- Default to cache-aside; it stays correct (just slower) when the cache is down.
- Set a TTL on every entry and add jitter so keys don't expire in lockstep.
- Add single-flight / request coalescing for hot keys *before* launch.
- Invalidate on write (or write-through, or short TTL) to a stated staleness budget.
- Size the cache to the hot set and alert on hit rate, evictions, and origin QPS.

**Don't**
- Don't add a cache before a number proves reads are the bottleneck (YAGNI).
- Don't cache data that must be exactly current — go to the source instead.
- Don't let a mass-expiry or cold start dump full load on the origin (warm, ramp).
- Don't ignore non-existent-key floods — cache negatives or front with a Bloom filter.
- Don't reuse a key across schema versions — bump the version (`...:v2`) instead.

## Numbers that matter
A cache node serves ~**100k–1M QPS**, far above an RDBMS (~1k). Memory access is
~100 ns vs ms-scale disk — the reason caching wins. Size the cache to the hot set
(~20% of data ≈ 80% of reads). Target hit rates are usually 90%+; below that,
question whether the data is cacheable. → `back-of-the-envelope`.

## Interface sketch
A cache entry is a contract: a **key** (stable, namespaced, e.g.
`user:123:profile`), a **value** (serialized; watch size), and a **TTL**. Decide
the **invalidation event** per key family (on write? on TTL? on version bump?).
Versioned keys (`...:v2`) make invalidation a write of a new key instead of a
delete race.

## Choosing a provider
Default to the generic recipe above (Redis or Memcached, self-hosted or managed).
If the user names a cloud, read `references/providers/<provider>.md` for the
managed-service mapping, limits, and provider-specific trade-offs. If no file
exists for that provider, the generic recipe is the answer.

## Diagram
To visualize the cache-aside read path (app → cache → miss → origin → populate)
or the stampede/fallback flow, use the in-plugin `architecture-diagram` skill —
cache nodes use the cache color, the origin its store color, and the miss path a
dashed arrow.

## Related building blocks
- `content-delivery` — *pairs with* this as the edge layer above it; CDN/edge caching for static and media (that concept lives there).
- `data-storage` — *depends on* this as the origin a cache protects; its read replicas are an *alternative to* caching reads, and cache-node sharding mirrors its partitioning. (Consistent-hashing theory is *owned by* `consistency-coordination`.)
- `consistency-coordination` — *pairs with* this when staleness is unacceptable and read-your-writes or stronger guarantees are required.
- `back-of-the-envelope` — *feeds into* this skill: it supplies the read ratio and hot-set size that justify a cache.
- `resilience-failure` — *pairs with* this; rate limiting and circuit breakers (*owned* there) help contain the retry storms a misbehaving cache can trigger.
- `system-design` — *owned-concept lives in* the orchestrator: the reasoning loop, the trade-off method, and the ten failure modes.

## References
- **`references/deep-dive.md`** — invalidation patterns, single-flight/coalescing, consistent hashing for cache sharding, Redis vs Memcached internals, local (L1) + distributed (L2) tiers. Read when designing the cache layer in detail.
- **`references/providers/{generic,aws,azure,gcp}.md`** — service mappings, limits, and pitfalls per environment.
