Index of every page on this site: llms.txt
All skills
Run it yourself

This skill should be used when the user asks about a "caching strategy", "cache invalidation", "what to cache", "read-through vs write-through vs write-back", "cache eviction" (LRU/LFU/TTL), "Redis vs Memcached", "stale reads", or hits "thundering herd", "cache stampede", "cache penetration", or "hot key" problems, or is choosing a Redis `maxmemory` eviction policy (allkeys-lru, volatile-lru, noeviction) vs TTL expiry for sessions or other cached state. Use it whenever a design is read-heavy or a datastore is overloaded by reads, even if the user doesn't say "cache".

Use this Skill: https://skilld.dev/gh/proyecto26/system-design-skills/caching

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

SKILL.md

≈146 tokens for metadata: the name and description. ≈2.4k when used: this file. ≈2.5k more on demand in 5 files.

Description uses 7.3% of example budget

Before choosing a Skill, your Agent reads its name and description. All available Skills share that space.

  • A shorter description leaves more room for other Skills. This entry exceeds our 1% size suggestion.

In our Claude Code example, all Skill names and descriptions share 8,000 characters. This Skill uses ≈583 characters, or 7.3%.

The 1% threshold is a size suggestion. Longer descriptions can still fit.

Your model, settings, and other Skills decide how much text your Agent can read.

Example settings and source

The example uses a 200k-token context and default Claude Code settings. The count includes the name, description, separators, and when_to_use when present. Codex also counts local file paths.

Skit's source and limits: Codex 0.160.1, Claude Code 2.1.292.

Caching

Put a copy of hot data closer to the reader so most requests skip the slow path. Caching is the highest-leverage move for read-heavy systems — and the easiest to get subtly wrong, because a cache adds a second source of truth that can serve stale or wrong data, and can amplify an outage when it misbehaves.

When to reach for this

Reads dominate (a high read:write ratio from back-of-the-envelope); the same data is read repeatedly; the datastore is the read bottleneck; or recomputation is expensive. A cache buys read latency and offloads the origin.

When NOT to

Write-heavy or read-once data (low hit rate — pure overhead). Data that must be exactly current with zero staleness (a cache is a stale copy by nature; when strict freshness is required, go to the source or use consistency-coordination). Don't add a cache before a number shows reads are the problem (YAGNI) — it's a new failure mode and a second thing to operate.

Clarify first

  • Read:write ratio and hit rate — is the working set cacheable? (→ back-of-the-envelope, 80/20.)
  • Staleness tolerance — seconds? minutes? must reads see their own writes?
  • Working-set size — does the hot set fit in RAM across cache nodes?
  • Consistency on write — can the cache briefly disagree with the store?
  • Eviction trigger — what's the access pattern (recency? frequency? time-bound?).

The options

Where to cache (often layered): client/browser → CDN edge (→ content-delivery) → application/in-process → distributed cache (Redis/Memcached) → database buffer pool. This skill focuses on the application and distributed layers.

Read strategy

  • Cache-aside (lazy): app checks cache, on miss reads the store and populates. Use when reads are unpredictable; the default for most systems.
  • Read-through: the cache library fetches from the store on miss. Use to keep app code simple and caching policy centralized.

Write strategy

  • Write-through: write cache and store synchronously. Use when reads right after writes must be fresh and slower writes are acceptable.
  • Write-back (write-behind): write cache now, flush to store async. Use for write-heavy/bursty paths that tolerate a small loss window.
  • Write-around: write only the store; let the cache fill on read. Use when written data is rarely re-read soon (avoids cache churn).

Eviction policy

  • LRU for recency-skewed access (most common); LFU for stable popularity; TTL to bound staleness; FIFO rarely. Match the policy to the pattern.

Trade-offs

Option What it solves What it worsens Change it when
Cache-aside Simple, resilient (cache down ⇒ just slower) First read per key is a miss; risk of stale after writes Misses are too costly → read-through + warming
Read-through Centralized, clean app code Couples app to cache lib; cold-start misses Custom per-key load logic is needed
Write-through Fresh reads after write Slower writes; writes cached data that may never be read Writes dominate and aren't re-read → write-around/back
Write-back Fast, absorbs write bursts Data loss window on crash; complex Durability of recent writes is required
Write-around No churn from write-only data Recently written keys miss on first read That data IS read right after write → write-through
TTL eviction Bounds staleness automatically Mass expiry can stampede the origin Add jitter / soft-TTL refresh

Behavior under stress

A cache that misbehaves doesn't just stop helping — it can take down the origin.

  • Thundering herd / stampede: a hot key expires (or the cache restarts) and thousands of concurrent misses hit the store at once. Mitigate: per-key locks / request coalescing (single-flight), early/probabilistic refresh, TTL jitter.
  • Cache penetration: requests for keys that don't exist bypass the cache every time (often malicious). Mitigate: cache the negative result (short TTL), or a Bloom filter in front.
  • Hot key: one key (a celebrity, a viral item) exceeds a single node's throughput. Mitigate: replicate the key across nodes, add a local/L1 tier, or shard the value.
  • Eviction storm / cold cache: after a flush or deploy, hit rate craters and the origin sees full load. Mitigate: warm critical keys; ramp traffic.
  • Stale-after-write: the store changed but the cache didn't. Mitigate: invalidate on write, or write-through, or short TTL — pick per staleness budget.

Monitor: hit rate, p99 latency, eviction rate, key distribution (hot spots), and origin QPS during cache restarts.

How to apply

  1. Clarify the inputs — confirm the read:write ratio, staleness budget, and hot-set size (see Clarify first). If no number yet shows reads are the bottleneck, stop — a cache is not needed yet (→ back-of-the-envelope).
  2. Pick the strategies from the trade-off table — choose a read strategy (cache-aside is the default), a write strategy keyed to the staleness budget, and an eviction policy matched to the access pattern.
  3. Set the key knobs — define the key naming scheme, TTL (with jitter), and the per-key-family invalidation event. Decide negative-caching and single-flight up front, not after the first incident.
  4. Stress-test the choice — walk each failure in Behavior under stress (stampede, penetration, hot key, cold cache, stale-after-write) and confirm a mitigation is in place for the ones the traffic profile can trigger.
  5. Size it with numbers — fit the hot set in RAM across nodes, sanity-check the target hit rate (90%+), and confirm the node count covers peak QPS (→ Numbers that matter).
  6. Pick a provider — default to the generic recipe; only open a provider file if the user named a cloud (see Choosing a provider).
  7. Show the trade-off in the answer — give a Decision | Solves | Worsens | Change it when… table with one row per choice you made here (e.g. the option, its store, its failure policy); the last column is a concrete trigger (a number or observable signal). Then name what breaks first.

Dos and don'ts

Do

  • Default to cache-aside; it stays correct (just slower) when the cache is down.
  • Set a TTL on every entry and add jitter so keys don't expire in lockstep.
  • Add single-flight / request coalescing for hot keys before launch.
  • Invalidate on write (or write-through, or short TTL) to a stated staleness budget.
  • Size the cache to the hot set and alert on hit rate, evictions, and origin QPS.

Don't

  • Don't add a cache before a number proves reads are the bottleneck (YAGNI).
  • Don't cache data that must be exactly current — go to the source instead.
  • Don't let a mass-expiry or cold start dump full load on the origin (warm, ramp).
  • Don't ignore non-existent-key floods — cache negatives or front with a Bloom filter.
  • Don't reuse a key across schema versions — bump the version (...:v2) instead.

Numbers that matter

A cache node serves ~100k–1M QPS, far above an RDBMS (~1k). Memory access is ~100 ns vs ms-scale disk — the reason caching wins. Size the cache to the hot set (~20% of data ≈ 80% of reads). Target hit rates are usually 90%+; below that, question whether the data is cacheable. → back-of-the-envelope.

Interface sketch

A cache entry is a contract: a key (stable, namespaced, e.g. user:123:profile), a value (serialized; watch size), and a TTL. Decide the invalidation event per key family (on write? on TTL? on version bump?). Versioned keys (...:v2) make invalidation a write of a new key instead of a delete race.

Choosing a provider

Default to the generic recipe above (Redis or Memcached, self-hosted or managed). If the user names a cloud, read references/providers/<provider>.md for the managed-service mapping, limits, and provider-specific trade-offs. If no file exists for that provider, the generic recipe is the answer.

Diagram

To visualize the cache-aside read path (app → cache → miss → origin → populate) or the stampede/fallback flow, use the in-plugin architecture-diagram skill — cache nodes use the cache color, the origin its store color, and the miss path a dashed arrow.

Related building blocks

  • content-delivery — pairs with this as the edge layer above it; CDN/edge caching for static and media (that concept lives there).
  • data-storage — depends on this as the origin a cache protects; its read replicas are an alternative to caching reads, and cache-node sharding mirrors its partitioning. (Consistent-hashing theory is owned by consistency-coordination.)
  • consistency-coordination — pairs with this when staleness is unacceptable and read-your-writes or stronger guarantees are required.
  • back-of-the-envelope — feeds into this skill: it supplies the read ratio and hot-set size that justify a cache.
  • resilience-failure — pairs with this; rate limiting and circuit breakers (owned there) help contain the retry storms a misbehaving cache can trigger.
  • system-design — owned-concept lives in the orchestrator: the reasoning loop, the trade-off method, and the ten failure modes.

References

  • references/deep-dive.md — invalidation patterns, single-flight/coalescing, consistent hashing for cache sharding, Redis vs Memcached internals, local (L1) + distributed (L2) tiers. Read when designing the cache layer in detail.
  • references/providers/{generic,aws,azure,gcp}.md — service mappings, limits, and pitfalls per environment.

Source: SKILL.md on GitHub

No rule matched.

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No alerts24d3 checks · Risk SAFE
  • Gen Agent Trust Hub24d

    This skill is a pure documentation and architectural guidance resource for system design, specifically focusing on caching strategies and failure modes across various cloud providers. It contains no executable code, network operations, or security risks.

  • Socket24d

    No alerts

  • Snyk24d

    Risk: LOW · No issues

Signed by skilld at 228d57d. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 3 weeks ago.

Activeupdated 4 weeks ago

README badge

README badge for proyecto26/system-design-skills/caching