Index of every page on this site: llms.txt
All skills
proyecto26 avatar

/sharded-counters

@228d57d
Run it yourself

This skill should be used when the user needs a "sharded counter", "distributed counter", to "count likes / views at scale", handles a "high-write counter" or "hot counter contention", asks about "approximate counting", "real-time counts", or "HyperLogLog". It gives the recipe for absorbing write-heavy counting without a single hot row. Use it whenever one row/key takes concurrent increments faster than it can serialize them, even if the user doesn't say "sharded counter".

Use this Skill: https://skilld.dev/gh/proyecto26/system-design-skills/sharded-counters

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

referencesprovidersgcp.md

≈622 tokens on demand. Your agent reads this file only when SKILL.md points to it.

Sharded counters — GCP

Only the contention/atomicity differences that change the recipe. Default to the generic recipe; this maps it to GCP services.

Service mapping

  • Firestore distributed counter — Google's documented pattern is exactly write-sharding: split a counter into N sub-documents, increment a random shard in a transaction, sum the shards on read. Firestore caps sustained writes to a single document (~1/sec), so sharding is mandatory above trivial rates.
  • Bigtable counter — atomic increment on a cell (ReadModifyWrite). Distribute load by designing the row key to avoid hot-spotting (salt/field-promote the key); a sequential or single hot row key serializes on one tablet.
  • Spanner — transactional UPDATE … SET c = c + 1; exact and strongly consistent, but a single hot row contends on locks — shard the row (N rows summed) for write-heavy counts.
  • Memorystore for Redis / Valkey — managed Redis for the fast path: INCR, HyperLogLog, key sharding. Use for microsecond tallies and HLL uniques.

When to pick which

Firestore distributed counter for app-level counts that must be durable (use the built-in shard pattern); Bigtable when increments are huge-scale and you control the row-key design; Spanner when the count must be transactionally exact (shard the row only if writes are hot); Memorystore Redis for ephemeral fast tallies.

Limits / things that bite (verify against current docs)

  • Firestore: ~1 write/sec sustained per document — a single counter doc hits this fast; the N-shard read sums all sub-docs.
  • Bigtable: a hot row key concentrates on one tablet; throughput is per-tablet, so row-key design (salting) is the real contention lever.
  • Spanner: a single hot row serializes on locks; cross-row reads to sum shards add latency but stay strongly consistent.
  • Memorystore Basic tier has no failover/SLA — a restart loses the cache; instances are regional and VPC-attached.

Pitfalls

  • Incrementing one Firestore document at high rate and hitting the per-document write limit — use the distributed-counter shard pattern.
  • A monotonic/sequential Bigtable row key creating a tablet hot spot — salt or field-promote the key.
  • Assuming Spanner's strong consistency removes contention — a hot row still serializes; shard it for write-heavy counts.
  • Lock-in: Firestore's shard pattern and Bigtable row-key design don't port directly to other clouds.

Source: SKILL.md on GitHub

No rule matched.

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No alerts24d3 checks · Risk SAFE
  • Gen Agent Trust Hub24d

    This skill consists purely of architectural documentation and design patterns for distributed counters. It contains no executable code, network operations, or data-handling scripts, and presents no security risks.

  • Socket24d

    No alerts

  • Snyk24d

    Risk: LOW · No issues

Signed by skilld at 228d57d. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 3 weeks ago.

Activeupdated 4 weeks ago

README badge

README badge for proyecto26/system-design-skills/sharded-counters