Index of every page on this site: llms.txt
All skills
proyecto26 avatar

/blob-store

@228d57d
Run it yourself

This skill should be used when the user wants a "blob store" or "object storage", names "S3" or an S3-compatible store, needs to "store images / video / files", asks about "multipart upload" or "resumable upload", "signed / presigned URLs", "media storage", "unstructured data at scale", object "versioning", storage "tiering" (hot/cold/archive), or "erasure coding" vs replication for durability. Use it whenever a design must hold large unstructured objects (photos, video, backups, logs, ML datasets) and serve them cheaply and durably, even if the user just says "where do we put the files".

Use this Skill: https://skilld.dev/gh/proyecto26/system-design-skills/blob-store

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

referencesprovidersgeneric.md

≈558 tokens on demand. Your agent reads this file only when SKILL.md points to it.

Blob store — generic / self-hosted

The vendor-neutral default. If no cloud is named, this is the answer.

Service mapping (generic recipe → open source)

  • S3-compatible object store → MinIO (simplest, S3 API, erasure coding within a node set; the on-prem S3 default), Ceph + RGW (RADOS data plane, replication or EC pools, S3/Swift gateways; scales to exabytes, heavier to operate), SeaweedFS (optimized for many small files — packs them into volume files to keep the metadata index tiny).
  • Durability scheme → replication or erasure-coded pools, configured per bucket/pool.
  • Tiering → hot pool on SSD/NVMe, cold pool on HDD; lifecycle/transition policies move objects between pools.
  • CDN fronting → any CDN or reverse-proxy cache in front (→ content-delivery).

Decision-changing limits (verify against current docs)

  • The metadata/index node is the scaling ceiling and the SPOF — replicate and shard it. Billions of tiny objects stress it long before disk capacity does; pick SeaweedFS or pack small objects if the count is huge.
  • Erasure-coded reads gather k shards across nodes — sensitive to network and per-node count; keep EC for large/cold pools.
  • Multipart part-size minimums and max-parts-per-upload exist (S3-compatible APIs commonly use a ~5 MB minimum part and a max part count); confirm for the chosen implementation.

Trade-offs (self-host vs managed)

  • You own placement, rebalancing, failure-domain layout, capacity planning, and upgrades — real operational weight, but no egress markup and no lock-in.
  • S3-compatible APIs make migration to/from a cloud feasible, if you avoid provider-only features (lifecycle quirks, server-side encryption modes, event hooks).
  • Capacity is provisioned, not elastic — you must plan headroom; a cloud absorbs spikes.

Pitfalls

  • Treating the index node as an afterthought — it is the first thing to fall over.
  • Erasure-coding a hot small-object pool, then chasing tail latency.
  • Skipping lifecycle/abort-incomplete rules — orphaned parts and old versions grow forever.
  • Assuming "S3-compatible" means feature-complete; verify the specific API surface you rely on.

Source: SKILL.md on GitHub

No rule matched.

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No alerts24d3 checks · Risk SAFE
  • Gen Agent Trust Hub24d

    The blob-store skill is a comprehensive educational resource and architectural guide for designing object storage systems. It contains technical documentation, trade-off analyses, and provider-specific configurations for major cloud platforms. No malicious patterns, executable code, or security risks were identified.

  • Socket24d

    No alerts

  • Snyk24d

    Risk: LOW · No issues

Signed by skilld at 228d57d. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 3 weeks ago.

Activeupdated 4 weeks ago

README badge

README badge for proyecto26/system-design-skills/blob-store