Index of every page on this site: llms.txt
All skills
proyecto26 avatar

/distributed-search

@228d57d
Run it yourself

This skill should be used when the user designs a "search system", needs "full-text search", asks about an "inverted index", "Elasticsearch / OpenSearch", "relevance ranking" (TF-IDF/BM25), "search autocomplete / typeahead", an "indexing pipeline", or "faceted search". It gives the crawl/index/search architecture, index sharding and replication, ranking, and near-real-time indexing. Use it whenever users must query text by relevance rather than fetch rows by key, even if they don't say "search engine".

Use this Skill: https://skilld.dev/gh/proyecto26/system-design-skills/distributed-search

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

referencesprovidersaws.md

≈481 tokens on demand. Your agent reads this file only when SKILL.md points to it.

Distributed search — AWS

Service mapping

  • Amazon OpenSearch Service — managed OpenSearch/Elasticsearch; provisioned domains (you size data/master nodes, shards, replicas) — the default managed full-text engine on AWS.
  • OpenSearch Serverless — auto-scaling capacity in "collections" (search / time-series / vector); removes node sizing at a price premium and with less low-level tuning control.
  • OpenSearch Ingestion — managed pipeline (collect/transform → index); pairs with the indexing pipeline. A durable buffer (e.g. Kinesis/MSK) still belongs in front (→ messaging-streaming).
  • CloudSearch — older managed search; prefer OpenSearch for new designs.

When to pick which

Provisioned domains when you want control over shard/replica sizing and steady cost; Serverless when load is spiky and you don't want to size nodes; Ingestion when you want managed transform-and-load rather than running your own indexer.

Limits / things that bite (verify against current docs)

  • Provisioned domains need master nodes for stability at scale; under-sizing master/data nodes causes cluster instability under load.
  • Storage, shard-per-node, and field-count limits per instance type — a hot or oversized shard still saturates one node and drags fan-out latency.
  • Serverless OCU (compute unit) billing can surprise at steady high load; it isn't automatically cheaper than a right-sized provisioned domain.
  • Version upgrades and blue/green domain changes can be disruptive — plan them.

Pitfalls

  • Assuming Serverless removes all tuning — you lose some shard/analyzer control.
  • Skipping dedicated master nodes on a production provisioned domain.
  • Treating the domain as a source of truth instead of a reindexable derived copy.
  • Lock-in: OpenSearch APIs are portable, but Serverless collections and Ingestion configs don't port directly to other clouds.

Source: SKILL.md on GitHub

No rule matched.

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No alerts24d3 checks · Risk SAFE
  • Gen Agent Trust Hub24d

    This skill is a technical architecture guide for designing distributed search systems and contains no security risks. It consists entirely of documentation and design patterns.

  • Socket24d

    No alerts

  • Snyk24d

    Risk: LOW · No issues

Signed by skilld at 228d57d. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 3 weeks ago.

Activeupdated 4 weeks ago

README badge

README badge for proyecto26/system-design-skills/distributed-search