Index of every page on this site: llms.txt
All skills
proyecto26 avatar

/messaging-streaming

@228d57d
Run it yourself

This skill should be used when the user designs a "message queue", reaches for "Kafka", "RabbitMQ", "SQS", "Kinesis", "pub/sub", or "event-driven" architecture, asks about "async processing", "background jobs", "stream processing", or wrestles with "exactly-once vs at-least-once", "delivery guarantees", "message ordering", "duplicate handling / dedup", "dead letter queue", "backpressure", "saga orchestration", or "durable workflow". Use it whenever a slow or spiky operation should move off the request path, or two services must be decoupled, even if the user doesn't say "queue".

Use this Skill: https://skilld.dev/gh/proyecto26/system-design-skills/messaging-streaming

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

referencesprovidersgcp.md

≈609 tokens on demand. Your agent reads this file only when SKILL.md points to it.

Messaging & streaming — GCP

Service mapping → the generic options

  • Pub/Sub — managed pub/sub and work queue in one; at-least-once by default, push or pull subscriptions, built-in dead-lettering and retry policy, auto-scaling throughput. The default for both fan-out and async jobs on GCP.
  • Pub/Sub (ordering keys) — opt-in per-key ordering when you set an ordering key; off by default.
  • Pub/Sub Lite — cheaper, lower-cost-per-message variant where you provision capacity (zonal); use for high-volume, cost-sensitive streaming where you accept managing capacity.
  • Dataflow — managed stream/batch processing (Apache Beam) that consumes Pub/Sub for windowing, aggregation, exactly-once processing within the pipeline. Use when you need stream processing, not just transport.

Decision-changing limits (verify against current docs)

  • At-least-once by default → duplicates happen; consumers must be idempotent. Pub/Sub offers an exactly-once delivery mode within a subscription (narrower scope; verify constraints).
  • Ordering requires an ordering key and same-region publish; it caps per-key parallelism (the usual ordering cost).
  • Ack deadline is the visibility-timeout analog; extend it for long jobs or get redelivery. Message retention window bounds replay/seek-to-timestamp.
  • Message size limit (~10 MB) larger than SQS, but large payloads still favor a claim-check via Cloud Storage.

Provider-specific trade-offs

  • One service (Pub/Sub) covers queue + pub/sub, which simplifies the decision but means you tune subscriptions (push vs. pull, ack deadline, ordering) rather than pick a different product.
  • Pub/Sub is not a long-retention replayable log like Kafka by default — for Kafka semantics use self-managed Kafka on GKE or a partner offering; for stream processing reach for Dataflow.
  • For durable workflows GCP offers Workflows (YAML orchestration) and Cloud Tasks (HTTP task queue with scheduling) — compare with Temporal in temporal.md.

Pitfalls

  • Assuming exactly-once because GCP offers the mode — it's per-subscription and bounded; keep consumers idempotent.
  • Forgetting ordering is off until you set an ordering key.
  • Ack-deadline shorter than processing time → silent duplicate processing.
  • No dead-letter topic configured → failed messages redeliver until retention ends.

Source: SKILL.md on GitHub

No rule matched.

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No alerts24d3 checks · Risk SAFE
  • Gen Agent Trust Hub24d

    The skill provides architectural guidance and reference material for messaging and streaming systems. It contains no executable scripts, network operations, or security risks. All content is strictly educational and design-oriented.

  • Socket24d

    No alerts

  • Snyk24d

    Risk: LOW · No issues

Signed by skilld at 228d57d. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 3 weeks ago.

Activeupdated 4 weeks ago

README badge

README badge for proyecto26/system-design-skills/messaging-streaming