/doca-compress

DOCA Compress workflows

Where to start: The verbs run configure → build → modify → run → test → debug. Skip ahead only when the user is already past a verb. The ## test verb is an iterative loop (cap check → permission cross-check → round-trip smoke → small-bulk → full-bulk → loop back if a task-type or buffer-sizing assumption changed), not a one-shot pass — see the eval-loop overlay in ## test below.

Read this file when the loader sent you here from SKILL.md. For the underlying capability surface, the compress-vs-decompress task split (including decompression-only as a standalone shape), per-task capability-query rules, error taxonomy, observability, and safety / path-selection policy, see CAPABILITIES.md. For the cross-library DOCA patterns layered under everything below (the universal lifecycle, the cross-library DOCA_ERROR_* taxonomy, the modify-a-shipped-sample workflow), see doca-programming-guide.

Each verb below describes the shape of the workflow, not a copy-paste recipe. The agent's job is to walk the user through the steps in order, verifying preconditions before recommending the next call.

configure

Goal: bring up a DOCA Compress context on a host or BlueField and confirm the device's accelerator supports the task type(s) the user actually intends to use, with buffer sizing that respects the per-task max.

Steps the agent should walk the user through:

1. Confirm the installed DOCA version. Use the procedure in doca-version TASKS.md ## configure. Quote the version observed (pkg-config --modversion doca-compress, then doca_caps --version); do not assume "latest". The four-way match rule lives in doca-version CAPABILITIES.md ## Version compatibility; if the observed sources disagree, route there before any Compress diagnosis.
2. Discover the device capability surface for Compress. Run doca_caps --list-devs (per doca-caps) to see which devices are visible, then run the per-doca_devinfo doca_compress_cap_* queries against the candidate device. Record the support and maximum-source-size result for every requested mode: doca_compress_cap_task_compress_deflate_is_supported(devinfo), doca_compress_cap_task_decompress_deflate_is_supported(devinfo), doca_compress_cap_task_decompress_lz4_stream_is_supported(devinfo), doca_compress_cap_task_decompress_lz4_block_is_supported(devinfo), and their matching doca_compress_cap_task_compress_deflate_get_max_buf_size(devinfo) / doca_compress_cap_task_decompress_deflate_get_max_buf_size(devinfo) / doca_compress_cap_task_decompress_lz4_stream_get_max_buf_size(devinfo) / doca_compress_cap_task_decompress_lz4_block_get_max_buf_size(devinfo) queries. For either LZ4 decode mode, also record its matching _get_max_buf_list_len result. Query only the modes the user intends to enable. The capability surface to compare against lives in CAPABILITIES.md ## Capabilities and modes.
3. Decide whether to offload at all. Per the size-threshold path-selection bullets in CAPABILITIES.md ## Safety policy, doca-compress is the right answer only when the input is bulk (rule of thumb: ≥ a few KiB), repeated, or already pinned in doca_mmap memory. For a tiny one-shot input (e.g. a 64-byte buffer), recommend CPU zlib / zstd — do not invent a doca-compress use case the user did not ask for. For outbound encoding in a non-DEFLATE algorithm, doca-compress is not the answer. Inbound decode supports DEFLATE plus the independently gated LZ4 stream and LZ4 block modes.
4. Pick the task type(s) to enable. DEFLATE compress (doca_compress_task_compress_deflate) for outbound encoding; DEFLATE decompress (doca_compress_task_decompress_deflate), LZ4 stream decompress (doca_compress_task_decompress_lz4_stream), or LZ4 block decompress (doca_compress_task_decompress_lz4_block) for inbound decoding; both DEFLATE directions for a round-trip flow. A decompress-only consumer is a fully valid shape — do not auto-enable the compress task on a user who only described an inbound decoding need. The trade-off table lives in CAPABILITIES.md ## Capabilities and modes.
5. Configure the Compress instance. Mandatory before doca_ctx_start(): enable at least one task type (the matching task configuration for DEFLATE encode, DEFLATE decode, LZ4 stream decode, and/or LZ4 block decode, each with its success and error completion callbacks plus its max-num-tasks budget); set source mmap permissions (doca_mmap_set_permissions to include DOCA_ACCESS_FLAG_LOCAL_READ_ONLY); set destination mmap permissions (DOCA_ACCESS_FLAG_LOCAL_READ_WRITE); size the source buffer to ≤ the matching _get_max_buf_size and the destination buffer to the worst-case output size (a compress destination may need to be slightly larger than the source for poorly-compressible inputs; a decompress destination may need to be substantially larger than the compressed source). Per the matrix in CAPABILITIES.md ## Safety policy.
6. Sanity check before any task submission. Confirm with the user: which task type(s), source-buffer size, destination-buffer size, and whether the intended source size is within the per-task _get_max_buf_size (and, for LZ4, the matching maximum buffer-list length). Run a round-trip smoke for DEFLATE encode plus decode, or for a decode-only consumer use a known matching CPU-produced fixture in the exact requested format (DEFLATE, LZ4 stream, or LZ4 block) together with its retained original. Verify the decoded result matches that original byte-for-byte before any user data flows. If any step fails with a DOCA_ERROR_*, route through the error taxonomy in CAPABILITIES.md ## Error taxonomy before retrying.

For the canonical DOCA universal lifecycle that underlies steps 4-6, see doca-programming-guide TASKS.md ## configure. This skill adds the Compress overlay; do not re-explain the lifecycle here.

build

Goal: produce a binary that links DOCA Compress against the user's installed DOCA, using the canonical cross-library build pattern.

The build pattern for any DOCA C/C++ consumer is identical across libraries — pkg-config for include + link flags, meson or CMake as the build system — and is fully documented in doca-programming-guide TASKS.md ## build. This skill carries only the Compress-specific overlay:

For non-C consumers (Rust, Go, Python), the link surface is the same *.so files; the FFI wrapper layer is the language-specific binding and is out of scope for this skill — but the four slots above are still the load-bearing inputs the wrapper needs.

modify

Goal: take a shipped DOCA Compress sample (or the File Compression reference application) as the verified starting point and apply a minimum-diff modification to express the user's intent.

The universal modify-a-shipped-sample workflow lives in doca-programming-guide TASKS.md ## modify. Use it as-is. The Compress-specific overlay is the modify-from-sample schema fill — the slots the agent must elicit from the user before recommending any code-level edit:

The agent emits an intent description + the filled slots; the actual unified diff against the sample source is produced by the modify-from-sample renderer (deferred to a future round, per doca-programming-guide TASKS.md ## modify). Until the renderer ships, the agent must walk the user through the diff line-by-line against the sample source they read on disk, and have the user paste back the result for validation.

run

Goal: actually execute the built binary against the user's installed DOCA on a host or BlueField, with a real input.

Steps the agent should walk the user through:

1. Confirm the device is reachable. Compress runs on a single side (no peer); the only env-side requirement is that the doca_dev the binary opens corresponds to a device whose accelerator the user expects. Mismatched doca_dev selection (opening a NIC without the requested Compress task on its accelerator) returns DOCA_ERROR_NOT_SUPPORTED at task submit, not at open. Re-quote the output of doca_caps --list-devs (doca-caps) and confirm the device the binary opens is the same one the cap-query ran against.
2. Run the round-trip (or decompress-known-fixture) smoke first. A binary that compresses one short input and decompresses the output, comparing to the original — or, for a decompress-only consumer, decompresses one small DEFLATE fixture produced by zlib and compares to the original — is the cheapest correctness signal. Do not bulk-encode or bulk-decode before this passes.
3. Capture the structured log. Set DOCA_LOG_LEVEL=trace for the first run (see doca-debug CAPABILITIES.md ## Observability). This is the cheapest way to make the lifecycle and task-submit transitions visible on first failure.
4. Capture the completion events on the PE. A run that produces no completion events but doesn't error is almost always a missed doca_pe_progress() call. Confirm the progress engine is being driven on the main thread.

test

Goal: prove the configured Compress context can actually produce correct encoded / decoded output at the user's intended throughput, on the user's hardware, and that the task selection, buffer sizing, and permission set were right.

Performance harness routing. For throughput / latency measurement on the configured Compress context (or for cross-library comparison against the other DOCA crypto primitives), route the user to doca-bench TASKS.md ## test — doca-bench is the cross-library performance harness with documented warm-up / steady-state / outlier semantics, and it explicitly supports Compress. The iteration loop below stays the correctness harness; doca-bench is the performance harness.

This is a loop, not a one-shot pass. Each iteration narrows either the task type, the buffer sizing, the permission set, or the path-selection assumption. The loop terminates when either (a) the user's intended workload produces correct output end-to-end with acceptable throughput, or (b) the agent has narrowed the failure cause to a layer outside DOCA Compress itself (driver / firmware / device) and escalated to the matching skill.

Iteration identity: compare the tuple of requested task / algorithm mode, support and max-size query results, mmap permission map, submitted/completed counts, and byte-comparison outcome. Two consecutive iterations with the same tuple are unchanged and trigger escalation; unchanged is never a green result.

Iteration shape:

1. Capability re-check. Re-run the exact support and _get_max_buf_size queries selected in ## configure for DEFLATE encode, DEFLATE decode, LZ4 stream decode, and/or LZ4 block decode against the active doca_devinfo; include the matching _get_max_buf_list_len result for LZ4. If any required task type returns false / unexpected → that's the answer; the user's device or DOCA version does not support the requested config. Update the intent or update the install.
2. Permission cross-check. Compare the configured source + destination mmap permissions against the matrix in CAPABILITIES.md ## Safety policy. Mismatches surface as DOCA_ERROR_NOT_PERMITTED on the first task submission, not at configure time.
3. Round-trip smoke (or decode-known-fixture smoke). Compress one short fixed input and decompress the output, comparing to the original byte-for-byte; OR, for a decode-only consumer, decode one known matching CPU-produced DEFLATE, LZ4 stream, or LZ4 block fixture and compare to its retained original byte-for-byte. If the comparison fails, the configuration is wrong — do not proceed to bulk input.
4. Completion drain. Confirm completion events arrive on the PE for every submitted task. Submitted but no completion is the most expensive class of bug to discover late; confirm it on the round-trip smoke before bulk submissions.
5. Small-bulk test. Submit a small series of tasks at sizes approaching the per-task _get_max_buf_size but staying under it; verify the outputs decompress back to the inputs (round-trip) or match a CPU reference. Throughput numbers come from this step; correctness comes from step 3.
6. Full-bulk run. Once the small-bulk passes, scale up to the user's intended size and submission rate. Watch for DOCA_ERROR_AGAIN (drain the PE before retrying) and for DOCA_ERROR_INVALID_VALUE (a per-task source-size or destination-size boundary was crossed — re-narrow to the cap-query axis or to the worst-case-output-sizing axis).
7. Negative test. Once the positive path works, intentionally request a task type the device should NOT support (per step 1), or submit a source buffer larger than _get_max_buf_size for the configured task, and confirm the failure is the expected DOCA_ERROR_NOT_SUPPORTED / DOCA_ERROR_INVALID_VALUE. This validates the agent's capability-discovery is itself correct.

Eval-loop overlay — why this is a loop, not a one-shot pass:

Loop termination: stop iterating once two consecutive iterations have the same task / algorithm mode, cap results, permission map, submitted/completed counts, and byte-comparison outcome. That unchanged identity means the cause is below DOCA Compress and is never success. Escalate to doca-debug TASKS.md ## debug with the captured cap-query snapshot + round-trip diff as evidence.

debug

Goal: when a DOCA Compress call returns a DOCA_ERROR_* (or the program produces no completion event), narrow the cause to a specific layer and act on it.

The cross-library debug ladder lives in doca-debug TASKS.md ## debug. Walk through it in order — install → version → build → link → runtime → program → driver — before recommending Compress-specific fixes. This skill's overlay names the Compress-specific manifestation at layers 5 (runtime) and 6 (program):

Layer 5 (runtime) — Compress overlay.

• Walk the lifecycle: was the context started? Were the task(s) enabled before start (doca_compress_task_*_set_conf before doca_ctx_start())? Submitting before the requested task is enabled returns DOCA_ERROR_BAD_STATE, not a clear symptom.
• Confirm the PE is being progressed. No completion events is almost always a missing doca_pe_progress() in the user's main loop.
• Confirm both the source mmap and the destination mmap are still alive at submit time. Destroying either before doca_ctx_destroy is a use-after-free that surfaces as DOCA_ERROR_BAD_STATE from subsequent calls.

Layer 6 (program) — Compress overlay.

• Buffer-sizing matrix: the most common Compress program-layer bugs are (a) a source buffer larger than the matching _get_max_buf_size for the enabled task — surfaces as DOCA_ERROR_INVALID_VALUE at submit; fragment at the application layer — and (b) a destination buffer too small for the worst-case output. For compress, worst-case output can be slightly larger than the input on incompressible data; for decompress, the decompressed output can be substantially larger than the compressed input.
• Task-type drift: an _alloc_init call that asks for a task type the cap query returns false for returns DOCA_ERROR_NOT_SUPPORTED. Re-run the matching doca_compress_cap_task_*_is_supported against the active doca_devinfo; do not assume from prior installs.
• Source / destination swap: a configuration that wires the encoded buffer as the source of a compress task (or the raw buffer as the source of a decompress task) will produce a completion that decodes back to garbage on round-trip. Walk the buffer wiring against the task direction in CAPABILITIES.md ## Capabilities and modes before any other diagnosis.
• Round-trip mismatch: if the decompressed output is the wrong length, the task type is likely wrong (asked for compress, configured decompress, or vice versa). If the length is right but the bytes don't match the original, the source / destination wiring is wrong or the fixture used for the smoke is itself corrupt.

Once the layer is identified, route to the matching debug verb on the matching skill: install / build / link / driver to doca-setup ## debug; version to doca-version ## debug; cross-cutting runtime to doca-debug ## debug; program-layer Core-context patterns to doca-programming-guide TASKS.md ## debug.

5-phase universal debug-loop instantiation (Compress). Layer identification above is phase 1 of the universal debug-loop contract. The agent MUST walk the remaining four phases on every Compress debug answer before declaring done:

1. Layer identification — above (capability / lifecycle / data-path).
2. Triple capture (READ-ONLY). Capture (a) capability map: doca_compress_cap_task_compress_deflate_is_supported(devinfo)
   • doca_compress_cap_* max buffer / algorithm support on the actual devinfo, (b) submitted-task vs completed-task counters from doca_pe_progress callback log, (c) DOCA log at DOCA_LOG_LEVEL=DEBUG for the offending task with the request / response buffer addresses. The triple is the rollback target.
3. Single-variable mutation SMALLER than the original change. Examples: shrink the input buffer to a single compressible payload (not the production batch); switch doca_mmap to a fresh pinned region (not the production pool); reduce parallelism to one outstanding task (not the pipeline depth). Larger mutations void the experiment.
4. Re-capture and compare. Re-run the triple; the request/response counter diff IS the evidence.
5. Exit with named green signal OR escalate. Green = one round-trip (compress → decompress) returns bytes-equal at the source. If two consecutive iterations don't change anything, escalate via the layer route table above with the captured triple.

rollback

Compress contexts are stateful (started context + registered mmap regions + in-flight tasks on the progress engine) and the agent's failure mode is to leave in-flight tasks dangling on a context that is being torn down, returning DOCA_ERROR_BAD_STATE on the next program run. The universal verification contract step 1 (preconditions) requires "the rollback path is documented" on every change-recommending answer; this is the Compress instantiation.

Snapshot before mutate. Before any change-recommending Compress answer, capture (a) the started-context registration map (mmap region IDs + task-type conf flags from ## configure step 3), (b) the outstanding-task count from doca_pe_progress, and (c) the input/output buffer ownership list. The triple IS the rollback target.

1. Drain outstanding tasks FIRST. Walk doca_pe_progress until the outstanding-task counter is zero. Do NOT submit new tasks after rollback intent is declared. If the drain stalls (counter not decrementing within the bounded debug-loop window), fire the deploy-loop bridge on the stalled-drain symptom before continuing the rollback.
2. doca_ctx_stop on the Compress context. Returns DOCA_ERROR_BAD_STATE if step 1 was skipped — that is diagnostic, not a retry trigger; re-walk step 1 with a higher-resolution drain log.
3. Destroy the stopped Compress context. doca_compress_destroy before destroying either mmap. The underlying doca_dev remains valid and must not be torn down by this step.
4. Unregister mmap regions in reverse-register order. doca_mmap_destroy on every region created with doca_mmap_create_*; the underlying host buffers may be freed after this step.
5. Re-verify with the shipped round-trip smoke. Re-run the round-trip from ## test step 1 against a fresh context to confirm the device + driver path is intact post-rollback. Green is exactly a byte-equal result. If the first fresh-context smoke is not byte-equal, permit one bounded debug-loop single-variable mutation and then one fresh-context recheck. If that recheck is still not byte-equal, surface the unresolved residual gap and do not retry.
6. Document the rollback verb in the verification contract preconditions block. The step 1 line for a Compress add reads: "the rollback path is the five-step reversal in ## rollback; the agent has captured the started-context registration map and outstanding-task count." Without that line, the contract is incomplete and the agent is NOT eligible to declare done.

The rollback is bounded to the initial fresh-context recheck plus one debug mutation and one final fresh-context recheck. Only a byte-equal comparison is green; a second non-green result is the unresolved residual gap.

Deferred task verbs

The following verbs are out of scope for this skill but are commonly asked in the same conversations. Route them as follows so the agent does not invent guidance:

• install. Installing DOCA, choosing packages, post-install verification, pkg-config wiring — defer to doca-setup and to the install-tree layout in doca-public-knowledge-map ## Layout of an installed DOCA package. This skill assumes DOCA is already installed.
• deploy. Deploying Compress-using applications at scale (compress-before-write storage workers across many hosts, decompress-on-read network receivers, Kubernetes operator workflows) — out of scope for Phase 1 and reserved for a future platform skill. For single-host first-run testing, the right verb in this skill is ## run; do not invent a "deploy" workflow.
• rollback. Coordinated rollback of Compress-using applications across many hosts — out of scope. For a single in-session Compress configuration rollback, the right verb in this skill is destroying the context (doca_ctx_stop → doca_ctx_destroy) and re-running ## configure with corrected parameters.
• kernel-level driver install / firmware burn. Compress depends on the underlying ConnectX firmware and BlueField BFB; if the debug ladder lands on a driver-layer issue (DOCA_ERROR_DRIVER from a Compress submit, repeated accelerator errors in dmesg), the fix is via mlxconfig / mlxfwreset / re-imaging the BFB, all of which belong to doca-setup ## debug layer

Command appendix

Every command below is cross-cutting on DOCA Compress — it answers a recurring class of question that comes up in the verbs above. The agent should treat the class as load-bearing; the worked example is a single instance. Run-as user is the unprivileged user unless noted. Rows that need elevated privileges call that out explicitly.

Infra-aware preamble (every row below). Per the bundle's detect → prefer → fall back → report contract documented in doca-structured-tools-contract ## The agent behavior contract, the agent should:

1. Probe for the matching structured helper FIRST (doca-env --json for version + devices + libraries + drivers + hugepages in one shot; doca-capability-snapshot for per-device capability flags; version-matrix.json for "available since" lookups).
2. If the probe succeeds, the structured tool's output is the authoritative answer and the agent SHOULD NOT also run the manual command in the row below. Report "using structured <tool>".
3. If the probe fails, fall back to the manual command in the row. Report "falling back to manual chain".
4. The schemas the structured tools emit are defined in doca-structured-tools-contract ## Schemas; the version-handling semantics (four-way match, NGC, headers-win) are owned by doca-version.

For commands shared across libraries (pkg-config --modversion, doca_caps, cat /opt/mellanox/doca/applications/VERSION, DOCA_LOG_LEVEL) the cross-library overlay is in doca-debug TASKS.md ## Command appendix; this table adds the Compress-specific rows on top.