/doca-setup

DOCA setup — capabilities, version compatibility, errors, observability, safety

Where to start: Pick the H2 anchor that matches your question (modes / version / errors / observability / safety) and read that section end-to-end before issuing a command. Tables in each section are the load-bearing content; the prose around them is interpretation.

Read this file when the loader sent you here from SKILL.md. For the env workflows that use the surface described here, see TASKS.md. For the program-side counterparts (build flavor selection rationale, the universal lifecycle, the cross-library DOCA_ERROR_* taxonomy, the program-side safety policy), see doca-programming-guide. For where to find official documentation, the on-disk layout of an installed DOCA package, or the official Installation Guide, route through doca-public-knowledge-map.

This file describes the install / build / runtime environment that any DOCA program assumes has been verified before its own workflows begin. It is deliberately library-agnostic and program-agnostic; everything here is about the host the program runs on, not about the program itself.

Lint note: this file references /opt/mellanox/doca and docs.nvidia.com paths several times. For most library skills the lint flags this and asks for cross-links to doca-public-knowledge-map instead, since URL/path duplication is normally drift waiting to happen. For doca-setup specifically the references are intrinsic — the skill's whole job is to operate on the install tree and verify it. The repeated paths are intentional; do not refactor them out.

Pattern overview

Every env-class concern this skill teaches resolves into one of FIVE patterns. Reach for the pattern first, then drill into the matching H2 anchor; the patterns are CLASSES, not use cases.

Two principles cut across all five patterns:

• Env layers run in order: install → capability → runtime. A hugepages question with an unreachable install is the install question. Skipping a layer is the single most common failure mode the env-debug ladder catches.
• Env-class only. The instant the answer becomes "rewrite the program", hand off to doca-programming-guide. The patterns above never tell the user to change their code.

Capabilities and modes

DOCA install is layered into three orthogonal env axes. Pick the right combination before writing or building any code. Program-side selection of which mode the program initializes (host vs DPU vs switch) and which build flavor it links against is in doca-programming-guide CAPABILITIES.md ## Capabilities and modes; this section covers what the env supports.

Install profiles. Determined by the package profile selected at install time (see the Installation Guide via doca-public-knowledge-map).

The agent's rule: if a sample build fails with a missing *.pc file (doca-flow.pc, doca-rdma.pc, …), the most likely cause is the wrong install profile, not a code bug. Surface this hypothesis before any code-level diagnosis.

Build flavor — env side. Two *.so trees ship with every DOCA install. The choice of which one a program links against is a programming decision (see doca-programming-guide CAPABILITIES.md ## Capabilities and modes); the env side is where each tree lives on disk and how to make the runtime see it.

Runtime modes — env-side enablement. Most DOCA libraries (Flow in particular) run in one of three host configurations. The env determines which modes are available on this host; the program selects one at init time.

The agent must clarify which mode the user expects before recommending any port-id or representor naming. Program-side mode-selection guidance is in doca-programming-guide.

Version compatibility

For the canonical DOCA version-detection chain (pkg-config --modversion doca-common → cat applications/VERSION → doca_caps --version → BFB version on BlueField), the four-way match rule, NGC container semantics, the headers-win-over-docs rule, and the routing to the DOCA Compatibility Policy, see doca-version. The body lives there; this skill does not duplicate it.

BFB-version probe ban (same as in doca-version). When the user is configuring a BlueField host and the agent reaches for the BFB-image DOCA version (the (d) leg of the four-source chain), the only legitimate probes are bfver and cat /etc/mlnx-release on the BlueField Arm console. Do NOT substitute mlxprivhost (configures privileged-host mode, not version) or bfb-info (not a real NVIDIA-documented tool) — both are common hallucinations even when the visible binary is present on /usr/bin/ of the x86 host. See doca-version CAPABILITIES.md ## Capabilities and modes.

The env-side overlay is responsible for making the detection chain work on this host before any version question can be answered:

• PKG_CONFIG_PATH must include /opt/mellanox/doca/infrastructure/lib/pkgconfig so that pkg-config --modversion doca-<library> resolves at all. The env-setup procedure in TASKS.md ## configure verifies this; partial-install diagnosis lives in doca-version TASKS.md ## debug layer 2.
• The on-disk paths the detection chain reads (/opt/mellanox/doca/applications/VERSION, $(pkg-config --variable=includedir doca-common)/doca_version.h, the *.pc directory) are env-side artifacts — see ## Capabilities and modes for the install-tree layout that places them where the chain expects.
• On the no-install path (NGC container, per TASKS.md ## no-install Path 0), the env-side overlay is that the four-way match is of the container tag the user pulled; the container's headers, *.so, samples, and doca_caps are guaranteed consistent by construction. The agent must still report which path was used so the user knows which install they verified.

Error taxonomy

Env-class errors that the agent should recognize and disambiguate before falling back to a program-internal diagnosis. The cross-library, program-side DOCA_ERROR_* taxonomy lives in doca-programming-guide CAPABILITIES.md ## Error taxonomy; the library-specific overlays live in the matching library skills.

The taxonomy above is env-class only. Program-internal DOCA_ERROR_* codes (DOCA_ERROR_BAD_STATE, DOCA_ERROR_NOT_SUPPORTED, etc.) live in doca-programming-guide CAPABILITIES.md ## Error taxonomy; the library-specific overlay (e.g. Flow's mapping from API call to which DOCA_ERROR_* it returns) lives in the matching library skill — for Flow, see doca-flow CAPABILITIES.md ## Error taxonomy.

Observability

What "healthy install" looks like under observation. The agent should run these checks (or have the user run them) before recommending any code-level change. Program-side observability (DOCA log levels, capability snapshots, library counters) lives in doca-programming-guide CAPABILITIES.md ## Observability.

Install layer:

ls /opt/mellanox/doca                                  # SDK root present
ls /opt/mellanox/doca/samples                          # samples shipped
ls $(pkg-config --variable=includedir doca-common)           # headers shipped
ls /opt/mellanox/doca/infrastructure/lib/pkgconfig/    # *.pc files shipped
pkg-config --list-all | grep -i doca                   # what the build env can find
pkg-config --modversion doca-common                    # the unified version

Capability layer (DPU and Flow):

doca_caps                                              # device capabilities snapshot
doca_caps --version                                    # runtime version

Runtime layer (host or DPU):

mount | grep huge                                      # hugepages mounted
cat /proc/meminfo | grep -i huge                       # hugepages reserved
devlink dev show                                       # devices visible
ls /sys/class/net/                                     # network interfaces
cat /sys/class/net/*/phys_port_name                    # representor names

On a hardware target, the presence of all of the above (without errors or empty output where output is expected) is the precondition for program-level runtime work. The agent's investigation order on an env-class report is exactly install → capability → runtime, in that order — never start at the application layer when these have not been verified.

Inside the NGC DOCA container, the install and capability layers above will respond normally; the runtime layer will partially or fully report no real hardware (no hugepages mounted by default, no devlink devices visible, no representors). That is the expected state for the build / read / learn loop the container is for; for runtime against real hardware, the user has to graduate to a hardware path (see TASKS.md ## no-install Paths A and C).

Safety policy

DOCA install and runtime preparation are shared system state. Several env actions affect more than just the user's program; the agent must surface this before recommending them. The program-side safety policy (validate-before-commit, stage-first-widen-later) lives in doca-programming-guide CAPABILITIES.md ## Safety policy.

1. Never modify /opt/mellanox/doca/lib*/. The shipped libraries are the release; rewriting them voids the install and breaks any other DOCA application on the host. Build flavor changes go via LD_LIBRARY_PATH or the -trace pkg-config module, never by editing the lib/ tree.

2. Hugepage mounts and reservations are global. Adding hugepages reduces memory available to the kernel and to other applications; removing or remounting hugepages while another DOCA / DPDK application is running will crash that application. Before recommending a hugepages change, ask the user whether anything else on this host is using DOCA or DPDK; if yes, do the change in coordination, not in isolation. The same applies to mlxconfig changes that require a reset, and to devlink dev eswitch set ... mode switchdev.

3. Never auto-reboot or auto-power-cycle. mlxconfig and BFB updates require a host reset to take effect. The agent should produce the commands and explain the requirement, but must not chain them with an unattended reboot — let the user confirm their environment can absorb the reboot.

4. The NGC container is build / read / learn only — not a runtime substitute. The public NGC DOCA container at nvcr.io/nvidia/doca/doca is the canonical Stage-1 path for any user on macOS, Windows, or Linux without DOCA, but it is not a substitute for running against real hardware. DPDK / DOCA calls that require a real NIC, real DPU, real hugepages on the host kernel, or real driver presence will fail inside the container — and that failure is correct. The agent must not work around this boundary by granting the container unrestricted host privileges or exposing host devices; when the user needs runtime, graduate them from Path 0 to Path A or Path C in TASKS.md ## no-install.

5. Be explicit about what changes when the user installs DOCA. Installing the host package modifies kernel modules (mlx5_core, OFED), adds udev rules, and (for doca-all) installs services that may auto-start. Surface this before recommending an install on a host that has other workloads.

Universal verification contract

Every answer that recommends a change (build / deploy / configure / modify / install / upgrade) MUST end with the 5-step verification contract below. This is the bundle-wide shape that every per-artifact ## test anchor instantiates with artifact-specific commands. Skipping any step makes the answer ineligible to declare the task complete.

Scope: the env-health loop in TASKS.md ## test runs first. This contract then governs a change recommendation. On Stage-1 NGC containers without hardware, hardware preconditions and traffic counters are explicitly not applicable; green means the selected sample builds, starts and exits cleanly at the no-traffic baseline, and doca_caps/pkg-config report the versions expected for the copied catalog tag. Never fabricate PCIe or representor evidence.

Stacking rule. When a per-artifact skill's ## test anchor is loaded (e.g. doca-flow ## test), the artifact-specific steps layer on top of the universal contract — they do not replace it. The agent runs both: the universal 5 steps as the spine, the artifact-specific overlay as the per-step instantiation.

Cross-cutting layering. When doca-hardware-safety is also loaded (see the activation triggers in AGENTS.md), step 1 (Preconditions) MUST include the pre-flight inventory from doca-hardware-safety TASKS.md ## configure and step 5 (Observability + declare done) MUST include the rollback path verification ("the rollback for this change is X; the agent verified it works before applying"). When doca-version is also loaded, step 1 MUST cite the four-source detection chain explicitly and step 3 MUST include a post-change doca_caps --version re-read to confirm the version state did not silently shift.

Deploy-loop bridge — step 5 not-green is the debug-loop trigger

The verification contract describes a convergent shape: step 3 smoke probe returns green, step 5 observability surface stays green, the task is done. Real deploys frequently land on not-green at step 5 (Ready 0/1, port Down, counter flat, log line absent, systemd active (running) followed by repeated restarts) and the agent's failure mode is to declare done anyway because the change "looks applied." The deploy-loop bridge prevents that.

The agent MUST treat "step 5 observability did NOT reach the named green signal within the expected window" as the symptom that fires the universal debug-loop contract on the change-not-converging symptom. That is: the deploy is now in the debug-loop's phase 1 (layer identification) on a deploy-class symptom — the agent walks layer identification → triple capture → single-variable mutation → re-capture → exit, exactly as it would for a DOCA_ERROR_* symptom. The deploy-loop bridge is the named hand-off that prevents the "applied but unverified" outcome.

The bridge converts a verification-only answer into a verification + iteration answer for every change-recommending prompt — not only error-symptom prompts. Deploy / configure / install / upgrade prompts that previously stopped at "and watch the metric for green" now must say "and watch the metric for green; if the metric is not green within X, walk the debug-loop on the not-green symptom; if the loop's second iteration does not converge, walk the rollback path." That is the cross-cutting auto-debug-loop coverage the bridge guarantees on change-recommending prompts where no failure has been reported yet.

Hardware binding-layer command stanza

Any answer that touches hardware state — recognizing the system shape, configuring representors, pinning workloads to NUMA / IRQ / queues, debugging a "does nothing on the wire" runtime symptom, applying a doca-hardware-safety overlay — MUST instantiate the binding-layer command stanza below. The stanza is the bundle-wide hardware-enumeration shape; per-artifact skills (doca-bare-metal-deployment, doca-flow, libs/doca-rdma, …) layer their library-specific binding details on top.

The agent's failure mode the stanza replaces: mentioning hardware ("you'll want to pin this to the right NUMA node") without naming the specific commands that produce the enumeration. Pointing at "check the binding" without running the commands is the same failure mode the debug-loop contract replaces for symptoms — hand-waving instead of evidence.

The stanza is read-only. Every row is a side-effect-free enumeration. Mutating commands (mlxconfig -d <bdf> set ..., devlink dev eswitch set ... mode switchdev, kernel-module reload, BFB reflash) are NOT part of the stanza — they belong in the per-artifact ## modify anchor and require the doca-hardware-safety overlay. Running the stanza is always safe; acting on what it reveals is what requires the safety overlay.

Stacking rule. When a per-artifact skill is loaded (e.g. doca-bare-metal-deployment, doca-flow), the per-artifact ## configure / ## run anchors layer library-specific binding details on top — for Flow, the steering-mode capability query (doca_caps) and the representor-to-port_id mapping; for bare-metal, the cgroup-v2 + namespace + taskset instantiation. The cross-cutting stanza is the spine; the per-artifact overlay names the artifact-specific calls.

Cross-cutting layering. When the ## recognize front-door routing decision fires, the auto-detect block at step 1 of that anchor is a subset of this stanza (PCIe presence + DOCA install presence). The full stanza is what ## recognize hands off to the routed skill so the downstream answer has the binding-layer picture in hand. When the universal verification contract fires, step 1 (Preconditions) MUST include the relevant rows of this stanza for any hardware-touching change.