/doca-dms

DOCA Management Service — Tasks

Where to start: The order is configure → build → modify → run → test → debug. The ## test verb is an iterative loop, not a one-shot pass — see the eval-loop overlay in ## test below. For DMS, build and modify are about daemon configuration (SystemD unit, flags, YANG-instance fragments), not about compiling source.

These verbs cover the in-scope DMS operational workflows for an external operator deploying and using DMS. Every step assumes the operator has consulted the live public DMS guide on docs.nvidia.com and is using it as the authoritative reference; this file prescribes the order and what to look up where, not a copy-paste runbook. If the version-matched live guide is unavailable, inspect the installed dmsd --help, SystemD unit, and installed documentation. If none is available for the installed release, stop and report the version/source gap; do not fall back to remembered flags or paths.

configure

Preparing the management endpoint and choosing a deployment shape.

1. Confirm the env is healthy first. This skill expects DOCA to be installed at the management endpoint (host, BlueField Arm, or pod image base). If that has not been verified, run doca-setup ## test first. If the user has no install yet, route to doca-setup ## no-install for the public NGC DOCA container path. Do not return until the doca-setup ## test install/version, capability, and smoke checks are green for the management endpoint; run that check once. If it errors or cannot complete under its documented timeout, abort the DMS workflow and route the captured result to doca-setup ## debug; do not retry from this skill. A package merely being present is not sufficient.
2. Identify the deployment shape. Read the public DMS guide's "Service Deployment" section and map the user's situation to one of the documented shapes (Host non-DPU / BlueField Arm / Kubernetes pod). Each shape has its own "Prerequisites" subsection — read it before any further step.
3. Identify the management protocol scope. Confirm whether the user needs gNMI (data configuration), gNOI (system operations), or both. gNMI Subscribe streaming IS supported (STREAM with SAMPLE 1s–60s and ONCE; only POLL / Aggregation are Unimplemented per gnxi/gnmi/server.go). For a turnkey, productized telemetry-aggregation surface separate from DMS's own gNMI Subscribe, route to the DOCA Telemetry Service (DTS) instead.
4. Plan the authentication mode. Choose one of the four documented modes (localhost-only / PAM / credentials / mTLS) and read the Security Best Practices subsection before committing. Localhost-only is for development only and must not be exposed. Record the bind scope, credential/certificate source, server trust policy, client-auth requirement, and -allowed_users behavior from the live guide; reject a plan that weakens any of those compared with the matching documented recipe.
5. Plan user authorization. Identify the human/service users who will issue DMS commands over gRPC and ensure they are in the -allowed_users list (default root; enforced by isUserAllowed in gnxi/utils/credentials/credentials.go). The dmsgroup Unix group is a separate, dmspe-only gate — add a user to it only if they also need to invoke the dmspe privileged helper directly on the endpoint.

build

DMS is a service, not a library. There is no DMS application artifact for the operator to build — the daemon (dmsd/dmspe) ships with the DOCA install, and clients are standard gNMI/gNOI tooling that the operator already has.

If the user is asking how to build a gNMI/gNOI client in their own language, the right routing is:

• C / C++ — generic gRPC + gNMI/gNOI client patterns. Outside DMS scope. The DMS-specific contribution is the path inventory and the operation list documented in CAPABILITIES.md ## gNMI client surface and ## gNOI client surface.
• Other languages (Go / Python / Rust / …) — same answer. Use the standard gNMI/gNOI client libraries for that language; DMS does not author wrappers and does not maintain language-specific clients.

If the user is instead asking about building a DOCA application in the general sense (linking against libdoca-*), that is not a DMS question — route them to doca-programming-guide ## build and the matching libs/<library> skill.

modify

DMS does not have a "modify a sample" workflow analogous to DOCA libraries; there is no DMS sample program a user starts from. The DMS analog of "modify" is adapt the documented launch / auth / deployment recipe to the user's environment:

1. Start from the documented recipe. Identify the public guide's recipe that matches the user's deployment shape and authentication mode. Quote it; do not author a new one.
2. Diff against the user's environment. Note the specific substitutions the user must make: hostnames, certificate paths, allowed_users lists for PAM mode, port assignments, network-exposure decisions.
3. Apply minimum-change. Change only what the user's environment forces. Every additional deviation from the documented recipe widens the surface for an unintended exposure.
4. Re-validate against the Security Best Practices subsection. Each substitution is a chance to accidentally weaken the documented posture. Re-check the five recorded controls from configure step 4: bind scope, secret/certificate source, server trust, client auth, and -allowed_users. Any unapproved widening (for example binding localhost-only mode to 0.0.0.0) is a stop condition.

run

Bringing up dmsd and exercising it.

1. Decide SystemD vs manual launch. SystemD is the documented recommended path for production. Manual launch is appropriate for bring-up, debugging, and walking the documented advanced configurations.
2. Apply the documented launch flags. Flag inventory lives under "DMS Server Flags (dmsd)" in the public guide, partitioned into General & Provisioning / Authentication & Security / Authentication Method / Security flag families. Quote flags from the live guide; do not infer flags from generic gRPC knowledge.
3. Verify the daemon is reachable. Use a documented gNMI Get against a known-supported State path as a connectivity probe. "Documented" matters: a Get against an unsupported path returns an error from dmsd regardless of whether the daemon is healthy.
4. Verify authorization works as expected. Confirm a user in the -allowed_users list can issue gRPC commands and a user outside it cannot. If the test fails, the failure is at the authorization layer, not the protocol layer — see ## debug step 2.
5. Issue the user's first real Set against a documented path. Then Get the corresponding State path to confirm the change landed. The Set / Get pattern across Configuration vs State branches is the documented validation idiom.

For gNOI operations: read the operation's documented sub-operation list first, then issue. Operations like OS install, Reboot, and Factory-reset change the managed device state in non-trivial ways — the operator should know which sub-operation they are invoking and why.

⚠️ Destructive and irreversible — confirm before issuing. OS install, Reboot, Factory-reset, and managed-file deletion are service-impacting and cannot be undone: they can take a production BlueField or ConnectX offline or wipe its configuration. Before issuing any of them the agent MUST (1) verify the target device identity and (2) obtain authorization bound to that target and action: an explicit user reply in an interactive session or an approved-system authorization artifact in unattended execution. Otherwise stop with confirmation_required. Never issue them speculatively or as an implicit side effect of another task.

test

DMS has no "compile and unit-test" workflow — testing is operational.

## test is a bounded iterative loop, not a one-shot pass. Every configuration mutation (auth mode, listener, -allowed_users membership, persistency setting) re-opens the smoke sweep. Skipping the re-run after a mutation is the failure mode this loop replaces. Permit one mutation-and-rerun cycle total per test invocation, regardless of which back-edge or failure class consumes it. If the second sweep reveals any failure, including a different failure class, do not take another back-edge. Return a diagnostic bundle to the operator containing all smoke results, collected debug-layer outputs, and the capability snapshot when step 4 completed; otherwise state that the snapshot was not reached. Mark the result requires_human_decision instead of iterating again or invoking another skill automatically.

Green criteria: the daemon returns the expected shape for a documented Get; the configured auth mode rejects the documented negative request (or, for localhost-only mode, the listener is loopback-only and unreachable remotely); the required isolated persistency probe survives restart and is rolled back; and the capability snapshot matches the selected deployment shape.

The eval-loop overlay (rows apply to every DMS deployment, not just one topology):

The agent's rule: every mutation re-opens the sweep. A configuration change followed by "it probably still works" is exactly the failure mode the iterative loop is here to prevent.

The destructive-operation gate remains active inside both ## test and ## debug: no OS install, reboot, factory reset, or managed-file deletion may be used as a probe or recovery action without re-verifying the exact target and obtaining confirmation for that specific operation.

1. Smoke-test the daemon. After launch (## run step 3), confirm the daemon answers a documented gNMI Get and returns expected shape.
2. Smoke-test the auth mode. For credentialed modes, confirm the chosen mode rejects unauthenticated/unauthorized requests as documented. For mTLS, this means an explicit "request without cert is rejected" check. For PAM, this means an "user not in allowed_users is rejected" check. Localhost-only has no authentication: its negative test is that dmsd is bound only to loopback and cannot be reached through a non-loopback interface.
3. Smoke-test persistency only on an isolated test path. If the deployment relies on persistency, have the operator select a documented, reversible path on a test-only interface/instance, snapshot its current value, set a distinct test value, restart dmsd, verify the value survives, then restore and verify the snapshot. If no isolated reversible path exists, skip the mutation and mark persistency not_tested; when persistency is required this keeps the sweep not-green and blocks completion. Never improvise a "benign" value on production state.
4. Capability snapshot. Save the as-deployed answer to: which gNMI paths your environment supports, which gNOI operations your environment supports, which auth mode is active, who is in the -allowed_users list. This snapshot is the artifact that lets future debug sessions skip rediscovery.

debug

Layered diagnosis. Walk the layers in this order; do not skip down without clearing the layer above.

1. Transport layer. Is the gRPC channel even reaching dmsd? Symptoms: UNAVAILABLE, DEADLINE_EXCEEDED, TLS handshake failures. Causes: wrong host/port, firewall, daemon not running, TLS material misconfigured. Resolution: confirm dmsd is running (SystemD status or process list), confirm the listener address matches the client's target, confirm TLS material if mTLS mode is in use.
2. Authentication / authorization layer. Symptoms: UNAUTHENTICATED, PERMISSION_DENIED, "user not authorized". Causes: wrong credentials, user not in the -allowed_users list (applies to ALL auth modes, enforced by isUserAllowed), certificate issuer not trusted by dmsd. Resolution: walk the documented authentication-mode troubleshooting in the public guide for the specific mode in use.
3. Path / operation layer. Symptoms: INVALID_ARGUMENT, "path not found", "operation not supported". Causes: the path is not in the DMS-supported set, the operation is not in the documented gNOI list, or an unsupported gNMI Subscribe MODE was attempted (POLL / Aggregation are Unimplemented; STREAM / SAMPLE 1s–60s and ONCE ARE supported). Resolution: re-read the supported-paths reference and the gNOI operation list in CAPABILITIES.md. If the path or operation is genuinely not in the public set, the answer is "not supported" — not "invent a workaround".
4. Backend / underlying-tool layer. Symptoms: the operation reached dmspe and the underlying tool (e.g. mlxconfig, OS installer, file system) returned an error. Resolution: extract the underlying-tool error from the response and consult that tool's own documentation. Preserve the raw gRPC status code, message, and documented details fields verbatim. If the selected release documents no nested error field, capture the complete sanitized response and stop rather than guessing which substring belongs to the backend. Return that response as the diagnostic artifact and mark the diagnosis blocked pending the underlying tool's documentation. DMS is the conduit, not the source of truth for the underlying tool's failures.
5. State persistency layer. Symptoms: a previously-set value does not survive a daemon restart. Resolution: confirm automatic recording is enabled (it can be disabled per the docs), confirm the configuration-persistency file is writable and not being overwritten by an external process, then consult the documented state restoration mechanism.
6. Library-level errors. If DMS is acting as a thin wrapper over a DOCA library call and that library returned DOCA_ERROR_*, the relevant cross-library taxonomy lives in doca-programming-guide CAPABILITIES.md ## Error taxonomy. The library-specific overlay (e.g. for Flow) lives in the matching libs/<library> skill.

Command appendix

DMS-specific commands the verbs above reach for, grouped by purpose so the agent picks the right family without searching prose. Every row is a class — the agent must not invent flags beyond what the row names; flag discovery is --help on the installed binary or the SystemD unit file, not prose recall.

Three cross-cutting rules for this appendix:

• Never invent a DMS flag. The public guide is the contract; dmsd --help against the installed binary is the only secondary source. Prose-derived flags are the most common hallucination failure for this skill.
• Frontend logs before backend logs. When triaging, read journalctl -u dmsd (or the documented frontend log) first; only drop to dmspe once the frontend confirms the request reached the backend.
• Cross-link instead of duplicate. Cross-cutting commands (the read-only triple, dmesg, mlxconfig -d <pcie> q) live in doca-debug TASKS.md ## Command appendix; this appendix names only the DMS-specific ones.

Deferred task verbs

• Installing DOCA on the management endpoint — out of scope here. Route to doca-setup ## configure for env preparation and doca-setup ## test for install health verification, or doca-setup ## no-install for the public NGC DOCA container Path 0 if there is no DOCA install yet.
• Building a custom DOCA application — not a DMS question. Route to doca-programming-guide ## build for the canonical build pattern, plus the matching libs/<library> skill for the API surface.
• Turnkey telemetry aggregation — out of scope for DMS. DMS's own gNMI Subscribe DOES stream (STREAM / SAMPLE 1s–60s and ONCE; only POLL / Aggregation are Unimplemented); for a productized telemetry-aggregation surface, route to the DOCA Telemetry Service (DTS), discoverable through doca-public-knowledge-map.
• Library-internal API questions (Flow pipe construction, RDMA queue setup, …) — outside DMS. Route to the matching libs/<library> skill.

Cross-cutting

• The public DMS guide is the single source of truth. Any flag, path, operation, or auth-mode detail the agent quotes must come from there, not from generic gNMI / gNOI / gRPC knowledge.
• DMS is currently beta with GA scoped to SPC-X use cases. Treat any "what is the long-term roadmap" question as out of scope and defer to the live public guide.
• Localhost-only auth is never safe to expose externally.
• All operational guidance must preserve the documented two-process separation (dmsd low-priv frontend, dmspe privileged backend).
• For URL routing to the DMS guide and other public DOCA documentation, see doca-public-knowledge-map ## DOCA services.