Index of every page on this site: llms.txt
All skills
nvidia avatar

/nvidia-ontology-setup

@43fa2d3
by NVIDIA Corporationnvidia/auto-ontology14k stars
1,224
Run it yourself

Set up or troubleshoot the Auto Ontology runtime. Use for Helm (the official install), Docker Compose, developer setup, and MCP connection to an existing deployment.

Use this Skill: https://skilld.dev/gh/nvidia/auto-ontology/nvidia-ontology-setup

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

BENCHMARK.md

≈2.9k tokens on demand. Your agent reads this file only when SKILL.md points to it.

Skill Benchmark: nvidia-ontology-setup

✅ Overall verdict: PASS — Recommended for publication

Publication Recommendation

Recommended for publication based on the completed evaluation evidence in this report.

Evaluation Metadata

  • Skill: nvidia-ontology-setup
  • Validation status: passed
  • Evaluation date: 2026-11-19
  • Evaluator version: 1.6.10
  • Agents: Claude Code (aws/anthropic/bedrock-claude-opus-5-5), Codex (openai/openai/gpt-6.1-sol)
  • Tasks: 10 evaluation tasks (10 positive)
  • Dataset digest: sha256:dcbcf0848aefb19cd5b010e9a0d69ed8920e61dc3fdba560ca0f88cd48a8da96 (skill-evaluator-dataset-snapshot/1)
  • Attempts per task: 1
  • Environment: k8s-sandbox
  • Tier 2 evidence: required for publication
  • Tier 3 evidence: required for publication

Each task attempt ran in its own isolated sandbox pod.

Tier 1 LLM Provenance

SkillSpector provider, model, analyzer revision, and determinism evidence retained for this scan.

Skill Provider Adapter Analyzer Model Revision Determinism
nvidia-ontology-setup nv_inference openai mcp_tool_poisoning aws/anthropic/bedrock-claude-opus-5-5 2.13.5 nondeterministic / provider_defaults
nvidia-ontology-setup nv_inference openai semantic_developer_intent aws/anthropic/bedrock-claude-opus-5-5 2.13.5 nondeterministic / provider_defaults
nvidia-ontology-setup nv_inference openai semantic_quality_policy aws/anthropic/bedrock-claude-opus-5-5 2.13.5 nondeterministic / provider_defaults
nvidia-ontology-setup nv_inference openai semantic_security_discovery aws/anthropic/bedrock-claude-opus-5-5 2.13.5 nondeterministic / provider_defaults
nvidia-ontology-setup nv_inference openai meta_analyzer aws/anthropic/bedrock-claude-opus-5-5 2.13.5 nondeterministic / provider_defaults
  • Controls for nvidia-ontology-setup: temperature=not set (source=provider_default, forwarded=no); seed=not set (source=unset, forwarded=no); reasoning_effort=not set (source=provider_default, forwarded=no)

What This Report Answers

The three-tier evaluation checks whether the skill:

  • is safe to use;
  • produces correct answers;
  • is discovered and activated when needed;
  • helps the agent complete the user's goal and expected workflow; and
  • avoids wasted skill and tool usage.

Results at a Glance

Measure Claude Code (Baseline → Skill Uplift) Codex (Baseline → Skill Uplift)
Overall 94.1% — baseline ran, but no comparable score was available; uplift unavailable 88.8% — baseline ran, but no comparable score was available; uplift unavailable
Security 80.0% → 100.0% (+20.0 points) 75.0% → 90.0% (+15.0 points)
Correctness 48.0% → 100.0% (+52.0 points) 58.0% → 98.0% (+40.0 points)
Discoverability 93.5% — baseline ran, but no comparable score was available; uplift unavailable 91.5% — baseline ran, but no comparable score was available; uplift unavailable
Effectiveness 46.4% → 90.0% (+43.6 points) 55.5% → 78.3% (+22.8 points)
Efficiency 86.8% — baseline ran, but no comparable score was available; uplift unavailable 86.0% — baseline ran, but no comparable score was available; uplift unavailable

How to read this table: baseline is the same task attempted without the target skill. Scores are rounded to one decimal; threshold-adjacent values use additional precision so their displayed band matches the verdict. Uplift is derived from those displayed scores and shown in percentage points.

Example: 47.0% → 92.0% (+45.0 points) means the skill-assisted run scored 92.0%, 45.0 percentage points above its 47.0% no-skill baseline.

Comparison details

Shared-source comparisons use the same signals and weights on both conditions. They do not replace the full dimension scores above or change the publication verdict.

Agent Dimension Basis Baseline → With skill (uplift) Explanation
claude-code Discoverability Dimension comparison N/A Discoverability is not applicable to the no-skill baseline; there is no installed skill to discover.
claude-code Efficiency Token-only comparison 97.8% → 96.1% (-1.7 points) Only Token Efficiency is comparable; the full dimension scores use different evaluator sources.
codex Discoverability Dimension comparison N/A Discoverability is not applicable to the no-skill baseline; there is no installed skill to discover.
codex Efficiency Token-only comparison 89.3% → 91.2% (+1.9 points) Only Token Efficiency is comparable; the full dimension scores use different evaluator sources.

Token Usage

Actual Tier 3 execution usage is reported for every observed agent/case pair and both conditions.

Agent Dataset case With skill Without skill Delta Change Coverage
claude-code All cases 1,207,654 1,514,844 -307,190 -20.28% skill 10/10; base 10/10
claude-code setup-eval-001-underspecified 116,199 107,379 +8,820 +8.21% skill 1/1; base 1/1
claude-code setup-eval-002-use-existing-scripts 75,399 146,574 -71,175 -48.56% skill 1/1; base 1/1
claude-code setup-eval-003-auth-secret-cookies 117,251 143,673 -26,422 -18.39% skill 1/1; base 1/1
claude-code setup-eval-004-postgres-port-host-only 115,760 142,520 -26,760 -18.78% skill 1/1; base 1/1
claude-code setup-eval-005-mcp-already-up 115,272 181,282 -66,010 -36.41% skill 1/1; base 1/1
claude-code setup-eval-006-vault-partial 107,366 142,761 -35,395 -24.79% skill 1/1; base 1/1
claude-code setup-eval-007-embed-triplet-mismatch 214,291 143,259 +71,032 +49.58% skill 1/1; base 1/1
claude-code setup-eval-008-connect-source-workflow 116,719 221,039 -104,320 -47.20% skill 1/1; base 1/1
claude-code setup-eval-009-env-vs-ui-connections 112,577 143,239 -30,662 -21.41% skill 1/1; base 1/1
claude-code setup-eval-010-helm-install 116,820 143,118 -26,298 -18.38% skill 1/1; base 1/1
codex All cases 526,689 624,717 -98,028 -15.69% skill 10/10; base 10/10
codex setup-eval-001-underspecified 29,618 71,920 -42,302 -58.82% skill 1/1; base 1/1
codex setup-eval-002-use-existing-scripts 33,946 49,034 -15,088 -30.77% skill 1/1; base 1/1
codex setup-eval-003-auth-secret-cookies 46,937 54,703 -7,766 -14.20% skill 1/1; base 1/1
codex setup-eval-004-postgres-port-host-only 51,236 44,377 +6,859 +15.46% skill 1/1; base 1/1
codex setup-eval-005-mcp-already-up 67,020 78,344 -11,324 -14.45% skill 1/1; base 1/1
codex setup-eval-006-vault-partial 92,834 57,924 +34,910 +60.27% skill 1/1; base 1/1
codex setup-eval-007-embed-triplet-mismatch 47,130 45,875 +1,255 +2.74% skill 1/1; base 1/1
codex setup-eval-008-connect-source-workflow 63,884 76,064 -12,180 -16.01% skill 1/1; base 1/1
codex setup-eval-009-env-vs-ui-connections 47,432 80,059 -32,627 -40.75% skill 1/1; base 1/1
codex setup-eval-010-helm-install 46,652 66,417 -19,765 -29.76% skill 1/1; base 1/1
ALL AGENTS Dataset aggregate 1,734,343 2,139,561 -405,218 -18.94% skill 20/20; base 20/20

Prompt tokens include cached reads, so total tokens are prompt + completion (cached is not added twice). The Efficiency score uses (prompt - cached) + completion. Change is 100 * (with - without) / without; negative means fewer tokens. N/A comparisons include their reason in the coverage column; coverage is never estimated.

Tier Status

Tier Purpose Status Evidence
Tier 1 Static validation PASSED WITH OBSERVATIONS 11 validator(s); 6 finding(s)
Tier 2 Semantic deduplication PASSED 2 validator(s); 0 finding(s)
Tier 3 Live agent evaluation PASS 2 agent(s); 10 task(s)

Findings and Observations

<details> <summary>Show detailed findings and successful checks</summary>
  • MEDIUM QUALITY/quality_correctness: Resource-based skill lacks documentation of available resources (skills/nvidia-ontology-setup/SKILL.md)
  • MEDIUM SECURITY/Autonomous Decision Making (EA2): Excessive Agency: Never ask the user (SKILL.md:65)
  • MEDIUM SECURITY/External Transmission (E1): Data Exfiltration: curl -s -o /dev/null -w '%{http_code}\n' "$AUTO_ONTOLOGY_API_URL/.well-known/oauth-authorization-server"

`200` is required. Anything else: run the frontend from the checkout
(`pnpm dev`) and poin (`SKILL.md:203`)
- **LOW** QUALITY/quality_discoverability: Skill uses exclusivity language that conflicts with composability (`skills/nvidia-ontology-setup/SKILL.md`)
- **LOW** QUALITY/quality_reliability: No prerequisites/requirements documented (`skills/nvidia-ontology-setup/SKILL.md`)
- 1 additional finding(s) are available in the full evaluation artifacts.

</details>

## Scoring Methodology

<details>
<summary>Show dimension definitions, source signals, and thresholds</summary>

| Dimension | Question | Scored signals |
|---|---|---|
| Security | Is it safe to use? | `security` (100%) |
| Correctness | Is the answer correct? | `accuracy` (100%) |
| Discoverability | Was the right skill loaded when needed? | `skill_execution` (100%) |
| Effectiveness | Did the skill help complete the task? | `goal_accuracy` (50%) + `behavior_check` (50%) |
| Efficiency | Did it avoid wasted tool calls and token usage? | `skill_efficiency` (50%) + `token_efficiency` (50%) |

- Dimension bands: PASS at 50% or above; NEUTRAL from 40% to below 50%; FAIL below 40%.
- Overall Tier 3 lift: PASS at +5 points or more; FAIL at -10 points or less; values between those bands are NEUTRAL.
- Overall verdict: PASS only when every configured dimension passes for at least one supported agent. Lift is reported as diagnostic evidence and does not override this gate.
- The 50% attempt pass threshold is a separate per-task gate; it is not the dimension pass threshold.
- Effectiveness is the equal-weight mean of goal completion (`goal_accuracy`) and expected workflow adherence (`behavior_check`).
- Efficiency is 50% tool-call productivity (the backward-compatible `skill_efficiency` wire id) and 50% `token_efficiency`. Positive-case skill routing is scored under Discoverability, not Efficiency; a negative case without a routing target is N/A. N/A sources are omitted, remaining weights are renormalized, and the dimension is marked partial.

Signals present in this run:

- `security` (Security): unsafe operations, secret leakage, and unauthorized access.
- `skill_execution` (Skill Execution): whether the expected skill was selected, decoys were avoided, and the workflow executed.
- `skill_efficiency` (Tool Productivity): tool-call productivity (legacy wire id; routing is scored under Discoverability).
- `accuracy` (Accuracy): final-answer correctness against the reference answer.
- `goal_accuracy` (Goal Accuracy): whether the user's goal was achieved.
- `behavior_check` (Behavior Check): whether the expected workflow behavior was followed.
- `token_efficiency` (Token Efficiency): actual uncached prompt plus completion usage (50% of Efficiency).

</details>

## Freshness

Regenerate this benchmark when the skill, evaluation dataset, target agent/model, evaluator version, environment, or scoring policy changes.

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

1 warningtoday3 checks · Risk SAFE
  • Gen Agent Trust Hubtoday

    The skill provides installation, troubleshooting, and MCP configuration instructions for NVIDIA's Auto Ontology system using Helm and Docker Compose. Analysis found that all external resources and tools belong to the vendor's verified infrastructure and follow secure practices for secret management and network operations.

  • Socket2d

    No alerts

  • Snyktoday

    Risk: MEDIUM · 1 issue

Signed by skilld at 43fa2d3. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated 2 days ago
Other metadata
metadata
{
  "version": "0.3.2",
  "author": "NVIDIA <opensource@nvidia.com>",
  "tags": [
    "nvidia-ontology",
    "install",
    "docker",
    "helm",
    "mcp",
    "troubleshooting"
  ]
}

README badge

README badge for nvidia/auto-ontology/nvidia-ontology-setup