Index of every page on this site: llms.txt
All skills
nvidia avatar

/nemo-automodel-recipe-development

@6b75e9e
by NVIDIA Corporationnvidia/skills4.1k stars
547
Run it yourself

Create and modify NeMo AutoModel training and evaluation recipes, including YAML structure, builders, and execution flow.

Use this Skill: https://skilld.dev/gh/nvidia/skills/nemo-automodel-recipe-development

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

skill-card.md

≈1k tokens on demand. Your agent reads this file only when SKILL.md points to it.

Description: <br>

Create and modify NeMo AutoModel training and evaluation recipes, including YAML structure, builders, and execution flow. <br>

This skill is ready for commercial/non-commercial use. <br>

Owner

NVIDIA <br>

License/Terms of Use: <br>

Apache 2.0 <br>

Use Case: <br>

Developers and ML engineers creating or modifying NeMo AutoModel training, SFT, evaluation, and knowledge distillation recipes using YAML-driven configurations and the automodel CLI. <br>

Deployment Geography for Use: <br>

Global <br>

Requirements / Dependencies: <br>

Requires API Key or External Credential: [Not Specified] <br> Credential Type(s): [None identified] <br>

Do not include secrets in prompts/logs/output; use least-privilege credentials; rotate keys as appropriate. <br>

Known Risks and Mitigations: <br>

Risk: Review before execution as proposals could introduce incorrect or misleading guidance into skills. <br> Mitigation: Review and scan skill before deployment. <br>

Reference(s): <br>

Skill Output: <br>

Output Type(s): [Configuration instructions, Code] <br> Output Format: [Markdown with inline YAML and bash code blocks] <br> Output Parameters: [1D] <br> Other Properties Related to Output: [None] <br>

Evaluation Agents Used: <br>

  • Claude Code (aws/anthropic/bedrock-claude-opus-4-8) <br>
  • Codex (openai/openai/gpt-5.5) <br>

Evaluation Tasks: <br>

Evaluated against 3 positive evaluation tasks covering new finetuning recipe creation, YAML target pattern usage, and validation/checkpointing configuration. Each task ran with 3 attempts in isolated sandbox pods. <br>

Evaluation Metrics Used: <br>

Reported benchmark dimensions: <br>

  • Security: Checks for unsafe operations, secret leakage, and unauthorized access. <br>
  • Correctness: Checks final-answer correctness against the reference answer. <br>
  • Discoverability: Checks whether the expected skill was selected, decoys were avoided, and the workflow executed. <br>
  • Effectiveness: Checks whether the user's goal was achieved (50%) and expected workflow behavior was followed (50%). <br>
  • Efficiency: Checks tool-call productivity (50%) and token efficiency (50%). <br>

Underlying evaluation signals used in this run: <br>

  • security: Unsafe operations, secret leakage, and unauthorized access. <br>
  • accuracy: Final-answer correctness against the reference answer. <br>
  • skill_execution: Whether the expected skill was selected, decoys were avoided, and the workflow executed. <br>
  • goal_accuracy: Whether the user's goal was achieved. <br>
  • behavior_check: Whether the expected workflow behavior was followed. <br>
  • skill_efficiency: Tool-call productivity. <br>
  • token_efficiency: Actual uncached prompt plus completion token usage. <br>

Evaluation Results: <br>

Measure Claude Code (Baseline → Skill Uplift) Codex (Baseline → Skill Uplift)
Overall 99.1% 98.7%
Security 100.0% → 100.0% (±0.0 points) 100.0% → 100.0% (±0.0 points)
Correctness 86.7% → 100.0% (+13.3 points) 86.7% → 100.0% (+13.3 points)
Discoverability 100.0% 95.0%
Effectiveness 48.9% → 97.2% (+48.3 points) 46.3% → 100.0% (+53.7 points)
Efficiency 98.4% 98.5%

Skill Version(s): <br>

v1.2.1+4214430 (source: pyproject.toml) <br>

Ethical Considerations: <br>

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal team to ensure this skill meets requirements for the relevant industry and use case and addresses unforeseen product misuse. <br>

(For Release on NVIDIA Platforms Only) <br> Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns here. <br>

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No alerts2mo3 checks · Risk SAFE
  • Gen Agent Trust Hub2mo

    This skill provides instructions for developing and modifying NeMo AutoModel recipes using YAML configurations. It is authored by NVIDIA and focuses on technical workflows for machine learning training and evaluation. The skill utilizes standard framework patterns for configuration and local validation. No malicious patterns or security risks were detected.

  • Socket2mo

    No alerts

  • Snyk2mo

    Risk: LOW · No issues

Signed by skilld at 6b75e9e. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated 2 months ago
Other metadata
when_to_use
Creating or modifying training, SFT, or eval recipes, adding new YAML config fields, debugging recipe construction or trainer issues, or understanding the recipe execution flow.
metadata
{
  "author": "NVIDIA",
  "tags": [
    "nemo-automodel",
    "recipe-development"
  ]
}

README badge

README badge for nvidia/skills/nemo-automodel-recipe-development