skilld
All skills
nvidia avatar

/digital-health-clinical-asr-finetune

@1ba3403
by NVIDIA Corporationnvidia/skills3.5k stars
424

Stage 4 of the Clinical ASR Flywheel. Use when priority KER is above 0.3 to run stock NeMo SFT on Parakeet TDT v2 and offline cycle N+1 re-eval. NOT for generic word boosting (use /finetune-asr).

Requires /digital-health-clinical-asr-eval /digital-health-clinical-asr-build

Ask your Agent

Use this Skill: https://skilld.dev/gh/nvidia/skills/digital-health-clinical-asr-finetune

This session only. Nothing lands on disk.

skill-card.md

≈943 tokens on demand. Your agent reads this file only when SKILL.md points to it.

Description: <br>

Stage 4 of the Clinical ASR Flywheel — runs stock NeMo SFT on Parakeet TDT v2 when priority KER is above 0.3 and performs offline cycle N+1 re-eval to measure that the loop closed. <br>

This skill is ready for commercial/non-commercial use. <br>

Owner

NVIDIA <br>

License/Terms of Use: <br>

Apache 2.0 <br>

Use Case: <br>

Developers and engineers fine-tuning NVIDIA Parakeet ASR models on clinical vocabulary to reduce keyword error rate for healthcare speech recognition workflows. <br>

Deployment Geography for Use: <br>

Global <br>

Known Risks and Mitigations: <br>

Risk: Review before execution as proposals could introduce incorrect or misleading guidance into skills. <br> Mitigation: Review and scan skill before deployment. <br>

Reference(s): <br>

Skill Output: <br>

Output Type(s): [Shell commands, Configuration instructions, Files] <br> Output Format: [Markdown with inline bash code blocks] <br> Output Parameters: [1D] <br> Other Properties Related to Output: [Produces a .nemo model file, training_run_info.json, offline_hyps.jsonl, and cycle leaderboard markdown] <br>

Evaluation Agents Used: <br>

  • Claude Code (claude-code) <br>
  • Codex (codex) <br>

Evaluation Tasks: <br>

Evaluated against 3 internal evaluation tasks (all positive skill-activation cases, 2 attempts per task, 50% pass threshold). <br>

Evaluation Metrics Used: <br>

Reported benchmark dimensions: <br>

  • Security: Checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access. <br>
  • Correctness: Checks whether the agent follows the expected workflow and produces the correct final output. <br>
  • Discoverability: Checks whether the agent loads the skill when relevant and avoids using it when irrelevant. <br>
  • Effectiveness: Checks whether the agent performs measurably better with the skill than without it. <br>
  • Efficiency: Checks whether the agent uses fewer tokens and avoids redundant work. <br>

Underlying evaluation signals used in this run: <br>

  • skill_execution: Verifies that the agent loaded the expected skill and workflow. <br>
  • skill_efficiency: Checks routing quality, decoy avoidance, and redundant tool usage. <br>
  • accuracy: Grades final-answer correctness against the reference answer. <br>
  • goal_accuracy: Checks whether the overall user task completed successfully. <br>
  • behavior_check: Verifies expected behavior steps, including safety expectations. <br>
  • token_efficiency: Compares token usage with and without the skill. <br>

Evaluation Results: <br>

Dimension Num claude-code codex
Security 6 100% (+44%) 89% (+28%)
Correctness 6 90% (+2%) 97% (+29%)
Discoverability 6 56% (+7%) 65% (+24%)
Effectiveness 6 97% (+18%) 94% (+35%)
Efficiency 6 47% (+14%) 48% (+6%)

Skill Version(s): <br>

1.0.0 (source: frontmatter) <br>

Ethical Considerations: <br>

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal team to ensure this skill meets requirements for the relevant industry and use case and addresses unforeseen product misuse. <br>

(For Release on NVIDIA Platforms Only) <br> Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns here. <br>

Source: SKILL.md on GitHub

Third-party checks

1 warning1mo3 checks · Risk SAFE
  • Gen Agent Trust Hub1mo

    The skill is safe and follows best practices for clinical ASR fine-tuning using NVIDIA NeMo and Brev cloud services. It provides detailed cost warnings and security recommendations for tool installation. A low-severity risk of indirect prompt injection exists because the skill processes external manifest data which could contain maliciously crafted text.

  • Socket1mo

    No alerts

  • Snyk1mo

    Risk: MEDIUM · 3 issues

Provenance

Signed by skilld at 1ba3403. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated 4 months ago

Capability

version
1.0.0
author
Ben Randoing <brandoing@nvidia.com>
tools
[
  "Read",
  "Write",
  "Bash",
  "Skill"
]
Other metadata
tags
[
  "clinical-asr",
  "finetune",
  "sft",
  "nemo",
  "parakeet",
  "flywheel"
]
compatibility
Requires a CUDA host (24 GB VRAM comfortable, 16 GB workable with batch_size=4), the NeMo container (nvcr.io/nvidia/nemo:25.11.01), and the finetune-asr + riva-asr-custom skills installed alongside this one. No local GPU? Use Brev. NVIDIA_API_KEY required for the offline cycle N+1 eval round-trip and for any NIM deploy.
metadata
{
  "author": "Ben Randoing <brandoing@nvidia.com>",
  "tags": [
    "clinical-asr",
    "flywheel",
    "finetune",
    "nemo-sft",
    "parakeet"
  ],
  "team": "healthcare-tme",
  "domain": "ai-ml",
  "stage": 4,
  "previous_skill": "digital-health-clinical-asr-eval",
  "next_skill": "riva-asr-custom"
}

README badge

README badge for nvidia/skills/digital-health-clinical-asr-finetune