Index of every page on this site: llms.txt
All skills
av avatar

/pull-llamacpp-model

@8f70341
by Ivan Charapanauav/skills28 stars
2
Run it yourself

Use when pulling or downloading a new llamacpp model. The active ROCm image (kyuz0/amd-strix-halo-toolboxes) fails to start in the ephemeral pull container without ROCm device access. Must temporarily switch to the standard CPU image.

Use this Skill: https://skilld.dev/gh/av/skills/pull-llamacpp-model

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

SKILL.md

≈64 tokens for metadata: the name and description. ≈376 when used: this file. ≈140 more on demand in 1 file.

Description uses 3.2% of example budget

Before choosing a Skill, your Agent reads its name and description. All available Skills share that space.

  • A shorter description leaves more room for other Skills. This entry exceeds our 1% size suggestion.

In our Claude Code example, all Skill names and descriptions share 8,000 characters. This Skill uses ≈256 characters, or 3.2%.

The 1% threshold is a size suggestion. Longer descriptions can still fit.

Your model, settings, and other Skills decide how much text your Agent can read.

Example settings and source

The example uses a 200k-token context and default Claude Code settings. The count includes the name, description, separators, and when_to_use when present. Codex also counts local file paths.

Skit's source and limits: Codex 0.160.1, Claude Code 2.1.292.

Pull a llamacpp Model

When using a custom ROCm image such as kyuz0/amd-strix-halo-toolboxes:rocm-7.2.4 for llamacpp inference on AMD Strix Halo / gfx1151, Harbor's pull mechanism starts an ephemeral container with --n-gpu-layers 0; the custom image fails in that context without ROCm device access. Use the standard CPU image just for pulling, then restore.

Steps

1. Save the current image and switch to the standard CPU image

original_rocm_image="$(harbor config get llamacpp.image.rocm)"
harbor config set llamacpp.image.rocm ghcr.io/ggml-org/llama.cpp:server

2. Pull the model

harbor pull <hf-owner/model-repo:quantization>
# Examples:
harbor pull bartowski/Qwen2.5-7B-Instruct-GGUF:Q4_K_M
harbor pull unsloth/Mistral-Small-3.1-24B-Instruct-2503-GGUF:UD-Q4_K_XL

Harbor detects the HuggingFace model spec and routes it through llamacpp automatically.

3. Restore the custom image

harbor config set llamacpp.image.rocm "$original_rocm_image"

Notes

  • Run these steps in the same shell so original_rocm_image survives. Restore it even if the pull fails.
  • The restore step is mandatory — without it, llamacpp will not use the GPU on the next harbor up llamacpp
  • Extra args (e.g. llama-server -fa 1 -dio --no-mmap -ngl 99 --no-mmproj) are stored separately in config and are not affected
  • To verify the model after pulling: harbor llamacpp model
  • To verify the image was restored: harbor config get llamacpp.image.rocm

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No alerts8d3 checks · Risk SAFE
  • Gen Agent Trust Hub9d

    The skill provides utility commands for managing llamacpp models using the harbor CLI, specifically addressing a configuration issue when pulling models in environments without ROCm device access.

  • Socket8d

    No alerts

  • Snyk9d

    Risk: LOW · No issues

Signed by skilld at 8f70341. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub last week.

Activeupdated last week

README badge

README badge for av/skills/pull-llamacpp-model