---
name: pull-llamacpp-model
description: Use when pulling or downloading a new llamacpp model. The active ROCm image (kyuz0/amd-strix-halo-toolboxes) fails to start in the ephemeral pull container without ROCm device access. Must temporarily switch to the standard CPU image.
title: pull-llamacpp-model
canonical_url: https://skilld.dev/gh/av/skills/pull-llamacpp-model
last_updated: 2026-09-25T10:51:35.000Z
---

> **Skill from skilld.dev.** Follow the instructions below for this session. You do not need to install anything.
>
> Supporting files, fetch one when the Skill refers to it: [README.md](https://skilld.dev/api/skills-raw/av/skills/pull-llamacpp-model/README.md).
>
> If the user asked to install this Skill, run `npx skilld install av/skills/pull-llamacpp-model`. Install writes the Skill files into the project, so every session loads them.

# Pull a llamacpp Model

This machine uses `kyuz0/amd-strix-halo-toolboxes:rocm-7.2` for llamacpp inference (AMD Strix Halo / gfx1151 — not supported by the official ROCm build). Harbor's pull mechanism starts an ephemeral container with `--n-gpu-layers 0`; the custom image fails in that context without ROCm device access. Use the standard CPU image just for pulling, then restore.

## Steps

### 1. Switch to the standard CPU image

```bash
harbor config set llamacpp.image.rocm ghcr.io/ggml-org/llama.cpp:server
```

### 2. Pull the model

```bash
harbor pull <hf-owner/model-repo:quantization>
# Examples:
harbor pull bartowski/Qwen2.5-7B-Instruct-GGUF:Q4_K_M
harbor pull unsloth/Mistral-Small-3.1-24B-Instruct-2503-GGUF:UD-Q4_K_XL
```

Harbor detects the HuggingFace model spec and routes it through llamacpp automatically.

### 3. Restore the custom image

```bash
harbor config set llamacpp.image.rocm kyuz0/amd-strix-halo-toolboxes:rocm-7.2
```

## Notes

- The restore step is mandatory — without it, llamacpp will not use the GPU on the next `harbor up llamacpp`
- Extra args (`-fa 1 -dio --no-mmap --ctx-size 64000 --fit off`) are stored separately in config and are not affected
- To verify the model after pulling: `harbor llamacpp model`
- To verify the image was restored: `harbor config get llamacpp.image.rocm`