Index of every page on this site: llms.txt
All skills
huggingface avatar

/hf-mem

@5291c2b official
by Hugging Facehuggingface/skills11k stars
797
Run it yourself

Estimate the GPU memory needed to run a Hugging Face model (weights, KV cache and inference overhead) and whether it fits on given GPUs, without downloading it. Use when asked how much VRAM a model needs, whether it fits on a GPU, the longest context that fits, or which GGUF quantization to pick.

Use this Skill: https://skilld.dev/gh/huggingface/skills/hf-mem

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

SKILL.md

≈76 tokens for metadata: the name and description. ≈389 when used: this file.

Description uses 3.8% of example budget

Before choosing a Skill, your Agent reads its name and description. All available Skills share that space.

  • A shorter description leaves more room for other Skills. This entry exceeds our 1% size suggestion.

In our Claude Code example, all Skill names and descriptions share 8,000 characters. This Skill uses ≈306 characters, or 3.8%.

The 1% threshold is a size suggestion. Longer descriptions can still fit.

Your model, settings, and other Skills decide how much text your Agent can read.

Example settings and source

The example uses a 200k-token context and default Claude Code settings. The count includes the name, description, separators, and when_to_use when present. Codex also counts local file paths.

Skit's source and limits: Codex 0.160.1, Claude Code 2.1.292.

hf-mem

uvx hf-mem <model> -o json [options]

<model> is a Hub ID (org/name, org/name@revision), a Hub URL, a local path, or org/name-GGUF:Q4_K_M for one GGUF quantization.

Options:

  • --gpu-memory 24 (GiB per GPU) or 8x80 (8 GPUs): adds fit.
  • --max-model-len N: context length, defaults to the model's maximum, which can dominate the total.
  • --batch-size N: concurrent sequences, defaults to 1.
  • --kv-cache-dtype fp8 (vLLM names for Safetensors, llama.cpp cache types such as q8_0 for GGUF).
  • --file <file or variant> and --format safetensors|gguf|onnx to pick the weights.
  • --max-num-batched-tokens 2048 on A100s.
  • Gated or private models need HF_TOKEN or --hf-token.

JSON (sizes in bytes, divide by 2^30 for GiB):

  • total is the GPU memory for one copy of the model: weights + kv_cache + overhead (vLLM for Safetensors, llama.cpp for GGUF, none for ONNX).
  • offloaded are weights the engine keeps in CPU memory: excluded from weights and total, but needed as free system RAM.
  • fit (with --gpu-memory): fits, needed GPUs, max_model_len (the longest context that fits), and the required vs usable memory across the given GPUs. Use required / GPUs for the memory per GPU, not total / GPUs, as each GPU also holds its own overhead and replicated KV cache.
  • GGUF and ONNX repositories with several files return a files list to compare quantizations; pick one with :TAG or --file to get fit.
  • Report warnings to the user, and errors explain how to fix them.

Source: SKILL.md on GitHub

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

1 warning2d3 checks · Risk SAFE
  • Gen Agent Trust Hub3d

    This skill provides a utility to estimate the GPU memory required to run Hugging Face models. It leverages a specialized tool to calculate memory for weights and cache without requiring a full model download, which is a standard developer workflow.

  • Socket2d

    No alerts

  • Snyk3d

    Risk: MEDIUM · 1 issue

Signed by skilld at 5291c2b. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 3 days ago.

Activeupdated 4 days ago
  • huggingface
  • memory-estimation
  • safetensors
  • gguf
  • inference
  • vram
  • llm
  • model-loading

README badge

README badge for huggingface/skills/hf-mem

Estimates GPU/CPU memory requirements for Safetensors and GGUF models on the Hugging Face Hub without downloading weights, using HTTP Range requests. Optionally includes KV cache calculations for LLMs and VLMs with configurable context window and precision settings.

Generated from the current SKILL.md.

Does this download or load model weights locally?
No. hf-mem uses HTTP Range requests to estimate memory requirements without downloading or loading any weights locally.
Does this support GGUF models?
Yes. hf-mem supports both Safetensors and GGUF formats. For GGUF repos with multiple quantizations, you can specify a particular file with --gguf-file.
Can this estimate KV cache memory for language models?
Yes, with the --experimental flag. It will calculate KV cache requirements for LLMs and VLMs, with customizable context window (--max-model-len) and KV cache precision (--kv-cache-dtype).
What authentication is required?
An HF_TOKEN env var or --hf-token flag is needed only for gated or private models. Public models require no authentication.
What models does this work with?
hf-mem supports Transformers, Diffusers, Sentence Transformers, and other models on the Hugging Face Hub with Safetensors or GGUF weights.

Generated from the current SKILL.md. These answers refresh after source changes.