Last verified: 2026-07-13 — refresh when a new model-size anchor is validated on Spark or the DGX Spark playbooks change quantization defaults.
UMA Memory Accounting
A worksheet for estimating whether a model/method/batch combination
fits DGX Spark's 128GB unified pool before a run, and for sanity
checking a plan against known-working anchors. This is planning
math, not a guarantee — always leave headroom rather than sizing to
the byte; see the OOM Ladder in SKILL.md for what to do when the
estimate turns out optimistic.
The Four Terms
Total footprint ≈ weights + optimizer states + gradients + activations, plus a near-zero term for LoRA/QLoRA adapters. Work each term from parameter count and dtype, then sum.
1. Weights
params × bytes/param, by dtype:
| dtype | bytes/param |
|---|---|
| fp32 | 4 |
| bf16 / fp16 | 2 |
| int8 | 1 |
| int4 (QLoRA NF4) | 0.5 |
A 70B model in bf16 is ~140GB — already over the pool before anything else loads. The same model in 4-bit (QLoRA) is ~35GB, which is why QLoRA, not bf16, is what makes 70B-class models reachable on Spark at all.
2. Optimizer states
Full fine-tuning carries optimizer state for every trainable parameter; LoRA and QLoRA carry it only for the adapter parameters, which is why this term is negligible for them regardless of base model size.
| Optimizer | bytes/param (trainable only) |
|---|---|
| AdamW, fp32 states | 8 (4B momentum + 4B variance) |
| AdamW 8-bit (bitsandbytes) | ≈2 (quantized momentum + variance) |
adamw_8bit is the Unsloth default for a reason on a 128GB
box — the fp32 variant roughly quadruples this term for any run
that isn't LoRA/QLoRA-adapter-only.
3. Gradients
Same dtype as the compute precision — typically bf16, so 2 bytes/param — and, like optimizer state, only for trainable parameters. Full fine-tuning pays this for every weight; LoRA and QLoRA pay it only for the adapter, since the frozen base weights never accumulate a gradient.
4. Activations
The hardest term to pin to a single number — it scales with batch size, sequence/packing length, and architecture (attention variant, hidden size, layer count), not just parameter count. Two levers matter more than exact estimation:
- Gradient checkpointing trades recompute for memory: expect
roughly 30% savings on this term versus no checkpointing, at
the cost of a recompute pass per checkpointed segment. Unsloth's
use_gradient_checkpointing="unsloth"is the default for this reason. - Packing/sequence length is the more direct lever than batch size
for this term — see the OOM Ladder in
SKILL.mdfor why packing length is the preferred first cut over batch size.
5. LoRA/QLoRA adapter overhead
A rank-r adapter on a linear layer adds r × (in + out)
parameters — A is r×in and B is out×r, so the two
matrices together contribute r·in + r·out. At the rank sizes in
normal use (1–32 for RL, up to ~256 for SFT-at-scale), this is a
small fraction of a percent of base model size — round it to zero
in the worksheet unless an unusually high rank is in play.
Anchors
Known-working combinations on a single Spark, to sanity check a new plan against rather than trusting the formula in isolation:
| Model class | Method | Observed total | Notes |
|---|---|---|---|
| 70B | QLoRA | ≈40GB | 30–48h for 3 epochs; the reference point for "70B fits via QLoRA, not bf16." |
| a ~120B-class MoE model | NVFP4-native LoRA | ≈68GB (per Unsloth's official DGX Spark tutorial, unsloth.ai docs, 2025-12) | Community recipe (nvfp4-lora-spark); experimental, not the default assumption for other 100B+ MoE models. |
| 27B | LoRA | fits at pack ≤1024 | The LoRA ceiling on a single Spark — larger dense models need multi-Spark or a smaller method. |
| 9B | Full fine-tune | fits comfortably | The full-FT ceiling — above this, full FT needs LoRA/QLoRA or multi-Spark instead. |
Treat "ceiling" entries as the largest class that fit in practice,
not a hard architectural limit — a smaller batch, shorter packing,
or a leaner optimizer can sometimes push slightly past one of these,
and a heavier configuration of the same model class can fail well
under it. Re-derive from the four terms above for anything outside
these four reference points, and confirm with the OOM Ladder in
SKILL.md if the estimate turns out optimistic.