The model inventory and the calibration behind the profile table
Read this file when calibrating the profile table or adding a model, never when answering a request: SKILL.md answers
routing requests on its own, and nothing here changes an answer.
Inventory
Each row is a model at one calibrated reasoning effort. The rows record what routing has calibrated so far, not the
efforts a model can run: a user may explicitly request an effort no row lists (see Routing model in SKILL.md). The
session's harness and the family determine whether a route is native or external (see Launch mechanism in SKILL.md);
sota marks models trusted with critical review and design decisions, not eligibility to orchestrate. Availability and
user overrides may remove or replace these defaults; see Local availability in SKILL.md.
Claude rows carry no version because they are harness aliases that follow each new release on their own. Their placements therefore move to a new release without anyone re-measuring them, the same as a GPT row whose id is bumped. Unlike a GPT bump, nothing records when that happens, so the evidence below does not say which Claude release it was measured on.
| model | family | sota |
|---|---|---|
| gpt-6-luna medium | gpt | |
| gpt-6-luna high | gpt | |
| gpt-6.2-sol low | gpt | |
| gpt-6.2-sol medium | gpt | |
| gpt-6.2-sol high | gpt | |
| gpt-6.2-astra high | gpt | yes |
| haiku high | claude | |
| sonnet low | claude | |
| sonnet medium | claude | |
| sonnet high | claude | |
| opus high | claude | |
| fable high | claude | yes |
| muse-spark-1.3-contributor low | muse | |
| muse-spark-1.3-contributor medium | muse | |
| muse-spark-1.3-contributor high | muse | |
| muse-spark-1.3-contributor xhigh | muse | |
| glm-5.3-flash low | glm | |
| glm-5.3-flash high | glm | |
| glm-5.3-flash max | glm |
Why the profile table looks the way it does
These assignments are defaults, not claims that every task in a profile is equivalent. Test quality is critical because weak tests are how correctness defects survive review. Reviews route above similarly sized implementation work because the reviewer is the gate: a missed finding may have no later backstop. Some profiles intentionally share routes today; keeping their semantics separate lets later calibration change one without conflating different failure costs. A second cross-family SOTA perspective may be worth its overhead for high-risk critical review. Orchestration is not a delegation profile; planning, decomposition, quality gating, and VCS ownership remain with the orchestrating SOTA session.
The mid tier, and what the eval behind the luna default measured
Within implementation work, sol is the escalation and long-context tier rather than a replacement for luna's workhorse role. Work that has to be reasoned across more input than the orchestrator can afford to spend its own context on (a change threaded through dozens of files, a diagnosis that means reading a large subsystem) starts at sol medium, because luna's long-context retrieval is weaker. Difficult debugging and ambiguity that survives decomposition also use sol, with high as the escalation rung when the delegate's own judgment matters mid-task. For mechanical and clear-spec work, sol medium is the terminal fallback after a Luna-high failure; sol high requires explicit reclassification as complex implementation or a deliberate new route. The eval behind these routes showed something narrower than "the mid tier is useless": in 36 runs every model passed every hidden test, so the tasks separated prices, not failure rates, and the expensive models' visible advantages were soft — documentation quality and benchmark discipline — which a reviewed assumptions list and a gate that reads the diff cover. The honest conclusion is that nothing there justified paying mid-tier prices for well-specified work, not that no task ever will. An orchestrator reaching for sol or opus to implement something well-specified usually has an unsettled design, and the fix is to settle it; the routine authored and mechanical review rows are unchanged by this reasoning — taste-dependent prose and review were not what the eval measured.
Luna is the workhorse on purpose, not as a compromise. Across six treeward features and a planted bug, every model from
luna medium up to sonnet passed every hidden test on the first attempt; the gate rejected three results for hidden
work-done regressions, none of them luna's, making luna medium the cheapest clean record in that eval. The current
Luna-high default is an explicit economic hypothesis: its incremental cost may be offset by fewer escalations, but no
direct medium-versus-high workhorse comparison has established that yet. Luna has two demonstrated weaknesses. Judgment
on open questions is the first, and the caller's checkpoint protocol moves that judgment to the orchestrator before any
code exists: in the guidance eval, the checkpoint arm went 8 for 8 across the four cheap models — luna medium and high
among them — on a feature the same models had gotten right once in eight runs without it. It does not move
mid-implementation judgment anywhere, which is why the caller's gate still reads the diff in full. The second weakness
is long context, covered by the exception in SKILL.md. What the workhorse needs is a clear spec and a reviewed
assumptions list, and those are the orchestrator's to supply. Expect luna's decision log to be short or empty — it does
not experience decisions as decisions — and a short or empty log is not evidence that the work was simple or that
nothing was decided.
Evidence behind the rules in SKILL.md
SKILL.md states the rules; this is what each rests on, so that calibration has one place to update.
- The GPT
sotamark. gpt-6.2-astra high carries the family'ssotamark; gpt-6.2-sol high does not. The mark follows the model trusted to orchestrate, not a measurement of sol against astra. The decision to mark astra as the flagship was made on 2026-09-06, before these sol and luna version changes. Their routes retain the earlier placements, but the measurements below have not been repeated on the new versions. Astra has no placement below critical review because nothing has calibrated it as a delegate; give it one when real use says so. - The opt-in families. The muse placements rest on vendor-reported benchmarks rather than calibrated use. The glm
family's model ran as the
ox-alphastealth preview on OpenRouter and OpenCode; its evidence base is one clean clear-spec smoke run plus vendor benchmarks, and its per-token price is roughly an order of magnitude below the other families. - The visual carve-out. Real-world feedback consistently rates sol below the Claude models on visual design taste even while its coding reputation holds up.
- The workhorse-writer default. The cross-family cost the native-path bias exists to weigh is small and
characterized for writers (measured through
codex exec; an external GPT route on another CLI has not been measured): the shell-out path to codex (launch, resume, and monitoring) is exercised end to end (two items there remain explicitly unverified), and the price gap to the same-family writer alternative is about 30× for no measured difference in first-attempt correctness (the eval above). The default rests on one eval in one small repository; if a follow-up on a larger, messier codebase finds luna's literalism surviving the caller's checkpoint protocol, the workhorse-writer paragraph inSKILL.mdis what to revisit.