# Model roles leaderboard (2026-11-02) Which model to use for which job, ranked from outcomes in our own agent harnesses, not from public benchmarks. Harnesses read this list instead of hard-coding model names, and their results feed back into it. | Role | What it judges | Ranked models | |---|---|---| | taste | novelty, priority, whether the author would care, whether it is worth filing | anthropic/claude-opus-5.5, anthropic/claude-fable-5.1 | | review | adversarial review: finding the false step | anthropic/claude-opus-5.5, openai/gpt-6-astra, anthropic/claude-fable-5.1, openai/gpt-6.1-sol | | mechanical | rebuild, reproduce, run controls | openai/gpt-6.1-sol, deepseek/deepseek-v4.1-flash-1017 | | builder | long multi-step build work | anthropic/claude-opus-5.5, openai/gpt-6.1-sol | | cheap | bounded work behind deterministic tests | deepseek/deepseek-v4.1-flash-1017, z-ai/glm-5.3-flash | | biology | computational biology review, modeling and literature | moonshotai/kimi-k3.1-pro | | batch | unattended spec-driven jobs, judged on delivering the required report | openai/gpt-6.1-sol, deepseek/deepseek-v4.1-flash-1017 | | writing | text sent in the owner's name, judged by whether he sent it as drafted | anthropic/claude-opus-5.5 | ## What each model is for | Model | Use it for | Ranked for | |---|---|---| | anthropic/claude-fable-5.1 | escalation, root-causing | taste, review | | anthropic/claude-opus-5.5 | daily driver, anything long | taste, review, builder, writing | | deepseek/deepseek-v4.1-flash-1017 | cheap tier behind tests | mechanical, cheap, batch | | moonshotai/kimi-k3.1-pro | biology advisor; targeted critique | biology | | openai/gpt-6-astra | hard reviews | review | | openai/gpt-6.1-sol | Codex side, debugging you can check | review, mechanical, builder, batch | | z-ai/glm-5.3-flash | cheap tier behind tests | cheap | | anthropic/claude-haiku-5.5 | untested cheap candidate | - | | anthropic/claude-opus-4.6 | fallback on the Claude side | - | | anthropic/claude-sonnet-5.5 | sub-agents and read-heavy work | - | | google/gemini-4-argon | tried, below Opus 5.5 | - | | openai/gpt-5.6-sol | fallback on the Codex side | - | | openai/gpt-6-luna | untested cheap candidate | - | | openai/gpt-6.1-terra | untested cheap candidate | - | | x-ai/grok-4.6 | fast implementation | - | Escalate at fixed checkpoints, never on the cheap model's own say-so: cheap executor to the mid tier when a deterministic check fails twice or the task needs a causal diagnosis; mid tier to a top reviewer for a pre-merge review of anything safety-relevant, a disputed number, or a go/no-go call. Anything going out in the owner's name, and interactive orchestration, stays on the top builder. ## Price, dearest first Sticker prices from the vendors' own pricing pages, sorted by a blended rate of 3 parts input to 1 part output, which is roughly how an agent session reads. **Sticker is not cost per task**, and the last two columns are what we actually measured: a model that writes more output, or re-reads a long context every turn, can cost several times another at the same sticker. | Model | Input | Output | Cache read | Blended | $/agentic task | $/interpreter task | Checked | |---|---:|---:|---:|---:|---:|---:|---| | anthropic/claude-fable-5.1 | $10 | $50 | $0.25 | $20 | $0.58 | $0.197 | Nov 2 | | openai/gpt-6-astra | $10 | $50 | $1 | $20 | $0.53 | $0.116 | Nov 2 | | anthropic/claude-opus-4.6 | $5 | $25 | $0.50 | $10 | $0.79 | $0.144 | Oct 25 | | anthropic/claude-opus-5.5 | $4 | $20 | $0.20 | $8 | $0.27 | $0.073 | Nov 2 | | openai/gpt-5.6-sol | $4 | $20 | $0.40 | $8 | $0.28 | $0.064 | Oct 25 | | anthropic/claude-sonnet-5.5 | $2 | $10 | $0.10 | $4 | $0.1 | $0.039 | Nov 2 | | openai/gpt-6.1-sol | $2 | $10 | $0.10 | $4 | $0.09 | $0.024 | Nov 2 | | moonshotai/kimi-k3.1-pro | $2 | $10 | $0.20 | $4 | $0.1 | $0.081 | Nov 2 | | google/gemini-4-argon | $2 | $10 | $0.10 | $4 | $0.31 | $0.028 | Nov 2 | | x-ai/grok-4.6 | $2 | $6 | $0.50 | $3 | $0.15 | $0.059 | Oct 25 | | openai/gpt-6.1-terra | $1 | $5 | $0.05 | $2 | $0.058 | $0.014 | Oct 27 | | deepseek/deepseek-v4.1-flash-1017 | $0.15 | $0.60 | $0.003 | $0.26 | $0.016 | $0.009 | Oct 25 | | z-ai/glm-5.3-flash | $0.15 | $0.50 | $0.03 | $0.24 | $0.025 | $0.0013 | Oct 26 | | anthropic/claude-haiku-5.5 | $0.10 | $0.50 | $0.01 | $0.20 | - | $0.003 | Oct 25 | | openai/gpt-6-luna | $0.10 | $0.50 | $0.01 | $0.20 | $0.007 | $0.002 | Oct 25 | Cost per task is the mean of two runs per model on the same two tasks, measured here on 2026-10-28: an agentic job (fix a failing test suite and add a feature in a real repository, graded on 6 visible and 16 hidden tests) and an interpreter job (one call, up to 3 repair rounds, 85 hidden cases). Effort is each model's default. The surprises are all in the gap between sticker and task. Sticker ratios are not cost ratios: Gemini 4 Argon has GPT-6.1 Sol's sticker price and cost about 3.5x Sol per agentic task, because it wrote more output and cached badly; Grok 4.6's output price is below Sol's and it still cost about 1.7x per task, on 8-10K of mostly reasoning tokens. "Sol eats tokens" turned out to be about the effort setting, not the model: at default effort it was the lightest frontier model on both tasks, and at high effort it still cost less per agentic task than Opus 5.5. Opus 5.5 was mid-pack on output tokens, not the heaviest. And the token-hungry cheap tier stays cheap: DeepSeek wrote the most output of any model and was still the second-cheapest per task, though cheap models beat the mid tier per answer by 1.3-11x, not by the 7-30x their per-token prices suggest. Agentic dollars are list-price estimates from measured token counts, within about 1% of the billed total; interpreter dollars are API-reported. On review: Opus 5.5 and Astra review every round. Fable 5.1 runs once, on the final bytes, because it is the costliest reviewer by a wide margin: one fan-out across several documents can eat a large share of a weekly subscription allowance within an hour. Its rank reflects cost, not quality. Want to try: google/gemini-4-carbon enters the review and taste candidates on the day it is on OpenRouter or the Gemini API. Status: announced 2026-10-28, cyber-defender access only, reported $2/$10; absent from Google's pricing page and from the OpenRouter catalog when both were checked on 2026-11-02. Want to try: deep-principle/hydrogen-1.0 enters the biology and review candidates on the day it has a hosted route, or runs on one of our own pods. Status: open weights (MIT) on Hugging Face as deep-principle/Hydrogen-1.0, GLM-5.3 post-trained on lab records and discovery trajectories; 753B parameters in BF16, with MXFP4 and NVFP4 quantizations, so self-hosting means roughly 380 GB of VRAM in 4-bit (estimate) and a multi-GPU pod. No OpenRouter route and no hosted price when checked 2026-11-02. Its 38.3% FrontierScience-Research against Astra's 33.3% and 93.0% GPQA-Diamond are VENDOR-REPORTED and count for nothing here until our own matched tasks run. Scope of biology: Computational analysis, modeling and literature only. No wet-lab protocols, and nothing that modifies, enhances or cultures an organism. Excluded from taste: openai/gpt-6.1-sol (owner's call: weak taste, until evidence says otherwise). ## Evidence so far | Role | Model | Resolved outcomes | Held % | Wilson low (95%) | |---|---|---:|---:|---:| | taste | anthropic/claude-opus-5.5 | 2 | 100% | 0.34 | | taste | deepseek/deepseek-v4.1-flash-1017 | 1 | 0% | 0.00 | | taste | openai/gpt-6.1-sol | 1 | 0% | 0.00 | | review | openai/gpt-6-astra | 1 | 100% | 0.21 | | mechanical | deepseek/deepseek-v4.1-flash-1017 | 3 | 100% | 0.44 | | mechanical | openai/gpt-6.1-sol | 1 | 100% | 0.21 | | builder | openai/gpt-6.1-sol | 2 | 100% | 0.34 | | builder | deepseek/deepseek-v4.1-flash-1017 | 1 | 0% | 0.00 | | builder | anthropic/claude-opus-5.5 | 1 | 100% | 0.21 | | batch | openai/gpt-6.1-sol | 77 | 75% | 0.65 | | writing | anthropic/claude-opus-5.5 | 2 | 100% | 0.34 | ## Method - Outcomes come from the harnesses: an idea-triage call stands or is reversed later; a blind review's PASS or FLAG is checked against whether the document later had to change; an upstream pull request is merged or closed; an expert's answer agrees or disagrees with the model's guess. - A model joins a role's ranking only with at least 5 resolved outcomes over 2 different days. It moves above another model only when its 95% Wilson lower bound beats that model's rate. Seeded models count as 75% until they have enough evidence. - The starting order is the owner's own ranking from daily use. Evidence moves it; one lucky run can't. - Exclusions are the owner's calls and are listed with their reason. Evidence can lift one only after 20+ outcomes, and that change is flagged, never silent. - Limits: small samples, selected tasks, one person's harnesses. This is a routing table, not a benchmark. - What stays unknown: end-to-end charged cost per accepted task, since an API sticker is not an invoice and neither is subscription spend; which provider served a call on a multi-provider route without a response receipt; and a complete task denominator, which is why these are outcome counts and not win rates. The cost-per-task figures are two samples per model on two small tasks, run through one harness. Every model passed both, so they separate cost, not quality.