Methodology
One rule governs the whole index. This page explains exactly how each figure is built, so you can decide whether to trust it — here is what it currently covers:
Every number is traceable to a source URL and a date it was verified. Quality is measured, never asserted; prices are cross-checked across independent sources and arbitrated by majority; anything we can't source, we don't print.
Pricing: sources and majority arbitration
Each endpoint's headline price comes from a primary source — the provider's own pricing page or public model catalogue, or the OpenRouter catalogue where it aggregates that provider. When a provider publishes its own catalogue and OpenRouter resells it at a different price, the provider's own price is the headline and the OpenRouter price is shown beside it. For a few providers with no machine-readable price list (Alibaba Cloud, DigitalOcean, OpenAI), their list price is read from the open models.dev index, kept only while it stays consistent with the OpenRouter route, and labelled as such. That figure is then cross-checked against independent secondary sources (LiteLLM, models.dev). Prices are per 1M tokens in USD. "Blended" means input + output added together — a coarse aid for ranking, not a workload estimate; each model page lets you size your own mix.
We never invent agreement: a secondary source only enters the comparison if it actually lists that model. When sources are present, the price carries one of four confidence levels, arbitrated on a tolerance of 20% blended.
At least one independent source agrees within tolerance. The primary isn't alone.
No secondary source lists this model, so there's nothing to corroborate against.
Sources exist and none agree, but every gap is a clean ×2 or ÷2 ratio: a probable batch-vs-standard tier confusion, not real dispersion.
Sources disagree by a non-tier amount: genuine dispersion, flagged for review rather than hidden.
Every model page shows the per-provider price, the source link, the confidence tag and the date it was verified — so a "diverges" flag is visible to you, not swept away.
Freshness and staleness
Each price stores the date it was last checked against its source. A compact freshness signal sits under the site banner on every page, recomputed on each rebuild — currently 864 prices, 83% verified within seven days. Category and ranking tables carry a per-row "Verified" column with the relative age of each price. Two thresholds keep stale numbers honest — a doubtful number does more harm than a missing one.
noindex)Quality benchmarks
Quality is borrowed from primary leaderboards, each read on a single dated snapshot to avoid mixing noise from different sources. Scores attach to the model, not the provider — the same model scores the same wherever it's served, which is why the cheapest host of a given model is also its best value.
The human-preference score is LMArena (Chatbot Arena) Elo, read directly from the primary LMArena text leaderboard. The intelligence score is the Epoch Capabilities Index (ECI) by Epoch AI: one score per model, combined from dozens of benchmarks, with a confidence interval. Epoch publishes it under CC BY 4.0; we read their data file once a day and credit them on every page that shows a score. Higher is better, and the scale has no zero (about 115 to 170 in October 2026), so charts start just below the lowest score and the interval matters: two models whose intervals overlap are treated as the same level. We use only what Epoch computes itself, never the third-party results it also republishes. Model matches are curated by hand, by the exact API identifier Epoch measured; ambiguous versions are left without a score rather than guessed. Embeddings use the MTEB multilingual mean task score on one dated snapshot.
Speed and uptime: read at the source, never our own
Since 21 October 2026 we no longer run our own weekly speed benchmark. Speed figures, where a page shows them, are read from OpenRouter — the median output speed and the median time to first token of each provider over the last 30 minutes, with the time we read them. We read them once a day, with the prices: each figure is a snapshot of real traffic in the half hour before the read, not a daily average. The source sits next to every number. We never publish tail latency (p95), error rates or sample counts.
Uptime is OpenRouter's share of successful requests to a provider over the last 24 hours, read every day. A provider under 97% is flagged; one that OpenRouter does not route shows « — ».
Why the change: these numbers already exist, sourced and fresh, for far more providers than a weekly run from two machines could reach. The readings of our own runs, up to the last one on 20 October 2026, stay available as an archive in benchmark-readings.csv.
Derived metrics
The rankings combine the figures above by simple, stated arithmetic — no black box.
Because quality is identical across providers, the cheapest endpoint of a model is also its best value — which is exactly why top-of-leaderboard models are rarely top value. A model is labelled frontier when it is statistically tied with the best-scored model: the top of its Epoch interval reaches the bottom of the leader's.
The maker spec sheet
A model page carries two kinds of fact that used to sit in the same row, and shouldn't. One is the object: what the maker built and published about it. The other is the market for that object: who serves it, at what price, with what window, at what speed. The first only changes when the maker republishes weights; the second changes daily and differs on every row. So the object gets its own block — the spec sheet — and one rule governs it: a field comes from the maker, or it does not exist. Price, served context window and served maximum output are properties of an endpoint, not of a model, and they stay on the provider rows.
Weights when we have them, docs when we don't
Where weights are public, the sheet is read from the maker's own repository and pinned to a commit: the context window, parameter count, active MoE experts, weight precision and input modalities all come out of config.json or the safetensors index at a specific revision, so the link we print reopens exactly the file we read. Where there are no weights, the sheet is read from the maker's documentation instead, and says so — a doc page can be rewritten without trace, a commit cannot, and the sheet marks which of the two backs each number. The index currently holds 219 spec sheets, 120 of them pinned to a weights commit.
An empty cell is itself a finding
The sheet is printed even when it is nearly empty. Each field shows a state: published by the maker; not published by the maker; not applicable (parameter counts for an API-only model); access-restricted (the repository exists but is gated); or still to be read by us. Only the last of those is our debt, and it is labelled as ours rather than dressed up as the maker's silence. A "not published" is never a guess: it means we opened that file, at that commit, on that date, and the field was not there — so it carries a source link and a date exactly like a filled one. Across every sheet, 945 of 2190 fields are published by their maker and 867 are still ours to read. The starkest column is the knowledge cutoff: on 139 models the maker's own page states no date at all.
Edge cases the sheet handles
When the two registers disagree
The two registers can disagree, and when they do we print both rather than arbitrate. A maker may document a 500k context window while every endpoint serving that model advertises 256k; both statements are true and they measure different things — what the weights declare, and what this host serves. The sheet says the first, the provider row says the second, each with its source. We do not tell you which is right.
Maker lifecycle status
The Maker lifecycle status row is the same split, in its sharpest form. It reports what the maker says about its own model — generally available, deprecated, retired — in the maker's vocabulary and with the maker's definition. It is not the availability of any endpoint. Mistral has marked mistral-saba retired since 30 September 2025 and defines retired as "no longer accessible", with requests returning a 404; the catalogue still lists it as generally available, with prices, on a host that serves it today. Both statements are sourced and dated on this site, and we arbitrate neither. A status inferred from an endpoint that answers would be an observation about a host, not a statement by a maker, so it never enters this row.
The License row and its scope
The License row carries a permanent note about its own scope, for a reason worth stating here. Three different things share that word: the licence on the published weights, which is what we read; the maker's acceptable-use policy, which is a separate document and is sometimes stricter; and the contract you sign with whichever host you call, which we do not read at all. A model whose weights are released under a non-commercial licence can be served by a paid endpoint without anything being amiss — the host may hold a separate agreement with the maker, or the maker may be the host. Reading "non-commercial" on this page and concluding something about a provider listed on it would be a mistake our own labelling invited, so the row now says what it measures. The note is the same under every licence, permissive or not: it describes the field, not the model, and never the host.
Where the sheet appears
Where the sheet appears follows from what it describes. A base served by two or more priced endpoints has a model page, and the sheet lives there, once. A base served by a single endpoint has no model page — there is no price spread to compare — so its sheet is printed on that endpoint's own page instead. 0 of 219 bases are in that situation, and until 7 September 2026 their sheets were computed and published in the data files but appeared on no page at all. That gap was worst exactly where it mattered: both models a maker currently marks retired are single-endpoint ones.
Licenses, context and cutoffs
Open-weight vs. proprietary status, context-window size, maximum output and knowledge-cutoff dates each carry their own source link, and three things about those links are worth knowing. Licenses are read model by model from the LICENSE file in the maker's own weights repository where we have verified them, and fall back to the family's reference license elsewhere — the link tells you which. Context window and maximum output, in the provider tables, are what each host declares for the endpoint it serves — not what the maker published: hosts routinely serve less, and occasionally declare more. What the maker published is on the spec sheet above, separately and separately sourced. Knowledge cutoff is the one field where a maker figure often does not exist at all — most open-weight families publish none. Where the maker states a date we print it and link their documentation; where it doesn't, we print a third-party estimate, marked ≈ and labelled as an estimate wherever it appears. It is the only editorial field on this site, and it is never ranked alongside maker data without saying so. For open-weight models we also estimate self-host memory: VRAM ≈ parameters × bytes-per-weight (0.5 at int4, 1.0 at int8, 2.0 at fp16) × 1.2 overhead for KV cache and activations, which feeds the self-host break-even calculator.
What this index does not do
- It invents nothing. A figure without a source and a date does not get printed.
- It is an index, not a gateway. We don't route your traffic or resell tokens, so a provider's ranking is never for sale.
- Affiliate links are disclosed where present, and never change the order of any table; rankings are computed purely from the sourced numbers.
All data on the site is free to reuse under CC BY 4.0with attribution.