AMR Score Methodology

What the AMR score measures

AMR measures general capability, not a specific job. It blends reasoning, knowledge, coding and math results with human preference voting, so the score is useful for a first-pass shortlist and close to useless for deciding whether a model fits one narrow workflow.

No model vendor publishes an AMR score. We build one so models from a dozen labs can sit in a single sortable table. A GPQA result from Anthropic and a SWE-Bench result from DeepSeek do not otherwise line up, and choosing between them by reading two spec sheets is guesswork dressed as analysis.

The five benchmarks behind it, and the arena

Five written benchmarks supply the capability signal: GPQA Diamond, MMLU-Pro, SWE-Bench Verified, AIME 2025 and LiveCodeBench v6. Community arena Elo supplies the human signal. Prices and context windows are tracked next to these but never enter the score, so a cheaper model cannot rank higher for being cheap.

Arena Elo is the one input that is not a percentage. It is mapped onto the same 0 to 100 range before it enters the composite, which keeps a popular model from outranking one that fails the written benchmarks.

How the composite is calculated

Each input is put on a common 0 to 100 scale and then combined into one weighted score. The weights are editorial, held fixed across the whole board, and only revisited when the entire leaderboard is recalibrated rather than tuned model by model.

We do not publish a coefficient list you can re-run, and that is deliberate. Published weights invite labs to optimize the index instead of the capability it stands for. What we publish instead is every input for every model, on its detail page and in the compare tool. Disagree with our weighting? The raw numbers let you apply your own.

On October 16, 2026 the whole board was calibrated against an independent published composite: 20 models carry directly comparable values and the remaining models are mapped onto the same scale. That work is what makes an 86.5 stable. Without it each new release would drift the scale.

Where each number comes from

Three source types feed the board: prices and context windows from OpenRouter's models API, benchmark results from what labs publish, and arena Elo from blind pairwise human voting. Where sources disagree, the table shows the value that a live API call or the original publication supports.

Prices and context windows

Synced from OpenRouter's public models API. When a vendor rate card and OpenRouter disagree, the OpenRouter figure is the one shown, because that is what your API bill will match.

Pricing policy

OpenRouter lists several endpoints per model, and most of them are service tiers of the same provider rather than competing vendors: one model can quote at 1x, 2x, 4x and 12x depending on capacity and priority. Taking the lowest would advertise a tier almost nobody runs.

So each price is the vendor’s own rate card: the endpoints served by the model’s maker, taking the lower median when that maker publishes several tiers. The chosen figure is always an actual published endpoint, never an average of tiers. Where the vendor does not serve the model on OpenRouter, the most common price across endpoints is used, and the provider is recorded either way.

Each model page names the provider and the date under its input price. The value ranking counts only models whose price carries that record, because a cheapest-per-point claim has to rest on numbers a reader can check. Models without a verified price stay on the main board and off the value track.

How the value ranking is calculated

Value is AMR points per blended dollar, with blended price weighted 25% input and 75% output. It is one assumption, not a law: a retrieval-heavy workload is mostly input tokens and a reasoning agent is mostly output, so treat the ordering as a starting point and re-weight with the numbers on each model page.

Benchmark scores

Taken from what the labs publish, plus independent evaluation runs where they exist. A vendor-published figure is recorded as published. We do not substitute our own numbers for it unless we can reproduce the run.

How each score is labelled

Every benchmark value carries a provenance label once its source is recorded. An official leaderboard submission means the benchmark’s own leaderboard accepted it with run logs. An independent re-run means a third party executed the harness. Vendor-reported means the lab published the number and nobody else has reproduced it yet. Where a value has no label, the source has not been recorded, and it should be read as vendor-reported at best.

Benchmarks that come in variants get extra care. DeepSWE, SWE-bench Multimodal, SWE-bench-Live and lab-internal suites are different measurements from SWE-Bench Verified, and a number from one of them is never placed in the SWE-Bench column. When a harness is disclosed (mini-SWE-agent, a vendor agent framework, an undisclosed scaffold), it is listed next to the score, because the same model can move several points between harnesses.

Arena Elo

Blind pairwise voting from a public chat arena, where voters pick between two anonymous responses. Elo moves slowly and reflects preference as much as raw capability, which is why it is one input rather than the score itself.

Update cadence and snapshot dates

Benchmarks, prices and rankings are re-checked weekly. The snapshot date sits next to every table heading and on every share card, so a screenshot cannot be mistaken for today's number. Usage rankings come from a separate weekly OpenRouter snapshot and carry their own date.

If a value changes, either a new benchmark run landed or the scale was recalibrated. Rows are not re-dated just because a model was re-listed. Scores are directly comparable within one snapshot; treat comparisons across snapshots with care, because a recalibration can move every model at once.

Limits: what the AMR score is not

AMR is one number with known gaps. No model vendor publishes it, it does not tell you which model wins your own task, and a recalibration can shift every score at once. Read it as a shortlist tool, not a verdict.

Not a vendor score

No lab publishes AMR. It is this site's index, built from public results.

Not task-specific

A model ranked lower overall can still win your job. The coding, speed, value and context tracks rank on a single axis for exactly this reason.

Not immune to self-reporting

Benchmarks a vendor publishes count as published until someone reproduces them.

Not permanent

Recalibration moves the scale, not just one model. Compare within a snapshot rather than across months.

Pending models carry a provisional score

A model with no published benchmarks yet stays on the main board at a provisional AMR and is held out of track rankings until its runs land.