Benchmark-aware fallback

A chat stops at a limit when its model is out on every subscription and key it may use, and the fallbacks you set up had nothing either. With benchmark-aware fallback on, the chat continues instead on the best model that can run at that moment: first by your preference order, then by a benchmark score for the kind of work the chat does.

Where it sits

It is the last step, and only that. At a limit n8v tries, in this order:

1. The same model on another subscription you opted in for auto-switch (Quota).
2. Another model of the same provider, by your rule per model (Limits and fallback per model).
3. The model you fixed for the provider in When a provider runs out.
4. Benchmark-aware fallback: the best model that has room right now.

It never runs while a subscription or a key n8v may use still has room for the chat's own model. n8v knows a subscription is out from the limits it records for the Quota page and for auto-switch, so one of the two has to be on. A chat whose own model is not in the table is left alone. It never moves a chat with a pinned model, an organization's run, a workstation task or a task in an enterprise workspace.

How the model is chosen

Preference order first. The models you list are tried in your order, whatever their scores. The default is Fable 5.2, then Opus 5.5.

Then the table. Each model has three scores from 0 to 100. The chat's work decides which one counts:

The work is read from the chat's title and first request. When in doubt it is coding.

Only what can run. A model is skipped when its rollout flag is off for your account, when it has no room, when a rule excludes it, or when a turn could not start on your credentials. Effort carries over to the closest level the new model has.

The default table

Every default figure comes from LMArena's own published data, the leaderboard data set (CC BY 4.0), and each figure keeps its own source and date:

A model's figure is its best-rated entry on that board on that date, for example Claude Opus 5.5 (Max) on WebDev. Each column is rescaled to 0 to 100 across the models n8v runs: the highest rating is 100, the lowest is 0. So a 0 means the lowest of these twelve on that board, not a model that cannot do the work.

Fable 5.2 was not on the Text boards on 9 November. Its reasoning and writing figures carry Fable 5.1's until it is rated, and say so: their source begins with "Provisional".

In Settings the date of each figure is under its score, and the pencil opens the source of each: the board, the entry and its rating. Benchmarks move: correct any score, source or date, or reset the table to these defaults. A score you change without naming a source is marked "Set on this account" with the day you set it.

What it never does

• Spend a Founding Member's key. On a Founding seat only subscriptions count as room. Keys and built-in credits are passed over, also when they belong to the best-scoring model.
• Break a repository's rules. Standing instructions on the account, the repository and the chat are read before picking. "Stay on Claude", "Claude only", "never use Gemini", "avoid Opus" or "do not fall back to GPT-5.5" each exclude what they name, by provider, model or family. A repository named for iOS or Swift stays on Claude unless a rule allows other providers. If the rules or your seat cannot be read at that moment, nothing is picked.
• Trade a chat back and forth. After a switch, a chat is not moved again for two minutes.

When nothing is left to pick, the chat waits for its reset as before.

Default rules

n8v applies no rule you cannot see. One rule comes as a default, listed in Settings under Default rules:

Named ios, swift, swiftui, xcode stays on Claude

A repository whose name has one of the words as a part (acme/ios-app, swift-kit) is kept on that provider: nothing of another provider is picked for its chats. A word is a whole part of the name, so acme/studios is not matched.

The rule is yours: change its words or its provider, remove it, add up to eight of your own, or restore the default. With no default rules, only the standing rules you wrote decide. A written rule that allows other providers ("allow all providers", "allow Codex") comes before a default rule.

With the fallback per model

Limits and fallback per model decides first. Say Fable continues on Opus 5.5 there:

1. Fable runs out. The chat continues on Opus 5.5 by your rule per model. Benchmark-aware fallback is not asked.
2. Opus 5.5 runs out too, and nothing else of the earlier steps has room. Now benchmark-aware fallback picks, for example GPT-6 Astra for coding. The line names the model that just ran out: "Opus 5.5 High's quota out".
3. Home is still the chat's own model, Fable. At the start of each turn n8v checks Fable, and returns to it once it has room.

Two things follow from that. The chat does not step back onto Opus 5.5 on the way: if Opus 5.5 has room again before Fable does, the chat stays on the pick until Fable is back. And while a chat is on a pick, your rule per model is not applied to it; a limit on the pick is answered here, home first.

With the default preference order (Fable 5.2, then Opus 5.5) the two settings agree: for a chat on any other model that runs out, Fable and then Opus 5.5 are tried first, as long as they have room.

In the chat

One line says what happened and why, with the reset in your own time:

switched to GPT-6 Astra High: Opus 5.5 High's quota out until 14:30, next best for coding by benchmark

A Fallback badge beside the model repeats it. When the pick came from your preference order, the line ends "by preference".

Going home

At the start of every turn n8v checks whether the chat's own model has room again: on the active subscription, or on another one when auto-switch allows it. If it has, that turn runs on it:

switched back to Opus 5.5 High: preferred model has room again

Before the reset the provider named, only a fresh quota reading counts as room. A model you pick yourself for a message ends the fallback there.

Settings and the dry run

In Settings, Agent, under Benchmark-aware fallback:

• Pick next best model by benchmark switches it on or off.
• Preference order: add, remove and reorder. Each model shows whether it has room now.
• Benchmark table: edit the three scores; each shows its date, and the pencil opens the source and date of each figure.
• Default rules: the rules by repository name, to edit, remove, add or restore.
• Dry run: choose the model that runs out, optionally a repository whose rules apply, and see the pick for coding, research and writing from your real quota, keys and rules. Nothing is switched, and it works while the setting is off.

API and MCP

Reading and the dry run need the chats:read scope; changing the setting needs chats:message. A key limited to repositories is refused, because the setting covers the whole account.

A table entry takes coding, reasoning and writing (0 to 100), and refs with a source (up to 160 characters) and date (YYYY-MM-DD) per figure: {"opus-5.5": {"coding": 91, "refs": {"coding": {"source": "SWE-bench Verified", "date": "2026-11-12"}}}}. A source and date on the entry itself stand for all three figures. null restores that model's default row. In the answer every row has refs for its three figures, and snapshot names the data set and the revision the defaults were read from.

rules is the list of default rules, each {"words": ["ios", "swift"], "agent": "claudeCode"} with agent one of claudeCode, codex, gemini, grok, and words of letters and digits. [] removes them all; null restores n8v's default.

The dry run takes current_model, repo (owner/name), prompt, title and assume_out. Without assume_out it answers what would happen at this moment: a model that still has room reports current_available and no pick. With it, the model is treated as out and current_has_room still says whether it really is.

curl -sS -X POST https://n8v.online/api/v1/subscriptions/benchmark-fallback/dry-run \
  -H "Authorization: Bearer $N8V_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"current_model": "opus-5.5-high", "repo": "acme/web", "assume_out": true}'

Each switch and return is in the audit log (limit.fallback, limit.fallback_return) with the chat's line, and in the event feed (chat.fallback.started, chat.fallback.ended). A change of the setting is limit.fallback_set.

Refusals: 403 benchmark_fallback_off (the flag is off for the account), 403 subscriptions_disabled, 403 repo_not_allowed, and 400 invalid_request with param naming the field.

Questions

Why did the chat not switch? Something for its own model still had room, an earlier fallback took the turn, the chat's model is pinned, or nothing else could run. The dry run shows which, per candidate.

Why not the model with the highest score? Your preference order comes first, and a model without room, behind an off flag or excluded by a rule is skipped.

Does it cost more? It uses only subscriptions and keys you already connected, under the limits you set for each key.