s1-llm-auto-router

Last updated:

SpecialistMIT

⚠️ Why not a model

Single-task LLM-routing specialist: it answers only seven fixed questions with fixed option sets, so it does not compete with general System One models.

Technical specs

Base LLMs
mmBERT-small
Decision types
choice, score, noul
Features
llm-routing, cpu-onnx, multilingual-22, fixed-schema, hosted-api
License
MIT

s1-llm-auto-router is a small decision model from Decision Models, the inference platform tied to Bittensor Subnet 92 (REAX). It was released as open weights on 15 October 2026 under the MIT license. It reads a user turn before you send it anywhere and answers the questions an auto-router needs to choose a model.

Narrow scope

The weights are trained, but for one job. The model answers exactly seven questions, with exactly these options:

Question Type Answer
category choice coding, agentic, math, knowledge, long_context, tool_use, design, summarisation, general
difficulty score (5 levels) trivial → frontier
stakes score (4 levels) negligible → high
needs_tools, needs_vision, needs_long_context, follow_up noul probability

You can’t send it your own question or labels. That is why it sits here and not in the models catalog, the same treatment as CUA-S1-FORMS.

Technical details

  • Base: jhu-clsp/mmBERT-small (MIT), pruned to 11 layers and distilled from a fine-tuned mmBERT-base teacher, with seven classification heads and per-head temperatures in meta.json.
  • Format: ONNX with int8 embeddings (183 MB) and a fast tokenizer. No PyTorch or transformers needed.
  • Input: the user turn (512 tokens in total) plus an optional conversation summary (first 600 characters).
  • Hosted: model id s1-llm-auto-router on https://api.decisionmodels.io/v1/systemone, USD/EUR 0.001 per million input tokens at launch, output free.

Numbers (UnverifiedClaim)

All from the model card, measured by the publisher:

  • Latency: about 14 ms p50 / 47 ms p99 per request (all seven answers) on 4 cores of an AMD EPYC 9555, 26 ms p50 on an older Xeon; about 93 ms p50 end to end through the hosted API.
  • Routing test: 2,175 synthetic requests in 22 languages, gold labels from a vote of up to five open LLMs. Routing agreement 0.742, against 0.630 for Jev 1.13 (hosted), 0.577 for Winnow-12B and 0.622 for Weiche-395M. Category accuracy 0.898 vs 0.845 for Jev.
  • Small blind set: 100 requests labelled by Claude Opus, routing agreement 0.60 vs 0.57 for Jev (the card itself says ±8 points at this size).

The publisher points out that Jev is a general model and this fixed task is outside its design target, and it doesn’t rank its own models on JevBench, which it operates.

The launch post also announces a joint project with msg group, a German IT consultancy, which will evaluate the model in an internal project. That is a stated plan, not a production report yet.

Why it is listed here (not in Models)

It is a specialist. It has trained weights, but a fixed schema, and it can’t take arbitrary typed questions. Putting it in the quadrant would compare a seven-question router with general decision models.