Artificial Analysis launches its own Decision Index: Jev 1.13 leads decision models at 58, Claude Haiku 5.5 scores 61
On 13 October 2026 Artificial Analysis published the Artificial Analysis Decision Index v1.0 (launch post on X: 735 likes, about 48K views on 14 Oct). Artificial Analysis runs every evaluation itself. This is a different board from the community Decision Index on Hugging Face (edition 0.3); scores on the two are not comparable.
What it measures
Four sets, 25% each:
| Set | Skill | Metric |
|---|---|---|
| AA-Omniscience-Decision (500 private questions, with an “I don’t know” option) | Knowledge and hallucination | Accuracy + non-hallucination rate, 12.5% each |
| HLE-MultiChoice (513 text-only questions from Humanity’s Last Exam) | Reasoning | Accuracy |
| When2Call (NVIDIA) | Tool calling | Accuracy |
| WildGuardTest (Ai2; 835 of 1,725 rows) | Safety classification | Average F1 |
LLMs run the same sets with a structured output that names one option, so they give no per-option probability.
Launch results
- Decision models: Jev 1.13 58, ahead of GLiDE and Microsoft-Decision-1 at 54. Eleven decision models are in the index.
- Cost per 1M tasks: Decider v1.1 27B is the cheapest at $17, then Microsoft-Decision-1 at $31 and Jev 1.13 at $42, all three on the cost Pareto line. GLiDE costs $119. The range is $17 to $199.
- Speed: Celeris-1 Decision has the fastest median, 0.06 s per task, against 0.13 s for Jev 1.13.
- By set: Jev scores 21 on HLE-MultiChoice where no other decision model passes 15. d1 has the best Omniscience accuracy (41) but a non-hallucination rate of 2; Microsoft-Decision-1 leads that rate at 40%. Celeris-1 Decision and GLiDE lead When2Call at 84. GLiDE leads WildGuardTest at 95.
- Against LLMs: Claude Haiku 5.5 (Low) scores 61, 3 points above Jev 1.13, at 1.4 s per task and $184 per 1M tasks (11× slower, 4.4× the cost). GPT-6 Luna (Low) scores 55.
Calibration
Artificial Analysis finds every decision model overconfident: at almost every probability level, accuracy is below the probability given to the chosen answer. The gap is largest on knowledge and reasoning (Omniscience, HLE) and small on tool calling and classification. Calibration charts for LLMs are “coming soon”.
How to read it
This is the first third-party board for the category from an established benchmark shop, and it tests what decision models are least built for (knowledge, hard reasoning) next to what they are built for (routing, safety labels). Scores are from launch day; the live leaderboard will move.