JevBench places Microsoft-Decision-1 sixth among hosted decision APIs
Source · Model card: Microsoft-Decision-1
Benchmark Heaven added Microsoft-Decision-1 to its hosted API leaderboard on 10 October. The run covers all 1,500 decisions in the JevBench v1.6.1 pool. On the current API board, Microsoft ranks #6 of 27 with a composite score of 69.1 and a capability score of 70.8. Jev is #4 at 71.5; Sage is #1 at 74.0.
The axes help explain that placement: Microsoft scores 57.2 on intelligence, 84.4 on calibration, 82 on speed and 61 on cost. Its median endpoint latency is 459 ms on the board’s 124-request speed subset, and its measured cost is $0.019 per 1,000 decisions. The run records 21 context-limit failures and keeps them in the scored denominator. Benchmark Heaven describes the board as independent of TypeSafe AI.
Where it beats Jev
The category radar favors Microsoft on several of the use cases named in the launch post: recruiting (29.7 vs Jev’s 0.0), risk assessment (51.2 vs 32.8), gaming (25.1 vs 5.1), advertising (37.0 vs 16.4), scientific discovery (36.9 vs 21.2), and moderation/trust and safety (67.2 vs 51.6). The board cautions that category pools and supported decision types can differ, so these scores are directional comparisons rather than a controlled head-to-head.
Its mixed-language radar is 88.32 across 79 completed responses. That is a strong signal, but the sample is small. The post says Microsoft is the strongest API model in the top ten on mixed-language tasks; the live board’s API listing confirms the measured value, while the all-systems ranking is reported by the post.
This result replaces the earlier catalog note that Microsoft had no independent benchmark placement. Capability is updated from 5.8 to 7.9; the JevBench measurements are third-party evidence. Microsoft’s own 36-benchmark and speed claims remain separately attributed to the vendor.
Sources
Clinical System One preview (10 Oct 2026)
A second public result comes from the EdgeEvals System One preview board: Microsoft-Decision-1 is tied #5 of 38 with 611/669 decisions credited, including 152/153 clinical-note cases. The board records two severe triage errors, 417 ms median latency and $0.03129 per 1,000 decisions (~$0.021 for this run). The irrelevant-text check held 130/130 previously correct cases, with no new severe errors.
The board’s methodology says its cases are synthetic and not yet reviewed by clinicians or certified coders, so this is not clinical validation. The tweet’s counts and cost are consistent with the board; it omits the two severe errors. This does not change catalog scores. Data · methodology · post.