AutoJev-27B
Quadrant scores
See the full quadrantScored with the public quadrant rubric (maturity × capability; bubble = adoption). Revise as evidence lands.
- Maturity6.6/10
Apache-2.0 weights and MIT code with a /v1/systemone server, and since 01/10 the same weights as a hosted, priced Perplexity API ($0.02/M input since 6 Oct) with quickstart, OpenAPI reference, rate limits and a cookbook. The hosted endpoint is a preview (gateway-preview v0.1.0) on its own path, with no SLA.
- Capability7.6/10
Official JevBench (17 Oct): Intelligence 55.6, calibration 87.6, p50 0.097 s. Noul, Choice and Score in one pass. Temperature scaling is documented. No latency figures published.
- Adoption24/100
Perplexity's launch post reached ~540 likes and ~116k views within hours, and Perplexity now serves the model. Before that: ~47 HF likes, ~1k downloads, 110 GitHub stars and a community GGUF. No named production users yet.
Vendor claims
- Overall accuracy 84.60% vs Jev 82.79% and Qwen3.8-27B 69.83%; ECE 0.043 vs Jev 0.053 (author's own held-out set)[Vendor claim — not independently verified]
- One H200, full-weight SFT, 73,000 unique training examples, 286 updates; released checkpoint is step 200[Vendor claim — not independently verified]
- Research, data generation, training, evaluation and deployment were executed by autonomous agents[Vendor claim — not independently verified]
- Perplexity Decisions API (pplx-decider-v1-27b): 85.71% overall vs Jev 84.51% on an 11-benchmark panel (7,210 rows, measured through Perplexity's API); Jev better on WinoGrande, BBH, JevBench hard, TruthfulQA, Belebele and JudgeBench[Vendor claim — not independently verified]
News covering this model
Board audit of 17 Oct 2026: see the “Board audit” section below; where it conflicts with older board statements on this page, it wins.
Read this first Perplexity’s Decisions API serves these exact weights.
perplexity-ai/pplx-decider-v1-27b(Hugging Face, 01/10 18:24 BRT, Apache-2.0) has byte-identical files todenis-pplx/autojev-27b: the same SHA-256 for all 11 safetensors shards,readout.safetensors,config.jsonanddecision_config.json. Its NOTICE calls it the “AutoJev 27B curated SFT checkpoint” and it bundles theautojevsource. So the hostedpplx-decider-v1-27bis this catalog entry, not a new model. Perplexity has not said so in its launch post; the hash comparison is ours (01/10).
AutoJev-27B is an open decision model published by the Hugging Face / GitHub user denis-pplx on 19 September 2026. It is a full-weight fine-tune of Qwen/Qwen3.8-27B that returns probabilities over the options you supply, in one forward pass per question. The repo ships a server that speaks POST /v1/systemone with choice, noul and score, a browser playground and optional base64 images.
The author describes it as an “independent implementation inspired by Jev”. It is not a TypeSafe product and is not trained on Jev outputs as far as the card says. It is also unrelated to AutoTrust’s JEV models (now cataloged as GEV-26B-Decide), which use a similar-sounding name.
v1.1 is a different model (6 Oct 2026). Perplexity’s pplx-decider-v1.1-27b is a new training run on the same Qwen3.8-27B backbone (noncausal attention, more data) and is now the API’s default model, with its own page. This page stays on v1 = AutoJev-27B, which the API still serves as
pplx-decider-v1-27b, now at $0.02/M input.
Perplexity Decisions API (1 October 2026)
@perplexitydevs launched a Decisions API on 1 October 2026 (15:24 BRT; ~540 likes and ~116k views by the evening).
| Attribute | Value |
|---|---|
| Endpoint | POST https://api.perplexity.ai/v1/decisions, model pplx-decider-v1-27b |
| Body | System One shape (state, questions) with noul, choice, score and images, on its own path (not /v1/systemone) |
| Price | $0.02 / M input tokens since 6 Oct 2026 (launched at $0.04); output tokens free; no per-request fee |
| Limits | 10 requests/s per organization; ~250k-token context per the launch post |
| Timing (Perplexity’s tests, 30/09) | a few hundred input tokens under 2 s; ~90k tokens 5 s; ~190k tokens 14 s |
| Status | OpenAPI labelled “gateway-preview” v0.1.0 |
| Weights | perplexity-ai/pplx-decider-v1-27b, identical to AutoJev-27B |
Perplexity’s chart (UnverifiedClaim): 85.71% overall vs Jev 84.51% on an 11-benchmark panel, 7,210 rows run through its own API; Jev is ahead on WinoGrande, BBH, JevBench hard, TruthfulQA, Belebele and JudgeBench. The cookbook shows ticket triage.
The company field now reads “denis-pplx · hosted by Perplexity”. Perplexity’s post calls it “our multimodal decision model” without mentioning AutoJev; the author’s handle suggests he works at Perplexity, but neither side states it.
Specs
| Attribute | Value |
|---|---|
| Author | denis-pplx (HF + GitHub) |
| Release | 19 Sep 2026 (HF repo 16:54 BRT; GitHub repo created the same day) |
| Base model | Qwen/Qwen3.8-27B |
| Parameters | 26.1B (BF16, ~49 GiB of weights) |
| Training | Full-weight SFT on one H200, 73,000 unique examples, 286 updates; checkpoint 200 released; scalar temperature fitted separately |
| Training data | Generated and curated by the author’s agent pipeline; the exact corpus is not bundled |
| API | POST /v1/systemone (choice, noul, score, optional images); FastAPI docs at /docs |
| License | Weights Apache-2.0, code MIT |
| Latency | No figures published (0–0ms in the frontmatter means “no data”) |
Decision Index 0.2.1 (official Space)
Measured by the official Decision Index Space (data snapshot 28 Sep 2026, 71 systems):
| System | Index (balanced skill) | Rank |
|---|---|---|
| Jev 1.13 | 57.91 | #1 |
| Surogate Rune 26B-A4B v3 | 57.44 | #2 |
| AutoJev-27B | 56.40 | #4 |
| simple-jev (Qwen3.8-27B, runtime) | 55.74 | #5 |
The index also measures calibration: for AutoJev-27B, accuracy 0.730, mean confidence 0.738, ECE 0.018 and Brier 0.361. That is tighter than the author’s own ECE figure.
Note the gap to simple-jev on the same Qwen3.8-27B base: 0.66 points. Most of AutoJev’s accuracy comes from the base model; the fine-tune adds a trained decision readout and better calibration.
Author’s own results (UnverifiedClaim)
| Model | Overall accuracy | ECE | Brier |
|---|---|---|---|
| Qwen3.8-27B | 69.83% | 0.0648 | 0.408 |
| AutoJev-27B | 84.60% | 0.0428 | 0.220 |
| Jev | 82.79% | 0.0527 | 0.254 |
These come from the author’s curated held-out set (holdout-v3), which is not public. Beating Jev on your own test set is a different claim from matching Jev across public suites; on the official index AutoJev sits 1.5 points below Jev.
Community GGUF
thomasgauthier/autojev-27b-GGUF ships a Q4_K_M quant and a vision projector (~3k downloads). It does not run on mainline llama.cpp or LM Studio: you need the jev.cpp fork, which adds the AutoJev classifier head and a --system-one server flag. Another copy, arafathusayn/autojev-27b, is a plain re-upload.
Fit / anti-fit
Fit when you need: a local, TypeSafe-compatible endpoint with good measured calibration, agent gating and routing decisions, or a base to study (training code is public).
Anti-fit when you need: a small GPU (about 49 GiB of BF16 weights plus overhead), published latency, natural-image accuracy (the author says image support is not an accuracy claim), or the training corpus.
Limits
- The training corpus is not published; the headline accuracy is on a private held-out set.
- The card still says “while the weights are private, authenticate”; the repo was public and ungated when we checked on 30 Sep 2026.
- “Built by autonomous agents” describes the author’s process; it does not change how the model is evaluated here.
- No p50/p99 latency numbers; Perplexity gives only coarse timings by input size.
Why it is in the catalog
The weights were fully fine-tuned for typed decisions and ship with an open System One-compatible server that returns probabilities. That passes the catalog criteria. The official index places it among the top open entries.
JevBench v1.6.1 (Benchmark Heaven, checked 7 Oct 2026)
On the v1.6.1 open-weights board (added 7 Oct; not comparable with v1.5.7), AutoJev-27B is #46 at 18.5. Its capability is 71.6 (Intelligence 55.6, Calibration 87.6, raw p50 0.34 s), but self-hosting costs $0.23 per 1,000 decisions (7.0× Jev), so it sits outside the Jev-class caps. Decision Index 0.3 dropped pplx-decider v1 (these weights) as superseded by v1.1. See news. No catalog score change.
JevBench v1.5.7 (earlier edition, checked 5 Oct 2026; not comparable with v1.6)
AutoJev-27B is #55 at official 19.5 on v1.5.7. See news. No catalog score change.
Board audit (17 Oct 2026)
Checked against the official boards on 17 Oct 2026: Decision Index 0.3.2 (data of 16 Oct, 19:00 UTC), JevBench and ImageJevBench live. The boards are separate sources and are never averaged. Older board statements and rankings further down this page are superseded where they conflict with this section.
| Board | Row | Result |
|---|---|---|
| JevBench open weights (live 17 Oct) | AutoJev-27B | Capability 71.6; Intelligence 55.6; calibration 87.6; p50 0.097 s; $0.23/1k |
| ImageJevBench v0.3.0 | AutoJev-27B | Capability 62.1; Intelligence 37.6; calibration 86.7; p50 0.48 s; $0.054/1k |
| ImageJevBench v0.1.5 archive (frozen) | AutoJev-27B | Capability 75.9; Intelligence 69.3; calibration 82.4; p50 0.43 s; $0.049/1k |
Score update (audit, 17 Oct 2026). Capability 7.2 → 7.6 = types × 0.30 + calibration × 0.25 + latency × 0.25 + accuracy × 0.20 = 8 × 0.30 + 8 × 0.25 + 7 × 0.25 + 7 × 0.20 = 7.55 → 7.6. Types, calibration and latency come from the earlier working (8 / 8 / 7); only accuracy is re-derived from the board (7; was 5): JevBench Intelligence 55.6 ÷ 63.6 × 8 = 7.0, half-up. Maturity and adoption unchanged.
Hugging Face on 17 Oct (denis-pplx/autojev-27b): 3,056 downloads, 52 likes, licence tag apache-2.0, last modified 2026-09-22.
Links
- Hugging Face: denis-pplx/autojev-27b
- GitHub: denis-pplx/autojev
- Perplexity: pplx-decider-v1-27b on Hugging Face · launch post · news
- Community GGUF (thomasgauthier) · jev.cpp fork
- Decision Index (official Space)
- News: Five open decision models from HF trending
- Compare: Jev · Surogate Rune · simple-jev
