Jev-Omni
Quadrant scores
See the full quadrantScored with the public quadrant rubric (maturity × capability; bubble = adoption). Revise as evidence lands.
- Maturity4.8/10
Public Apache-2.0 weights + loader/example; GPU-class ops (~50GB FP32), custom head stack, no hosted SLA, early release.
- Capability6.3/10
Official boards (17 Oct): Decision Index 0.3.2 Full 42.35 (#88); JevBench Intelligence 55.5, calibration 87.0, p50 0.155 s.
- Adoption38/100
Viral X launch (~2k likes / ~170k views on author post) + ~107 HF likes; still niche outside open-Jev wave.
Vendor claims
- DecisionBench Medium 87.57% / micro 86.01% (80 scenarios / 293 questions) — author card[Vendor claim — not independently verified]
- Author JevBench: 86.15% matched (self-reported) vs independent JevBench v1.4.2: #7 overall (51.3)[Vendor claim — not independently verified]
- MMAU 63.10% (1000 questions); MVBench 53.10% (14 tasks / 2786 questions) — author card[Vendor claim — not independently verified]
- Warm H200: 83 ms (~2k text), 26 ms image, 31 ms 13s audio, 504 ms 16-frame video — author card[Vendor claim — not independently verified]
- Card: 30,000-question fine-tune; decision_config recipe size 24000; tweet cites 30k examples / <100ms on 1 H100 — numbers conflict across sources[Vendor claim — not independently verified]
- X launch post: on par with Jev on typed benchmarks; first-multimodal framing is marketing — catalog treats four-modality ship as open peer fact only[Vendor claim — not independently verified]
News covering this model
Board audit of 17 Oct 2026: see the “Board audit” section below; where it conflicts with older board statements on this page, it wins.
JevBench v1.4.2 (independent)
| Metric | Score | Rank |
|---|---|---|
| Overall | 51.3 | #7 |
| Intelligence | 44.8 | - |
| Calibration | 71.2 | - |
| Speed | 76.9 | - |
| Cost | 52.4 | - |
The independent benchmark places Jev-Omni at #7 overall — a strong result for a community multimodal model, though below the author’s “on par with Jev” framing (Jev is #2 at 63.29). The 12B Gemma backbone handles decision quality well; speed (76.9) reflects the heavier compute.
Jev-Omni is an independent open multimodal decision classifier — Gemma 4 12B IT plus a custom head (head.pt) and loader (jev_omni.py) — not a TypeSafe product and not trained on Jev output (author card disclaimer).
State + question + options in; {prediction, prediction_index, confidence, probabilities} out in one forward pass with no text generation. Modalities: text | image | audio | video via media= path. Card: Hugging Face akhilaaa3/Jev-Omni (Apache-2.0). Dataset mentioned: akhilaaa3/decision-bench.
Catalog decisionTypes list choice and boolean (noul via Yes/No options). The card footer also names “score”; the public loader is an option classifier (2–256 slots, best ≤20) — treat score-as-buckets only if you encode buckets yourself. Not a pure transformers AutoModel drop-in: load through the shipped helper.
Among open catalog peers this is the first with all four modalities shipped (text/image/audio/video). openjev already covers image/video/text (audio flagged as coming); Prosodia is audio-specialist and stays off-quadrant.
Public intro: author X post 21 Sep 2026 (@Akhila_988); HF created ~20 Sep 2026. Likes ~107 as of 23 Sep 2026 (downloads API showed 0 at check).
Limits
- Independent JevBench v1.4.2 places Jev-Omni at #7 (51.3) — solid, but a 12-point gap vs Jev (#2 at 63.29) contradicts “on par” framing
- DecisionBench, MMAU, MVBench and H200 latency rows remain self-reported → UnverifiedClaim, not ModelSystem.One numbers
- Card “30k” FT vs
decision_configrecipesize: 24000vs tweet “30k examples / <100ms on 1 H100” — source tension; prefer card tables carefully and keep tweet claims unverified - Launch X video is polished motion-graphics packaging (not raw live UI). On-screen latencies (e.g. text ~17 ms, image ~36 ms, audio ~11 ms / 13s, video ~201–211 ms / 8 frames) differ from the HF H200 table (83 / 26 / 31 / 504 ms) — treat both as UnverifiedClaim
- Demo hygiene: animated bars can show option probs that do not sum to 1 (e.g. YES 53.3% / NO 6.5% on an audio clip) — packaging, not a catalog calibration proof
- CUDA / GPU-class ops (~50 GB FP32 before overhead; BF16 autocast at inference); custom head stack; no hosted SLA
- Best at ≤20 options; head accepts 256 but quality above 20 is not established
- Not affiliated with TypeSafe; “Jev” in the name is the interface shape only
Why it is in the catalog
Trained multimodal decision weights + open inference that returns typed option probabilities across four modalities (peer class of openjev / Tiny-Jev, heavier GPU footprint than mini open peers; distinct from hosted Jev). Radar: 2026-09-23.
Board audit (17 Oct 2026)
Checked against the official boards on 17 Oct 2026: Decision Index 0.3.2 (data of 16 Oct, 19:00 UTC), JevBench and ImageJevBench live. The boards are separate sources and are never averaged. Older board statements and rankings further down this page are superseded where they conflict with this section.
| Board | Row | Result |
|---|---|---|
| Decision Index 0.3.2 (16 Oct 19:00 UTC) | Jev-Omni | Full 42.35 (#88, tie group 21); public 41.04 (#96); same-skills 42.18; new-domains 39.24 |
| JevBench open weights (live 17 Oct) | Jev-Omni | Capability 71.3; Intelligence 55.5; calibration 87.0; p50 0.155 s; $0.029/1k |
| ImageJevBench v0.3.0 | Jev-Omni | Capability 54.3; Intelligence 37.5; calibration 71.0; p50 0.32 s; $0.022/1k |
| ImageJevBench v0.1.5 archive (frozen) | Jev-Omni | Capability 76.9; Intelligence 63.9; calibration 89.9; p50 0.30 s; $0.022/1k |
Board-status statements on this page to recheck against the table (may be superseded):
- “Independent JevBench v1.4.2 places Jev-Omni at #7 (51.3) — solid, but a 12-point gap vs Jev (#2 at 63.29) contradicts “on par” framing”
Score update (audit, 17 Oct 2026). Capability 6.9 → 6.3, first full working (the earlier score had no stored breakdown) = types 6 × 0.30 + calibration 6 × 0.25 + latency 7 × 0.25 + accuracy 6 × 0.20 = 6.25 → 6.3. Types from decisionTypes; calibration from the JevBench axis 87.0 (≥90 → 7, 85–89.9 → 6, 75–84.9 → 5, 60–74.9 → 4, below 60 → 3; no author ECE counted); latency from the board’s self-hosted p50 0.155 s without p99 (≤0.25 s → 7, ≤0.5 s → 6, ≤1 s → 5, above → 3); accuracy Decision Index Full 42.35 ÷ 60.11 × 8 = 5.6, half-up. Maturity and adoption unchanged.
Hugging Face on 17 Oct (akhilaaa3/Jev-Omni): 5,272 downloads, 415 likes, licence tag apache-2.0, last modified 2026-10-13.
