Jev-Omni

Last updated:

Open80–500ms$0/M input

Quadrant scores

See the full quadrant

Scored with the public quadrant rubric (maturity × capability; bubble = adoption). Revise as evidence lands.

  • Maturity4.8/10

    Public Apache-2.0 weights + loader/example; GPU-class ops (~50GB FP32), custom head stack, no hosted SLA, early release.

  • Capability6.3/10

    Official boards (17 Oct): Decision Index 0.3.2 Full 42.35 (#88); JevBench Intelligence 55.5, calibration 87.0, p50 0.155 s.

  • Adoption38/100

    Viral X launch (~2k likes / ~170k views on author post) + ~107 HF likes; still niche outside open-Jev wave.

Vendor claims

News covering this model

Board audit of 17 Oct 2026: see the “Board audit” section below; where it conflicts with older board statements on this page, it wins.

JevBench v1.4.2 (independent)

Metric Score Rank
Overall 51.3 #7
Intelligence 44.8 -
Calibration 71.2 -
Speed 76.9 -
Cost 52.4 -

The independent benchmark places Jev-Omni at #7 overall — a strong result for a community multimodal model, though below the author’s “on par with Jev” framing (Jev is #2 at 63.29). The 12B Gemma backbone handles decision quality well; speed (76.9) reflects the heavier compute.

Jev-Omni is an independent open multimodal decision classifier — Gemma 4 12B IT plus a custom head (head.pt) and loader (jev_omni.py) — not a TypeSafe product and not trained on Jev output (author card disclaimer).

State + question + options in; {prediction, prediction_index, confidence, probabilities} out in one forward pass with no text generation. Modalities: text | image | audio | video via media= path. Card: Hugging Face akhilaaa3/Jev-Omni (Apache-2.0). Dataset mentioned: akhilaaa3/decision-bench.

Catalog decisionTypes list choice and boolean (noul via Yes/No options). The card footer also names “score”; the public loader is an option classifier (2–256 slots, best ≤20) — treat score-as-buckets only if you encode buckets yourself. Not a pure transformers AutoModel drop-in: load through the shipped helper.

Among open catalog peers this is the first with all four modalities shipped (text/image/audio/video). openjev already covers image/video/text (audio flagged as coming); Prosodia is audio-specialist and stays off-quadrant.

Public intro: author X post 21 Sep 2026 (@Akhila_988); HF created ~20 Sep 2026. Likes ~107 as of 23 Sep 2026 (downloads API showed 0 at check).

Limits

  • Independent JevBench v1.4.2 places Jev-Omni at #7 (51.3) — solid, but a 12-point gap vs Jev (#2 at 63.29) contradicts “on par” framing
  • DecisionBench, MMAU, MVBench and H200 latency rows remain self-reported → UnverifiedClaim, not ModelSystem.One numbers
  • Card “30k” FT vs decision_config recipe size: 24000 vs tweet “30k examples / <100ms on 1 H100” — source tension; prefer card tables carefully and keep tweet claims unverified
  • Launch X video is polished motion-graphics packaging (not raw live UI). On-screen latencies (e.g. text ~17 ms, image ~36 ms, audio ~11 ms / 13s, video ~201–211 ms / 8 frames) differ from the HF H200 table (83 / 26 / 31 / 504 ms) — treat both as UnverifiedClaim
  • Demo hygiene: animated bars can show option probs that do not sum to 1 (e.g. YES 53.3% / NO 6.5% on an audio clip) — packaging, not a catalog calibration proof
  • CUDA / GPU-class ops (~50 GB FP32 before overhead; BF16 autocast at inference); custom head stack; no hosted SLA
  • Best at ≤20 options; head accepts 256 but quality above 20 is not established
  • Not affiliated with TypeSafe; “Jev” in the name is the interface shape only

Why it is in the catalog

Trained multimodal decision weights + open inference that returns typed option probabilities across four modalities (peer class of openjev / Tiny-Jev, heavier GPU footprint than mini open peers; distinct from hosted Jev). Radar: 2026-09-23.

Board audit (17 Oct 2026)

Checked against the official boards on 17 Oct 2026: Decision Index 0.3.2 (data of 16 Oct, 19:00 UTC), JevBench and ImageJevBench live. The boards are separate sources and are never averaged. Older board statements and rankings further down this page are superseded where they conflict with this section.

Board Row Result
Decision Index 0.3.2 (16 Oct 19:00 UTC) Jev-Omni Full 42.35 (#88, tie group 21); public 41.04 (#96); same-skills 42.18; new-domains 39.24
JevBench open weights (live 17 Oct) Jev-Omni Capability 71.3; Intelligence 55.5; calibration 87.0; p50 0.155 s; $0.029/1k
ImageJevBench v0.3.0 Jev-Omni Capability 54.3; Intelligence 37.5; calibration 71.0; p50 0.32 s; $0.022/1k
ImageJevBench v0.1.5 archive (frozen) Jev-Omni Capability 76.9; Intelligence 63.9; calibration 89.9; p50 0.30 s; $0.022/1k

Board-status statements on this page to recheck against the table (may be superseded):

  • “Independent JevBench v1.4.2 places Jev-Omni at #7 (51.3) — solid, but a 12-point gap vs Jev (#2 at 63.29) contradicts “on par” framing”

Score update (audit, 17 Oct 2026). Capability 6.9 → 6.3, first full working (the earlier score had no stored breakdown) = types 6 × 0.30 + calibration 6 × 0.25 + latency 7 × 0.25 + accuracy 6 × 0.20 = 6.25 → 6.3. Types from decisionTypes; calibration from the JevBench axis 87.0 (≥90 → 7, 85–89.9 → 6, 75–84.9 → 5, 60–74.9 → 4, below 60 → 3; no author ECE counted); latency from the board’s self-hosted p50 0.155 s without p99 (≤0.25 s → 7, ≤0.5 s → 6, ≤1 s → 5, above → 3); accuracy Decision Index Full 42.35 ÷ 60.11 × 8 = 5.6, half-up. Maturity and adoption unchanged.

Hugging Face on 17 Oct (akhilaaa3/Jev-Omni): 5,272 downloads, 415 likes, licence tag apache-2.0, last modified 2026-10-13.