Bespoke-Nimble-9B

Quadrant scores

See the full quadrant

Scored with the public quadrant rubric (maturity × capability; bubble = adoption). Revise as evidence lands.

• Maturity4.5/10

  Complete open recipe (data/train/serve/evals). Since 30 Sep 2026 also a one-command ollama pull nimble behind Ollama's native /v1/systemone endpoint. Still a one-epoch LoRA, no hosted SLA, no RLCD.

• Capability6.8/10

  Official JevBench (17 Oct): Bespoke Nimble 9B Intelligence 43.1, calibration 69.9, p50 0.145 s; v3 is Decision Index Full 54.67 (#49). Choice / boolean / rubric score via candidate logits; close to Jev on the author holdout, weaker ECE.

• Adoption23/100

  ~1.9k GitHub stars, 213 HF likes / ~3.7k downloads, and Ollama's showcase model for /v1/systemone (~2.7k pulls in the first ~14h; launch post 234k views). Growing niche adoption, not the category default.

Vendor claims

• ~100ms median on H100 for a scoring subset (author timing; not full holdout)[Vendor claim — not independently verified]
• Holdout accuracy 90.1% vs base Qwen 66.4% vs Jev 1.13 93.2% (author 324-example set)[Vendor claim — not independently verified]
• Public human-label suite ~74.8% vs Jev ~76.0% (author-reported)[Vendor claim — not independently verified]
• Nimble 9B averaged 91ms per decision locally on a MacBook Pro M5 Max (Ollama Pac-Man demo)[Vendor claim — not independently verified]
• 75.7% mean accuracy on Bespoke's 13 public human-label datasets run on Ollama, vs Jev 1.13 76.0% (Bespoke's run)[Vendor claim — not independently verified]
• Did not distill from Jev — contrastive curation + LoRA on answer tokens only[Vendor claim — not independently verified]
• Bespoke-Nimble-9B-v3 (LoRA, CC BY-NC-4.0) scores 56.88 on Decision Index 0.2.1 (Bespoke's own run; not on the official board)[Vendor claim — not independently verified]

News covering this model

• Bespoke Labs ships Nimble V3, a LoRA with a claimed Decision Index of 56.88 and a non-commercial license
• Ollama adds a local /v1/systemone endpoint and three Jev-style decision models
• Bespoke Nimble adds image support — open LoRA now multimodal

publicado (OK do Fabricio 17/10 BRT). Board audit of 17 Oct 2026: see the “Board audit” section below; where it conflicts with older board statements on this page, it wins.

Bespoke-Nimble-9B is not a frontier base model — it is a LoRA adapter (PEFT r=16, ~165 MiB) on Qwen3.5-9B, shipped by Bespoke Labs as an open data / open model / open recipe for an open Jev.

Given text + a flat schema, it returns typed answers and probabilities over allowed tokens. It does not generate free-form reasoning as the primary output. Inspired by TypeSafe’s System One / Jev; the authors state they did not distill from Jev.

What it is

• Adapter: bespokelabs/Bespoke-Nimble-9B (Apache-2.0; needs Qwen/Qwen3.5-9B base)
• Recipe + local scoring: github.com/bespokelabsai/nimble
• Schema fields: boolean, enum (≤26 choices), rubric scores as ordered enum strings; returns selected value + per-option probabilities
• Method highlight: contrastive data curation (flip one fact → negative) + LoRA on answer tokens only
• Launch note: Mahesh Sathiamoorthy (@madiator) (18 Sep 2026)

See also catalog peers: Kev (smaller LoRA reconstruction), Jev (hosted reference).

Limits (read before you ship)

• Built in ~1–2 days — expect rough edges; research prototype, not a hosted product
• Max 26 choices per field; prompts over 2,048 tokens are rejected
• Author holdout is synthetic / model-checked (324 examples); public suite labels are human but criteria wording is a free parameter
• Probabilities are softmax over candidates at temperature 1.0 — not claimed as temperature-tuned calibration; ECE trails Jev on the public suite (author docs)
• Merged Mac path needs substantial RAM (~18 GB+ BF16 base); quality checks emphasized on the adapter CUDA path
• Do not use for production decisions that affect people without your own labelled eval

Image support (19 Sep 2026)

On 19 September, author @madiator announced that Nimble can now process images. A demo showed the model correctly counting bedrooms and bathrooms in a confusing floor plan where the base Qwen struggled. The author noted that Jev does not have multimodal support.

UnverifiedClaim: Image counting demo (1 floor-plan example); no multimodal benchmark published yet. (source)

This makes Nimble one of the few open System One–style adapters with image input. The underlying mechanism (vision encoder path) is not detailed in the demo; verify against the repo for implementation.

Run it with Ollama (30 Sep 2026)

Nimble is the showcase model for Ollama’s native decision-model support. With Ollama 0.35 or later:

ollama pull nimble

Ollama ships it as a merged 9.5 GB model (“fine-tuned from Qwen3.5-9B”, Apache-2.0), so you don’t need to load the Qwen base and the adapter separately. Requests go to http://localhost:11434/v1/systemone in TypeSafe’s format, and the official TypeSafe Python SDK works against it. The endpoint takes up to 64 questions per request and 2–26 options per question. Its confidence field measures how peaked the distribution is; it is not calibrated correctness.

UnverifiedClaim: Ollama reports Nimble “averaged 91ms per decision” in a Pac-Man demo on a MacBook Pro M5 Max, and 75.7% mean accuracy on Bespoke’s 13 public human-labelled datasets (3,880 decisions) against 76.0% for Jev 1.13 (Jev figure from Bespoke’s published run). (blog · library · news)

Nimble V3 (30 Sep 2026): non-commercial license

Bespoke-Nimble-9B-v3 (first published as bespokelabs/Nimble-V3, renamed since; the old URL redirects) is a new LoRA adapter on Qwen3.5-9B for typed choice and yes/no questions. Bespoke reports Decision Index 0.2.1 = 56.88, with the run, inference code and instructions in bespoke-nimble-9b-v3-decision-index. UnverifiedClaim: this is Bespoke’s own run, and V3 was not on the official Decision Index board when we checked on 2 Oct 2026.

The license changed. V3 is CC BY-NC-4.0: no commercial use. The original Bespoke-Nimble-9B, v2 (23 Sep) and Ollama’s nimble stay Apache-2.0. If you ship commercially, stay on those or get a separate grant from Bespoke. V3 is a family note on this page, not a second quadrant point; the scores above still describe the Apache-2.0 line. See news.

Nimble v4 (13 Oct 2026): API-only

Bespoke’s newest models, nimble-v4 and nimble-v4-max, are closed and hosted, so they have their own page: Nimble v4. nimble-v3 is listed as deprecated on the Bespoke API. This page still scores the open 9B LoRAs.

Why it is in the catalog

Public typed decision I/O, open weights + full recipe, and explicit System One framing. Plotted next to Kev as an open LoRA point — with artifactKind: lora-adapter so the card does not pretend this is a standalone frontier model.

Board audit (17 Oct 2026)

publicado (OK do Fabricio 17/10 BRT). Checked against the official boards on 17 Oct 2026: Decision Index 0.3.2 (data of 16 Oct, 19:00 UTC), JevBench and ImageJevBench live. The boards are separate sources and are never averaged. Older board statements and rankings further down this page are superseded where they conflict with this section.

Board	Row	Result
Decision Index 0.3.2 (16 Oct 19:00 UTC)	Bespoke Nimble 9B v3	Full 54.67 (#49, tie group 11); public 57.19 (#46); same-skills 52.34; new-domains 49.05
Decision Index 0.3.2 (16 Oct 19:00 UTC)	Bespoke Nimble 9B v2	Full 40.95 (#92, tie group 22); public 39.9 (#99); same-skills 39.51; new-domains 39.68
JevBench open weights (live 17 Oct)	Bespoke Nimble 9B v3	Capability 66.8; Intelligence 56.5; calibration 77.0; p50 0.094 s; $0.063/1k
JevBench open weights (live 17 Oct)	Bespoke Nimble 9B	Capability 56.5; Intelligence 43.1; calibration 69.9; p50 0.145 s; $0.13/1k

Board-status statements on this page to recheck against the table (may be superseded):

• “Bespoke reports Decision Index 0.2.1 = 56.88, with the run, inference code and instructions in bespoke-nimble-9b-v3-decision-index.”

Score proposal (from scratch, audit follow-up 17 Oct 2026; publicado (OK do Fabricio 17/10 BRT)). Capability 5.0 → 6.8 = types × 0.30 + calibration × 0.25 + latency × 0.25 + accuracy × 0.20 = 10 × 0.30 + 4 × 0.25 + 7 × 0.25 + 5 × 0.20 = 6.75 → 6.8. Types 10 (choice + score + boolean). Calibration 4: JevBench axis 69.9 for Bespoke Nimble 9B (60–74.9 → 4); the author flags weaker ECE. Latency 7: board p50 0.145 s (≤0.25 s → 7), no p99. Accuracy 5: the Decision Index has no row for this v1 adapter (it lists v2 and v3), so JevBench Intelligence 43.1 ÷ 63.6 × 8 = 5.4 → 5. The v3 row (Full 54.67, accuracy 7) is not applied to this v1 page; family text: the board lists Nimble 9B v2 (Full 40.95, #92) and v3 (54.67, #49), v4 has its own page. Maturity and adoption unchanged.

Hugging Face on 17 Oct (bespokelabs/Bespoke-Nimble-9B-v3): 531 downloads, 13 likes, licence tag cc-by-nc-4.0, last modified 2026-09-30.