Nimble v4 / v4-max
Quadrant scores
See the full quadrantScored with the public quadrant rubric (maturity × capability; bubble = adoption). Revise as evidence lands.
- Maturity6.1/10
Paid hosted API (POST /v1/systemone) with docs, quickstart, TypeSafe SDK support, an agent skill and prepaid credit. Closed weights, unlike the open Nimble 9B LoRAs. nimble-v4 is now deprecated and aliases v4-max. No SLA published.
- Capability8.1/10
Noul, choice (2–255 options) and score, up to 64 questions per request. Independent JevBench API board: v4-max composite 66.2 (#14), Intelligence 68.2, Calibration 91, capability 79.7, p50 0.25 s. Not on DI 0.3 or AA. No ECE or calibration method published.
- Adoption3/100
Launch thread 167 likes / 26K views (13 Oct). Bespoke’s open Nimble line already has an Ollama showcase and ~1.9k GitHub stars; v4 itself has no named users yet.
Vendor claims
- nimble-v4-max and nimble-v4 score +10 and +6 over Jev 1.13 on Decision Index public scores, making v4-max #1; 43% fewer mistakes than Jev and +50 pp on model-routing decisions (internal evals)[Vendor claim — not independently verified]
News covering this model
Read this first. These are API-only models. The open Bespoke-Nimble-9B LoRAs are a separate page. Bespoke’s numbers are UnverifiedClaim: the post gives deltas against Jev 1.13, not absolute scores or a run on the board.
Bespoke Labs announced nimble-v4 and nimble-v4-max on 13 Oct 2026. Per the docs, both are called at POST https://api.bespokelabs.ai/v1/systemone, by plain HTTP or TypeSafe’s Python SDK. nimble-latest points to v4-max (see pricing below). Each request answers up to 64 questions about one state.
Pricing (docs): input only, output free. nimble-v4-max and nimble-latest $0.04/M. nimble-v4 is deprecated and now points to nimble-v4-max, so it costs the same; nimble-v3 is deprecated. New accounts get $1 of credit. (At launch on 13 Oct the page listed v4 at $0.04/M and v4-max at $0.09/M; Benchmark Heaven’s 16 Oct re-measure notes the cut. We re-read the docs page on 17 Oct.)
What “#1” would mean. On the Decision Index 0.3 board Jev’s public part is 57.96, so +10 would put v4-max near 68, above the current top public score (65.64). Neither model is on the DI 0.3 board (Benchmark Heaven has since run v4-max on its JevBench API board, a different board; see below), and the post doesn’t say which DI edition it used.
Independent result: JevBench API board (Benchmark Heaven, board v1.7.46, checked 17 Oct 2026)
On the API board (1,500-item full set; est. $0.0516 per 1,000 decisions for both rows) Bespoke Nimble-v4-max is #14 on composite (66.2), Intelligence 68.2, Calibration 91, capability 79.7 (third by capability on that board, ahead of Sage 1.3.0’s 78.6 and Jev 1.13.0’s 77.1), raw p50 0.25 s. The composite is held down mostly by cost (the board’s cost term is the lowest of its four) while Intelligence (68.2), Calibration (91) and Speed (91) are all solid (kettei-fat’s Intelligence is 80). For comparison on the same board: kettei-fat 77.8, Sage 1.3.0 74.0, Jev 1.13.0 71.5, Jev 1.14.0 70.2. The old Nimble-v4 row scored 38.4 (#24; Intelligence 41.4, below the board’s gate), measured before v4 became an alias of v4-max; read it as history.
What it does and does not settle: this is a third-party board, and it does not confirm the vendor’s “#1 on the Decision Index” claim, because neither Nimble model is on Decision Index 0.3 and the Artificial Analysis index. The two boards are not mixed.
A community bench (not an official board): S1Bench by @ItsCuthulhu (15 Oct 2026)
@ItsCuthulhu posted a run of both Nimble models on his own S1Bench (checked 17 Oct): six public suites (VitaminC, MASSIVE en-US, BoolQ, HelpSteer2, Aegis2, PAWS), 1,999 decisions per model, 0 errors, scored against Jev 1.13.0’s published numbers on the same subsets. Read from the page’s data on 17 Oct:
| Model (as labelled on the bench) | Macro accuracy | Δ vs published Jev 1.13.0 (0.768) | ECE | Median per decision |
|---|---|---|---|---|
| nimble-v4-max (“27B”) | 0.783 | +0.014 (paired p = 0.52, not significant) | 0.024 | 0.84 s |
| nimble-v4 (“9B”) | 0.764 | −0.004 (paired p = 0.04) | 0.026 | 0.23 s |
The post quotes Jev at 0.775 and a +0.008 gap for v4-max; the live page uses 0.768 as the Jev baseline and shows +0.014. We report both and do not pick one. The author’s point is that Bespoke’s Decision Index claim of +10.5 over Jev does not show up on a different battery, and on this one the difference for v4-max is within noise.
How we treat it: it is an independent hobby bench with its own harness, not one of the official boards (JevBench, Decision Index 0.3, Artificial Analysis), so it does not enter the score working and it is not mixed with those boards. The parameter counts (27B, 9B) are the bench’s labels; Bespoke has not confirmed them (UnverifiedClaim for sizes). The bench’s price column is also the author’s. The 0.84 s median is slower than the 0.25 s Benchmark Heaven measured for v4-max; the two harnesses differ and we show both. As of 17 Oct nimble-v4 is an alias of nimble-v4-max, so the 9B row is history.
Score working (17 Oct 2026, re-score)
- Maturity = availability 7 × 0.30 + docs 7 × 0.25 + integrations 5 × 0.25 + support 5 × 0.20 = 2.10 + 1.75 + 1.25 + 1.00 = 6.10 → 6.1 (unchanged)
- Capability = types 9 × 0.30 + calibration 7 × 0.25 + latency 8 × 0.25 + accuracy 8 × 0.20 = 2.70 + 1.75 + 2.00 + 1.60 = 8.05 → 8.1 (was 6.35 → 6.4)
- Adoption = engagement 5 × 0.35 + ecosystem 3 × 0.35 + production 1 × 0.30 = 1.75 + 1.05 + 0.30 = 3.10 → 3 (unchanged)
What moved: calibration 5 → 7 (an independent calibration axis of 91, with no ECE or method, the same treatment as Sage); latency 4 → 8 (0.25 s measured by the board); accuracy 7 → 8 (independent board result, replacing the vendor deltas). The price cut does not enter the formula. The older working, kept for the record: maturity 6.10, capability 6.35 → 6.4, adoption 3.10 → 3 (13 Oct).