EmbeddingGemma 2 Typed Decision

Quadrant scores

See the full quadrant

Scored with the public quadrant rubric (maturity × capability; bubble = adoption). Revise as evidence lands.

• Maturity4.4/10

  Apache-2.0 fine-tune of EmbeddingGemma 2 (~0.3B) with a documented decide() call. Public weights only: no /v1/systemone server, package, third-party integration or SLA.

• Capability6.8/10

  choice, score and noul (boolean). One temperature per type, fit on held-out train rows. No published latency. The 76.4% test figure is the author's, and hyperparameters were chosen on the test split.

• Adoption2/100

  X post on 11 Oct: 130 likes, 6,098 views, 163 bookmarks (captured 12 Oct). HF: 10 likes and 0 downloads last month. No third-party integration or reported production use.

Vendor claims

• Author-reported 76.4% argmax accuracy on LocalLLaMA/typed-decisions (all, test): choice 71.2%, score 74.0%, noul 84.8%; soft CE 0.887, Brier 0.065. The card says hyperparameters were chosen while looking at the test split.[Vendor claim — not independently verified]
• The launch post says the model is about 6 points behind a 27B LLM with about 100× fewer parameters. That comparison is not on the model card. The card lists 0.3B parameters (~90× smaller than 27B).[Vendor claim — not independently verified]

Read this first

1. Open weights, local Python, custom code. Load it with trust_remote_code=True. There is no /v1/systemone server and no package.
2. 76.4% is the author’s test number. It matches the card’s phase-2 row. The same card says hyperparameters were chosen while looking at that split, so treat it as slightly optimistic.
3. The 27B comparison is only in the post. “~6 points behind a 27B LLM” and “~100× fewer parameters” are not on the card. 0.3B versus 27B is about 90×.

What it is

Manu Romero published mrm8488/embeddinggemma-2-ft-typed-decision on 11 October 2026 (card created 18:31 UTC). It is a text-only fine-tune of google/embeddinggemma-2 plus a schema-conditioned scoring head of about 0.5M parameters. The card lists the checkpoint at 0.3B parameters, Apache-2.0.

You pass a state and a schema of questions. model.decide(state, questions) returns a typed value and a probability for each question: choice (string), score (int level) and noul (boolean). The head scores each option’s text against the state and the question, so option names and descriptions are data. The card says answers drift when the schema is far from the training data.

How it was trained

The encoder uses the Classification prompt, mean pooling, 768-d, L2-normalized embeddings. State is rendered as key: value lines, not raw JSON. The logit is a zero-shot cosine prior plus a gated trained correction.

Phase 1 trains the head on cached embeddings. Phase 2 fine-tunes encoder and head together (encoder 2e-5, head 1e-4), with token embeddings frozen, 5 epochs, 32 rows per batch. The loss is cross-entropy on the annotators’ soft labels, a Brier term, and an ordinal distance for score. One temperature per question type is fit on held-out training rows and stored in the weights.

float16 overflows and returns NaNs. The card says to use bfloat16 or float32.

Author’s test numbers

On LocalLLaMA/typed-decisions, subset all, test split, argmax versus the gold label:

	Accuracy	Soft CE	Brier
Phase 1 (frozen encoder)	67.7%	0.968	0.113
Phase 2 (this checkpoint)	76.4%	0.887	0.065

By type, phase 1 to phase 2: choice 66.2% to 71.2%, score 61.8% to 74.0%, noul 77.0% to 84.8%.

Schemas were trained in English. The card says other languages still run, with softer probabilities. It also says numbers inside the state are harder for an embedding model than words, and that score is the weakest type.

Score working (12 Oct 2026)

• Maturity 4.4 = availability 8 × 0.30 + docs 6 × 0.25 + integrations 1 × 0.25 + support 1 × 0.20 = 2.40 + 1.50 + 0.25 + 0.20.
• Capability 6.8 = types 10 × 0.30 + calibration 7 × 0.25 + latency 4 × 0.25 + accuracy 5 × 0.20 = 3.00 + 1.75 + 1.00 + 1.00 = 6.75 → 6.8. Latency is 4 because none is published. Accuracy is 5 because the figure is the author’s and the card flags test-split optimism.
• Adoption 2 = engagement 5 × 0.35 + ecosystem 1 × 0.35 + production 1 × 0.30 = 1.75 + 0.35 + 0.30 = 2.40 → 2.