ENPT

StartLux-Prism serves StartLux-Decision GGUFs on NVIDIA at 19 ms per question, author says

Last updated:

Source · Model card: StartLux-Decision

publicado (OK do Fabricio 24/10 BRT).

StartLux Labs’ StartLux-Prism (repo since 13 Oct, covered by Pandaily on 24 Oct) serves StartLux-Decision v1 and v1.1 AutoThink (4B, 9B, 27B) straight from the released GGUF files on NVIDIA GPUs, behind /v1/systemone. Q8_0 runs as int8 and Q4_K_M as W4A8; in 4 bits the 27B fits a 32 GB RTX 5090.

Author figures (UnverifiedClaim): 27B v1.1 on one H200 answers a ~256-token question in 19.0 ms against 53.1 ms for the official engine in FP8 (vLLM 0.31), a three-question request 5.3× faster, and int8 keeps 92.94 vs 92.97 for BF16 on v1’s suites, with bit-identical answers across batches. Catalogued as a runtime; the model’s weights stay CC BY-NC-4.0.