StartLux-Prism
⚠️ Why not a model
Inference engine for the existing StartLux-Decision GGUF files on NVIDIA GPUs. Prompts, read-outs and decision rules come from the model; only the forward pass is new.
Technical specs
- Base LLMs
- StartLux-Decision v1 (4B, 9B, 27B), StartLux-Decision v1.1 AutoThink (4B, 9B, 27B)
- Decision types
- choice, noul, score
- Features
- http-systemone, int8, w4a8, gguf, nvidia, tensor-parallel, data-parallel, prefix-cache, multimodal, bit-exact
- License
- not declared (no LICENSE detected by GitHub)
Links
publicado (OK do Fabricio 24/10 BRT).
StartLux-Prism (StartLuxLabs/StartLux-Prism, repo since 13 Oct; covered by Pandaily on 24 Oct) serves StartLux-Decision v1 and v1.1 AutoThink (4B, 9B, 27B) straight from the released GGUF files, behind the /v1/systemone API, on NVIDIA GPUs from Ampere to Blackwell. Q8_0 files run as int8 (W8A8) and Q4_K_M as W4A8, with no conversion; in 4 bits the 27B fits one 32 GB RTX 5090. Linux, Python 3.12, CUDA 12 or 13; kernels compile on first start.
Why it is not a model
No new weights. Prompts, read-outs, thoughts and decision rules come from StartLux-Decision; only the forward pass is reimplemented. The weights it serves keep their CC BY-NC-4.0 licence.
Author figures (UnverifiedClaim)
- 27B v1.1 on one H200: 19.0 ms per ~256-token question vs 53.1 ms for the official engine in FP8 (vLLM 0.31); three-question request 17.4 vs 92.9 ms (5.3×).
- 24 questions about one video (8 frames): 0.21 s vs 0.72 s.
- Eight H200s: 238 requests/s vs 138 for the official engine in BF16.
- Accuracy on v1’s seven suites: int8 92.94, W4A8 93.07, official BF16 92.97. int8 sums exactly in int32, so answers are bit-identical whatever the batch.