Independent BANKING77 test: Kev 98%, Verdict 89% dynamic vs Qwen JSON 66%
On 15 October 2026 Towards Data Science published Lambert Leong’s hands-on comparison (article) of seven open projects versus a pinned Qwen3-0.6B JSON baseline on BANKING77 routing (12 intents; fixed 8-route menus and dynamic 4-candidate menus; two repeats with identical predictions).
Fixed routes (200 req): Kev 0.8B 98.0%, Laya 92.0%, Verdict 86.5% (8 abstentions), Qwen JSON 83.0%, AnyJev 79.5%, NanoJev 52.0%.
Changing candidates (100 req): Kev 98.0%, Verdict 89.0%, Laya 74.0%, Qwen JSON 66.0%, NanoJev 17.0%. Nine of Qwen’s dynamic misses were markdown-fenced JSON rejected by a strict parser.
Median warm latency (author table): Verdict ~17–18 ms, Kev ~125–130 ms, Qwen JSON ~331–357 ms. Scope is narrow (Colab, batch 1, twelve intents). The author treats results as leads, not a universal ranking. Useful third-party evidence for catalog notes on kev, laya and verdict. No score change from this single eval.
