Taktile Labs benchmark: a decision model fine-tuned on 64 bank statements beats every out-of-the-box model
publicado (OK do Fabricio 30/10 BRT).
Taktile Labs, the research arm of the lending-software company Taktile, published a bank-statement classification benchmark (David Ahn, Maximilian Eber), announced on 29 October 2026. It compares 23 methods, from keyword rules and frontier LLMs to fine-tuned decision models, on 72 synthetic statements (25,593 transactions, 25 categories, 8 bank formats).
Findings (author-run, UnverifiedClaim). Without labelled data, Claude Opus 5.5 was the most accurate (95.5%, $2.52 per 1,000 transactions); the best decision models out of the box scored 92.1 to 93.0% for $0.002 to $0.107. After fine-tuning on 24,400 labelled transactions (about 64 three-month statements), Taktile’s decision model on Qwen3.5-4B reached 97.3% on the main test set (97.2% across three) at about $0.03 per 1,000. Kev-4B, fine-tuned on the same data, went from 80% to 96%.
Confidence. Below 0.90 confidence the Taktile model flagged 16% of transactions; sending those to a second model first left a person to review 4.4%, with 98.8% right or reviewed.
Why it matters here. It is an independent, domain-specific evaluation that names a catalogued model (Kev) and measures calibration as a workflow, not just accuracy. The data is synthetic and the study is Taktile’s own; no catalog scores change.