Language models are often asked to generate an answer token by token when software only needs to choose from a bounded set of options. A banking customer who says they were charged twice for the same transfer needs to be routed to a support team, and generating a JSON label adds unnecessary text generation and parsing to that decision.
This article tests seven open "Jev" decision models against a Qwen JSON-generation baseline. Instead of generating an answer, Jev-style models score the available candidates and return a typed decision. The comparison covers banking requests with both stable route lists and changing candidate menus, which matters because agent tools and workflow options often change at runtime.
Kev led on accuracy, reaching 98% on both fixed routes and changing candidates. Verdict reached 89% on changing candidates, compared with 66% for the LLM baseline Qwen3-0.6B, which scored 83% on fixed routes. JSON generation still beat the three Qwen readout configurations. Both Kev and Verdict generated zero answer tokens, versus about 10 per request for Qwen JSON generation.
Recorded median request latency was 125–130 ms for Kev and 17–18 ms for Verdict, compared with 331–357 ms for the LLM baseline across the two panels. The starter kit and benchmark are available for readers who want to try a decision model and compare JSON generation with a direct readout.
Source: Towards Data Science · Summarized by HeadlinesBriefing