Sep 24, 2026 Typed decisions with a probability for every option, in a single forward pass: matching Jev's accuracy and speed with an LLM. Marko Rosenmüller, Ph D Technical Lead AITL; DR: In this post, we show how an off-the-shelf LLM can make typed decisions in a single forward pass. This approach makes it possible to turn an LLM into a Jev-like decision model.

We evaluate the approach using GLM-5.3-Flash running on Privatemode. Using a benchmark constructed from public data sets, we show that this setup delivers results that are on par with Type Safe's Jev in terms of decision accuracy/correctness and speed. As a bonus, the setup with GLM-5.3-Flash enables typed decisions on images, which is not possible with Jev.

How it works Playground Benchmark Why typed decisions Much of what software asks an LLM is a decision. "Which team should handle this ticket?", or "Does this contract clause belong in the liability section?". In such cases, software typically requires that the LLM's response follows a certain format like JSON and that it comes from a pre-defined set like "yes" and "no". Given the right instructions, LLMs can typically already fulfill this reliably.

However, in the basic approach, speed and costs become an issue: For each decision, the LLM needs to write a whole JSON object, and a reasoning model may think for hundreds of tokens before that. Further, you also don't learn the confidence of the model (unless you explicitly ask it). All these aspects can matter a lot in practice and have so far prevented people from employing LLMs for decision making in high-volume/high-throughput scenarios.

Specialized decision models (or "System One" models) like Jev and Laya are designed to address this. You pass in a piece of state and a set of named options, and you get back the chosen option together with a confidence value (i.e., probability) for each one. Turning an LLM into a decision model Initially, we asked ourselves if an LLM could be turned into a decision model with Jev-like properties.

The short answer is: "yes". In the following, we show how it works. To understand our approach, it's important to understand how LLMs work: An LLM never writes text directly.

Given a prompt, an LLM outputs a probability distribution over its entire vocabulary of tokens. In text generation, in the simplest case, the token with the highest probability is selected as the next token. The selected token then is appended to the prompt and the whole process repeats.

As described above, this is costly and slow if you just want to set a few fields in a JSON object. Our core insight is that it's unnecessary to have the LLM predict the whole JSON object, as we already know its shape. We're only interested in the LLM's typed judgement for a given input.

We realized that it's possible to craft prompts so that we get the typed judgement in a single run of the LLM — with no fine-tuning, on the model exactly as it ships. This is the difference to Jev and Laya, which are models trained for the purpose. The basic steps are as follows: Number the options.

The state, the question, and the output options go into the prompt as JSON, with an index on every option. The instruction asks the model to an...