Type Safe's Jev introduced a new spin on large language models that has taken the AI world by storm. According to Vercel, "Jev was adopted faster than any other model in AI Gateway history." But clouds are forming. OpenAI is undoubtedly paying attention – and deciding what to do next. I wish all the best for Type Safe, but if they truly live up to their promises, then I'm concerned that OpenAI is well positioned to fast-follow – not only to replicate Jev's flagship product, but also to fold that capability into upcoming models and agents and offer some really useful new behavior that Jev is not positioned to reproduce.

OpenAI has for years used their LLMs as implicit classifiers; they just haven't trained them for general classification tasks and they haven't packaged up general classification as a stand-alone product. If OpenAI can replicate the training, then they will be able to replicate Jev in short order. Moreover, OpenAI is positioned to use this new classifier inside of their existing models and agents which can be useful for quick model selection, more efficient thinking, better security guardrails, and generally smarter, faster, and cheaper models.

The key factor deciding all of this is whether or not Type Safe has a moat. The biggest moat I see is in Type Safe's training data and training processes. My main assumption is that Jev is using something quite close to a conventional large language model. As evidence, Latent Space reports that many of the early clones are indeed LLM-based. Given a state and a set of questions, Jev's LLM generates a single token or, more accurately, generates the probability distribution over all possible next tokens. The logprobs associated with every possible token at that one step are then massaged into whatever format Jev needs to return. For a noul question, Jev looks at just two tokens, true and false, ignores everything else, and normalizes their probabilities into a single probability that the answer is true. For a choice question, Jev can be prompted with a list of possibilities and it looks at the relative probabilities of those tokens to build out the full distribution, selecting the highest as the winner. The choice pattern is pretty much what I blogged about way back in 2025 in Supercharging LLM Classifications with Logprobs, and even without fine-tuning it was already showing promise. I haven't thought hard about the score primitive, but I suspect it's a variant of the same pattern. Part of the premise of this post is that OpenAI might be poised to quickly take advantage of this idea, and this becomes clearer if you understand how. OpenAI has been using large language models implicitly as specialized classifiers since at least the introduction of tool calling. Back in early 2024 I wrote Tool Invocation – Demonstrating the Marvel of GPT's Flexibility, where I coaxed a GPT model into revealing exactly how it decides to call a tool. The following is what a chat session looks like internally. Here there is a user message, then an assistant response without a tool call followed by a user message with a tool call: I've color-coded the text to indicate token boundaries. If you haven't seen Chat ML before, it's the internal markup language that OpenAI introduced for organizing user-agent conversation prompts. <|im_start|> and <|im_end|> are reserved tokens that delimit the messages, and the first token after <|im_start|> identifies the speaker, either user or assistant. Right after <|im_start|>assistant, the ve...