HeadlinesBriefing favicon HeadlinesBriefing.com

Jev-like LLM Wrapper with Vision Model Support

Hacker News •
×

A developer discovered Jev and self-hostable projects like Open Jev and Sem If, which use an old trick: reading LLM token probabilities. The method involves crafting prompts with state and multiple-choice questions, then sending Chat Completions requests with parameters like {"max_completion_tokens": 1,"logprobs": true,"top_logprobs": 20} to get single-token answers with alternative probabilities. This approach works with vision models too.

The author extended Jev's text/JSON format by adding an attachments field for images in local experiments. Their Python example captures webcam frames, sends base64 JPEGs, and evaluates person visibility, indoor/outdoor setting, and scene brightness. Using Gemma 4 12B on RTX 3090, they achieved ~1 frame per second with three questions per frame.

Open AI's gpt-6-luna yielded ~0.2 FPS due to separate connections. The flexibility allows changing conditions via plain text descriptions rather than specialized CV models.