HeadlinesBriefing favicon HeadlinesBriefing.com

Jevchat Turns Jev Into a Chatbot

Hacker News •
×

Jevchat turns Jev into a chatbot by asking one question at each step: Given the user's question and the reply written so far, which symbol comes next? The options are an alphabet plus an option to stop emitting. Jev returns a probability for each one, and the sampler draws the next symbol from that normalised distribution. Append, repeat, and stop when STOP is drawn. There are several alphabets (including truncated token lists) and sampling strategies available. The idea is for fun, the cost is somewhat impractical, and the results are hilarious. This was a Claude accelerated experiment. The user described the sampling algorithms, strategies, and so on, and Claude implemented them.

Setup: Put your Jev key in .env next to pyproject.toml (git-ignored) as JEV_API_KEY. TYPESAFE_API_KEY is also accepted. Values in .env win over exported ones, so editing the file is enough to switch keys. Commands: poetry run jevchat (interactive), poetry run jevchat ask "do people need water?", poetry run jevchat alphabets, poetry run jevchat bench.

The reply appears as it is sampled, in a panel with a live readout of the generation rate symbols/s, characters/s, milliseconds per API call, elapsed time, and the top few symbols Jev scored at the last step. Ctrl-C cancels. The first press stops generation once the in-flight request returns; a second press aborts immediately.

Modes: Two things are swappable: strategy (-s/--strategy) and alphabet (-a/--alphabet). Strategies include choice (one question over the whole alphabet), bisect (yes-no questions narrowing the group), buckets (alphabet split across many questions), and refine (buckets then a question over winners). Presentations include hypothesis (default) and symbol.