I made Lathoa for kids aged 10 to 14. A robot named Errol solves math problems step by step, but one step is intentionally wrong. The child must find the error and explain why it's wrong to earn XP.
Sometimes nothing is wrong, so guessing 'there's a mistake' doesn't work—explanations are required. Speed also affects scoring. There's no direct LLM chat; Lathoa uses stable harnessing with evaluation steps to catch inconsistencies and prompt injections.
You can play a case on the homepage without signing up. Getting an LLM to be wrong on purpose is hard: half the time it gives the right answer and calls it wrong, or flags a correct step as a mistake. So every case is pre-checked.
Where possible, a plain arithmetic check redoes the math exactly. A second model solves the problem without seeing Errol's work. If they disagree, the case is discarded.
The weakness is that the second model might make the same mistake as the first. The arithmetic check helps, but only works for English cases so far—German and Greek use commas for decimals, and parsing isn't fixed yet. I wonder: does finding someone else's mistake teach more than solving the problem yourself? I'm not sure, and I'd like input from educators.
Source: Hacker News · Summarized by HeadlinesBriefing