Researchers tested whether Jev and Luna could solve identity resolution problems. While both models performed well initially (88.8–97.3% accuracy), combining Jev with Opus achieved frontier accuracy at over 4x lower costs. Luna initially outperforms Jev, but the gap disappears in harder cases.
Jev's confidence scores are better calibrated, allowing it to excel in hybrid configurations. Frontier models remain more accurate than decision models, but decision models are up to 90x cheaper and over 10x faster, costing $0.50 to $0.10 per 1,000 identities and answering in 0.15 seconds. However, they trail Opus accuracy by about 6 percentage points on difficult benchmarks.
Identity resolution maps data records to real people, essential for customer outreach and personalization. The process faces challenges with data errors and over-merging, where identities incorrectly correspond to multiple people. Human sampling is slow, averaging 50 identities per hour, and small samples provide poor accuracy estimates.
The article details error types including typos, misread characters, and nicknames that complicate the matching process.
Source: Hacker News · Summarized by HeadlinesBriefing