HeadlinesBriefing favicon HeadlinesBriefing.com

200‑Line Meta‑Harness Beats 5 Tasks on Islo

Hacker News •
×

Yossi Eliaz released a 200‑line proof‑of‑concept that demonstrates a self‑optimising meta‑harness loop on the Islo sandbox. The orchestrator snapshots a reproducible eval environment, forks a candidate per task, and harvests raw execution traces for a proposer agent to rewrite the harness. In four proposer steps the system climbs from a zero‑score baseline to a perfect 5/5 pass rate on modest hardware.

Yoonho Lee’s framing argues that diagnostic context, not summary statistics, is the real bottleneck for LLM optimisation. Islo’s primitives—snapshot save, use --snapshot for parallel forks, and logs for durable traces—provide exactly what the meta‑harness requires. The POC wires these primitives with a deterministic Python simulator, allowing the loop to run in seconds without consuming Claude credits. The design keeps agent usage cheap while preserving deterministic debugging for developers.

The five‑task suite—FizzBuzz, primes, list reverse, sum‑of‑evens, palindrome check—shows monotonic improvement: 0/5 → 2/5 → 3/5 → 4/5 → 5/5, staying well below the ten‑iteration cap. A surprising cross‑task fix occurs when a fizz‑buzz hint containing “inclusive” also resolves the sum‑evens task, illustrating the value of full‑trace visibility. The same orchestrator can swap to a real Islo‑Claude backend with minimal changes.