Ploy announced that its production AI agent now runs on GPT-5.6 Sol, OpenAI's latest flagship model released this morning. In head‑to‑head tests against Claude Opus 4.8, GPT‑5.6 completed website builds 2.2 × faster and at 27 % lower cost, while achieving a higher visual score (0.970 vs 0.936). The agent, which plans pages, reads codebases, writes components, generates images, and validates output, required extensive engineering work to migrate from the Claude stack.

The migration exposed hidden dependencies in Ploy’s evaluation harness. Tool‑call budgets tuned for Opus’s sequential calls were exceeded by GPT‑5.6’s parallel pattern, and the harness lacked support for batched file reads, causing roughly a third of initial failures. After fixing the harness, the team measured mean build cost of $2.22 and wall‑clock time of 3 min 42 s for GPT‑5.6, compared with $3.06 and 8 min for Opus.

A key technical challenge was GPT‑5.6’s tendency to emit all optional parameters in function calls, inserting default values that led to empty file reads and unnecessary tool calls. Ploy solved this by transforming the schema at the provider boundary: optional fields become required‑nullable, allowing the model to explicitly return nulls, which are stripped before validation. This reduced empty reads from 52 % to 0 % and cut tool calls by ~30 %.

Prompt caching also differed between providers. Claude’s cache shared a 29K‑token prefix across all workspaces with 92‑96 % hit rates, while OpenAI’s cache required custom breakpoints. Adjusting the cache configuration eliminated the perceived cost premium for GPT‑5.6. The effort demonstrates that moving to a new LLM involves more than model selection; it demands revisiting harness assumptions, tool schemas, and caching strategies to realize performance gains.