HeadlinesBriefing favicon HeadlinesBriefing.com

Les Sorties LLM Structurées peuvent Retourner un JSON Valide mais Erroné

Towards Data Science •
×

After enabling Structured Outputs for parsing payment confirmations, I noticed 2-3% of transactions had correct amounts and senders but wrong dates. The issue: messages never mentioned dates, yet the model filled transaction_date anyway—often with the job run date. The schema required the field, so the model invented values to satisfy type checking.

This revealed a quieter failure mode: valid JSON masking incorrect data. Before Structured Outputs, reliability meant regex parsers and retry loops. Now, with OpenAI's Python SDK and Pydantic models, malformed JSON errors disappear.

But the schema doesn't know when data is missing. The fix requires mental shifts: making fields nullable so models return null instead of guessing, and distinguishing extraction (what's explicitly stated) from inference (what's implied). Nullable fields prevent invented values, while also enabling better evidence tracking for what was actually read versus pattern-matched.