HeadlinesBriefing favicon HeadlinesBriefing.com

Implement Structured Output with Local LLMs

Towards Data Science •
×

Local LLMs keep sensitive data in-house and reduce cloud dependence. But free‑form text is hard to process in larger workflows, so we use Structured Output to enforce a predictable shape.

The example uses Gemma 4 as a local LLM, Ollama as the serving runtime, and Pydantic to define and validate the output schema. In a smart‑home scenario, the user asks, "Should the dishwasher run now or later?" The LLM transforms household notes into a sanitized object that contains only scheduling facts.

We first draft Pydantic models for the desired structure: a clock time type, a device schedule entry, and a full scheduling context. Ollama is then configured to return JSON that matches this schema. The call uses the format argument to constrain generation and model_validate_json to parse the response.

The result is a clean Python object with devices to schedule. However, the model mistakenly includes the robot vacuum, showing that Structured Output enforces shape but not content correctness. This illustrates the need for careful prompt engineering and post‑processing checks.