Validating LLM Output Against Schemas: JSON Schema, Zod, Retries

An LLM returning valid JSON most of the time is not an API contract — it's a hope. Real applications parse model output into types, feed it into databases, and hand it to other systems, so a single malformed response can crash a whole pipeline. Validation turns that from a crash into a recoverable event.

The pattern is simple: define the shape you expect, check every response against it, and handle failures deliberately. This guide covers the tooling, the retry strategy, and the mistakes that turn validation into its own source of cost.

Define the contract first

Write the schema before you write the prompt. The schema is the source of truth: it goes into your prompt as the requested format, into your validation layer as the check, and into your error handling as the reference for what went wrong. Two popular approaches: JSON Schema for language-agnostic validation, and TypeScript-first validators like Zod that give you typed output for free.

JSON Schema vs Zod-style validators

• JSON Schema: portable, works in any language, easy to embed in prompts verbatim.
• Zod (and similar): type-safe, concise, and it derives TS types from the schema.
• Many stacks use both: Zod for internal validation, JSON Schema rendered into the prompt.
• Some provider SDKs now ship structured output modes that constrain generation to a schema.

Even with structured output modes, validate on your side. Provider constraints reduce but don't eliminate malformed responses, and you should never trust a model's output because the system promised to shape it.

Handling malformed output

1. Catch parse and schema errors separately so you know which failed.
2. Log the raw output and the error — undebugable failures are expensive failures.
3. Try a repair pass: ask the model to fix its own JSON with the error message.
4. Retry the full call with a tightened prompt if repair fails.
5. Cap total attempts and fail loud instead of looping forever.

Retry budgets that stay sane

Retries multiply cost and latency, so budget them explicitly. A common policy is one automatic repair attempt plus one full retry — beyond that, surface an error to the caller rather than hammering the provider. Track your retry rate; a rising rate is usually a prompt or schema problem, not bad luck.

Test your pipeline with validation failures

Don't discover malformed-output handling in production. Include seeded bad responses in your tests — truncated JSON, wrong types, extra keys, empty arrays — and verify your retry and fallback paths handle each one without crashing and without runaway spend.

FAQ