AI Agent Guide Interactive
Part 1 · LLM Engineering Foundations 3 interactive labs · Zod schemas, native constraints & retry pipelines

Prompt Engineering in Practice: Structured Outputs & Validation

In the previous chapter we asked the model to "output JSON only." That helps, but it hits a hard limit: prompts are probabilistic nudges, not deterministic guarantees. In production, your code needs its own defense layer to detect, repair, and retry bad outputs.

30-Second Version · 3 Takeaways Labs simulate classic LLM failure modes
  1. 01 Prompting alone for JSON will eventually break in production; always back it up with schema validation.
  2. 02 Define one Zod schema as your single source of truth for TypeScript types, runtime checks, and field-level errors.
  3. 03 Verify endpoint support before relying on output_config.format; fall back to Tool Use or local retries when unsupported.

Why Prompting Alone Always Breaks

LLMs sample tokens probabilistically. Telling the model to "return only a JSON array" works most of the time because it matches patterns in training data, not because the model obeys hard syntax rules. Any of these three failure modes can crash an unguarded parser:

Drift 01 · Chatty Preamble
Conversational filler before JSON

The model opens with a polite acknowledgment or wraps output in Markdown fences, causing JSON.parse to fail on character zero.

Sure, here is the code review: ```json {"summary":"..."} ```
Drift 02 · Schema Drift
Silent field or type mutations

You asked for a positive integer line, but the model emits line_number or an out-of-range score, breaking downstream assumptions.

{"line_number": 12, "score": 120}
Drift 03 · Truncated Output
Long payloads hit max_tokens

Generation stops mid-array at the token ceiling, leaving unclosed brackets ]} and an unparseable payload.

{"issues":[{"type":"bug","line":49

Endless prompt tweaking will not eliminate these failures. The reliable architecture is two lines of defense: reduce the odds of drift at the prompt or API layer, and enforce deterministic extraction, validation, and retries in application code. Drop either layer and your pipeline stays brittle.

Defense 1 · Prompt / Native Constraints
Lower the probability of format drift

Push format constraints upstream using system instructions, few-shot examples, or server-side output_config.format.

→
Defense 2 · Code-Level Schema Validation
Deterministically catch, repair, and retry

Chain extractJson() + JSON.parse() + Zod safeParse() to turn noisy text back into type-safe objects.

Make the Contract Explicit: Zod Schema

Before writing any parser or retry loop, codify the expected output shape. In TypeScript, Zod gives you compile-time types, runtime validation, and field-level error diagnostics from a single schema. Prefer safeParse() over parse() in production so malformed responses become structured errors rather than uncaught exceptions.

Zod Method Purpose Example
z.object({}) Define object shape and required keys z.object({ summary: z.string() })
z.enum([...]) Restrict a string to a closed set of values z.enum(["bug", "style", "performance", "security"])
z.number().int().positive() Enforce a positive integer (> 0) Ideal for source line numbers line; blocks negatives and floats
z.optional() / z.catch() Allow missing fields or fall back to safe defaults on failure z.array(ReviewIssue).catch([])
Inspect ReviewResult schema definition & safeParse usage
review-schema.ts
import { z } from "zod";

const ReviewIssue = z.object({
  type: z.enum(["bug", "style", "performance", "security"]),
  line: z.number().int().positive(),
  message: z.string().min(1),
  suggestion: z.string().optional(),
});

const ReviewResult = z.object({
  summary: z.string(),
  issues: z.array(ReviewIssue),
  score: z.number().min(0).max(100),
});

type ReviewResult = z.infer<typeof ReviewResult>;

// Prefer safeParse() in production: returns structured issues instead of throwing
const parsed = ReviewResult.safeParse(rawOutput);
if (!parsed.success) {
  console.error("Validation failed:", parsed.error.issues);
} else {
  const result: ReviewResult = parsed.data;
}
Interactive Lab 01 · Code-Layer Defense X-Ray

JSON Extraction, Syntax Parsing & Zod Validation

Pick a failure preset or edit the raw model output on the left to watch extractJson → JSON.parse → safeParse catch issues stage by stage.

Presets:
Raw Model Output (editable) 0 chars
1. extractJson(raw) Extraction —
2. JSON.parse(jsonStr) Syntax Check —
3. ReviewResult.safeParse() Schema Check —

Going Further: output_config.format & Compatible Endpoints

Zod validates data after generation. When your provider supports native structured outputs, you can pass output_config.format via the Anthropic SDK's client.messages.parse() so the server enforces your JSON Schema during decoding (Anthropic Structured Outputs docs).

Follow a clear priority order: use native strict response formats whenever the provider genuinely supports them, and treat manual parsing plus retries as the portable safety net. Watch out for one subtle trap: being "Anthropic API compatible" does not mean every advanced parameter is honored.

Take Alibaba Cloud Model Studio's Anthropic-compatible endpoint as an example. Its official documentation separates behavior by model: DeepSeek and GLM strictly enforce the JSON Schema, while Qwen silently downgrades to plain JSON mode—guaranteeing valid JSON syntax without enforcing schema constraints. Likewise, DeepSeek's Anthropic-compatible API docs note that output_config only supports the effort parameter.

Rule for Compatible Endpoints
Never assume the server enforced your schema just because the request did not return HTTP 400. Whenever you route across multiple models or third-party compatible endpoints, keep local Zod validation and retries in place.
Inspect full examples: bailian-output-format.ts & parseOutput() retry wrapper
structured-output-sync.ts (live-synced with Lab 02 controls)
import Anthropic from "@anthropic-ai/sdk";
import { zodOutputFormat } from "@anthropic-ai/sdk/helpers/zod";

const client = new Anthropic();
const MODEL = process.env.MODEL ?? "qwen-plus";

// 1. Native output_config.format (first choice when strict schema is supported)
const response = await client.messages.parse({
  model: MODEL,
  max_tokens: 2048,
  messages: [{ role: "user", content: `Review the following code:\n${code}` }],
  output_config: {
    format: zodOutputFormat(ReviewResult),
  },
});

// 2. Portable fallback: extract + Zod validation + retry (works on any text model)
const result = await parseOutput(
  [{ role: "user", content: `Review this code: ${code}` }],
  ReviewResult,
  { maxRetries: 2, withFeedback: true }
);

Runnable source code: bailian-output-format.ts · parse-with-retry.ts

Interactive Lab 02 · Endpoint Quirks & Retry Pipeline

Compatible Endpoint Simulation & Feedback-Driven Retries

Make a quick prediction first, then toggle providers and retry settings to see how parseOutput() feeds Zod errors back into the model for self-repair.

Predict First You call qwen-plus through Model Studio's Anthropic-compatible endpoint with output_config.format, and the model omits the required score field. What happens?
Provider & Endpoint Capability
Local Retry Policy (syncs code above)

Streaming JSON & Tool Use as an Alternative

Waiting for a full code review to finish before showing anything leaves users staring at a blank screen. The Anthropic SDK supports passing output_config.format into messages.stream(): stream text deltas to the UI for live progress, then read the validated parsed_output from stream.finalMessage() once the stream closes.

Keep one boundary in mind: parsed_output is only available after the entire stream completes. If your UI must render an interactive card the instant each individual issue is found, consider NDJSON (Newline Delimited JSON)—instruct the model to emit one standalone JSON object per line, then split on newlines and JSON.parse() incrementally. Because NDJSON is more sensitive to prompt phrasing, reserve it for true item-by-item streaming.

Before native structured outputs became widespread, teams relied on Tool Use / Function Calling to enforce structure (Anthropic Define Tools · OpenAI Structured Outputs): define a virtual tool such as submit_review, force its invocation with tool_choice, and validate tool_use.input with Zod. That exact pattern remains a cornerstone when we build agents later in the book.

Inspect streaming output_config.format & forced Tool Use examples
streaming-and-tool-use.ts
// Option A: Streaming output_config.format (stream text to UI, read parsed_output at end)
const stream = client.messages.stream({
  model: MODEL,
  max_tokens: 2048,
  messages: [{ role: "user", content: prompt }],
  output_config: { format: zodOutputFormat(ReviewResult) },
});

stream.on("text", (text) => process.stdout.write(text));
const message = await stream.finalMessage();
const finalResult = message.parsed_output;

// Option B: Virtual tool to carry structured output
const res = await client.messages.create({
  model: MODEL,
  max_tokens: 2048,
  tools: [{
    name: "submit_review",
    description: "Submit structured code review findings",
    input_schema: { type: "object", properties: { /* ... */ }, required: ["summary", "issues", "score"] },
  }],
  tool_choice: { type: "tool", name: "submit_review" },
  messages: [{ role: "user", content: `Review this code: ${code}` }],
});
const toolUse = res.content.find((c) => c.type === "tool_use");
if (toolUse?.type === "tool_use") {
  const checked = ReviewResult.parse(toolUse.input);
}

Full streaming source: bailian-streaming-output-format.ts

Interactive Lab 03 · Streaming Consumption

Whole-JSON Stream Wait vs. Incremental NDJSON Rendering

Start the simulated stream to compare when the client can turn incoming tokens into structured UI cards under each protocol.

Raw Token Stream (SSE text delta) Ready
Click "▶ Start Stream" to watch tokens arrive...
parsed_output Consumption Status 0 items parsed
Waiting for stream...

Which Path to Choose: Engineering Decision Matrix

You do not need a single mechanism across every feature. Pick the strategy that matches your provider's capabilities and your UI's consumption pattern:

Provider & Product Scenario Recommended Approach Why It Fits
Provider strictly enforces output_config.format / response_format Native Structured Outputs Enforces schema during server-side decoding for the highest first-pass success rate
No native format constraint, but Tool Use is supported Tool Use + Zod Validation Forces structured arguments via tool_choice and aligns with agent toolchains
Endpoint only guarantees valid JSON, or you switch models frequently JSON Extraction + Zod + Feedback Retries Most portable defense layer across any text-completion backend
Long generation where users want live progress and code needs one final object Streaming output_config.format Combines responsive streaming UX with a type-safe final parsed_output
Array output where the UI must render each item the moment it is generated Incremental NDJSON Parsing Lightweight protocol that enables zero-wait, item-by-item object consumption

Once output determinism is locked down with native constraints, schema validation, and feedback retries, the next bottleneck shifts to the input side: what belongs in the prompt window, what should stay out, and how to stop multi-turn sessions from degrading. That is the focus of the next chapter, Context Engineering.