Decode vs Decide
With the AI SDK, asking an LLM for structured output and asking Jev for an evaluation look almost identical: text and a type definition go in, a typed object comes out.
What happens between those two lines is very different. Scroll through both calls on the same input and watch.
"I was charged twice. Please refund the duplicate."
The LLM writes it
A typical structured-output call.
You pass a prompt and a zod schema, and you get back a typed output. Nothing at the call site tells you how that object got made.
The schema becomes a writing rule.
The SDK converts your zod schema to JSON Schema and sends it with the prompt. The provider uses it to limit which tokens the model is allowed to write next.
The model reads everything once.
Prompt and schema go through the model in one pass. This is where your instructions are understood, and it's the same kind of understanding Jev does.
Then it writes, one token per pass.
Left alone, the model would rather start with "Here". The schema blocks every token that can't begin valid JSON, so it writes {".
Structured output doesn't stop the model from generating text. It only narrows which text is allowed.
Most tokens are already decided.
department and ":" are the only legal next tokens, so they're forced. You still pay for the pass that writes each one.
Token 4 is the first real decision: billing at 0.91. The model samples a word and the 0.91 is thrown away.
Seven more passes to close the object.
requestsRefund takes three forced tokens. true is sampled at 0.96, and that number is discarded too. Then }.
The SDK turns the text back into data.
The model returned a string. The SDK parses it, validates it with zod, and hands you a typed object. If validation fails, the call throws.
11 passes and 11 billed output tokens. Two of those tokens were decisions.
import { generateText, Output } from 'ai';import { anthropic } from '@ai-sdk/anthropic';import { z } from 'zod';const { output } = await generateText({model: anthropic('claude-haiku-4-5'),output: Output.object({schema: z.object({department: z.enum(['billing', 'support']),requestsRefund: z.boolean(),}),}),prompt: 'I was charged twice. Please refund the duplicate.',});output.department; // 'billing'output.requestsRefund; // true
Jev picks it
The same shape, with questions instead of a schema.
You pass state (the text) and questions (what you want to know, in English), and you get back typed answers.
The questions are the answer space.
Every allowed answer is listed up front: billing or support, true or false. A boolean question maps to TypeSafe's "noul" type, the probability that a statement is true.
It reads everything once.
The state and questions go through the model in one pass. This input is the only thing TypeSafe bills for, at $0.042 per million tokens.
That same pass scores every answer.
Nothing is written. The model puts a probability on every option of every question at once. Adding a question adds input, not passes.
It returns what you defined, with the probabilities.
Because no text came back, there's nothing to parse. The SDK only checks that the probabilities add up to about 1 (they're rounded to two decimals).
One pass. No billed output. Choosing a threshold, like auto-refund above 0.9, is left to your code.
import { experimental_evaluate as evaluate } from 'ai';import { typeSafeAi } from '@ai-sdk/typesafe-ai';const { answers } = await evaluate({model: typeSafeAi.evaluationModel('jev-latest'),state: 'I was charged twice. Please refund the duplicate.',questions: {department: {type: 'choice',instructions: 'Which team should handle this?',criteria: {billing: 'Charges and refunds',support: 'Other requests',},},requestsRefund: {type: 'boolean',instructions: 'Is the customer requesting money back?',},},});answers.department.choice; // 'billing'answers.requestsRefund.probability; // 0.97
The same moment, side by side
After one pass
Both models have read the input. The LLM has written {". Jev already has every answer.
After eleven passes
The LLM has finished writing the JSON. Jev returned ten passes ago.
What you're holding
The same two answers. The LLM gives you one sample of each. Jev gives you the pick and how sure it was.
Neither approach guarantees a correct answer. "Can't hallucinate" only means Jev's answer is always one of your options, and it can still pick the wrong one. The trade is flexibility for speed and cost.
| LLM + structured output | Jev evaluation | |
|---|---|---|
| How the answer is made | Written token by token, with the schema blocking invalid tokens | Every option scored in one pass |
| Passes in this example | 11 | 1 |
| Output you pay for | 11 tokens, mostly structure the schema already fixed | None. Input only |
| What comes back | Sampled values | Pick plus probabilities |
| What it can return | Anything a schema can describe, including free text | Choice (up to 255 options), score (2–10 levels), or boolean |
Notes
The code is real, written against ai@7.0.111 and @ai-sdk/typesafe-ai@3.0.4. Token splits, pass counts, and probabilities are illustrative, and model internals are simplified. Whether the schema is enforced during writing depends on the provider; without that, the SDK relies on the prompt and validates afterward. TypeSafe hasn't published Jev's architecture. "One pass" is from their docs ("generates all outputs in a single query") and matches independent latency tests.
Sources: TypeSafe docs, models and pricing, jev-1.13 known weaknesses, AI SDK TypeSafe provider, AI SDK docs.