Compare · Explainer · Last verified 2026-10-16

Decision model vs LLM: which one should make the call?

Short answer

Use a decision model when the result is a yes/no, one option from a fixed list, or a level on a rubric that your code will threshold. It returns a probability for every option and bills input tokens only. Use an LLM (with structured output) when you need text, extracted fields, an explanation, or tool-call arguments. On the vendor pages we checked on 2026-10-16, LLM structured output gives you a schema-valid label but no per-option probability. Many production systems use both: the decision model gates or routes, then the LLM writes.

Pick a decision model for routing, triage, moderation gates, tool-call approval, and grading against a rubric, when you want a number to set a cutoff on.
Pick an LLM for anything that needs words: summaries, replies, extraction into your own JSON schema, or reasons a person will read.

Side-by-side

	Decision model API	LLM + structured output
Examples	OpenAI Decisions, Microsoft-Decision-1, Jev, Perplexity pplx-decider, Cloudflare Clef, H2O-Lightning-4B (open weights)	Claude JSON outputs / strict tool use, Gemini response_schema / text/x.enum
What comes back	A probability (yes/no), a chosen option plus a probability for every option, or a probability-weighted score over ordered levels, with a confidence value. [OpenAI, Microsoft, Perplexity docs, 2026-10-16]	Text or JSON that matches your schema, or one enum value. [Anthropic and Google docs, 2026-10-16]
Native probabilities	Yes, per option. [2026-10-16]	Not documented on Anthropic’s or Google’s structured-output pages. [absence noted 2026-10-16]
Billing	Input tokens only. Hosted list prices: Perplexity $0.02, Clef-flash $0.038, Jev and Microsoft-Decision-1 $0.042, OpenAI Decisions $0.10 per 1M input tokens; output free. [vendor pages, 2026-10-16]	Input and output tokens. Claude Haiku 5.5 $0.10/$0.50, Haiku 4.5 $1/$5, Sonnet 5.5 $2/$10 per 1M (input/output). Structured outputs add an injected system prompt to input. [Anthropic docs, 2026-10-16]
Many questions, one input	Yes, in one request (Perplexity allows 1–128 questions). Question text and, on some APIs, the shared state count again per question. [Perplexity, LLM Gateway docs, 2026-10-16] [per-question state billing on Perplexity is from its community forum, not the docs]	One schema with many fields; every field you ask for is generated output you pay for.
Explanations and tool calls	No. OpenAI says to use Structured Outputs for written explanations and function calling for tool calls. [OpenAI Decisions guide, 2026-10-16]	Yes. Strict tool use guarantees schema-valid tool inputs. [Anthropic docs, 2026-10-16]
Known failure modes	Refusals come back as a typed refusal answer (OpenAI). Scores shift with question wording and option order (Microsoft). [2026-10-16]	Refusals and max_tokens cut-offs can break the schema; enum capitalization isn’t guaranteed (Anthropic). JSON mode without a schema is only a hint (Google). [2026-10-16]
Speed signals	OpenAI says Decisions is about 10x faster than the Responses API [vendor claim, no benchmark published]. OpenRouter shows a 0.20 s p50 for Microsoft-Decision-1 [OpenRouter, 2026-10-16].	No like-for-like latency data from these vendors; measure on your workload.

Worked cost example: 1 million support tickets

Our arithmetic from list prices checked 2026-10-16. Assumptions: 1,000,000 tickets, 500 input tokens each, one three-way routing question. For the LLM rows we assume a 10-token JSON label as output and ignore the structured-output system prompt, whose size Anthropic does not publish. Tokenizers differ: Anthropic says its newer tokenizer (Claude 4.7 and later) produces about 30% more tokens for the same text, which would raise the Claude rows by roughly that much.

Option	Price per 1M tokens (input / output)	Tokens billed	Estimated total
Perplexity pplx-decider (decision)	$0.02 / $0	500M in	$10
Cloudflare Clef-flash (decision)	$0.038 / $0	500M in	$19
Jev or Microsoft-Decision-1 (decision)	$0.042 / $0	500M in	$21
OpenAI Decisions (decision)	$0.10 / $0	500M in	$50
Claude Haiku 5.5 (LLM)	$0.10 / $0.50	500M in + 10M out	$55
Claude Haiku 4.5 (LLM)	$1 / $5	500M in + 10M out	$550
Claude Sonnet 5.5 (LLM)	$2 / $10	500M in + 10M out	$1,100

What this shows: decision APIs are not automatically cheaper than the smallest LLMs. The price gap is large against mid-size models, but against Claude Haiku 5.5 it is small. Choose on output type first (probability vs text), then on price. Add a second or third question and the decision rows grow only by the question text, while the LLM grows by every extra output field.

Same question, both ways

Request shapes from each vendor’s docs, trimmed. We did not run these calls (no paid keys); they show the difference in what you get back.

Decision model (OpenAI Decisions, POST /v1/decisions). Response: a choice plus a probability for each department.

{
  "model": "gpt-6-luna",
  "input": "I was charged twice for my order.",
  "questions": [{
    "type": "choice",
    "name": "department",
    "instructions": "Which department should handle this complaint?",
    "choices": [
      {"value": "billing"}, {"value": "technical"},
      {"value": "shipping"}, {"value": "other"}
    ]
  }]
}

LLM with structured output (Claude Messages API, output_config.format). Response: JSON with one department string and no probabilities.

{
  "model": "claude-opus-5-5",
  "max_tokens": 50,
  "messages": [{"role": "user",
    "content": "Which department should handle: I was charged twice for my order."}],
  "output_config": {"format": {"type": "json_schema", "schema": {
    "type": "object",
    "properties": {"department": {"type": "string",
      "enum": ["billing", "technical", "shipping", "other"]}},
    "required": ["department"], "additionalProperties": false}}}
}

Anthropic’s own examples use claude-opus-5-5. For the cost rows above you would swap in a Haiku model ID from Anthropic’s models page.

Checklist

Need a number to threshold, rank, or escalate on? Decision model.
Need several judgments about the same input in one call? Decision model.
Need text a person reads, extracted fields, or tool arguments? LLM.
Need images? OpenAI Decisions, Perplexity, Clef, and H2O-Lightning-4B accept images; Jev and Microsoft-Decision-1 are text/JSON only.
Need to pin exact behavior? Prefer a versioned ID (Jev jev-1.13.0) or open weights (H2O-Lightning-4B, Strands Decider). OpenAI documents only the gpt-6-luna alias, and OpenRouter says Microsoft-Decision-1 weights are updated continually.
Consequential decisions about people? Both Microsoft and OpenAI say to keep a human in the loop. Use the model as decision support, not the final say.

FAQ

Is a decision model an LLM?

It is built from one but used differently. Perplexity describes its decision model as reading text and images the way a multimodal language model does, but returning typed answers with probabilities instead of writing text. OpenRouter says Microsoft-Decision-1 is post-trained from Qwen3.5-9B, and H2O-Lightning-4B is built on Qwen3.5-4B. None of them write replies, code, or explanations.

Can I get probabilities from an LLM with structured output?

Not from the structured-output features we checked. Anthropic’s structured outputs page and Google’s Gemini structured output page (both checked 2026-10-16) document schema-valid JSON or a single enum value, not per-option probabilities. A "confidence" field you add to your schema is text the model writes, not a calibrated probability.

Is a decision model always cheaper than an LLM?

Not always. Decision APIs bill input tokens only ($0.02 to $0.24 per 1M on the hosted APIs we track), but a small LLM like Claude Haiku 5.5 lists $0.10 input and $0.50 output per 1M tokens, so for short labels the bill can be close to OpenAI Decisions. The bigger differences are native probabilities, many questions per request, and no output parsing.

Jev vs LLM: what is Jev?

Jev is TypeSafe AI’s hosted decision model (POST /v1/systemone). As of 2026-10-16 it lists $0.042 per 1M input tokens with free output, accepts text only, and jev-latest points to jev-1.13.0. OpenRouter’s Armor tool-call guard uses it. OpenRouter also claims Jev 1.13 beat Claude Code’s use of Sonnet 5 on an internal tool-call test set, but published no test set, so treat that as unverified.

When should I use both?

When a request needs a gate and a written answer. Microsoft’s docs suggest using Microsoft-Decision-1 to route or gate a request, then sending approved requests to a generative model. OpenAI’s voice guide shows the same split: Decisions picks the action, and the Responses API writes the explanation.

Methodology

Every fact on this page was checked against the vendor’s own docs or pricing page on 2026-10-16. Items marked in amber are vendor claims or unofficial sources we could not confirm. Cost figures are our arithmetic from list prices, before regional premiums, batch discounts, or caching. See our methodology.