AI decision models like Jev, and when to use one instead of an LLM

An AI decision model reads text, and sometimes images, and answers questions you define in advance with typed values instead of generated text. Each answer is a yes/no probability, one option from a list you supply, or a score on a scale, and it comes with a probability for every option. TypeSafe AI made the category popular with Jev, launched in early access on 15 September 2026. The launch post reached 1,989 points on Hacker News, and by our count at least 20 more stories about decision models passed 100 points there over the next three weeks, Cloudflare’s Clef launch among them at 641.¹

Use a decision model when software needs the same bounded judgement many times, within about a second, and the categories change too often to retrain a classifier. Use an LLM with structured output when the answer needs words, arithmetic, dates or several steps of reasoning. Fine-tune a classifier when the categories are stable, you have thousands of labelled examples and answers must arrive in tens of milliseconds.

This post explains how decision models differ from autoregressive LLMs, lists the options as of 7 October 2026 and summarises four independent tests, then gives a decision rule, patterns and a checklist. We haven’t benchmarked these models ourselves. Every figure links to its source, and vendor claims are marked as such.

How an AI decision model differs from an LLM

An autoregressive LLM writes its answer one token at a time. A JSON schema can stop it writing anything outside your labels, but the label is still decoded token by token, and a reasoning model thinks first. Output tokens also cost more: GPT-6 Luna and Claude Haiku 5.5 both charge $0.10 per million input tokens and $0.50 per million output tokens.²³

A decision model doesn’t write at all. The published designs share a shape: the input and the candidate answers go through a transformer once, and a small head scores each candidate. Cloudflare says Clef runs its frozen Qwen backbone in a prefill-only pass and scores the valid choices in parallel.⁴ Strands Decider 2B swaps the language-model head of Qwen3.5-2B for a pointer head of just over a million parameters, which scores the hidden state at each option against the hidden state at the answer position.⁵

TypeSafe hasn’t published Jev’s architecture. Its launch post describes Jev as “unstructured state in, typed probabilistic decisions out”, and its docs say Jev reads the input once and evaluates every question against it in parallel.⁶⁷ Since the work is mostly reading, the vendors charge for input. Jev costs $0.042 per million input tokens with free output, and OpenAI’s Decisions API bills input tokens only.⁸

The hosted APIs share three kinds of question. A yes/no question returns the probability that a statement is true; TypeSafe calls it a noul, short for Bernoulli, and OpenAI calls it a predicate.⁹⁸ A choice question returns one of your options with a probability for each, and a score question returns a probability-weighted position on ordered levels you describe. The answer can’t fall outside your options, which TypeSafe presents as a model that never makes type errors.⁶ Whether the answer is right is what the tests below measure.

TypeSafe trains Jev with an unpublished method it calls Reinforcement Learning for Calibrated Decisions (RLCD). The goal is that, over many predictions, answers given 0.8 are right about 80% of the time.¹⁰ The closest published method, according to Sebastian Raschka, is RLCR, which adds a Brier score to the correctness reward; he notes there’s no known link between the two.¹¹¹² Cloudflare trained Clef with cross-entropy plus a Brier loss.⁴

A classic classifier differs in where the labels live. A fine-tuned ModernBERT learns a fixed label set from labelled examples, so a new category means new labels and another training run.¹³ A decision model reads the options and their descriptions in each request, so one model can route tickets in one call and grade search results in the next.

Decision models give up text. They can’t write a reply or explain an answer, and TypeSafe suggests extracting values by having Jev choose among candidates your code has found. Its list of known weak spots for jev-1.13, last reviewed on 2 October, adds arithmetic and counting, comparing dates, multi-hop instructions, long inputs padded with irrelevant detail, adversarial content and, in some cases, a lean towards the first option in a choice.¹⁴ The Strands team says the single parallel pass makes its model much worse than reasoning models at complex problems.⁵

Jev, OpenAI’s Decisions API, Cloudflare Clef and the open models

These are the decision models available as of 7 October 2026, with two small LLMs and a fine-tunable encoder for comparison. Prices are list prices.

Jev. TypeSafe is admitting developers from a waitlist. The current model, jev-1.13.0, takes up to 64K tokens per request (32K for the input plus the longest question), reads text only and allows up to 255 options in a choice.⁷⁶ The launch post claims end-to-end responses in 70 to 500 ms. It traces the 193.6-times speed and 444.6-times cost advantages on TypeSafe’s home page to four in-house workflow evals, which score each model against the average answer of GPT-6 Astra and Claude Fable 5.1 instead of labelled ground truth.⁶ Against GPT-5.6 Terra, AY Automate measured 3.6 times the speed and 40 to 49 times lower cost per decision.¹⁶

OpenAI’s Decisions API has been in public beta since 6 October.² It runs only on gpt-6-luna, takes text and inline base64 images, and can return a refusal in place of an answer. OpenAI says it answers about 10 times faster than the Responses API, without saying how that was measured.⁸

Cloudflare’s Clef and Clef-flash came out on 1 October as open weights, built on Qwen3.8-27B and Qwen3.5-9B. Both take images, have a 65,536-token context and, Cloudflare says, are fully compatible with Jev’s API.¹⁷⁴ Cloudflare’s performance figures are its own. It says Clef leads the Jev Decision Index, a community leaderboard on Hugging Face,¹⁸ and beats Jev in three of four TypeSafe workflow evals, by 0.3 to 2.9 points, while losing the fourth by 3.1. It reports median latencies of 209 ms for Clef and 39 ms for Clef-flash, against 524 ms for Jev.⁴

Strands Decider 2B was published on 1 October by the Strands Agents team, with Apache 2.0 weights, training data and scripts. The team reports median latencies of about 115 ms on an Nvidia RTX 3090 and about 153 ms for small tasks on an M3 MacBook, and third place of 33 models in its size class on JevBench’s public set.⁵

llama.cpp gained a /v1/systemone server endpoint for decision models in release v0.6.0 on 5 October. It serves Clef, with image input, and the open Jev-style models laya, julia-1, lev, openjev, kev and nimble, and build b11447 added pplx-decider on 6 October.¹⁹ Strands Decider wasn’t in llama.cpp’s release notes up to 7 October.

Jev vs LLM vs classifier in published tests

Four independent tests published by 7 October compare decision models with LLMs or classifiers on labelled data. Each group of rows is one test, so compare rows inside a group. Costs are the published cost per 1,000 decisions multiplied by 1,000.

1. Red Hat ran the classifiers on a MacBook Pro M1 CPU. Jev’s figure includes the round trip from a machine in the UK to TypeSafe’s servers in the US, which Red Hat says adds at least about 56 ms. The article’s detailed table gives DeBERTa a median of 80.4 ms and a P95 of 276.4 ms, against 54.1 ms in its summary table.
2. Median across all three tasks in the study, through OpenRouter from one laptop.
3. Time to score all 25,000 reviews, which measures throughput. ModernBERT ran on a DGX Spark after 3 hours of fine-tuning, and Jev ran through TypeSafe’s API.

Red Hat’s two tests went opposite ways. The DeBERTa model, running on a laptop CPU, beat Jev by 2.7 points on prompt injection, and Jev beat the 125M-parameter granite classifier by 5.9 points on content safety.²⁰ The authors concluded that pre-trained classifiers remain very competitive, and that apart from the compact Laya model, the decision models they tested weren’t faster, cheaper or more accurate than an LLM judge.

Within the category, models differ widely. On the same 77 routing cases, Clef and Jev were 20.8 points apart. The pilot is small, and Jev’s 95% interval runs from 64.6% to 83.6%, but Clef also got 6 cases right that GPT-6 Astra missed while Astra got none that Clef missed. GPT-6 Luna got one more case right through the Decisions API than through chat, and answered in 0.31 s instead of 1.07 s.²²

Against small LLMs, Jev was about as accurate and much faster and cheaper. In AY Automate’s test it was 2.0 to 3.6 times faster than the four LLMs at the median, and 4.7 to 7.5 times cheaper per decision than the two cheapest. On 77-way routing only GPT-5.6 Terra was reliably more accurate, by 5.2 points.¹⁶

Per-token prices don’t settle the cost per decision. In S Anand’s pilot, GPT-6 Luna through chat cost less per decision than Jev or the Decisions API. The decision models were also sent label descriptions, and Jev read 1,706 input tokens per case on average against 485 for Luna through chat.²² Tokenizers differ too. AY Automate found that Jev counted 360 input tokens for a prompt GPT-5.6 Terra counted as 153.¹⁶

How the published tests were run

We didn’t run any of these tests. These are their setups, and what each leaves out.

• Red Hat (Rob Geada, Mac Misiura and Shelton Cyril; updated 5 October) used two class-balanced yes/no benchmarks from EvalHub’s NeMo Guardrails library and doesn’t give their sizes. Latency is the round trip to a NeMo Guardrails server. The classifiers ran on a laptop CPU, the LLM judges on a GPU cluster and Jev over TypeSafe’s API. The experiments were in English, and two of the article’s tables disagree on some latencies.
• AY Automate (Adel Dahani) ran 791 labelled decisions on 19 September. There were 160 for 8-way routing, 231 for 77-way routing and 400 from deepset’s prompt-injection set, all with labels and no descriptions. The LLMs ran at temperature 0 with minimal reasoning. Every call went through OpenRouter from one laptop, and the 95% intervals are 6 to 13 points wide.
• S Anand’s pilot, in the version committed on 7 October, used one case from each of BANKING77’s 77 categories, scored by exact match. Decision models got label descriptions as well as names, chat models names only. Latency is end-to-end from one machine.
• Sebastian Raschka scored the 25,000-review IMDb test set, with ModernBERT fine-tuned on the 25,000-review training set. He couldn’t check whether IMDb was in Jev’s training data, and AY Automate couldn’t check its public datasets either.

Vendor-reported figures appear only in the options section, labelled there. We found no independent classification results for Claude Haiku 5.5, which came out on 7 October.

What the confidence numbers are worth

The probability on every answer is a large part of the appeal, because code can act on confident answers and escalate the rest. The published results on how far to trust it are mixed.

In S Anand’s pilot, Jev’s mean confidence was 87.4% against 75.3% accuracy, and the Decisions API’s was 85.5% against 79.2%. Clef and Clef-flash were slightly underconfident, at 90.6% and 92.8% against 96.1% and 94.8%, and had the pilot’s best Brier scores.²² Prompted LLMs varied as much. Claude Opus 5 and Claude Fable 5.1 averaged within a point of their accuracy, while GPT-6 Luna through chat was 14 points overconfident.

A model can be overconfident on average and still rank its answers well. In AY Automate’s test, keeping only Jev’s answers at 0.90 confidence or above raised accuracy from 83.8% to 95.5% on 8-way routing, with 70% of items answered. A cascade that sent everything under 0.80 to GPT-5.6 Terra matched Terra’s accuracy at 26% to 28% of its cost and about half its mean latency.¹⁶

Confident errors never reach the fallback. At 0.90 or above, Jev was still wrong on 5 of 112 answers in the 8-way task, and all five filed a direct-debit complaint under card payments. Terra made the same five calls, because the two labels overlap.¹⁶ OpenAI’s guide says to set thresholds from labelled examples, weighing the cost of false positives against false negatives.⁸ TypeSafe’s docs say to pin a model version once thresholds are tuned, since the jev-latest alias moves when a new model ships.⁷

When to use a decision model, an LLM or a classifier

The table turns the evidence above into a starting rule. It weighs the latency budget, the cost per million decisions, the need for calibrated probabilities, the labelled data you have and how often the categories change.

Start with the latency budget. A check that must finish in under 100 ms points to a classifier running next to your code. Red Hat’s pre-trained classifiers had medians of 33 to 80 ms on a laptop CPU, while a hosted model also pays for the network, at least about 56 ms in Red Hat’s setup.²⁰ Hosted decision models fit budgets from about 300 ms to a second. An LLM for classification still makes sense when a second or more is acceptable and you want its reasons as well as its label.

Then count your labels. Whichever option you pick, you need a labelled set from your own traffic to measure accuracy and choose thresholds. At a few hundred items per task, AY Automate’s 95% intervals were 6 to 13 points wide, so treat smaller gaps between models as noise until you have more data.¹⁶

Patterns for intent routing, guardrails and rule checks

A study of 2,170 public GitHub projects that used Jev, collected on 22 September, found attribute judgement and scoring in wide use.²³ The three patterns below come from the vendors’ docs and examples.

Intent routing. Classify each request once, act on confident answers and send the rest to a person or a stronger model. TypeSafe’s docs route support messages this way. Order-status questions go to plain code, product and returns questions to specialist LLMs, and unclear messages or complex complaints to a person.²⁴ In the illustrative sketch below, decide stands in for any decision API, and the thresholds are set on labelled data.

# Illustrative sketch: `decide` stands in for any decision API.
INTENTS = {
    "order_status": "Asks where an existing order is",
    "refund": "Wants money back for an order or a charge",
    "technical": "Reports a problem using the product",
    "other": "Fits none of the above",
}

def route(message, order):
    r = decide(state=message, questions={
        "intent": {"type": "choice", "options": INTENTS},
        "upset": {"type": "yes_no", "question": "Is the customer upset?"},
    })
    intent = r["intent"]
    if intent["confidence"] < MIN_CONFIDENCE:  # chosen on labelled data
        return "human"
    if intent["choice"] == "refund":
        # Amounts and dates are arithmetic, so code checks them.
        if order.total > REFUND_LIMIT or order.days_since_delivery > RETURN_DAYS:
            return "human"
        return "refund_flow"
    if intent["choice"] == "technical":
        upset = r["upset"]["probability"] > UPSET_CUTOFF
        return "senior_support" if upset else "support_llm"
    return "order_lookup" if intent["choice"] == "order_status" else "human"

The “other” option follows OpenAI’s advice to give unmatched inputs somewhere to go, and the refund limits stay in code because TypeSafe lists arithmetic and dates as weak spots.⁸¹⁴

Guardrails. A guardrail asks a yes/no question about an input or an action before it runs. The Strands example asks whether a proposed tool call’s arguments come from something the user said, and whether it’s too early to call the tool. Code turns the two probabilities into an action, which in the demo sends the agent back to ask the user.⁵ Jev scored highest on content safety in Red Hat’s test. For prompt injection we’d put a dedicated classifier first, since DeBERTa did better there and TypeSafe says content written to steer the model can move Jev’s answer.²⁰¹⁴

Rule checks. Anything code can compute exactly should stay in code, and the model should be asked only for the judgement. It’s the split we use around LLM judges, where ordinary code checks allowed values, ranges and required fields on every answer, as in our pipeline with two reviews and a judge. A decision model’s typed output removes the parsing step, but range and policy checks still run in your code.

A checklist before you ship one

1. Build a labelled set from your own traffic, a few hundred items per decision, and hold part of it back.
2. Run your shortlist on it with your real prompts and option descriptions. Record accuracy, p50 and p95 latency from your own region, and cost per million decisions from billed tokens.
3. Compare mean confidence with accuracy, and read every confident error. Overlapping categories show up there first.
4. Choose thresholds on one part of the set and confirm them on the part you held back.
5. Add an “other” option, and a path to a person or a stronger model.
6. Shuffle the option order in tests and check the answers hold, since Jev can lean towards the first option.
7. Pin the model version once thresholds are tuned, and re-run the set before you move to a new one.
8. Keep arithmetic, dates and policy limits in code.
9. Log the model version, probabilities and final route for every decision, and count escalations and refusals. Our post on a pipeline that reported 0 dropped shows what goes missing when only failures are counted.

Frequently asked questions