When to Use a Decision Model

By Jhony Vidal
Published 21 November 2026
5 min read

Compare how many tickets reach the right desk, how many need review and what each decision costs.

On this page

Audience · Product owners and engineers deciding which support requests to route automatically

“You charged my card twice.” Someone on the support desk reads the message and forwards it to billing. They make no promise about a refund. All they have decided is who should read it next.

That is a good job for a decision model. You give it the message and a narrow question. It returns a score your application can use. If the same customer has already submitted the billing form, an ordinary rule can send it straight to the right desk. Paying a model to read that ticket again buys very little.

Write the question a colleague could answer

For this inbox I would ask, “Is this about an existing charge, invoice or refund?” A duplicate charge belongs with billing. “Does the annual plan cost more?” is a question about a future purchase. Writing down that difference gives two people a fair chance of labelling the same tickets consistently.

The model estimates how likely the answer is yes. The team decides what to do with that number. In our example, a score of 0.90 or higher sends the message to billing. Everything else stays in the ordinary inbox for a person to read. An unavailable service or missing score also leaves the ticket there.

The same approach works for other bounded choices. A help centre can ask which of five guides covers a question. An editor can ask whether a draft names a source. If the customer needs a written explanation, a text-generating model can draft one after a person or a lookup has established the facts. Routing a billing question gives the software no authority to move money.

See what changes on eight tickets

Here is a small offline exercise. The messages, correct labels and model scores are fictional. Two tickets came through the billing form. Six arrived as free text. The replay counts what each policy would do with those same eight tickets.

PolicyScores requestedSent to billingLeft for reviewSent to billing in error
Billing form rule alone0260
Model alone, cutoff 0.908350
Rule first, then model at 0.906440
Rule first, then model at 0.706621

Look at the last two rows. Lowering the cutoff sends two more messages straight to billing. One is a real billing question about two entries on a bank statement (0.72). The other is the annual-plan question (0.84). A tidy number can still send a customer to the wrong desk.

Try it without an account. Save these two files in the same folder, then run the commands below. Open the JSON to see which message changes queue when you move the cutoff. Python replay · Eight labelled tickets

python3 routing.py --policy rules
python3 routing.py --policy model --cutoff 0.90
python3 routing.py --policy hybrid --cutoff 0.90
python3 routing.py --policy hybrid --cutoff 0.70

Connect one inbox to a provider

The examples below ask the same question. In each, text is the customer’s message and question or QUESTION is the billing question above with the instruction to exclude prices before purchase. Choose the example that fits your application. The complete files include imports, timeouts and the route back to a person.

TypeScript with the OpenAI SDK. A yes/no question is called a predicate. Check the answer type before reading the probability because the API can also return a refusal. Decisions guide

const result = await client.decisions.create({
model: "gpt-6-luna", input: text,
questions: [{ type: "predicate", name: "billing", instructions: question }],
});
const answer = result.answers[0];
return answer?.type === "predicate" ? answer.probability : null;

Save the TypeScript file beside the tickets, install the SDK with npm install openai, then run node route.mts with Node 24. The default replays the example. With OPENAI_API_KEY set, node route.mts --live scores the six free-text tickets. TypeScript example

If you already use a text-generating model, give it the same question and ask for one of two labels in JSON. In the TypeScript example, set OPENAI_TEXT_MODEL to a Responses model you have access to that supports structured output, then run node route.mts --text. It uses the same billing form rule and the same six messages. You can now compare wrong queues, reviews, elapsed request time and usage. The text version picks a queue directly. The decision version gives you a score so you can change the cutoff without asking the model again.

Python with Gemini. Google’s name for the yes/no question type is noul. Gemini takes a dictionary of named questions. Read the answer using the name you supplied. Google’s worked codelab

from google import genai
client = genai.Client(http_options={"api_version": "v1beta"})
result = client.decisions.create(
model="gemini-decision-1", input=text,
questions={"billing": {"type": "noul", "instructions": QUESTION}},
)
probability = result.answers["billing"].noul

Install google-genai>=3.2 in your Python environment and set GEMINI_API_KEY. Run python3 routing.py --provider gemini. For the Python OpenAI SDK, install openai, set OPENAI_API_KEY and change the provider to openai. Each live run makes billable requests to the selected provider.

Python with Microsoft Foundry. Foundry calls the input state. The request uses your deployment name, which can differ from the model name in the catalogue. Deploy Microsoft-Decision-1.1, then copy the resource endpoint and deployment name from its details page. Foundry deployment and authentication guide

body = {"model": deployment, "state": text, "questions": {
"billing": {"type": "noul", "instructions": QUESTION}
}}
request = Request(endpoint + "/providers/microsoft/v1/systemone",
data=json.dumps(body).encode(),
headers={"Content-Type": "application/json", "api-key": key})
with urlopen(request, timeout=10) as response:
probability = json.load(response)["answers"]["billing"]["noul"]

Set AZURE_ENDPOINT, DEPLOYMENT_NAME and AZURE_API_KEY, then run python3 routing.py --provider foundry. The guide also shows Microsoft Entra authentication. Choose a cutoff for each provider using labelled tickets. Two services returning 0.90 may make different mistakes.

Put the decision inside a workflow

A single inbox only needs a function. A workflow earns its keep when another step needs the result, such as looking up an invoice before drafting a reply. Keep the rule, the cutoff and the fallback together in one function called classify. It returns either billing or review.

In LangChain and LangGraph, a runnable wraps the function. A graph node stores the returned queue in the ticket’s state. A later node can read that queue to choose the next step. LangGraph guide

route = RunnableLambda(classify)
graph = StateGraph(State)
graph.add_node("route", lambda state: {"queue": route.invoke(state)})
graph.add_edge(START, "route")
graph.add_edge("route", END)

In Microsoft Agent Framework, an executor receives the ticket and returns the queue as the workflow’s output. Both frameworks can use any of the three scoring functions above. Switching frameworks need not change who gets a message. Agent Framework executors

@executor(id="route")
async def route_ticket(ticket: dict, ctx: WorkflowContext[Never, str]) -> None:
queue = await asyncio.to_thread(classify, ticket)
await ctx.yield_output(queue)
workflow = WorkflowBuilder(start_executor=route_ticket, output_from=[route_ticket]).build()

Save the workflow file alongside routing.py. With Python 3.12, install langchain-core langgraph agent-framework-core, then run python workflow.py. Both workflows print billing for the duplicate-charge example using a fixed score. Add --provider gemini, openai or foundry to call the service once per workflow. Complete workflow example

Measure the wait and the wrong turns

Google reported 150–250 ms end-to-end latency in its launch tests and nearly five times lower cost than Gemini 3.5 Flash-Lite on its comparison suite. Microsoft’s October benchmark reported about 35 times lower median latency for Decision-1 than GPT-6 Sol on its structured decision tasks. The comparisons use different models, workloads and baselines. They tell you why a local trial is worth running. Google benchmark and conditions · Microsoft launch benchmark

For a larger worked comparison, DataCamp takes 12 easily confused banking intents and samples 15 labelled messages per intent. That is a useful next step after the eight-ticket replay. Banking intent experiment

One useful observation from the earlier Jev work is that a model can return a valid answer from your list and still pick the wrong one. Check the decision against what actually happened to the ticket. TypeSafe’s introduction to Jev

Before moving a single real message, run the candidate alongside the current inbox. Let support colleagues label a held-out sample without seeing the scores. For each ticket record the policy version, model deployment, score or failure, chosen queue, final queue, time spent and billed usage. A ticket ID is enough to join those events without copying the customer’s message into telemetry.

Three numbers would decide whether I kept it:

  • Wrong routes per 100 tickets. Also count reviews and billing requests left waiting. A system can avoid wrong routes by sending everything to a person.
  • Time to the right desk. Measure the median and the slowest five per cent, including retries, handovers and review. The replay’s live mode reports the 95th percentile of API request times. Eight tickets are enough to check the wiring, far too few to estimate a busy inbox’s slow tail.
  • Cost per correctly routed ticket. Add model charges and staff time for review and rework. For example, at 30 seconds per review and three minutes per wrong transfer, saving two reviews but adding one wrong transfer costs two extra minutes across our eight tickets. Substitute your own timings.

Start with one queue where a mistake is easy to undo. Read the messages on either side of the cutoff with the people who handle them. The annual-plan question is exactly the kind of case that can change a team’s mind about what to automate.

The Author

Jhony Vidal

Jhony Vidal

Lead AI Engineer