Decision Models Turn AI Routing Into a Small, Testable Contract

A fictional billing ticket arrives: “I was charged twice, but I only want the extra amount reversed. Please keep my subscription active.” A chatbot can respond conversationally. A routing system first needs a much smaller answer: is this about a duplicate charge, cancellation, or something else?

Turn a conversation into a contract

Free-form language is flexible, but it is awkward when software needs a predictable next step. A decision interface makes the task explicit: here is the ticket, here are the allowed categories, and here is the question the result must answer. The system can then validate that the response belongs to the allowed set before acting on it.

The llama.cpp decision-model description covers questions over a state and supports choice, boolean and score-style decisions. It notes that option descriptions matter, multiple questions can share state processing, and scores or thresholds vary by model. That is an interface pattern, not a guarantee that every output is calibrated or suitable for automatic action. llama.cpp decision models

An option named billing may be too broad. If the system must distinguish a duplicate payment from a renewal complaint, those descriptions need boundaries and examples. Otherwise, the model may make a plausible choice that downstream logic interprets too specifically. The contract includes not only the allowed labels, but what each label means and what the system should do when none fit.

This distinction is important because classification can be right while the action is still unauthorized. A model may correctly identify a refund request, but only a human reviewer or deterministic policy may be allowed to issue a refund. Put that rule in the application boundary, not in a hope that the model will remember it.

Describe options so they can be distinguished

Suppose a support team offers three routes: account access, billing, and product use. A customer says, “I cannot open the course after my payment went through.” The issue touches both access and billing. If options merely list category names, the system has no guidance about which route owns a cross-cutting case.

Better descriptions explain the routing rule: account access covers sign-in and entitlement failures; billing covers payment disputes and invoices; product use covers a feature question after access works. The system can ask a second question if needed, such as whether the user can sign in. That keeps the first decision narrow and turns ambiguity into a visible branch.

Multiple questions may be asked against the same state, but shared processing does not make the questions interchangeable. One answer can classify topic; another can detect whether personal data is included. Each output should have its own schema, allowed values and interpretation. Combining them into a single vague “risk score” makes debugging difficult.

Treat scores as measurements to calibrate

A score that resembles a probability is easy to overread. If a model returns 0.82 for “billing,” that number does not automatically mean 82 out of 100 similarly scored tickets are billing cases. Different models, prompts, option wording and incoming populations can shift the meaning.

Calibration begins with examples that resemble the live workflow and have reviewed labels. Separate development examples from a held-out evaluation set. Group results by decision type and inspect false positives and false negatives. If a threshold controls whether a human sees a case, measure the missed cases and unnecessary escalations at several thresholds before choosing one.

Any numeric illustration should be treated as hypothetical until measured. For instance, a team might provisionally send scores below an invented threshold to manual review, but it must validate that rule on its own labeled tickets. Do not transfer a threshold from another model or dataset as if confidence were a universal unit.

A conceptual request shape

The following sketch illustrates the idea of structured input; it is conceptual and is not asserted as runnable code for a particular library version:

{
  "state": "Customer reports a duplicate charge and asks to keep service active.",
  "question": "Which support queue should review this?",
  "choices": ["billing_review", "cancellation", "other", "unclear"]
}

The output contract might allow only one of those choices plus an optional explanation or uncertainty marker. The application should reject unexpected values, log the model and configuration, and route unclear safely. It should retain enough of the input and decision record for an authorized reviewer to reconstruct why a ticket went to a queue.

Do not mistake a structured response for a trustworthy one. A model can choose a valid label for the wrong reason. Validation should combine format checks with representative evaluation and sampling of real decisions. If the state omits essential context, the right response may be a follow-up question rather than a forced classification.

Design for ambiguity and change

Real requests do not always fit the category tree. Provide an “unclear” or “needs human review” path and make it operationally useful. If every ambiguous ticket lands in an overloaded queue, staff may ignore the signal. Track how often the fallback occurs, what information would resolve it, and whether the categories need revision.

Monitor drift. A new subscription plan, support policy or product feature can change the meaning of old labels. Version the option descriptions and evaluation examples. When performance changes, inspect which category boundaries moved before adjusting thresholds. Retain a route to revert a decision contract that produces unexpected behavior.

The strongest case for free-form conversation is that users do not speak in taxonomies. They should not have to. The interface can translate their ordinary language into a narrow internal decision, while the product preserves their actual request for review. Structure belongs at the software boundary; it need not constrain the person asking for help.

The interface can be modular without making the goal obvious. Granite Switch combines several capabilities behind an endpoint. The shopping-agent opinion asks whether an agent has understood the user’s priorities before a decision becomes a commitment.

A small, testable decision contract is useful precisely because it is modest. It names the question, limits the outputs and keeps action authority elsewhere. That creates something a team can evaluate, inspect and improve without pretending the model’s label is the policy itself.