Decision models (Jev) — lab.decide

From docs/decision-models.md on the 3.0.0-GA branch (updated 2026-11-16, 17e721c). Code and examples: github.com/ancientcomputing/locallm. Overview: LocalLM SDK.

Contents

Decision models answer typed questions (yes or no, one of several, a scale) instead of writing text. A separate feature of the SDK, used with lab.decide.

Reach for this when your app needs a small, fixed decision about some text — which route, which team, does this need the web, is this sensitive — and you'd otherwise ask a chat model and parse its reply.

New to decision models? They behave nothing like a chatbot. Read Decision models (Jev) in your app first (what they are, for people who know chatbots, with links to TypeSafe's and others' docs), then JevDK's developer's guide for the workflow.

Examples that use it: JevDK.

A decision model (a decider) takes one input and a few typed questions, and answers each with one of the options you allowed plus a probability for every option. It writes no text. TypeSafe's Jev is the original; the SDK uses Jev's vocabulary and asks the same questions of four backends:

Decider Runs Module OS Decision.fidelity
OpenJev (OpenJevDecisionProvider) On the Mac, on an MLX model Inference macOS 27 .tokenScored(calibrated:)
TypeSafe Jev via OpenRouter (JevDecisionProvider(.openRouter(key:))) Hosted Remote macOS 26+ .native
Featherless Simple Jev (.featherless(key:), .featherlessDemo()) Hosted Remote macOS 26+ .tokenScored(calibrated: false)
OpenAI Decisions (JevDecisionProvider(.openAI(key:)), openai:gpt-6-luna; since SDK 2.3) Hosted Remote macOS 26+ .native

lab.decide and the decision types are in Core and work on macOS 26. The deciders live where the table says: OpenJev in Inference, the hosted ones in Remote.

↑ Top

Register a decider and route to it

Deciders are registered and routed separately from chat models (they never appear in a chat model picker), with their own scheme namespace:

import LocalLMLabSDKInference   // OpenJev
import LocalLMLabSDKRemote      // hosted Jev

try lab.models.register(decision: OpenJevDecisionProvider(mlx: mlx))        // the MLXModelProvider you already use
lab.models.route(decision: "support", to: ModelID("openjev:mlx-community/Qwen3-4B-4bit")!)

// or hosted — off until you register one; questions and input are then sent to that service
try lab.models.register(decision: JevDecisionProvider(.openRouter(key: key)))
lab.models.route(decision: "support", to: ModelID("openrouter:typesafe/jev-1.13")!)

ModelRegistry also has replace(decision:), removeDecisionProvider(scheme:), modelID(forDecision:), decisionProvider(for:) and decisionAvailability(for:) (a ModelAvailability, e.g. not downloaded or needs a key). Decision routes are saved by lab.snapshot() like chat routes.

↑ Top

Ask

let d = try await lab.decide(route: "support", state: .text(message), questions: [
    .choice("team", "Which team should handle this message?", criteria: [
        "billing": "payments, invoices and refunds",
        "technical": "bugs, outages and product errors",
        "account": "profile, login and subscription changes",
    ]),
    .noul("refund", "Does the customer explicitly ask for money back?"),
    .score("urgency", "How urgent is it?", criteria: ["can wait", "soon", "now"]),
])

d.choice("team")      // ChoiceAnswer: .choice "technical", .confidence, .probabilities ["billing": …, …]
d.noul("refund")      // Double: probability of yes
d.score("urgency")    // ScoreAnswer: .score 1.6 (expected level, 0-based), .level, .probabilities, .legend
d.fidelity            // .native or .tokenScored(calibrated:)
d.modelID, d.latency, d.usage   // usage: tokens and cost, from hosted backends
d.isRefused("refund")  // true if the backend declined that question (OpenAI Decisions only)
↑ Top

Confidence, fidelity and calibration

.native probabilities come from a model trained to make decisions (TypeSafe). .tokenScored ones are read off an ordinary language model — how likely it was to say each option's label — and on a small model they're often 100% even when it's wrong. Don't put tight thresholds on uncalibrated .tokenScored answers.

For a local decider you fit a calibration on your own marked answers and pass it in; fidelity then says .tokenScored(calibrated: true). It holds for one model, revision, wrapper and kind of question, so the SDK ships no fitted values. The usual way is to fit it in JevDK and use the result in your app, with no fitting code of your own:

// 1. In JevDK: mark answers, Calibrate… → Fit calibration, then File → Export Questions….
//    The .decisions.json carries the calibration with the questions it was fitted for.
let set = try DecisionQuestionSet(contentsOf: url)
let openjev = OpenJevDecisionProvider(mlx: mlx, tunedWith: set)       // applies set.tuning?.calibration

// 2. Or copy the snippet (Calibrate window → Copy code): the values plus the system instructions
//    they were fitted with — the calibration only holds with the same wrapper.
let openjev2 = OpenJevDecisionProvider(mlx: mlx,
    wrapper: OpenJevWrapper(system: "…", inputLabel: "…"),
    calibration: DecisionCalibration(noulTemperature: X, choiceTemperature: Y, scoreTemperature: Z))
// X, Y, Z: the Temperature column of JevDK's Calibrate window (yes/no, choice, score; 1.0 = no change).

To fit in code instead (for example from lab.evaluate's samples):

let cal = DecisionCalibration.fit(samples)            // [DecisionCalibration.Sample], from marked answers
cal.measure(samples)                                   // accuracy, expectedCalibrationError, logLoss

Calibration is temperature scaling per question kind: it never changes which answer wins, only how sure it says it is.

↑ Top

OpenJev: the local decider

↑ Top

Hosted Jev

JevProviderConfig presets set the endpoint, scheme, fidelity, limits and model list: .openRouter(key:) (TypeSafe's typesafe/jev-1.13; typesafe/jev-router picks a model for you), .featherless(key:) and .featherlessDemo() (free, rate-limited, no key). validated() applies the chat providers' checks (https, no line breaks in keys). Questions and choice options keep your order on the wire, JSON state is sent as JSON, and d.usage carries the tokens and cost. Like other remote providers, it's off by default and your UI should say where inputs are sent.

OpenAI's Decisions API (since SDK 2.3) uses the same provider with a different wire format, JevProviderConfig's wire: .openAIDecisions (the other presets use .jev):

try lab.models.register(decision: JevDecisionProvider(.openAI(key: key)))
lab.models.route(decision: "support", to: ModelID("openai:gpt-6-luna")!)
↑ Top

A decider and your chat model: one model or two

The pattern the SDK is built for is two models per feature: a decider for the typed choices (does this need the web, which tool, is it sensitive) and a chat model for the text, both on the Mac.

↑ Top

From JevDK to your app: question sets, answer sets, lab.evaluate

Questions are the work, so the SDK carries them from testing to shipping intact:

// The questions exactly as tested in JevDK (File → Export Questions…), with the model, revision,
// wrapper and calibration they were tested with.
let set = try DecisionQuestionSet(contentsOf: Bundle.main.url(forResource: "support.decisions", withExtension: "json")!)
let openjev = OpenJevDecisionProvider(mlx: mlx, tunedWith: set)     // the tested wrapper + calibration
try lab.models.register(decision: openjev)
let decider = set.tuning?.model ?? ModelID("openjev:mlx-community/Qwen3-4B-4bit")!
lab.models.route(decision: "support", to: decider)
if let why = openjev.tuningMismatch(for: decider) { log(why) }      // a different model or revision on this Mac
let d = try await lab.decide(route: "support", state: .text(message), questions: set.questions)

// In CI: score any decider against your answer set (CSV: input, then one column per question id).
let answers = try DecisionAnswerSet(csv: String(contentsOf: answersURL, encoding: .utf8))
let result = try await lab.evaluate(route: "support", questions: set.questions, answerSet: answers)
#expect(result.score("refund")!.accuracy >= 0.95)
↑ Top

Serving decisions over HTTP: jev-serve

To use a local decider from another process or language, run the jev-serve example. It answers hosted Jev's API, OpenRouter's POST /api/alpha/decisions and Featherless's POST /v1/classifier, with OpenJev on this Mac, so existing hosted-Jev client code switches by changing its base URL. It serves the setup you tested in JevDK (File → Export Server Config…): each model pinned to its tested revision (downloaded if missing), with its wrapper and calibration; loopback by default, an optional bearer token. Responses add "fidelity": "tokenScored" and "calibrated", which JevDecisionProvider reads.

The wire format is JevWire (Core), used by both the client (JevDecisionProvider) and the server:

let wire = try JevWire.decodeRequest(body)                          // model?, DecisionRequest (order kept)
try wire.request.validate(against: provider.limits)
let decision = try await provider.decide(wire.request, using: id)
let response = try JevWire.encodeResponse(decision, model: repo, request: wire.request)
↑ Top

Decisions that need your product's knowledge

A decider sees only state and the questions (there's no system prompt), so knowledge it needs, such as which part of your software a bug report concerns and who owns it, has to come through those, or stay in your code:

Don't put the knowledge in an OpenJevWrapper's system text: hosted backends never see it, and a calibration only holds for the wrapper it was fitted with. Full walkthrough with code: JevDK's guide.

↑ Top

Writing questions that work

↑ Top