Online providers — GPT, Claude online, OpenRouter (LocalLMLabSDKRemote)

From docs/remote-providers.md on the 3.0.0-GA branch (updated 2026-11-08, 6676a71). Code and examples: github.com/ancientcomputing/locallm. Overview: LocalLM SDK.

Contents

Using hosted models through LocalLMLabSDKRemote. Part of the model layer.

Reach for this when you want a hosted model API — OpenAI, Anthropic's Messages API, OpenRouter, or any OpenAI-compatible server — behind the same lab.makeSession(route:) call site as the on-device and local models, ideally with the provider running web search for you.

Examples that use it: model-switch is the reference app — add a provider, tick web search, switch models mid-conversation, streamed search activity + citation links, all through Components' AIModelsSettingsView.

LocalLMLabSDKRemote.xcframework is a 4th binary on the release, added the same way as Inference / Claude — a binaryTarget keyed off LOCALLM_SDK_VERSION. It has no third-party dependencies (pure URLSession). Its manifest floor is macOS 26, so linking it does not force a macOS 27 deployment target; RemoteModelProvider itself is @available(macOS 27) and reports .requiresOS("macOS 27") on 26 — so a macOS-26 app can link it and just not register it below 27.

import LocalLMLabSDKRemote

// One data-driven config per provider. A preset fills in the dialect + base URL + auth shape;
// YOU supply the model ids — the SDK ships none (which id is current/good is your call).
var openai = RemoteProviderConfig.openAI(apiKey: key, models: [RemoteModel(id: "gpt-…")])
openai.capabilities.insert(.webSearch)
openai.defaultOptions.webSearch = true              // provider runs the search; no MCP needed

lab.models.replace(try RemoteModelProvider(openai)) // validates, then registers / re-registers
lab.models.route("chat", to: ModelID(scheme: "openai", rest: "gpt-…")!)
let answer = try await lab.makeSession(route: "chat").respond(to: prompt)

RemoteModelProvider.init(_:) always validates config for you — baseURL and auth are just data, and if your UI lets a user type or import them (a custom .openAICompatible endpoint, an imported settings file), that string is an SSRF-shaped primitive inside your app's sandbox that decides where a real API key gets sent. init(_:) runs RemoteProviderConfig.validated() internally and throws RemoteProviderConfig.ValidationIssue for a non-HTTPS baseURL to a non-local host (the key would go over the wire in clear text), a non-http(s) scheme (file:, ftp:, …), a malformed custom header name, or a CR/LF in a credential (header injection) — catch it and reject the input rather than registering the provider anyway. This is a syntax check, not a trust check: it doesn't confirm the host is the one the user meant to talk to, so still warn before sending an existing key to a changed host, and tell users a remote request carries the full prompt, transcript, and tool definitions to that endpoint. If you're certain a config needs to skip this (e.g. a hardcoded corporate-proxy profile where validated()'s rules are known to be wrong for it), use the non-throwing RemoteModelProvider(unchecked:) escape hatch instead — never for anything user-supplied.

Settings UI: Components' AIModelsSettingsView renders the whole "AI Models" panel (add provider + key, per-model rows, web-search toggle, Test connection). Components does not link Remote — it calls back through onSave / onRemove / onTest closures with plain RemoteProviderDraft / ProviderTestOutcome values, so a macOS-26 host app can host the panel and hand the actual RemoteModelProvider work to a 27-only helper. → the full pattern is examples/model-switch.

↑ Top