localai-cli
Call your Mac's on-device model from your own Swift or Python app — no server, no HTTP
LocalLM Lab's API Lab gives you an OpenAI-compatible HTTP
endpoint for scripts and tools that already speak that protocol. localai-cli is for the
other case: you're building a native app — a game, a utility, a menu bar tool — and you want to reach
Apple's on-device model directly from Swift or Python, without running an HTTP server at all.
localai-cli is a small, self-contained binary: your app spawns it as a subprocess, writes
one JSON request to its stdin, and reads one JSON response back from its stdout. That's the entire
interface.
It's config-aware: it reads the same localai-config.json that LocalLM Lab's
Local AI Settings and MCP Servers panels write, so whatever
connectors and MCP tools your user has already granted through the app's UI are exactly what your code
is allowed to request — nothing more. localai-cli never edits that file, only reads it.
The model
Every request goes to Apple's FoundationModels framework running directly on the Mac's
Neural Engine — the same on-device model Prompt Playground and API Lab use. No network call, no API
key, no per-token cost. Requires macOS 26+ on Apple Silicon with Apple Intelligence
enabled.
Whether LocalLM Lab itself needs to still be running (not just installed) depends on
the request. A request with no connectors and no mcp_tools never touches
LocalLM Lab at all — localai-playground-run talks to FoundationModels
directly, so you can quit the app and keep only localai-config.json plus the two toolkit
binaries around. A request that names any connector or MCP tool relays that one call
through LocalLM Lab's menu bar tray process over a local Unix socket, which has to be up at the
moment the model actually reaches for the tool — config validation itself
(connector 'x' is not enabled, etc.) still works with LocalLM Lab quit, since that only
reads the config file; it's specifically the live tool call that needs the app running.
Get it
- Run LocalLM Lab at least once, and use Local AI Settings / MCP
Servers to enable whichever connectors and tools you plan to call with. This writes
~/Library/Application Support/LocalLM Lab/localai-config.json. - Download the
localai-toolkit-<version>-arm64.zipasset from the LocalLM GitHub page and unzip it — it contains two binaries as a matched pair:localai-cliandlocalai-playground-run(the raw model-invocation helperlocalai-cliwraps). Keep them together in the same directory;localai-cli's default--helper-pathassumes "next to me." - Full runnable Swift and Python examples — including the ones on this page — live in examples/localai-cli-swift and examples/localai-cli.
Interface
localai-cli --config /path/to/localai-config.json --status
localai-cli --config /path/to/localai-config.json --run <<< '{"system_prompt": "...", "user_input": "...", "connectors": ["clock"]}'
localai-cli --config /path/to/localai-config.json --chat <<< '{"messages": [...], "connectors": ["filesystem"]}'
localai-cli --version
| Flag | What it does |
|---|---|
--config <path> | Required for --run/--chat. Path to a localai-config.json-shaped file — no implicit lookup, so you can ship a copy anywhere. |
--status | Checks on-device model availability. Doesn't touch the config file. |
--run | system_prompt/user_input request on stdin, one-shot, no conversation memory. |
--chat | OpenAI-style messages array on stdin instead — use this shape, not --run's, when you have a conversation history. |
--helper-path <path> | Override where localai-playground-run lives. Default: same directory as localai-cli itself. |
--version | Prints the build version and exits. |
Connectors and MCP tools: request vs. enabled
connectors_enabled in the config is the ceiling — what your user has
granted via LocalLM Lab's own settings UI. The connectors field in each request is the
selection — which of those this one call actually wants active. The rule is
requested ⊆ enabled:
- Request something not in
connectors_enabled→localai-clirejects it with a JSON{"error": "..."}, before the model is ever invoked. - Enabled but not requested → simply not given to that call, no error.
- Omit the field entirely → nothing is active. This is a deliberate least-privilege default, not an oversight — every tool a given call can use has to be named explicitly in that call's own request.
MCP tools work the same way, at {server, tool} granularity, via a separate
mcp_tools array (kept distinct from connectors — see
MCP Servers for how those get enabled in the first place):
{
"system_prompt": "You are a concise assistant. Use the available tool to answer.",
"user_input": "What documentation topics are available for nickclyde/duckduckgo-mcp-server?",
"mcp_tools": [{"server": "https://mcp.deepwiki.com/mcp", "tool": "read_wiki_structure"}]
}
Quickstart: two ways to try it in under a minute
Pick your language. Both do the exact same thing: ask the on-device model the current time, using the
clock connector — the only one with no macOS permission dialog, so there's nothing to
approve beyond one toggle.
One-time setup: open Local AI Settings in LocalLM Lab and turn on System Clock. That's it.
Save as quickstart_clock.swift, place localai-cli and
localai-playground-run next to it, then run swift quickstart_clock.swift —
no Xcode project needed, swift <file>.swift runs a script file directly.
#!/usr/bin/env swift
import Foundation
let env = ProcessInfo.processInfo.environment
let cliPath = env["LOCALAI_CLI_PATH"]
?? URL(fileURLWithPath: #filePath).deletingLastPathComponent().appendingPathComponent("localai-cli").path
let configPath = env["LOCALAI_CONFIG_PATH"]
?? NSHomeDirectory() + "/Library/Application Support/LocalLM Lab/localai-config.json"
let request: [String: Any] = [
"system_prompt": "You are a concise assistant.",
"user_input": "What time is it right now?",
"connectors": ["clock"],
]
let requestData = try! JSONSerialization.data(withJSONObject: request)
let process = Process()
process.executableURL = URL(fileURLWithPath: cliPath)
process.arguments = ["--config", configPath, "--run"]
let stdinPipe = Pipe(), stdoutPipe = Pipe()
process.standardInput = stdinPipe
process.standardOutput = stdoutPipe
process.standardError = Pipe()
try! process.run()
stdinPipe.fileHandleForWriting.write(requestData)
try! stdinPipe.fileHandleForWriting.close()
let stdoutData = stdoutPipe.fileHandleForReading.readDataToEndOfFile()
process.waitUntilExit()
let response = try! JSONSerialization.jsonObject(with: stdoutData) as! [String: Any]
if let error = response["error"] as? String, !error.isEmpty {
print("FAIL - \(error)")
exit(1)
}
print(response["answer"] as? String ?? "")
Full version with error messages tailored to each failure mode: quickstart_clock.swift.
Save as quickstart_clock.py, place localai-cli and
localai-playground-run next to it, then run python3 quickstart_clock.py
(use python3 explicitly — plain python isn't guaranteed on current macOS).
import json, os, subprocess
CLI_PATH = os.environ.get(
"LOCALAI_CLI_PATH", os.path.join(os.path.dirname(__file__), "localai-cli")
)
CONFIG_PATH = os.environ.get(
"LOCALAI_CONFIG_PATH",
os.path.expanduser("~/Library/Application Support/LocalLM Lab/localai-config.json"),
)
request = {
"system_prompt": "You are a concise assistant.",
"user_input": "What time is it right now?",
"connectors": ["clock"],
}
result = subprocess.run(
[CLI_PATH, "--config", CONFIG_PATH, "--run"],
input=json.dumps(request),
capture_output=True,
text=True,
)
response = json.loads(result.stdout)
if response.get("error"):
print(f"FAIL - {response['error']}")
else:
print(response.get("answer", ""))
Full version with error messages tailored to each failure mode: quickstart_clock.py.
MCP quickstart: a real tool call, no auth required
Same idea, but the tool call goes out to a real MCP server instead of a local connector — DeepWiki, which needs no OAuth or API key.
One-time setup: in LocalLM Lab's MCP Servers panel, Add Server with
URL https://mcp.deepwiki.com/mcp, auth type None, then enable its
read_wiki_structure tool.
let request: [String: Any] = [
"system_prompt": "You are a concise assistant. Use the available tool to answer.",
"user_input": "What documentation topics are available for the GitHub repo "
+ "nickclyde/duckduckgo-mcp-server?",
"mcp_tools": [["server": "https://mcp.deepwiki.com/mcp", "tool": "read_wiki_structure"]],
]
// ...same Process/Pipe subprocess call as above, using this request instead.
Full script: quickstart_mcp_deepwiki.swift.
request = {
"system_prompt": "You are a concise assistant. Use the available tool to answer.",
"user_input": (
"What documentation topics are available for the GitHub repo "
"nickclyde/duckduckgo-mcp-server?"
),
"mcp_tools": [{"server": "https://mcp.deepwiki.com/mcp", "tool": "read_wiki_structure"}],
}
# ...same subprocess.run() call as above, using this request instead.
Full script: quickstart_mcp_deepwiki.py (and quickstart_mcp_deepwiki_chat.py for the --chat shape instead of --run).
Example: Plate Today — "what's on my plate today"
A fuller example than the quickstarts above: a script that checks Calendar, Reminders, and Todoist for
what's due today and asks the on-device model to summarize the day. It's a plain script — no app, no
EventKit calls of its own, no TCC prompts triggered by the script itself — it relies entirely on
connectors and MCP tools LocalLM Lab already has permission for. Calendar and Reminders come in as
connectors; Todoist comes in as an MCP tool call against Todoist's own hosted MCP server,
find-tasks-by-date.
FoundationModels has no live wall-clock awareness on its own, so the script also requests the
clock connector and instructs the model to look up the actual current date first, then
match "today" against calendar/reminders/Todoist using that date — rather than guessing.
Before ever calling localai-cli, the script reads localai-config.json itself
and checks all four sources are actually usable — the three connectors enabled, and the Todoist server
connected and enabled with find-tasks-by-date enabled on it — and prints a checklist. If
anything's missing, it names the exact panel/toggle to fix and exits without ever invoking the model, so
you don't burn a model call on a request that was always going to fail.
One-time setup: in Local AI Settings, turn on System
Clock, Calendar, and Reminders. In MCP
Servers, add Todoist (https://ai.todoist.net/mcp) and enable its
find-tasks-by-date tool.
Save as plate_today.swift, place localai-cli and
localai-playground-run next to it, then run swift plate_today.swift.
let requiredConnectors = ["clock", "calendar", "reminders"]
// ...preflight: read localai-config.json, check requiredConnectors are all
// in "connectors_enabled" and the Todoist server/tool are connected +
// enabled, print a checklist, exit(1) before calling localai-cli if
// anything's missing...
let request: [String: Any] = [
"system_prompt": "You are a friendly, concise personal assistant. Always start by "
+ "calling getCurrentTime to find out today's actual date — never assume or guess "
+ "it. Then use the available tools to check the user's calendar, reminders, and "
+ "Todoist tasks for that date, and summarize their day.",
"user_input": "What's on my plate today? First check the current date with your clock "
+ "tool, then check my calendar events, my reminders due today, and my Todoist "
+ "tasks due today (exclude overdue tasks), then give me a friendly, concise "
+ "summary of my day.",
"connectors": requiredConnectors,
"mcp_tools": [["server": "https://ai.todoist.net/mcp", "tool": "find-tasks-by-date"]],
]
// ...same Process/Pipe subprocess call as the quickstarts above, using this request instead.
Full script: plate_today.swift.
Save as plate_today.py, place localai-cli and
localai-playground-run next to it, then run python3 plate_today.py.
REQUIRED_CONNECTORS = ["clock", "calendar", "reminders"]
# ...preflight: read localai-config.json, check REQUIRED_CONNECTORS are all
# in "connectors_enabled" and the Todoist server/tool are connected +
# enabled, print a checklist, sys.exit(1) before calling localai-cli if
# anything's missing...
request = {
"system_prompt": (
"You are a friendly, concise personal assistant. Always start by calling "
"getCurrentTime to find out today's actual date — never assume or guess it. "
"Then use the available tools to check the user's calendar, reminders, and "
"Todoist tasks for that date, and summarize their day."
),
"user_input": (
"What's on my plate today? First check the current date with your clock tool, "
"then check my calendar events, my reminders due today, and my Todoist tasks "
"due today (exclude overdue tasks), then give me a friendly, concise summary "
"of my day."
),
"connectors": REQUIRED_CONNECTORS,
"mcp_tools": [{"server": "https://ai.todoist.net/mcp", "tool": "find-tasks-by-date"}],
}
# ...same subprocess.run() call as the quickstarts above, using this request instead.
Full script: plate_today.py.
Pasting this into a real app
The quickstarts above are standalone scripts (top-level executable statements) — fine for
swift file.swift or python3 file.py, but top-level statements don't compile
inside a regular Swift file in an Xcode app target. What you actually want to copy into your project
is a plain function wrapping the same subprocess call:
struct LocalAIError: Error, CustomStringConvertible {
let description: String
}
/// Invokes `localai-cli --run` and returns its parsed JSON response.
/// Throws only if localai-cli itself couldn't be executed (e.g. missing
/// binary) - a rejection or model error comes back as a normal dictionary
/// with an "error" key, same as localai-cli's own stdout contract.
func runLocalAI(
cliPath: String,
configPath: String,
systemPrompt: String,
userInput: String,
connectors: [String] = [],
mcpTools: [[String: String]] = []
) throws -> [String: Any] {
guard FileManager.default.isExecutableFile(atPath: cliPath) else {
throw LocalAIError(description: "localai-cli not found at \(cliPath)")
}
var request: [String: Any] = [
"system_prompt": systemPrompt,
"user_input": userInput,
"connectors": connectors,
]
if !mcpTools.isEmpty { request["mcp_tools"] = mcpTools }
let requestData = try JSONSerialization.data(withJSONObject: request)
let process = Process()
process.executableURL = URL(fileURLWithPath: cliPath)
process.arguments = ["--config", configPath, "--run"]
let stdinPipe = Pipe(), stdoutPipe = Pipe(), stderrPipe = Pipe()
process.standardInput = stdinPipe
process.standardOutput = stdoutPipe
process.standardError = stderrPipe
try process.run()
stdinPipe.fileHandleForWriting.write(requestData)
try stdinPipe.fileHandleForWriting.close()
let stdoutData = stdoutPipe.fileHandleForReading.readDataToEndOfFile()
process.waitUntilExit()
guard let json = try? JSONSerialization.jsonObject(with: stdoutData) as? [String: Any] else {
throw LocalAIError(description: "localai-cli produced no parseable JSON")
}
return json
}
Source with the fuller error-handling variant (also reads stderr, supports an --mcp
demo mode): run_localai.swift.
import json
import subprocess
def run_localai(cli_path, config_path, system_prompt, user_input, connectors=None, mcp_tools=None):
"""Invoke localai-cli --run and return its parsed JSON response.
Raises RuntimeError only if localai-cli itself couldn't be executed
(e.g. missing binary) - a rejection or model error is NOT raised, it
comes back as a normal dict with an "error" key, same as localai-cli's
own stdout contract.
"""
request = {
"system_prompt": system_prompt,
"user_input": user_input,
"connectors": connectors or [],
}
if mcp_tools:
request["mcp_tools"] = mcp_tools
result = subprocess.run(
[cli_path, "--config", config_path, "--run"],
input=json.dumps(request),
capture_output=True,
text=True,
)
try:
return json.loads(result.stdout)
except json.JSONDecodeError:
raise RuntimeError(
f"localai-cli produced no parseable JSON (exit {result.returncode}): "
f"{result.stdout!r} / {result.stderr!r}"
)
Source with the fuller error-handling variant: run_localai.py.
--chat takes a messages array instead of
system_prompt/user_input — the two shapes aren't interchangeable. Sending a
--run-shaped request to --chat (or vice versa) fails with
{"error": "The data couldn't be read because it is missing."}, which is really just a
request-shape mismatch, not a config or MCP problem.
Troubleshooting
{"error": "connector '...' is not enabled in the config"}— turn it on in Local AI Settings.{"error": "MCP server \"...\" is not configured"}— the URL in your request doesn't match any server inlocalai-config.json; check it against the MCP Servers panel.{"error": "MCP server \"...\" is not enabled"}/"MCP tool \"...\" ... is not enabled"— the server or that specific tool's toggle is off.{"error": "MCP tool \"...\" is not known on server \"...\""}— tool names are case-sensitive and come directly from the server; double-check spelling.- No output, or a connection-style failure instead of a JSON
{"error": ...}— make sure LocalLM Lab is actually running, not just installed: connector and MCP calls relay through its background process, which must be up. {"error": "The operation couldn't be completed. (FoundationModels.LanguageModelSession.GenerationError error -1.)"}— a generic on-device generation failure from FoundationModels itself, not a config/connector rejection (the request already passed validation). Treat it as transient — just retry.- A reply that doesn't look like it used any tool at all — check whether your prompt actually needs the connector/tool you requested; the model won't reach for a tool a prompt doesn't call for, even if it's enabled.
{"error": "config file not found: ..."}— fix the--configpath;localai-clinever falls back to an implicit location.
Contact
Questions, or building something with this? Contact neuron@thisbrain.ai or join our Discord.