localai-cli

Call your Mac's on-device model from your own Swift or Python app — no server, no HTTP

LocalLM Lab's API Lab gives you an OpenAI-compatible HTTP endpoint for scripts and tools that already speak that protocol. localai-cli is for the other case: you're building a native app — a game, a utility, a menu bar tool — and you want to reach Apple's on-device model directly from Swift or Python, without running an HTTP server at all. localai-cli is a small, self-contained binary: your app spawns it as a subprocess, writes one JSON request to its stdin, and reads one JSON response back from its stdout. That's the entire interface.

It's config-aware: it reads the same localai-config.json that LocalLM Lab's Local AI Settings and MCP Servers panels write, so whatever connectors and MCP tools your user has already granted through the app's UI are exactly what your code is allowed to request — nothing more. localai-cli never edits that file, only reads it.

The model

Every request goes to Apple's FoundationModels framework running directly on the Mac's Neural Engine — the same on-device model Prompt Playground and API Lab use. No network call, no API key, no per-token cost. Requires macOS 26+ on Apple Silicon with Apple Intelligence enabled.

Whether LocalLM Lab itself needs to still be running (not just installed) depends on the request. A request with no connectors and no mcp_tools never touches LocalLM Lab at all — localai-playground-run talks to FoundationModels directly, so you can quit the app and keep only localai-config.json plus the two toolkit binaries around. A request that names any connector or MCP tool relays that one call through LocalLM Lab's menu bar tray process over a local Unix socket, which has to be up at the moment the model actually reaches for the tool — config validation itself (connector 'x' is not enabled, etc.) still works with LocalLM Lab quit, since that only reads the config file; it's specifically the live tool call that needs the app running.

Get it

  1. Run LocalLM Lab at least once, and use Local AI Settings / MCP Servers to enable whichever connectors and tools you plan to call with. This writes ~/Library/Application Support/LocalLM Lab/localai-config.json.
  2. Download the localai-toolkit-<version>-arm64.zip asset from the LocalLM GitHub page and unzip it — it contains two binaries as a matched pair: localai-cli and localai-playground-run (the raw model-invocation helper localai-cli wraps). Keep them together in the same directory; localai-cli's default --helper-path assumes "next to me."
  3. Full runnable Swift and Python examples — including the ones on this page — live in examples/localai-cli-swift and examples/localai-cli.

Interface

localai-cli --config /path/to/localai-config.json --status
localai-cli --config /path/to/localai-config.json --run   <<< '{"system_prompt": "...", "user_input": "...", "connectors": ["clock"]}'
localai-cli --config /path/to/localai-config.json --chat  <<< '{"messages": [...], "connectors": ["filesystem"]}'
localai-cli --version
FlagWhat it does
--config <path>Required for --run/--chat. Path to a localai-config.json-shaped file — no implicit lookup, so you can ship a copy anywhere.
--statusChecks on-device model availability. Doesn't touch the config file.
--runsystem_prompt/user_input request on stdin, one-shot, no conversation memory.
--chatOpenAI-style messages array on stdin instead — use this shape, not --run's, when you have a conversation history.
--helper-path <path>Override where localai-playground-run lives. Default: same directory as localai-cli itself.
--versionPrints the build version and exits.

Connectors and MCP tools: request vs. enabled

connectors_enabled in the config is the ceiling — what your user has granted via LocalLM Lab's own settings UI. The connectors field in each request is the selection — which of those this one call actually wants active. The rule is requested ⊆ enabled:

MCP tools work the same way, at {server, tool} granularity, via a separate mcp_tools array (kept distinct from connectors — see MCP Servers for how those get enabled in the first place):

{
  "system_prompt": "You are a concise assistant. Use the available tool to answer.",
  "user_input": "What documentation topics are available for nickclyde/duckduckgo-mcp-server?",
  "mcp_tools": [{"server": "https://mcp.deepwiki.com/mcp", "tool": "read_wiki_structure"}]
}

Quickstart: two ways to try it in under a minute

Pick your language. Both do the exact same thing: ask the on-device model the current time, using the clock connector — the only one with no macOS permission dialog, so there's nothing to approve beyond one toggle.

One-time setup: open Local AI Settings in LocalLM Lab and turn on System Clock. That's it.

Save as quickstart_clock.swift, place localai-cli and localai-playground-run next to it, then run swift quickstart_clock.swift — no Xcode project needed, swift <file>.swift runs a script file directly.

#!/usr/bin/env swift
import Foundation

let env = ProcessInfo.processInfo.environment
let cliPath = env["LOCALAI_CLI_PATH"]
    ?? URL(fileURLWithPath: #filePath).deletingLastPathComponent().appendingPathComponent("localai-cli").path
let configPath = env["LOCALAI_CONFIG_PATH"]
    ?? NSHomeDirectory() + "/Library/Application Support/LocalLM Lab/localai-config.json"

let request: [String: Any] = [
    "system_prompt": "You are a concise assistant.",
    "user_input": "What time is it right now?",
    "connectors": ["clock"],
]
let requestData = try! JSONSerialization.data(withJSONObject: request)

let process = Process()
process.executableURL = URL(fileURLWithPath: cliPath)
process.arguments = ["--config", configPath, "--run"]

let stdinPipe = Pipe(), stdoutPipe = Pipe()
process.standardInput = stdinPipe
process.standardOutput = stdoutPipe
process.standardError = Pipe()

try! process.run()
stdinPipe.fileHandleForWriting.write(requestData)
try! stdinPipe.fileHandleForWriting.close()

let stdoutData = stdoutPipe.fileHandleForReading.readDataToEndOfFile()
process.waitUntilExit()

let response = try! JSONSerialization.jsonObject(with: stdoutData) as! [String: Any]
if let error = response["error"] as? String, !error.isEmpty {
    print("FAIL - \(error)")
    exit(1)
}
print(response["answer"] as? String ?? "")

Full version with error messages tailored to each failure mode: quickstart_clock.swift.

MCP quickstart: a real tool call, no auth required

Same idea, but the tool call goes out to a real MCP server instead of a local connector — DeepWiki, which needs no OAuth or API key.

One-time setup: in LocalLM Lab's MCP Servers panel, Add Server with URL https://mcp.deepwiki.com/mcp, auth type None, then enable its read_wiki_structure tool.

let request: [String: Any] = [
    "system_prompt": "You are a concise assistant. Use the available tool to answer.",
    "user_input": "What documentation topics are available for the GitHub repo "
        + "nickclyde/duckduckgo-mcp-server?",
    "mcp_tools": [["server": "https://mcp.deepwiki.com/mcp", "tool": "read_wiki_structure"]],
]
// ...same Process/Pipe subprocess call as above, using this request instead.

Full script: quickstart_mcp_deepwiki.swift.

Example: Plate Today — "what's on my plate today"

A fuller example than the quickstarts above: a script that checks Calendar, Reminders, and Todoist for what's due today and asks the on-device model to summarize the day. It's a plain script — no app, no EventKit calls of its own, no TCC prompts triggered by the script itself — it relies entirely on connectors and MCP tools LocalLM Lab already has permission for. Calendar and Reminders come in as connectors; Todoist comes in as an MCP tool call against Todoist's own hosted MCP server, find-tasks-by-date.

FoundationModels has no live wall-clock awareness on its own, so the script also requests the clock connector and instructs the model to look up the actual current date first, then match "today" against calendar/reminders/Todoist using that date — rather than guessing.

Before ever calling localai-cli, the script reads localai-config.json itself and checks all four sources are actually usable — the three connectors enabled, and the Todoist server connected and enabled with find-tasks-by-date enabled on it — and prints a checklist. If anything's missing, it names the exact panel/toggle to fix and exits without ever invoking the model, so you don't burn a model call on a request that was always going to fail.

One-time setup: in Local AI Settings, turn on System Clock, Calendar, and Reminders. In MCP Servers, add Todoist (https://ai.todoist.net/mcp) and enable its find-tasks-by-date tool.

Save as plate_today.swift, place localai-cli and localai-playground-run next to it, then run swift plate_today.swift.

let requiredConnectors = ["clock", "calendar", "reminders"]

// ...preflight: read localai-config.json, check requiredConnectors are all
// in "connectors_enabled" and the Todoist server/tool are connected +
// enabled, print a checklist, exit(1) before calling localai-cli if
// anything's missing...

let request: [String: Any] = [
    "system_prompt": "You are a friendly, concise personal assistant. Always start by "
        + "calling getCurrentTime to find out today's actual date — never assume or guess "
        + "it. Then use the available tools to check the user's calendar, reminders, and "
        + "Todoist tasks for that date, and summarize their day.",
    "user_input": "What's on my plate today? First check the current date with your clock "
        + "tool, then check my calendar events, my reminders due today, and my Todoist "
        + "tasks due today (exclude overdue tasks), then give me a friendly, concise "
        + "summary of my day.",
    "connectors": requiredConnectors,
    "mcp_tools": [["server": "https://ai.todoist.net/mcp", "tool": "find-tasks-by-date"]],
]
// ...same Process/Pipe subprocess call as the quickstarts above, using this request instead.

Full script: plate_today.swift.

Pasting this into a real app

The quickstarts above are standalone scripts (top-level executable statements) — fine for swift file.swift or python3 file.py, but top-level statements don't compile inside a regular Swift file in an Xcode app target. What you actually want to copy into your project is a plain function wrapping the same subprocess call:

struct LocalAIError: Error, CustomStringConvertible {
    let description: String
}

/// Invokes `localai-cli --run` and returns its parsed JSON response.
/// Throws only if localai-cli itself couldn't be executed (e.g. missing
/// binary) - a rejection or model error comes back as a normal dictionary
/// with an "error" key, same as localai-cli's own stdout contract.
func runLocalAI(
    cliPath: String,
    configPath: String,
    systemPrompt: String,
    userInput: String,
    connectors: [String] = [],
    mcpTools: [[String: String]] = []
) throws -> [String: Any] {
    guard FileManager.default.isExecutableFile(atPath: cliPath) else {
        throw LocalAIError(description: "localai-cli not found at \(cliPath)")
    }

    var request: [String: Any] = [
        "system_prompt": systemPrompt,
        "user_input": userInput,
        "connectors": connectors,
    ]
    if !mcpTools.isEmpty { request["mcp_tools"] = mcpTools }
    let requestData = try JSONSerialization.data(withJSONObject: request)

    let process = Process()
    process.executableURL = URL(fileURLWithPath: cliPath)
    process.arguments = ["--config", configPath, "--run"]

    let stdinPipe = Pipe(), stdoutPipe = Pipe(), stderrPipe = Pipe()
    process.standardInput = stdinPipe
    process.standardOutput = stdoutPipe
    process.standardError = stderrPipe

    try process.run()
    stdinPipe.fileHandleForWriting.write(requestData)
    try stdinPipe.fileHandleForWriting.close()

    let stdoutData = stdoutPipe.fileHandleForReading.readDataToEndOfFile()
    process.waitUntilExit()

    guard let json = try? JSONSerialization.jsonObject(with: stdoutData) as? [String: Any] else {
        throw LocalAIError(description: "localai-cli produced no parseable JSON")
    }
    return json
}

Source with the fuller error-handling variant (also reads stderr, supports an --mcp demo mode): run_localai.swift.

⚠️ --chat takes a messages array instead of system_prompt/user_input — the two shapes aren't interchangeable. Sending a --run-shaped request to --chat (or vice versa) fails with {"error": "The data couldn't be read because it is missing."}, which is really just a request-shape mismatch, not a config or MCP problem.

Troubleshooting

Contact

Questions, or building something with this? Contact neuron@thisbrain.ai or join our Discord.