localai-cli

Script or experiment against a configured model from Swift or Python — no app to build, no server to run

Three ways to reach a model from outside LocalLM Lab, for three different jobs. API Lab gives you an OpenAI-compatible HTTP endpoint for scripts and tools that already speak that protocol. The SDK is for building a real native macOS app around the model layer, connectors, and MCP — signing, entitlements, a UI, the works. localai-cli is for the case in between: you want to call a model from a script or a quick prototype, with none of that — no HTTP server, no app to build, no Xcode project. Just a subprocess and JSON on stdin/stdout.

That makes it a good fit for two things in particular. Scripting tests or experiments without the overhead of building an app around them — try a connector, a new MCP tool, or a model switch from a one-off script instead of wiring up a UI to poke at it. And prompt testing: loop over a list of prompts — or variations of one prompt, different phrasing, different system instructions — and capture each response so you can see, side by side, which version actually gets the result you're after.

localai-cli is a small, self-contained binary: your script spawns it as a subprocess, writes one JSON request to its stdin, and reads one JSON response back from its stdout. That's the entire interface.

It's config-aware: it reads the same localai-config.json that LocalLM Lab's Connectors and MCP Servers panels write, so whatever connectors and MCP tools your user has already granted through the app's UI are exactly what your code is allowed to request — nothing more. localai-cli never edits that file, only reads it.

Get it

1. Run LocalLM Lab at least once, and use Connectors / MCP Servers to enable whichever connectors and tools you plan to call with. This writes ~/Library/Application Support/LocalLM Lab/localai-config.json.
2. Download the localai-toolkit-<version>-arm64.zip + .sha256 — 1.0.0-beta.4 through 1.0.0-RC.1 shipped it as a release asset on ancientcomputing/locallm-releases; from 1.0.0-GA on, get localai-toolkit-1.0.0-GA-arm64.zip + its checksum from this repo's own releases instead, alongside the app DMG (earlier versions are checked into locallm/toolkit/) — and unzip it: two binaries as a matched pair, localai-cli and localai-playground-run (the raw model-invocation helper localai-cli wraps). Keep them together in the same directory; localai-cli's default --helper-path assumes "next to me."
3. Full runnable Swift and Python examples — including the ones on this page — live in examples/localai-cli-swift and examples/localai-cli.

Interface

localai-cli --config /path/to/localai-config.json --status
localai-cli --config /path/to/localai-config.json --run   <<< '{"system_prompt": "...", "user_input": "...", "connectors": ["clock"]}'
localai-cli --config /path/to/localai-config.json --chat  <<< '{"messages": [...], "connectors": ["filesystem"]}'
localai-cli --version

Connectors and MCP tools: request vs. enabled

connectors_enabled in the config is the ceiling — what your user has granted via LocalLM Lab's own settings UI. The connectors field in each request is the selection — which of those this one call actually wants active. The rule is requested ⊆ enabled:

• Request something not in connectors_enabled → localai-cli rejects it with a JSON {"error": "..."}, before the model is ever invoked.
• Enabled but not requested → simply not given to that call, no error.
• Omit the field entirely → nothing is active. This is a deliberate least-privilege default, not an oversight — every tool a given call can use has to be named explicitly in that call's own request.

MCP tools work the same way, at {server, tool} granularity, via a separate mcp_tools array (kept distinct from connectors — see MCP Servers for how those get enabled in the first place):

{
  "system_prompt": "You are a concise assistant. Use the available tool to answer.",
  "user_input": "What documentation topics are available for nickclyde/duckduckgo-mcp-server?",
  "mcp_tools": [{"server": "https://mcp.deepwiki.com/mcp", "tool": "read_wiki_structure"}]
}

Quickstart: two ways to try it in under a minute

Pick your language. Both do the exact same thing: ask the on-device model the current time, using the clock connector — the only one with no macOS permission dialog, so there's nothing to approve beyond one toggle.

One-time setup: open Connectors in LocalLM Lab and turn on System Clock. That's it.

#!/usr/bin/env swift
import Foundation

let env = ProcessInfo.processInfo.environment
let cliPath = env["LOCALAI_CLI_PATH"]
    ?? URL(fileURLWithPath: #filePath).deletingLastPathComponent().appendingPathComponent("localai-cli").path
let configPath = env["LOCALAI_CONFIG_PATH"]
    ?? NSHomeDirectory() + "/Library/Application Support/LocalLM Lab/localai-config.json"

let request: [String: Any] = [
    "system_prompt": "You are a concise assistant.",
    "user_input": "What time is it right now?",
    "connectors": ["clock"],
]
let requestData = try! JSONSerialization.data(withJSONObject: request)

let process = Process()
process.executableURL = URL(fileURLWithPath: cliPath)
process.arguments = ["--config", configPath, "--run"]

let stdinPipe = Pipe(), stdoutPipe = Pipe()
process.standardInput = stdinPipe
process.standardOutput = stdoutPipe
process.standardError = Pipe()

try! process.run()
stdinPipe.fileHandleForWriting.write(requestData)
try! stdinPipe.fileHandleForWriting.close()

let stdoutData = stdoutPipe.fileHandleForReading.readDataToEndOfFile()
process.waitUntilExit()

let response = try! JSONSerialization.jsonObject(with: stdoutData) as! [String: Any]
if let error = response["error"] as? String, !error.isEmpty {
    print("FAIL - \(error)")
    exit(1)
}
print(response["answer"] as? String ?? "")

import json, os, subprocess

CLI_PATH = os.environ.get(
    "LOCALAI_CLI_PATH", os.path.join(os.path.dirname(__file__), "localai-cli")
)
CONFIG_PATH = os.environ.get(
    "LOCALAI_CONFIG_PATH",
    os.path.expanduser("~/Library/Application Support/LocalLM Lab/localai-config.json"),
)

request = {
    "system_prompt": "You are a concise assistant.",
    "user_input": "What time is it right now?",
    "connectors": ["clock"],
}

result = subprocess.run(
    [CLI_PATH, "--config", CONFIG_PATH, "--run"],
    input=json.dumps(request),
    capture_output=True,
    text=True,
)
response = json.loads(result.stdout)

if response.get("error"):
    print(f"FAIL - {response['error']}")
else:
    print(response.get("answer", ""))

MCP quickstart: a real tool call, no auth required

Same idea, but the tool call goes out to a real MCP server instead of a local connector — DeepWiki, which needs no OAuth or API key.

One-time setup: in LocalLM Lab's MCP Servers panel, Add Server with URL https://mcp.deepwiki.com/mcp, auth type None, then enable its read_wiki_structure tool.

let request: [String: Any] = [
    "system_prompt": "You are a concise assistant. Use the available tool to answer.",
    "user_input": "What documentation topics are available for the GitHub repo "
        + "nickclyde/duckduckgo-mcp-server?",
    "mcp_tools": [["server": "https://mcp.deepwiki.com/mcp", "tool": "read_wiki_structure"]],
]
// ...same Process/Pipe subprocess call as above, using this request instead.

request = {
    "system_prompt": "You are a concise assistant. Use the available tool to answer.",
    "user_input": (
        "What documentation topics are available for the GitHub repo "
        "nickclyde/duckduckgo-mcp-server?"
    ),
    "mcp_tools": [{"server": "https://mcp.deepwiki.com/mcp", "tool": "read_wiki_structure"}],
}
# ...same subprocess.run() call as above, using this request instead.

Example: Plate Today — "what's on my plate today"

A fuller example than the quickstarts above: a script that checks Calendar, Reminders, and Todoist for what's due today and asks the on-device model to summarize the day. It's a plain script — no app, no EventKit calls of its own, no TCC prompts triggered by the script itself — it relies entirely on connectors and MCP tools LocalLM Lab already has permission for. Calendar and Reminders come in as connectors; Todoist comes in as an MCP tool call against Todoist's own hosted MCP server, find-tasks-by-date.

FoundationModels has no live wall-clock awareness on its own, so the script also requests the clock connector and instructs the model to look up the actual current date first, then match "today" against calendar/reminders/Todoist using that date — rather than guessing.

Before ever calling localai-cli, the script reads localai-config.json itself and checks all four sources are actually usable — the three connectors enabled, and the Todoist server connected and enabled with find-tasks-by-date enabled on it — and prints a checklist. If anything's missing, it names the exact panel/toggle to fix and exits without ever invoking the model, so you don't burn a model call on a request that was always going to fail.

One-time setup: in Connectors, turn on System Clock, Calendar, and Reminders. In MCP Servers, add Todoist (https://ai.todoist.net/mcp) and enable its find-tasks-by-date tool.

let requiredConnectors = ["clock", "calendar", "reminders"]

// ...preflight: read localai-config.json, check requiredConnectors are all
// in "connectors_enabled" and the Todoist server/tool are connected +
// enabled, print a checklist, exit(1) before calling localai-cli if
// anything's missing...

let request: [String: Any] = [
    "system_prompt": "You are a friendly, concise personal assistant. Always start by "
        + "calling getCurrentTime to find out today's actual date — never assume or guess "
        + "it. Then use the available tools to check the user's calendar, reminders, and "
        + "Todoist tasks for that date, and summarize their day.",
    "user_input": "What's on my plate today? First check the current date with your clock "
        + "tool, then check my calendar events, my reminders due today, and my Todoist "
        + "tasks due today (exclude overdue tasks), then give me a friendly, concise "
        + "summary of my day.",
    "connectors": requiredConnectors,
    "mcp_tools": [["server": "https://ai.todoist.net/mcp", "tool": "find-tasks-by-date"]],
]
// ...same Process/Pipe subprocess call as the quickstarts above, using this request instead.

REQUIRED_CONNECTORS = ["clock", "calendar", "reminders"]

# ...preflight: read localai-config.json, check REQUIRED_CONNECTORS are all
# in "connectors_enabled" and the Todoist server/tool are connected +
# enabled, print a checklist, sys.exit(1) before calling localai-cli if
# anything's missing...

request = {
    "system_prompt": (
        "You are a friendly, concise personal assistant. Always start by calling "
        "getCurrentTime to find out today's actual date — never assume or guess it. "
        "Then use the available tools to check the user's calendar, reminders, and "
        "Todoist tasks for that date, and summarize their day."
    ),
    "user_input": (
        "What's on my plate today? First check the current date with your clock tool, "
        "then check my calendar events, my reminders due today, and my Todoist tasks "
        "due today (exclude overdue tasks), then give me a friendly, concise summary "
        "of my day."
    ),
    "connectors": REQUIRED_CONNECTORS,
    "mcp_tools": [{"server": "https://ai.todoist.net/mcp", "tool": "find-tasks-by-date"}],
}
# ...same subprocess.run() call as the quickstarts above, using this request instead.

Pasting this into a real app

The quickstarts above are standalone scripts (top-level executable statements) — fine for swift file.swift or python3 file.py, but top-level statements don't compile inside a regular Swift file in an Xcode app target. What you actually want to copy into your project is a plain function wrapping the same subprocess call:

struct LocalAIError: Error, CustomStringConvertible {
    let description: String
}

/// Invokes `localai-cli --run` and returns its parsed JSON response.
/// Throws only if localai-cli itself couldn't be executed (e.g. missing
/// binary) - a rejection or model error comes back as a normal dictionary
/// with an "error" key, same as localai-cli's own stdout contract.
func runLocalAI(
    cliPath: String,
    configPath: String,
    systemPrompt: String,
    userInput: String,
    connectors: [String] = [],
    mcpTools: [[String: String]] = []
) throws -> [String: Any] {
    guard FileManager.default.isExecutableFile(atPath: cliPath) else {
        throw LocalAIError(description: "localai-cli not found at \(cliPath)")
    }

    var request: [String: Any] = [
        "system_prompt": systemPrompt,
        "user_input": userInput,
        "connectors": connectors,
    ]
    if !mcpTools.isEmpty { request["mcp_tools"] = mcpTools }
    let requestData = try JSONSerialization.data(withJSONObject: request)

    let process = Process()
    process.executableURL = URL(fileURLWithPath: cliPath)
    process.arguments = ["--config", configPath, "--run"]

    let stdinPipe = Pipe(), stdoutPipe = Pipe(), stderrPipe = Pipe()
    process.standardInput = stdinPipe
    process.standardOutput = stdoutPipe
    process.standardError = stderrPipe

    try process.run()
    stdinPipe.fileHandleForWriting.write(requestData)
    try stdinPipe.fileHandleForWriting.close()

    let stdoutData = stdoutPipe.fileHandleForReading.readDataToEndOfFile()
    process.waitUntilExit()

    guard let json = try? JSONSerialization.jsonObject(with: stdoutData) as? [String: Any] else {
        throw LocalAIError(description: "localai-cli produced no parseable JSON")
    }
    return json
}

import json
import subprocess


def run_localai(cli_path, config_path, system_prompt, user_input, connectors=None, mcp_tools=None):
    """Invoke localai-cli --run and return its parsed JSON response.

    Raises RuntimeError only if localai-cli itself couldn't be executed
    (e.g. missing binary) - a rejection or model error is NOT raised, it
    comes back as a normal dict with an "error" key, same as localai-cli's
    own stdout contract.
    """
    request = {
        "system_prompt": system_prompt,
        "user_input": user_input,
        "connectors": connectors or [],
    }
    if mcp_tools:
        request["mcp_tools"] = mcp_tools

    result = subprocess.run(
        [cli_path, "--config", config_path, "--run"],
        input=json.dumps(request),
        capture_output=True,
        text=True,
    )
    try:
        return json.loads(result.stdout)
    except json.JSONDecodeError:
        raise RuntimeError(
            f"localai-cli produced no parseable JSON (exit {result.returncode}): "
            f"{result.stdout!r} / {result.stderr!r}"
        )

Troubleshooting

• {"error": "connector '...' is not enabled in the config"} — turn it on in Connectors.
• {"error": "MCP server \"...\" is not configured"} — the URL in your request doesn't match any server in localai-config.json; check it against the MCP Servers panel.
• {"error": "MCP server \"...\" is not enabled"} / "MCP tool \"...\" ... is not enabled" — the server or that specific tool's toggle is off.
• {"error": "MCP tool \"...\" is not known on server \"...\""} — tool names are case-sensitive and come directly from the server; double-check spelling.
• No output, or a connection-style failure instead of a JSON {"error": ...} — make sure LocalLM Lab is actually running, not just installed: connector and MCP calls relay through its background process, which must be up.
• {"error": "The Security settings for Calendar don't allow changes, so existing events can't be modified. Set the Security settings for Calendar to Changes."} (or the same for reminders and contacts, or "…don't allow deleting, so … can't be deleted … to Full") — the Security settings for that connector in LocalLM Lab don't allow the tool: updateCalendarEvent/updateReminder/addContact/updateContact need Changes and deleteCalendarEvent/deleteReminder/deleteContact need Full. A level change takes effect immediately in the running app and doesn't require restarting your localai-cli script.
• {"error": "The operation couldn't be completed. (FoundationModels.LanguageModelSession.GenerationError error -1.)"} — a generic on-device generation failure from FoundationModels itself, not a config/connector rejection (the request already passed validation). Treat it as transient — just retry.
• A reply that doesn't look like it used any tool at all — check whether your prompt actually needs the connector/tool you requested; the model won't reach for a tool a prompt doesn't call for, even if it's enabled.
• {"error": "config file not found: ..."} — fix the --config path; localai-cli never falls back to an implicit location.

Contact

Questions, or building something with this? Contact neuron@thisbrain.ai or join our Discord.