User Guide

Every panel in the LocalLM Lab app, what it does, and how to use it

The LocalLM Lab app is a tool for experimenting with AI on your Mac. Run models fully locally — Apple's on-device model, or the latest open-weight models (Qwen, DeepSeek, Llama, and others) you download and run yourself — or reach for a cloud provider (Claude, GPT, OpenRouter) when you want one, side by side in the same app. Connect real data sources and tools — your calendar, your files, MCP servers — and watch exactly how a model uses them. Iterate on system prompts and inputs until a model does what you actually want, with every call, tool result, and raw response visible, not hidden behind a chat bubble.

It's one app with several windows. Each window is a panel for one part of the system: where you run prompts, which model answers, what the model can see, and how much it's allowed to do without asking. This guide walks through every panel in order, from a fresh install to advanced use.

If you're building your own app rather than using this one, see the SDK guide instead — the app is built on that SDK, but this page is about the app.

Before you start

LocalLM Lab runs on Apple Silicon Macs with macOS 26 or newer and Apple Intelligence enabled (System Settings → Apple Intelligence & Siri). It's free, needs no account, and is signed and notarized. Download it from the home page, drag it to Applications, and open it.

On first launch the app starts a small background helper — the inference process that actually runs the models — and supervises it for as long as the app is open. You don't manage the helper yourself; if it crashes, the app restarts it. Quitting LocalLM Lab (⌘Q) shuts the helper down too. Closing the last window does not quit the app, so a script or application pointed at API Lab keeps working with no window open.

macOS 26 vs. macOS 27

One build covers both. On macOS 26 you get Apple's on-device model, every connector, MCP servers, and API Lab. The full model layer — Private Cloud Compute, locally-run open-weight (MLX) models, and online providers like OpenAI and Claude — needs macOS 27, so the AI Models panel only appears there.

The windows

Every panel opens from the Window menu, or with a keyboard shortcut. Prompt Playground is the main window and opens on launch; the rest open on demand.

The File menu has Save Configuration As… (⇧⌘S) and Load Configuration… (⇧⌘O) — see Configuration Profiles.

Prompt Playground

Three panes: a system prompt, a user input, and the model's output. Write or paste a system prompt, type or import an input, and press Run. A stopwatch shows how long the response took, and the "via" line shows which model actually answered.

The two inputs aren't concatenated — they're sent as distinct roles through Apple's FoundationModels API: the system prompt becomes the session's instructions, the user input is the message. Output renders verbatim, including raw JSON, so structured-output prompts are easy to test.

Single-shot vs. multi-turn

By default every Run is independent — no memory carried between runs — so you can vary the prompt or input freely and see how each change lands. Flip Multi-turn on to keep a conversation: the button becomes Send, replies accumulate, and New Conversation clears it. A multi-turn conversation is a live session on the helper; it holds the connectors, MCP tools, and security settings that were in effect when it started, so change those and start a new conversation to pick them up.

No thinking

With an open-weight (MLX) model selected, a No thinking switch appears next to Multi-turn. On, it asks the model to skip its <think>…</think> reasoning pass and answer directly — faster, and no reasoning block to read past. It works on models whose chat template supports the toggle (the Qwen3 family and similar); a model that always reasons (such as DeepSeek-R1) reports that it can't be run this way rather than ignoring the setting. The switch is hidden for Claude, online providers, and Apple's on-device model — none of them expose a reasoning-off control. It takes effect on the next run (or the next conversation, in multi-turn).

Tools, resources, and confirmations

If you've enabled connectors or MCP tools, the model can call them mid-response. You'll see each call and its result inline. When Security is set to confirm a given call, a yes/no card appears and the turn waits for you. Resources… and Prompts… open pickers for MCP resources and prompt templates you've turned on — a resource is only attached to a turn if you pick it here.

Every run is also appended to a plain-text log at ~/Library/Logs/LocalLM Lab/dev.log — one timestamped chunk per run, not cleared between launches.

AI Models

Requires macOS 27. On macOS 26 the on-device model is the only option and this panel is hidden.

This panel picks the default model — the one a request uses when it doesn't name one. Apple's own LanguageModel protocol is what puts every one of these behind a single API: Apple's on-device model, Private Cloud Compute, and Claude (via Anthropic's ClaudeForFoundationModels package) all conform to it directly. LocalLM Lab's SDK extends the same protocol rather than working around it — open-weight (MLX) models and online providers get their own conformances too — so every model answers through the exact same call, however it's actually hosted. This is where you choose the default:

A request to API Lab can still override the default per call, by naming its own model explicitly — an explicit model always wins over whatever's selected here. Prompt Playground doesn't have a separate per-run picker; it always runs whichever model is currently the default, set on this panel. The built-in Claude model keeps its key on its own row; each online provider (OpenAI, Anthropic Messages, OpenRouter, or any OpenAI-compatible base URL) keeps its key separately in the Keychain — nothing is shared between them.

Advanced → Tool result size (Compact / Balanced / Full) controls how much of a single MCP tool result the model keeps before it's trimmed. It's sized to the model automatically; reach for it only if a small model is giving incomplete answers on large tool results. Ordinary file reads aren't affected.

More on models, MLX, and adding a Hugging Face model →

Connectors

Connectors give the model real information from your Mac instead of only what it already knows. Every connector is off by default and turned on individually.

Two rules hold across all of them. Every connector is read-only by default — Calendar and Reminders are the only ones that can also create (a new event or reminder), and nothing can modify or delete anything unless you raise its level in Security. And everything stays on your Mac except Weather, which is clearly flagged.

For Calendar, Reminders, Contacts, and Location, turning the connector on for the first time triggers the matching macOS permission prompt. If you previously denied one, macOS won't ask again — the panel shows an "Open System Settings" link for that case.

Filesystem needs a folder: click Grant folder… and pick one. The model can read files inside that folder and nowhere else. Edit descriptions on any connector lets you rewrite what the model is told each tool does — useful when tuning a small model.

What each connector does, with example prompts →

Security

By default the model can use every tool you've enabled without asking. The Security panel is where you tighten that — per connector and per MCP server. It applies to both Prompt Playground and API Lab.

Per-connector level

Each connector that can change data gets a level:

• Read-only — the model can look things up, nothing more.
• Changes — read, plus create and edit. The default.
• Full — read, create, edit, and delete.

Confirm each change (Playground) adds a yes/no prompt before any create / edit / delete call from that connector.

Per-MCP-server

An MCP server's tools are written by its operator — LocalLM Lab can't vouch for them. For a server you don't fully trust, turn on Confirm each tool call. Once a call is confirmed you can mark that tool "always allow" so the same server still runs it without prompting. A separate toggle controls whether the model may read that server's resources — off by default, because resources are where injected instructions can enter.

MCP Servers

The Model Context Protocol is a standard way for AI apps to reach real tools and data. This panel connects production MCP servers straight to your model and lets you watch the protocol work — capability negotiation, tool discovery, OAuth, live calls.

Adding a server

1. Paste the server's URL and a display name.
2. Pick an auth type:
   • None — no sign-in, or the server discovers OAuth itself on the first call.
   • Personal Access Token — paste a token the server issued (e.g. GitHub).
   • OAuth (manual client) — for servers with no dynamic registration; paste a Client ID from the server's developer console (e.g. Slack).
3. Click Add. The server connects, negotiates, and lists its tools.

Adding a server is itself the consent step — you're vouching for it. Tokens are stored in the Keychain, never in a file.

Choosing tools

Every tool starts disabled — a real server can expose 50+ tools and the on-device model has a small context budget, so you opt in to the handful you need. The context budget line shows the running token cost of what's enabled. Resources work the same way and are enabled per resource.

Connect, disconnect, remove

Copy Diagnostics puts a redacted connect/negotiate/authorize trace on the clipboard for a bug report. Save As exports a connected server's tool and resource list to a text file.

Every server that's been tested, and how to connect each →
Building an MCP server? Test it against LocalLM Lab →

Configuration Profiles

A configuration profile is a saved snapshot you can switch between — built for one specific job: figuring out which combination of MCP tools and tool descriptions a small model needs to reliably complete a task. Set up a combination, save it, try another, and flip back to compare.

What's in a profile

The model and provider setup is deliberately left out — the point is to vary the tools around the model, not the model itself. Secrets are never written to the file, so a profile is safe to keep in a folder or a repo; it references Keychain items that already exist on this Mac.

Saving

File → Save Configuration As… (⇧⌘S). Profiles default to the same folder as the app's own config (~/Library/Application Support/LocalLM Lab/); the file name becomes the profile name.

Loading

File → Load Configuration… (⇧⌘O). Loading a profile:

1. Is refused while a Prompt Playground run or an API Lab request is in flight — finish or cancel it first.
2. Replaces the saved settings; leaves the model, providers, and Filesystem folder untouched.
3. Resets Security to its cautious baseline (as if you'd relaunched).
4. Turns API Lab off.
5. Disconnects every MCP server and relists them disconnected, with the profile's tool and resource picks restored. Reconnect each when you're ready; a server you'd already signed into reconnects without a new sign-in.
6. Closes the settings windows and opens a "needs setup" summary — it lists only what needs your attention: a Filesystem folder to re-grant, a macOS permission that will be asked for, or an MCP server that needs a fresh sign-in.
7. Opens Prompt Playground.

API Lab

API Lab turns your Mac into an AI server, with the selected model running inference behind an OpenAI-compatible API. Point any tool or script written for the OpenAI API at your Mac instead of a paid cloud endpoint. It's a zero-cost way to learn how AI app development actually works.

Multi-turn conversations sent as OpenAI-style messages arrays are an approximation, not full multi-turn parity: Foundation Models takes instructions plus one prompt string, so system messages become instructions and the rest are flattened into a single prompt rather than replayed as a live, stateful session the way Prompt Playground's own Multi-turn mode works.

1. Turn On. A localhost endpoint starts on the first free port from 8765.
2. Copy the base URL and the token (hidden for safety; Copy fetches it from the Keychain). Regenerate issues a new one.
3. Optionally turn on Allow LAN access to reach it from another machine over HTTPS — this exposes the endpoint to your whole network, so anyone with the token can use it. Download the self-signed certificate and trust it on the client. Localhost doesn't need the certificate.
4. Turn on Log incoming API calls to append every request to ~/Library/Logs/LocalLM Lab/api-requests.log — method, path, status, duration, and full bodies.

Sample clients, including a Python chatbot UI, are in the examples repo.

CLI & SDK

localai-cli is a small binary you drive from a script: write a JSON request to stdin, read a JSON response from stdout. No server, no HTTP. It's config-aware — the connectors and MCP tools you've granted in the app stay the ceiling on what a request can use. Interface and examples →

The SDK is for building your own native macOS app on the same on-device model + MCP client the app uses. SDK guide →

Where things live

macOS permissions (Calendar, Reminders, Contacts, Location) are held by macOS itself, in System Settings → Privacy & Security — not by LocalLM Lab. The app can only ask; granting, revoking, and re-granting happen there.

Troubleshooting

"LocalLM Lab isn't running" from a connector tool

The background helper reaches connector data through the app. Make sure LocalLM Lab is open, then try again.

A connector tool says access was denied

macOS recorded a "no" for that permission and won't prompt again. Open System Settings → Privacy & Security, find the category (e.g. Calendars), and enable LocalLM Lab.

"Grant Calendar access…" does nothing

You're running an unsigned build with no usage description — macOS silently refuses. Use the signed release from the home page.

An MCP server's resources come back unchecked after reconnecting

Newly discovered resources default to disabled for safety; your saved picks are re-applied once the server finishes connecting. If they don't stick, disconnect and reconnect once more.

A Security setting reverted on its own

Expected — Full levels and "allow all" choices don't survive a relaunch or a configuration Load. Set them again for the session.

Reporting a bug — what to include

Nothing here is sent anywhere automatically; you attach it yourself when you file an issue.

• An MCP problem (a server won't connect, an auth flow fails, a tool call errors): open the MCP Servers panel and click Copy Diagnostics. It puts a redacted trace of the recent connect / negotiate / authorize activity on the clipboard — no tokens, no tool results, just the protocol milestones and errors. Paste that into the issue.
• A model / prompt problem (wrong output, a run that hangs or fails): the last run is at the bottom of ~/Library/Logs/LocalLM Lab/dev.log — model, system prompt, user input, and output, one timestamped block per run. Copy the relevant block. (It records your prompts and the model's output, so review it before sharing.)
• An API Lab problem: turn on Log incoming API calls, reproduce it, and grab the matching lines from ~/Library/Logs/LocalLM Lab/api-requests.log.

For a stubborn MCP issue on a release build, you can turn on verbose protocol logging without a rebuild — run this in Terminal, relaunch LocalLM Lab, reproduce, then Copy Diagnostics again:

defaults write ai.thisbrain.locallmlab.chooser MCPLogLevel debug

Set it back to info (or delete the key) afterwards.

Full list of known issues →