Run AI models locally on your Mac

Why LocalLM Lab matters

An Apple Silicon Mac is a genuinely capable place to run language models. Every Mac with Apple Intelligence already ships one — Apple's Foundation Models framework runs it directly on your Mac's silicon — and the same unified memory and GPU can run open-weight models too, downloaded once and run via MLX, from small, fast ones to large models that take on work you'd otherwise send to a cloud API. No network calls, no per-token costs, no data leaving your machine. LocalLM Lab exists to make running them easy — pick a model, download it if it isn't already there, and find out what it can and can't do on your own hardware, whether you're experimenting, building against it, or just learning about AI, before committing to it in a real product.

It's also more than an app to try that out in. The same SDK LocalLM Lab runs on is free to build your own native macOS apps on — turning a Mac into a real AI powerhouse, two ways. Run models fully locally — Apple's on-device model or open-weight models you download and run yourself — for zero cloud cost and a genuinely different privacy and security posture, since a prompt or tool result never has to leave the machine. Or wire the Mac up as an AI hub: a real MCP client gives your app's model discovery and tool-calling access to the services you actually use — Slack, GitHub, Notion, Linear, and the rest — OAuth included. That hub still does the connecting even when the model itself is a hosted one — Claude, GPT, whatever you've configured never talks to those services directly; every tool call is proxied through your local MCP client, so your Mac, not the cloud provider, is what every tool call and credential actually passes through. Most real apps end up doing both at once — reasoning locally where that's enough, handing off to a bigger model when it isn't, tools reachable either way. What the SDK gives you →

AI Models

The AI Models panel is where you pick which model runs on your Mac. Apple's on-device model is there out of the box; on macOS 27 you can also download open-weight models — from small, fast ones to large models that use most of your Mac's memory — and run them fully locally via MLX. When you do want a bigger model, add a hosted provider (Claude, GPT, OpenRouter, or any OpenAI-compatible server) with your own key. Whichever one you pick, the rest of the app — your connectors, your MCP servers — reaches it the same way.

Downloaded models are validated against your Mac's memory before they run, pinned to one exact version, and every file is hash-checked. Each has a sliders button to tune it — temperature, output length, thinking on or off, and more — and can be paired with a smaller speed helper model or a LoRA adapter for faster replies or a different writing style. How model selection works, a field report on which open-weight models tool-call, and how pairing works →

Prompt Playground

Prompt Playground is where you try a model out — including how it behaves with tools. Pick a model, turn on the connectors or MCP servers you want it to have, write or paste in a system prompt, type or import a user input, and hit Run to see exactly how it responds — tool calls and all — with a stopwatch showing how long it took.

Turn on Benchmark (macOS 27) to time a run in detail: time to first token, prompt and generation speed, and how much memory the model actually uses on your Mac. Save as CSV after each run to line up models — or the 4-, 6- and 8-bit versions of one — side by side, or copy a one-line summary to share. How Benchmark works →

By default every run is independent — no conversation memory carried between runs — so you can freely vary the system prompt, the input, the tools you've enabled, or the model itself to see how each change shapes the response. Flip Multi-turn on to keep a conversation instead. When a connector or MCP tool is enabled, a Tool calls list shows every call the model actually made, not just the final answer. Output renders verbatim, including raw JSON, so you can test structured-output prompts too; for open-weight models that emit chain-of-thought, a "Thinking…" pane keeps it separate from the answer (or a per-model Turn thinking off setting skips it).

The system prompt and user input aren't simply concatenated into one blob of text — they're sent as distinct roles through Apple's FoundationModels API. The system prompt becomes a session's instructions; the user input is the message passed to respond(to:):

LanguageModelSession(instructions: systemPrompt)
    .respond(to: userInput)

Every run is appended to a plain-text log at ~/Library/Logs/LocalLM Lab/dev.log — one timestamped entry per run, with the model, system prompt, user input, each tool call and its output, and the answer each clearly labeled. Open it in any text editor to review or search past runs; it isn't cleared between launches.

SDK

Building your own native macOS app? LocalLM Lab's SDK gives it a real MCP client and a model layer — Apple's on-device model, Claude, open-weight MLX models, and hosted providers behind one protocol, with routing and residency the SDK owns. For open-weight models it also covers the risky part of downloading them at runtime: a trust policy, hash-verified files, exact-version pins, and deliberate updates and cleanup. Your app's AI features can discover and call tools on real MCP servers (Slack, Todoist, GitHub, Notion, Linear, and others), including their OAuth flows — and with the on-device model, without a single prompt or tool result ever leaving the device. That's real privacy, tighter security, and no per-token API bill for the reasoning step itself.

It also gives your app instant Calendar, Reminders, Contacts, Location, and Filesystem access — no wrappers to write, and ready-made Tools so most apps don't need to custom write their own tool-calling adapters — and is proven to work inside App Sandbox, with a signed, verified path to a Mac App Store .pkg. It's not a demo: LocalLM Lab itself runs on this SDK.

New in 2.0: decision models. Instead of asking a chat model and parsing its reply, ask a few typed questions about a message — yes / no, one of several options, a scale — and get one allowed answer each, with how sure the model is, in milliseconds. On the Mac (OpenJev on an MLX model) or hosted (TypeSafe's Jev, Featherless), behind one API, and next to your chat model. Decision models in your app →

See what it does and how to get started →

API Lab

Not in the Mac App Store version of LocalLM Lab — use the downloadable app.

Turn your Mac into an AI server. API Lab turns whichever model you've selected into a local, OpenAI-compatible /v1 API — point existing tools and scripts built for the OpenAI API at your own Mac instead of a paid cloud endpoint. In 2.0 it's a native Swift HTTP server running in the app itself (localhost, plus optional LAN HTTPS with a self-signed certificate) — no separate server process.

It's a way to learn how to build AI-enabled apps against a real API without needing to hand over a credit card or pay per token just to get started. Because everything runs on your machine, it's also naturally private and free to experiment with — but the real point is giving you a zero-cost way to learn how AI app development actually works.

Turning it on is a toggle away in a simple settings screen, where you'll find the local address and access token your code needs to connect. Turn on request logging in that same screen and every incoming call is appended to ~/Library/Application Support/LocalLM Lab/api-requests.log — method, path, status, duration, and the full request/response bodies — handy for confirming your code is actually reaching it.

Example code for the OpenAI-compatible API here. Examples include a Python-based chatbot UI.

localai-cli

Not in the Mac App Store version of LocalLM Lab — use the downloadable app.

Want to experiment with the local AI programmatically — trying it against different connectors and MCP servers from a script, without building a real app? localai-cli is a small, self-contained binary you drive from a script: write a JSON request to its stdin, read a JSON response from stdout. No server, no HTTP, no app to build — and it's config-aware, so whatever connectors and MCP tools you've already granted through LocalLM Lab's own settings stay the ceiling on what a given request can use.

See the interface, plus Swift and Python examples →

Model Context Protocol

MCP is quickly becoming the standard way AI applications connect to real tools and data — think of it as a USB-C port for AI. Most developers have read about it; few have watched the protocol work end to end. LocalLM Lab is built to be that hands-on place, with a full client at the current spec revision (2026-07-28 — new in 2.0) that falls back automatically to 2025-11-25 and earlier for servers that haven't moved yet.

Connect production servers. Slack, Linear, GitHub, Notion, Todoist and others, straight to whichever model you're running — and watch the real exchange: capability negotiation, tool discovery, OAuth, live tool calls. No sandbox, no toy server, no cost. Which servers have been tested, and how to connect each →

Your Mac is the hub, even for hosted models. Route a session to a hosted provider — Claude, GPT, OpenRouter — and its tool calls still go through this same local client: the hosted model sends back a tool call, LocalLM Lab is what actually reaches Slack or GitHub with your own OAuth token from your own Keychain, and only the result goes back to the model. The cloud model never talks to your tools directly.

Building an MCP server? LocalLM Lab is an independent client you can drive with a range of models — from Apple's small on-device model to Claude — to see how each one actually handles your tools, with a wire-level protocol log. Testing your MCP server →

Want an MCP client in your own app? The LocalLM Lab SDK ships this same one as a Swift package — no protocol or OAuth stack to write. The MCP client SDK →

Connectors

Turn on connectors in the Connectors panel to give the model real information from your Mac to answer with — your calendar, reminders, contacts, current location, or the weather — instead of only what it already knows. Every connector is off by default and stays on your Mac except Weather, which is clearly flagged as the one exception that leaves the device. In Security you set how much each connector may do — Read-only, Changes (create and edit) or Full (also delete) — and whether Prompt Playground asks you to confirm a change first.

See what each connector does →

Getting started

1. Download the app and launch it once — macOS will ask for Apple Intelligence / model access on first run.
2. Pick a model in AI Models — Apple's on-device model needs nothing else; an open-weight model downloads on demand; a hosted provider needs your own API key.
3. Turn on what you need — a Connector for real data from your Mac, an MCP server for a real external tool. Both are off by default.
4. Reach it the way that fits — Prompt Playground to experiment by hand, API Lab or localai-cli to script it, or the SDK to build it into your own app.

New to the app? The User Guide walks through every panel — Prompt Playground, AI Models, Connectors, Security, MCP Servers, Configuration Profiles, and API Lab.

Support LocalLM Lab

LocalLM Lab is built by ThisBrain AI as an independent project — free, no paid tier, not funded by anyone. If it's useful to you, you can help keep it maintained. Support covers the Apple Developer Program, test hardware for new Apple Silicon, the model testing behind the open-weight field reports, and the hours of keeping it working as macOS, the models, and the Mac hardware change.

Buy me a coffee