Build with an AI coding agent
Set up Claude Code, Codex or another coding agent so that what it writes against the SDK compiles, and is safe
For developers who already build Mac apps and want an agent to do the SDK part. Terms: see the glossary.
You need a Mac with Apple Silicon, macOS 27 and Xcode 27, and Apple Intelligence turned on. Apple’s on-device model also runs on macOS 26; open-weight, hosted and Private Cloud Compute models need macOS 27. The first chat needs no account or key.
The short version. The repo ships an AGENTS.md and a skill, locallm-swift-app. Start your
agent in a clone of the repo and it finds both. Then tell it what it can’t read from the SDK: the version, which models, how
the app is distributed, who approves tool calls, and how to check the result.
Set it up
Two ways, depending on where the agent works.
- Work inside a clone.
git clone https://github.com/ancientcomputing/locallm, check out the3.0.0-GAbranch, and start your agent in that folder. Claude Code picks the skill up from.claude/skills/. Other agents readAGENTS.md, which points to the skill’s files. - Work in your own project. Copy
skills/locallm-swift-appinto~/.claude/skills/(Claude Code), and give the agent the docs: clone the repo beside your project and start withclaude --add-dir ../locallm. If it can’t see the clone, the skill falls back to the same files on the3.0.0-GAbranch.
An agent that reads the web can start from llms.txt, which lists the docs worth fetching, with links to the raw files.
Two prompts to start from
A good prompt names the things an agent can’t infer. Change the details to fit your app.
Starting from nothing (no project yet, new to Mac apps):
Use the locallm-swift-app skill. I have a Mac with Apple Silicon, macOS 27 and Xcode 27, but I have never built a Mac app.
Create a new SwiftUI Mac app called Chatty, using LocalLM SDK 3.0.0-GA, with a chat window that talks to Apple's on-device
model. Start from docs/quickstart.md in the repo. I don't use Xcode much, so make it a Swift package I can open in Xcode and
run. Build it, then tell me exactly how to run it, and explain in plain language what each file does and where the SDK is used.
Adding to an existing app: a chat on Apple’s on-device model with a tool:
Use the locallm-swift-app skill. In this Xcode project, add a SwiftUI chat on LocalLM SDK 3.0.0-GA using Apple's
on-device model and the Calendar tool. It is a direct-download app with a window, so ask the user before any tool that
changes data. Start from examples/mcp-chat for the chat window and examples/plate-today-tools for the Calendar tools, and confirm
API names in docs/api-surface.md, not from memory. When you are done, run a turn that creates a calendar event and show me
it was not denied.
An open-weight model that is downloaded with the user’s consent:
Use the locallm-swift-app skill. Add Qwen3-4B-4bit through MLX to this app. Loading must not download silently: check
the size, ask the user, then download. Show a progress bar, and handle a model that is not on disk.
We ran all three against a fresh agent. Each produced code that compiled against the 3.0.0-GA frameworks and followed the rules below. We did not run the resulting apps with a person at the keyboard, so the approval sheet and the download were not exercised.
↑ TopWhat an agent gets wrong in 3.0
These are the points AGENTS.md puts first, because each one fails quietly or only at build time.
- A session with tools needs an
authorizer:. With none, a 3.0 session refuses anything that may change data or send it out, and the model readsDENIED: …. Nothing throws. An app with a window asks the user (ConfirmingToolAuthorizer); a batch job with no one to ask passesAllowingToolAuthorizer()and says why. See the tool authorizer. - Loading a model never downloads it. Check
availability, callvalidate(itsweightBytesis the size to show), ask the user, thendownload. - The SDK’s enums are not frozen. A
switchover one needs@unknown default. - Tools given directly to Apple’s
LanguageModelSession(tools:)bypass the authorizer. Give them to the SDK’smakeSession. - A Swift package program needs a linker line. Without
-Xlinker -rpath -Xlinker @executable_pathit builds, then fails at launch withLibrary not loaded: @rpath/…. The quickstart manifest has it; an Xcode app project doesn’t need it. - Names come from
api-surface.md, not from memory. It is generated from the SDK and lists every public name. - MCP tools reach the model as
<prefix>_<tool>, a server starts untrusted, and a server that changes its tools is held for review. See MCP security in your app.
Check what it built
- Build it, then run a turn that calls a tool that writes, and confirm the result is not
DENIED. A refused call does not throw. - Anything that uses Calendar, Reminders, Contacts, Location, OAuth, Keychain or App Sandbox needs a real, signed
.appto test. A bareswift runcan pass or fail for the wrong reason. - Read the diff for any place the agent hands tools to
LanguageModelSessiondirectly, or passesAllowingToolAuthorizer()in an app where a person is present.
More: Quickstart · Examples · Security in the SDK · the skill · llms.txt