Testing your MCP server with LocalLM Lab

An independent client, a range of models, and a wire-level view

If you're building an MCP server, you want to test it against a client that isn't the one you developed against. LocalLM Lab is a free, native macOS app with a full Model Context Protocol 2026-07-28 client, falling back to 2025-11-25 and earlier — a fourth independent implementation alongside Claude Desktop, Cursor, and the MCP Inspector — and it lets you drive your server with several very different models to see how each one actually handles your tools.

Why a second client matters

• Independent implementation. Conformance bugs hide in the gap between two implementations of the same spec. LocalLM Lab negotiates 2026-07-28 and falls back cleanly to older revisions, so you can check your server against a client built from the spec rather than from the same SDK you used.
• A range of models, not just one. Switch the model between runs and send the same prompt again. Most of these need macOS 27; Apple's on-device model runs on macOS 26 too.

  Model	Runs	Good for surfacing
  Apple on-device	On the Mac, ~4K context	tool descriptions that are too long, schemas that are too loose, too many tools at once
  Open-weight (MLX)	On the Mac, via MLX — any Hugging Face model you download	how a 3B–14B open model orchestrates a multi-call tool chain
  Claude, via Apple's Foundation Models	Anthropic, through the on-device model API (needs an API key)	a frontier baseline through the same code path as the local models — what should work
  An online provider	A separate path: OpenAI, Anthropic's Messages API, OpenRouter, or any OpenAI-compatible endpoint, each with its own key	the specific model your users will actually run

• The small-model stress test. A frontier model papers over a verbose tool description, a loose enum, an oversized result, or forty tools where three would do. A 3B on-device model does not — it picks the wrong tool, mangles the arguments, or loops. A server whose tools a small local model can drive is a well-designed server. LocalLM Lab makes that test a one-click thing.

The test loop

1. Connect. MCP Servers panel → Add Server → your URL. Auth type None covers no-auth and any OAuth server (the client discovers the challenge and runs the flow — OAuth 2.1 + PKCE, DCR, CIMD). Use Personal Access Token or OAuth (manual client) for the servers that need them.
2. Enable a tool or two. Every tool starts disabled. The context-budget line shows the running token cost — a useful number on its own.
3. Pick a model in AI Models, write a prompt that should exercise the tool, and Run. Each tool call and its result show inline.
4. Switch models and run the same prompt again. Save the setup as a Configuration Profile so you can flip between tool combinations while you compare.
5. Read the wire. See Diagnostics below.

Seeing the protocol traffic

The MCP client logs every decision and failure to Apple's unified logging system — connects, negotiation, fallbacks, and sign-ins, with no setting to turn on. Individual requests, tool calls, and each SSE frame are debug-level detail; to include them, turn the detail up first, then relaunch the app:

defaults write ai.thisbrain.locallmlab.chooser MCPLogLevel debug

Then run this in Terminal while you drive the app:

/usr/bin/log stream --predicate 'subsystem == "ai.thisbrain.locallmlab.sdkcore" AND category BEGINSWITH "MCP"' --level debug

Use the full /usr/bin/log path: in zsh, the default macOS shell, a bare log runs a shell builtin and fails with "too many arguments".

The four categories are MCP.connection (negotiation, HTTP, auth challenges), MCP.manager (add-server outcomes, and any tool result whose structuredContent fails its own outputSchema), MCP.oauth (the whole flow, step by step), and MCP.stream (each SSE frame as it's classified). Access and refresh tokens, auth codes, and client secrets are never written.

For a report you can paste into an issue, use MCP Servers panel → Copy Diagnostics — a redacted trace of recent connect / negotiate / authorize activity. The MCPLogLevel debug setting above makes this trace detailed too.

Save As… on a connected server exports its full tool / resource / prompt list, with each tool's token cost, to a text file.

Scripting test runs

For a batch of prompts against your server — does the model pick the right tool? call it with valid arguments? recover from an error? — localai-cli is usually the better fit than API Lab: no HTTP layer, and each request names the exact {server, tool} pair to activate via mcp_tools, so a script can loop over tools and models without touching the app's UI between runs (the server still has to be added once in MCP Servers first).

import json, os, subprocess

CLI_PATH = os.environ.get("LOCALAI_CLI_PATH", "./localai-cli")
CONFIG_PATH = os.path.expanduser(
    "~/Library/Application Support/LocalLM Lab/localai-config.json"
)

request = {
    "system_prompt": "You are a concise assistant. Use the available tool to answer.",
    "user_input": "What documentation topics are available for nickclyde/duckduckgo-mcp-server?",
    "mcp_tools": [{"server": "https://mcp.deepwiki.com/mcp", "tool": "read_wiki_structure"}],
}

result = subprocess.run(
    [CLI_PATH, "--config", CONFIG_PATH, "--run"],
    input=json.dumps(request), capture_output=True, text=True,
)
response = json.loads(result.stdout)
print(response.get("error") or response.get("answer"))

Loop that over a list of prompts — and models, and {server, tool} pairs — for a batch pass; each call is independent, so a thread pool works fine too. Swap --run for --chat if you want to send an OpenAI-style messages array instead. Full CLI reference →

API Lab is the other option, useful when your harness is already written against an OpenAI SDK or you'd rather drive it from curl: it exposes the model — with your enabled MCP tools — over a local OpenAI-compatible /v1/chat/completions endpoint. The model is whatever's set in AI Models, the tools are what's enabled in the panel, and the assistant's text reply comes back — the individual tool calls are in the log stream output above, not the HTTP response.

To call a tool with exact arguments and no model in the loop at all, use the MCP Inspector — that's what it's for. A first-class "call this tool with this JSON" action inside LocalLM Lab, and an in-app protocol-log panel, are planned for a later release.

What this covers, and what it doesn't

LocalLM Lab exercises, against your server:

• version negotiation: server/discover for 2026-07-28, falling back to the initialize handshake down to 2025-03-26, with the MCP-Protocol-Version header sent only where the negotiated revision defines it. A server that only offers 2024-11-05, or uses the old two-endpoint HTTP+SSE transport, is refused with a protocol-mismatch error rather than a bare HTTP 404
• the 2026-07-28 request rules — the _meta envelope on every request, Mcp-Method / Mcp-Name, and x-mcp-header parameters mirrored into Mcp-Param-* headers (a tool with invalid declarations is left out, as the spec requires)
• structured tool results — structuredContent validated against your outputSchema, with a mismatch surfaced as a tool error
• resource_link results, isError, tool titles and icons, nextCursor pagination
• server-initiated elicitation — the client renders a typed, validated form and names your server as the one asking (what it looks like). On 2026-07-28 it arrives as a multi-round-trip input_required result, and the client retries with your requestState echoed back unchanged
• change notifications — subscriptions/listen: when your server reports a tool, prompt, or resource list change, the client re-lists, and the server's row shows Live
• the full OAuth story — DCR, CIMD, the RFC 9728 .well-known fallback, incremental-scope 403 step-up, and the stricter 2026-07-28 checks (protected-resource resource match, metadata issuer match, required iss, authorization-server migration)

The client side of all this is checked against the official MCP conformance suite: it passes the client scenarios 2026-07-28 requires, apart from confidential-client authentication, which a desktop app doesn't use.

It is not a substitute for the MCP Inspector: there's no in-app raw-request editor or JSON tree view yet (use log stream), and no hand-crafted tools/call — the Inspector is still the tool for calling a tool with exact arguments and no model. LocalLM Lab also doesn't act as a sampling provider, and connects over remote Streamable HTTP only, not stdio. What it adds that the Inspector can't: a real model — several of them — actually deciding to call your tools.

Next

• MCP Servers — every server tested so far, and how to connect each
• The elicitation UI — what the client renders when your server asks the user for input
• The MCP client SDK — this same client as a Swift package for your own app
• mcp-diagnostics.md — the client's logging, in full

Contact

Building an MCP server and want to compare notes? neuron@thisbrain.ai or our Discord.