Choosing tools per turn
How a chat app with many tools decides which ones the model sees on each message
For developers building a chat on the SDK with MCP servers or built-in tools. Terms: see the glossary.
The short version. Every tool you give a model is described to it, in text, on every message. With a handful of tools that is fine. With dozens it is slow, and a small model gets worse at choosing. In SDK 3.0 your tools can be available rather than attached: give a session a picker and it reads each message and attaches only the few tools it probably needs. The picker is a small question-answering job, so it can be a small model, including Apple’s on-device model, which needs no download.
Why pick at all
A model cannot see your tools except through their descriptions, and those descriptions are sent with every request. A chat with eleven built-in plug-ins (Clock, Weather, Calendar, Reminders, Contacts, Location, Documents, Files, Tables, Web, Images) plus a few MCP servers can send thousands of words of descriptions before the user has typed anything. That costs time, and it gives a model more ways to pick the wrong tool. A picker trades a small extra step for a shorter, more focused request. MCP in your app has the cost side in detail.
How the SDK does it
You give the session a picker with dynamicTools:. There is no default, because the choice affects privacy
and speed. Each turn, the picker looks at the message and the tools on offer and returns a short list. Only that list
goes to the chat model. A server is contacted only when one of its tools is actually used.
let dynamicTools = DynamicTools.matching(
SystemToolMatcher(maxTools: 6),
options: ToolMatchingOptions(maxTools: 6, additionThreshold: nil))
let session = try lab.makeSession(route: .chat, instructions: "You are a helpful assistant.",
dynamicTools: dynamicTools, authorizer: authorizer)
The authorizer is the gate every tool call passes. Leave it out and the session refuses anything that
writes or sends data (the tool authorizer shows how to ask the user instead).
Three pickers ship with the SDK, and you can write your own:
SystemToolMatcher: Apple’s on-device model. It reads a short summary of each plug-in or server and the name and first sentence of each tool’s description, and says which fit the message. Nothing to download, and the message stays on the Mac.DecisionToolMatcher: a decision model answers the same question. This suits a small model trained to give fixed answers, such as OpenJev on a model like Qwen3-4B. A hosted decider works too, but then the message and your tool descriptions go to that provider.KeywordToolMatcher: no model. It matches words in the message to server names, keywords and tool names. Fast and predictable, but it misses paraphrases.
ToolMatchingOptions sets how many tools to attach (maxTools, default 5) and how confident the
picker must be to include a source or tool (defaults 0.7 for a source and 0.3 for a tool). A picker can have its own,
lower cap: SystemToolMatcher attaches at most four tools unless you pass it maxTools, which is
why the sample sets six in both places.
A worked case: ChatAgain
ChatAgain (our reference chat app, coming soon to the Mac App Store) uses SystemToolMatcher, so the picker
is Apple’s on-device model, with the chat itself on Private Cloud Compute. It offers the picker the 11 built-in
plug-ins, attaches at most 6 tools per message, and turns off the mid-chat history check
(additionThreshold: nil). We turned that check off because it kept a new topic’s tool out: asking
“what time is it in Tokyo?” after a run of questions about files did not get the Clock tool. An extra tool on a
follow-up costs a few words of description; a missing one costs the answer.
What we measured, with the usual caution (our runs, one machine, a small sample):
- Speed: the pick took about 1.6 seconds at the median, over 42 messages. A whole message, tools included, averaged about 1.8 minutes in our full test run, so the pick is a small part of the wait.
- Accuracy: in a 255-message run, the picker chose the right plug-ins for 254. The one miss was “what did i finish in the last two weeks?”, which got a clarifying question instead of the Reminders tool. We judged that reasonable for an ambiguous message.
We have not measured how many tools the picker attaches per message in ChatAgain. For DecisionToolMatcher
we have one figure, from our turns evaluation: with a small local decider, picking took about half a second a turn
(MCP in your app).
Where it goes wrong
- A miss can cost the answer. The chat model sees only the tools it was given. The SDK tells it
what was left out (
DynamicTools.missingToolGuidanceandheldBackNote), so it can say what is missing instead of guessing. WithrequestsToolson (the default inChatSetup) the model can callrequestToolsto ask for more, and a reply that says a tool is missing makes the session look for one and run the turn again, once a turn. In our test runs the model never calledrequestToolsitself, so the retry, not the model, is the safety net. - Wording matters. The picker judges from a plug-in’s summary and each tool’s first sentence only. Put the most useful sentence first in every description, and re-test when you change one.
- Ambiguous messages can go to the wrong plug-in. That is the nature of guessing intent from one line.
- A permission can still stop a picked tool. The picker chooses what is offered, not what is allowed: Location can be picked and then refused without permission, and every call still passes the authorizer. See MCP security.
- Speed varies. In long runs with a locked screen we saw picks run two to three times slower (seen with a different on-device decider; we expect the same for any on-device model).
More: tool cost and the picker in MCP · decision models · the prompt cache · SDK overview