Choosing tools per turn

How a chat app with many tools decides which ones the model sees on each message

For developers building a chat on the SDK with MCP servers or built-in tools. Terms: see the glossary.

The short version. Every tool you give a model is described to it, in text, on every message. With a handful of tools that is fine. With dozens it is slow, and a small model gets worse at choosing. In SDK 3.0 your tools can be available rather than attached: give a session a picker and it reads each message and attaches only the few tools it probably needs. The picker is a small question-answering job, so it can be a small model, including Apple’s on-device model, which needs no download.

Why pick at all

A model cannot see your tools except through their descriptions, and those descriptions are sent with every request. A chat with eleven built-in plug-ins (Clock, Weather, Calendar, Reminders, Contacts, Location, Documents, Files, Tables, Web, Images) plus a few MCP servers can send thousands of words of descriptions before the user has typed anything. That costs time, and it gives a model more ways to pick the wrong tool. A picker trades a small extra step for a shorter, more focused request. MCP in your app has the cost side in detail.

How the SDK does it

You give the session a picker with dynamicTools:. There is no default, because the choice affects privacy and speed. Each turn, the picker looks at the message and the tools on offer and returns a short list. Only that list goes to the chat model. A server is contacted only when one of its tools is actually used.

let dynamicTools = DynamicTools.matching(
        SystemToolMatcher(maxTools: 6),
    options: ToolMatchingOptions(maxTools: 6, additionThreshold: nil))

let session = try lab.makeSession(route: .chat, instructions: "You are a helpful assistant.",
                                  dynamicTools: dynamicTools, authorizer: authorizer)

The authorizer is the gate every tool call passes. Leave it out and the session refuses anything that writes or sends data (the tool authorizer shows how to ask the user instead).

Three pickers ship with the SDK, and you can write your own:

ToolMatchingOptions sets how many tools to attach (maxTools, default 5) and how confident the picker must be to include a source or tool (defaults 0.7 for a source and 0.3 for a tool). A picker can have its own, lower cap: SystemToolMatcher attaches at most four tools unless you pass it maxTools, which is why the sample sets six in both places.

A worked case: ChatAgain

ChatAgain (our reference chat app, coming soon to the Mac App Store) uses SystemToolMatcher, so the picker is Apple’s on-device model, with the chat itself on Private Cloud Compute. It offers the picker the 11 built-in plug-ins, attaches at most 6 tools per message, and turns off the mid-chat history check (additionThreshold: nil). We turned that check off because it kept a new topic’s tool out: asking “what time is it in Tokyo?” after a run of questions about files did not get the Clock tool. An extra tool on a follow-up costs a few words of description; a missing one costs the answer.

What we measured, with the usual caution (our runs, one machine, a small sample):

We have not measured how many tools the picker attaches per message in ChatAgain. For DecisionToolMatcher we have one figure, from our turns evaluation: with a small local decider, picking took about half a second a turn (MCP in your app).

Where it goes wrong

↑ Top

More: tool cost and the picker in MCP · decision models · the prompt cache · SDK overview