200 Tools All Stuffed into a Prompt? Dynamic Tool Retrieval Goes Live in the Gemini API
Dynamic tool retrieval is live in the Gemini API. Each tool sends only a one-line summary, and full parameters are loaded only when needed. Two modes, code examples, and common questions—all here.
Gemini API has launched dynamic tool retrieval in preview. If your Agent is still sending full schemas for 200 tools to the model on every turn, this feature is exactly what you need.
Dynamic tool retrieval solves the problem of tool-list bloat. As tools multiply, sending complete schemas to the model in full burns tokens and lowers tool-selection accuracy. Now an Agent can upload only compact tool names and descriptions—about 26 tokens per tool by default—and load full parameters only when the task requires them.
Two modes.
Built-in server-side retrieval: add {type: 'tool_search'} to the tools list, and set defer_loading: true for functions or remote MCP servers. Gemini keeps full parameter schemas out of the initial prompt, so the model sees only a one-line summary. When needed, the model performs search, loads the schema, and issues a function_call within the same API request. Note: the API does not retain tool declarations across turns; even if you pass previous_interaction_id, you must resend the full tools array with every request. When using @google/genai, remember to pass apiVersion: 'v1beta'.
The key loop in server-side mode (simplified illustration):
function runLocalTool(name: string, args: Record<string, unknown>) {
if (name === 'issue_refund') {
return { status: 'refunded', order_id: args.order_id, amount: args.amount };
}
return { error: `Unknown tool: ${name}` };
}
const tools = [
{ type: 'tool_search' },
{
type: 'function',
name: 'issue_refund',
description: 'Refund an order to the original payment method.',
short_description: 'Refund an order.',
defer_loading: true,
parameters: {
type: 'object',
properties: {
order_id: { type: 'string' },
amount: { type: 'number' }
},
required: ['order_id', 'amount']
}
}
];
while (true) {
const interaction = await ai.interactions.create({
model: 'gemini-4-neon',
previous_interaction_id: previousId,
input: nextInput,
tools
});
previousId = interaction.id;
const calls = (interaction.steps ?? []).filter(s => s.type === 'function_call');
if (calls.length === 0) break;
nextInput = calls.map(fc => ({
type: 'function_result',
name: fc.name,
call_id: fc.id,
result: [{ type: 'text', text: JSON.stringify(runLocalTool(fc.name, fc.arguments)) }]
}));
}
In the first response, steps will appear in order as tool_search_call, tool_search_result, function_call. You only need to execute your own function_call; the search part is handled by the server.
Custom client-side retrieval: declare a tool_search tool, set execution: 'client', and define the name and parameters yourself. All functions are fully managed externally and do not go into tools[]. After the model calls the search tool, you retrieve with any logic—for example, vector search, filtering permissions by user, or scoring in parallel with the Gemini Decisions API—and then place the matching full schemas into function_result.result.
A user ran into a pitfall in client mode: the first time, without a threshold, the system matched 'without the tilt' to a tool whose description contained '40 degree tilt'. It only worked correctly after adding a noul threshold of 0.5. Thresholds are not decoration.
Another user worried: when retrieval misses, will the model confidently grab a suboptimal tool? In server-side retrieval, does search ranking look only at descriptions, or also at schemas? The original article does not expand on these questions, so it is recommended to test them yourself. When there are fewer than 20 tools, static loading is still simpler; with more than 20, deferring the long tail can significantly reduce input tokens. Dynamic retrieval has no additional fee and uses standard token pricing.
For the full guide, error reference, and token calculation, see the official documentation. Original article: Original article
发布时间: 2026-11-21 02:41