Gemini API Tool Retrieval: How an Agent Loads a Schema on Demand

Follow a refund from schema search to function call, choose server-side or client-side retrieval, and see when deferring Gemini tools costs more.

Published

Gemini API Tool Retrieval: How an Agent Loads a Schema on Demand

A support agent looking up an order can carry the full refund schema through every turn, even if nobody asks for a refund. Gemini API Tool Retrieval lets the model see that a refund tool exists and load its parameters when a customer actually needs it. For a developer with a large catalog, the decision turns on how much schema the agent can leave unread and how often it has to search.

Google launched Tool Retrieval in preview on November 19, 2026. As of November 2026, it requires v1beta on the Interactions API. Interactions itself is generally available. An agent using generateContent needs to move its tool loop to Interactions to use retrieval. API changelog · Interactions API overview

Google’s announcement names Gemini 3.6 and later, including all Gemini 4 models. The retrieval guide explicitly lists gemini-3.6-flash, gemini-3.8-flash, gemini-4-argon and gemini-4-hydrogen. Hydrogen is the model in Google’s examples below. Launch announcement · Supported models

With default, server-side retrieval, your application still sends the complete declarations. What changes is the input Gemini sees. Each deferred function initially occupies a line with its name and short_description, or the first sentence of its description. The API holds the parameter types, properties and required fields until the model asks for the function by name. This distinction matters if you log HTTP payload sizes to estimate model costs. Tool retrieval guide

A complete declaration can travel in the request and still stay out of the initial promptWhere the refund schema lives during server-side retrieval
From declaration to call
Your applicationDeclare issue_refundSend the full parameters with defer_loading enabled.
Initial model contextOffer the refund by nameThe description says what the action does.
Gemini APILoad the requested schemaReturn the parameter definitions for order_id and amount.
Model responseRequest the refundReturn the order and amount as function arguments.

As of Nov 20, 2026 · Source: Google, Gemini API Tool Retrieval

A refund in one server-side request

Here is Google’s Python example. The plain tool_search entry enables server-side lookup. The flag belongs to the function. Keep its complete parameters object here so that Gemini has something to retrieve. Google’s refund example

Python
from google import genai

client = genai.Client()

tools = [
   {"type": "tool_search"},
   {
       "type": "function",
       "name": "issue_refund",
       "description": "Refund an order to the original payment method.",
       "defer_loading": True,
       "parameters": {
           "type": "object",
           "properties": {
               "order_id": {"type": "string"},
               "amount": {"type": "number"},
           },
           "required": ["order_id", "amount"],
       },
   },
   # ... 50+ other deferred functions (leave frequent tools undeferred)
]

interaction = client.interactions.create(
   model="gemini-4-hydrogen",
   input="Order A-1042 arrived damaged. Please refund 40 dollars.",
   tools=tools,
)
How Google’s refund example appears in interaction.steps
You · Input

Order A-1042 arrived damaged. Please refund 40 dollars.

Agent · tool_search_call

function_names = ["issue_refund"]

tool_search_result

issue_refund requires order_id (string) and amount (number).

Agent · function_call

issue_refund · {"order_id": "A-1042", "amount": 40}

The first two steps record work the API has already done. The last gives your application work to do. At that point the customer has requested a refund and Gemini has supplied arguments. Your existing handler validates the request, runs the refund and sends back a function_result with the call’s ID. Inspect interaction.steps on every response. A function_call can be present even when the status differs from requires_action. Google’s response format · Function calling explained

Which schemas should stay loaded?

Google’s rule is “keep your most-used tools loaded up front and defer the long tail.” For a support agent that looks up an order on most turns and issues refunds occasionally, keep the lookup loaded and defer the refund. If refunds are the agent’s primary job, the same refund schema belongs in its initial context. Google’s launch guidance

There is an exception worth checking in your prompts. If a system instruction or another tool’s output mentions a function by name, leave that function loaded. The model can otherwise emit a call directly, skip the search that supplies its schema, and leave the API unable to continue when you return the result. For tools you do defer, make the first line distinguish similar operations. Google’s examples billing_get_customer and crm_get_customer give the model a service to choose before it sees either parameter list. Google’s best practices

When your app owns the registry

Consider an agent whose user connects a billing integration halfway through a conversation. Your registry knows which billing tools that user may access. With client-side retrieval, you initially declare a search function with a name, description and query schema. You can make a newly available function discoverable without having declared it in the original request. Each search costs one extra application-to-API round trip. Server-side and client-side search are mutually exclusive within a request. Retrieval modes

Google’s client-side excerpt starts with the search declaration and resumes inside the handler for a search call. Here interaction is the preceding interaction, step is its ordinary function_call for search_tools, and my_registry is the lookup your application supplies. Google’s client-side example

Python
tools = [{
    "type": "tool_search",
    "execution": "client",
    "name": "search_tools",
    "description": "Search the internal tool registry for functions.",
    "parameters": {
        "type": "object",
        "properties": {"query": {"type": "string"}},
        "required": ["query"],
    },
}]

# When the model emits a function_call for "search_tools",
# return matching {"type": "function", ...} schemas in function_result:
followup = client.interactions.create(
    model="gemini-4-hydrogen",
    previous_interaction_id=interaction.id,
    tools=tools,
    input=[{
        "type": "function_result",
        "name": "search_tools",
        "call_id": step.id,
        "result": my_registry.search(**step.arguments),
    }],
)

Return complete function declarations as objects in function_result.result. The call_id ties those matches to the search that requested them. The next function_call can name a discovered billing function, which your normal handler executes. A registry that finds nothing can return a short text result naming the categories it can search so the model can revise its query. Client-side retrieval

What changes for remote MCP servers?

A remote Model Context Protocol (MCP) server already supplies tool definitions and runs its tools. Add defer_loading: true to the mcp_server declaration alongside the server-side tool_search entry. The flag covers every tool the server exposes. You can declare your own functions alongside it. Google’s MCP configuration · What is MCP?

Google’s weather and currency example shows why the execution boundary matters. A single search requests both weather_service:get_weather and convert_currency. For London weather the API contacts the server and returns an mcp_server_tool_call followed by an mcp_server_tool_result. For the currency conversion, a local function, the response ends with a function_call for your app. Dispatching every step as local work would repeat work the remote server already completed. Google’s mixed-tool response

What it costs, and when it costs more

Tool Retrieval has no separate fee. The model’s standard input, cached-input and output rates apply. Enabling server-side search adds a small fixed instruction block. Each deferred tool still contributes its catalog line. Once retrieved, its schema stays in context and is billed on subsequent calls for the rest of that conversation. Google’s pricing explanation · Gemini API rates by model

A server-side search saves a network round trip for your app, but the API makes another model call with the accumulated context and newly loaded schemas. In usage, the initial prompt is counted in total_input_tokens and the subsequent pass in total_tool_use_tokens, billed at input rates. Check total_cached_tokens for the portion served at the cached rate. Retrieved schemas are appended where they were loaded, preserving the earlier prefix for reuse. A smaller initial prompt can also fall below the model’s cache minimum. Google’s billing guide

Google illustrates the trade-off over a call and its follow-up, with two schemas retrieved and 400 tokens per schema. These are worked examples that exclude output and caching. Google’s token arithmetic

Catalog and conversationStatic inputRetrieval input including search passes
200 tools, 2,000 tokens of conversation164,000About 23,650
10 tools, 30,000 tokens of conversation68,000About 92,830

In the smaller catalog, a search makes the model read the long conversation again to recover a small amount of schema. For a short, stable list, Google suggests static loading for fewer than roughly 20 tools. For a large, fixed list where most tasks touch only a few tools, try server-side retrieval. Choose client-side search when a live registry, namespace ranking or per-user filtering justifies the extra request. Google’s mode comparison

How the search differs across labs

APIWhat the model can see before discoveryHow the hosted search supplies a tool
Gemini InteractionsA name and brief description for each deferred toolGemini requests names. The API returns their schemas in a tool_search_result.
Anthropic MessagesThe search tool and any tools you leave loadedClaude can search the catalog using regex or BM25. The API expands matching tool_reference blocks into definitions.
OpenAI ResponsesA deferred standalone function’s name and description, or the high-level description and name of a searchable namespace or MCP serverThe response includes tool_search_output with the loaded tools. OpenAI recommends grouping related functions so the model can discover the group from its description.

If you port a tool catalog, review the text the model sees before search. A service description that makes a namespace easy to find in Responses may still leave similarly named Gemini functions hard to distinguish. Google’s retrieval guide · Anthropic’s tool search guide · OpenAI’s tool search guide

Migration checklist for an existing agent

  1. Record a baseline. Pick representative conversations, including one long conversation and one that needs a rare function. Record billed usage, latency and whether the agent called the right function with usable arguments.
  2. Choose the retrieval boundary. Use Interactions with a supported model and the preview API version. Start with server-side lookup for a known catalog. Choose client-side lookup when the app must decide which tools a user can discover.
  3. Audit names before setting flags. Keep frequent tools and functions named in instructions or tool outputs loaded. Give each deferred function a distinct first-line description and its full schema.
  4. Check the follow-up request. Resend the complete declared tools array even with previous_interaction_id, including when returning a function result. That ID preserves history. Tool declarations apply to each interaction separately.
  5. Preserve stateless history. With store=false, replay the response steps in order, including the original retrieval calls, results and opaque signatures. A dropped search result can lose the loaded schema. A missing call or signature can surface as a 400 on the follow-up.
  6. Compare the whole conversation. Count search passes and cached input alongside initial input. If the agent searches for the same frequently used capability across most conversations, try keeping that schema loaded and compare the totals again.

Google’s multi-turn and stateless examples

Published
Category
Build
Related Articles
Genie 4 API: How One Live World Session Works, and What It Costs

Genie 4 API: How One Live World Session Works, and What It Costs

Genie 4 opened on the Gemini API on Nov 19, 2026. How one live world session runs, in Google's own code, what a minute costs and where it stops.Nov 19, 2026Build
MAI Code 2 in GitHub Copilot

MAI Code 2 in GitHub Copilot

Choose MAI Code 2 or Flash by the scope of a fix. Check which Copilot plans include each model, where to find the picker and how organization policy works.Nov 19, 2026Build
GitLab Alternatives in 2026: Gitea, GitHub, Forgejo, Bitbucket, Azure DevOps, Codeberg and OneDev

GitLab Alternatives in 2026: Gitea, GitHub, Forgejo, Bitbucket, Azure DevOps, Codeberg and OneDev

Compare seven GitLab alternatives by hosting, CI capacity, registry, security and seat price. See what each importer moves and which pipelines need rewriting.Nov 19, 2026Build
Social Media API in 2026: Zernio, Ayrshare, Phyllo and the Native APIs

Social Media API in 2026: Zernio, Ayrshare, Phyllo and the Native APIs

Compare Zernio, Ayrshare, Phyllo and four native APIs for publishing and creator data. See how customers connect and which features need approval.Nov 18, 2026Build
AGENTS.md vs CLAUDE.md

AGENTS.md vs CLAUDE.md

Which file should your coding agents read? See how Claude Code, Codex, Cursor and Copilot load instructions and how one import keeps team rules in sync.Nov 18, 2026Build
Thunkable Review

Thunkable Review

Can one Thunkable app serve your business on iOS and Android? Compare its blocks, data connections, publishing plans and limits on team editing.Nov 18, 2026Build
LLM Wiki: Build a Knowledge Base Your AI Maintains

LLM Wiki: Build a Knowledge Base Your AI Maintains

Set up an LLM wiki in a folder of Markdown files. Follow one team decision from source memo to cited answer, then update it when the decision changes.Nov 18, 2026Build
Codex Alternatives in 2026: Claude Code, Cursor, GitHub Copilot and 5 More

Codex Alternatives in 2026: Claude Code, Cursor, GitHub Copilot and 5 More

Compare eight Codex alternatives by model, editor, billing and team controls. See when to switch agents and when to keep Codex.Nov 18, 2026Build
Newsletter

One letter, every Sunday.Working systems, not hot takes.

Weekly. No spam. Unsubscribe anytime.