Function Calling Explained: How LLMs Use Your Tools

How Function Calling Works

The model produces a request. Your application performs the operation. For a developer wiring an LLM into an API or database, that separation is the most useful fact to understand before writing code.

Imagine a restaurant order ticket. A waiter can request a dish, but the ticket does not cook anything. The kitchen checks the order, prepares what it can and reports back. In your application, a tool call is the ticket, and your function is the kitchen.

You give the model a tool schema, a description of an available operation and the input fields it accepts. The model can respond with the function's name and arguments, the values to put into those fields. Your application checks the request, invokes the function, then returns its output so the model can answer or request another operation. Google's function calling guide explicitly assigns execution to your application.

A tool can read as well as write. Looking up an order, searching a document collection and checking availability are tool operations even when they change nothing. For a first integration, a narrow lookup is a useful starting point: you can inspect the entire path without introducing a transaction that needs undoing.

Function Calling vs Tool Calling

For custom functions, the names often describe the same mechanism. OpenAI uses both names, and Anthropic describes tool use as also being called function calling. The broader word “tool” can include provider-hosted capabilities as well as functions you implement. Execution location matters more than the label: for your own database function, your application is responsible for running it.

The distinction becomes concrete in the three makers' documentation below. Their message formats differ, so keep each provider's request and result envelopes together.

A Minimal End-to-End Function Calling Example

Expose an order-status lookup, then return only the status the database supplied. The following example uses a deliberately invented order, ORD-1042, in a local SQLite database. It is an educational fixture, not a customer record or an observed model response.

OpenAI Function Calling: Run the Round Trip

OpenAI's function calling documentation shows the Responses API round trip: declare the function, receive a function_call, execute your code and return a function_call_output carrying the same call_id.

Install the Python SDK with python -m pip install openai, set OPENAI_API_KEY in your environment, and save this as order_tool.py. The example uses gpt-6-astra, the model in the current documentation's Responses example; it is not a claim that this is the only suitable model. Running the script makes live API requests, while the database itself stays in memory.

import json
import re
import sqlite3

from openai import OpenAI

client = OpenAI()
model = "gpt-6-astra"

# Demo identity. In a web app, obtain this from your authenticated session.
tenant_id = "tenant_demo"
db = sqlite3.connect(":memory:")
db.execute("CREATE TABLE orders (tenant_id TEXT, order_id TEXT, status TEXT)")
db.execute(
    "INSERT INTO orders VALUES (?, ?, ?)",
    (tenant_id, "ORD-1042", "shipped"),
)

# The tool schema: a name, a purpose, and an input contract.
tool = {
    "type": "function",
    "name": "get_order_status",
    "description": (
        "Look up the current status of an existing order by order ID. "
        "Use for status questions, not refunds or address changes. "
        "Returns the order ID and status, or an error."
    ),
    "strict": True,
    "parameters": {
        "type": "object",
        "properties": {
            "order_id": {
                "type": "string",
                "description": "The order ID supplied by the user, such as ORD-1042.",
            }
        },
        "required": ["order_id"],
        "additionalProperties": False,
    },
}


def execute(name, raw_arguments):
    if name != "get_order_status":
        return {"error": "unknown_tool"}
    try:
        args = json.loads(raw_arguments)
    except (json.JSONDecodeError, TypeError):
        return {"error": "invalid_arguments"}
    if not isinstance(args, dict) or set(args) != {"order_id"}:
        return {"error": "invalid_arguments"}
    order_id = args["order_id"]
    if not isinstance(order_id, str) or not re.fullmatch(r"ORD-[0-9]+", order_id):
        return {"error": "invalid_order_id"}
    row = db.execute(
        "SELECT status FROM orders WHERE tenant_id = ? AND order_id = ?",
        (tenant_id, order_id),
    ).fetchone()
    if row is None:
        return {"error": "order_not_found"}
    return {"order_id": order_id, "status": row[0]}


history = [{"role": "user", "content": "What is the status of order ORD-1042?"}]
response = client.responses.create(
    model=model,
    input=history,
    tools=[tool],
    tool_choice={"type": "function", "name": "get_order_status"},
    parallel_tool_calls=False,
)

# Keep every output item, including any reasoning items.
history.extend(response.output)
calls = [item for item in response.output if item.type == "function_call"]
if len(calls) != 1:
    raise RuntimeError("Expected one tool call; inspect the response before continuing.")
call = calls[0]
result = execute(call.name, call.arguments)
history.append({
    "type": "function_call_output",
    "call_id": call.call_id,
    "output": json.dumps(result),
})

answer = client.responses.create(
    model=model,
    input=history,
    tools=[tool],
    tool_choice="none",
    instructions="Answer from the tool result. If it contains an error, report the failure.",
)
print(answer.output_text)
db.close()

This deliberately bounded example makes 2 model requests and 1 database lookup on its successful path. It forces the known lookup on the first request and disables further tool calls on the second. That makes the mechanics visible; a general assistant would also need to handle requests where no tool is appropriate, information is missing or another lookup is needed.

The tenant_id is absent from the tool schema because the application already knows who is signed in. The database query uses that trusted identity alongside the requested order ID. Moving identity into a model-supplied argument would remove the boundary the example is trying to teach.

The Model's Call and the Result You Return

The request and the result are separate messages joined by a call identifier. A representative call would look like this. Identifiers and values are illustrative; this is not a captured API response.

{
  "type": "function_call",
  "id": "fc_demo",
  "call_id": "call_demo",
  "name": "get_order_status",
  "arguments": "{\"order_id\":\"ORD-1042\"}"
}

After the handler returns the fixture's status, the application appends this result. In a live response, copy the actual call.call_id; never manufacture a new one.

{
  "type": "function_call_output",
  "call_id": "call_demo",
  "output": "{\"order_id\":\"ORD-1042\",\"status\":\"shipped\"}"
}

The model now has evidence for an answer such as “Order ORD-1042 has shipped.” The status came from the database. The model's contribution was translating the request into arguments and translating the returned data into an answer.

The Same Idea in the Three Makers' Docs

The execution pattern travels across providers; the wire format does not. Verified against the makers' live documentation on 17 October 2026, the mapping below distinguishes the current examples rather than treating similarly named fields as interchangeable.

Anthropic Tool Use

Anthropic's tool use documentation defines your function's inputs with input_schema. For a client tool, your code handles the assistant's tool_use block and sends a tool_result inside the next user message, preserving the assistant message in the conversation.

The useful boundary is explicit in those docs: client tools run in your application, while server tools run on Anthropic's infrastructure. A hosted search tool therefore does not demonstrate that Claude can directly execute your private database function. You still supply that implementation.

Gemini Function Calling

Google's Gemini function calling documentation also separates the model's request from application execution. Its current guide uses the Interactions API, with function_call and function_result steps and previous_interaction_id to continue the interaction.

Follow the example for the API surface you are using. Copying a declaration from one API and a result message from another is not portability. Preserve the common operation, such as get_order_status, and adapt its declaration and result envelope together.

Why Tool Calls Are the Working Loop Inside an Agent

A tool-using agent repeatedly chooses work, receives its result and decides what comes next. Function calling supplies the handoff between that decision and an operation your software can perform.

For example, an order assistant could look up a shipment, inspect the returned carrier reference, then request tracking information before answering. The second operation depends on the first result. That feedback loop is more useful than asking the model to invent every step in advance.

Anthropic's agent architecture guidance distinguishes fixed workflows from agents that direct their own tool usage. It also describes tool feedback and stopping conditions as parts of the loop.

Calling this “the base of every agent” is useful shorthand only up to a point. The docs do not establish that every system called an agent uses a particular function-calling API. The durable idea is the model-to-action loop. A single lookup can stay a fixed workflow; it does not need an autonomous planner.

Three Things That Break in Production

Treat schema quality, tool selection and argument validation as separate problems. A fix for one does not prove the others are solved.

Bad Schemas: The Contract Leaves Room for the Wrong Meaning

A field named id does not tell the model whether you want an order number, customer identifier or shipment reference. An unconstrained string called action invites values your handler may not recognize. Improve the contract before adding more instructions around it.

Name parameters precisely, describe their meaning and use an enum when only a fixed set of values is accepted. Keep the first tool narrow enough that its purpose fits in a plain sentence. In the example, order_id is explicit and the description excludes refunds and address changes.

OpenAI's strict mode requires additionalProperties: false on objects and all properties to appear in required; a value that may be absent can allow null. Its guide documents these strict-mode requirements. Google also documents schema restrictions, so a schema accepted by your local validator is not automatically accepted by every model API.

The wall: a schema can constrain representation, but a valid order ID can still be the wrong order. That is an application check.

The Model Calls the Wrong Tool: Valid Arguments, Wrong Operation

A customer asking “Where is my refund?” needs a status lookup, not a new refund. If both tools say “handles refunds,” you have made the choice ambiguous.

Describe when each tool should and should not be used, and expose only operations appropriate to the current task and user. Anthropic's tool-definition guidance specifically recommends explaining purpose, parameter meaning, usage boundaries and limitations.

When your application already knows which operation must happen, use a supported tool-choice control or call that operation directly. The example forces a known lookup for teaching; do not turn that into “every user message must trigger a tool.” A greeting or an incomplete order question may need a conversational response instead.

The wall: well-formed arguments cannot tell you whether the model chose the right business action. Evaluate the chosen tool against representative user requests, including cases where no call should happen.

Unvalidated Arguments: A Correct Shape Crosses the Wrong Boundary

The handler is an input boundary. Check the function against an explicit allowlist, parse its arguments, validate their shape and meaning, and enforce access using your application's authenticated identity.

For the order example, a useful failure probe is an order belonging to another tenant, meaning another customer account. The query must not return it even if its identifier is a perfectly valid string. Other probes include a missing order ID, an unexpected field and a request for an operation the handler does not implement.

Anthropic's overview notes that missing information can lead a model to infer argument values. Do not assume a required field proves the user supplied it. For a consequential action, compare the requested operation and its inputs with what the user actually authorized.

Return a recognizable failure when execution fails. A timeout is not evidence that an operation never happened: before retrying a write, use an application-level idempotency mechanism, a way to recognize the same intended operation and avoid repeating its effect. These are implementation recommendations, not guarantees supplied by the model API.

When to Use Structured Outputs Instead

Use structured outputs when you need a shaped answer from information already available. Use a tool when you need an external operation. A support-message classifier that returns a category and urgency can produce structured output. An assistant that needs the current order status must obtain that state through a lookup.

OpenAI's Structured Outputs guide makes this distinction between connecting application functionality and formatting the model's response. It also explains that strict function schemas use Structured Outputs underneath. These are complementary capabilities, not mutually exclusive products.

Suppose you already supplied an order record and only need {order_id, status, explanation} for your interface. A structured answer is sufficient. If the record is missing or must be refreshed, call the lookup first. You can then structure the final answer as well.

A JSON object saying that an email was sent is still only an output object. Sending the email requires an executed operation and a result you can check.

Function Calling vs MCP

Function calling handles the model's request for work. Model Context Protocol (MCP) provides a shared interface for exposing tools and other context to compatible applications. You can wire a function directly into one app before introducing a separate tool server.

MCP vs Tool Calling: When Sharing Becomes Useful

If several applications need the same order lookup, an MCP server can expose the tool for discovery and invocation. The MCP architecture documentation describes tools/list for discovery and tools/call for execution requests. The server still needs an implementation and access checks.

Read what MCP is and how its host, client and server fit together for that next step. Once your direct lookup works, the MCP server implementation guide shows how to package capabilities for reuse. Introducing MCP does not remove the need to validate the operation being requested.

What Is Overhyped

A successful schema check proves a format constraint, not a completed task. The operation may be inappropriate, the record inaccessible or the underlying service unavailable. A fluent final sentence does not settle any of those questions.

Nor does a tool list create an agent by itself. You still need rules for which tools are available, how results enter the conversation, when to stop and what to do after an error. A framework can package this work, but the execution policy remains a design decision.

The first useful milestone is modest: a request reaches the intended handler, only permitted data comes back, and the answer reflects that result. Make that path visible before expanding the system's authority.

Your First Shipping Decision

Start with a read-only operation that answers a recurring user question. The order-status example is a useful pattern because its expected result and access boundary are easy to inspect.

For builders, act now if the answer depends on data your app can retrieve reliably. Keep a fixed workflow when the next operation is already known. Wait to introduce model-selected writes until you have explicit permissions, failure handling and a way to prevent repeated effects.

For operators, inspect what ran and what came back. Log the selected tool, validation outcome, execution status and call correlation, while keeping sensitive arguments out of ordinary logs. Set timeouts and a maximum number of tool rounds appropriate to the job; do not let a conversational loop continue without a boundary.

For buyers, ask the implementation team to show a denied lookup and a failed operation as well as a successful answer. If the product only summarizes information already supplied, function calling may be unnecessary. Structured output or ordinary text generation can be enough.

Your Monday move: implement one narrow lookup, try a permitted record, an inaccessible record and an incomplete request, then inspect the result returned to the model in each case. Expand the tool list only after you can explain why each request was accepted or rejected.

Frequently Asked Questions

For practical guides to building with AI, subscribe to the newsletter.