What Is Function Calling in 2026? How OpenAI, Gemini, and Claude Invoke APIs via JSON

Function calling (tool calling) is the default Agent protocol in 2026: models no longer pretend to hit APIs in prose. They emit JSON under an agreed schema, and your code actually fetches weather, queries a database, edits files, or runs tests. The difference is not who chats better—it is whether schema is validated, execution is audited, and side effects are isolated.

Last updated 18 August 2026. Field names drift across SDK versions; the loop does not: request → execute → feed back. If a tool can mutate disk, pair this with an Agent File System sandbox. If you need preferences across sessions, store them in a dedicated Memory tier, not inside the next arguments blob.

Why free-text prompts are not enough (Why)

The old trick was “please call get_weather(city=Beijing)” plus a regex. Demos pass; production hits three walls:

1. Unstable parsing—polite padding, reordered args, missing quotes; JSON.parse dies.
2. No types—enums become free text; required fields vanish; parallel calls cannot be correlated.
3. Side effects mixed with chat—the model claims a file was deleted when you never ran anything, or you ran it and the model never saw the result.

Function calling keeps choice in the model and execution in the runtime. HTTP, SQL, and shell never happen inside the model process. That is the same idea as CI: one job, one workspace—execution boundary before cleverness.

Three layers of function calling (What)

Wiring “OpenAI tools” is not an architecture. Split at least three layers:

Asymmetric takeaway: vendor APIs standardize the L1 JSON envelope. They do not finish L2 authorization or L3 process isolation. MCP is discovery and sessioning—not automatic safety.

The loop looks like this. The model never holds your API keys; secrets live only in the runtime.

┌──────────────┐     JSON tool request      ┌──────────────────┐
│  LLM API     │  ────────────────────────► │  Your runtime    │
│  (OpenAI /   │     name + arguments       │  (Agent loop)    │
│   Gemini /   │                            │                  │
│   Claude)    │  ◄──────────────────────── │  execute tool    │
└──────────────┘     tool result JSON       │  • HTTP APIs     │
                                            │  • DB / search   │
                                            │  • shell / MCP   │
                                            └────────┬─────────┘
                                                     │
                                                     ▼
                                            ┌──────────────────┐
                                            │ Policy + secrets │
                                            │ allowlist, audit │
                                            │ Cloud Mac / VPC  │
                                            └──────────────────┘

Core compare: OpenAI vs Gemini vs Claude

The divider in 2026 is not “does it support function calling” (all three do). It is string vs object arguments, how parallel calls line up, and how failures are written back. Gateways usually normalize to an internal ToolCall{name, id, args} and dispatch to one L2 runtime.

How the JSON looks (How)

Same get_weather tool. In production, descriptions should state boundaries (“public weather only, never personal location”) so the model does not stuff sensitive fields into parameters.

OpenAI: tools + tool_calls

Attach tools on the request. If the model calls, the assistant message includes tool_calls. arguments is a stringified JSON object—parse, then validate.

{
  "model": "gpt-4.1",
  "messages": [{"role": "user", "content": "What's the weather in Beijing?"}],
  "tools": [{
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "Current weather for a city",
      "parameters": {
        "type": "object",
        "properties": {
          "city": {"type": "string", "description": "City name"},
          "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
        },
        "required": ["city"]
      }
    }
  }]
}

Simplified model output:

{
  "role": "assistant",
  "tool_calls": [{
    "id": "call_abc123",
    "type": "function",
    "function": {
      "name": "get_weather",
      "arguments": "{\"city\":\"Beijing\",\"unit\":\"celsius\"}"
    }
  }]
}

After execution, send a role: tool message (or the Responses API equivalent) with matching tool_call_id and string content. The model then answers in prose or issues another call.

Gemini: functionDeclarations + toolConfig

Declarations live under tools[].functionDeclarations. Type names are often uppercase. toolConfig.functionCallingConfig.mode can be AUTO, ANY (must call a tool), or NONE (must not). That is a product switch, not a sandbox.

{
  "contents": [{"role": "user", "parts": [{"text": "What's the weather in Beijing?"}]}],
  "tools": [{
    "functionDeclarations": [{
      "name": "get_weather",
      "description": "Current weather for a city",
      "parameters": {
        "type": "OBJECT",
        "properties": {
          "city": {"type": "STRING"},
          "unit": {"type": "STRING", "enum": ["celsius", "fahrenheit"]}
        },
        "required": ["city"]
      }
    }]
  }],
  "toolConfig": {"functionCallingConfig": {"mode": "AUTO"}}
}

The model typically returns a functionCall part (name + args object). You reply with a functionResponse part. Do not assume field names match OpenAI; Chat-compatible vs native Gemini SDKs differ.

Claude: tools + tool_use / tool_result

Messages API tools use input_schema. The assistant reply is an array of content blocks: optional text, then tool_use. input is already an object (one less parse) but you still validate—models omit fields.

{
  "model": "claude-sonnet-4-6",
  "max_tokens": 1024,
  "tools": [{
    "name": "get_weather",
    "description": "Current weather for a city",
    "input_schema": {
      "type": "object",
      "properties": {
        "city": {"type": "string"},
        "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
      },
      "required": ["city"]
    }
  }],
  "messages": [{"role": "user", "content": "What's the weather in Beijing?"}]
}

{
  "role": "assistant",
  "content": [{
    "type": "tool_use",
    "id": "toolu_01XYZ",
    "name": "get_weather",
    "input": {"city": "Beijing", "unit": "celsius"}
  }]
}

Write results back as role: user tool_result with a matching tool_use_id:

{
  "role": "user",
  "content": [{
    "type": "tool_result",
    "tool_use_id": "toolu_01XYZ",
    "content": "{\"temp_c\": 28, \"condition\": \"clear\"}"
  }]
}

Claude Code and Cursor add MCP on top: the model still sees a catalog of JSON-schema tools; processes hot-plug over an MCP session. Transport changes; the loop does not.

How to choose (decision matrix)

Recommended stacks

Stack A — fastest solo (half a day)

• One model (OpenAI or Claude); register 1–3 read-only tools (search, weather, docs).
• Validate with a JSON Schema library; reject extra properties; secrets in env, never in the prompt.
• Log tool, args_hash, latency, ok—never raw keys or PII.

Stack B — small-team production

• Normalize inbound to ToolCall; adapters out to OpenAI / Gemini / Claude.
• L2 policy: per-tool timeout, concurrency, daily caps; dangerous tools confirm or run only on isolated nodes.
• Read-only APIs on functions; repo writes and tests on Cloud Mac sessions with the same destroy discipline as remote Mac automation.
• Keep Memory out of tool arguments.

Stack C — enterprise

• Tool catalog via review; schema changes as PRs.
• MCP or custom servers in a locked VPC; audit JSONL leaves the workspace.
• Never ship unconstrained bash without path policy.

Common pitfalls

1. Believing the model authenticates—keys live in the runtime; the model sees schema and prior results.
2. Treating OpenAI arguments as an object—parse first; treat parse failure as a model error.
3. Descriptions that say “do anything”—vague copy makes the model over-call; state when not to call.
4. Ignoring parallel tool_calls—shared mutable state races; read-only can fan out, writes serialize.
5. Swallowing failures—return structured errors so the model can retry or switch tools.
6. MCP equals security—MCP is transport; a home-directory root is still full disk.

Rollout (7 steps)

1. Inventory tools—name, side-effect class (read / write / irreversible), timeout, data residency.
2. Author JSON Schema—required fields, enums, no additionalProperties; descriptions with boundaries.
3. Minimal loop—user message → optional calls → execute → fill → final answer; one tool first.
4. Validate and isolate secrets—reject parse errors, schema misses, unknown names; audit them.
5. Second vendor adapter—prove internal ToolCall maps to Claude blocks and Gemini parts.
6. Sandbox write tools—path allowlists or remote workspaces; run deny tests from the filesystem guide.
7. Observe 7 days—mis-call rate, round-trip, parallel conflicts; tighten schemas that misfire.

FAQ

How is function calling different from JSON mode?

JSON mode / structured output constrains the final reply shape (extract an order). Function calling constrains intermediate tool requests and usually spans multiple turns after execution. You can combine them; they solve different jobs.

Can the model query my database directly?

No—unless your runtime exposes arbitrary SQL as a tool without authorization. In the default loop the model only proposes schema-valid parameters.

Can three vendors share one schema?

Yes for the core JSON Schema, then wrap envelopes (OpenAI parameters, Gemini uppercase types, Claude input_schema). Regression-test enums and required fields; strict-mode behavior is not identical.

Are Claude Code tools function calling?

Same loop: structured calls, host executes (including MCP). The catalog is supplied dynamically by the IDE/MCP instead of a handwritten tools array on every HTTP request.

Should tool results go into long-term memory?

Raw API payloads are large and ephemeral. Summarize into Memory; do not treat full tool_result as durable state. See the help center and the Memory guide.

Conclusion

Treat 2026 function calling as schema-typed RPC. OpenAI, Gemini, and Claude are JSON dialects of “model proposes, runtime executes, result returns.” Choose by gateway and side-effect class; secure with L2 validation and isolated execution hosts—not marketing names.

Before you ship, ask: if the model emits write_file right now, which machine and directory receive the bytes? If you cannot answer, do not attach write tools yet. Read-only APIs can ship today; code edits and tests belong on a disposable Cloud Mac node—check pricing first.

Further reading

• Agent File System & sandboxes →
• Self-hosted vs SaaS Agent Memory →
• OpenClaw cloud automation →