Beyond Tool Calling: Why Code Mode is the Final Boss of LLM Execution

The Agentic Shift: From Chatting to Orchestrating

For the past two years, we’ve been obsessed with "chat." We’ve treated Large Language Models (LLMs) like super-powered interns who can write emails, summarize PDFs, and occasionally hallucinate a plausible-sounding legal brief. We marveled at the ability of a machine to hold a conversation, to mimic human empathy, and to explain quantum physics to a five-year-old. But the industry is quietly, rapidly moving past the chat interface. We are entering the era of the Agent, where the goal isn't just to talk about doing work, but to actually do it.

In this new paradigm, the LLM isn't just a text generator; it’s an orchestrator. It’s the brain sitting at the center of a complex web of APIs, databases, and infrastructure. It is the decision-making engine that decides which lever to pull, which query to run, and which alert to escalate. Until recently, our primary way of letting these brains interact with the world was through Tool Calling (or Function Calling). You give the model a JSON schema of your API, and it spits back a JSON object telling you which function to run and with what arguments.

It worked. It got us from GPT-3.5 to the first generation of autonomous agents. It allowed us to build the first "connected" AI applications. But as we try to move from toys to tools—as we try to build systems that handle thousands of endpoints, manage global cloud infrastructure, or perform multi-step data processing across disparate silos—we’re hitting a massive, expensive, and conceptually rigid wall.

Enter Code Mode.

Code Mode isn't just another feature; it’s a fundamental architectural shift in how LLMs execute tasks. Pioneered by Cloudflare in their Workers AI ecosystem and increasingly adopted by the vanguard of AI engineering, it moves us away from synthetic JSON handshakes and toward a world where LLMs write, test, and execute actual code to solve problems. It is the transition from the model being a caller to the model being a coder.

In this deep dive, I want to explore why the current JSON-based status quo is failing, the historical evolution that led us here, the technical brilliance of the "Dynamic Worker" sandbox, and why Code Mode represents the most important leap in AI engineering since the original transformer paper. This is my take on why Code Mode is the final boss of LLM execution.

{
  "subject": {
    "identity": "A high-tech digital brain or core",
    "features": "Luminescent neural pathways, glowing data streams, intricate mechanical integration",
    "expression": "Powerful, focused energy"
  },
  "environment": {
    "setting": "A vast, dark digital void or server rack room",
    "elements": ["Infinite rows of glowing servers", "Floating code fragments", "Ethernet cables weaving like vines"],
    "background_blur": "Deep bokeh"
  },
  "lighting": {
    "primary": "Cool electric blue and violet",
    "secondary": "Golden highlights on the circuitry",
    "style": "Cinematic, high-tech noir"
  },
  "camera": {
    "angle": "Wide angle, low perspective",
    "lens": "35mm",
    "format": "Digital 3D render"
  },
  "style": {
    "aesthetic": "Futuristic, clean, cybernetic",
    "color_palette": ["#00d4ff", "#6a00ff", "#121212"]
  }
}

A Brief History of LLM Interaction: From ReAct to Code Mode

To understand where we are going, we have to look at how we got here. The journey of making LLMs "do things" has been one of increasing structure and decreasing friction.

The ReAct Era (2022-2023)

In the early days of GPT-3 and GPT-4, we used a prompting pattern called ReAct (Reason + Act). We would tell the model: "You are an agent. You have access to a tool called 'Search'. If you need to know something, output Thought: I need to search for X. Action: Search(X). I will then provide the result."

This was brilliant but incredibly brittle. The model would often forget the formatting, hallucinate action names, or fail to parse the output. It was a "natural language handshake" that was prone to breaking whenever the model got slightly confused. We spent months fighting with "Regular Expressions" to parse LLM outputs. It was the Stone Age of agents.

The Function Calling Era (2023-2025)

OpenAI changed the game with native Function Calling. They moved the tool definitions out of the system prompt and into a dedicated API parameter. The model was fine-tuned to specifically output structured JSON.

This was a massive leap. It made agents reliable. We could finally build systems where the model would consistently say {\"name\": \"get_weather\", \"arguments\": \"{\\\"location\\\": \\\"San Francisco\\\"}\"}. This ushered in the era of the "Copilot." Every SaaS company on earth added a side panel where an AI could trigger a few pre-defined actions.

But as our ambitions grew, the limitations of Function Calling became apparent. We started hitting the "JSON Tax"—the cost of describing hundreds of functions in every prompt. We started hitting the "Latency Wall"—the multiple round-trips required for even simple logic. And most importantly, we hit the "Logic Gap."

The Code Mode Era (2026+)

Code Mode is the logical conclusion of this evolution. Instead of trying to teach a model a synthetic language (JSON-based tool calling), we give it the language it was already trained on: TypeScript.

In Code Mode, the model doesn't just pick a tool; it scripts the entire orchestration. It handles the if/else logic, the for loops, and the error handling internally within a single generated script. It’s no longer a remote control; it’s a localized brain.

The JSON Tax: Why Tool Calling is Breaking

Let's get technical about the "JSON Tax." Why is standard tool calling reaching its limits?

1. The Token Bloat Problem

Prompt engineering is essentially real-estate management. Every token you use to describe a tool is a token you can't use for the user's data or the model's reasoning.

Consider a modern enterprise API like Cloudflare or AWS. If you want an agent to be truly useful, it needs to understand hundreds, if not thousands, of endpoints. If each endpoint description takes 100-200 tokens (for the name, description, and JSON schema of parameters), you quickly find yourself in a situation where the "system overhead" of just explaining what the tools are consumes 20,000+ tokens.

You’re paying for those tokens on every single turn. Even if the user just says "Hello," you're sending the entire 20k-token manual. This isn't just a cost issue; it’s an Attention issue. Transformers have a finite "attention" budget. When you cram 500 tool definitions into the context, the model’s ability to reason about the user's specific problem degrades. It gets lost in the noise.

2. The Multi-Turn Latency Trap

In the JSON model, the LLM is a "caller." It issues a command, waits for the result, processes the result, and issues the next command.

Imagine a task: "Find all DNS records matching '*.dev.example.com' and change their TTL to 3600."

• Turn 1: LLM calls list_zones.
• Turn 2: LLM calls list_dns_records for the specific zone.
• Turn 3: LLM receives 1000 records, filters them in its head (risky!), and then issues the first update_record call.
• Turns 4-103: LLM issues 99 more update_record calls.

This is a disaster. Each turn takes 2-5 seconds of model latency, plus network latency. A simple task that should take 500ms of execution time takes 10 minutes of AI reasoning time.

3. Logic is Hard in JSON

JSON is static. It has no loops. It has no variables. If the LLM needs to do something conditional based on the output of tool A before calling tool B, it has to wait for tool A to finish, see the result, and then manually decide to call tool B.

We are essentially forcing a genius-level reasoning engine to act like a primitive 1970s mainframe terminal.

{
  "subject": {
    "identity": "A heavy, glowing golden coin embossed with a JSON brace '{ }'",
    "features": "Cracked surface, digital 'leaking' light",
    "expression": "Weighted, burdensome"
  },
  "environment": {
    "setting": "A digital balance scale",
    "elements": ["The JSON coin on one side", "A tiny, nimble code snippet on the other", "Floating binary numbers"],
    "background_blur": "Soft"
  },
  "lighting": {
    "primary": "Amber warmth on the JSON coin",
    "secondary": "Sharp white light on the code snippet",
    "style": "Dramatic contrast"
  },
  "camera": {
    "angle": "Close-up",
    "lens": "50mm macro",
    "format": "3D photorealistic"
  },
  "style": {
    "aesthetic": "Metaphorical, high-concept",
    "color_palette": ["#FFB800", "#FFFFFF", "#2B2B2B"]
  }
}

What is Code Mode? (The LLM as Software Engineer)

Code Mode flips the script. Instead of the LLM picking a tool from a list, you give the LLM a Typed SDK and the permission to write and execute its own scripts.

This is a subtle but profound shift. In Code Mode:

1. Discovery: The model uses a search() tool to find the specific parts of the SDK it needs. (This solves the Token Bloat problem—you only provide documentation for what's relevant to the current task).
2. Creation: It writes a TypeScript/JavaScript function that uses those tools.
3. Execution: It sends that code to a secure, sandboxed environment to run.

Why TypeScript? The Linguistic Advantage

LLMs are trained on billions of lines of code. They are actually better at writing valid, type-safe TypeScript than they are at following a custom, synthetic JSON schema. TypeScript is a "native" language for the model. It understands imports, it understands async/await, and critically, it understands type definitions.

When you provide the model with a .d.ts file (a TypeScript definition file), you aren't just giving it a list of functions. You are giving it a map of the entire system’s possibilities. The model can see what the inputs are, what the outputs are, and how they relate. It can reason about the structure of the API before it ever makes a call.

The MCP Connection (Model Context Protocol)

Code Mode is a perfect companion to the Model Context Protocol (MCP). MCP allows servers to expose tools and resources to LLMs in a standardized way. Code Mode takes this a step further: instead of the LLM consuming those tools one-by-one, it consumes the capability of the MCP server as a library.

Imagine an MCP server for your database. In the old world, the LLM would issue individual SQL queries via tool calls. In the Code Mode world, the LLM writes a script that uses the database client to perform complex joins, aggregations, and data transformations locally in the sandbox before returning the final, cleaned answer to you.

The Secret Sauce: Cloudflare’s Dynamic Workers and V8 Isolates

This is where the "magic" happens. If you let an AI write code and run it, you need to be absolutely sure it doesn't delete your production database, steal your secrets, or start mining Bitcoin.

Most people think of "running code" in terms of Containers (Docker) or Virtual Machines. But for AI-generated code, containers are too slow and VMs are too heavy.

V8 Isolates vs. Containers

Cloudflare Workers (and the Code Mode execution layer) run on V8 Isolates.

• Containers (Docker): When you start a container, you are starting an entire operating system kernel. It takes seconds. It consumes hundreds of megabytes of RAM. If an AI writes a script, waiting 5 seconds for a container to boot kills the user experience.
• V8 Isolates: V8 is the engine that runs JavaScript in Chrome. An "Isolate" is a lightweight sandbox within that engine. You can spin up thousands of Isolates in the same process. They boot in less than 10 milliseconds. They consume kilobytes of memory.

The Dynamic Worker Sandbox

Cloudflare’s implementation of Code Mode uses Dynamic Workers. When an LLM generates code, Cloudflare creates a one-time-use, ephemeral Worker.

This worker is a "Zero Trust" environment:

• No Public Internet: By default, the AI's code cannot call out to the internet. It can't "phone home" to a malicious server.
• Scoped Bindings: You only give the worker "bindings" (permissions) to the specific APIs it needs. If the agent is managing DNS, it doesn't get access to your billing information or your customer data.
• No Persistent State: The worker exists for seconds. It runs the script, returns the result, and is immediately destroyed. There is no way for a "malicious" AI script to hide in a corner of your infrastructure.

This is the "Security-First" approach to AI. You aren't giving the LLM the keys to the castle; you're giving it a highly restricted, temporary workspace with a specific set of tools and a guard at the door.

{
  "subject": {
    "identity": "A translucent, glowing crystal sphere",
    "features": "Inside the sphere, complex code is swirling like a storm",
    "expression": "Contained, controlled power"
  },
  "environment": {
    "setting": "A sleek, white laboratory or futuristic clean room",
    "elements": ["Minimalist tech tables", "Floating holographic shields around the sphere", "Laser sensors"],
    "background_blur": "High"
  },
  "lighting": {
    "primary": "Bright white clinical light",
    "secondary": "Soft blue glow from the sphere",
    "style": "High-key, futuristic"
  },
  "camera": {
    "angle": "Eye-level",
    "lens": "85mm",
    "format": "Digital painting"
  },
  "style": {
    "aesthetic": "Clean, secure, high-tech",
    "color_palette": ["#FFFFFF", "#E0F7FA", "#0097A7"]
  }
}

The Token Revolution: 99.9% Savings

Let’s talk economics. In the old JSON tool-calling world, as your API grows, your costs grow linearly. If your API doubles in size, your prompt doubles in size. This is a "Prompt Inflation" crisis that keeps architects up at night.

In Code Mode, your token footprint remains fixed.

Cloudflare’s Code Mode uses a two-tool strategy that I call "Pull vs. Push":

• The Push Model (JSON): This is the equivalent of trying to teach a student every single fact in an encyclopedia before they take a test. You push the entire documentation of the world into the model's context window. You hope the model remembers step 5 when it's at step 500. You hope it doesn't get "Lost in the Middle" (a known phenomenon where LLMs ignore information in the center of long prompts). Most importantly, you pay for every single word of that encyclopedia on every single turn of the conversation. If your encyclopedia is 100,000 tokens long, and you have a 10-turn conversation, you've just paid for 1,000,000 tokens of "knowledge" that the model might never have even used.

• The Pull Model (Code Mode): This is the equivalent of giving a student a library card and a search engine. You give the model a search() tool. The model says: "I have been asked to manage R2 buckets. I don't know the exact syntax for that. Let me search." The search tool returns only the relevant TypeScript interfaces and documentation for the R2 SDK. The model then "pulls" that specific knowledge into its working memory.

This shift from "pushing" to "pulling" documentation is the fundamental breakthrough of Code Mode. It decouples the complexity of your API from the cost of your inference. You can have an API with 10,000 methods, and the LLM will still only use a few hundred tokens of "discovery" overhead to find the three methods it actually needs.

For enterprise applications with massive, sprawling SDKs, this is the difference between a project being financially viable or a money pit. It also means you can support infinite tools. Since you aren't limited by the hard cap of the context window, your agent can effectively have access to every single API and database in your organization simultaneously.

The Developer Experience (DX) Shift: From Schema Writing to SDK Design

As developers, we’ve spent the last two years becoming "AI Plumbers." We’ve spent countless hours writing OpenAPI specs, JSON schemas, and natural language descriptions of our functions just so an LLM could understand them. We were essentially writing "manuals for machines."

Code Mode changes the Developer Experience (DX) fundamentally. It aligns AI development with traditional software engineering.

Type-Safe AI

In the JSON world, tool calling is "stringly typed." You hope the model outputs the right string. You hope it formats the date correctly. You hope it uses the right enum value. If it doesn't, your code crashes at runtime.

In Code Mode, we use TypeScript. We provide the model with a .d.ts file. If the model tries to call a function with the wrong number of arguments or the wrong type of data, the TypeScript compiler (running in the execution layer) will catch it before the code even runs. We can provide the model with immediate, actionable feedback: "Hey, you tried to call setLifecycle(days: number), but you passed a string. Here is the correct signature."

The model then fixes its own code. This "Reflection Loop" is much faster and more reliable than the traditional multi-turn error-handling loops of JSON tool calling.

The Rise of "LLM-First" SDKs

We are starting to see the emergence of "LLM-First" SDKs. These are libraries designed specifically to be consumed by AI agents in Code Mode. They feature:

1. Hyper-Descriptive Types: Using TypeScript's template literal types and JSDoc comments to give the model context directly in the code.
2. Atomic Operations: Functions that perform a single, clear task that the model can easily chain together.
3. Discovery-Friendly Structures: Hierarchical namespaces that make the search() tool more effective.

The best "AI Engineers" of the future won't be prompt wizards; they will be Library Architects.

The Human in the Loop: Supervision in Code Mode

One of the biggest pushbacks against Code Mode is the "Black Box" problem. "If an AI is writing and running code, how do I know what it's doing? How do I stop it if it goes rogue?"

The beauty of Code Mode is that code is Auditable.

The "Dry Run" Pattern

Before an agent executes a script in production, we can run it in a "Dry Run" mode. Because the code is just TypeScript, we can use static analysis tools to check for dangerous patterns. We can check if it's trying to access unauthorized bindings. We can even ask a second, more conservative LLM to "Code Review" the generated script.

Human-in-the-Loop Interruption

In a traditional JSON loop, the human is often a bottleneck. In Code Mode, the human is a Supervisor.

We can set up "Checkpoints." The agent writes a script to perform a complex migration. It then presents the script to a human engineer: "I have written this script to migrate 500 buckets. Here is the code. Do you want me to execute it?" The human can read the code (which is much clearer than a list of 500 JSON tool calls), approve it, and the agent executes it in milliseconds.

This allows us to scale human expertise. One engineer can supervise a fleet of ten Code Mode agents, only stepping in to "LGTM" (Looks Good To Me) the generated scripts.

{
  "subject": {
    "identity": "A high-tech control center",
    "features": "Multiple holographic screens displaying streaming code, a human hand hovering over a glowing 'Approve' button",
    "expression": "Strategic, oversight"
  },
  "environment": {
    "setting": "A bridge of a spaceship or a futuristic SOC (Security Operations Center)",
    "elements": ["Command consoles", "Stellar background through a window", "Floating data visualizations"],
    "background_blur": "Medium"
  },
  "lighting": {
    "primary": "Cool cyan and magenta accents",
    "secondary": "Soft ambient light on the consoles",
    "style": "Cinematic, tech-noir"
  },
  "camera": {
    "angle": "Over-the-shoulder",
    "lens": "35mm",
    "format": "Digital 3D render"
  },
  "style": {
    "aesthetic": "Professional, futuristic, high-stakes",
    "color_palette": ["#00BFFF", "#FF00FF", "#000000"]
  }
}

Orchestration Mastery: The Power of the Single Turn

The most profound change when you switch to Code Mode is how you think about "reasoning."

In the multi-turn JSON loop, the LLM has to hold the state of the task in its own "short-term memory" (the context window). If the task is long, the model gets distracted. It suffers from "context drift."

In Code Mode, the "reasoning" is baked into the code. The LLM creates a plan, writes the logic to handle that plan, and then hands it off to the sandbox.

Real-World Example: The Infrastructure Audit

User Prompt: "Audit all my R2 buckets. If any bucket doesn't have a 'lifecycle-policy' tag, add it and set the policy to delete files after 30 days. Report back with a summary."

The JSON Approach (The "Old Way"):

1. Turn 1: Call list_buckets. (Wait 3s).
2. Turn 2: Receive list of 50 buckets. LLM realizes it needs to check tags for each. Calls get_bucket_tags for bucket #1. (Wait 3s).
3. Turns 3-51: LLM calls get_bucket_tags for the remaining 49 buckets. (Wait 150s).
4. Turn 52: LLM processes the results. It finds 20 buckets need updates. Calls update_bucket_tags for bucket #1. (Wait 3s).
5. Turns 53-72: LLM calls update_bucket_tags for the rest. (Wait 60s).
6. Turn 73: LLM summarizes. Total time: 4 minutes. Total cost: 73 round-trips to GPT-4o ($1.50).

The Code Mode Approach (The "New Way"):

1. Turn 1: Model calls search('R2 bucket management'). Receives SDK docs.
2. Turn 2: Model writes a script:
   text

   Copy

   const buckets = await r2.listBuckets();
   const results = [];
   for (const bucket of buckets) {
     const tags = await r2.getTags(bucket.name);
     if (!tags['lifecycle-policy']) {
       await r2.setTags(bucket.name, { ...tags, 'lifecycle-policy': 'active' });
       await r2.setLifecycle(bucket.name, 30);
       results.push(bucket.name);
     }
   }
   return `Updated ${results.length} buckets: ${results.join(', ')}`;

3. Turn 3: Model calls execute(script). The script runs in the Cloudflare edge in 200ms.
4. Turn 4: Model receives the summary and reports back to the user. Total time: 10 seconds. Total cost: 4 round-trips ($0.05).

The Code Mode agent is 24x faster and 30x cheaper. It handles the "logic" (the loops and conditionals) where it belongs: in the execution environment, not in the model's head.

My Take: Why This is the "End Game" for AI Engineering

As someone who spends a lot of time thinking about the "plumbing" of the internet, I believe Code Mode is the moment AI becomes "Real."

We’ve had our fun with chatbots. We’ve enjoyed the novelty of a machine that can write poems and summarize emails. But if we want AI to manage our infrastructure, optimize our databases, or automate our security operations, we cannot rely on brittle JSON handshakes. We need the same tools that humans have used to build the modern world: Code.

By giving LLMs the ability to write and run code in a safe, sandboxed environment, we are essentially giving them "hands."

The Death of the "Prompt Engineer"

For a long time, the industry was obsessed with "Prompt Engineering"—the art of coaxing a model to output exactly the right JSON. Code Mode makes this largely obsolete. We don't need to "coax" the model to follow a synthetic schema; we just need to provide a well-typed SDK.

The best "AI Engineers" of the future won't be prompt wizards; they will be API Designers. If you build a clean, typed, and well-documented SDK, the AI will be able to use it perfectly. If your SDK is messy, inconsistent, and poorly typed, the AI will struggle. Code Mode brings AI engineering back into the realm of traditional software engineering best practices.

The Competitive Advantage

If you are building an AI-powered product today, and you are still using 20k-token prompts to describe your tools, you are already behind. You are paying more for a slower, less reliable product.

Code Mode allows you to:

1. Scale to Infinite Tools: Your agent can navigate thousands of APIs without breaking the context window.
2. Ship Faster: You don't need to manually write JSON schemas for every new feature. Just give the agent access to the SDK.
3. Improve Reliability: Code is easier to test and debug than a black-box reasoning loop. You can even run "pre-flight" checks on the AI's generated code before you execute it.

{
  "subject": {
    "identity": "A professional developer and an AI agent working together",
    "features": "Handshake between a human hand and a robotic hand composed of data streams",
    "expression": "Collaboration, trust"
  },
  "environment": {
    "setting": "A bridge between two worlds: physical and digital",
    "elements": ["Floating code architecture", "Sunrise in the background", "Sleek glass walkway"],
    "background_blur": "Medium"
  },
  "lighting": {
    "primary": "Warm golden sunrise light",
    "secondary": "Soft blue ambient light from the data",
    "style": "Optimistic, epic"
  },
  "camera": {
    "angle": "Eye-level, medium shot",
    "lens": "35mm",
    "format": "Cinematic photography"
  },
  "style": {
    "aesthetic": "Hopeful, futuristic, clean",
    "color_palette": ["#FF9E00", "#007AFF", "#FFFFFF"]
  }
}

Looking Ahead: The Self-Healing Infrastructure

Where does this go?

I envision a world where "DevOps" becomes "AgentOps." Instead of humans writing Terraform files and monitoring dashboards, we’ll have a fleet of specialized Code Mode agents.

The Self-Healing Cloud

Imagine a Cloudflare Worker that detects a DDoS attack. In the old world, it would trigger an alert, and a human engineer would wake up at 3 AM to adjust firewall rules.

In the Code Mode world, a "Sentinel Agent" sees the alert. It writes a script to analyze the traffic patterns, identifies the malicious signatures, and then writes a new set of WAF (Web Application Firewall) rules. It executes those rules, verifies they are working, and then sends a Slack message to the human engineer: "I detected an attack and mitigated it. Here is the post-mortem script I ran."

Autonomous Cost Optimization

Imagine an agent that has a "Read-Write" binding to your entire cloud billing and infrastructure. Every night, it runs a script to find under-utilized R2 buckets or over-provisioned database instances. It doesn't just "recommend" changes; it writes the migration scripts, executes them in a sandbox, verifies the data integrity, and applies the changes.

The Rise of "Agentic Memory"

One of the biggest challenges in AI is "Memory." How does an agent remember what it did yesterday? In Code Mode, the agent can write its own "logs" to a KV store or a database. It can build its own knowledge base. If it encounters a bug in an SDK, it can write a "patch" script and store it for future use. The agent starts to build its own library of successful execution patterns.

The transition from "Chat" to "Code Mode" is the transition from AI as a toy to AI as a foundational layer of our digital infrastructure.

Conclusion

The "JSON Tax" is a relic of the early days of LLM integration. It served us well as a bridge, but it’s time to move on.

Code Mode—especially when paired with the security and performance of Cloudflare’s Dynamic Workers—represents the mature path forward. It respects the constraints of the context window, it leverages the native strengths of the model, and it maintains a hard line on security through sandboxing.

It is the moment where we stop treating the LLM as a "user" of our APIs and start treating it as a Developer of our systems.

If you haven't explored the @cloudflare/codemode SDK yet, now is the time. The era of the "Tool Caller" is over. The era of the "Code Executing Agent" has begun.

Are you ready to stop chatting and start executing?

What’s your take on Code Mode? Are you still paying the JSON Tax, or have you moved to sandboxed execution? Let's discuss on X/Twitter.