Controlling agent costs starts with measuring where tokens actually go, since a single agent task can trigger dozens of model calls across planning, tool selection and result synthesis, and unmonitored deployments routinely surprise teams with their bill. The highest-leverage lever is model routing, using a small, cheap model for classification, extraction and simple tool selection, and reserving an expensive frontier model only for the reasoning steps that genuinely need it, which can cut average cost per task substantially without hurting quality on the steps that matter. Prompt caching, supported by both Anthropic and OpenAI, meaningfully reduces cost for agents that repeatedly send the same system prompt or tool definitions, and trimming tool schemas and context to only what a given step needs avoids paying to process irrelevant tokens on every call. Setting a hard step limit and a cost budget per task prevents a stuck or looping agent from running up unbounded charges, and this should be enforced in code rather than left to the model's judgment. Ongoing cost monitoring per workflow, broken down by model and step, lets you catch regressions after a prompt or tool change. Nanobase AI builds this cost telemetry and model-routing logic into agent deployments from the start rather than retrofitting it after a billing surprise.
You cannot control what you have not broken down
Teams that get an unpleasant billing surprise from an agent deployment almost always share the same root cause: they measured total spend without ever breaking it down by where the tokens in a single task actually went. A single agent run is not one model call; it is often a dozen or more, each carrying its own system prompt, tool schema definitions, and accumulated context from prior steps, and without a per-component breakdown, cost optimization becomes guesswork. The first cost-control step is instrumenting token usage by component, not by task.
| Cost component | What drives it | Primary lever |
|---|---|---|
| System prompt | Sent on every single model call in the run | Prompt caching |
| Tool schema definitions | Sent alongside the system prompt on every call | Trim to only tools relevant to the current step |
| Reasoning and planning tokens | Grows with task complexity and step count | Model tier routing for simple steps |
| Tool call results | Size of data returned from each tool | Truncate or summarize large results before feeding back to the model |
| Final synthesis | One larger call combining everything | Model tier routing, since this step often benefits from a stronger model |
Model routing is the highest-leverage lever
Of every cost-control technique, routing steps to different model tiers based on actual difficulty delivers the largest reduction with the least risk to quality, because most agent tasks contain a mix of simple, high-volume steps like classification or extraction alongside a small number of genuinely hard reasoning steps. Sending everything to a frontier model pays a premium on every step regardless of whether that step needed it, while reserving the frontier model specifically for the steps that benefit from it, and routing the rest to a smaller, cheaper model, cuts average cost per task substantially without touching the accuracy that matters most.
Prompt caching pays off on repeated structure, not repeated content
Both Anthropic and OpenAI support prompt caching, which meaningfully reduces cost for agents that repeatedly send the same system prompt and tool definitions across many calls within a single task or across many tasks in a session, since the cached portion is billed at a fraction of the cost of processing it fresh each time. This benefit is largest specifically for agents with a large, stable tool schema and a long system prompt relative to the actual task-specific content, which describes most enterprise agents with a well-developed tool catalog.
A cost audit checklist
A per-component breakdown, not a single total, is what turns a rising bill into a specific, fixable line item.
- Instrument token usage per model call, tagged by which component of the task it belongs to, not just total tokens per task.
- Identify the steps consuming the most tokens and check whether they genuinely need the model tier currently assigned to them.
- Confirm prompt caching is actually enabled and effective, since a system prompt that changes slightly between calls can silently defeat caching.
- Check tool result sizes flowing back into context; a tool returning a full database row set when only three fields are needed wastes tokens on every subsequent call in that task.
- Set a hard cost budget per task, enforced in code, so a stuck or looping agent cannot run up unbounded charges regardless of what caused the loop.
Enforcing budgets in code, not in the prompt
A cost budget instructed to the model in a prompt is not a budget; it is a suggestion the model can ignore under an unusual input or a long, complex task. The actual enforcement needs to live in the orchestration layer: a hard step count limit, a token budget tracked across the task, and a circuit that halts execution and escalates to a human once either limit is reached. This is the same principle that applies to safety controls generally, that anything genuinely required should be enforced outside the model rather than requested of it.
Frequently asked questions
Does switching to a cheaper model always save money without hurting quality?
Not universally; it depends on whether the specific step actually needs the reasoning capability of the more expensive model. Blind downgrading without measuring task-specific accuracy on your evaluation set can silently degrade quality on steps that genuinely needed the stronger model.
How much can prompt caching realistically save?
The savings depend heavily on how large and stable your system prompt and tool schemas are relative to task-specific content, and on how many calls within a task or session reuse that same cached portion. Agents with large tool catalogs and long system prompts tend to see the most benefit.
Should we track cost per task or cost per model call?
Both, but cost per task is the metric that maps to business value and is what a budget should be set against, while cost per model call breakdown is what actually tells you where to optimize. Tracking only the aggregate hides which specific component is driving spend.
What is the first sign that agent costs are becoming a problem?
A rising average cost per task that is not matched by a corresponding rise in task complexity or volume, which usually indicates either a model routing issue, an ineffective cache, or an agent occasionally looping longer than expected on certain inputs.
How Nanobase AI helps
Nanobase AI, a Silicon Valley enterprise AI engineering company, builds this cost telemetry and model-routing logic into agent deployments from the start rather than retrofitting it after a billing surprise. Related infrastructure decisions are covered in running agents fully on-premise with open-weight models and in the broader cost comparison of owned GPUs versus cloud API pricing.
Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.