Controlling AI agent costs: budgets, routing and loops
Agents don't make one model call — they make dozens. Without controls, costs grow quietly and then all at once.
A chatbot makes one model call per message. An agent might make twenty: planning, calling tools, reflecting, retrying. Multiply by thousands of tasks a day and spend can move fast. Here's how to keep it predictable without starving your agents.
Know where the money goes
You can't control what you can't see. Track cost at every level:
- Per model call: tokens in, tokens out, price
- Per run: total cost of completing one task
- Per agent, team and customer: who is spending what
Cost per successful task is the number that matters. A cheap agent that fails half the time is expensive.
Set budgets at several levels
- Per run: a hard cap on how much one task may cost. This stops runaway loops.
- Per agent per day: an alert at 80%, a stop or a downgrade at 100%.
- Per organisation: a monthly ceiling, with clear owners.
When a budget is hit, the agent should fail gracefully. It can hand off to a human, switch to a cheaper path or queue the work, rather than crashing mid-task.
Route to the right model
Not every step needs your most capable model. Common patterns:
- Tiered routing: classify, extract and format with a fast model; save the strongest model for reasoning-heavy steps.
- Escalation routing: try the cheaper model first, then retry with a stronger one only when confidence is low.
- Fallback routing: if a provider is slow or down, switch automatically.
Watch for loops
The most expensive agent bugs are loops: an agent that retries the same failing tool, or two agents handing a task back and forth. Defend against them with:
- A maximum step count per run
- Detection of repeated identical tool calls
- Per-run budgets, which cap the damage even when the first two miss
Cache what repeats
System prompts, tool definitions and shared context often repeat across calls. Prompt caching, where your provider supports it, can cut costs on long-running agents significantly.
Cost control isn't about spending less on AI. It's about knowing exactly what each outcome costs, so you can confidently spend more where it pays off.
See Argos Nexus AI in action
Governed agents, human review and full audit — in one platform.