Cut AI Agent Costs: Smart Harness Tricks
Reduce operational expenses by optimizing tool call delegation in AI agents
Recently, I spent quite a bit of time optimizing the cost of my agent harness. If you read my last article, you know my agents can already handle most of my daily routine automatically. However, the operational cost turned out to be huge. Once the architecture stabilized, my next step was to cut spending and improve efficiency.
The most urgent task in reducing costs is finding out where the money actually goes. We can only save money if we know what we are spending it on.
I initially thought that unstable cache prefixes were causing frequent cache misses and driving up total spending. I spent some time researching my session logs, only to discover a surprising fact. My cache hit rate was actually very high, hovering around at least 95%. Of course, there is still room for improvement here since this relates to how my loop engineering works. I hand over tasks between agents quite often, so cold starts happen frequently. This is an area we can tweak, but the room for optimization is limited. Cache pricing is low to begin with, and raising the hit rate from 95% to 99% brings less than a 5% gain.
So where is the real bottleneck? To explain the core issue, let’s make a few assumptions to keep our mathematical model simple.
Here are the assumptions:
Each tool call costs 1 unit, including input and output.
Each turn requires at least 10 tool calls.
Under these rules, our cost structure looks like this:
The cost for Turn k is 10k, making the total cost:
Because agents have no built-in memory, each turn carries forward the cumulative output of all previous turns as the session continues. Since this follows an O(N²) sequence, costs skyrocket very quickly over long turns.
We can flatten this cost curve in two ways.
First, let’s talk about the approach that is hard to implement: reducing the value of k. This means finishing the work in as few turns as possible. If turn counts grow too long, we must either fix issues in the agent’s workflow or compact the context and hand off the task to another agent. When task complexity is high, striking the right balance becomes extremely difficult. How long is long enough? When is the ideal moment to compact? These decisions are tough even for human engineers, let alone getting an agent to judge them autonomously.
The alternative approach is reducing the compounding cost created by tool calling output. Specifically, we delegate all tool calls from the main session to subagents, letting the main session receive only the final conclusion.
Let’s look at another mathematical model.
We keep the previous assumptions and introduce a new notation M, which represents the setup cost of delegating to a subagent. We can simplify the subagent’s output cost to 1.
The cost for Turn k is M+10+k, making the total cost:
As shown in the table above, the execution volume per turn stays constant at 10 tool calls. The difference is that we preserve only the final output in the main session while discarding intermediate steps. This transforms the compounding 10k factor into a simple k.
With formulas for both models, the rest is straightforward math. We need to find the Break-Even Point, calculating the cost difference and the threshold where savings exceed zero.
Subtracting the two equations gives us the cost difference:
The threshold condition is:
Let’s assume M costs 10, making it equal to the cost of a single turn. The resulting break-even point occurs when N is greater than or equal to 4.
In simple terms, following this strategy makes any session lasting 4 turns or longer cheaper. This model relies on ideal assumptions, such as a delegation cost of 10 and 10 tool calls per turn. But if you handle complex engineering tasks, you know how easily real workloads hit these numbers. These assumptions actually underestimate real-world usage.
The benefits are clear.
However, this strategy introduces a key challenge: what conditions should the main session use to decide whether to delegate a task instead of handling it directly?
Delegating brings overhead. If we delegate simple commands like ls or grep, we end up losing money. What objective criteria should an agent use to make this call? Delegation only makes sense when the total cost of tool calling exceeds M, yet an agent cannot predict how many tool calls a turn will need. How can we help the agent make the right choice?
This is where the classic Knowledge Matrix comes in handy.
Known knowns: The agent handles this quadrant easily using its training data or the context we provide.
Unknown knowns: This quadrant is trickier, but a comprehensive knowledge base reduces the gap significantly.
An agent can tackle these first two quadrants through brute-force effort. When we handle complex workloads, however, we usually hit the unknown territory.
Known unknowns: The agent uses reasoning to identify missing information. For example, it might check git history, verify ticket status in Jira, and validate deployment via kubectl. This sequence of actions generates tool calls.
Unknown unknowns: This quadrant is even more complex. Lacking sufficient context, the agent must rely on trial and error, digging up details through various tool calls while analyzing the results on the fly.
Our proposed solution targets the Known Unknowns quadrant directly.
With a clear objective and a working formula, our next step is simple. We instruct the main session to list its required information explicitly before starting a task. If it can generate this list, it has entered the known unknowns space. When the item count exceeds a set threshold, it delegates the job to a subagent. I set my threshold at 3, though this remains a magic number.
This approach cuts costs effortlessly.
Wrap Up
I reviewed my setup using past prompts, and the performance improvement on long sessions was remarkable. Single-session costs dropped to 20% of their original baseline, yielding an 80% discount.
Although this approach sounds simple on paper, fine-tuning the details takes real effort. How do we prompt the main session effectively? Should we use skills or AGENTS.md? What instructions work best? How do we teach the main session to handle each quadrant of the Knowledge Matrix?
Because every engineer’s workload differs, I cannot offer a one-size-fits-all solution. My work focuses heavily on research and investigation, naturally generating many tool calls. If your daily work centers on writing code, your tasks will involve basic file access, which follows a completely different pattern.
As with my previous posts, I am sharing the underlying methodology so you can customize it for your needs. The core principle remains simple: convert repetitive intra-turn tool calls into fixed costs, preventing total expenses from compounding out of control.
Originally published on Medium