.avif)
.avif)
.avif)
The industry did the right thing when it broke the single all-knowing agent into specialists. One prompt carrying every instruction, every tool and every rule loses focus as it grows, hallucination risk rises with it, and adding one scenario means retesting everything. Distributing that load across specialists, each with its own instructions, knowledge and tools under a supervisor, keeps changes local and instructions short. We build on that architecture, and we still think it is right.
What gets discussed less is what the shape costs once it is carrying real traffic. Every handoff carries the conversation history forward. Every return trip to the supervisor is another model call over that history. Every tool result stays in the context window long after the agent has finished with it, and is paid for again on every turn that follows.
That cost is easy to miss, because it is not charged on every turn. A sub-agent that has the conversation answers the customer directly, for as many turns as the topic stays with it, and the supervisor is not involved. It comes back in only when the conversation needs to move to a different specialist. So the cost does not spread evenly across a session. It comes in spikes, and each spike lands at the worst possible moment: the history is at its longest, and the customer is mid-problem and waiting.
Take a customer calling to arrange a service visit. One specialist establishes which product it is. Another takes the name and phone number. Another confirms the address. Another finds an appointment slot. To the customer that is one request, asked once. Before this work, every one of those steps ended with the conversation going back up to the supervisor, which re-read everything said so far before deciding who should speak next.
Anthropic published its own engineering measurement of the wider cost in June 2025, writing up the multi-agent research system it built: “In our data, agents typically use about 4× more tokens than chat interactions, and multi-agent systems use about 15× more tokens than chats.” That is a model provider describing its own system rather than independent analyst research, and it should be read that way. The same write-up reports:
“We found that token usage by itself explains 80% of the variance, with the number of tool calls and the model choice as the two other explanatory factors.”
Source: Anthropic Engineering, “How we built our multi-agent research system” (June 13, 2025) https://www.anthropic.com/engineering/multi-agent-research-system .
So the thing that made the system manageable is also what makes it slow and expensive, and the expense is not where the architecture diagram puts it. What a multi-agent system costs in production is decided at the seams between agents, not inside them.
There is a reflex that says context is free insurance. Keep everything, in case the agent needs it. The measured evidence does not support the reflex.
Chroma Research published an evaluation in July 2025 of 18 models across the GPT-4.1, Claude 4, Gemini 2.5 and Qwen3 families, varying input length while holding the task fixed. It found that “Model performance varies significantly as input length changes, even on simple tasks”, and that performance “grows increasingly unreliable as input length grows”. The model coverage is broad, and the direction matches what anyone who has debugged a long session already recognizes.
Source: Chroma Research, “Context Rot: How Increasing Input Tokens Impacts LLM Performance” (July 2025).
This is what stops the whole subject from being an efficiency announcement. A diagnostics call returns a large structured payload. The agent reads one field from it and answers the customer. Fifteen turns later, that entire payload is still in every request, competing for the model's attention with the sentence the customer just typed. Removing it is not housekeeping. Cost and quality move the same way here, so there is nothing to trade off.
Each of those costs has a specific place where it can be removed. In one production deployment, three changes went live together, all of them about how much context and control passes between agents.
Two specialists who need each other should not have to talk through a third.
The obvious design keeps the supervisor as the single router, because then there is one place to reason about routing. That is free while a topic stays with one specialist. It gets expensive the moment the topic moves, because the return trip is another model call over the whole conversation so far, and the customer waits through it.
Sub-Agent Collaborations let one sub-agent hand a conversation straight to another, in one direction or in both. So the agent that establishes which product it is now hands directly to the one that takes the contact details, and that one hands directly to the one that confirms the address. The supervisor is never asked. The model call over the accumulated history is not made shorter. It is not made at all. History is shared, so the receiving agent picks up where the last one stopped and the customer repeats nothing.
When the routing decision is already known, do not ask a model to make it.
The obvious design lets the agent decide who speaks next, since deciding is what agents are for. But not every handoff is a decision. Once a tool has recorded which product the customer has, who speaks next is already settled: one product category always goes to one specialist, another always goes to another, and there is nothing left to work out.
So the tool names the next agent inside its own response, at the moment it finishes. The same happens when a tool records what kind of case it is, sending the conversation to the specialist for a replacement or the one for a spare part. The handoff costs no model call at all, because nothing had to be asked.
That also marks the limit of the technique. Where the next step genuinely cannot be known in advance, the model should still make the call, and paying for that decision is cheaper than guessing wrong.
A tool result that has done its job should stop being paid for on every turn.
The counter here is real: prune something the agent still needed and you have traded cost for a failure. That is why Context Pruning is set per tool rather than globally. A tool's output can be dropped after a set number of customer turns, or the moment the conversation moves to another agent, and what stays behind is a note that the call happened.
The address check and the warranty lookup go as soon as the conversation transfers. Once the address is verified the verification stands, and the payload behind it is dead weight. The list of available appointment slots is different, because the customer needs a turn or two to pick from it. So it is kept for exactly that long, then dropped before it becomes a long stale list paid for on every request.
That is the difference between a setting and a decision.
We compared two stretches of production traffic, the same days of the week one week apart, with tens of thousands of calls on each side. What follows are averages over real conversations rather than over a sample.
Two stretches of production traffic, the same days of the week one week apart, tens of thousands of calls on each side.
The one change a caller can feel is also the largest. Average time to first token fell 31%. That is the gap between the customer finishing their sentence and the system starting to answer, and it is where responsiveness gets judged. A caller cannot tell you why a system feels slow, but they can tell you that it does.
What the system is asked for fell as well. Average tokens per session fell 17%.
A conversation that ends sooner can also mean people giving up, so on its own that figure proves nothing. The self-service rate held at close to 70%: the same share of callers finished what they came for without needing a person. That is what makes the other numbers mean anything. Without it they describe a system doing less, not a system doing the same for less.
It is worth being clear about what this comparison is. Two stretches of live traffic set side by side is not an experiment, and since all three changes went live at once, none of them can be given the credit on its own.
Those figures have a consequence beyond the invoice. Capacity here is bought in advance, so it is a ceiling that was fixed long before any of this shipped. What consumes that ceiling is how many requests arrive in the busy hour and how large each one is. The first belongs to the customers. The second is what the wiring between the agents decides.
Nothing was bought to get this. The model is the same, the vendor is the same, the contract is the same. What changed is how control and context move between the agents, and the same work now costs less and answers faster.
Everyone building one of these systems draws the same picture: boxes for the agents, lines between them. The boxes get the prompts, the tools, the test sets, the arguments in review. The lines get drawn last and are never costed, and the lines are where the bill is set.
The familiar account of why pilots never reach production runs through goals that were never defined, integrations that were skipped, data nobody collected. One item is missing from it, and it only shows up at volume: what the architecture costs to run.
So the price of this technology is not something you are quoted; it's something you design.
Nobody is graded on the diagram they drew. The grade arrives with the traffic, on an ordinary morning.
.avif)
