Why does an AI agent use hundreds to thousands of times a single request?_

The International Energy Agency (Key Questions on Energy and AI, April 2026) puts the GPU electricity of a request at 0.05 Wh for a medium model without reasoning, and at 50 Wh for an agent that reasons across four to six successive calls: a ratio of 1,000, by our calculation on its Figure 2.1, which the IEA gives as an order of magnitude. There is nothing mysterious about the mechanism: on every turn, the client sends the entire history back to the model.

The arithmetic consequence: doubling the number of turns quadruples the input tokens billed. No hardware efficiency gain absorbs a factor of 4 per doubling.

A loop, not an exchange

Two mechanical properties matter for what follows.

The API is stateless. The model remembers nothing. On every turn the client hands it the whole conversation, and the provider processes all of it.

The context holds far more than the conversation. There is the system prompt, the tool definitions, the instruction files and the results of previous calls. On a coding agent, the system prompt alone runs to tens of thousands of tokens, resent on every turn. MISSING DATA on the exact value: no vendor publishes the size of its system prompt, and it changes with every product release. You can measure it client side, by counting the input tokens of the first turn in an empty session.

The quadratic growth of a dialogue

If each turn adds A tokens, question and answer included, turn number i bills i times A input tokens. The running total after N turns is A × N × (N+1) / 2, growth in the square of the number of turns.

With 1,000 tokens added per turn:

At the Claude Sonnet 5.5 input rate, $2 per million tokens, a 50-turn conversation costs $2.55 in input tokens alone, against $0.10 if the history were not resent.

One necessary caveat: this formula is an elementary piece of arithmetic, not a citation. No published source documents or quantifies the phenomenon explicitly. Anthropic's engineering piece on context engineering, published on 29 September 2025, covers the subject with no numbers at all: it talks about "context rot" and about context as a finite resource with diminishing returns, and recommends condensed summaries "often 1,000 to 2,000 tokens".

What caching fixes, and what it does not

This quadratic term is the same one described in the chapter on long context. The prefix is precisely the part that repeats, so caching cuts exactly that quadratic term.

On the 50-turn example, with Sonnet 5.5's cache read at 5% of the input price, the $2.55 falls to about $0.25: the 1,000 new tokens of each turn stay at full price, plus 25% for the cache write. Measured on real sessions rather than in theory, the gain is a little smaller: Lumer et al. (Don't Break the Cache, arXiv:2601.06007, 9 January 2026), across more than 500 agent sessions from the DeepResearch Bench, record −79.6% cost on GPT-5.2, −78.5% on Claude Sonnet 4.5 and −41.4% on Gemini 2.5 Pro. The gap with the read-price discount, −90% at 10%, is not a disagreement, it is a change of denominator: the discount applies to re-read tokens, the −78.5% to the whole session.

What caching does not change, Anthropic states outright: "Cached prompt prefixes still occupy the context window: prompt caching changes what you pay for those tokens, not whether they count." The window fills at the same rate, and rate limits apply to total volume.

A second effect is specific to recent models: reasoning blocks from previous turns are kept by default and count towards the window as input tokens. A reasoning token is therefore billed once at the output rate when it is produced, then at the input rate on every subsequent turn.

And one underlying question is still open. No provider documents whether it keeps computed state in memory or recomputes it. Only one measurement compares the energy of a cache hit with that of a miss, Irminsul, and only on a 4,096 token prefix. The fact that Google bills hourly storage for its explicit cache, up to $4.50 per million tokens per hour, does suggest a physical resource is being tied up. Writing that caching divides the footprint by ten would be extrapolating from price to energy.

What an agentic task actually consumes

Public figures from the Terminal-Bench 4.0 leaderboard, recorded on 3 September 2026 and rechecked on 10 October 2026. These are aggregate totals across the whole benchmark, all attempts combined.

The leaderboard publishes 66 tasks and 330 attempts per model, five attempts per task, which makes the per-task cost computable: about $50 per task for GPT-6 Astra, $10 per attempt. The top of the leaderboard has moved since the first reading (Opus 5.5, Sonnet 5.5 and GPT-6.1 Sol went ahead); the table keeps the original six models. On the SWE-bench side, the official leaderboards publish a cost for only some submissions, and no token counts, so the comparison stops there.

Running this benchmark once costs 1.5 to 17 billion tokens and $1,800 to $6,200, across the six models in the table.

Now look at the last two columns. Gemini 3.8 Flash uses 11.5 times more tokens than GPT-6 Astra, and yet it is the cheapest run in the table. Between the most and least expensive, the cost gap is only 3.4. Price per token varies so much between models that it reverses the consumption ranking. Cost alone is not enough to steer energy.

What the harness decides

Ismael Velasco's opinion, on the "Conceptual definitions and indirect impacts" panel of the ENS/OBVIA day on 9 June 2026: "If you have a really efficient model in a really inefficient harness, the impacts will be way worse than a really inefficient model in a really efficient harness." MISSING DATA on the reference: no public recording of the talk could be found. It is a formulation to reuse, not a result to cite.

That last figure has a direct methodological consequence. The only published measurement of the energy gain from a cache read was made on a 4,096 token prefix. Transposing that result to a regime thirty times longer would be a genuine change of scale, not a detail. Hence a conservative retained value.

One overhead appears on no invoice: provider-side monitoring of agentic models, which uses inference compute never billed to the client. The system prompt, for its part, is billed on every turn, but no vendor publishes its size: you only know it by measuring it. MISSING DATA on both: a figure of around 20% of inference compute circulates for monitoring at OpenAI, with no traceable publication behind it. A methodology still has to decide whether its per-token factor includes that share or documents it separately, and write down which it chose.

What this changes for an audit

The relevant unit becomes the task completed, rather than the request or the token. The SCI for AI standard has already put this on record, setting workflow execution as the functional unit for agentic AI.

Ask for three things: the number of turns per session, the median context length, and the token split between fresh input and cache writes on one side, cache reads and output on the other. Those three numbers are enough to rebuild most of the footprint, and they all sit in the usage exports.

The lever is rarely where teams look for it. On a real base of agentic sessions, the genuinely compressible share of the footprint, fresh input and cache writes together, is 28.8%. A perfect prompt compressor applied to new tokens only, with no effect on the history already in cache, would therefore never get below 71% of the total. What moves the needle acts on output and on the choice of model.

FAQ

Does an AI agent really use thousands of times more than a simple request?

That is our calculation on the International Energy Agency's Figure 2.1 (April 2026), between its two extremes: 0.05 Wh for a request without reasoning, 50 Wh for an agent that reasons across four to six calls. Without reasoning, the IEA's agent is only about 23 times. The IEA's text speaks of "hundreds or thousands of times" for video, reasoning and agents taken together. The Green IT 2025 study, cited with caution by Arcep, puts forward a factor of 60 between an agentic service and a classic generative AI service. The two measure different things, and the literature on this is still young.

Why does the cost of a conversation with an agent rise so fast?

Because the API is stateless: on every turn, the client resends the entire history. Turn ten therefore bills ten times the volume of one turn, and the running total grows as the square of the number of turns. Doubling the number of turns quadruples the input tokens billed, and a 50-turn conversation costs 25 times what a linear total would.

Is prompt caching enough to solve the problem?

It largely solves the price. OpenAI and Google bill a cache read at about 10% of a fresh input token, and Anthropic at 5% on Opus 5.5 and Sonnet 5.5, 2.5% on Fable 5.1 and Mythos 5.1, 10% on its other models (price lists as of 10 October 2026), and measurements across more than 500 agent sessions record cost reductions of 41 to 80% depending on the model. It solves neither the occupation of the context window nor the rate limits, and nobody has measured its exact energy gain.

How many tokens does an automated development task consume?

For the six models listed above, the Terminal-Bench 4.0 leaderboard publishes totals ranging from 1.5 to 17.2 billion tokens and $1,800 to $6,200 to run the whole benchmark once, across 66 tasks and 330 attempts. Worth noting: the token gap between models reaches a factor of 11.5 while the cost gap is only 3.4, and the hungriest in tokens, Gemini 3.8 Flash, is the cheapest: price per token hides the consumption gap.

What data should I ask for to quantify an agentic footprint?

Three things, all present in provider usage exports: the number of turns per session, the median context length, and the token split between fresh input and cache writes on one side, cache reads and output on the other. A global token volume without that split leads to gaps of a factor of 3.3 to 3.6 depending on the rate applied to context re-reads.