Why does an AI agent use up to 1,000 times a single request?_

The International Energy Agency puts an agentic task at up to 1,000 times more electricity than a simple text request. There is nothing mysterious about the mechanism: on every turn, the client sends the entire history back to the model.

The arithmetic consequence: doubling the number of turns quadruples the input tokens billed. No hardware efficiency gain absorbs a factor of 4 per doubling.

A loop, not an exchange

Two mechanical properties matter for what follows.

The API is stateless. The model remembers nothing. On every turn the client hands it the whole conversation, and the provider processes all of it.

The context holds far more than the conversation. There is the system prompt, the tool definitions, the instruction files and the results of previous calls. On a coding agent, the system prompt alone runs to tens of thousands of tokens, resent on every turn. MISSING DATA on the exact value: no vendor publishes the size of its system prompt, and it changes with every product release. You can measure it client side, by counting the input tokens of the first turn in an empty session.

The quadratic growth of a dialogue

If each turn adds A tokens, question and answer included, turn number i bills i times A input tokens. The running total after N turns is A × N × (N+1) / 2, growth in the square of the number of turns.

With 1,000 tokens added per turn:

At the Claude Sonnet 5 input rate, $2 per million tokens, a 50-turn conversation costs $2.55 in input tokens alone, against $0.10 if the history were not resent.

One necessary caveat: this formula is an elementary piece of arithmetic, not a citation. No published source documents or quantifies the phenomenon explicitly. Anthropic's engineering piece on context engineering, published on 29 September 2025, covers the subject with no numbers at all: it talks about "context rot" and about context as a finite resource with diminishing returns, and recommends condensed summaries "often 1,000 to 2,000 tokens".

What caching fixes, and what it does not

This quadratic term is the same one described in the chapter on long context. The prefix is precisely the part that repeats, so caching cuts exactly that quadratic term.

On the 50-turn example, the $2.55 becomes $0.26. Measured on real sessions rather than in theory, the gain is a little smaller: Lumer et al. (Don't Break the Cache, arXiv:2601.06007, 9 January 2026), across more than 500 agent sessions from the DeepResearch Bench, record −79.6% cost on GPT-5.2, −78.5% on Claude Sonnet 4.5 and −41.4% on Gemini 2.5 Pro. The gap with the headline −90% is not a disagreement, it is a change of denominator: −90% on cached tokens, −78.5% on the whole session.

What caching does not change, Anthropic states outright: "Cached prompt prefixes still occupy the context window: prompt caching changes what you pay for those tokens, not whether they count." The window fills at the same rate, and rate limits apply to total volume.

A second effect is specific to recent models: reasoning blocks from previous turns are kept by default and count towards the window as input tokens. A reasoning token is therefore billed once at the output rate when it is produced, then at the input rate on every subsequent turn.

And one underlying question is still open. No provider documents whether it keeps computed state in memory or recomputes it. Nobody has measured the energy of a cache hit against a miss. The fact that Google bills hourly storage for its explicit cache, up to $4.50 per million tokens per hour, does suggest a physical resource is being tied up. Writing that caching divides the footprint by ten would be extrapolating from price to energy.

What an agentic task actually consumes

Public figures from the Terminal-Bench 4.0 leaderboard, accessed 9 September 2026. These are aggregate totals across the whole benchmark, all attempts combined.

One piece of data is missing and it blocks everything: neither the number of tasks in the benchmark nor the number of attempts per task is documented, on the leaderboard or in the project repository. No per-task cost can be derived. On the SWE-bench side, the official leaderboards publish neither cost nor tokens.

What can be said without that data: running an agentic benchmark once costs 1.5 to 17 billion tokens and $1,700 to $7,300.

Now look at the last two columns. The ratio between the hungriest and the leanest in tokens is 11.5, the ratio of costs only 4.3. Price per token varies between models and therefore hides part of the real consumption gap. A client steering on cost alone is using the wrong metric.

The harness matters more than the model you pick

Ismael Velasco's formulation, at an OBVIA conference: "If you have a really efficient model in a really inefficient harness, the impacts will be way worse than a really inefficient model in a really efficient harness." MISSING DATA on the reference: neither the date of the talk nor a public recording could be found. It is a formulation to reuse, not a result to cite.

That last figure has a direct methodological consequence. The only published measurement of the energy gain from a cache read was made on a 4,096 token prefix. Transposing that result to a regime thirty times longer would be a genuine change of scale, not a detail. Hence a conservative retained value.

Two more overheads appear on no invoice. The system prompt, resent on every turn, whose size no vendor publishes. And provider-side monitoring of agentic models, which uses inference compute never billed to the client. MISSING DATA on both: a figure of around 20% of inference compute circulates for monitoring at OpenAI, with no traceable publication behind it. A methodology still has to decide whether its per-token factor includes that share or documents it separately, and write down which it chose.

What this changes for an audit

The relevant unit becomes the task completed, rather than the request or the token. The SCI for AI standard has already put this on record, setting workflow execution as the functional unit for agentic AI.

Ask for three things: the number of turns per session, the median context length, and the token split between fresh input and cache writes on one side, cache reads and output on the other. Those three numbers are enough to rebuild most of the footprint, and they all sit in the usage exports.

The lever is rarely where teams look for it. On a real base of agentic sessions, the genuinely compressible share of the footprint, fresh input and cache writes together, is 28.8%. A perfect prompt compressor would therefore never get below 71% of the total. What moves the needle acts on output and on the choice of model.

FAQ

Does an AI agent really use 1,000 times more than a simple request?

That is the upper order of magnitude the International Energy Agency uses for an agentic task compared with a simple text request. The Green IT 2025 study, cited with caution by Arcep, puts forward a factor of 60 between an agentic service and a classic generative AI service. The two measure different things, and the literature on this is still young.

Why does the cost of a conversation with an agent rise so fast?

Because the API is stateless: on every turn, the client resends the entire history. Turn ten therefore bills ten times the volume of one turn, and the running total grows as the square of the number of turns. Doubling the number of turns quadruples the input tokens billed, and a 50-turn conversation costs 25 times what a linear total would.

Is prompt caching enough to solve the problem?

It largely solves the price. All three major providers bill a cache read at 10% of a fresh input token, and measurements across more than 500 agent sessions record cost reductions of 41 to 80% depending on the model. It solves neither the occupation of the context window nor the rate limits, and nobody has measured its exact energy gain.

How many tokens does an automated development task consume?

The Terminal-Bench 4.0 leaderboard publishes totals ranging from 1.5 to 17.2 billion tokens and $1,700 to $7,300 to run the whole benchmark once. Since the number of tasks and attempts is not documented, no per-task cost can be derived from it. Worth noting: the token gap between models reaches a factor of 11.5 while the cost gap is only 4.3.

What data should I ask for to quantify an agentic footprint?

Three things, all present in provider usage exports: the number of turns per session, the median context length, and the token split between fresh input and cache writes on one side, cache reads and output on the other. A global token volume without that split leads to gaps of a factor of 5 to 10 depending on the rate applied to context re-reads.