How do you cut the carbon footprint of AI? Ten levers ranked by impact_

The ranking below is ordered by measured gain, and it is the reverse of the one teams apply by instinct. They go for the prompt first, which comes tenth and returns a few per cent. Model choice, which outweighs everything else, comes last when it comes at all.

A client polishing prompts on the heaviest model, in the most carbon-heavy region, is working the weakest lever while two factor-of-10 levers sit untouched.

The table, with who controls what

The last column decides what you can act on in a given engagement. The numbers behind each lever come from the chapter on the formula.

Lever 1: do not use an LLM

The most frugal compute is the compute that never happens, and a good share of deployed use cases have a deterministic substitute that is cheaper, more reliable and auditable.

A number to place the rungs, measured by Luccioni, Jernite and Strubell (Power Hungry Processing, FAccT 2024) on eight A100 accelerators with no request batching, in kWh per thousand inferences: classifying text 0.002, extractive question answering 0.003, generating text 0.047, generating an image 2.907. Generating rather than classifying costs about 25 times more.

The first rung is a question, not a technique: does the need really call for a feature built with AI, or something else, or nothing at all. The first deliverable of a sufficiency review is the list of use cases that never needed a frontier model.

Lever 2: do not oversize the model

The gap between commercial models exceeds 65 from the most frugal to the greediest, across the 30 models in Jegham et al., How Hungry is AI? (v6, November 2025). Those values are estimates, not measurements: API latency plus inferred hardware attribution. Quote them with that limit attached.

On the claude-carbon agentic session base (January to August 2026, 17,402 million tokens), two models from the top of the range account for 99% of the carbon, while a mid-range model weighs 9 kg out of 1,052.

Providers do it for themselves. At Google, the strategy is to route consumer requests to smaller models as soon as a new generation matches the previous one's performance. For product integrations, it is never the generalist models running but specialised versions.

Three limits to state, two of them awkward. Economic lock-in: investment in data centres sized for large models creates pressure to earn that money back. The training legacy: many small models are distilled from large ones, which reduces their upstream benefit. And a stacking risk: if small models get added to large ones rather than replacing them, the gain evaporates. Ask for proof of replacement.

Lever 3: switch off unnecessary reasoning

Reasoning mode costs 92% more energy on average, and up to 849% more on a code generation task, for a performance gain that varies by task. On tasks where it adds nothing, it is pure overhead.

And even where it does add something, half of it is surplus: a published method brings the token volume from 2,408 down to 1,331 on a maths set, −44.8%, for an accuracy of 92.8% against 93.0%.

This is an API parameter, not a project. Measure at the client before recommending a level: the same task set at the different settings, reading the output token counter. One afternoon, and you have the only number that matters, the one for their tasks.

Lever 4: choose the region

Same energy, same model, same code. Only the electricity mix changes.

Two different quantities circulate under this one lever, and confusing them wrecks the argument.

The intensity ratio between grids is 14 at Ember (production, LCA, 2025) and 21 at Electricity Maps (traced consumption, LCA, 2025) between France and Poland. That is a factor per kilowatt-hour, and it survives the choice of database, which is rare.

The annual inventory ratio between two sites is 126: Meta publishes its location-based scope 2 emissions per data centre for 2024, and the gap between Luleå in Sweden (5,298 tCO2e) and Altoona in Iowa (666,434 tCO2e) reaches that factor, at identical hardware and practices. That number aggregates grid intensity and the volume of electricity consumed at each site. It illustrates the lever, it does not measure it.

Three limits, rarely stated.

This lever has a scale ceiling. Lin et al. (2023) note that hyperscale data centres exceed 15% of the load on some regional grids; at that size, moving the load towards decarbonised electricity disturbs the very grid you are trying to spare. The lever works for one player alone, not for everyone at once.

On an already clean grid, this lever returns far less. The carbon composite moves by a factor of 4.1 on the total, not 20, and at zero intensity the hardware remains.

And on a proprietary API, you do not choose the region. The lever is real for self-hosting or a cloud with region selection, unavailable for most usage. Saying so avoids selling a lever nobody can pull.

Lever 5: cut the context

The cost of context is superlinear, and providers bill it with a 2× multiplier on input beyond their threshold. Routing each question to a targeted search rather than to the full context cuts token volume by 61.6% on Gemini 1.5 Pro, in Li et al. (Google DeepMind, EMNLP 2024), across nine datasets.

The gain is in tokens, not joules, and the retrieval pipeline is not free. And on agentic use the ceiling is low: the genuinely compressible share is 28.8% of the footprint, so a perfect compressor does not get below 71% of the total.

Lever 6: prompt cache

A gain with no quality loss at all, which makes it unusual: the result is identical, it is the compute that disappears. Announcements go as far as 90% off the cost of the tokens concerned, and the measurement by Lumer et al. (Don't Break the Cache, arXiv:2601.06007, January 2026) across more than 500 agent sessions from the DeepResearch Bench gives −79.6% on the full GPT-5.2 bill.

The boundary trap is here. The cache removes prefill, that is, the cheapest part per token, about 35 times cheaper than an output token at batch size one. A 90% discount on the price of input tokens is not 90% of energy saved.

Nobody has measured the energy of a cache hit against a miss. Defensible position: the cache reduces the compute actually performed without degrading the output, its energy magnitude is unknown and lower than the commercial discount.

The cache is a prefix: anything that changes the start of the context mid-session invalidates it for everything after, so add nothing along the way. The duration counter starts at the beginning of the request, not at the end of the answer. And the pricing models differ, one charging for writes, another for hourly storage.

Speculative decoding, same family, same trap

A small "draft" model proposes several tokens ahead, the large model checks them in a single pass and keeps the longest prefix consistent with what it would have produced itself.

Same property as the cache: at zero temperature the output is strictly identical. This is not an approximation, it is a rearrangement of the compute. Published speed-ups run from 2 to 3.6 times, all measured in latency: 2 to 3 times in Leviathan, Kalman and Matias (arXiv:2211.17192, ICML 2023) on T5-XXL at batch size one, up to 3.6 times for Medusa (arXiv:2401.10774).

The trap is nastier than the cache one, because it can reverse the gain. The only two published energy measurements give −12% to −27% with a well-matched draft (Nik, Riegler and Halvorsen, SimulaMet, arXiv:2502.11723, peer-reviewed version Scientific Reports 2026; Qwen2.5-7B on an A100, n-gram draft, greedy decoding), and 30% more consumption despite a 1.19 times speed-up when the draft is badly matched (Dutta et al., arXiv:2602.09113, Findings of EACL 2026; Vicuna-7B on an A5000, CodeCarbon measurement). Both run at batch size one, the most favourable regime.

Speed and energy do not necessarily move together, then. Like batching, this lever belongs to the provider.

Lever 7: cut output tokens

An output token costs about 35 times an input token, measured at batch size one. On an agentic session, output is 0.73% of the volume and 48.5% of the carbon.

Three ways to act: constrain the answer length and switch off reasoning, but also pick a model that answers briefly. That last point is poorly known: on the same task, models produce answers of different lengths, and that verbosity appears in no comparison grid anywhere.

One trap worth knowing. Optimising joules per token encourages generating more, since the fixed costs get amortised: Vellaisamy et al. (Carnegie Mellon) measure on Qwen3-8B and MATH-500, on an H200, 7.46 J per token at 10 output tokens against 0.72 J at 512, while the energy of the request is multiplied by 5. Optimise joules per task solved.

Lever 8: batching and sharing

Decoding is memory-bound, so handling several requests in the same pass amortises reading the weights. Two distinct quantities, to be given separately.

The gain runs from 17 to 25 between batch 1 and batch 32, depending on the attention architecture (Ma et al., The Illusion of Power Capping in LLM Decode, May 2026; one H200, models of around 4 billion parameters, decode phase isolated).

The total observable gap between an unbatched measurement and a production measurement reaches around 35 on models of the same class. That one compares two complete protocols, not two batch sizes: 1.72 Wh per request on Llama-3-70B at AI Energy Score against 0.0486 Wh on Llama 3.3 70B at ML.Energy v3.0. The weights are not exactly the same, the order of magnitude holds.

Hence the counter-intuitive conclusion you have to be willing to state: self-hosting a model at low volume is worse energetically than calling a shared API. The numbers: generating at batch size one, the GPU draws 137 to 300 W out of 700, energy per token is 17 to 25 times that of batch 32, and on BLOOM, measured over 18 days on 16 A100s with no batching, roughly 75% of the energy goes simply to keeping the model in memory (Luccioni, Viguier and Ligozat, JMLR 2023).

Two caveats, or you are selling cloud. First, separate this physical argument from the hyperscalers' marketing argument, which compares their fleet with a deliberately inefficient installation. And at high volume with steady load, self-hosting becomes defensible again, with levers the API forbids. The threshold can be calculated.

Lever 9: quantisation and distillation

Reducing numerical precision gains 39% on average according to the measurement by PEReN for Arcep, the French telecoms and digital regulator (May 2026, 22 open models on the Jean Zay supercomputer, GPU only). And distillation gains 83% to 90% of inference energy for 13 to 15 ROUGE-L points lost, on CodeT5+ and an RTX 4090 (arXiv:2608.17515).

The result that stops you recommending it blind: at small batch, between 8 and 16 requests, reduced precision consumes more than standard precision in 7 measured cases out of 7, with a median of 30% more (ML.Energy v3.0, observation 7, H100 and B200). The gain only shows up at large batch. Low precision only pays in compute-bound regimes; in a memory-bound regime it pays nothing.

On quality, do not repeat the "negligible degradation" of the method papers. The most honest source is Meta's, which writes that standard benchmarks do not properly reflect the effects of quantisation, and that without guardrails the model occasionally produces corrupted answers while the scores stay good.

Lever 10: cut the retries

A failed answer is energy spent for nothing, plus a second request. In a conversation, the retry costs more than the initial request, since the whole history goes back up the wire.

Prompt engineering is energy saved too, in the literal sense: a prompt that gets the right result first time removes entire requests. The Arcep report of May 2026 cites a piece of green prompting work measuring 32% to 48% less inference energy at equal or better quality. The report does not name that work, so the number travels on a single relay.

The moves that work. Frame it once rather than ten round trips: a longer but precise prompt costs less than a short prompt that forces iteration, since input is 35 times cheaper than output at batch size one. Write every prompt as if you had one shot, and send text rather than a scanned document. Last point, do not fill the window to the brim: quality degrades well before the technical limit, so retries go up. MISSING DATA on the threshold: no published measurement links fill rate to error rate. It would take running the same evaluation at several fill rates, with the model and task held constant.

The limit to state, or this lever becomes manipulation: its real value is educational, it makes the subject tangible for a user who chooses neither their model nor their region. Its risk is shifting responsibility onto the end user while the factor-of-10 decisions get taken elsewhere. Putting it tenth is not an accident, and you should say so out loud when you present it.

The fake lever you will be offered

Lower the temperature to consume less. Someone will suggest it, it is wrong, and knowing that saves you a meeting.

Temperature is a division applied after all the compute, on a vector the size of the vocabulary. It adds no layer, no pass, no weight read. Structurally, it cannot matter.

Measurement confirms it: Nik, Riegler and Halvorsen (SimulaMet, arXiv:2502.11723, peer-reviewed version Scientific Reports 2026) sweep temperature from 0.1 to 1.2 on Qwen2.5-7B, two A100-PCIE-40GB, batch size one, and find a spread of −0.2% to +2.6% against greedy decoding depending on the task. Noise. On the same rows, switching to the DoLa decoding strategy costs 51.5% more.

Two indirect channels do exist, and deserve naming so you do not get caught out. A high temperature can make the model write for longer, and the cost is then in the token count, not in the setting. It also degrades the acceptance rate of speculative decoding: EAGLE-2 (arXiv:2406.16858) measures a speed-up loss of 5.4% to 21.6% going from zero temperature to 1, on the badly matched pairs only. Nobody has measured the chain from there to energy.

Nobody has measured average output length as a function of temperature with the model and prompt held fixed. MISSING DATA. The experiment takes two days for anyone with a GPU: output length and end-of-sequence triggering at several temperatures, on a fixed set of prompts.

What this list does not settle

Efficiency per process tells you nothing about the consumption of the system. Google divided its energy per request by 33 while its electricity consumption rose 38% and its emissions 18%, in the same year. The ten levers above are real and quantified, and all of them have been pulled at scale by the players who master them best.

Left to voluntary action, one party's frugality frees up capacity for another. Only a constraint on the total, a cap or a binding budget, beats a rebound effect.

The line to hold: individual levers exist and are quantified, but they are not enough. Selling the first half without saying the second is professional malpractice.

To apply this ranking to real usage, the calculator quantifies a mix of models and the methodology publishes the parameters used.

FAQ

What is the most effective lever for reducing the footprint of AI?

Not using a language model when a deterministic substitute is enough, since the compute then never happens. Model choice comes next, with a gap of more than 65 between the most frugal and the greediest commercial model, estimated by Jegham et al. across 30 commercial models. Prompt optimisation comes tenth and returns a few per cent.

Does optimising your prompts really reduce the carbon footprint?

Marginally, and far less than most other levers. The Arcep report of May 2026 cites a piece of green prompting work measuring 32% to 48% less energy at equal quality on the part concerned, without naming it, but input is only a quarter of the energy on chat and less still on agentic use. Its real value is educational: it makes the subject tangible for a user who controls nothing else.

Should you self-host a model to reduce its footprint?

Not at low volume, where it is worse energetically than a shared API. Generating one request at a time, an accelerator draws only 137 to 300 W out of a 700 budget and more than 88% of its compute units sit idle, while roughly 75% of the energy goes to keeping the model loaded. At high volume with steady load, the trade-off flips.

Does switching off reasoning mode degrade quality?

On tasks that do not warrant it, no: the French regulator measures an average overhead of 92% energy for a performance gain that varies by task. And even where it does help, a published method removes 44.8% of the reasoning tokens for an accuracy loss of 0.2 points. The setting should rest on an evaluation, not on intuition.

Are efficiency gains enough to bring down the total footprint of AI?

No, and the best counter-example comes from a provider. Google divided the energy of its median request by 33 between May 2024 and May 2025, while the electricity consumption of its data centres rose 38% and its total emissions 18%. Only a constraint on total volume, as a cap or a budget, offsets a rebound effect.