AI Inference Costs: Why Tokens Get Cheaper but Bills Rise
AI inference costs per token fall about 13× a year for a fixed capability, yet each task uses more tokens. Track cost per task, with a worked example, not price per token.
AI inference costs per token keep falling, yet company AI bills often rise, because each task now consumes far more tokens than it did two years ago. As of 4 October 2026, Epoch AI estimates that the cost of a fixed level of AI performance falls about 13× per year, while reasoning models and agent loops multiply the tokens behind every answer. The number to manage is cost per task, not price per token.
What are AI inference costs and how are they billed?
Inference means running a trained model to produce an output. Providers bill it in tokens, chunks of text of roughly four characters in English. A bill has up to four lines: fresh input tokens, cached input tokens (much cheaper), output tokens (the priciest per token) and reasoning tokens, the model's internal thinking, which Anthropic's cost documentation says are billed as output tokens. Unit price means dollars per million tokens (MTok). Cost per task means the full bill for one finished job, including every model call in a loop. Unit economics means the cost and revenue of one such unit.
Is the price per token really falling?
Yes, when you hold capability constant. In a September 2026 study, Epoch AI's Luke Emberson and David Roodman found that the cost of a given level of AI performance has fallen about 47% per quarter, or 13× per year, since 2023, across five benchmarks. In their example, OpenAI's o3 scored 75% on GPQA Diamond at roughly $0.30 per question in January 2025, and just under 18 months later GPT-5.6 Luna matched it for $0.0004. The authors caution that models may be trained for benchmarks and that almost nobody stays on the cheapest frontier model for every task.
Vendor price lists point the same way, with exceptions. List prices per million tokens, checked on 4 October 2026:
| Model | Status | Input $/MTok | Output $/MTok |
|---|---|---|---|
| GPT-4 (8K context) | Launch price, 14 March 2023 | 30 | 60 |
| o3 | Listed 4 October 2026 | 2 | 8 |
| Claude Opus 4.1 | Retired on the first-party API | 15 | 75 |
| Claude Opus 5.5 | Listed 4 October 2026 | 4 | 20 |
| Claude Haiku 3.5 | Retired on the first-party API | 0.80 | 4 |
| Claude Haiku 4.5 | Listed 4 October 2026 | 1 | 5 |
| Claude Fable 5.1 | Listed 4 October 2026 | 10 | 50 |
Sources: the GPT-4 launch post, OpenAI's pricing page and Anthropic's pricing page. From GPT-4 at launch to o3 today, input falls from $30 to $2 (15×) and output from $60 to $8 (7.5×). Inside one Anthropic tier, Opus 4.1 to Opus 5.5 cuts both prices 3.75×. The fall inside a named product line is therefore slower than the capability-adjusted 13×, and two rows go the other way. Haiku 3.5 to 4.5 raised list prices 25%, and Fable 5.1 lists at twice Opus 5 ($10/$50 against $5/$25). Anthropic's page adds that Claude 4.7 and later models use a newer tokenizer that produces approximately 30% more tokens for the same text, so an unchanged list price can still raise the cost per document.
Why do AI bills rise when the price per token falls?
A bill is price times tokens times tasks. Price is falling while the other two factors grow, for four reasons.
- Reasoning multiplies output tokens. Thinking is billed at the output rate, the highest line. Artificial Analysis, an independent benchmarking firm, publishes cost per task. On 1 September 2026 it reported that Claude Fable 5.1 kept Fable 5's $10/$50 list price but cost about 20% more per task, because it used about 1.7× the output tokens. On 9 September it reported that GPT-6 Astra, priced 2.5× above GPT-5.6 Sol, matched Fable 5.1 at about 40% of the cost per task, using 27,000 output tokens per task against 78,000. A higher price per token can give a lower price per task, and an unchanged price per token can give a higher one.
- Agent loops multiply calls. Anthropic reports that in its research system agents use about 4× more tokens than chat and multi-agent systems about 15×, because each step re-sends a growing context. Danube Labs' own usage export for the ten weeks from 27 July to 4 October 2026 (one Claude Code account on a Max 5x plan) shows 50.0 million output tokens against 21.08 billion cache-read tokens. Cache reads were 98.2% of all tokens, about 420 for every output token. It is one account and one tool, so it illustrates the shape of agent traffic and proves nothing general.
- Tokenizers and tiers move the unit. A new tokenizer changes how many tokens a document becomes, and a new top tier changes what the best model costs, so the price of the best available model can rise in the same year that the price of a fixed capability falls.
- Cheaper models invite bigger jobs. Teams that could not afford to run a model over every ticket, contract or log line now do.
Is this the Jevons paradox applied to tokens?
William Stanley Jevons observed in The Coal Question (1865) that more efficient steam engines raised total coal use, because cheaper work found more uses. For tokens, total spend rises when volume grows by more than the price falls. Write V for volume growth and P for the factor by which price falls, and spend grows when V is greater than P.
A plausibility check: Epoch's 13× per year compounds to 169× over two years, while Alphabet's June 2026 investor presentation says it processed 9.7 trillion tokens a month two years ago and 3.2 quadrillion today, about 330 times as many. The measures differ, since Epoch tracks the price of constant performance and Alphabet counts every surface, free consumer products included. So the check shows that V greater than P is plausible and does not estimate demand elasticity. Our read is that for agent-heavy workloads, volume is winning.
Which discounts actually reduce the bill?
| Lever | Documented effect (Anthropic pricing, 4 October 2026) | Best for |
|---|---|---|
| Prompt caching | Reads cost 0.1× the input price on most Claude models (0.05× on Opus 5.5, 0.025× on Fable 5.1). Writes cost 1.25× for 5 minutes or 2× for 1 hour. | Long stable prefixes, loops |
| Batch API | 50% off input and output, stacking with caching. | Work that can wait |
| Lower effort | Thinking is billed as output, and effort levels steer how much the model thinks. | Simple requests |
| Model routing | Haiku 4.5 at $1/$5 against Opus 5.5 at $4/$20 is a 4× gap. | Tasks a small model passes on your tests |
Our explainer on prompt caching covers the mechanics. The write-up on how we cut LLM costs by 99% without hurting quality combined routing, caching, prompt reduction and batching, checked against a 400-example evaluation set.
Subscription or metered pricing: who carries the usage risk?
A flat plan is a bundle priced on an expected average, so the vendor insures usage variance, which works while usage per user is bounded. Anthropic lists Pro at $20 per month and Max from $100, describes Max as “5x or 20x more usage per 5-hour session than Pro” and adds that “paid plans add weekly limits on top” (claude.com/pricing, 4 October 2026). The limits are the vendor's loss cap.
The gap between flat price and metered value can be large. Over the ten weeks above, ccusage valued the account's usage at $13,166 at API list prices, against about $230 for ten weeks of a plan listed from $100 per month, roughly 57 times. That is not what the account was billed, it hit the weekly limit in every one of those weeks, and it is a single account. It shows why vendors cap flat plans.
Agents remove the natural bound on usage, so flat pricing is retreating where cost scales with tokens. The New Stack reported in June 2026 that GitHub retired Copilot's fixed subscription for token-based billing and Cursor restructured its team tiers, with some subscribers reporting projected bills up tenfold. Anthropic's documentation puts the average Claude Code cost at about $13 per developer per active day, $150–250 per month, and below $30 per active day for 90% of users. Use flat seats for people and metered API keys for products, and read the provider's terms before using a subscription for anything beyond your own work. Our comparison of Claude subscription vs API for coding covers the trade-offs.
How do you calculate cost per task? A worked example
Cost per task adds up every call in the loop: fresh input × input price, cached input × cache-read price, and output (reasoning included) × output price. Cost per successful outcome divides the cost of an attempt, plus human review, by the success rate.
Assumptions, illustrative and not measured. A support-ticket agent makes 4 model calls per ticket. Each call sends 9,000 input tokens (a 6,000-token stable prefix of instructions and policy read from a warm cache, plus 3,000 fresh) and produces 400 visible plus 600 reasoning tokens. The model is Claude Sonnet 5.5 at $2 input, $0.20 cache read and $10 output per MTok. Per ticket that is 12,000 fresh input, 24,000 cache-read and 4,000 output tokens. The arithmetic is 12,000 × $2 ÷ 1,000,000 = $0.024, plus 24,000 × $0.20 ÷ 1,000,000 = $0.0048, plus 4,000 × $10 ÷ 1,000,000 = $0.040, which totals $0.0688 per ticket, or $688 per 10,000 tickets.
| Scenario | Fresh input | Cache read | Output | Cost per ticket | Change |
|---|---|---|---|---|---|
| Base case | 12,000 | 24,000 | 4,000 | $0.069 | baseline |
| No caching | 36,000 | 0 | 4,000 | $0.112 | +63% |
| Reasoning tripled | 12,000 | 24,000 | 8,800 | $0.117 | +70% |
| Prices halved, reasoning tripled | 12,000 | 24,000 | 8,800 | $0.058 | −15% |
| Batch API, base tokens | 12,000 | 24,000 | 4,000 | $0.034 | −50% |
Caching alone removes 39% of the bill. Halving every price while tripling reasoning leaves the bill only 15% lower, because output tokens rose 2.2× while input stayed flat. If a person needs 6 minutes per ticket at a loaded cost of €16 per hour (our assumption), a ticket costs €1.60 in labour, and the model call is about 4% of that, treating $1 as €1 for scale. That flatters the model, because review time and wrong answers are the larger costs, which our guide to AI agent ROI for business shows how to count.
How should a company budget for AI tokens?
- Log tokens per task by type (fresh, cached, output, reasoning).
- Cap output tokens and set an effort level per workflow, and test quality on an evaluation set before changing either.
- Put the long stable part of the prompt first so cache hits stay high, and route easy tasks to a small model.
- Price each task at 0.5× and 2× your measured tokens and re-price quarterly, because vendors change prices and tokenizers. Anthropic's pricing page records a Sonnet 5 price rise planned for 1 September 2026 that did not take place.
How Danube Labs works with AI costs
Danube Labs builds My Claude Code, a local gateway that routes coding agents across 57 model providers and keeps a request log with per-request cost, which is the data this article says you need. For client projects, an AI feasibility study starts with an evaluation set and a cost-per-task estimate before anything is built. See our AI systems service or ask for a free intro call.
Frequently asked questions
- Why is AI getting cheaper per token?
- Cheaper hardware, lower-precision (quantised) inference, software optimisation, smaller models that match older large ones, and competition between vendors all push prices down. Epoch AI measures about a 13× fall per year since 2023 in the cost of a fixed level of AI performance, though it warns that almost nobody stays on the cheapest model for every task.
- Why does my AI bill go up when the price per token falls?
- A bill is price times tokens times tasks. Reasoning models bill their thinking as output tokens, agents call the model many times per task, and cheaper prices encourage bigger jobs. Anthropic reports that agents use about 4× more tokens than chat and multi-agent systems about 15×, which can cancel a large price cut.
- How do I calculate the cost of an AI task?
- Add every model call in the task: fresh input tokens times the input price, cached tokens times the cache-read price, and output plus reasoning tokens times the output price. Then divide by the success rate and add human review time. In our worked example, a four-call support ticket on Claude Sonnet 5.5 costs about $0.069 under stated assumptions.
- Is a subscription cheaper than the API?
- For heavy individual use, usually yes. In one Claude Code account over ten weeks, usage valued at $13,166 at API list prices ran under a plan listed from $100 per month, though weekly limits applied. A product backend needs metered API access with spend controls, and you must follow the provider's terms on what a subscription may be used for.
- What is the Jevons paradox in AI?
- It is the observation, from William Stanley Jevons in 1865, that efficiency gains can raise total consumption. Applied to AI, total token spend rises when volume grows faster than the price per token falls. Alphabet reports about 330 times more tokens a month than two years ago, against Epoch's roughly 169 times price fall over two years.
