Token Minimizing: Why AI Keeps Getting Cheaper – and Your AI Bill Still Rises

AI gets cheaper per unit every year, and yet AI costs are rising in many companies. The reason: modern AI, and AI agents in particular, consumes many times more tokens. Keeping costs under control therefore does not take a smaller AI budget, but token minimizing: as little usage as each result needs.

The paradox: cheaper tokens, bigger bills

A token is the unit in which AI models process and bill text. According to Gartner, one token is roughly four characters. Every question, every attached document and every answer counts.

The price per token has collapsed in a short time. According to Stanford University's AI Index 2025, the cost of a model performing at GPT-3.5 level fell more than 280-fold between November 2022 and October 2024. Gartner expects providers to run a large language model in 2030 at over 90 per cent less than in 2025.

At the same time, Gartner warns that these savings will not be fully passed on to customers. And AI agents need 5 to 30 times more tokens per task than a standard chatbot. Because usage grows faster than prices fall, total costs go up.

In August, Gartner went further: providers' compute costs per agent workflow are expected to rise more than fivefold by 2028. Gartner calls this the «Inference Paradox»: each unit gets cheaper, AI as a whole gets more expensive.

Why this affects your company

As long as your team uses AI on fixed per-user licences, you notice little of this. But some offerings already bill by usage, such as the new Copilot Business licences. And as soon as you run AI agents or your own AI applications, you almost always pay per token.

At that point a licensing question becomes an operating question. A poorly built agent that rereads the entire manual at every step costs many times more than a well-built one, for the same result. You do not see that difference in the demo; you see it on the monthly invoice.

Token minimizing, not token maximizing

Token minimizing does not mean using less AI. It means spending no more tokens than each result needs. High usage is not a sign of progress; often it is a sign of waste.

Gartner recommends the same principle: routine tasks belong on smaller, efficient models, while the expensive frontier models are reserved for complex work. The main levers:

• The right model for each task: Not every question needs the strongest model. On our own website, a small, inexpensive model answers questions in the chat. We use the large model only for the AI potential analysis, where depth matters.
• Cache what repeats: Rules and knowledge that are identical in every request can be cached with the provider. At Anthropic, reusing them costs at most a tenth of the normal price (documentation). Our website assistant does this with every question.
• Batch what is not urgent: Analyses that do not need to finish immediately run as a batch. Anthropic halves the cost for this, and most batches finish in under an hour (documentation).
• Send only what is needed: Instead of packing the whole archive into every request, a good application first finds the relevant passages and sends only those. Short, clearly bounded answers save tokens too.
• Set limits for agents: Cap the steps, run time and budget of each agent, and monitor usage. Our post on user accounts for AI agents shows how to keep AI agents in Microsoft 365 under control.

Where AI costs are heading

Our view: prices per token will keep falling, models will become more capable, and the tasks companies hand to AI will grow. On balance, AI will not become cheaper, but more valuable, provided someone manages the costs.

There will be no single model for everything. In its own words, Gartner sees no reliable, economical one-size-fits-all model on the horizon. Companies that take AI seriously will combine several models and choose per task.

For SMEs this means AI costs become an ongoing management task, much like cloud costs. Not set once and forgotten, but reviewed regularly: where does usage come from, and is it worth it?

What to do now

• Make usage visible: Find out which of your AI tools bill by usage, and set up budgets and alerts.
• Choose models by task: For each use case, check whether a smaller model is enough.
• Limit agents: Set caps on steps and costs before an agent goes live.
• Use repetition: Have your AI applications checked for caching and batch processing.
• Find out where you stand: Our free AI Check shows in a few minutes where your company stands with AI.