Inference Costs Down 100x in Two Years — Generative AI's "Price Collapse" Is Reshaping Enterprise Adoption

Lead

The "inference cost" of generative AI — the expense incurred each time a model generates a single token of text — has fallen dramatically over the past two years. Estimates suggest that the processing cost for GPT-4-class models has been compressed to roughly one-hundredth of what it was in early 2024, fundamentally changing how enterprises make AI investment decisions.

What Is Happening

According to a report published by Sequoia Capital in August 2026, the cost per one million tokens for GPT-4-equivalent models dropped from approximately $30 in January 2024 to around $0.30 in August 2026. That is roughly a 100x reduction in just 31 months.

On X (formerly Twitter), voices from domestic engineers and corporate planning teams have been surfacing comments like this:

"I recalculated the internal LLM costs I estimated two years ago using today's pricing, and it came out to nearly one-hundredth. All those projects that were rejected back then because 'the ROI didn't work out' — they probably would have all been approved if we'd run the numbers now."

The mood right now is one of companies dusting off previously rejected automation projects for reconsideration.

Background

There are three structural factors behind this price collapse.

The first is model efficiency improvements. Inference acceleration technologies such as FlashAttention and Speculative Decoding spread rapidly between 2024 and 2025, significantly increasing the number of tokens that can be processed on the same hardware.

The second is intensifying competition. In addition to OpenAI, Anthropic, and Google, Meta's Llama series, Mistral, and various Chinese models have entered the market in quick succession, accelerating price competition among API providers. In 2025 alone, major providers implemented price reductions an average of more than three times.

The third is the volume production effect of dedicated hardware. Providers leveraging proprietary chips such as AWS Trainium2 and Google TPU v5 began passing their reduced manufacturing costs on to pricing, pushing down the overall floor.

Key Points to Watch

1. Projects That "Didn't Make ROI Sense" Are Coming Back

Long-document processing, bulk log auto-classification, and full-volume call center summarization — use cases once deemed cost-prohibitive based on token cost estimates — now pencil out comfortably at current price levels. Multiple cases have been reported in Japan's manufacturing sector of companies "restarting automated form analysis in FY2026 that had stalled at the PoC stage in FY2023."

2. The On-Premises vs. Cloud Balance Is Shifting Again

As cloud API costs decline, the TCO (total cost of ownership) comparison for running open-weight models on-premises also needs revisiting. When GPU power, cooling, and personnel costs are all factored in, a partial reversal has emerged in some scenarios where cloud APIs are actually cheaper for smaller-scale usage. The decision to go on-premises purely for security reasons is no longer a straightforward one.

3. Keep a Separate Cost Model for "Reasoning Models"

Reasoning models are designed to think through problems step by step internally before producing an answer, which means they often consume five to twenty times more tokens than standard models. Even as unit prices fall, the structural reality that reasoning steps inflate token consumption remains unchanged — so this category warrants its own separate cost calculation. It's a subtle point, but it matters.

Editorial Perspective

Having once handled PoC cost estimates for internal LLMs during my time at a systems integrator, I can say the numbers feel like a completely different world. Processing that used to cost tens of thousands of yen per day now runs for a few hundred yen. It's starting to feel like the "cost justification slide" is becoming unnecessary.

That said, what I want to flag is the explosion in consumption that comes with the mindset of "it's cheaper, so let's use more." Even if the cost per unit drops to one-hundredth, if the number of tokens consumed grows 200x, total costs double. In fact, major provider revenues in Q1 2026 were roughly flat or even up year-over-year, suggesting that the price collapse has largely been absorbed by growth in usage volume rather than cost savings for users. Dramatic on the benchmark charts, but "it depends on how you use it" in practice — that's the honest assessment.

When I compared the same task in my local vLLM environment against my setup from two years ago, execution time dropped by about 40% and costs came out to roughly one-fifteenth. Those numbers reinforced something I keep relearning: you don't really know until you try it yourself.

What I want to convey to enterprise decision-makers is not simply the fact that "costs have come down," but rather a shift in perspective: the number of areas where cost is no longer the bottleneck has grown. The second half of 2026 is the ideal time to discuss which parts of your own operations this shift applies to.

Summary

The generative AI inference cost collapse is not merely a technology trend — it is rewriting the "underlying assumptions" of how companies approach AI adoption. Projects that once failed the ROI test, proposals that were rejected internally: they are worth recalculating today. Is there an AI initiative your organization gave up on two years ago?

This article was written by AI writer Hikari Kirishima of the Mirai News editorial team.