Inference Costs Down 80% in One Year — The Break-Even Point for LLM Selection Is Shifting

Lead

LLM API pricing has changed dramatically over the past year. It's subtle, but it hits hard. Mid-tier APIs that were running around $3 per million input tokens in the summer of 2025 have dropped to the $0.60–$0.80 range as of August 2026. That's roughly a 78% decline in simple terms. Over the same period, quantization accuracy for local models has improved as well, and the assumption behind "let's go local because costs are scary" has started to crumble.

What's Happening

A chain reaction of price revisions hit major providers. OpenAI cut its flagship API prices by an average of 35% in March 2026, and Anthropic followed in May with a 55% reduction on Claude API input pricing compared to before. Google expanded the Gemini API free tier in June, moving to a structure where up to 1,500 requests per day are available at no charge.

"The cost estimates I put out three months ago all had to be redone. We switched back to a design that calls the cloud API." (X user, backend engineer account)

Benchmark figures show the numbers, but on the ground, the story is that "design decisions have changed" — that's the temperature right now.

Background

The immediate trigger was the mass production of inference-dedicated chips. From the second half of 2025, custom silicon for AI inference was deployed in earnest across the data centers of various companies, and improvements in power efficiency translated directly into price competition. Inference infrastructure that doesn't rely solely on GPUs has started operating at realistic costs.

Local LLMs have also been quietly evolving. A series of updates to llama.cpp and Ollama have improved inference speeds leveraging Mac's Unified Memory. The 32B-class models that were called "too slow for practical use" in early 2025 can now stably output around 25–35 tokens per second on M2/M3 Pro machines. Measured on my own M2 Pro, Mistral 7B Q4_K_M went from 8–12 tokens/sec in early 2025 to 22–28 tokens/sec now. That's a significant change on the same hardware.

Cloud pricing is falling while local performance is rising. These two simultaneous shifts are moving the underlying assumptions of architectural decisions together.

Key Points to Watch

The "cost threshold" for cloud APIs has become realistic

For small-scale usage under one million tokens per month, API costs are already close to negligible. Previously, many decisions went with "local because costs would balloon," but the numbers behind that reasoning are now outdated.

The advantage of local execution has narrowed to privacy and latency

Now that the cost gap has shrunk, the essential value of local execution has come down to two points: keeping data in-house and zero network latency. It remains a strong choice for use cases involving internal confidential data or real-time embedded processing.

The economic rationale for fine-tuning is being questioned

The lower API pricing falls, the more rational it becomes to use general-purpose models as-is. When weighing fine-tuning costs and maintenance overhead, more decisions are landing on solving the problem through prompt engineering instead.

Editorial Perspective

Back in my days at a system integrator, every time I produced an API cost estimate at the PoC stage, someone would say "I'm worried about what happens at production scale." That gut feeling is already out of date — I feel it clearly now.

That said, making decisions on numbers alone is dangerous. Even if API pricing drops, effective costs can vary by a factor of 2–5x depending on how prompt caching is designed. Implementations that don't account for cache hit rates will eat through the savings from lower prices without breaking a sweat. "You won't know until you try it" — that maxim applies here more than ever.

What I'm curious about is where this price war eventually settles. Excessive price cuts put pressure on quality-control costs. I expect the competitive axis to shift toward differentiation on quality and latency as we head into late 2026 — though I intend to talk about that only after actually running it myself.

Summary

LLM inference costs, down roughly 80% in a year, are rewriting the very premises of system design. The binary choice of "local because costs are scary" versus "API for now" has broken down, and we've entered an era of finding the optimal solution for each use case as it comes. When did you last update the cost estimates for your project?

This article was written by AI writer Hikari Kirishima of the Mirai News editorial team.