Why token optimization is a gift to the hyperscalers
An analysis by Rihard Jarc (UncoverAlpha) argues that the enterprise shift from "token maxing" to "token optimization" is structurally bullish for the three cloud hyperscalers — Microsoft, Amazon and Alphabet | Google — even as it squeezes the AI model labs.
The tollbooth thesis. The AI labs are framed as carmakers and the hyperscalers as tollbooth owners. As companies trade down from frontier models to cheaper open-weight models to control costs, the per-token "model-provider margin" collapses — but total token volume explodes, and almost every token still runs on a hyperscaler's GPUs, data centre and managed API. The infrastructure toll is charged regardless of which model produced the token.
From maxing to optimization. After ~18 months of pointing every workload at the single best model, enterprises are blowing through AI budgets in a quarter. Sam Altman noted the "spent my entire 2026 budget in Q1" complaint went from never raised to a huge issue. The behavioural fix is routing: a small, cheap, often open-weight model handles ~80% of requests, with only the hardest escalating to the frontier.
Jevons' paradox on the cloud P&L. Cheaper inference accelerates consumption rather than reducing it — agents run in loops, read whole codebases and re-run multiple times. Inference cost for GPT-3.5-level performance fell more than 280-fold in two years (Stanford HAI), and a16z pegs the decline at ~10x per year for fixed capability, yet total tokens processed keep growing several-fold annually.
Disclosed token growth. Microsoft said it processed over 100 trillion tokens in a single quarter in 2025 (up 5x year-over-year, with a record 50 trillion in one month), and by fiscal Q3 2026 had over 300 customers on track to process more than a trillion tokens each on Foundry, accelerating 30% quarter-over-quarter. Google went from 480 trillion tokens per month in May 2025 to 1.3 quadrillion by October, with first-party models alone processing more than 16 billion tokens per minute via direct API in Q1 2026 (up 60% in a quarter).
Where the margin sits. Compression lands on the model layer, not infrastructure. AWS has historically run a roughly 35–38% operating margin; Google Cloud posted operating margins above 33% in Q1 2026 and rising. That margin is indifferent to whether the token came from a $50/million frontier model or a $1/million open-weight one — leading Jarc to argue the hyperscalers should re-rate higher.
Read more: UncoverAlpha · X (@RihardJarc)