After the GPU: memory, power and the photonics bottleneck
Key points
Key takeaways from Sourcery interview with imec VP AI Steven Latré and Strike Capital investor Adam Chambers, recorded at the Strike Capital Summit:
Moore's Law is flattening. Transistor features are now around one nanometer deep, roughly one hundred thousand times thinner than a human hair, so imec sees scaling approaching the end of physics, forcing hardware innovation to become far more inventive than in the past two decades.
The post-GPU bottlenecks. GPUs handle training well, but Chambers maps the inference layer's constraints to power and energy, memory, cooling and data transfer, citing bandwidth and latency of getting memory to the GPU as the key build area, plus liquid cooling moving closer to the die over time.
A memory oligopoly under strain. Just three makers, Micron, SK Hynix and Samsung, control about ninety-five percent of memory production, hyperscalers absorbing their inventory helped memory prices rise about seven hundred percent in two thousand twenty-six alone.
Manufacturing, not headcount, is the bottleneck. Demand for chips is effectively endless, says Chambers, every startup can bring a unique design, so the real constraint is manufacturing the right components in the right regions, citing TSMC's foundries as crucial,and more diversified hardware beyond GPUs will be needed as algorithms diversify.
Photons over copper. Photonics attacks energy, cooling and data-transfer bottlenecks at once, Steven Latré expects optics to migrate from pluggable transceivers between clusters toward co-packaged optics right next to the GPU within the next five years,calling photonics the enabling technology for chips of the future.
Scaling the biggest models is ending. Latest open-source models already pack more parameters than the human brain,yet the chips behind them remain roughly one million times less energy-efficient than biological neurons,so companies are abandoning the pure scaling hypothesis and attacking memory bottlenecks instead.
Wanted: European science, US business sense. Chambers' hottest take is to recruit or invest in people with European academic and technical training plus US commercial instincts,saying the best founders he has backed combine both mindsets.
The software revolution is still coming. Latré argues today's large language models are brute force,and in five to ten years we will laugh at them the way we laugh at dial-up internet,expecting new software evolutions tied much closer to hardware.
Read more: Sourcery on YouTube
