Former Oculus CTO Carmack floats NAND flash to cut AI-accelerator memory cost
John CarmackDealroom has a profile for this one. Try Dealroom → — id Software co-founder, former Oculus CTO and founder of AGI startup Keen TechnologiesDealroom has a profile for this one. Try Dealroom → — argued in a thread on 6 July 2026 that AI inference could run on far cheaper memory than today's HBM.
Core idea: model inference has a deterministic memory access pattern, unlike game rendering, so weights do not truly need random-access memory. Cold-start latency in the millisecond range is tolerable provided sequential bandwidth is high.
The play: NAND flash is over 100x cheaper per gigabyte than HBM. Give a flash controller a 1024-bit, HBM-bandwidth interface and stream 16KB+ pages into accelerator scratchpad — either via a specialised pipelined pin protocol, or by emulating RAM (easier, but sequential reads only; random reads fall off a 1000x+ cliff).
Trade-offs: a split flash-plus-HBM design is suboptimal versus uniform memory but far cheaper, and lets accelerators run much larger models. The case is strong for inference and weaker for training, where flash wears out from writes and cheap high-latency DRAM may fit better.
Who it touches: the thesis lands on the memory incumbents — SK HynixDealroom has a profile for this one. Try Dealroom →, whose "H3" flash-for-AI concept replies noted is close to Carmack's pitch, plus Micron TechnologyDealroom has a profile for this one. Try Dealroom → and Samsung ElectronicsDealroom has a profile for this one. Try Dealroom →.
Read more: John Carmack on X
