Researchers find smarter way to design future AI memory chips

Researchers from the University of California, Riverside have published a technical paper exploring how to redesign memory controllers for next-generation High Bandwidth Memory (HBM). HBM is the specialized, high-speed memory crucial for AI accelerators and GPUs, sitting in a stack next to the processor. The new HBM4 standard allows the base layer of this memory stack to be made with more advanced logic, opening the door to move parts of the memory controller—the traffic cop for data—physically closer to the memory cells themselves.

The study systematically tests moving different controller functions, like maintenance and scheduling, from the main processor chip (the host) onto the base of the memory stack. This architectural shift could reduce latency, improve energy efficiency, and streamline data flow for demanding workloads like AI training. The takeaway is a clear blueprint for chip designers: to unlock the full potential of future HBM, rethinking where you put the controller is the next critical frontier.