[ad_1]
John von Neumann’s authentic pc structure, the place logic and reminiscence are separate domains, has had a great run. However some firms are betting that it’s time for a change.
In recent times, the shift towards extra parallel processing and a large improve within the dimension of neural networks imply processors must entry extra knowledge from reminiscence extra shortly. And but “the efficiency hole between DRAM and processor is wider than ever,” says Joungho Kim, an knowledgeable in 3D reminiscence chips at Korea Superior Institute of Science and Expertise, in Daejeon, and an IEEE Fellow. The von Neumann structure has turn out to be the von Neumann bottleneck.
What if, as a substitute, no less than a number of the processing occurred within the reminiscence? Much less knowledge must transfer between chips, and also you’d save vitality, too. It’s not a brand new thought. However its second could lastly have arrived. Final 12 months, Samsung, the world’s largest maker of dynamic random-access reminiscence (DRAM), began rolling out processing-in-memory (PIM) tech. Its first PIM providing, unveiled in February 2021, built-in AI-focused compute cores inside its
Aquabolt-XL high-bandwidth reminiscence. HBM is the type of specialised DRAM that surrounds some prime AI accelerator chips. The brand new reminiscence is designed to behave as a “drop-in substitute” for atypical HBM chips, mentioned Nam Sung Kim, an IEEE Fellow, who was then senior vp of Samsung’s reminiscence enterprise unit.
Final August, Samsung revealed outcomes from checks in a accomplice’s system. When used with the
Xilinx Virtex Ultrascale + (Alveo) AI accelerator, the PIM tech delivered an almost 2.5-fold efficiency acquire and a 62 % minimize in vitality consumption for a speech-recognition neural web. Samsung has been offering samples of the know-how built-in into the present era of high-bandwidth DRAM, HBM2. It’s additionally creating PIM for the following era, HBM3, and for the low-power DRAM utilized in cellular gadgets. It expects to finish the usual for the latter with JEDEC within the first half of 2022.
There are lots of the way so as to add computational smarts to reminiscence chips. Samsung selected a design that’s quick and easy. HBM consists of a stack of DRAM chips linked vertically by interconnects referred to as through-silicon vias (TSVs). The stack of reminiscence chips sits atop a logic chip that acts because the interface to the processor.
The very best knowledge bandwidth within the stack lies inside every chip, adopted by the TSVs, and at last the connections to the processor. So Samsung selected to place the processing on the DRAM chips to make the most of the excessive bandwidth there. The compute items are designed to do the commonest neural-network calculation, referred to as multiply and accumulate, and little else.
Different designs have put the AI logic on the interface chip or used extra complicated processing cores.
Samsung’s two largest opponents,
SK hynix and Micron Expertise, aren’t fairly able to make the leap on PIM for HBM, although they’ve every made strikes towards different forms of processing-in-memory.
Icheon, South Korea–based mostly SK hynix, the No. 2 DRAM provider, is exploring PIM from a number of angles, says
Il Park, vp and head of memory-solution product improvement. For now it’s pursuing PIM in commonplace DRAM chips quite than HBM, which is likely to be easier for patrons to undertake, says Park.
HBM PIM is extra of a mid- to long-term chance, for SK hynix. In the intervening time, prospects are already coping with sufficient points as they attempt to transfer HBM DRAM bodily nearer to processors. “Many specialists on this area don’t wish to add extra, and fairly vital, complexity on prime of the already busy scenario involving HBM,” says Park.
That mentioned, SK hynix researchers labored with Purdue College pc scientists on a complete design of an
HBM-PIM product referred to as Newton in 2019. Like Samsung’s Aquabolt-XL, it locations multiply-and-accumulate items within the reminiscence banks to make the most of the excessive bandwidth throughout the dies themselves.
“Samsung has put a stake within the floor,” —Bob O’Donnell, chief analyst at
Technalysis Analysis
In the meantime, Rambus, based mostly in San Jose, Calif., was motivated to discover PIM due to power-consumption points, says Rambus fellow and distinguished inventor
Steven Woo. The corporate designs the interfaces between processors and reminiscence, and two-thirds of the ability consumed by system-on-chip and its HBM reminiscence go to transporting knowledge horizontally between the 2 chips. Transporting knowledge vertically throughout the HBM makes use of a lot much less vitality as a result of the distances are a lot shorter. “You is likely to be going 10 to fifteen millimeters horizontally to get knowledge again to an SoC,” says Woo. “However vertically you’re speaking on the order of a pair hundred microns.”
Rambus’s experimental PIM design provides an additional layer of silicon on the prime of the HBM stack to do AI computation. To keep away from the potential bandwidth bottleneck of the HBM’s central through-silicon vias, the design provides TSVs to attach the reminiscence banks with the AI layer. Having a devoted AI layer in every reminiscence chip may permit reminiscence makers to customise recollections for various functions, argues Woo.
How shortly PIM is adopted will rely on how determined the makers of AI accelerators are for the memory-bandwidth aid it supplies. “Samsung has put a stake within the floor,” says Bob O’Donnell, chief analyst at
Technalysis Analysis. “It stays to be seen whether or not [PIM] turns into a business success.
This text seems within the January 2022 print challenge as “AI Computing Involves Reminiscence Chips.”
[ad_2]
