TECH NEWS – SanDisk is trying to break new ground, a move ultimately forced by artificial intelligence.
The rapid rise of AI and the resulting increase in demand for computing power have created bottlenecks, compelling DRAM and NAND manufacturers to pursue more integrated solutions. Historically, chip makers solved these problems by introducing new memory technologies, with DRAM serving as the primary component. Rising costs, development and yield challenges, and increased power consumption have shifted attention toward other viable approaches, however. HBM, or high-bandwidth memory, has evolved steadily, but shortages are quickly turning it into a bottleneck as well. HBM also has other drawbacks, such as lower capacity, and while DRAM manufacturers are achieving higher speeds and capacities with each new generation, they have so far been unable to keep pace with demand. HBM is also located beside the main chip, which comes with latency disadvantages.
NAND, meanwhile, offers higher capacity at a lower price, but it is located farther from the chip, resulting in slower data-transfer speeds. NAND has not been able to match the speed of DRAM, or HBM, either. To address this, NAND manufacturer SanDisk previously revealed its plans for an HBF, or High-Bandwidth Flash solution. HBF reportedly uses an architectural hierarchy similar to HBM, meaning it stacks multiple layers of NAND Flash on top of one another. The layers are connected using multiple TSVs, or Through Silicon Vias, combining the NAND packages into a single stack. While HBM currently offers 32-64 GB of capacity per stack, HBF could scale up to 4 TB.
While this could address part of the capacity and speed problem, future demands from AI and high-performance computing, or HPC, require more. This is where SanDisk’s latest patent, US 12,430,274 B2, enters the picture. The patent explores the idea of stacking a NAND Flash module with CBA, or CMOS Bonded Array technology, beneath the main compute module in three dimensions. That module could be an AI accelerator or a GPU. The solution would continue to use HBM DRAM on the same interposer, but for a different purpose. HBM would handle memory tasks requiring immediate attention, while the NAND Flash on the memory module would be used for read and write operations and for managing larger data sets. NAND Flash could provide broader connections between the compute chip and the memory module, reducing latency, cost, and power consumption.
The processing core contains a multi-core processor directly integrated with high-bandwidth, high-capacity non-volatile memory. The processor could be a high-performance graphics processing unit or an artificial intelligence processor, for example. The non-volatile memory may include a CBA memory module comprising a large NAND memory module and a CMOS logic circuit module. The integrated processor and CBA memory module can be mounted on an interposer. The processing core may also contain layers of large HBM semiconductor chips attached to the interposer on one or more sides of the processor and CBA memory module.
Although this vision offers a glimpse into future methods for overcoming memory bottlenecks, it is important to note that it is currently only a patent. Numerous questions, such as power consumption, the manufacturing cost of a chip combining NAND and DRAM in a single package, and many other practical issues, need to be solved before anything close to this can become reality. The patent establishes a serious and carefully considered defensive position around the processor-on-NAND architecture, especially in terms of wide-interface cross-die routing that would be difficult to replicate. The product moving toward standardization, however, takes the simpler, more market-ready beside approach.
The most interesting part of the story is still unfolding: will SanDisk ultimately bridge the gap between its protected technology and products that actually reach the market?
