AI is reshaping the memory chip market: Why are HBM, DDR5, and NAND demands continuing to grow?

When it comes to AI servers, many people immediately think of GPUs.  
But if you take apart an AI server, you'll find that the GPU is only one component. What truly powers AI model operations are also HBM, DDR5, NAND Flash, SSDs, and the supporting power management, controllers, and high-speed interface chips for these memory components.  

By 2026, the storage demands driven by AI have expanded beyond just HBM. HBM, DDR5, and NAND Flash are now forming a new AI storage supply chain.  
SEMI forecasts that global investment in storage equipment at 300mm wafer fabs will exceed $50 billion in 2026, with HBM, DDR5, and 3D NAND serving as key growth drivers.  

Why does AI require more and more memory chips?  

One major difference between AI models and traditional software is the massive volume of data they need to process and transfer.  
Take large language models as an example: model parameters, training data, intermediate computation results, and cache generated during inference all require rapid exchange across different levels of memory.  

Simplified, the storage architecture of an AI server can be understood as:  
CPU → DDR5 → AI accelerator → HBM → SSD/NAND  

Each type of memory plays a distinct role:

Therefore, AI is not simply about "adding more GPUs."  
In reality, the stronger the computing power of an AI server, the higher the demands for memory bandwidth, capacity, and storage speed.  

HBM: The Key Memory Behind AI Chips  

HBM serves as the high-speed data reservoir closest to the GPU. One of the key differences between HBM and traditional DDR memory lies in its use of 3D stacking and ultra-wide interface technologies.  
Multiple DRAM dies are vertically stacked and then connected to the AI processor via advanced packaging, enabling extremely high memory bandwidth within a compact physical footprint. This is crucial for AI computing, as AI models frequently need to access vast amounts of parameters during computation. If the processing speed is fast but memory cannot deliver data quickly enough, the GPU may end up "waiting for data."  
Thus, as AI computing power increases, so does the demand for memory bandwidth, making HBM increasingly vital.  

Illustrative Example: NVIDIA's Data Center AI GPUs  

Modern high-end AI accelerators typically integrate HBM near the chip rather than relying solely on conventional DDR memory for high-speed computation.  
The reason is simple:  
AI GPUs require "high bandwidth," not just "large capacity." For instance, NVIDIA's H100 uses HBM3, while subsequent high-end AI GPU platforms have adopted even more advanced technologies like HBM3E. AMD's Instinct series of AI accelerators also employs HBM technology. This means that as AI GPU shipments grow, the impact extends beyond the GPUs themselves.  
Components related to HBM-including DRAM dies, TSVs, interposers, advanced packaging, substrates, high-speed signal connections, power management, and thermal systems-will all be affected. This explains why HBM has become the critical memory behind AI chips in recent years.  

A Typical Case of Rising HBM Demand: SK hynix

SK hynix is one of the key suppliers in the HBM market at present. Its HBM products are mainly targeted at the high-performance computing and AI accelerator markets. From the perspective of the industry chain, the biggest difference between HBM and ordinary DRAM is not merely "larger capacity". HBM has higher requirements for the entire manufacturing and packaging process. The expansion of HBM production capacity is not simply increasing a few DRAM production equipment, but also involves wafer manufacturing, packaging, testing and supply chain management. Therefore, when the demand for AI grows rapidly, the supply of HBM cannot be expanded quickly in a short period of time.

DDR5: The storage chip that is easily overlooked in AI servers

When it comes to AI storage, many people only focus on HBM. But in fact, DDR5 is also an important component of AI servers.
HBM mainly serves AI accelerators, while DDR5 mainly undertakes the system memory task on the CPU side of the server.
An AI server usually not only has GPUs or AI Accelerators, but also requires the CPU to run:

• Operating System
• Data preprocessing
• Data Management
• Model loading
• Network task
• Storage Management
• Other server applications

All these tasks require a large amount of system memory.
So it can be simply understood as:

• HBM is responsible for "high-speed AI computing"
• DDR5 is responsible for "server system operation"

They are not in a competitive relationship, but a complementary one.

For example: Why does DDR5 increase along with AI?

Suppose a server uses the AMD EPYC or Intel Xeon platform.
The CPU connects to multiple DDR5 memory channels, and at the same time, the server is configured with multiple AI accelerators.
Then the entire system may form such a structure: CPU -- DDR5 -- PCIe / High-speed interconnection -- AI Accelerator -- HBM.
At the same time, the server also needs: Enterprise SSD → NAND Flash.
That is to say, an AI server actually consumes multiple storage technologies simultaneously.
Therefore, AI is simultaneously driving up the demand for HBM and DDR5 in servers.

NAND Flash: Another Piece of the Puzzle in the AI Era

If HBM addresses the question: "How can data be quickly read during AI computing?",
then NAND Flash addresses the question: "Where are all these data stored?"
AI training generates a large amount of data, including:

• Training Dataset
• Model Checkpoint
• Log Data
• Vector Database
• Inference Data
• Backup Data

These data ultimately need to be stored.
Therefore, the demand for enterprise-level SSDs and NAND Flash in AI data centers is also increasing.

To illustrate: Why did Micron simultaneously launch HBM and DDR5?

Unlike focusing solely on a specific type of memory, Micron has made a comprehensive layout:

• HBM
• DDR5
• Server DRAM
• NAND
• SSD

This actually reflects an important change in the AI storage market: AI does not merely require a single type of Memory.
For example:

• The AI Accelerator needs HBM to solve the problem of high-speed data access.
• The Server CPU needs DDR5 to provide large-capacity system memory.
• The AI Data Center requires Enterprise SSD + NAND to store massive amounts of data.

Therefore, for storage chip suppliers, the opportunities brought by AI are for the entire product line, rather than just for a specific model.

Summary

AI is causing changes in the entire server hardware architecture.
HBM might be the currently most talked-about storage technology, but DDR5, NAND Flash, and enterprise-level SSDs are also indispensable parts of AI infrastructure.
For the electronics component industry, what truly deserves attention is not "How many GPUs does AI need this year", but:
AI is redefining the entire memory and semiconductor supply chain.
From HBM to DDR5, and then to NAND and SSDs, AI is connecting the previously relatively independent storage market together.
For chip buyers, electronics component suppliers, and end-device manufacturers, understanding this complete storage architecture will also become an important foundation for future inventory planning, alternative selection, and supply chain management.