Roughly 15bn 2GB DRAM chips shipped in 2023, and on track for 16bn this year.
HBM chips are 1.2 cm2 or a bit less than 2x the size of DDR or LPDDR chips (typically .65 to .7 cm2).
Let's say the shipments of H100-equivalent this year come to 4M, each with 128GB average HBM and 512GB average ancillary DRAM. Around 256M HBM and 1024M commodity DRAM chips. If we reckon the average yield loss of HBM in assembly (the loss at wafer level will be very low) at 30% then we come to under1.9 billion LPDDR-equivalents of manufacturing needed at the wafer level for the AI market. About 14% of production capacity this year, of which 6% is the actual HBM and the rest will be LPDDR and DDR with the LPDDR share rising sharply later this year.
There may have been a compensating fall in other markets, servers and mobile have both been flat to down IIRC. There seems no basis for saying there is a 30% capacity shock to the DRAM market. There has been a severe shortage in HBM, but that is due mostly to downstream assembly capacity needs.
Laptop/PC market and smartphones are using more DRAM on average going forward, right? Microsoft is setting minimum requirements for "copilot+" branding.
Microsoft specs won't matter for mobile - though others may make similar decisions. Use of LPDDR is likely to rise. Laptops are a smaller market and the copilot+ likely to be just a fraction of that.
2025 is likely a surge and I would expect to see DRAM vendors investing in commodity capacity to get ready.
Any thoughts on WDC or other memory companies? Seeing a lot of reports of pricing leverage on the NAND side too. Not the type of company I’m looking to invest in, but I’m mildly curious as a fellow MU call holder
About 90 exabytes of SSD shipped in 1Q24 which is about 12x the capacity of DRAM shipments. HDD should ship around 250 to 300 EB per quarter. Total shipments of flash are on par with HDD capacity, most of it going into mobiles. AI generates a lot new output but it is not clear where that is going and how much of it is discarded rapidly. If you can find solid figures for what happens to the data generated by AI that might help with analyzing the impact to storage markets (SSD, HDD, mobile flash).
I suspect generated AI data will mostly drive the HDD market because of its low value and predictable lifetime and usage patterns. HDD is the best economic match. The HDD makers may be glad to crank up capacity, some of which may be idle and cheap if you buy the right kinds of drives.
You don't need random IO for training set. There may be some strategies reordering it but you can probably finesse that by sequential processing and selection into different streams before writing them out.
That’s my point. If SEC had to go back to the drawing board with HBM then the latent capacity would switch to DDR5 and impact that market.
SK and MU will enjoy more market share at NVDA as a result of SEC’s issues but the benefit will be limited, as their HBM capacity is effectively capped for the next 12 months.
Personally I’m a bit concerned about which way the [memory] market might go after the Blackwell rollout.
Depends how much they run the line on wasted chips, but I would expect reconfiguring the line for HBM is a slow process which was already done on some slightly optimistic basis (the Samsung culture does not seem one where you confess to problems in advance) and if they want to grab more of the DDR capacity to allocate to HBM that will be slow, especially if they do not even have firm orders yet and so could be stockpiling chips while they fix the yield problems downstream.
"The one part of Doug’s thesis I do agree with is the stock grants Marvell management gave themselves. Encourages sandbagging of the Amazon capital bonfire opportunity."
Can you elaborate on this a little more? I'm not familiar with Doug's thesis.
There is a belief that Marvell will defend their market share in DSP/optical because they have been successful since acquiring Inphi. I think Broadcom is now motivated and will taking share going forward.
Roughly 15bn 2GB DRAM chips shipped in 2023, and on track for 16bn this year.
HBM chips are 1.2 cm2 or a bit less than 2x the size of DDR or LPDDR chips (typically .65 to .7 cm2).
Let's say the shipments of H100-equivalent this year come to 4M, each with 128GB average HBM and 512GB average ancillary DRAM. Around 256M HBM and 1024M commodity DRAM chips. If we reckon the average yield loss of HBM in assembly (the loss at wafer level will be very low) at 30% then we come to under1.9 billion LPDDR-equivalents of manufacturing needed at the wafer level for the AI market. About 14% of production capacity this year, of which 6% is the actual HBM and the rest will be LPDDR and DDR with the LPDDR share rising sharply later this year.
There may have been a compensating fall in other markets, servers and mobile have both been flat to down IIRC. There seems no basis for saying there is a 30% capacity shock to the DRAM market. There has been a severe shortage in HBM, but that is due mostly to downstream assembly capacity needs.
Laptop/PC market and smartphones are using more DRAM on average going forward, right? Microsoft is setting minimum requirements for "copilot+" branding.
Microsoft specs won't matter for mobile - though others may make similar decisions. Use of LPDDR is likely to rise. Laptops are a smaller market and the copilot+ likely to be just a fraction of that.
2025 is likely a surge and I would expect to see DRAM vendors investing in commodity capacity to get ready.
Any thoughts on WDC or other memory companies? Seeing a lot of reports of pricing leverage on the NAND side too. Not the type of company I’m looking to invest in, but I’m mildly curious as a fellow MU call holder
WDC is complicated because of the spinoff. There indeed is pricing leverage as NAND is hitting a cycle.
Ah that’s right… That should be finalizing somewhat soon IIRC. I guess my best bet is to stick with MU for any NAND-related gambling impulses, too.
About 90 exabytes of SSD shipped in 1Q24 which is about 12x the capacity of DRAM shipments. HDD should ship around 250 to 300 EB per quarter. Total shipments of flash are on par with HDD capacity, most of it going into mobiles. AI generates a lot new output but it is not clear where that is going and how much of it is discarded rapidly. If you can find solid figures for what happens to the data generated by AI that might help with analyzing the impact to storage markets (SSD, HDD, mobile flash).
I suspect generated AI data will mostly drive the HDD market because of its low value and predictable lifetime and usage patterns. HDD is the best economic match. The HDD makers may be glad to crank up capacity, some of which may be idle and cheap if you buy the right kinds of drives.
What are people using for storing the training dataset? I thought large NVMe arrays for the fast random I/O but would like your opinion.
You don't need random IO for training set. There may be some strategies reordering it but you can probably finesse that by sequential processing and selection into different streams before writing them out.
Really interesting breakdown. I appreciate the insights. Very new to this stuff but the memory/storage markets have me increasingly intrigued
From what I understand, Anthropic is training their model on Amazon ASICs? Any thoughts?
So what happens to Samsung’s DDR5 output if they are not able to produce as much HBM as forecasted?
Good question. Depends on if they give up and cut HBM production I suppose?
That’s my point. If SEC had to go back to the drawing board with HBM then the latent capacity would switch to DDR5 and impact that market.
SK and MU will enjoy more market share at NVDA as a result of SEC’s issues but the benefit will be limited, as their HBM capacity is effectively capped for the next 12 months.
Personally I’m a bit concerned about which way the [memory] market might go after the Blackwell rollout.
Depends how much they run the line on wasted chips, but I would expect reconfiguring the line for HBM is a slow process which was already done on some slightly optimistic basis (the Samsung culture does not seem one where you confess to problems in advance) and if they want to grab more of the DDR capacity to allocate to HBM that will be slow, especially if they do not even have firm orders yet and so could be stockpiling chips while they fix the yield problems downstream.
So the Dell’s storage business really has 70% gross margins??
The high-end NVMe storage servers. Regular HDD and SATA SSD much cheaper.
"The one part of Doug’s thesis I do agree with is the stock grants Marvell management gave themselves. Encourages sandbagging of the Amazon capital bonfire opportunity."
Can you elaborate on this a little more? I'm not familiar with Doug's thesis.
There is a belief that Marvell will defend their market share in DSP/optical because they have been successful since acquiring Inphi. I think Broadcom is now motivated and will taking share going forward.