Is NAND Flash a hidden AI play?
Video models could drive explosive growth.
IMPORTANT:
Irrational Analysis is heavily invested in the semiconductor industry.
Please check the ‘about’ page for a list of active positions.
Positions will change over time and are regularly updated.
Opinions are authors own and do not represent past, present, and/or future employers.
All content published on this newsletter is based on public information and independent research conducted since 2011.
This newsletter is not financial advice and readers should always do their own research before investing in any security.
Remember when Joanna Stern (WSJ) interviewed OpenAI CTO Mira Murati on their ‘Sora’ video generative AI model?
This glorious meme was inspirational… for a niche investment idea: NVMe SSDs.
Contents:
Why video model training may cause a NAND demand spike.
What is NAND flash memory?
State of the industry, featuring WDC’s lame 2024 investor event.
NVMe: Components and Key Attributes
The Players (Investment Vectors)
Companies that also make DRAM
WDC + Kioxia + Bain Capital Clown Show
The Merchant Silicon Providers
Conclusion and Risks
[1] Why video model training may cause a NAND demand spike.
Why are we here (this particular corner of the public internet) today?
The answer is OpenAI Sora and the hilarious face CTO Mira Murati made when asked if the model was trained on YouTube video data.
(they definitely used YouTube data to train Sora)
Also inspired by a discussion with
. Check out their great content.How though? YouTube has an API but data cannot be pulled live during training. Latency from pinging public Google datacenters would kill efficiency and MFU.
Full-resolution video also cannot be used effectively. OpenAI must have created a local repository of pre-processed YouTube data. Compressed, quantized, and (most importantly) local.
[2] What is NAND flash memory?
NAND flash is non-volatile memory. This means when it loses power, the memory retains data. USB flash drives, SATA SSDs, NVMe SSDs, SD cards, and eMMC storage are all based on NAND flash.
Essentially, a long string of transistors are used to store data. This data remains in the absence of a supply power because some transistors are in depletion mode.
This means the transistor behaves like a short-circuit in the absence of a gate voltage. So always on without any power.
For more detailed information on the tech itself, check out this excellent resource.
I don’t want to dwell on the technical details of NAND because of how degenerate this industry is. NAND truly is the bottom-feeding, shitco pity party of semiconductor.
Key Concepts:
NAND is the most commoditized corner of semiconductor-land.
COST BASE IS EVERYTHING.
In general, more layers is better because it reduces cost.
Bit’s per cell (SLC, MLC, TLC, QLC) has interesting tradeoffs that you need to understand to follow industry presentations.
More bits per memory cell mean higher density (good) but lower write cycle endurance (bad) but much lower price (good).
Almost everything these days (2024) TLC with some QLC penetration.
Many drives have SLC caches to improve responsiveness.
There is a huge difference between SATA and NVMe SSDs.
Components other than the NAND itself (logic controller, DRAM cache) have an outsized impact on drive performance.
[3] State of the industry, featuring WDC’s lame 2024 investor event.
On June 10th, 2024, Western Digital ( WDC 0.00%↑ ) held an investor webinar titled ‘The New Era of NAND’.
A brief summary of the event is as follows:
We are a bunch of bottom-feeding losers and intend to not be losers in the future.
Thier presentation is a good vector for explaining the state of the NAND industry.
How NAND got here.
Why everyone is so malnourished, miserable, and traumatized.
Key technology trends over the last decade+.
They last five years have been painful.
YMTC (China) has been flooding the global market with dirt cheap NAND while retaining profitability. Protectionist sanctions did not do much to slow them down.
General complaints on CapEx intensity for the transition to 3D NAND.
3D NAND stacks many layers (we are at 200+ now) to get huge density gains. Printing such complex structures is difficult. Companies like Lam Research, Tokyo Electron, Applied Materials, and so on have made great money selling tools to the bottom feeders. The NAND manufacturers themselves… no lol.
WD has been trying to push PLC (5-bit-per-cell) for quite some time. QLC has existing endurance and reliability challenges.
An intuitive way for understanding why logical scaling harms endurance/durability is as follows.
With more bits per cell, you need to much more accuracy to charge and measure the cell voltage.
As the same cell is written/read from over time, the transistors degrade. Electrons get “stuck” in the cell.
Eventually, a cell will be unable to reach all the necessary voltage states and must be disabled, permanently reducing product bit capacity.
Drive manufacturers must over-provision NAND as a reserve/backup for this inevitability, driving up cost.
I find it amusing that WD spent a third of their presentation complaining about CapEx and promising to reduce future equipment spend while simultaneously hinging most of their technological differentiation on hybrid bonding.
Allways nice to see a hyper-commoditized bottom-feeder claiming their business has entered a new era of premium pricing power.
The performance of products that use NAND flash heavily depends on DRAM caching and controller (built on logic node) quality.
EVERYTHING IS NEW EVERYONE. WE WILL NOT EAT SHIT IN THE INEVITABLE, VIOLENT DOWNCYCE.
Layers race is over! WD has declared it so! YMTC, Micron, and Samsung definitely will not continue racing even though we are publicly begging everyone to slow down.
[4] NVMe: Components and Key Attributes
Broadly speaking, there are three primary components to a solid-state drive:
NAND flash chips
DRAM cache
Logic (interface) controller
Cheapo drives skip the DRAM cache and have terrible performance. Mid-tier drives use lower-quality controllers.
Ironically, the NAND itself tends to have the least impact of the three on performance.
Here is an example using two drives I happen to have in my primary machine. Both drives use 64-layer TLC NAND but the performance is widely different. This is due to the following differences:
The NVMe drive is not limited by an ancient standard (SATA) that was designed for hard-drives.
The NVMe drive has double the ratio of DRAM (cache) to NAND.
The NVMe drive uses a custom-made controller with optimized firmware while the SATA drive uses an off-the-shelf merchant silicon controller.
In general, it is rare for real-world workloads to be sequential in nature. A sequential write is like copying a large file all at once.
Most workloads are random and low-queue depth. This means the program (operating system, database, web browser, …) reads/writes in small 4-16 KB chunks randomly spread out across the memory. The software/filesystem tries to queue up several small tasks to speed up operation but is somewhat limited.
Every workload is different. To properly evaluate drives for AI training workloads, it is necessary to profile the workload directly instead of relying on toy benchmarks or manufacturer-provided IOPS specifications.
The key takeaways are:
NVMe is way better than SATA.
The NAND flash itself has less impact compared to the DRAM cache and logic controller.
[5] The Players (Investment Vectors)
Comments on the TrendForce table:
Solidigm was Intel’s Flash business before it got sold to SK Hynix.
Kioxia was Toshiba Flash before it was spun out to Bain Capital.
“Others” is dominated by YMTC who is likely severely undercounted because sanctions.
Broadly speaking, there are three categories the investment… opportunities available to NAND degenerates.
[5.a] Companies that also make DRAM
All memory is a commodity but not all commodities are created equal. DRAM is going through a super-cycle because of HBM supply-destruction via 3-5x area intensity. Samsung, SK, and MU stocks are all moving largely due to DRAM, not NAND. If you want to find an investment vector for NAND as a pure-play, these three are not it.
However, pure-plays are not necessarily the best. All good NVMe drives need DRAM cache. In theory, the companies that make both DRAM and NAND have an intrinsic advantage.
[5.b] WDC + Kioxia + Bain Capital Clown Show
WDC and Kioxia (formerly Toshiba Flash) have a long, complicated history.
Key Points:
Bain Capital owns Kioxia and is in deep ****.
SK Hynix is also a private shareholder of Kioxia.
WDC and Kioxia have had a technology partnership for many years.
BiCS flash.
Essentially, R&D is shared amongst these sub-scale shitcos.
WDC has been desperately trying to merge with Kioxia for YEARS.
Bain wants to unload bags.
SK Hynix opposes this merger.
WDC and Kioxia are self-aware and realize they are sub-scale.
This saga has been going on for so long that Bain is trying to IPO their bags.
TOKYO, April 16 (Reuters) - Bain Capital has proposed an initial public offering (IPO) of Japan's Kioxia Holdings as part of a plan to allow the money-losing chipmaker to refinance a $5.8 billion loan coming due in June, according to a person familiar with the matter.
Bain and Kioxia met with a group of banks including Sumitomo Mitsui (8316.T), Mizuho (8411.T), and Mitsubishi UFJ (8306.T), on Monday after the lenders urged Kioxia to come up with a recapitalisation plan because it looks likely to run afoul of the terms of a 900 billion yen ($5.8 billion) syndicated loan, the person said.
Formerly Toshiba Memory, Kioxia was acquired by a Bain-led group in a $18 billion carve-out in 2018. The maker of memory chips is expected to have posted a loss for the year that ended last month, according to the person. That would mark its second straight annual loss and push its net assets below the roughly 500 billion yen required under the terms of the loan, the person said.
The banks want the refinancing to pave the way for reviving merger talks with U.S. chipmaker Western Digital (WDC.O), according to the source.
Talks between the two to merge and create a memory chip powerhouse stalled last year due to opposition from South Korea's SK Hynix (000660.KS), which is an investor in Kioxia.
The banks had previously pledged to provide a loan of about 1.9 trillion yen on the condition that Kioxia would merge with Western Digital.
Source: Reuters
[5.c] The Merchant Silicon Providers
Most drive manufactures make their own controllers.
There are two big merchant silicon providers of NVMe controllers, Phison (TPEX:8299) and Silicon Motion (SIMO 0.00%↑).
A few other tiny players exist (Microchip, Marvell) but are not worth taking seriously for NVMe.
[6] Conclusion and Risks
Look at this graph.
NAND is not a good long-term investment. Terrible compared to every other sub-sector within semiconductor.
This chart puts into perspective all the cope WD presented in their “NEW ERA OF NAND” public therapy session.
However, NAND is an interesting opportunity right now because of AI driving almost all attention to HBM/DRAM, historically low bit growth, and a possible incoming demand-shock driven by video model training.
Good luck, have fun.
Pure storage is another type of investment into the NAND space. They don’t manufacture any NAND, they take Microns NAND, package it into an all flash data array, and sell that.