Broadcom 2024 AI Infrastructure Day
Irrational Recap
IMPORTANT:
Irrational Analysis is heavily invested in the semiconductor industry.
Please check the ‘about’ page for a list of active positions.
Positions will change over time and are regularly updated.
Opinions are authors own and do not represent past, present, and/or future employers.
All content published on this newsletter is based on public information and independent research conducted since 2011.
This newsletter is not financial advice and readers should always do their own research before investing in any security.
Link to the vod here.
Timestamps are with respect to the official vod.
[18:00] Customer List
#1 Google/TPU.
#2 Meta/MITA (some kind of recommendation network accelerator)
#3… that everyone has been trying to guess.
Before talking to anyone else, I believed this new design win must be a DPU // networking heavy ASIC for a Chinese hyperscaler such as Baidu or Alibaba.
In private conversations, several folks suggested Bytedance and this makes a ton of sense. Latency-sensitive, massively paralel, network heavy workloads. Clearly falls under Broadcom’s “consumer AI” category.
The export control rules are very specific and place a large emphasis on compute density and memory bandwidth. Not that hard to design a chip that is network heavy (PCIe, Ethernet) and thus complies with USA commerce department rules while still being useful.
Now let’s take a look at sell-side’s attempts.
Amazon is with Marvell. Everyone knows this.
Apple will make their own.
Before talking to some friends, I was 99% confident customer #3 was not a U.S.-based hyperscaler. After speaking with others, I am 99.9% confident.
Amazon is with Marvell. This socket has not changed. Timeline shown in the slide makes it impossible.
Microsoft is flailing about trying to cobble something competative together on the cheap. This means not paying Broadcom’s margins.
Oracle is best friends with Nvidia, going all-in.
This leaves no other options for Piper Sandler analyst’s incorrect guess.
They did not even try to guess customer #3. At least they re-confirmed correct information. Meta is not “likely” BTW. It is 100%.
[35:00] Inadvertant Infiniband Advertisment
If the network is the computer, then it would be really nice to get a lot of compute done in network. Say… by using Nvidia SHARP over Infiniband for all-reduce and lots of other cool compute operations.
Broadcom spends a lot of time trash talking Infiniband and promoting how awesome open standards like Ethernet are.
Once again, I feel it is important for subscribers to be reminded of my biasies.
In my dayjob (not Nvidia or Broadcom), I work on Ethernet.
~50% of my portfolio is Nvidia shares. ~5% is Broadcom shares.
All of this debate between Ethernet and Infiniband is stupid. Both interconnects have pros and cons. Both will continue to exist. Nvidia sells switching products for both, which is the correct decision.
[46:00] Niche NIC
The market is rapidly moving towards DPU. Get compute done in the network cards to improve effective bandwidth and free up GPU/CPU/XPU/ASIC resources.
Basically, the network card is becoming a tiny computer with some ARM CPU cores, fixed-function logic, and accelerators. You can SSH into your network card now because it runs Linux.
Broadcom Thor2 is the polar opposite of what industry is gravitating towards in the AI era. There exists a place in the market for an ultra-power effecient NIC but this is not the future. Misleading commentary on this part of the presentation.
[48:00] More Infiniband Trash Talk
Broadcom feels the need to continuously roast infiniband. It’s almost as if all-reduce in-network leads to massive gains in bus-bandwidth.
Broadcom is repeatedly sharing willfully misleading information on Infiniband vs Ethernet. This includes contrived benchmarks that don’t utilize any of the compute features/advantages of Infiniband…
[1:15:00] Co-Packaged Optics Reliability
The presenter discusses reliability but glosses over servicability A dead pluggable optical trancever can be easily replaced. This means CPO has much stricter reliability requirments. Orders-of-magintute higher. Everything that is not the lazer needs to be really damn reliable for CPO to be economically viable.
[1:50] Incredible Custom Silicon Design Speed
This presentors section is super well done. I strongly reccomend watching his entire appearance for yourself. Very impressive it only takes 7-9 months for Broadcom to get all pre-silicon activities done.
Commentary on the 3D-wafer design for 2026 and beyond is incredible.
Opinions/Conclusions
I am an incrimental buyer of AVGO 0.00%↑. Cash generation from dayjob is not great after deducting 401K, ESPP, and paying rent. Slows down how fast I can buy more shares.
Broadcom is one of my favorite stocks to add right now. Well diversified, competent management, great existing tech, incredible technology roadmap.
Sidenote: I absolutly hate it when companies present willfully misleading statements/data/benchmarks. Ethernet vs Infiniband section is full of bovine excriment.
ByteDance would want video compression pipeline, too.