I suspect there's another reason why Softbank bought Ampere : prevent Mediatek from pulling another Nuvia/Qualcomm on ARM by acquiring Ampere. Ampere developed their own ARM ISA-compatible custom core, with a license to do so. Given the substantial royalties that a company like Mediatek (largest ARM SoC maker and licensee in the world!) pays to ARM under their current license, Mediatek buying Ampere on the cheap to get their custom core designs would have been a great way to reduce those high licensing fees by quite a lot. And with the Waterloo moment ARM had in their legal dispute with Qualcomm, it would have been a pretty safe bet that it would've worked. Well, would have, because Softbank just slammed that door shut.
Can at least confirm everything about Azure is shit and has been since day one, it's way too enmeshed with the 90% of the company that's still run like it's 1995 to ever change.
Didn't JHH mention earlier gen GPUs are being repurposed for inference at CSPs and lots of them were still in service? Useful life should be longer than 3 yrs.
If the useful life of DC GPUs is indeed 2-3 years, then what's the point of making custom ASICs, which would probably be comparable in performance to last-gen GPUs?
Say I'm Meta and I need a ton of inference. I can $ into:
1) designing MTIAs
2) ordering them from TSMC
3) hoping that models' paradigm doesn't change so much that my ASICs would become absolutely worthless
And in the end the cost of production would be way higher for me, than for Nvidia, because it's R&D is spread across $200bn in sales and it can buy wafers for less @TSMC. And I can't sell my ASICs to anyone in 2-3 years, because they were designed for Meta only.
Or I can just buy cheap second-hand last-gen GPUs from Azure / GCP whatever other firm.
Two possible reasons, even if proprietary ASICs aren't cheaper (I write possible, as Meta, MS, AWS won't say 😁):
1. Not having to wait in line - Nvidia has a huge back log.
2. Reduce dependance on Nvidia for software and updates. And likely soon, compete with them for customers. Nvidia (Jensen Huang) has now clearly stated that they see Hyperscalers also as competitors. It's very uncomfortable and increasingly risky to depend so much on hardware that is made by one's competitor.
While not directly comparable, AWS had their reasons why they first bought Anapurna and then designed their ARM-based Graviton CPUs there. As far as we know, that decision paid off for AWS/Amazon.
Forgot to add this: used/last gen GPUs/accelerators are either really no longer competitive, very inefficient (power consumption) and unattractive for customers or, if still viable, basically unavailable.
Do you mean that by 2027 Nvidia's Blackwell with all of their rnd budget and a decade of experience will be behind Meta or AWS's napkin chip design created in under 2 years and for a few hundred million $?
No, but for Hyperscalers like AWS and Microsoft, the question is more about lcompeting with Nvidia's then-current accelerators available directly from Nvidia. Also, both AWS and Microsoft have a lot of incentive to nudge (or push/armtwist) their customers to use their own respective solutions, if only to justify the investment. AWS did that pretty successfully with the first generation(s) of Graviton. But of course, AI is a very different situation.
Maybe old news but this concept was not taught to me on CFA course:-
SoftBank's stake in Arm ~$158 billion
SoftBank's total market value: ~ $77 billion
interesting 👍
Just curious, what do you think of QRVO?
I suspect there's another reason why Softbank bought Ampere : prevent Mediatek from pulling another Nuvia/Qualcomm on ARM by acquiring Ampere. Ampere developed their own ARM ISA-compatible custom core, with a license to do so. Given the substantial royalties that a company like Mediatek (largest ARM SoC maker and licensee in the world!) pays to ARM under their current license, Mediatek buying Ampere on the cheap to get their custom core designs would have been a great way to reduce those high licensing fees by quite a lot. And with the Waterloo moment ARM had in their legal dispute with Qualcomm, it would have been a pretty safe bet that it would've worked. Well, would have, because Softbank just slammed that door shut.
Can at least confirm everything about Azure is shit and has been since day one, it's way too enmeshed with the 90% of the company that's still run like it's 1995 to ever change.
lol random Rivos burn. Looking forward to you changing your tune on Rivos after they invite you for a briefing and give you some merch!
Didn't JHH mention earlier gen GPUs are being repurposed for inference at CSPs and lots of them were still in service? Useful life should be longer than 3 yrs.
What do you think of Nebius in light of the Coreweave's IPO?
Sell, plenty of red flags even before IPO
If the useful life of DC GPUs is indeed 2-3 years, then what's the point of making custom ASICs, which would probably be comparable in performance to last-gen GPUs?
Say I'm Meta and I need a ton of inference. I can $ into:
1) designing MTIAs
2) ordering them from TSMC
3) hoping that models' paradigm doesn't change so much that my ASICs would become absolutely worthless
And in the end the cost of production would be way higher for me, than for Nvidia, because it's R&D is spread across $200bn in sales and it can buy wafers for less @TSMC. And I can't sell my ASICs to anyone in 2-3 years, because they were designed for Meta only.
Or I can just buy cheap second-hand last-gen GPUs from Azure / GCP whatever other firm.
Two possible reasons, even if proprietary ASICs aren't cheaper (I write possible, as Meta, MS, AWS won't say 😁):
1. Not having to wait in line - Nvidia has a huge back log.
2. Reduce dependance on Nvidia for software and updates. And likely soon, compete with them for customers. Nvidia (Jensen Huang) has now clearly stated that they see Hyperscalers also as competitors. It's very uncomfortable and increasingly risky to depend so much on hardware that is made by one's competitor.
While not directly comparable, AWS had their reasons why they first bought Anapurna and then designed their ARM-based Graviton CPUs there. As far as we know, that decision paid off for AWS/Amazon.
Forgot to add this: used/last gen GPUs/accelerators are either really no longer competitive, very inefficient (power consumption) and unattractive for customers or, if still viable, basically unavailable.
Do you mean that by 2027 Nvidia's Blackwell with all of their rnd budget and a decade of experience will be behind Meta or AWS's napkin chip design created in under 2 years and for a few hundred million $?
No, but for Hyperscalers like AWS and Microsoft, the question is more about lcompeting with Nvidia's then-current accelerators available directly from Nvidia. Also, both AWS and Microsoft have a lot of incentive to nudge (or push/armtwist) their customers to use their own respective solutions, if only to justify the investment. AWS did that pretty successfully with the first generation(s) of Graviton. But of course, AI is a very different situation.
Available directly from Nvidia as in renting instances directly from Team Green.