Huawei shows off AI computing system to rival Nvidia's top product
Technology
15
Beiträge
7
Kommentatoren
85
Aufrufe
-
This post did not contain any content.
-
This post did not contain any content.
People will fall over each other to explain exactly why these devices are no match for nvidia’s top cards like H100/B100, but that’s besides the point. For a lot of people out there top tier nvidia products are basically unobtainium anyway.
If they manage to actually get this into peoples hands, this is a VERY big deal.
-
People will fall over each other to explain exactly why these devices are no match for nvidia’s top cards like H100/B100, but that’s besides the point. For a lot of people out there top tier nvidia products are basically unobtainium anyway.
If they manage to actually get this into peoples hands, this is a VERY big deal.
Can you explain for those of us who don’t know those words and ackronyms?
-
Can you explain for those of us who don’t know those words and ackronyms?
I edited with a bit more context. They are mostly just product identifiers.
Unobtainium just nerd speak for “things that are nominally available but impossible to actually get your hands on”. It’s rooted in sci fi tropes that are in themselves very interesting but besides the point right now.
-
People will fall over each other to explain exactly why these devices are no match for nvidia’s top cards like H100/B100, but that’s besides the point. For a lot of people out there top tier nvidia products are basically unobtainium anyway.
If they manage to actually get this into peoples hands, this is a VERY big deal.
Sucks you can’t get Chinese kit in USA though. At least I’ve never seen it available; not sure if it’s outright banned or US suppliers just don’t carry it
-
I edited with a bit more context. They are mostly just product identifiers.
Unobtainium just nerd speak for “things that are nominally available but impossible to actually get your hands on”. It’s rooted in sci fi tropes that are in themselves very interesting but besides the point right now.
Let me rephrase my question: Please fall over yourself to explain to me why these devices are no match for Nvidia’s top cards like the H100/B100. I wish to understand.
Cheers
-
Let me rephrase my question: Please fall over yourself to explain to me why these devices are no match for Nvidia’s top cards like the H100/B100. I wish to understand.
Cheers
He's arguing that comparing raw performance is moot in his comment, since having affordable/available supply that can undercut NVIDIA in the same role would be quite a blow to their market dominance (especially outside the US).
-
Let me rephrase my question: Please fall over yourself to explain to me why these devices are no match for Nvidia’s top cards like the H100/B100. I wish to understand.
Cheers
When making high performance chips, the main figure of merit is how small you can make individual switching elements. Smaller means faster switching but also less energy needed per switch, which in turns means less heat generation etc.
The smallest transistors can only be made by a specific company in Taiwan, and companies like nvidia and apple compete for every single wafer (unassembled chips) that comes out of that factory. This company sits at the end of a global supply chain: basically these chips can only be made if a bunch of countries all work together. One of the main policy goals of the western allies in the last decade or so has been to shut China out of this industry to prevent them from developing this capability.
If you don’t have access to the smallest transistors, you are going to have to make some pretty dire trade offs. Slower chips. Fewer cores per chip. That kind of stuff. That’s the problem Huawei is facing: no matter how good of a chip they design, it will always be at a disadvantage unless they can access the technology to make smaller transistors.
The catch here is that that factory is operating at capacity and big firms are snapping up most supply as soon as/before it hits the market. And that’s before we take into account various sanctions. So for many users, a slower chip that you can get will always beat the fast one that you can’t get.
-
When making high performance chips, the main figure of merit is how small you can make individual switching elements. Smaller means faster switching but also less energy needed per switch, which in turns means less heat generation etc.
The smallest transistors can only be made by a specific company in Taiwan, and companies like nvidia and apple compete for every single wafer (unassembled chips) that comes out of that factory. This company sits at the end of a global supply chain: basically these chips can only be made if a bunch of countries all work together. One of the main policy goals of the western allies in the last decade or so has been to shut China out of this industry to prevent them from developing this capability.
If you don’t have access to the smallest transistors, you are going to have to make some pretty dire trade offs. Slower chips. Fewer cores per chip. That kind of stuff. That’s the problem Huawei is facing: no matter how good of a chip they design, it will always be at a disadvantage unless they can access the technology to make smaller transistors.
The catch here is that that factory is operating at capacity and big firms are snapping up most supply as soon as/before it hits the market. And that’s before we take into account various sanctions. So for many users, a slower chip that you can get will always beat the fast one that you can’t get.
Thank you, that explains it excellently. I understand now.
-
This post did not contain any content.
It's not theoretical. They've already released an 300B LLM dubbed Pangu Pro, trained on Huawei NPUs:
Paper page - Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity
Join the discussion on this paper page
(huggingface.co)
And it's open weights!
IntervitensInc/pangu-pro-moe-model · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
(huggingface.co)
It's actually a really neat model: the experts are split into 8 'groups' and routed so that the same number are active in each group at any given time. In other words, it's specifically architected for 8X Huawei NPU servers, so that there's no excessive cross-communication or idle time between them.
So yeah, even if it's not a B200, proof's in the puddin, and huge models are being trained and run on these things.
-
People will fall over each other to explain exactly why these devices are no match for nvidia’s top cards like H100/B100, but that’s besides the point. For a lot of people out there top tier nvidia products are basically unobtainium anyway.
If they manage to actually get this into peoples hands, this is a VERY big deal.
If they manage to actually get this into peoples hands
To be clear, I think they're talking about mega-pricey server products, where the minimum size is usually 8 of them in a box. This isn't a home lab kind of thing.
-
Thank you, that explains it excellently. I understand now.
Just to add to this, the biggest moat Nvidia has is not transistor density, but their software ecosystem.
Every since like the GTX 200 series in 2008, Nvidia stuff has been the standard for academic research, and it basically only works on their GPUs. Anything for research is done on Nvidia GPUs, which is tweaked for enterprise deployment on GPUs... if you want it on something else, you basically have to start from scratch. And dump a tremendous amount of brainpower put into optimization.
AMD's in an interesting position here because they've been making Nvidia GPU competitors for literally decades. Their architectures are actually quite similar, hence it's easier to 'emulate' Nvidia on AMD than pretty much anything else.
...That being said, the Chinese have made tremendous progress busting out of the Nvidia software ecosystem, hence these chips are actually being used for real work.
-
If they manage to actually get this into peoples hands
To be clear, I think they're talking about mega-pricey server products, where the minimum size is usually 8 of them in a box. This isn't a home lab kind of thing.
Oh absolutely. The NV equivalent is priced at multiple millions of dollars. If you can get it.
-
Just to add to this, the biggest moat Nvidia has is not transistor density, but their software ecosystem.
Every since like the GTX 200 series in 2008, Nvidia stuff has been the standard for academic research, and it basically only works on their GPUs. Anything for research is done on Nvidia GPUs, which is tweaked for enterprise deployment on GPUs... if you want it on something else, you basically have to start from scratch. And dump a tremendous amount of brainpower put into optimization.
AMD's in an interesting position here because they've been making Nvidia GPU competitors for literally decades. Their architectures are actually quite similar, hence it's easier to 'emulate' Nvidia on AMD than pretty much anything else.
...That being said, the Chinese have made tremendous progress busting out of the Nvidia software ecosystem, hence these chips are actually being used for real work.
I remember when you needed a VGA video card to play Space Quest
-
Sucks you can’t get Chinese kit in USA though. At least I’ve never seen it available; not sure if it’s outright banned or US suppliers just don’t carry it
Huawei was uniquely, specifically, forced out of the US market around the time they were completing for 5G Tower standards.
