[HN Gopher] A800 40GB Active Graphics Card
___________________________________________________________________
A800 40GB Active Graphics Card
Author : kristianp
Score : 43 points
Date : 2024-02-18 10:39 UTC (1 days ago)
(HTM) web link (www.nvidia.com)
(TXT) w3m dump (www.nvidia.com)
| jsheard wrote:
| This isn't new, it was announced a while ago. The A800 is a
| variant of the previous generation A100 chip which Nvidia cooked
| up specifically to sneak under the sanctions imposed on China,
| but with the US cracking down further they are now cutting their
| losses and selling them off in other markets.
| intpx wrote:
| Announced awhile ago but available now.
| jsheard wrote:
| True, and it's still in-stock if you happen to have PS15,000
| burning a hole in your pocket.
| brudgers wrote:
| For business at scale, a capital cost of 15,000 money units
| can be rounding error.
|
| And for some people -- though not me -- it is actually
| pocket money and less than a weekend in Vegas.
| vjjsejj wrote:
| r/iamveryrich leaking?
| karolist wrote:
| "some people have more money than you", still, 15k USD is
| a lot, even without this perspective.
| adastra22 wrote:
| I've got a fire-proof pocket you can put it in.
| verall wrote:
| "sneak under" vs "comply with"
|
| If the US did not want NV to sell any GPU to any Chinese
| company, they could make that the law instead
| jsheard wrote:
| Well the US did follow up by also banning the exports of the
| A800/H800 models tailored to precisely comply with the
| earlier sanctions, so presumably the intent was to stop
| export of GA100/GH100 silicon altogether and they didn't
| anticipate Nvidia rules-lawyering the sanctions by
| reconfiguring it.
| photonbeam wrote:
| They absolutely anticipate that, its just about how well
| the people actually draft regs, or not
| declaredapple wrote:
| > so presumably the intent was to stop export of
| GA100/GH100 silicon altogether
|
| They should have just written this in the regulations.
|
| Instead they're making nvidia waste resources making a
| product that complied with regulations.
| forgetfreeman wrote:
| I mean fuck around and find out. Folks want to slide
| right up to the edge of legal definitions that's fine,
| but playing victim when the powers that be see you do it
| and nudge the line closer to the spirit of the original
| intent is laughable. It's not like the concept of
| legislation was invented two weeks ago and folks are
| shambling around trying to figure out what it all means.
| declaredapple wrote:
| It's just frustrating for me to see people write rules,
| and then say "hey not like that!". It wastes everyone's
| time, the point of regulations is to draw the line and
| have no ambiguous meaning.
|
| It's like putting up a speed limit sign that says "55mph"
| and then ticketing them for speeding going "53mph".
|
| If they want people to go 35mph, they should just made
| the limit 35mph!
| shawnz wrote:
| Consider that regulations are hard to get right and all
| the people responsible might not have known in advance
| all the edge cases that were possible. After all, if they
| knew it in advance, the regulation would have been made
| before it ever became a problem in the first place
| oivey wrote:
| This is one of those situations where how the government
| works is not like your personal life experiences. Nvidia
| wasn't some anonymous individual out of hundreds of
| millions subjected to a capricious law. They were the
| direct target of the law, and almost certainly had a seat
| at the table when it was written. I'm sure it wasn't what
| they wanted, but they certainly had to know what the
| intent was, including backroom, private conversations.
|
| They thought the government wouldn't dare stop them. The
| government called their bluff.
| forgetfreeman wrote:
| I sympathize with your position but have you ever
| participated in any activity where large numbers of
| relatively intelligent people are incentivized to probe
| the boundaries of "permissible" behavior? Video game
| design and development would take a tenth of the effort
| if corralling users away from game-breaking edge cases
| wasn't required, and that's a zero risk low reward
| endeavor. Now just imagine what those same folks would
| get up to if there was a few billion in loose cash thrown
| into the mix. All of that is to say defining the
| boundaries of acceptable behavior precisely is easily one
| of the hardest things to get right, and usually
| impossible to get right the first time.
| adastra22 wrote:
| Eh, there's some reason to be annoyed here. They would
| have banned GPU export entirely, I'm sure, except for
| industry lobbyists who said that doing so would make
| Chinese companies fill the void in offering GPUs for the
| lucrative Chinese gaming market, then it'd be a short
| step from there to having created the capability they
| rely on the west for. NVIDIA, AMD, Nintendo, Sony,
| Microsoft, etc. all lobbied for this outcome saying,
| essentially, "let us keep the gaming market, but just
| outlaw export of GPUs for AI training: that will be more
| effective because then there's no market for a local
| competitor to serve."
|
| Then they went and made AI-trining equipment anyway that
| skirted the letter of the law.
| parineum wrote:
| I don't disagree with you but this is still very much on
| NVidia. They have the resources to contact the government
| and ask, "what if?" but they didn't because they were
| trying to pull a fast one.
| Salgat wrote:
| NVidia knew that was a possibility when they were working
| around the intent of the trade restrictions. The
| Department of Commerce was quite transparent in their
| intentions, and NVidia undermined that and the obvious
| happened.
| olliecornelia wrote:
| Aw, poor fuckin' NVIDIA.
| 2OEH8eoCRo0 wrote:
| That's not what the US wants. The US wants to sell to China
| but not boost their military.
|
| > U.S. Commerce Secretary Gina Raimondo, speaking in an
| interview with Reuters on Monday, said Nvidia "can, will and
| should sell AI chips to China because most AI chips will be
| for commercial applications."
|
| Gina Raimondo thinks it's sneaking
|
| > "That's not productive," Raimondo said. "I am telling you
| if you redesign a chip around a particular cutline that
| enables them to do AI, I am going to control it the very next
| day."
| halJordan wrote:
| Not particularly as you're ignoring the environment when the
| rules were made. When the rules were made they made them as
| constrained (we want that right?). So the remaining chips
| were the ones nvidia had two/three product segments down.
| Then nvidia introduced a new chip that was 99% instead of
| just selling the ones they had that were 75/80%. You can see
| how nvidia is participating in bad faith over this. Blaming
| the g-men for releasing constrained, market-aware rules is
| itself a bad faith participation.
| cherioo wrote:
| IMO, regulation is supposed to be "we think this is the
| line that makes chip too dangerous". And nvidia is
| complying with that. If the danger line is 80%, why not set
| the line at 81%
|
| The administration feels like it's throwing a tantrum about
| an extra 19% of performance. The fact that that even
| matters, or the premise that chip is dangerous, are both
| just funny (not so fun for nvidia)
| bee_rider wrote:
| If there's a line that actually makes product dangerous,
| the government should set it there. Designing laws they
| just-so happen to only hit specific SKUs seems like the
| original bad-faith act.
| declaredapple wrote:
| I don't get it, the rules were "Do not cross this line",
| nvidia, being a business that makes products that comply
| with regulations, made a product that stopped just short of
| the line.
|
| It's like putting up a speed limit sign that says "55mph"
| and then ticketing them for speeding going "53mph"
| parineum wrote:
| Nvidia clearly attempted to circumvent the _spirit_ of
| the law. The intent of the ban was clear and Nvidia
| ignored that.
|
| It's much less like the speed limit example you gave and
| much more like a parent telling one of their children to
| stop touching the sibling so they just hover an inch away
| from them saying, "I'm not touching you!". The parent is
| going to very obviously modify their rule immediately to
| curb this circumvention.
|
| If I had to guess, Nvidia hoped the regulations were just
| signalling rather than an actual attempt to prevent the
| specific hardware from getting to the Chinese.
| __loam wrote:
| It's not like that at all. It's more like "don't sell
| F22s to anyone else" then Lockheed makes a slightly
| slower and less stealthy export model to skirt the rules.
| It's obvious that Nvidia is acting in bad faith, and it's
| funny that the same people who said we need to maintain
| an edge over China are probably defending Nvidia doing
| this.
| declaredapple wrote:
| > "don't sell F22s to anyone else"
|
| What they actually said (abbreviated) was anything with
| more then
|
| > "600 Gbyte/s [memory transfer speed]" or 4800TOPS
|
| Page 103 https://public-
| inspection.federalregister.gov/2022-21658.pdf
|
| They set very specific performance restrictions, not a
| general "don't sell fast stuff plz"
| Salgat wrote:
| They're referring to the spirit of the law versus the letter
| of the law. NVidia knew they were not following the spirit of
| the law, and they understood that a result might be that the
| restrictions would simply be updated if they tried to
| undermine it.
| dheera wrote:
| NVIDIA's new GPUs have horribly confusing names.
|
| Are higher numbers better or lower numbers better? A6000? A100?
| A800?
|
| And H > A > V > K, I think? And there's no T100 for Tesla and
| J100 for Jetson? But there's a Tesla V100 (Tesla Volta 100) and
| a Jetson Xavier which is neither Jetson nor Xavier architecture
| but Volta architecture?
|
| Can we please just have monotonically increasing product names
| in terms of compute capacity and stop jumping around the
| alphabet?
|
| I know they name architectures after scientists, that's cool,
| but at _least_ they could do what Ubuntu does with release
| names and go in alphabetical order.
| justinclift wrote:
| (2023)
|
| As per:
| http://web.archive.org/web/20231103222724/https://www.nvidia...
| zingelshuher wrote:
| For those who want to run big models: AGX Orin 64GB is 'just'
| $2K. Not much processing power, but 64GB of shared memory is
| quite a lot for 2K.
| wmf wrote:
| A slightly more expensive Mac Studio is probably a better buy
| than Orin at this point.
| ComputerGuru wrote:
| Care to explain why? The Orin option seems massively more
| scalable and you can put in as beefy a machine as you want
| without paying the Apple tax which scales exponentially with
| machine specs.
| jjtheblunt wrote:
| exponentially is some exponentially scaled hyperbole.
|
| that said, i agree with your question!
| FlyingAvatar wrote:
| Not the OP, but I would suspect the Mac Studio will retain
| its value better than an application specific product.
| karolist wrote:
| You will be able to resell that 192G Mac Studio for 3-4k
| after a few years easily, which you've bought for 6k.
| There's nothing competing with Apple Silicon right now
| for local LLMs when you look at TCO.
| wmf wrote:
| I don't understand anything you're saying. Orin is not
| scalable at all and (IMO) overpriced given its slow CPU and
| GPU and it's limited to 64GB of soldered RAM while Macs can
| go up to 192GB. AFAIK Orin is limited to a special build of
| Ubuntu and has no community.
| abdullahkhalids wrote:
| How large models can this run?
|
| Generally, is there a resource that has a table of how much VRAM
| each of the open source models require, especially at different
| quantization levels? I am pretty confused about these Mixture of
| Experts models like Mixtral 8x7B.
| wmf wrote:
| I would expect a 40GB GPU to run 35B models.
| karolist wrote:
| 35B models need about 22G vram with Q6 quantization, you
| should be able to run 70B models with low Q
| eterevsky wrote:
| It looks like it's pretty expensive, but it's possible to buy A16
| with 64G of memory for around $3k
| olliej wrote:
| I recognize that these massive memory GPUs are for "AI"
| applications, but it really is amazing (exponential growth, I
| know, I know) when I think back to being at uni when I was super
| excited to get a Real(tm) High performance(r) GPU that had a
| truly massive 16Mb of ram :D (and a whopping 64Mb of system ram!)
___________________________________________________________________
(page generated 2024-02-19 23:02 UTC)