[HN Gopher] OpenAI is exploring making its own AI chips
___________________________________________________________________
OpenAI is exploring making its own AI chips
Author : rasbt
Score : 95 points
Date : 2023-10-06 12:47 UTC (10 hours ago)
(HTM) web link (www.reuters.com)
(TXT) w3m dump (www.reuters.com)
| makestuff wrote:
| Meta was rumored to want to layoff its chip fab team for VR. I
| wonder if OpenAI will work out a deal with them.
| drexlspivey wrote:
| Masayoshi Son is funding them so you know they're gonna lose a
| ton of money
| [deleted]
| jedberg wrote:
| My guess is that this leak is in response to the announcement
| from Anthropic that they will be using Amazon's custom AI
| silicon.
| Oras wrote:
| It means Microsoft is involved too. From consumer perspective,
| following the performance of Apple silicon, I am excited for this
| news.
| vhiremath4 wrote:
| It's interesting to think, when a company starts to vertically
| integrate, how deep do you go?
|
| Seems like OpenAI is exploring its own devices/OS as well, which
| makes sense to me, but it's a vertical integration bet. This
| seems to be another big bet, but they could benefit from having
| their own optimized chips regardless of whether the device/OS bet
| wins out.
|
| Extremely exciting times for OpenAI!
| theptip wrote:
| > how deep do you go?
|
| As far as gives you a competitive edge! For AI, to win you need
| better data and compute than your competitors.
|
| Going up the stack to consumer devices seems like a somewhat
| speculative move, though I understand the underlying desire to
| secure a data moat.
|
| Going down the stack to chips makes a lot of sense; if you can
| secure an edge in compute efficiency then you will beat anybody
| that doesn't have substantially more data than you do.
| hashtag-til wrote:
| Very exciting (from an engineer point of view), but there is
| much much more in a chips/devices/OS than "AI". It's a trap!
|
| To me, it seems like a distraction for them to go into devices
| stuff, rather than making their stuff so relevant that device
| vendors (Android/iOS) can't ignore. At the moment they can
| because they got good enough competitor solutions.
| baq wrote:
| Depends what they've got under the hood.
|
| If they found a theoretical way to infer something like
| GPT-3.5 without using so much RAM and can build a chip which
| makes this feasible in laptops or (holy grail) phones,
| they've got their moat for a 12-24 months, possibly more if
| they manage to patent it. Big if though.
| leetcodesucks wrote:
| [dead]
| cpersona wrote:
| When building out an initiative like this, how do companies avoid
| IP issues? They are looking to build technology that competes
| with the best in class to make it worth the effort without having
| to reinvent the wheel.
| wing-_-nuts wrote:
| I said a while back that I expect the major cloud vendors (Azure,
| AWS, GCP, etc) to start trying to develop their own chips for AI
| work. Google already does to some extent with their tpus. At the
| very least, this is saber rattling trying to convince nvidia to
| lower prices.
| ma2rten wrote:
| Both Amazon and Google already do this, there are reports that
| Microsoft does as well.
| okdood64 wrote:
| Why Google to only some extent?
| diggan wrote:
| Apple as well, if I understand correctly what the "Apple Neural
| Engine" is all about.
| cmrdporcupine wrote:
| Wish I could buy stock in Tenstorrent.
| trident5000 wrote:
| The economy is in an interesting place when in house chip making
| efforts or startups are popping up now. It used to be a task like
| landing on the moon. Still a difficult effort but it looks like
| this industry is expanding. It speaks to the rapid nature of
| technology in general where insurmountable tasks over time become
| closer to trivial.
| hamilyon2 wrote:
| I think that endgame for generative language models is models
| embedded directly into chips. Computers that run english language
| instead of machine code and for which CPU, GPU and what is
| currently known as PC is more like a peripheral IO device.
| intrasight wrote:
| Such a model could not be updated nor learn. Even very simple
| biological organisms can learn.
|
| The end-game is growing brains ;) (only partially kidding)
| layer8 wrote:
| That might not be a serious issue if we keep updating our
| personal hardware at the present frequency. So you'll be able
| to decide whether to update to the new iPhone with English
| 4.7, or rather stay with English 3.5 for another year.
| dragonwriter wrote:
| > Such a model could not be updated nor learn.
|
| It might have some limited ability for updates if the
| hardware had the model code but the weights were in memory
| that was updatable.
|
| It might do in-context learning even without that.
| intrasight wrote:
| > weights in memory
|
| That's basically how our neurons work. New neuron growth
| and connection isn't much of a factor in learning. Rather
| it's the synaptic restructuring (equivalent to AI model
| weights) that change relatively quickly.
|
| So we need to figure how how to "grow" mechanical brains. I
| envision this being done with a new generation of FPGAs
| tailored to this task.
| dragonwriter wrote:
| > I think that endgame for generative language models is models
| embedded directly into chips. Computers that run english
| language instead of machine code and for which CPU, GPU and
| what is currently known as PC is more like a peripheral IO
| device.
|
| That may be the endgame, but I think if it is there is a _long_
| time before attempts to jump to it aren 't going to fail like
| every high-level-system-in-hardware for other than very niche
| applications, because general purpose (comparatively, even if
| specialized for running AI models) hardware will be good enough
| that the value of being able to upgrade the models it is
| running will outweigh any marginal temporary edge that current-
| models-in-hardware have.
| mwbajor wrote:
| As soon as the investors and boardmembers realize that chips mean
| "hardware design" they will quickly put an end to any of these
| efforts.
| n40487171 wrote:
| I was just thinking the inference cost could be reduced by making
| hardware with less error correction in specific areas to get
| higher density, and let the NN work around the limitations.
| johngossman wrote:
| Given the cost of gpus it would be negligent if they weren't at
| least looking for alternatives. A story like this could also help
| negotiate prices with their suppliers. And everyone is looking at
| the success Apple had with custom silicon. But I suspect they'd
| prefer to partner or find alternative (cheaper) suppliers
| causi wrote:
| Am I the only person who thinks OpenAI just doesn't know where to
| go from here?
| vb-8448 wrote:
| I guess they are trying to find a path to profitability and
| scalability: the current setup will not to scale much further,
| and they need some more energy efficient solution.
| digging wrote:
| Only in the sense that no-one on earth knows how to get from
| here to true AGI. But they're making moves.
| mathisfun123 wrote:
| this isn't going to be a product, this is going to be for
| internal use only...
| theptip wrote:
| Are we watching the same company? They have been shipping new
| features like crazy. Code interpreter blew peoples' minds, they
| are training GPT-5 and I bet they are leaning into multi-modal
| more than with GPT-4V. Strategically multimodal is the big
| frontier on which they are going to continue expanding
| datasets.
|
| With that in mind, expanding their apps to ingest more audio
| and image data is an obvious strong move. (And you can see why
| a consumer device would help them get even more data from the
| real world, though it's less obvious to me that this is a win
| vs. just shipping apps.)
| ikeashark wrote:
| OpenAI confirms they aren't working on or training a a
| "GPT-5" https://techcrunch.com/2023/06/07/openai-gpt5-sam-
| altman/#:~....
| n40487171 wrote:
| It sounded like they weren't training but they were trying
| to figure out what gpt-5 structure, tooling, and training
| data will look like
| theptip wrote:
| 4 months is an eternity in AI right now. I would be shocked
| if they are not training a new GPT (at least in
| experimentation/research mode, if not full pretraining) -
| else what are their GPUs spinning on? That is not a capital
| investment you just let sit idle.
|
| But I concede that I don't have any concrete proof so I
| should modulate my certainty of tone.
| blovescoffee wrote:
| This was in June and they've continued to improve GPT-4
| since then. Surely they're building on GPT-4 to learn how
| to approach GPT-5
| bcjordan wrote:
| Still feels like they're laser-focused on achieving AGI/aligned
| ASI. Improving chips and evolving the UX feel like cohesive
| (possibly requisite) intermediate steps
| baq wrote:
| It might be that, or they might know _exactly_ where to go from
| here, just need the hardware.
| rvz wrote:
| They do, and it is to _accelerate_ their closed AI ecosystem
| and maintain their first mover advantage.
| hcks wrote:
| They stumbled upon a mass market product while releasing a
| rough research poc...
|
| If anything it vindicates even more their initial thesis about
| pursuing AGI as a business goal.
| aportnoy wrote:
| Note Sam Altman is a Cerebras investor.
| the-dude wrote:
| The first valuable comment imho, thanks!
|
| Let me add : there are numerous other AI chip startups.
| asciimike wrote:
| From the last thread on this
| (https://news.ycombinator.com/item?id=32610780):
|
| - https://sambanova.ai/ (Enterprise AI and dataflow-as-a-
| service for established models)
|
| - https://www.cerebras.net/ (AI accelerator, trying to
| compete with Nvidia)
|
| - https://www.graphcore.ai/ (Another AI accelerator company,
| UK based)
|
| - https://femtosense.ai/ (Sparse NNs on very low power chips,
| cool hardware and software challenges)
|
| - https://sima.ai/ (ML accelerators for embedded
| applications)
|
| - https://ambiq.com/ (Not AI, but low power chips for
| wireless using some fancy tech that reduces energy leakage)
|
| - https://www.esperanto.ai/ (RISC-V based Tensor computes
| chip, founded by Intel Hybrid Parallel Computing Vice
| President Dave Ditzel)
|
| - https://www.furiosa.ai/ (AI accelerator company which show
| good results in MLPerf benchmark)
|
| - https://groq.com/ (From the team that built the original
| TPU at Google)
|
| - https://lightmatter.co/ (Light tubes instead of copper)
|
| - https://www.untether.ai/
| rvz wrote:
| Exactly. We are now starting to slowly realize... [0]
|
| [0] https://news.ycombinator.com/item?id=35490837
| hef19898 wrote:
| Ah, so _that_ is how Sam and Co. cash out on the 10 billion
| from MS!
| seydor wrote:
| they should also buy reddit
| amelius wrote:
| I'm so tired of all this vertical integration.
|
| Can't we have hardware companies that make hardware, software
| (AI) companies that make software, and data companies (or
| government institutions) that run the software on the hardware
| and deal with our data?
| keenmaster wrote:
| I bet OpenAI is trying to figure out how to have self-improving
| AI which suggests modifications to the hardware that runs it.
| You'd need some hardware expertise in-house for that (in
| addition to software of course), though not necessarily a big
| chip company.
| blibble wrote:
| you'd need a lot more capital too
|
| a 3mn mask costs what, $20 million each time?
|
| using AI generated vomit for that would get expensive pretty
| quickly
| keenmaster wrote:
| They won't have trouble getting capital. It's open
| checkbooks for them all around.
| blibble wrote:
| even small chip manufacturing at the cutting edge has
| costs 2-3 orders of magnitude above what they're paying
| to rent some GPUs in azure
| hef19898 wrote:
| All those sweet Microsoft money has to somewhere, doesn't it?
| cmrdporcupine wrote:
| Every company is going to want to avoid paying rent on key
| infrastructure that they rely on for their business, thus the
| push to own their own IP and means of production in that
| regard.
|
| It doesn't work out all the time, for sure. In fact it probably
| fails more than it succeeds. But the motivation is pretty
| clear.
| llm_nerd wrote:
| This is especially true when you are beholden to a single
| supplier, with basically no competitive options.
|
| Right now nvidia completely owns the AI market, and is
| exploiting that absolute monopoly with ever escalating
| pricing, licensing and restrictive usage models. Everyone
| keeps trying to escape this -- see Tesla and their super-
| hyped and now apparently abandoned Dojo thing, while they put
| in their orders for tens of thousands of H100s -- but instead
| they keep being beholden to nvidia.
| screye wrote:
| When hardware companies fail to provide sufficient competition
| to an extortionate monopoly, the layer1 companies react after
| feeling the burn for too long.
|
| If Nvidia, Amd and Intel were in a battle for offering the best
| VFM, none of these companies would be hopping into hardware.
| Apple's strong commitment to chip making coincides with years
| of stagnation from Qualcomm and Intel.
|
| From my experience, companies love nothing more than a 3rd
| party that solves your problem for you, better than you and at
| a price that's easily cheaper than what I'd cost to build it
| in-house. This is especially true when the 3rd party product is
| an internal spec (gpu, cpu) rather than a competing platform
| (android auto)
|
| There is a reason car companies don't build their own speakers
| or tires....but still try to build their own UI (no matter how
| bad)
| artursapek wrote:
| ...why? Products are better when you can control the whole
| stack.
| amelius wrote:
| That may _seem_ to be an advantage. But because there is only
| 1 (or a few) companies now controlling the whole stack, you
| have less choice in the products you can buy. And since the
| incentives of the vendor may not be aligned with yours, which
| is increasingly the case if they are a monopoly, then the
| product is _not actually better_ from the consumer 's point
| of view.
|
| Don't like the way OpenAI treats your data, or how you can
| only run it in the cloud and not on an on-premises server? Or
| what dataset they used for training? You're out of luck!
|
| But if the market were more modular, and lots of small
| companies could use the same hardware in their products,
| you'd have something to choose from!
| jejeyyy77 wrote:
| that's what we have. Companies have discovered building their
| own is better.
| throwaway19423 wrote:
| Dealing with this professionally for DNNs. It just doesn't
| work. The large, important DNN models are so complicated, the
| toolchains for optimized execution don't do sane things unless
| you do some sort of vertical integration. The community tried
| with things like TVM, Halide, ONNX and others .. it is just
| crazy if you don't have a fully opinioned pipeline. Just my
| personal opinion.
| mensetmanusman wrote:
| Moores law is dead when it comes to power consumption, that
| means compute is getting more capital intensive.
|
| If you have an algorithm that works and need scale, you must
| vertically integrate to maintain an edge over those using more
| general compute architectures.
| [deleted]
| icapybara wrote:
| We do have those. It sounds more like you're saying that we
| shouldn't have vertically integrated companies.
| amelius wrote:
| The vertical integration allows them to corner the market and
| destroy the competition or prevent them from entering the
| market, which is bad for the consumer and bad for society,
| eventually.
| RicoElectrico wrote:
| I would go even further. I was wondering why most chip
| companies are so good at being mediocre. Like TI OMAP
| dropping out of smartphones and so on. But I think it's
| worthwhile to invert the question. It is quite
| extraordinary in the silicon space for there to be 1 clear
| winner like Nvidia or Qualcomm (or Intel not so long ago).
| So much so, that we can assume they got there by anti-
| competitive means and rent-seeking measures.
| mschuster91 wrote:
| > I was wondering why most chip companies are so good at
| being mediocre.
|
| Because anything to do with hardware, _particularly_
| anything with silicon, has immense startup costs.
| rootusrootus wrote:
| If we're making bets on OpenAI vs Nvidia in the hardware
| space, I know who I'm picking.
| icapybara wrote:
| That's a broad claim, you have to prove your case. You're
| asking for a ban on vertical integration as a business
| strategy.
| notaustinpowers wrote:
| Vertical integration is already under scrutiny by the
| antitrust folks. Even more so now after everything Google
| was able to get away with.
|
| OpenAI wanting to vertically merge to make their own AI
| chips may seem harmless enough (it's a good business
| move, we can cut expenses)! But we can't forget that Sam
| Altman just a few months ago told Congress he supports
| making an organization that companies need permission
| from to being creating/utilizing advanced AI systems. And
| he's such a kind man he's willing to lead that
| organization himself.
|
| Obviously someone integrating the chips to train AI, and
| having the ability to approve/deny his own competition is
| a huge red flag.
| stale2002 wrote:
| Well kind of. You are ignoring the elephant in the GPU
| hardware room right now, which is NVidia.
|
| NVidia has a near monopoly on the AI hardware market
| right now, so some vertical integration of alternative AI
| hardware doesn't seem nearly as big if a deal if it is
| needed to fight that current monopoly.
| notaustinpowers wrote:
| That's true, but again, him attempting to put himself as
| the gatekeeper of who can and cannot train/utilize
| advanced AI systems at scale gives him (almost)
| unilateral control over Nvidia's own AI chip
| manufacturing business. Deny startups the ability to
| train AI systems, and you deny Nvidia the opportunity to
| sell them their chips. Nvidia loses that revenue stream
| and stops producing AI chips due to high production costs
| and "lack" of demand. Ultimately leading to a market
| where only the obscenely wealthy companies who can
| manufacturer their own in-house chips can train AI, and
| even then, Sam Altman can deny even that.
| hashtag-til wrote:
| Ironically, Open AI are going to face the same gigantic
| barrier to enter the market.
|
| As of today, I say if they go into consumer market they
| will fail.
|
| If they go into specialised server chips, then they would
| have a chance, with some sort of accelerator over some arm-
| based chip - similar to what Nvidia is doing with Grace.
| Still big money to be spent on supporting the existing
| ecosystem on their hardware.
| feoren wrote:
| It's lack of anti-trust laws and (more importantly)
| enforcement that allow companies to corner the market.
| nVidia and ARM _should_ be worried about competition from
| OpenAI and Google: that 's the good kind of competition
| that we want. If only "hardware companies" are legally
| allowed to make hardware, that _increases_ their moat, not
| decreases it.
|
| Let's instead go back to when we actually enforced the
| anti-trust laws that we have. That was nice.
| ysavir wrote:
| Would OpenAI be competing with them, though? If they just
| manufacture their own and use their own, but don't sell
| them, then nVidia and ARM are just losing a client, not
| getting competition.
|
| Ideally what we would see is OpenAI investing in a new
| but independently operated chip manufacturer that makes
| chips to their standards. Though is it also possible that
| a chip of such standard would be so specialized that it
| wouldn't be usable by others? I'm not a hardware person,
| so that's a genuine question.
| JumpCrisscross wrote:
| > _If they just manufacture their own and use their own,
| but don 't sell them, then nVidia and ARM are just losing
| a client, not getting competition._
|
| That's competition. Apple Silicon completes with Intel at
| an ecosystem level.
| ysavir wrote:
| I'm very hesitant to define competition as "can't sell to
| them because they make theirs in house". Losing a sale
| isn't competition. Competition is someone threatening to
| take your clientele from you with their own offering.
| Apple can afford to design and use its own chips, but
| they're in a pretty unique position to do so. It's not
| like every other consumer of their chips is going to say
| "hey, if it's that easy, I'll do it too".
| layer8 wrote:
| It's inevitable(?) that the trend to vertical integration
| will continue, slowly pushing more and more non-vertical
| companies into irrelevance.
| scottiebarnes wrote:
| We had that, then we realized that companies who understand all
| the pieces lead to better user experiences, which is why we all
| have MacBooks and iPhones.
| e12e wrote:
| And Next cubes and UltraSPARCs? Or maybe vertical integration
| isn't a silver bullet?
| slg wrote:
| This is a funny example because Apple has disproven this
| logic just as often as they have proven it. Carplay and the
| App Store are two examples. People generally prefer the
| software Apple makes over the software car manufacturers
| make. People also like to install their own software on their
| devices beyond the software that Apple makes.
|
| Just because vertical integration occasionally works doesn't
| mean it is actually good for the consumer.
| kajecounterhack wrote:
| > People also like to install their own software on their
| devices beyond the software that Apple makes.
|
| Omg this, can Apple please stop sabotaging Google Maps and
| Google Photos :(
| killerdhmo wrote:
| ... what is Apple sabotaging? Google intentionally holds
| back features on iOS...
|
| Source: former Xoogler PM
| catchnear4321 wrote:
| "good for the consumer" is as subjective as the consumer.
|
| the eu disagrees in that it views itself as representative
| of the consumer. similar to how us states set laws that may
| be more strict than others.
|
| get a large enough government of a populace with a large
| enough portion of the sales, and companies can be made to
| act. which can be good, but isn't a guarantee.
|
| just because companies can be forced to act doesn't mean
| the forced actions are actually good for the consumer.
|
| (plus a decent amount is political theatre.)
| rgrieselhuber wrote:
| It also creates walled gardens and reduced incentives to
| invest in user experience once those walls are built.
|
| I switched from Mac to Linux precisely for this reason and
| the biggest surprise has been that whenever I touch a Mac
| again it feels like poverty.
|
| Not because the UX is bad but because I know how the company
| behind it operates and I have zero trust for anything that
| happens on the machine.
| mschuster91 wrote:
| > It also creates walled gardens and reduced incentives to
| invest in user experience once those walls are built.
|
| Which is why the EU has decided to break up the walled
| gardens - although IMHO they could ramp up their efforts a
| bit.
| tw04 wrote:
| I dunno, the last two generations of MacBook pro pretty
| much feature for feature provide fixes for all the widely
| criticized issues of the last Johnny Ive models. Apple has
| plenty of faults but they seem to actually be listening to
| user feedback in some areas despite a captive audience.
| conradfr wrote:
| But you can't run Windows or Linux (not sure of the state
| of Asahi) on it anymore.
| homarp wrote:
| natively. But can you really blame apple for Windows not
| having an ARM version?
|
| https://support.microsoft.com/en-us/windows/options-for-
| usin...
| Rohansi wrote:
| Windows does have an ARM version. You can even run it on
| the Raspberry Pi 4. That article is just about Apple's
| hardware specifically.
| lghh wrote:
| > reduced incentives to invest in user experience
|
| Using my Apple Silicon Macbook is a way better user
| experience than my Intel Macbook ever was. It certainly
| feels like they invested in user experience well after the
| walls were up around the garden.
| tyre wrote:
| > It also creates walled gardens and reduced incentives to
| invest in user experience once those walls are built.
|
| > Not because the UX is bad
|
| I don't see the connection. The first sentence says that
| walled gardens create bad UX, but Macs are the premier
| mainstream walled gardens and you don't find their UX bad.
| amelius wrote:
| Can you give some other examples besides Apple? One anecdote
| does not make a theory.
| boplicity wrote:
| I have a windows laptop for 1/3rd of the price, and am
| perfectly happy with it. (After switching from a Macbook
| Pro.)
| [deleted]
| btbuildem wrote:
| This makes a lot of sense. When ChatGPT initially broke the mold,
| I was hoping someone would find a way to repurpose all the
| silicon the crypto-bros' nonsense has commandeered -- alas, the
| problems are too different.
|
| Making specialized chips to run LLMs is the logical next step.
___________________________________________________________________
(page generated 2023-10-06 23:02 UTC)