[HN Gopher] Nvidia predicts AI models 1,000,000X more powerful t...
___________________________________________________________________
Nvidia predicts AI models 1,000,000X more powerful than ChatGPT
within 10 years
Author : CharlesW
Score : 31 points
Date : 2023-02-25 18:01 UTC (5 hours ago)
(HTM) web link (www.pcgamer.com)
(TXT) w3m dump (www.pcgamer.com)
| fidgewidge wrote:
| In this case "powerful" means faster chips. It's not clear that
| this translates into 1,000,000 smarter given that current AI
| might end up bottlenecked on availability of text and information
| to train it on. I guess if you could speed things up by a
| million-fold though, you could have way bigger context windows
| and you could potentially set a LLM talking to itself as it
| explores a problem through web search, writing and running
| programs, etc. Exciting times.
| pixl97 wrote:
| If I had to make a guess, I would think that text based LLMs
| will run into that wall sooner than later. But the rapid
| increase in compute availability wouldn't limit us to text
| either. Also imagine the models working on image input, sound
| data, video, temperature, touch, and whatever other inputs you
| could encode and present to them. This brings it more in line
| with the human experience of inputs.
| weinzierl wrote:
| Not saying it could no be true, but that's exactly what a company
| like Nvidia would say. Do we have any reasonably unbiased info on
| what the boundaries of the current approach are and when we will
| reach them?
| rhtgrg wrote:
| "More powerful" here just means larger (another commenter says
| faster chips, but you don't even need that if you spend the
| next ten years training a model, although that could be a more
| accurate representation of what Nvidia is trying to say, since
| the alternative is a no-brainer). AI isn't really "smart," and
| it seems unlikely we're going to get there anytime soon (in
| other words, the "singularity" is still firmly out of our
| grasp). What we're seeing today is the fruit of training models
| with large swaths of data, and the limits of this approach are
| predictably based on the availability of such data and the
| resources required to convert them into these gigantic models.
| Framed in those terms, the "limit" we should be considering
| ought to be more about utility than how many resources we can
| afford to throw at models that might ultimately have no use.
|
| Discussions around these topics tend to get dramatized, though.
| In my opinion we haven't made a lick of qualitative progress in
| recent years in the "IQ" of AI, but we do get better at
| utilizing what we have, and making quantitative progress in
| building ever larger models.
| pixl97 wrote:
| Please lay out your definition of smart?
|
| For the task they are given, take textual tokens as input and
| make coherent output, the models seem 'smart enough' for me.
|
| The problem here is all the text in the world is still one
| dimension of information. The emergence of human
| intelligence, at least I believe, isn't from one dimensional
| information, but from being able to get feedback and prune
| the bits of information that are inconsistent with reality.
| There is no continuous feedback loop in our current LLMs to
| prune/down weight bad information in that fashion that the
| mind works in. There is also the "the LLM universe is what
| humans tell it", if we feed it bad data that's incorrectly
| weighted, then we'll get bad output. There is currently no
| scientific method model internal for the AI to test its
| predictions on reality.
|
| I believe all the above issues are solvable, but no idea on
| the time and resources necessary to accomplish them.
| rhtgrg wrote:
| > Please lay out your definition of smart?
|
| Firstly, I agree that "smart enough" is what we should
| focus on here, as I tried to emphasize with my focus on
| utility.
|
| Secondly, I'm assuming your intent is to ask me when I
| would consider an AI to be "smart" in the context of my
| comment. I think I should avoid that question, and perhaps
| even retract my statement. There is plenty of evidence that
| ChatGPT is smarter than _some_ humans. All of your
| 'problems', as you might realize, are equally applicable to
| some people.
|
| Putting _that_ aside, a model would need to at least be
| able to internalize any corrections made (by a user, and
| not necessarily only in the context of training) and
| reliably produce results consistent with such an
| internalization, for me to consider it smart- _er_ (i.e.,
| an improvement on today 's AI IQ). To be _smart_ without
| qualification, it would need to go further, into the realm
| of being able to hold opposing ideas while functioning
| consistent to a given user 's expectations (this one will
| be big, because it will act as a defense against trolls and
| misinformation). Bonus points if it's able to improve upon
| its own learning algorithms and ruminate on ideas
| (unsupervised learning that is inspired by previous input
| and not stochastic in nature) without necessitating
| constant external dialogue.
| pixl97 wrote:
| Ya, as Bing showed us AI rampency is an issue that we've
| not tackled even in a tiny prompt space. Going to be
| interesting to see how we figure out how to deal with
| that.
| CharlesW wrote:
| This is hyperbolic, of course. But it makes Kurzweil's 2045
| prediction of the singularity not as outlandish as it first
| seemed. https://www.smithsonianmag.com/air-space-
| magazine/reaching-s...
| oneoff786 wrote:
| Still outlandish. Not especially well defended either.
| junglistguy wrote:
| ok
| Temporary_31337 wrote:
| Translation: nVidia is talking up the hype that can increase its
| stock price (now that the cryptocurrency hype is over)
| A_D_E_P_T wrote:
| As Scott Alexander at ACX recently said:
|
| "AGE OF MIRACLES AND WONDERS: We seem to be in the beginning of a
| slow takeoff. We should expect things to get very strange for
| however many years we have left before the singularity. So far
| the takeoff really is glacially slow (everyone talking about the
| blindingly fast pace of AI advances is anchored to different
| alternatives than I am) which just means more time to gawk at
| stuff. It's going to be wild."
|
| Any model even just one or two orders of magnitude more powerful
| than ChatGPT is going to make for a wild future. 10^6x is hard to
| imagine. And the implications for the philosophy of mind are
| staggering.
| uejfiweun wrote:
| Gradually, then suddenly.
| sdwr wrote:
| _These are the days of miracle and wonder
|
| This is the long-distance call
|
| The way the camera follows us in slo-mo
|
| The way we look to us all_
| throwuwu wrote:
| It always seems slow until you're on the right hand side of the
| elbow
| abudabi123 wrote:
| Could Ai learn from 3D mapping daytime soap operas and trace
| character development along the narrative arc? Then it has to
| function in the real world.
| [deleted]
| epups wrote:
| There is not enough written text to scale a model like ChatGPT at
| even 10x its current parameters, so improvements will have to
| come from optimizations and/or longer training (at diminishing
| returns).
| motoxpro wrote:
| Enough data wont really be a problem.
| https://us.sganalytics.com/blog/2-5-quintillion-bytes-of-dat...
|
| I wouldn't be surprised if we create more data in one year than
| is in the entire internet from the start
| KeplerBoy wrote:
| quintillion bytes of social media probably have very limited
| value.
| motoxpro wrote:
| Really? All of humanity imparting everything they think and
| know publicly seems incredibly valuable. 100% serious.
| BooneJS wrote:
| If you're looking to make a tweet generator, sure. If
| you're looking to harness the power of facts, no.
| ThrowawayTestr wrote:
| How do you know that? There are thousands upon thousands of
| books that haven't been digitized.
___________________________________________________________________
(page generated 2023-02-25 23:02 UTC)