[HN Gopher] ChatGPT turned generative AI into an "anything tool"
___________________________________________________________________
ChatGPT turned generative AI into an "anything tool"
Author : CharlesW
Score : 78 points
Date : 2023-08-23 14:01 UTC (8 hours ago)
(HTM) web link (arstechnica.com)
(TXT) w3m dump (arstechnica.com)
| yenwodyah wrote:
| When you have a hammer...
| Someone1234 wrote:
| Although let's see if it is sustainable.
|
| I pay for ChatGPT Plus but most people aren't, and from what I've
| read they're losing money like crazy. We may wind up seeing this
| as the technical high point before the various competitors cut
| back on expensive hardware/energy usage for the free tier, and
| the quality of the responses takes a dive.
|
| This might sound a bit conspiratorial so apologies for that but
| back when the competition for Google Assistant/Siri/Cortana/Alexa
| was super hot, the responses to the various voice assistance was
| almost eerie in that it could infer what you wanted/needed. Then
| as things cooled off, they got gradually dumber/worse every year
| since then. They're legitimately bad today (Siri in particular,
| but even Google Assistant is much worse than itself from back
| then). I suspect it is because hardware costs were too high so
| they found cheaper models that could run on a potato/locally.
|
| From what I've read to get ChatGPT responses in the times it
| currently takes, they may need to be running something like two
| 3080 GPUs, 64 GB of RAM, and a high end CPU, with the power draw
| associated. So to make the economics work (even with ad revenue
| for example), either a technological breakthrough has to occur to
| make running today's models much cheaper OR they will cut back
| making the responses objectively worse (they arguably already did
| this once with a 3.5 model change).
| malwrar wrote:
| The only moat openai has is their financial resources to pay
| for training and their vast troves of data scraped before
| reddit/twitter/etc started trying to lock things down. I think
| at this point there are enough people knowledgeable and
| interested in open LM technology that, if openai begins to
| degrade in quality, we won't lose this technology forever.
|
| I've personally been obsessively reading papers about new
| models and transformer modifications, and having succeeded in
| implementing some I think the experience has taught me that
| this tech is probably where web was in the 90s: easy in
| concept, expensive and hard to scale due to a few missing easy
| pieces thatll come later.
| viscanti wrote:
| > The only moat openai has is their financial resources to
| pay for training and their vast troves of data scraped before
| reddit/twitter/etc started trying to lock things down.
|
| Do we know that there's zero benefit from the user generated
| data they're getting now? They know when someone clicks the
| button to regenerate, and in theory they can have GPT4 review
| all the responses and classify if they're good or not. I
| don't know how beneficial that data is, so I'm curious if
| it's been proven to be completely worthless or not. If it's
| valuable at all, and openai is using it to continually
| improve the performance of ChatGPT, then maybe it will be
| difficult for a competitor to ever get traffic and data to
| their alternative.
| malwrar wrote:
| I don't know about "zero benefit"--certainly their access
| to customer queries gives them a gods eye view into how
| people are using their LM implementation and to some degree
| could probably be used to fine-tune their model--but I
| doubt it gives them a leg up that couldn't be matched
| through the sheer scale of academics and volunteers who
| contribute to open solutions & research. Maybe we invent a
| method for directly fine-tuning based on user interactions,
| or we discover that there's some shelf where further fine-
| tuning only confuses the model, or discover some magical
| architectural modification or dataset that enables high-
| quality interactions. We're still at the stage where basic
| modifications to the model breaks seemingly insurmountable
| problems. ALiBi for example swaps in a new attention
| mechanism, literally just a matrix addition operation, and
| found that it enables inference far past the context limit.
|
| I'm ultimately optimistic, their data is valuable but I
| don't think it's insurmountable.
| ColonelPhantom wrote:
| One thing about LLMs is that from what I know, batching is very
| effective with them, because to generate a token you need to
| basically go through the entire network for just that one
| sample. If you batch, you still need to stream the entire
| network, but that doesn't get any more expensive; you use each
| piece of data more often on hardware that used to just sit
| idle.
|
| So I assume that at an OpenAI-scale, they're more than able to
| batch up requests to their models, which gives a tiny latency
| increase (assuming they're getting many requests per second),
| but massively improves compute utilization.
| Madmallard wrote:
| We need a strong localized version at least for software
| development that is free as soon as possible. It is such a
| competitive advantage there's no way it will stick around
| forever for as cheap as it is.
| spott wrote:
| If it was just computer that caused the various assistants to
| get bad, then why haven't they gotten good again?
|
| Google assistant came out in 2016! That is 7 years ago! Since
| then, GPU compute in flops has 5xed... for single precision. If
| you include half-floats (not a thing back then), it is 75x the
| single precision flops.
|
| I think the reason that assistants have "gotten bad" is a
| combination of edge case controls (avoiding politically
| problematic results), increased capabilities mean more
| opportunities for failures, and their failures are amplified,
| and their successes ignored (in other words they aren't
| necessarily worse, we just expect more out of them).
| assistantist wrote:
| If it was just a matter of flops then AI would already be
| driving cars and making door to door food deliveries. The
| problem is neither the energy expenditure nor the compute
| capacity. No one knows what the proper architecture should be
| for general purpose intelligence assistants so everyone has
| to fine-tune the existing models on their specialized use
| cases and even then it's a hit or miss depending on how much
| training data you have available.
|
| No one currently has any ideas on how to deploy auto-adaptive
| ML software that continuously adapts to new data so the
| industry has settled on deploying snapshots and then hoping
| the data set that was used for the snapshot is going to be
| good enough for most use cases. It seems to work well enough
| for now and since Facebook has decided to open source most of
| their work someone in the open source community might figure
| out how to continously update these models without
| deterioration in output quality.
| spott wrote:
| I think you are missing my point.
|
| The OP said Siri et al. got worse because their creators
| optimized them to reduce compute costs. I'm saying that if
| that were true, then they should be at least as good as
| they were because you can get tons more compute for cheaper
| these days. So it isn't compute that makes the various
| assistants "worse" than when they were introduced.
|
| I made no statements about their quality in general, just
| their relative quality compared to when they were released.
| thfuran wrote:
| The difficulty of improving AI capability towards AGI isn't
| really related to a purported worsening of an already-
| implemented system for financial reasons.
| financialisty wrote:
| If money is the bottleneck then we will never get AGI. A
| superior machine intelligence would have no use for
| money.
| steno132 wrote:
| It's sustainable. Llama.cpp is orders of magnitude more
| efficient than Llama because it's in Rust.
|
| OpenAI hasn't even touched low level languages yet.
| [deleted]
| wongarsu wrote:
| So far OpenAI's modus operandi was "does it get better if we
| make it bigger". But now we've reached "good enough" (arguably
| already with GPT3.5, as evidenced by free ChatGPT running on
| GPT3.5-turbo).
|
| There has been promising work about making models smaller
| without losing performance (e.g. by training them longer), or
| quantizing the weights (llama.cpp etc) to make the model
| cheaper to run. I believe we will see a lot more focus on this
| over the next couple years, and given how there's been
| comparatively little work done in this direction I wouldn't be
| surprised by 10x or 100x efficiency gains.
| sharemywin wrote:
| let's not forget about good old fashion moore's law. which I
| believe in GPUs is even faster. and I'm not sure if some of
| these exotics Hardware architectures will pane out or not
| either.
| SubiculumCode wrote:
| efficiency efficiency efficiency will be the call. How do we
| reduce model size without losing ability. Can hardware be
| designed to run a particular set of weights efficiently?
| theRealMe wrote:
| I don't want to make unsubstantiated guesses, but we should
| remember that GPT3.5 was originally just GPT3.5 and when they
| came out with turbo it not only sped it up but also reduced
| the cost (to us) by like 10x of what it used to be if I
| remember right. So they are (or at the very least "were")
| working on and succeeding on reducing its cost to run.
| unshavedyak wrote:
| On the plus side i think at-home LLMs will continue to get a
| lot better. ChatGPT 3.5/4 both seem in reach for Llama2 and
| seeing all the amazing fine tunings that were available for
| Llama1, i suspect we have a very functional future of at home
| LLMs.
|
| I say functional because i don't think we've seen the larger
| reasoning skills scale well for smaller deployments, so that
| may be out of reach. But i think a future were we run something
| far better than Siri, entirely at home, hooked up to our home
| APIs in a functional spoken interface seems in reach and
| awesome (to me hah).
|
| I look forward to hooking all my at-home cameras up to a voice
| recognition, image recognition, and LLM detection suite and
| just speak aloud and have the AI-suite "do stuff". It sounds
| really, really fun to me.
|
| If it's cheap it'll be slow, but imo that's okay. For home use
| i think these systems can be the glue that tie our
| DBs/Ingestions/etc together, rather than the big-brain that
| knows everything.
| NBJack wrote:
| Except for the part Llama 2 weights are still being gated by
| Meta. There are certainly ways around this, but officially
| this hasn't even changed, even for Llama 1. I'm not certain
| Meta has any real incentive to do so. This arguably puts a
| bit of a damper on home solutions.
|
| Falcon looks very promising, but we seem to be at the early
| stages of its release, and I haven't seen much validating the
| claims made about it.
|
| I'd love to host an in-home alternative to Alexa, but I don't
| think I have the time to train a replacement on my own.
| Here's hoping this changes
| JimtheCoder wrote:
| "I look forward to hooking all my at-home cameras up to a
| voice recognition, image recognition, and LLM detection suite
| and just speak aloud and have the AI-suite "do stuff". It
| sounds really, really fun to me."
|
| Alternatively, this sounds like the starting plot to a
| crossover sci-fi/horror movie...
| dandellion wrote:
| The door refused to open. It said, "Five cents, please."
|
| He searched his pockets. No more coins; nothing. "I'll pay
| you tomorrow," he told the door. Again he tried the knob.
| Again it remained locked tight. "What I pay you," he
| informed it, "is in the nature of a gratuity; I don't have
| to pay you."
|
| "I think otherwise," the door said. "Look in the purchase
| contract you signed when you bought this conapt."
|
| In his desk drawer he found the contract; since signing it
| he had found it necessary to refer to the document many
| times. Sure enough; payment to his door for opening and
| shutting constituted a mandatory fee. Not a tip.
|
| "You discover I'm right," the door said. It sounded smug.
|
| From the drawer beside the sink Joe Chip got a stainless
| steel knife; with it he began systematically to unscrew the
| bolt assembly of his apt's money-gulping door.
|
| "I'll sue you," the door said as the first screw fell out.
|
| Joe Chip said, "I've never been sued by a door. But I guess
| I can live through it.
|
| -- Philip K. Dick, Ubik
| jstarfish wrote:
| They never read Demon Seed.
| renewiltord wrote:
| Of course it does. That's the structure of all sci-fi
| horror https://dresdencodak.com/2009/09/22/caveman-science-
| fiction/
| sharemywin wrote:
| isn't ChatGPT just an investor demo anyway?
|
| I doubt they run out of cash at least until they fund building
| GPT 5. If it flops then who knows.
| supportengineer wrote:
| I asked it to plan a date night in my city. What it came up with
| was... not terrible.
| TySchultz wrote:
| I've been building a daily news digest with intent of learning
| something new and creating something cool.
|
| Although AI is the heart of the product, I have noticed that I
| have been using it as an "anything tool". Though the articles
| talks about more broad use cases like hardware and robotics, I've
| found its an anything tool for most of the boring use cases.
|
| Using python for the first time in a while, ask GPT. Need a tweak
| on the app store copy, ask GPT. What are the top news sources,
| ask GPT. Data formatting & manipulation, save 5 minutes ask GPT.
|
| https://apps.apple.com/us/app/quill-news-digest/id1669557131
| josefresco wrote:
| Tried using Bing ChatGTP to create a logo for me because I saw a
| cool demo on Instagram. It couldn't write text correctly and I
| tried numerous times. Not only that, the logos were terrible and
| didn't "mimic" the image I uploaded as a guide (which was in the
| demo) Today I needed to transcribe an MP3. Nope, can't do that
| either.
|
| However on Monday I had it write some PHP and that was neat!
|
| 50/50 for me.
| CamperBob2 wrote:
| For logos you'd be better off with DALL-E 2, no? How do you
| even get a useful graphical logo out of a text-based language
| model?
| josefresco wrote:
| Bing chat uses DALL-E for image generation. The results were
| complete gibberish. Characters that looked like an alien
| alphabet and the graphic was also so abstract to not
| represent anything.
| vunderba wrote:
| Generative image systems are notoriously poor at producing
| legible text without significant rerolls. You'd be far
| better off generating a textless logo and then layering
| text afterwards using PS, Photopea, Krita, etc,
| kenjackson wrote:
| Transcribing an MP3 is a solved problem, in some contexts. What
| were you doing to make this a fail?
| josefresco wrote:
| Gave Bing Chat an MP3 URL and it just refused. I know I can
| do this with software on my PC, or maybe there's an online
| service where I can upload but I was hoping this would be
| something Bing Chat/ChatGTP would excel at. I tried first
| giving it the URL of the article where the MP3 was embedded,
| and then tried a direct link.
| BoorishBears wrote:
| I'm still lost on how you reached the conclusion Bing Chat
| should do be able that? Does MS imply it can do that
| somewhere?
|
| Did you assume Whisper is from OpenAI => OpenAI is related
| to Microsoft => Bing Chat can transcribe MP3s?
|
| I'm really just lost on the thought process here.
| lakomen wrote:
| I'd like to have LawGPT, where it reads all laws of a certain
| country, and I can ask it things like, "what are my options when
| the neighbors tree crosses over into my property?"
| party_zebra wrote:
| I bet we're going through the cycle netflix went through.
|
| Cable was fractured and expensive, then netflix made it cheap and
| monolithic, then everyone realized there was money to be made in
| this new paradigm, then there was a legal fight over usage
| rights, then it got split up and partitioned into a bunch of
| different services, the combined cost of which is pretty
| expensive and individually each service kind of sucks.
|
| So I'm betting we're a few years away from stackoverflow,
| wikipedia, imdb, goodreads?, etc all having their own ai
| interfaces.
|
| Fine by me. chatgpt is a way better user experience than
| stackoverflow, and I'd probably use the wikipedia one, don't
| really care about anything else. It being an "anything tool" is
| kind of a novelty, I liked that I could make it "write an
| advertisement for a housecat in the style of a ford truck
| commercial, and use plenty of made up trademarks and patents for
| normal cat behaviors/features", but then lost interest when it
| refused to do the same thing for boobs.
| og_kalu wrote:
| Large Language Models learn to complete arbitrary token
| sequences, not just about language.
|
| https://arxiv.org/abs/2307.04721
| atemerev wrote:
| This is what we used to call an "AGI" back in 2013-2014.
| Generalized artificial intelligence. An AI which is general
| enough to be used for diverse tasks in multiple fields, unlike
| specialized AIs of that time (e.g. for image recognition or self-
| driving cars). Nothing more than that.
|
| Now the goalposts are moved way forward, and now "AGI" is
| something that have conscious, with superhuman intelligence, and
| probably evil intent. But back in a day, AGI was just what we
| have today -- an universal chat bot.
|
| We live in the magic land.
| danpalmer wrote:
| This doesn't match what I've seen of usage of the term AGI
| since ~2010. I think we've "solved" chatbots more than was
| imagined at the time, and that involves being able to produce
| readable and arguably useful answers about a range of topics,
| but we haven't got game playing sorted yet, we don't have
| proactive models, and there's nothing in these models for
| movement or robotics (which until recently has been considered
| an important necessary step).
|
| We have taken a noticeable step forwards towards AGI in the
| last few years, but AGI is still a long way off.
| atemerev wrote:
| Games are solved since AlphaZero. Just set up an adversarial
| model, and voila. Any game, any conditions.
|
| Proactive models are simple -- just generate the "internal
| monologue" periodically from the system prompt, mix it with
| the compressed state, then feed it to another model (the
| outputs are used as state updates). I think this is how human
| agency works as well.
|
| Movement is a different story (LLMs do not work here, except
| as planners), but it pretty much solved too. Robot hardware
| is too expensive, though, and it is not clear how to power it
| autonomously.
| danpalmer wrote:
| AlphaZero is amazing, but it only plays 2D games with well
| defined win conditions.
| atemerev wrote:
| Yes, like military strategy.
| danpalmer wrote:
| Indeed, this is something that neither AlphaZero nor
| anything else has managed to do at anything approaching
| human level.
| lucubratory wrote:
| You really shouldn't say that so confidently. I know for
| a fact that at least in the US military (well, DoD
| civilian analysts), they're using models like AlphaZero
| for theatre-win condition modelling. It's probably being
| used because it allows analysts to find better solutions
| and they probably started using it after winning wargames
| with it - both guesses on my part, all I know for sure is
| that they are using the tech to model westpac.
| lucubratory wrote:
| "there's nothing in these models for movement or robotics"
|
| You should read the article. It talks about exactly this use
| case, using an LLM as part of robotics control systems.
| og_kalu wrote:
| It's not just about language.
|
| https://general-pattern-machines.github.io/
|
| >but we haven't got game playing sorted yet
|
| 4 can play chess, Minecraft fine. So I imagine a lot of games
| fall down to data.
|
| >and there's nothing in these models for movement or robotics
|
| The point is that there doesn't need to be.
|
| There are many projects using LLMs to pilot robots off text
| prediction ability.
|
| https://tidybot.cs.princeton.edu/
| https://innermonologue.github.io/
| https://www.deepmind.com/blog/rt-2-new-model-translates-
| visi...
|
| I think most from the 2000s would be calling what we have
| today AGI.
| dr-detroit wrote:
| [dead]
| Sohcahtoa82 wrote:
| > But back in a day, AGI was just what we have today -- an
| universal chat bot.
|
| I don't know anybody that used AGI to mean that. I've always
| thought of AGI as an AI that can learn how to perform tasks
| with very simple instructions, rather than spending weeks
| training a neural network on a data center full of GPGPUs.
|
| Basically, my idea of AGI has always been what you're calling
| the "new" definition of AGI.
| Kerb_ wrote:
| Remember that people have decades of generalized training
| before becoming able to perform tasks with very simple
| instructions. I would consider years of pre-training to be
| acceptable if the model is usable and trainable quickly in
| practice as well
| ChatGTP wrote:
| Mostly "training" to do abstract things, not really for
| survival or fun.
|
| I could catch fish since a fairly early age, mostly because
| I enjoyed it.
|
| At about age 12 I used to drive stolen cars around without
| any training at all except observation.
|
| It too me decades to become a corporate robot though :)
|
| I think training is a weird word, life is an experience,
| not just training. I think your comment demonstrated the
| limitations of language to describe things.
| danielbln wrote:
| Aren't you effectively describing few-shot learning in LLMs?
| Teach the model a new task by describing it and maybe give a
| few examples?
| e_y_ wrote:
| I would call a chat bot a specialized application. And I
| dispute that a chat bot was ever considered the end goal of
| AGI. The idea is that if we had AGI, we could build a chat bot
| with it to _test /demonstrate_ its general reasoning
| capabilities. Simply being effective at chatting is not
| sufficient to be called an AGI. And today's chatbots are quite
| stupid.
|
| Sure we're starting to give chat bots the ability to query
| information and even interpret user-provided images, but AFAIK
| the image interpretation is done using a different model rather
| than being an extension of the LLM.
| og_kalu wrote:
| LLMs aren't chatbots. That's just a nice consequence of what
| they can do.
|
| Their prediction abilities stretch beyond just language even
| if they're only trained on it.
|
| https://general-pattern-machines.github.io/
|
| There's nothing specialized about being able to predict
| meaningful completions from arbitrary token sequences.
|
| There's a whole lot that can be done with that even if they
| were constrained to language.
|
| https://tidybot.cs.princeton.edu/
| https://innermonologue.github.io/
|
| There's nothing stupid about GPT-4. There's no testable
| definition of general intelligence GPT-4 would fail that a
| chunk of humans also wouldn't.
|
| Vision understanding does not have to be a separate model.
| https://arxiv.org/abs/2306.14824
___________________________________________________________________
(page generated 2023-08-23 23:00 UTC)