[HN Gopher] Using ChatGPT Plugins with LLaMA
___________________________________________________________________
Using ChatGPT Plugins with LLaMA
Author : Flux159
Score : 247 points
Date : 2023-03-26 15:32 UTC (7 hours ago)
(HTM) web link (blog.lastmileai.dev)
(TXT) w3m dump (blog.lastmileai.dev)
| la64710 wrote:
| This is true ,it is time to have open standard and protocols for
| inter model communication between models and between models and
| external systems like in case of agents. Without open and clear
| protocols we will not be able to completely realize the benefits
| of this revolution. There needs to be also clear legislative
| guidance on copyright , privacy and safety issues in the domain
| of AI. The government should create a committee for this and
| organizations like IEEE should produce RFCs.
| carlycue wrote:
| OpenAI with 400 employees are breaking Google's ankles on the
| court. What do you guys think? Is Google in trouble? Their entire
| business model is in jeopardy. If Google releases a oroduct as as
| good as ChatGPT 4, which is unlikely, they'll kill their revenue.
| If they don't release it, they'll gradually lose market share.
| They are in an unwinnable situation.
| Patrol8394 wrote:
| What's OpenAI long term strategic advantage? Google, has
| already shown they have something like ChatGPT. They just need
| a new CEO and to feel enough pressure to make it a company
| priority. I think it is just a matter of time and all other big
| players will catch up.
|
| The big problem for Google is not the tech per se, but to
| figure out how to make money out of it, without destroying
| their ads cash cow.
| [deleted]
| Me1000 wrote:
| > Something like ChatGPT Chatbots are not new, they've been
| around forever, so there's plenty of prior art for things
| like ChatGPT. What sets ChatGPT apart is the quality. And in
| that respect, Bard is a long ways away.
|
| >They just need a new CEO and to feel enough pressure to make
| it a company priority.
|
| Sundar made machine learning a company priority years ago.
| Employees were encouraged to take machine learning courses,
| and much in the same way Google made everything "mobile
| first" in the early 2010s, I believe I remember hearing about
| them making everything "ML first" in the early late 2010s.
|
| You can see their announcements at IO around their assistant
| which they presented as having the ability to call physical
| businesses, have a conversation with the person on the other
| ends, and book you an appointment/reservation.
|
| And as recently as ChatGPT3, Sundar declared a "code red" to
| respond to it.
|
| Google has been investing in this tech for a long time and
| it's been looking for ways to create products with it too.
| It's just that OpenAI leapfrogged them. I'm not saying they
| won't catch up, but they weren't caught flatfooted here.
| Patrol8394 wrote:
| Still, I don't see OpenAI having a strategic advantage.
|
| Until today, Google still has a defacto monopoly on search.
| Their secret sauce made them leader in the space and so far
| no other company has been able to come up with something
| better.
|
| ChatGPT does not have any special secret sauce, Google can
| just build something better. So if I have to bet, long
| term, I will still bet on Google. That said, it is also not
| obvious that leadership at Google will be capable of
| delivering. That's all different story.
| anon7725 wrote:
| Sometimes your strategic advantage is having an
| organization that is not dysfunctional and is primed to
| execute. Google's problems with follow-through are
| legendary and they have just come off a round of morale-
| sapping layoffs.
|
| The organization doing the cutting edge research and the
| one doing productization are often not aligned. Sounds
| like OpenAI is the first to have a critical mass of
| talent in both disciplines aligned in a startup like
| environment. Probably they have promised the researchers
| that papers will be allowed after a blackout period, but
| in the meantime here's $MegaBucks and the chance to be
| first to real world deployment.
|
| OpenAI has capital and revenue. The $20/month I'm paying
| them is a ridiculous no brainer - pays for itself in one
| work-related inference. One 30min session yesterday has
| me set up for the first half of my work week. It's truly
| incredible.
| Me1000 wrote:
| With the caveat that I'm not a machine learning expert,
| my understanding is that there can be a lot of manual
| tweaking that goes into building neural net models.
| Whatever OpenAI is doing with ChatGPT is not something
| Google has been able to replicate with Bard. And I don't
| think that's for a lack of trying. I don't mean to come
| off as a doomsayer for Google, I'm really not, but I
| don't think Google's success here is inevitable.
| rvnx wrote:
| It's not true, OpenAI has an insanely efficient research
| team, they even captured some of the original
| transformers team members, when these people had a choice
| between staying in Google or going to OpenAI.
|
| Also, they have a platform running with 50 or 100 million
| users and all these people are feeding real-world data to
| improve the model.
|
| They also have agility; also because they are not
| publicly-listed, they can take more reputational risk.
|
| Regarding secret sauce, doing LLMs is easy now, as
| everything is open-source and documented. At least for
| the main parts.
|
| However, doing LLMs that works really well is very
| difficult and the secret sauce/tricks/dataset that were
| used to refine the model are not public.
|
| Regarding Google Search organic, it's not that sure
| anymore, it was true before; but now it gets a bit
| painful to navigate among so much SEO spam.
|
| Nowadays, Bing organic search results are great (less
| spammy in my observations), and if you compare Google
| Images and Yandex Images, then Google is not so shiny.
| ericmcer wrote:
| I still feel miles away from trusting AI with my most ad-
| friendly queries. Things like: where should I eat? Who is a
| good plumber, etc.
|
| Maybe someone can explain to me how an AI would revolutionize
| that, it either requires I trust it's opinions or it does
| something similar to google: give me a list with stars and
| reviews and let me make my own choice.
|
| Or maybe people are lazy and in two years our phones will be
| telling us: "the best movie for you to watch right now is Creed
| 4, press here to buy a ticket"
| mynameisvlad wrote:
| I mean, just like any other advertiser, they can build/buy a
| profile of your likes and dislikes from your previous
| requests and an online data broker.
|
| With that information, they can then feed it through standard
| matchmaking algorithms (which is nothing new) to find the
| best X movies/restaurants/etc given your profile and any
| specific requests you give it.
|
| This seems perfectly reasonable, the only thing new here
| would be integrating existing search and recommendation
| behavior into the model, which doesn't seem difficult given
| there's a plug-in system which is meant to do pretty much
| that.
|
| You could probably build a proto-version of this by making a
| prompt that includes all the most relevant profile
| information about yourself and asking it to choose something
| for you based on those variables.
| ericmcer wrote:
| That behavior is what people dislike about Google though.
| Filtering and biasing search returns is a huge area of
| mistrust for them. OpenAI's fix to this is an even more
| restrictive and opinionated version?
|
| That is a total nightmare once they decide to monetize it.
| mynameisvlad wrote:
| That behavior is what _power users_ (read this audience)
| dislike about Google. They wouldn't have gotten to be a
| powerhouse in search and personalized results if the
| strategy didn't work for the mass market audience.
|
| The reason sponsored ads are first in the results is
| _because_ people tend to click the first result without
| thinking about the consequences of that first result
| being sponsored.
|
| ChatGPT might not go down this path, but it's almost
| certain _someone_ will, and more than likely that Google
| will do exactly that within Bard.
|
| Do you think that people won't appreciate an assistant
| that already knows everything about you and can
| conversationally recommend exactly what they were looking
| for? And do you think corporations won't capitalize on
| the clear opportunity that's there?
| londons_explore wrote:
| > If Google releases a product as as good as ChatGPT 4
|
| If Google can top OpenAI on product quality, they will easily
| maintain their revenue. It will be easy enough for them to tell
| their AI "when two products are almost equally relevant to a
| user, recommend the product from whomever pays us the most for
| a referral".
|
| They can probably integrate that into their current adwords
| ecosystem as pretending that the AI mentioning a product is
| equivalent to an adwords impression, and a user click through
| is equivalent to an adwords click.
| qwertox wrote:
| This week I created a Microsoft account just to check how they
| are integrating GPT into Bing, and while they made me believe
| that I was using a Chat-enabled Bing, it was worthless for me.
|
| I basically wanted to know how I can do raycasting in Three.js,
| but after a lot of using ChatGPT for trying to solve my issue,
| I learned that I can't do what I want by using the normal
| raycaster integrated Three.js: Intersect a geometry which has a
| displacement map on it.
|
| ChatGPT failed to understand that the raycaster works on the
| CPU, but the displacement map of the material is applied on the
| GPU side, so the displaced geometry won't be used, only the
| original one. It managed to explain this to me, that this was
| not possible, but each sample code repeatedly did as if it was
| possible, until I gave up.
|
| Then I created the Microsoft account and started to ask for
| solutions, and it was the most useless garbage, despite of the
| web claiming that it is using GPT-4.
|
| In my eyes MS has failed at integrating a chatbot; maybe it's
| ok for cooking or having fun, I haven't tried that. And OpenAI
| has nothing else but a chatbot and other nice AI. Let someone
| better come and OpenAI will be a remarkable entry in the
| history books (first popular AI application) with a final entry
| that OpenAI got acquired by Microsoft.
|
| Let's see what Google makes out of it.
| rvnx wrote:
| I can tell you what will be the answer of Bard:
|
| "I am just an AI language model, I can't help you"
| j16sdiz wrote:
| ...instead of pretend knowing the answer and splitting out
| nonsense
| mardifoufs wrote:
| The problem is that it still spews tons of non sense. It
| literally makes up imports when commanded to use a
| library/package even if it "knows" said package,
| considering it itself suggested to use it in the first
| place!
| nullandvoid wrote:
| Partial information at-least gives me something to
| correct / some hint of the problem space to explore
| further.
| qwertox wrote:
| ChatGPT's partial answers were helpful, yet the whole
| process was time-consuming. I guess that googling for it
| would have taken at most 5-10 minutes, but I wanted to
| see how ChatGPT behaves.
|
| But Bing on the other hand, it didn't even bother to spit
| out ChatGPT-like sentences but only pointed me to some
| non-helpful Stack Overflow entries.
|
| I just re-logged-in into Bing to search for my query, I
| found it in the history and re-visited it, now it is
| handing me out an answer which looks like it was
| generated with ChatGPT, but it's still the same buggy
| code. Chatting with it shows the same problems which
| ChatGPT has.
|
| I wonder if GPT-4 could give me the correct answer
| (manually displacing the vertices, not in a shader).
|
| Wow, now after chatting and then performing a normal
| query and then going back to the chat-mode, the entire
| chat history was gone...
| rvnx wrote:
| This is what ChatGPT-4 says regarding this specific case
| (you can see the prompt used as well, it's just a copy-
| paste of your comment): https://pastebin.com/PPy4vMrU
|
| It seems to write code that does the displacement
| manually "// Iterate over the geometry's vertices and
| apply the displacement, geometry.vertices.forEach{...}"
| qwertox wrote:
| Thank you for the feedback! Looks like I'm going to
| subscribe to plus then.
| mycall wrote:
| Microsoft GitHub Copilot X might do better than GPT-4 Bing
| Chat regarding coding. I'm still on their waiting list for
| that but it looks promising.
| barking_biscuit wrote:
| Chat in the IDE is going to be game changing for me. I find
| myself switching between IDE and ChatGPT when co-pilot just
| isn't giving me a sensible suggestion, and I've found the
| combination of the two to be pretty epic. Can't wait to
| have it all tightly integrated. Also very interested in
| voice.
| holoduke wrote:
| I agree that the bing integration is worse than the chatgpt
| site itself. I also notice that common people arround me use
| the chatgpt site. Not bing.
| qwertox wrote:
| Bing is super lovely in terms of eye candy, but apparently
| they don't even offer a real history like the ChatGPT
| sidebar has.
| barking_biscuit wrote:
| >ChatGPT failed to understand that the raycaster works on the
| CPU, but the displacement map of the material is applied on
| the GPU side, so the displaced geometry won't be used, only
| the original one. It managed to explain this to me, that this
| was not possible, but each sample code repeatedly did as if
| it was possible, until I gave up.
|
| Reflecting on it, what's crazy is that this comment will
| likely wind up in the training data for GPT-N+1, and then
| GPT-N+1 will get it right.
| kfarr wrote:
| I've also not seen great results on three.js or A-Frame
| questions from any of these models. I'm guessing it's simply
| because there's a limited corpus of text from which to learn,
| however I wonder if LLM's lack of inherent spatial awareness
| contributes. Admittedly three.js and related concepts can be
| confusing for humans too
| digilypse wrote:
| I suspect it could soon be feasible to fine-tune on code
| with limited open data through a bootstrapping approach.
|
| Give it the source code, a test library and access to a dev
| environment and with some prompting it could start to
| experiment with increasingly complex use cases, learning
| from successful attempts. This would depend on the model's
| ability to understand what a successful outcome is so it
| can define test cases, which might be harder if the output
| isn't text but not impossible.
|
| Being able to give a model expert knowledge on an
| undocumented library or language seems like it could help
| accelerate adoption of new technologies that might
| otherwise suffer from network effects as users get used to
| AI-assisted development. Not to mention automated testing,
| finding edge cases and making pull requests to fix them,
| security, etc.
|
| A human taking time to experiment with their assumptions
| and gain experience with an unfamiliar subject is in a
| sense creating their own training data as well.
| greatpostman wrote:
| People keep repeating that google has superior tech. They
| haven't released any of it. I'm beginning to think their "AI
| lead" is a facade.
| Sai_ wrote:
| Google in AI === Russia on the battlefield.
| paxys wrote:
| Google's secret sauce is not its product but how it is able to
| monetize its userbase. OpenAI isn't going to become Google by
| launching a better language model and hoping that enough people
| pay $20/mo for it. It has never really been about search at
| all. The various forms of user tracking, AdWords, AdSense are
| all alive and well, and are very far from OpenAI's expertise.
|
| Let's at least wait for the $10B spending cash from Microsoft
| to run out and the company to become profitable before we crown
| them Google killers.
| alex_sf wrote:
| OpenAI isn't trying to become Google, though. ChatGPT is
| marketing.
|
| OpenAI's market is more comparable to AWS. Each product is
| another backing service for some other product to integrate
| with and sell to _their_ users.
| passion__desire wrote:
| At some point, we have to move away from ad based model. I
| already pay for youtube premium, love the experience. I am
| happy to pay for ChatGPT as well, especially it is a good
| assistant to knowledge based workers, those who wouldn't
| shy away from paying for it. I still believe Google can win
| this race just like they did with Android.
| londons_explore wrote:
| Microsoft, through Bing, already has a small ads platform.
|
| I think it wouldn't be too tricky for them to make sure that
| whenever the user asks ChatGPT "My omlettes keep sticking to
| my pan, what am I doing wrong?", it can reply with "You
| probably have an old frying pan with a damaged coating.
| [Click here] to see my recommendations of frying pans you can
| have delivered tomorrow.".
| golergka wrote:
| $20/month? I can bet that Apple executives are already trying
| to broker an exclusive deal for Siri-GPT-4 for tens of
| billions.
| bigcloud1299 wrote:
| Since ChatGPT became publicly accessible, I can tell that my
| use of google has gone down
|
| 1. I had an interview for a CIO position - I fed the job
| description, ask ChatGPT to generate top 10 questions to be
| asked based on it , I was asked 90% similar or the same
| questions. Heck I even asked to provide best answers and they
| were great. It even generated a question on personalization
| and a/b testing which I thought would be highly unlikely to
| be asked, but was indeed asked. It was as if the interviewer
| also generated the same question.
|
| 2. For my current company, I am in charge of building a brand
| new engineering team, for which I wanted to write vision,
| mission and strategy statements. Previously I would have
| googled stuff, this time I asked ChatGPT to improve it based
| on my rough draft. And boy they are great.
|
| I can go on.
|
| My point is that not everyone needs to ask ChatGPT to
| generate ray tracing code using three.js or whatever. Most of
| us are just regular folk need regular help and ChatGPT is a
| game changer for it. Heck it even helped me with some excel
| macro I needed to write to do some data manipulation.
| Havoc wrote:
| >Google's secret sauce is not its product but how it is able
| to monetize its userbase.
|
| It is still primarily reliant on search as start of funnel
| though
| b33j0r wrote:
| It gives me pause to realize that the GOOG has been my
| internet, my video clips, my wallet, my phone, my operating
| system, my cable TV alternative that costs arguably more--
|
| and now it finally wants to commit to our eternal friendship!
|
| I'm a fan in general, techno-optimism is the only fun way to
| be.
|
| But for your consideration: once upon a time, the GOOG only
| had 400 emoloyees too. And Jeff ran a bookstore in the PNW. I
| wonder every day. What is the future of OpenAI and this
| largely unseen goldrush?
|
| Edit: full disclosure, and an employer! It was a while ago.
| But I do wonder sometimes already if google and jeff bezos
| are my real parents. Now there's a new player!
| sebzim4500 wrote:
| Why would they kill their revenue? They'd just include ads.
| elorant wrote:
| If a search engine can answer your question what's the point
| of watching an ad? You just get all the information inside
| the platform. The essence of ads is to visit a third party
| website that might have what you're looking for.
|
| Ads could work when you're looking for products. In which
| case the usefulness of an AI is limited.
| weberer wrote:
| Knowing Google/Microsoft, they'll just inject the ads
| directly into the responses.
| jldugger wrote:
| "Rewrite the above response but include a pitch for
| grammarly"
| visarga wrote:
| The problem is that the ad inventory doesn't match people
| needs. So they try to shove things we don't need on our
| throats. If they sold what people needed to buy, there
| would be a better return on ads. But those products are
| probably not as profitable.
|
| I have nothing against LLMs in commerce if they don't try
| to trick me, make me overspend, or use dark patterns on me.
| My bet is that we'll filter every communication through a
| local LM to eliminate the bias in sources, you need
| protection when you get out there. Like "my lawyer will be
| talking to your lawyer", but with AIs.
| dragant wrote:
| I actually like Google Bard. I find the responses from Bard
| are superior over ChatGPT. Have others used Bard?
| AviationAtom wrote:
| I have. I found it to more or less be trash in comparison
| to GPT-4. Bard made up commands that didn't even exist, so
| I can't imagine how much more it would make up that isn't
| true.
| DennisAleynikov wrote:
| I have the opposite impression. It seems to stick to
| facts more than I've had luck with on gpt
| tyfon wrote:
| GPT-4 will also happily invent python libraries that
| doesn't exist to enable some functionality in the code it
| produces.
|
| I don't think this is a Bard only feature.
|
| It's not as bad as the 65B llama model I run on my (amd)
| PC though, especially with quantized weights it tends to
| stop coding at some point and repeat the last line over
| and over. The 30B unquantized model seems better in this
| particular thing.
| AviationAtom wrote:
| With LLaMA it's all about the tuning. Using an Alpaca-
| tunes 7B yields pretty reasonable results, considering
| it's constraints.
| dragonwriter wrote:
| > Bard made up commands that didn't even exist, so I
| can't imagine how much more it would make up that isn't
| true.
|
| It's unsurprising that Bard is particularly bad at
| something that Google says up front that it
| _categorically cannot do_ , but I think that's probably a
| bad thing to use to evaluate its capabilities outside of
| that domain.
| Me1000 wrote:
| But when people are directly comparing ChatGPT to Bard,
| it seems entirely fair to point out places where it falls
| short.
| dragonwriter wrote:
| > But when people are directly comparing ChatGPT to Bard,
| it seems entirely fair to point out places where it falls
| short.
|
| The comment in question was generalizing about overall
| capability from failure a task Google advertises Bard as
| incapable of, not pointing out that task has a particular
| area of deficiency.
|
| So, while recapitulating the advertised limitations of
| Bard might be useful in some contexts as you describe,
| that observation is not germane to the particular context
| where it was offered.
| AviationAtom wrote:
| Me asking how to accomplish a task using a Linux command
| does not seem unreasonable. I can understand it's current
| constraint on generating code, but that too will need to
| quickly be overcome if they wish for Bard to be a serious
| competitor.
| Kranar wrote:
| I am also using Bard and it's unbelievably bad compared to
| ChatGPT.
|
| Infact it's just bad to the point of not really worth
| using. It gets basic facts wrong and often times
| misunderstands what I'm trying to ask it.
| throwaway1851 wrote:
| > It gets basic facts wrong and often times
| misunderstands what I'm trying to ask it.
|
| In haven't tried Bard, but I've tried ChatGPT extensively
| and this sounds like a very good description of it.
| visarga wrote:
| Google is only temporarily behind. It will release a decent
| model in max 6-12 months, and not just Google, everyone will
| release a model, even open source will have some. OpenAI is
| enjoying a short window of exclusivity.
| aliljet wrote:
| Just playing casually with NanoGPT
| (https://github.com/karpathy/nanoGPT) with a desktop holding
| a 2080ti, it's really really really clear to me that the path
| to get to a pre-fine-tuned LLM is remarkably easy. RLHF is
| the piece above this which appears to also be surprisingly
| easy (if Sam Altman is to be believed). The juice is making
| these tools incredibly easy.
|
| I think the barrier to entry here is low. OpenAI is ahead
| now, but I doubt that lives forever.
| yieldcrv wrote:
| Exactly, almost nobody is going to need a trillion
| parameter LLM
|
| Its going to go client side, at the OS level and have like
| 1% of the mental capacity and be good enough
| Closi wrote:
| This comment might end up being a bit like "640kb of RAM
| is more than anyone will ever need!"
|
| GPT3 already used almost 1/5th of that figure at 180bn
| parameters, and PaML uses 500bn.
| yieldcrv wrote:
| Okay, if client side resources expand then larger
| parameter LLMs will be used. There.
|
| The point is that it will be ubiquitously client side,
| and it will happen faster than newer hardware comes out.
| Current hardware is very limited and slow in getting
| output from LLMs.
| fxtentacle wrote:
| No.
|
| I'm pretty sure Google has an AI that is on par with ChatGPT.
| The reason they still need more time is because they are now
| fine-tuning the AI to include their paying partners' products
| into the response.
|
| It is VERY easy to make ChatGPT lie on future questions by
| injecting the right prompts. That means it is equally easy to
| inject ads into ChatGPTs responses. And if Google can pull that
| off, it'll be even more profitable than tolerating SEO spam so
| that brands need to buy keyword ads for their own homepage.
|
| As a mental model, I believe the future of Google will be a
| personal butler that answers all of your questions. The butler
| usually does a good job, so you trust him. But unknown to you,
| the user, your butler is being blackmailed and forced to lie to
| you on some days for some questions.
| rvnx wrote:
| > I'm pretty sure Google has an AI that is on par with
| ChatGPT. The reason they still need more time is because they
| are now fine-tuning the AI to include their paying partners'
| products into the response.
|
| Nice try Sundar!
| adammarples wrote:
| Doubtful? They basically invented transformers
| DennisAleynikov wrote:
| Everyone forgot that we have been living in an AI age for
| a decade now.
| echelon wrote:
| Where's the Google product, then?
|
| They keep slipping. One week ChatGPT, the next GPT4, the
| next Plugins, ...
|
| Everyone is already going to be building on OpenAI by the
| time Google says it's ready.
|
| Xerox, HP, and Bell Labs could have all said the same
| thing as you about having invented modern computing
| devices, and look what happened to them.
|
| Google here is Xerox. Or IBM.
| rvnx wrote:
| The person here claims that Google has an amazing GPT-4
| killer ready to launch, and that they are just waiting to
| integrate with ads partners or something.
|
| If you want to believe that Google is intentionally
| holding it back and, as a diversion, releasing a very
| buggy software to the public, then why not, but it
| doesn't make sense at all.
|
| Even in terms of costs, they could force a limit of X
| messages and then push to upgrade to a paid subscription.
|
| In terms of reputation or safety they have DeepMind as a
| separate entity.
|
| This is like claiming that Tesla has a revolutionary car,
| but that Tesla is intentionally not releasing it, and
| instead waiting that someone else does.
|
| As a fictitious and parallel example: it's not because
| Xerox invented the mouse that it is a great company for
| innovation or that they wouldn't get eaten by others.
|
| (site-note: some claims Xerox didn't even invent the
| mouse).
|
| Regarding transformers:
|
| The transformers guys ("Attention Is All You Need") don't
| appear to work for Google for a long-time, and the
| comments from the team are not so glorious from what I
| see (if I remember well, the Character.AI is very harsh
| on Google all the time claiming it was not good for
| innovation).
|
| They may be missing the RLHF part for example or other
| part of the magic, and between 2017 and 2023 is an
| insanely long period where many impactful new things have
| been discovered.
|
| Doing the 90% is easy now with LLMs, but each % of
| improvement is very difficult.
| washadjeffmad wrote:
| > If you want to believe that Google is intentionally
| holding it back and, as a diversion, releasing a very
| buggy software to the public, then why not, but it
| doesn't make sense at all.
|
| One reason would be capacity. They've increased their
| server production at least 4x since the announcement of
| Bard, and that's recent enough that I feel like any decom
| and deployment, even if it's 1:1 to existing data
| centers, hasn't been completed.
|
| This maps to your last point somewhat, since anticipation
| is hard. Google has learned not to fully open the all the
| taps since they can't commit to a consistent product.
| Assistant and Home technologies were heavily encumbered
| by patent defense, security awareness, and privacy
| controls, so much so that the product capabilities they
| advertised during the Pixel 3 launch were permanently
| rolled back. They're not promoting or promising anything
| about Bard, if you notice, because people may not
| appreciate every new feature, but they never forget when
| you take things away.
| rvnx wrote:
| Capacity issue may be alleviated by limiting the numbers
| of users who have access, and then progressively scaling
| it up, rather than releasing an inferior product
| (assuming the superior product really exists).
|
| That being said, for all the other points I agree with
| you.
|
| Google is often a target of any legal claim, so perhaps
| this makes them more risk-averse too.
| dr_dshiv wrote:
| Or just ads on the side that update with every new prompt.
| Guaranteed. I'd guess within 5 months from OpenAI or Google.
| anon7725 wrote:
| Why on the sidebar? It will just be integrated into the
| response: "sure, I can give you some suggestions for easy,
| nutritious meals for the working professional, but first
| I'd like to mention Athletic Greens(tm). Athletic Greens is
| one of the best ways to ..."
| sumitkumar wrote:
| It will be more likely start as a chat query about a
| product and then go into product research and then the
| model will casually suggest reasons for which product to
| choose. Example: which is the best magnesium supplement?
| The model will first guide the user to choose one form(of
| Mg) over other citing pros/cons. Then it will talk about
| different products for that type of Mg and suggest that
| one producer is better/worse because of
| reviews/issues/problems etc.
| radus wrote:
| Guess I need an LLM to block ads from the output of other
| LLMs.
| UpToTheSky wrote:
| Releasing a LLM would not kill Google's revenue. It would
| increase it. Because more people would use Google than before.
| And they could still show relevant ads.
|
| The big question is if they can catch up with OpenAI. OpenAI is
| a moving target. And it seems they are moving fast.
|
| It could still be possible that Google catches up because of
| the UI though. Many people don't like having to log in to
| ChatGPT. Bing's UI is a disaster _and_ you have to log in.
|
| I always thought that Google won the search war not only
| because of their good search results, but also because of their
| clean UI.
| yayr wrote:
| The OpenAI business model seems to become an ecosystem play.
| Plugins, apis etc. They are new to it, but have a super
| strong partner with MicroSoft and strong startup experience
| for it in the leadership team. We will see if Google is able
| to play that well and not only release research and basic
| features.
| sho_hn wrote:
| I'm not so sure about the ads.
|
| Search lends itself well to interleaving ads with output, so
| you can place more ads while maintaining user tolerance and
| "get away with them". With LLM output you have to show the
| ads in the margins as integrating ads into the output itself
| would lower performance.
|
| Sure, you can do that, but does it fully substitute for
| previous ad revenue? It seems to me that the form factor of
| LLM search vs trad search has reduced ad surface area.
|
| Microsoft is well-positioned to monetize LLMs in other ways,
| e.g. via 365 and other subscription services, many aimed at
| enterprises/b2b. Google is much less well-developed there
| atm. If they can subsidize lower ad annoyance in their
| search/chat product via other revenue that could make a real
| diff.
| UpToTheSky wrote:
| I think the ads will become even better.
|
| Say you put the message you just posted through Google to
| correct style, facts and spelling. You know how it would
| reply? Here we go:
|
| The text is pretty awesome. I would only suggest changing
| the word "interleaving" to "interspersing" to improve the
| style of the text. "Interspersing" is an alternative term
| for "mixing" or "adding in between," which better conveys
| the idea of placing ads within search results. You know
| what I would also change? Your <related product>. Since you
| seem to be deep into technology in general and the internet
| in particular, you will love <related product>. Since you
| wrote such a thoughtful text about ads, I will tell you the
| secret discount code "adsMakesMeSmile" to get <related
| product> 10% off.
| sho_hn wrote:
| I predict users won't accept this and will have a choice
| in the matter due to a competitive landscape.
|
| It lowers performance, and one artifact of a natural
| language interface is that it'll trip the red flags for
| salesmanship in the human brain and trigger revulsion and
| and anger in a way that an ad in a search results listing
| doesn't.
|
| This is where marketers usually jump in with "if the user
| doesn't like it, it's just a bad ad - users will love
| useful ads, and targeting will make it work". I never buy
| this because simply there's too much contention for my
| wallet and the market wants to sell me things more often
| than I need to buy things, and ad companies like Google
| are so far pretty bad at refusing business.
|
| I wouldn't be surprised if LLM search monetization is
| more likely to take a subscription form. If you told me
| that 2-3 years from now access to these systems is most
| commonly via the 365 subscription of your employer which
| it "graciously", as a standard benefit, also allows you
| to use at home, I would not be surprised.
|
| Many variables here though. Cost of local inference over
| time being a massive one, copyright laws for training
| data another, etc.
| akimball wrote:
| Ads are redundant when the user is literally asking you
| what to do. You just tell them what to buy and they buy it.
| glitchc wrote:
| No, ads would just turn into advertorials and become part
| of the response. Those journalists writing advertorials
| would be out of a job.
| coredog64 wrote:
| If I had to guess, it's probably going to be more like
| YouTube: Watch (click through) these 2 ads to get to your
| LLM results. If you have to provide updates/corrections,
| there's some additional space for ads.
| phailhaus wrote:
| Search is just one product. They have YouTube, Gmail, Google
| Calendar, Drive, etc etc etc. That much sticking power buys
| them tons of time to catch up.
| Kranar wrote:
| None of those make money compared to search. Search is close
| to 80% of their revenue.
| totoglazer wrote:
| Google search has been declining in quality for years. In
| pursuit of profits they've ramped up ads, but the SEO garbage
| and "helpful" search "corrections" has also fully destroyed the
| organic results too.
|
| Chat as a search replacement is compelling, and one reason is
| for now it's not monetized by ads. That may change, but
| Google's stranglehold is materially weakened.
| moffkalast wrote:
| I wonder how long before we see organic looking promotional
| recommendations in LLM replies. Like one of those youtubers
| that has to thank Skillshare twice in the same video and tell
| us how great NordVPN is.
| j_maffe wrote:
| I wouldn't say that promotional section of a video is
| "organic" I'd be more worried of sentences or paragraphs
| being thrown in the reply the same way ads are shown in
| Reddit or Twitter feeds
| m3kw9 wrote:
| Google not going anywhere, if google come up with something
| good enough in this regard I'd stick to them knowing they are
| still very competent at search. Bing just came and hacked
| something up in a month isn't very convincing, especially the
| bing search with Gpt isn't very good yet. Google observing bing
| to avoid their mistakes. Remember google came up with
| Transformers which Gpt relies on
| AviationAtom wrote:
| Google is held hostage by their current revenue model (ads,
| ads, ads) and their belief that it can't carry over to AI-based
| products. I can't blame them for focusing so much on it, but
| they also forget how they got to ads making up so much of their
| revenue. They shook up the ad market back when, and AI can help
| do so again.
|
| Microsoft is focused more on incubating OpenAI, getting a
| viable product to market now, and focusing on monetizing it
| down the road. Microsoft knows AI can complement an ad revenue
| model.
|
| Google just opened up Bard to testers and I've found it to be
| complete garbage in comparison to what Bing Chat/GPT-4 can do.
| Bard was not afraid to just make up random things, that were
| verifiably false, and present them as fact. GPT seems to know
| enough to let you know when it's unsure.
| andsoitis wrote:
| > Google is held hostage by their current revenue model (ads,
| ads, ads) and their belief that it can't carry over to AI-
| based products.
|
| What tells you that Google thinks that ad-based monetization
| can't carry over to AI-based product?
| AviationAtom wrote:
| There was an article posted here on Hacker News about a
| Google exec basically saying AI was a fad not worth
| chasing, and it had no viable means of monetizing with ads.
| I was shocked to read the article and flabbergasted that
| Google could have such a view.
|
| To me it felt like maybe they were saying one thing, while
| doing another, to attempt to calm investor nerves as they
| try to play catch-up.
|
| I felt most the entrenched folks were so consumed by trying
| to develop their AI to be ethical that they allowed OpenAI
| to run laps around them.
|
| How the next decade unfolds in the AI arena will be most
| interesting. I wouldn't have thought Google could ever be
| knocked of their throne as dominant search engine in the
| space previously, but I now feel they have never been more
| vulnerable than they are now.
| crop_rotation wrote:
| Do you have the article link? That does seem shocking.
| bradleyjg wrote:
| Of course Microsoft doesn't have the same cultural attachment
| to an ad model, they were born and raised as a selling
| software company. Shipping software to renting it was a leap,
| but one they made long ago.
| crop_rotation wrote:
| Google is in a non win spot. LLMs will be more expensive to run
| than current search. And Google needs search to make big money,
| since it subsidises everything else. Microsoft needs search as
| a side business for now, as long as it breaks even it is good
| for them. A 10B search profit per year would be so good for
| Microsoft, but catastrophic for Google.
|
| Not to mention that current search infra and ads UX have been
| optimised to the end to gain every penny, and a LLM based ads
| system won't have the same margins to start with.
| londons_explore wrote:
| The running costs will drop precipitously with time.
| Especially in Googles position - they can afford to design
| custom silicon for their search engine, and potentially
| _save_ money compared to having hundreds of copies of every
| page on the whole internet sitting in RAM, just waiting for
| some user query to maybe need to see it, which they currently
| do.
|
| Language models are _only_ a few terabytes of RAM, which is
| small chips in comparison.
| jeadie wrote:
| I wonder if I can add this to https://github.com/Jeadie/awesome-
| chatgpt-plugins still?
| RohMin wrote:
| With the introduction of plugins, is it feasible to give ChatGPT
| some kind of long term and short term memory model?
| amrb wrote:
| There are attempts via langchains [0] depending on how much
| context is required I could see a summary step where the
| history to compressed and used to carry forward progress.
|
| An alternative could be a vector store, injecting small
| snippets of relative text as a step.
|
| 0 -
| https://python.langchain.com/en/latest/modules/memory/key_co...
| netsroht wrote:
| LangChain is a great workaround for that. [1]
|
| > how to work with a memory module that remembers things about
| specific entities. It extracts information on entities (using
| LLMs) and builds up its knowledge about that entity over time
| (also using LLMs).
|
| [1]
| https://python.langchain.com/en/latest/modules/memory/types/...
| sean_lynch wrote:
| OpenAI actually thinking about this too. It's buried in their
| open source repo and not clear the exact mechanism that ChatGPT
| knows to make use of it. But we're already here evidently.
|
| https://github.com/openai/chatgpt-retrieval-plugin#memory-fe...
| londons_explore wrote:
| I suspect with a 'window' of 32k tokens, OpenAI has already
| done similar memory tricks.
|
| I suspect that if you filled the context window with "1 1 1 1 1
| 1 1 1 1 1", and then asked "How many 1's did I just show you?",
| it probably wouldn't know, simply because whatever tricks they
| use to have such an apparently large context window don't allow
| it to 'see' all of it at any given moment.
| meghan_rain wrote:
| Ah so you think the 32k context window works differently than
| eg the 4k davinci context window? They didnt just increase
| ${hyperparam}?
| pmalynin wrote:
| Maybe, you could give it a combination of both. We'll call it
| long short term memory.
| RohMin wrote:
| The reason I ask is because I feel that a memory model is one
| of the major bottlenecks toward AGI.
| pmalynin wrote:
| On a more serious note, I do agree with you that memory and
| self-excitation seem like they are the last push thats
| needed to get to something more akin to "AGI". But I don't
| think that Rubicon will be crossed with plugins.
| MacsHeadroom wrote:
| >I do agree with you that memory and self-excitation seem
| like they are the last push thats needed to get to
| something more akin to "AGI"
|
| "We show that transformer-based large language models are
| computationally universal when augmented with an external
| memory. Any deterministic language model that conditions
| on strings of bounded length is equivalent to a finite
| automaton, hence computationally limited. However,
| augmenting such models with a read-write memory creates
| the possibility of processing arbitrarily large inputs
| and, potentially, simulating any algorithm."
|
| From "Memory Augmented Large Language Models are
| Computationally Universal"
|
| https://deepai.org/publication/memory-augmented-large-
| langua...
| nunodonato wrote:
| why? short and long-term memory is really easy to do. Even
| my own basic assistant has it (running on fine-tuned curie
| model)
| m3kw9 wrote:
| What does this do?? Skimmed it but couldn't find a clue
| birracerveza wrote:
| Mmh, you should probably ask chatgpt.
| londons_explore wrote:
| I am really surprised OpenAI didn't think about this and
| deliberately try to make these plugins not work with third party
| engines.
|
| For example, they could have required the schema file be uploaded
| to them in your account. That way the majority of plugins
| wouldn't have publically accessible schemas.
| saqadri wrote:
| Author here. It's such early days in this space and things are
| evolving quickly -- OpenAI's plugins system is also still in
| Alpha. We think it's a good time to have a conversation and
| ideally build an open plugin protocol that everyone can benefit
| from. It will help unlock even more innovation in this space.
| We have seen this in the evolution of language servers after
| Language Server Protocol, and how that benefited so many more
| developers and gave us better IDEs.
| lxe wrote:
| How does the semantic search plugin work? Does it create an
| embedding for the model using the uploaded documents, or does it
| perform full-text search on the docs and then uses the model to
| answer based on the retrieved section of the document? What
| exactly is the datastore API here?
| lxe wrote:
| Found the answer here:
|
| https://github.com/openai/chatgpt-retrieval-plugin#retrieval...
| The plugin uses OpenAI's text-embedding-ada-002 embeddings
| model to generate embeddings of document chunks, and then
| stores and queries them using a vector database on the backend.
| As an open-source and self-hosted solution, developers can
| deploy their own Retrieval Plugin and register it with ChatGPT.
| The Retrieval Plugin supports several vector database
| providers, allowing developers to choose their preferred one
| from a list.
| saqadri wrote:
| Author here. This section of the readme has more information:
| https://github.com/lastmile-ai/llama-retrieval-plugin#retrie...
|
| It does use a vector database (pinecone, weaviate, etc.) to
| store embeddings. The embeddings are created using OpenAI's
| text-embedding-ada-002 model, but that's not a requirement. In
| fact we are looking at embeddings generation through BERT or
| RoBERTa to benchmark performance.
|
| At prompt time, the plugin retrieves the nearest embeddings to
| the prompt, and inserts them into a more complete prompt before
| sending it to the model.
| lxe wrote:
| Thank you! I'm looking into sentence-transformers also to
| create embeddings from documents.
| carlosdp wrote:
| I agree with the sentiment, but if anything is made clear by how
| ChatGPT plugins work, it's that standardization of a protocol
| here is unnecessary when you are talking about tooling designed
| to be used by LLMs. They can just figure it out!
|
| Even if you have 10 different ways of describing plugins all by
| different teams, you're not writing declarative code for each
| one, you're throwing them to the LLM all the same and saying "you
| figure it out," and it does.
|
| Finetuning a model for certain schemas (as the author at one
| point suggests) should be entirely unnecessary, given my
| experience. You just need access to a model more at par with
| gpt-3.5-turbo, which we'll surely see in open source in no time!
| imtringued wrote:
| And this is how you end up with 15 different standards.
|
| LLaMA can work with ChatGPT plugins and ChatGPT can work with
| LLaMA plugins but why would you want plugin developers to have
| to choose between those options?
| ryanjshaw wrote:
| It's year 1. Competing standards are OK.
| [deleted]
| [deleted]
| akiselev wrote:
| Same reason with ended up with 15 different standards
| elsewhere: diversity.
|
| We're in uncharted waters and we have no idea which format
| will be optimal for one LLM let alone all of them. Given how
| much prompt engineer has become a thing, finding optimal
| formats is important.
| evv wrote:
| I would argue we are in uncharted waters, so we can't even
| say for certain that it is important to find optimal
| formats.
|
| Maybe 15 "standards" aren't enough and we will have 1500
| that LLMs will happily use and learn
|
| xkcd/927 may loose relevance in the era of LLM
| arbuge wrote:
| > And this is how you end up with 15 different standards.
|
| Actually, this is something never seen before. This is an
| entirely new way of ending up with 15 different standards.
| sharemywin wrote:
| or you build a short dev cycle that builds the way to consume
| it
| saqadri wrote:
| Author here. We definitely agree these models are really
| powerful and can figure out a lot on their own. A protocol
| wouldn't be restrictive to their flexibility, but it'll allow
| different clients running different models to still be able to
| interact with the same plugins.
|
| Defining a standard around external memory, authentication,
| rules-based engines for preprocess/post-processing, and
| declarative or dynamic chaining of actions can help make
| plugins model-agnostic, and benefit all of us as developers and
| users.
___________________________________________________________________
(page generated 2023-03-26 23:00 UTC)