[HN Gopher] OpenAI's chatbot store is filling up with spam
___________________________________________________________________
OpenAI's chatbot store is filling up with spam
Author : mfiguiere
Score : 120 points
Date : 2024-03-20 17:34 UTC (5 hours ago)
(HTM) web link (techcrunch.com)
(TXT) w3m dump (techcrunch.com)
| ActionHank wrote:
| Maybe they should use an AI to automate moderation.
| timetraveller26 wrote:
| Yes, they should have a GPT that runs other GPT's, an Universal
| GPT we could call it.
|
| They do use some sort of automation according to the article
|
| > We use a combination of automated systems, human review and
| user reports to find and assess GPTs that potentially violate
| our policies.
| ActionHank wrote:
| Then they can market this universal GPT as an AGI.
| bink wrote:
| It is a bit funny that if you apply for a job with many of
| these companies they ask they you do not use AI for the
| application.
| throwaway11460 wrote:
| As any company should. Not sure what's funny about it - I'm
| interested in the person. If using AI is one of required
| capabilities I will ask them to perform a task using it
| during the interview so I can review their process. In the
| same way I don't expect to receive an executable binary as a
| cover letter for a SWE position.
|
| I had many candidates submit absolutely terrible cover
| letters clearly made by ChatGPT. The number one sign - it
| talks about their experience exactly in the order I listed it
| on the job post; number two - I don't get to know anything
| about the person. I asked them to submit a cover letter they
| wrote themselves, and while not everyone's writing was as
| good as ChatGPT can write, I got to know the person, and
| hired few of these who definitely wouldn't make the cut if I
| went by the AI made letter.
|
| If you can have AI generate your letter so well I can't tell,
| you get a pass. Nobody was able to do that yet, though (or at
| least didn't admit to it, even though I'm asking after they
| are hired and saying that at this point it would only help
| them if they did so).
| herval wrote:
| most companies are using AI to filter out CVs these days,
| and cover letters written with ChatGPT seem to have a
| higher success ratio. I wouldn't blame anyone for using it,
| unfortunately.
| throwaway11460 wrote:
| Maybe in first rounds. I am the third/fourth round
| interviewer. If a cover letter like that gets to me, it's
| automatically a no - not because of the fact itself but
| simply because the cover letters are terrible.
| crooked-v wrote:
| In a third round, why are you even looking at cover
| letters?
| badwolf wrote:
| Who is even looking at cover letters at all?
| kergonath wrote:
| I suspect it depends on the field, but in my experience
| it's often a good indicator of success. A shoddy letter
| betrays a lack of professionalism. If you're not going to
| spend 10 minutes running a spell checker it means that
| either it's not important to you, so good bye, or that
| your standards are really low and you will keep producing
| garbage if I hire you, so good bye as well.
|
| The problem with cover letters is false positive (people
| who did not write the letter themselves, or who did it to
| a much higher standard than their usual). But then,
| that's what interviews are for.
|
| I did hire a couple of people with either a subpar CV
| (some good people sometimes end up in dead ends or
| difficult situations), or cover letter (not everyone is a
| great writer), or reference (your issues with your former
| boss are not always your fault), or interview (you can
| have a bad day), so I would not rely on a single factor.
| But a combination of 2 dodgy elements is an automatic
| rejection. Each one tends to surface different aspects.
| ungreased0675 wrote:
| Consider that job seekers today sometimes have to apply
| to hundreds of positions before being hired. For most of
| those applications, they won't hear anything back. Given
| that dynamic, would you spend time polishing a cover
| letter for each application?
| rvnx wrote:
| A guess: First round was "AI" (aka keyword matching),
| second round was recruiter who doesn't understand
| anything and doesn't read
| throwaway11460 wrote:
| First round is phone call prescreen, second round is non-
| technical interview with a recruiter. No GPT or keyword
| matching involved.
| throwaway11460 wrote:
| How am I supposed to learn something about the person?
| herval wrote:
| at the third or fourth round? By talking to them,
| obviously?
| basil-rash wrote:
| Don't you see how the incentives become impossible? The
| AI stage needs you to be as obvious and direct about your
| experience as possible, the human stage gets irritated
| when you write like the reader doesn't have a brain.
|
| Personally I send my resumes through a gpt with the job
| posting and ask if I'd be a good fit. Almost always the
| GPT will initially say no because I'm using some
| terminology that anyone in the industry would understand
| is a form of the posting's requirements, but the GPT does
| not. But then perhaps the screening recruiter doesn't
| either. So why not be specific? But then anyone else at
| the company might think I'm a moron... ugh.
| StableAlkyne wrote:
| > The number one sign - it talks about their experience
| exactly in the order I listed it on the job post
|
| I wouldn't attribute that to AI; it's a very natural way to
| write a cover letter. You want to hit the bullet points in
| the job posting, and the most logical way to do that is to
| go down the list and check boxes.
| throwaway11460 wrote:
| Yes, but this goes much further - on a word by word,
| sentence by sentence basis.
| ben_w wrote:
| I've never seen that from an LLM; usually the error they
| make is I've asked them for too many things (a quantity
| which varies wildly), I get a response for about 3 of
| those things and a note saying something to the effect of
| "put more here".
| exe34 wrote:
| This could be a way to separate the wheat from the chaff - if
| your AI can be detected through your writing, then you aren't
| who they're looking for!
| 1oooqooq wrote:
| on that note, will only bother with AI when "open"ai announces
| an AI ceo.
| ben_w wrote:
| Then you've set your standards way too high.
|
| Loads of places hire interns, no sane person would make an
| intern CEO. LLMs today are kinda like weird interns at every
| subject all at once.
| 1oooqooq wrote:
| who wants to pay for weird interns? even good ones are
| free. and might even grow to good hires.
| ben_w wrote:
| Paid internships are common, and LLMs are so much cheaper
| than humans they might as well be free.
|
| The prices on OpenAI's website are listed per _million_
| tokens, the cheaper model can read a typical book for
| less than the cost of that book in paperback, second
| hand, during a closing down discount sale, and even the
| fancy model is still in range of a cheap second-hand
| paperback (just not during a closing-down sale) for
| reading and just about the price of a _new_ cheap
| paperback for writing -- cheap enough it might as well be
| free.
|
| Plus, they're an intern at _everything all at once_. You
| don 't get to hire someone who has a late-degree-level
| grasp of programming (in every major programming language
| at the same time) _and_ genetics _and_ law _and_ physics
| _and_ medicine _and_ who has a business-level grasp of
| _all_ the most common written languages on Earth -- there
| is no such human.
|
| (If and when we can make an AI, be it an LLM or
| otherwise, which can act at the level of a senior in
| whichever field it was trained on, even if that training
| takes a gigawatt-year to compute per field, and even if
| inference costs 100 kW just to reach human output speed,
| it's going to seriously mess up the whole world economy;
| right now it's mainly messing up the economic incentives
| to hire people fresh from university while boosting the
| amount of simple stuff that actually gets done because
| normal people and not just business can now also afford
| to hire the weird intern).
|
| > and might even grow to good hires.
|
| Even if they do, will you be the one to hire them?
|
| Also, ChatGPT has so far existed for about as long as I
| spent on my (paid) year in industry that formed part of
| my degree. It wasn't _called_ an internship, but it
| basically was. In that time, the models have grown and
| improved significantly, suggesting (but not proving,
| because induction doesn 't do that) that they will
| continue to grow and improve.
| 1oooqooq wrote:
| business idea: put a chatgpt plugin moderator on the store,
| priced for a single purchase by them.
| Areibman wrote:
| As a (former) ChatGPT plugins developer, our business has
| absolutely tanked due to GPTs. Discoverability is nonexistent
| because search is just buried by spam. At least with plugins,
| there were only a few hundred to sort through, and most had some
| unique API they would plug into.
|
| For context, we built ChatOCR- an OCR tool that lets users
| extract text from photos and PDFs. We made roughly $20k from
| 39,000 users over 6 months on the plugins catalog
| danpalmer wrote:
| Do end users have to pay you each time or buy access or credits
| or something, or do you just get a cut of ChatGPT paid
| subscriptions when a paying user uses your plugin?
| Areibman wrote:
| We use a third party plugins manager called pluginlab.ai. It
| manages auth and subscriptions for plugins users by prompting
| them to sign in when they hit a paywall.
| jbverschoor wrote:
| Wait.. I thought the whole discoverability by Apple on the App
| Store was worthless (why pay Apple tax?), but when it's about
| <insert poster child>, it's something we actually need?
| csallen wrote:
| Nobody in this thread said they need OpenAI to provide them
| discoverability.
| refulgentis wrote:
| Not sure what you mean: OpenAI's App Store doesn't charge and
| has discoverability. (hence the viability of the spam, hence
| TFA)
|
| OP is likely affected by the change from plugins to GPTs
|
| Now, users have to manually select an app. Whereas before,
| you'd have a list of, say, 5 apps and the AI would attempt to
| intelligently determine which app to use.
|
| OpenAI said plugins didn't find product-market fit, and
| that's why they moved on.
|
| It's worth opining there's an air of rushing around doing
| nothing with their product development. Like instead of "what
| should it be?", and building it, and sustaining it, it's a
| pastiche of every startup shibboleth you've ever heard -
| sprints, if 10-20% of the user base / 10^7 users aren't using
| it within 3-5 months, it's time to Pivot (throw away the
| working solution we invested in)
|
| That makes sense when you're at 100 users in an uncertain
| market but at their scale, it reminds me more of how Google
| ended up with the reputation it has.
|
| * forgive me for not air-quoting app and app store, these
| aren't __apps__ per se, but it was immensely distracting,
| with an air of condescention, when I air-quoted everything
| chrsig wrote:
| > Not sure what you mean: OpenAI's App Store doesn't charge
| and has discoverability. (hence the viability of the spam,
| hence TFA)
|
| I think the point was that the app store charge acts as an
| effective filter for spam, enough so as to determine the
| sustainability of the app store ecosystem.
|
| I could very well be mistaken though, that was just how I
| interpreted it.
| matthewmacleod wrote:
| It's almost like different people can think differently about
| different things!
| xandrius wrote:
| Wrong thread.
| barfbagginus wrote:
| There's a conflict in value proposition here, which is more
| drastic than the discovery issue. That might help you better
| contextualize the failure, and avoid similar risks in the
| future.
|
| Namely, GPT is now a multimodal trained llm capable of doing
| OCR over PDFs and images. Given the accessibility of that
| feature, we expect that users do not try to discover OCR
| plugins anymore- they feel no need.
|
| Out of curiosity, what was the contingency plan in the case
| that OpenAI did this? What rationales did you use to estimate
| the likelihood and severity of that risk? Were there good
| reasons to discount that risk?
| Areibman wrote:
| We knew the time would come, but we built ChatOCR in a week.
| If we overthought the time horizon problem, we'd have 0 users
| and $0.
|
| But also, GPT-4-Vision is multi-modal but does not
| specifically use OCR. Our tool is used mostly to extract text
| from documents and load it into context, and we still saw
| growth after OpenAI built this feature into ChatGPT.
| yaj54 wrote:
| What resources do you suggest for learning how to do this
| kind of risk analysis well?
| codethatwerks wrote:
| Is there a name for this rugpull that is a recurring theme in
| tech, when you rely solely on another service bringing
| attention to your service.
|
| E.g. when youtube cancels you or your adwords account is
| blocked so you no longer make money
| jewel wrote:
| I've heard it called "Platform risk". Also "Playing in
| someone else's walled garden", or something along those
| lines. I realize that's not a term for the inevitable
| rugpull, but that's the closest I can think of.
| d13 wrote:
| You live on that platform, you die on that platform.
| snazz wrote:
| For Apple specifically it's called "Sherlocking".
| geoelectric wrote:
| That one is a little different. That's when Apple clones
| your app into their OS as a core feature, thereby
| completely killing your market.
|
| It usually implies Apple deliberately studied your specific
| app or replicated it based on details revealed in B2B
| licensing/acquisition meetings, similar to what MS pulled
| with Stacker back in the 90s.
|
| https://thehustle.co/sherlocking-explained
|
| https://en.wikipedia.org/wiki/Stac_Electronics
| paulpauper wrote:
| it is impossible to divorce AI from spam. the two go hand in
| hand. the spam situation will get far far worse than ever
| imagined with AI. The web will be increasingly clogged with AI-
| generated content (e.g. videos and articles ) that can do a
| reasonably good job impersonating human-generated content to the
| undiscriminating reader. Same for AI-produced research papers,
| plagiarism, etc. AI-generated content can still be identified,
| but that is not the point. It has to only be good enough to pass
| muster for the average person or anyone who is not paying close
| attention.
| visarga wrote:
| One area that's getting close to hitting the mainstream is AI
| agents that control web browsers or desktop apps. Basically,
| you tell your AI what you want to do, and it does the computer
| work for you, showing the results the way you like. You won't
| have to dig through tons of useless apps anymore. It'll search,
| sort, and create feeds for you. This is what scares advertisers
| the most - when we start saying, "Just talk to my AI." God
| knows I'd love an AI that can fill tax forms for me.
|
| Operating on top of UI it is compatible with all devices, kind
| of like an android robot would fit well with all human spaces
| and tools. There is plenty of screen capture with narration to
| learn from. You can generate more training data by essentially
| giving free reign over a VM to an AI agent. Ask it to do office
| work. We can also collect this kind of data very easily.
| falcor84 wrote:
| I recall imagining something s quite like this as a kid when
| I first encountered the term "User Agent".
|
| I'd be all in on this if I knew I had full control and
| explainability of its decisions, but am afraid that reality
| will be a lot more depressing than what I imagine.
| crooked-v wrote:
| For a very immediate example, I can already see a clear path
| to somebody coming up with an LLM service that will
| automatically filter out political spam texts and emails.
| (Maybe it already exists and I just haven't seen it yet.)
| kaibee wrote:
| I am endlessly frustrated every day that this is not a
| thing yet. I also don't want to send all of my emails to
| OpenAI though.
| samus wrote:
| I recently did a Kaggle experiment and finetuned a LLM to
| classify SMS as SPAM or HAM. It was not so difficult;
| following the text classification example in the
| Hugginface transformers docs was enough. Without trying
| hard at all, accuracy was above 90%. Impressive even
| though even the messages marked as HAM seemed to be quite
| trash. The model itself was small - few hundred million
| parameters only. Models of this size run well on CPUs and
| an even smaller might have workd too. And non-neural
| classifiers might also do fine.
|
| You could do something similar and unleash it on your
| mailbox. The tricky part are integrations. At work, I
| don't even bother since interacting with the email system
| there is way more tricky than with GMail.
| jart wrote:
| You can configure Postfix to filter content based on a
| shell script. https://www.postfix.org/FILTER_README.html
| You can turn the email into plain text from your shell
| script using links and then ask Mistral if it's spam or
| political: https://justine.lol/oneliners/#url You can use
| the grammar flag to force Mistral to only provide a YES
| or NO answer, which is easy to check from your script
| using an IF statement. Then boom, no more spam or
| politics.
| chasd00 wrote:
| seems like that would be very straightforward to the point
| that i could probably do it but maybe not in the most
| elegant way. I would be surprised if there wasn't a model
| on hugging face trained to identify the political leaning
| of some text. I bet it would be small enough to run
| locally, then it's a matter of creating a plugin/module or
| something that you add to your email client that runs on
| every new mail. Based on the result, move the email to your
| political spam folder.
| samus wrote:
| LLM is just quite compute-intensive. You could send emails
| to the OpenAI API to classify them as SPAM or HAM, but
| you'd probably cry at the bill. That should explain why it
| doesn't exist in free-tier accounts. Worry not, even in the
| event you belong to the GPU-poor, a beefy CPU should allow
| you to run a small LLM on your local machine. And if you
| find a good dataset for it, you can train the model in a
| Google Collab.
| pavel_lishin wrote:
| > _Basically, you tell your AI what you want to do, and it
| does the computer work for you, showing the results the way
| you like. You won 't have to dig through tons of useless apps
| anymore. It'll search, sort, and create feeds for you. This
| is what scares advertisers the most_
|
| Not just advertisers. There isn't much point in creating any
| sort of fun content on the web if it never actually gets seen
| by human eyes, but instead is just regurgitated into a
| slurry.
| cdme wrote:
| A tool to generate spam content becomes plagued by it. Seems
| predictable.
| golergka wrote:
| LLMs are no more a tool to generate spam than text and image
| editors or programming languages are.
| nirvdrum wrote:
| Almost immediately after OpenAI opened its API there were a
| raft of services to send auto-generated sales and marketing
| emails. Sure, there are other ways to generate unsolicited
| commercial emails. There are even services that will spam on
| your behalf. But, there's obviously a huge difference between
| an empty text editor and a service that generates plausibly
| phrased emails with virtually no effort.
| jsheard wrote:
| Similarly, the worst kind of SEO grifters are now providing
| services which can mass-generate thousands of longform
| "articles" on your behalf based on nothing but a handful of
| keywords, or even scrape a competitors site and bulk
| generate articles based on their content. SEO spam isn't
| new but automating it on this level is unprecedented.
| a_wild_dandan wrote:
| Yes, historically people abuse tools for spam. And also
| golergka is still completely correct.
| phoe-krk wrote:
| Similarly, the Haber-Bosch process is no more a tool to
| produce crop fertilizer than feeding corn to a cow is.
|
| Seriously though - the ability to industrialize and mass-
| produce that both LLMs and the industrial ammonia process
| have given us, for the bad and the good, is the real game-
| changer here. Treating two tools the same because both can
| give you the same singular result is an error if they have
| dramatically different - by several orders of magniture -
| speeds at which these results are achieved.
| ben_w wrote:
| Like all automation tools, LLMs are very effective at
| _getting more done_.
|
| They're not content-neutral (no violence, no sex, etc., which
| some people complain about _very loudly_ ), but for anything
| not forbidden, gpt-3.5-turbo-0125 can output more for about
| $4233.60 than a human record holding stenographer can type
| _in a working lifetime_.
|
| When it doesn't have to be amazing, and I think spam is one
| of many things in that category, this is absolutely a huge
| deal compared to a text editor.
| solumunus wrote:
| Bombs are no more a tool to kill people than rocks or knives
| are.
| golergka wrote:
| Are you claiming that overwhelming majority of bombs are
| used to cook and build? Please do share these intriguing
| cuisine and masonry techniques!
| pavel_lishin wrote:
| I think the majority of explosives are used for fireworks
| and mining, but I honestly have no idea how to get an
| accurate count of explosives set off in mining vs.
| explosives set off in warfare.
| golergka wrote:
| I'm pretty sure that majority of explosives by equivalent
| TNT or by dollar value are used in nuclear warheads for
| strategic deterrence, and thankfully have never been set
| off. But it would an interesting challenge to estimate
| all of these indeed.
| bdjsiqoocwk wrote:
| He wasn't claiming that, he was just applying your stupid
| logic elsewhere.
| probably_wrong wrote:
| Have you ever found this quote?
|
| > _These findings, combined with earlier results on synthetic
| imagery, audio, and video, imply that technologies are
| reducing the cost of generating fake content and waging
| disinformation campaigns._
|
| That paragraph comes from this [1] now-deleted blog post from
| OpenAI in 2019 when they decided not to release GPT-2 due to
| "concerns about malicious applications of the technology".
| It's hard to argue that GPT and friends are not "tools to
| generate spam" when the researchers themselves argued that
| point years ago.
|
| [1] http://web.archive.org/web/20190214171619/https://blog.op
| ena...
| bdjsiqoocwk wrote:
| Yes, they ARE more a tool to generate spam than text, because
| they can do everything a single text does, but in addition it
| can also generate text. So strictly more spam.
| codethatwerks wrote:
| > Seems Predictable
|
| Nice pun
| pants2 wrote:
| It was a huge mistake to monetize GPTs. I and many other AI
| enthusiasts would have probably developed plenty of useful GPTs
| for free. But now I would never bother because nobody will see my
| projects under all the SEO spam and fake ratings.
| ilaksh wrote:
| When you say it's monetized, are they actually giving anyone
| any money? The details of that were to "to be announced" but I
| haven't heard any announcement.
| Implicated wrote:
| It seems that just the 'threat' of monetizing it would cause
| an absolute flood of GPTs to be launched, under the
| assumption that they would eventually get paid.
| ado__dev wrote:
| When the barrier to entry is non-existent, this outcome is
| inevitable.
| rvz wrote:
| OpenAI GPTs Store was supposed to be the "iPhone moment" which it
| changes everything and upends the App Store. ...Or so I was told
| by the AI bros.
|
| So what _really_ happened? Most likely little quality control,
| garbage copy cats running around and now there is an
| insurmountable amount of spam.
|
| That is nothing like the quality of the Apple App Store.
| duskwuff wrote:
| > So what really happened? Most likely little quality control,
| garbage copy cats running around and now there is an
| insurmountable amount of spam.
|
| That, and there isn't a lot of space for real innovation within
| the "GPT" framework that OpenAI provides. Most of what's
| available in their store is literally just prepackaged prompts,
| and there's only so much that can accomplish. You can ask a
| chatbot to pretend to be an expert in a topic all day, but that
| won't actually make it into one.
|
| If Apple's App Store were just a form you filled out to release
| versions of Flappy Bird with new graphics or whatever, that'd
| fail too. Developers need conceptual "space" to innovate; I'm
| not convinced that's possible when the interface (a chat
| window) and the back end (a language model) are both set in
| stone by the platform.
| jart wrote:
| It was doomed from the start, because monetizable computing
| platforms that target consumers can't succeed if they don't
| have high barriers to entry.
|
| Low quality Atari games were responsible for the 1983 video
| game crash. Anyone could publish an Atari game. So what ended
| up happening was people made games about things like raping
| Native American women (Custer's Revenge) and they were sold
| just like any other Atari cartridge.
|
| Nintendo pioneered the closed platform. No one was allowed to
| build games for Nintendo without Nintendo's permission.
| Microsoft and Apple did things a bit differently, by making
| their platforms as difficult to develop for as possible. WIN32
| is byzantine. Making iPhone apps when the App Store was
| launched required learning Objective C, which was an entirely
| new language to most developers at the time.
|
| OpenAI on the other hand created an app store which, in their
| own words, "is simple and doesn't require any coding skills."
| https://openai.com/blog/introducing-the-gpt-store
| eightman wrote:
| The use case for AI is spam.
| jsemrau wrote:
| As I had written about publicly already, this doesn't surprise me
| a bit. Same with poe.com, GPTs can be generated in just a few
| clicks by anyone anywhere with no quality control in sight.
|
| My guess is that it was always a play about having access to
| high-quality data.
| paxys wrote:
| A spam-filled online marketplace is almost becoming a feature.
| That way the top developers have to pay and outbid each other to
| get featured and show up in search results, and that becomes a
| lucrative revenue source for the platform.
| ionwake wrote:
| Can anyone recommend a good chatgpt chatbot from the store? I
| havent had a poll recently. Any suggestions would be great,
| thanks.
| tkgally wrote:
| > OpenAI's terms explicitly prohibit developers from building
| GPTs that promote academic dishonesty. Yet the GPT Store is
| filled with GPTs suggesting they can bypass AI content detectors,
| including detectors sold to educators through plagiarism scanning
| platforms.
|
| The use of generative AI in academic contexts--both education and
| research--is turning out to be a vast ethical gray area. It seems
| that some people regard any use of AI for writing under one's own
| name to be dishonest. Others are willing to allow some AI use,
| but where they draw the line varies a lot. Is it okay, for
| example, to have an LLM correct word-level grammatical mistakes,
| rewrite a paragraph to make the author's point clearer, write a
| paragraph based on the author's bullet points, write a full first
| draft that the author then checks and revises by hand, translate
| phrases or sentences that the author has written in their native
| language, translate an entire paragraph or paper that the author
| wrote, etc.?
|
| Over the past year, I have conducted several workshops on LLMs
| for academics in Japan, and when I have polled them about the
| acceptability of each of those use cases, their responses have
| varied widely.
|
| At the institutional level, some universities discourage their
| students from using AI for writing at all, while others seem to
| be encouraging it. Just last week, I heard about two public
| universities in Japan that have contracted with Microsoft to
| provide Copilot to all of their students starting in April, when
| the new academic year begins here.
___________________________________________________________________
(page generated 2024-03-20 23:01 UTC)