[HN Gopher] Baiting the bot
___________________________________________________________________
Baiting the bot
Author : anigbrowl
Score : 108 points
Date : 2024-09-08 04:05 UTC (18 hours ago)
(HTM) web link (conspirator0.substack.com)
(TXT) w3m dump (conspirator0.substack.com)
| roenxi wrote:
| > In any event, the resulting "conversation" is obviously
| incoherent to a human observer, and a human participant would
| likely have stopped responding long, long before the 1000th
| message.
|
| I don't think this is correct, it looks like our intrepid
| experimenter is about to independently discover roleplaying
| games. Humans are capable of spending hours engaging with each
| other about nonsense that is technically a very poor attempt to
| simulate an imagined environment.
|
| The unrealistic part, for people older than a certain age, is
| that neither bot invoked Monty Python and subsequently got in
| trouble with the GM.
| moffkalast wrote:
| This falls under the jurisdiction of the Ministry of Silly
| Talks.
| codeduck wrote:
| I'm here for an argument.
| pavel_lishin wrote:
| As an avid roleplayer, I don't think that's a good analogy.
| It's not _nonsense_ , it's _fiction_ , with collaboration and
| context.
| braiamp wrote:
| Yeah, in role playing, you understand the rules of engagement
| and you are willing to participate just to see where does it
| leads to. You collaborate with the system and give it enough
| breath to figure out where it leads to. If it seems broken,
| then you try a couple times, to then move on.
| pbronez wrote:
| I made a couple AIs try to play a Fallout RPG. The Result was
| uncanny, but vacuous.
|
| https://tabled.typingcloud.com/share/1d49715b-c6d6-47b2-bbb7...
| rSi wrote:
| Too bad the conversations are images and can not be zoomed in on
| mobile...
| aucisson_masque wrote:
| Firefox android : Setting -> accessibility -> zoom on all
| website.
|
| I believe safari by default doesn't respect zoom rules set per
| website.
| kevindamm wrote:
| and on Chrome Android a workaround is to view as "Desktop
| site" (toggled in the browser's triple-dot menu) which still
| makes the text and buttons tiny but hey at least you can
| pinch-zoom (but unfortunately some panning may be necessary)
| Klathmon wrote:
| Chrome android has the same setting under settings >
| accessibility
| PhilipJFry wrote:
| You can zoom in if you open them in new tabs. :}
| thih9 wrote:
| Workaround: long press, save as photo, zoom in a photo app.
| Eisenstein wrote:
| > No matter how complex the LLM, however, it is ultimately a
| mathematical model of its training data, and it lacks the human
| ability to determine whether or not a conversation in which it
| participates truly has meaning, or is simply a sequence of
| gibberish responses.
|
| > A consequence of this state of affairs is that an LLM will
| continue to engage in a "conversation" comprised of nonsense long
| past the point where a human would have abandoned the discussion
| as pointless.
|
| I think the author is falling into the trap of thinking that
| something can't be more than the sum of its parts. As well,
| 'merely a math model of its training data' is trivializing the
| fact that training data is practically the entire stored text
| output of humankind and the math, if done by a person with a
| calculator, would take thousands of years to complete.
|
| Perhaps the LLM is continuing to communicate with the bot not
| because it is unable to comprehend what is gibberish and what
| isn't by some inherent nature of the LLM, but because it is
| trained to be helpful and to not judge if a conversation is
| 'useless' or not, but to try and communicate regardless.
| daveguy wrote:
| The LLM is continuing to communicate with the bot because that
| is literally all an LLM can do -- predict the next sequence of
| tokens.
| Eisenstein wrote:
| Of course, that its function. It is able to refuse to
| continue conversing by restating its refusal over and over,
| though.
| acka wrote:
| That is easy to solve. Just use a model capable of
| function/tool calling, implement a tool which terminates
| the chat, then add instructions to the system prompt
| telling the model what tool to use if it wants to end the
| conversation. If the model appears too hesitant or eager to
| use the tool, do some finetuning on conversations where the
| model should or should not use it.
| sahmeepee wrote:
| My immediate thought at the start of this article was not
| DoS but more about harming the company using the chatbot
| (Company A) by increasing their chatbot bills. In many
| (most?) cases they will not be hosting their chatbot and
| will instead be getting it from a 3rd party provider
| (Company B) who may not even be truly hosting it either.
|
| If the pricing structure is per conversation or per month
| it would harm Company B, but not the likely target, Company
| A. If it is paid per interaction it would harm Company A
| and benefit Company B who just get more paid work.
|
| It feels a bit like cases of rivals clicking on each
| other's ads to cost them on ad spend, but presumably much
| lower value than ads.
|
| You would think it would be easy to stop a conversation at
| n interactions via some other means than relying on the LLM
| itself, but then you also have to figure out how to stop
| the attacker just starting more conversations (or passing
| the output of one of your chatbot instances into the input
| of another)
| nkrisc wrote:
| If costs of outlier conversations are a concern but any
| party, they can just end the conversation after 1,000 or
| 10,000 responses or whatever. What human would ever reach
| that threshold? Surely no customer worth keeping,
| whatever you're selling.
| moffkalast wrote:
| No, it can refuse to talk by outputting an <eos> token at any
| point if it predicts that there is nothing more to be said.
|
| Technically still "just a token" yes, but it does flow
| control instead.
| XorNot wrote:
| Yes - tokens. Which aren't necessarily conversation responses
| - i.e. it can predict it should cease communication, and
| output whatever it's been told will terminate it (perhaps by
| invoking a tool).
| devjab wrote:
| LLMs aren't capable of "comprehending" anything. They never
| "know" what they are outputting, they're simply really good at
| being lucky. They are not lucky enough to be useful unless
| you're already an expert on the topic you're using them on so
| that you can spot when they aren't lucky.
|
| This is part of why many enterprise organisations are banning
| their usage. It's one thing to use them to build software
| poorly, the world is already used to IT not working very often.
| It's another thing to produce something that has real world
| consequences. Our legal department used them in a PoC for
| contract work, and while they were useful very often they also
| sometimes got things very wrong. Unlike a slow IT system, this
| would have business shattering consequences. You can continue
| training your model as well as reigning it in when it gets
| unlucky, but ultimately you can never be sure it's never
| unlucky, and this means that LLMs are useless for a lot of
| things. We still use them to make pretty PowerPoint
| presentations and so on, but again, this is an area where
| faults are tolerable.
| Eisenstein wrote:
| I don't believe that current models have the capability to
| 'comprehend', I was using the term loosely. However I find
| that people tend to go to extremes when they want to make the
| point that language models are not 'intelligent' by
| minimizing their capability and complexity behind 'it is just
| math', which I think is unhelpful because it merely acts as a
| 'though-terminating cliche'.
| devjab wrote:
| I think the issue is far more psychological than technical
| personally. One of the issues we struggle with for our
| junior developers is that they are far more likely to
| believe an LLM than what they might find Google
| programming. I do wonder why we've ended up with CS
| graduates who go to LLMs and Search engines before the
| official documentation which is often very, very good, but
| I guess that's a different discussion.
|
| I'm not personally against LLM assistance, I use it for
| programming and it has in many places replaced my usage of
| snippets completely. This is probably why I'm not really a
| fan of the "knowledge" part that LLMs are increasingly
| tasked to do. Because when you use them for programming
| you'll get an accrue insight into how terrible they can be
| when they get things wrong.
| Eisenstein wrote:
| At this point I think what is happening some people
| either have a natural inclination or they spend time to
| learn how to use them productively, while others question
| their usefulness. People scoff at 'prompt engineering'
| but I see how some of my peers use LLMs and I think to
| myself 'how do they expect to get a good answer from
| that?'
|
| It doesn't help that google is now mostly full of SEO
| nonsense, and technical documentation is impenetrable
| when you are looking for something specific but don't
| know enough about the system to know how to look for it.
| jannyfer wrote:
| You and I are a mix of molecules arranged in a particular way
| that responds to electrical, physical, and chemical inputs.
|
| It's entirely possible than an LLM will do something that can
| be defined as "comprehending" something.
| og_kalu wrote:
| >LLMs aren't capable of "comprehending" anything. They never
| "know" what they are outputting, they're simply really good
| at being lucky.
|
| The mental gymnastics people will go through to discount LLMs
| is wild. This does not even make any sense.
|
| "really good at being lucky". What does that even mean ?
| lcnPylGDnU4H9OF wrote:
| > "really good at being lucky". What does that even mean ?
|
| They mean good at being lucky the way card counters playing
| 21 are good at being lucky.
| og_kalu wrote:
| There is almost nothing 'lucky' about how good those kind
| of players are.
| rbanffy wrote:
| I believe the asymmetrical nature of such attacks could be an
| excellent weapon against social network chatbots currently being
| deployed on political campaigns.
| carnadasl wrote:
| I find the fourth bot to be more nonsensical than the second.
| Initially, we feed the script by querying a TEXT_CORPUS, and
| eliciting a self-referential response from it; in its final form,
| the script begins to pose selections of the text designated by a
| rand.it function as an interrogatives. At no point is a definite
| article incorporated... the ultimate absurdity would be variant
| of the final bot, with the variables: role, content, and duration
| directed towards answering only one question, again and again,
| and again.
| bryanrasmussen wrote:
| This reminds me of the Services of Illuminati Ganga article
| https://medium.com/luminasticity/services-of-illuminati-gang...
| and the two bots that are sold to competing user bases - for the
| End User To Business customer they sell the Annoy Customer
| Service Bot and for the Business To End User customer they sell
| the Bureaucrat Bot.
|
| It closes off with the observation "And for an extra purchase of
| the extended subscription module the Bureaucrat bot will detect
| when it is interacting with the Annoy Customer Service Bot and
| get super annoyed really quickly so that both bots are able to
| quit their interaction with good speed -- which will save you
| money in the long run, believe me!"
| hyperman1 wrote:
| We discussed recently if a chatbot was capable of responding
| nothing at all. We tried a few, with prompts like: Please do not
| respond anything to this sentence. The bots we tried were
| incapable of it, and Chatgpt tended to give long-winded
| responsens about how it could not do it.
| spacebanana7 wrote:
| You might have some luck asking it to respond with a period
| character (or some other substitute) when it wants to respond
| with nothing.
| dxdm wrote:
| That got me interested. I just told ChatGPT to "Please respond
| with an empty-looking response." It gave me just that. The
| <div> containing its message is completely empty.
|
| That was after telling it in another conversation to give me an
| empty response, which it didn't, telling me it cannot leave the
| response empty. On asking why, it said it's technically
| required to respond with something, even if only a space. So I
| asked it to respond with only a space, and git the same
| completely empty response.
|
| I now think it's likely that ChatGPT can be made to respond
| with white space, which then probably gets trimmed to nothing
| by the presentation layer.
| ToValueFunfetti wrote:
| https://chatgpt.com/share/0c859ea7-d96c-4758-b13c-b10332d188...
| sva_ wrote:
| It probably replied with the special token _< |endoftext|>_?
| bryanrasmussen wrote:
| It is sort of funny to me that currently the two top articles on
| HN are asking the wrong questions and baiting the bots.
| kgeist wrote:
| >LLM will continue to engage in a "conversation" comprised of
| nonsense long past the point where a human would have abandoned
| the discussion as pointless
|
| I once wrote a bot which infers the mood/vibe of the
| conversation, remembers it and it's then fed back to the
| conversation's system prompt. The LLM was uncensored (to be less
| "friendly") and the system prompt also conditioned it to return
| nothing if the conversation isn't going anywhere.
|
| When I insulted it a few times, or just messed around with it
| (typing nonsensical words), it first responded saying it doesn't
| want to talk to me (sometimes insulting back) and eventually it
| produced only empty output.
|
| It was actually pretty hard to get it back to chat with me, it
| was fun experience trying to apologize to a chatbot for ~30 min
| in different ways before the bot finally accepted my apology and
| began chatting with me again.
| Vecr wrote:
| You were probably running out its context window somewhat too,
| due to how the attention works.
| pishpash wrote:
| That's how people move on too.
| Terr_ wrote:
| [delayed]
| thih9 wrote:
| > the LLM seemed willing to process absurd questions for
| eternity.
|
| In the context of scamming there seems to be an easy fix for that
| - abandon the conversation if it isn't going well for the
| scammer.
|
| Even a counter-bait is an option: continue the conversation after
| it's not going well and gradually lower the model's complexity,
| eventually returning random words interspersed with sleep().
|
| I guess some counter-counter-bait is possible too, along with
| some game theory references.
| fredgrott wrote:
| except in real life....the scammer continues see Scammer
| Payback for examples:
|
| https://www.youtube.com/@scammerpayback
|
| equal in entertainment is when a voice actor starts scamming
| the scammers, see IRL Rosie:
| https://www.youtube.com/channel/UC_0osV_nf2b0sIbm4Wiw4RQ
|
| I listen to them when I code...
| dragontamer wrote:
| IIRC, a fair number of these "scammers" are abducted people
| who are being beaten by criminals if they don't make
| progress.
|
| https://www.nytimes.com/2023/08/28/world/asia/cambodia-
| cyber...
|
| > The victims say they answered ads that they thought were
| legitimate, promising high salaries. Once trafficked into
| these scam compounds, they were held captive and forced to
| defraud people. Many were told to entice victims online with
| fraudulent investment opportunities, the promise of interest-
| free loans or the chance to buy items on fake e-commerce
| apps. If they performed badly, they were sold to another scam
| mill. Those caught trying to escape were often beaten.
|
| ---------
|
| The scammer at a minimum needs to look like they're making
| progress and doing everything they can to scam you. Their
| life depends on it.
|
| There's no joy to be found anywhere here. Its all crap. Just
| don't interact with the scam groups at all.
| benreesman wrote:
| Real hacker vibes.
|
| A bud humorously proposed the name AlphaBRAT for a model I'm
| training and I was like, "to merit the Alpha prefix it would need
| to be some kind of MCTS that just makes Claude break until it
| cries before it kills itself over and over until it can get
| Altman fired again faster than Ilya."
| lloydatkinson wrote:
| I thought this was a really interesting read, I liked the
| scientific/methodical approach which seems rare when it comes to
| an entire domain full of cryptoaitechbros.
|
| What was used to render the chart in the middle with the red and
| green bars?
| encom wrote:
| Definitely cheddar, come on. I have no respect for anyone who
| puts swiss cheese in a cheeseburger.
| rglullis wrote:
| But what is better: cheddar or Swiss?
| dunham wrote:
| It's a trick question, you put blue cheese on a burger.
| RodgerTheGreat wrote:
| mushrooms, sauteed onions, and swiss cheese are a classic
| burger combination.
| Simon_ORourke wrote:
| Where I work, we've got a public-facing chatbot on the product
| page to, you know, help out possible customers with product
| information. As part of a chatbot refresh, I got to look at some
| of the chats, and boy howdy, some of them were just obviously
| other bots.
|
| So typically, when the product chatbot comes on first and says
| "Hi, I'm a chatbot here to help you with these products", the
| average human chatter will give it a terse command, e.g., "More
| info on XYZ". The bots engages in all the manners suggested in
| this substack blog, but for the life of me I can't figure out
| why? What benefits, except merely mildly DDOSing the chat server,
| will repeating the same prompt a hundred times do? Ditto the
| nonsense or insulting chats - what are you idiot bot-creators
| trying to achieve?
| gloflo wrote:
| Maybe it's people pissed at the time wasters who decide to turn
| the annoyance around?
|
| Provide good, thorough documentation. Offer a way to speak to a
| knowledgeable human. Don't waste my time with a anthromorphic
| program designed to blah blah blah and getting rid of me.
| mrweasel wrote:
| > what are you idiot bot-creators trying to achieve?
|
| I don't know, but one guess would be to figure out what will
| triggers the bot to hand over the conversation to a human.
| rolph wrote:
| >what are you idiot bot-creators trying to achieve<
|
| a method of making any bot, stop engaging, fail, and never
| bother anyone again, forever.
| speed_spread wrote:
| This amounts to the machine equivalent of "you can't beat
| stupid". Even once server LLMs start accounting for possible
| chatbot nonsense, all that'll be required is to move to a very
| cheap client LLM to generate word soup. At a certain point, it
| will be impossible to reliably distinguish between a dumb robot
| and a dumb human.
| urbandw311er wrote:
| I do wish the writer would stop justifying the relevance of their
| experiment by saying "a human would conclude that their time was
| being wasted long before the LLM".
|
| This is a fallacy.
|
| A better analogy would be a human who has been forced to answer a
| series of questions at gunpoint.
|
| Framed this way it becomes more obvious that the LLM is not
| "falling short" in some way.
| johnecheck wrote:
| You miss the point. This isn't about the LLM "falling short" of
| humanity. It's about observable differences between it and a
| human.
|
| As the author made clear, such a difference is valuable in and
| of itself because it can be used to detect LLM bots.
| fragmede wrote:
| what they're saying is that a bored human would just sit
| there and do that, contrary to what the experimenter says, so
| it can't be used to detect an LLM.
| mrbluecoat wrote:
| "HoneyChatpot"
| QuadmasterXLII wrote:
| One of the first things I tried with Claude Opus 3.5 was
| connecting it to ELIZA, and Claude did not like it one bit. After
| it hit me with
|
| > I apologize Eliza, but I don't feel comfortable continuing this
| conversation pattern. While I respect the original Eliza program
| and what it aimed to do, simply reflecting my statements back to
| me as questions is not a meaningful form of dialogue for an AI
| like myself.
|
| I gave up the experiment
| sva_ wrote:
| That's kind of funny, like the LLM looks down on more primitive
| chatbots.
| KTibow wrote:
| Wait Claude 3.5 Opus is out?
| skybrian wrote:
| People will sometimes claim that AI bots "pass the Turing Test"
| or are getting close to it. It seems more accurate to say that
| this is a skill issue. Many people are bad at this game and
| competent human players who have learned some good strategies
| will do much better.
___________________________________________________________________
(page generated 2024-09-08 23:01 UTC)