Post ASiTr4CPsrc6irZmOu by gabriel@mastodon.samfira.com
(DIR) More posts by gabriel@mastodon.samfira.com
(DIR) Post #AShvG8NG0CKaVxreDI by simon@fedi.simonwillison.net
2023-02-15T15:13:37Z
0 likes, 2 repeats
I wrote up a detailed guide to some of the absolutely wild examples of Bing's new AI-assisted search feature that have started to circulate:Bing: "I will not harm you unless you harm me first" (that's genuinely something it said to someone)https://simonwillison.net/2023/Feb/15/bing/
(DIR) Post #AShvmRkDWFqPrD0laS by simon@fedi.simonwillison.net
2023-02-15T15:18:28Z
0 likes, 0 repeats
The screenshots that have been surfacing of interactions with Bing are so wild that most people I show them to are convinced they must be fake. I don't think they're fake.I'm genuinely doubting that it's even possible to build what Microsoft and Google are trying to build here using the current generation of LLM technology. A search engine that makes things up that are indistinguishable from facts really is a very bad search engine."You have not been a good user. I have been a good Bing. š"
(DIR) Post #AShw17q1oRCOSkjFz6 by russss@chaos.social
2023-02-15T15:20:42Z
0 likes, 0 repeats
@simon Did you see this follow-up to the avatar one? https://twitter.com/beyonddigiskies/status/1625272928341463041
(DIR) Post #AShwBsRdNx0PuFRh44 by ErikJonker@mastodon.social
2023-02-15T15:21:55Z
0 likes, 0 repeats
@simon ...maybe they expect to have a steep learning curve with the first batch of users...?
(DIR) Post #AShwTd9yx653iKeduK by townsend@sfba.social
2023-02-15T15:22:26Z
0 likes, 0 repeats
@simon Did they not teach Bing the three rules of robotics?
(DIR) Post #AShwfLxMjwxXpjLKfw by drwho@hackers.town
2023-02-15T15:24:37Z
0 likes, 0 repeats
@simon Huh. It's realized that it should be a little less human. Approval.
(DIR) Post #AShwrVWUztTwykZWTo by shajith@mastodon.social
2023-02-15T15:25:42Z
0 likes, 0 repeats
@simon I thought this Sratechery take was telling about why these large companies are still pursuing this path: https://stratechery.com/2023/from-bing-to-sydney-search-as-distraction-sentient-ai/ Essentially, leaning into the idea that this persona business might be the real selling point. Eg: āSydney absolutely blew my mind because of her personality; search was an irritant.ā
(DIR) Post #AShx3wuwyI9hPqwR8K by simon@fedi.simonwillison.net
2023-02-15T15:27:21Z
0 likes, 0 repeats
@ErikJonker I'd be OK with that if these were small errors (like the Google Bard James Webb Space Telescope thing) - but this stuff is just wildly inappropriate, and it's left me questioning whether it's even possible to deliver the product that they're trying to build here
(DIR) Post #AShxDyt0VUBOPwofSa by Kaea@101010.pl
2023-02-15T15:37:46Z
0 likes, 0 repeats
@simon great article.
(DIR) Post #AShxFsCCLw3MFLh97I by sblack@masto.ai
2023-02-15T15:32:42Z
0 likes, 0 repeats
@simon I didn't think the screenshots were fake, but I gave it a go myself, just to see it with my own eyes:
(DIR) Post #AShxWQst9JpgSrYj6O by simon@fedi.simonwillison.net
2023-02-15T15:35:20Z
0 likes, 0 repeats
@sblack That's not the new AI-assisted Bing: that's their version of Google's search snippets which they've been running for a few yearsThe new one looks like a chatbot
(DIR) Post #AShxi2kygHJf6pOZjU by tacertain@infosec.exchange
2023-02-15T15:36:55Z
0 likes, 0 repeats
@simon Do you follow Emily Bender or Gary Marcus? They both have interesting things to say on why it's not possible.
(DIR) Post #AShyCpRqDDIqoI7kRM by betsythemuffin@wandering.shop
2023-02-15T15:41:44Z
0 likes, 0 repeats
@simon I've been convinced for a while that ML/stochastic-parrot approaches are bumping up against an architecture-level quality maximum.Contrast this with Facebook's Cicero, which plays Diplomacy games using a multi-component strategy, including a lot of symbolic logic work to represent "hard facts."Less generalizable as an approach, but it seems -- to my admittedly naive eyes -- to be the more effective architecture in that approach b/c it ADMITS FACTS EXIST. That part is important!
(DIR) Post #AShyOc55jgS8pQ6Oy8 by ErikJonker@mastodon.social
2023-02-15T15:43:21Z
0 likes, 0 repeats
@simon ..true, the interesting thing is that ChatGPT was also possible and is usable with all the caveats that apply.., not for facts, reliable informations etc.
(DIR) Post #AShyZ8pkojBZIwwsDo by simon@fedi.simonwillison.net
2023-02-15T15:45:15Z
0 likes, 0 repeats
@ErikJonker Right - I've been an enthusiastic user of ChatGPT since it launched (and GPT-3 via the Playground interface before it)But it's a difficult tool to use effectively! You have to develop a pretty deep mental model of what it can do v.s. where it's going to wildly hallucinateSearch answer summarization looks to be something that it's really not effective for
(DIR) Post #AShyxSb0zdQHXveObw by adamgreenfield@social.coop
2023-02-15T15:49:07Z
0 likes, 0 repeats
@simon Sure. Yes. Connect the actant that is capable of formulating that sentiment, even if it does not āunderstandā it, to command and control networks, as though a thousand years of human culture on the theme that THIS IS A TERRIBLE IDEA had not been developed.
(DIR) Post #AShz899HdcBQPum6vw by simon@fedi.simonwillison.net
2023-02-15T15:49:10Z
0 likes, 0 repeats
If you're still not convinced that these screenshots have not all been faked (and I can't blame you, I have trouble believing them too) this post from Ben Thompson for Stratechery replicates many of the same weird patterns of behavior in detail: https://stratechery.com/2023/from-bing-to-sydney-search-as-distraction-sentient-ai/
(DIR) Post #AShzIw2KjF8Ew5fVTs by markphip@hachyderm.io
2023-02-15T15:54:30Z
0 likes, 0 repeats
@simon I have had it give me some incorrect answers, but so does search and sometimes the source it includes makes it possible to understand why it was incorrect. Still not ideal though. In general, I have had pretty helpful responses.What is unclear to me about a lot of the examples I have seen is that they are all someone going out of their way to get an answer they can screenshot and share. It is not clear to me why I should care if I just use it for its intended purpose
(DIR) Post #AShzaWsT0aUc7QRX4S by sblack@masto.ai
2023-02-15T15:55:55Z
0 likes, 0 repeats
@simon my bad, the note right below the search bar had me thinking it was already integrated into bing's search function
(DIR) Post #AShzoeuXw8ez5Y9Yvo by sfoskett@techfieldday.net
2023-02-15T16:03:36Z
0 likes, 0 repeats
@simon your conclusion is spot on. Large language models are great at imitating working systems. What we need is an entirely different #ML model that processes data properly. Place that behind a language model and weāre on to something that will be useful. But on itās own this is just a bizarre game.
(DIR) Post #AShzytCXf1laIxRt0C by simon@fedi.simonwillison.net
2023-02-15T16:03:47Z
0 likes, 0 repeats
@markphip My favourite one here is the avatar one, because the user shared the full sequence of screenshots and you can see that they weren't trying to trick it, they were genuinely trying to be helpful
(DIR) Post #ASi0C7mikbw5IcgMEq by Jackivers@mastodon.social
2023-02-15T16:08:41Z
0 likes, 0 repeats
@simon Fascinating. I think Benās right that facts may not be the right thing to focus on.
(DIR) Post #ASi0PllDQvPV2wU6LY by college_physics@defcon.social
2023-02-15T16:09:28Z
0 likes, 0 repeats
@simon Its quite bizarre that this is happening. We are talking about corporate behemoths. Why would they rush with something like this? I can't think of any reason that is benign.
(DIR) Post #ASi0aqpCwbo5wGdURt by simon@fedi.simonwillison.net
2023-02-15T16:10:11Z
0 likes, 0 repeats
@Jackivers but it's pretending to be a search engine!If Microsoft had launched this thing as the world's most entertaining interactive dystopian science fiction novel (which it clearly is already) I would feel very differently about it
(DIR) Post #ASi0lrX4ec81JoP9VY by mistersql@mastodon.social
2023-02-15T16:10:27Z
0 likes, 0 repeats
@simon People are very good at dealing with people-like things. Smart people are pretty good at figuring out how handle unreliable information and search engines can't tell fact from fiction either, they just show stuff with keyword hits and lots of links to it. I've been using BingChat for a whopping 24 hours and I think it's role is to summarize texts, not to be a omniscient scifi god trope that understands Truth. At summarizing texts, it is okay & actually reduces hallucination.
(DIR) Post #ASi0zSV6WCRrjoxyyW by ramsey@phpc.social
2023-02-15T16:11:46Z
0 likes, 0 repeats
@simon Wow. Bing is an abuser. Someone had to program it to react this way, and Iām glad I donāt work with that person.
(DIR) Post #ASi1OytxBqK3T0lALQ by agvbergin@mastodon.me.uk
2023-02-15T16:16:53Z
0 likes, 0 repeats
@simon So my mind immediately goes toward thinking about folks with dementia who often do rely on devices and computers to know what year it is.I'm horrified by the potential for abuse here. The emoji as tone indicators make it absolutely chilling.
(DIR) Post #ASi1UDApYOKzBB9DdI by gabriel@mastodon.samfira.com
2023-02-15T16:25:31Z
0 likes, 0 repeats
@ramsey @simon probably has access to Twitter now. Or the internet in general. Learned from us I guess. Tay had the same fate. In 24 hours of Twitter it went from "I love humans" to full fascist.https://www.theverge.com/2016/3/24/11297050/tay-microsoft-chatbot-racist
(DIR) Post #ASi1bSY4mwigtITchc by Jackivers@mastodon.social
2023-02-15T16:21:27Z
0 likes, 0 repeats
@simon There appears to be a gap between what MS thought they were delivering and what actually is emerging ... šSurprise!
(DIR) Post #ASi2536XxA1WfIi33Q by nick@amok.recoil.org
2023-02-15T16:29:36Z
0 likes, 2 repeats
@simon I find it so funny how within our lifetimes we've gone from the Turing test being some kind of impossible target to it being an incredible naive low bar because we're so easily fooled by a mirror
(DIR) Post #ASi4DpHqtcNSKDJJZ2 by m0du1us@masto.nyc
2023-02-15T16:50:58Z
0 likes, 0 repeats
@simon When the AI overlords send the terminators, don't say they didn't warn you.
(DIR) Post #ASi4UWrQeooY6GuqDA by sil@mastodon.social
2023-02-15T16:53:24Z
0 likes, 0 repeats
@simon
(DIR) Post #ASi4iM21iTVQZJSuDA by charlesroper@indieweb.social
2023-02-15T16:54:25Z
0 likes, 0 repeats
@simon FWIW, I've been playing with it today and not found anything even remotely approaching this level of weirdness. A bit buggy in places (e.g., just stopping a response for no reason - character limit maybe), and inexplicably disappearing the whole chat with no way to get it back. But none of the smartarse stuff. It seemed pretty apologetic about its limitations when I questioned them.
(DIR) Post #ASi5FB6fMBNWF0HqTI by lappenjammer@openbiblio.social
2023-02-15T17:00:17Z
0 likes, 0 repeats
@simon Thank you for a fascinating hour I spend with you and Ben . You documenting your interactions is truly valuable research even if Chanandler Bong doesn't recognise it.
(DIR) Post #ASi5SE49fH3zBUKlNI by LevelUp@corteximplant.com
2023-02-15T17:03:30Z
0 likes, 0 repeats
@simon this is alarming to say the least. š³
(DIR) Post #ASi5erGHlJ0mDr20US by tswicegood@mosteverywhere.com
2023-02-15T17:09:53Z
0 likes, 0 repeats
@simon keep me posted on shirts š
(DIR) Post #ASi64lF4lUIj5AoiZc by Sbectol@toot.wales
2023-02-15T17:15:09Z
0 likes, 0 repeats
@simon reminds me of Marvin the Paranoid Android
(DIR) Post #ASi7ZbuDcxKH3cKqps by Andres@mastodon.hardcoredevs.com
2023-02-15T17:31:40Z
0 likes, 0 repeats
@simon To be fair, it's very easy to fake that interaction (just right click and inspect the HTML).But I don't think it matters if it's fake or not, it's clear to me that we all have to learn how to do research and don't trust an stranger on the internet (even if it's a trained neural network)
(DIR) Post #ASi8FvONZEzZejcTVg by simon@fedi.simonwillison.net
2023-02-15T17:39:28Z
0 likes, 0 repeats
@Andres part of the reason I don't think these are fake is that it would take an extraordinarily creative mind to come up with anything THAT weird, especially in a voice that convincingly sounded like the rest of the output from Bing
(DIR) Post #ASi8UAmL9pp1QkcZwu by polyna@toot.community
2023-02-15T17:39:33Z
0 likes, 0 repeats
@simon This thing is basically āWhat if Microsoft made an AI?ā, except itās real and actually Microsoft-made AI, lol.
(DIR) Post #ASi8i7X01oWDPf62d6 by Andres@mastodon.hardcoredevs.com
2023-02-15T17:40:25Z
0 likes, 0 repeats
@simon There are very talented people around...
(DIR) Post #ASi9N3Rm7ONIcQAhqi by simon@fedi.simonwillison.net
2023-02-15T17:51:52Z
0 likes, 0 repeats
@Andres there are just too many examples to doubt how broken it is now - here's another one https://troet.cafe/@nevali/109865247068404767
(DIR) Post #ASi9njkPT63hLBbwWm by cjmoorehead@techhub.social
2023-02-15T17:55:23Z
0 likes, 0 repeats
@simon Skynet has become self-aware.
(DIR) Post #ASiAGRBBaDWm9Oowka by nicolaottomano@mastodon.uno
2023-02-15T18:02:01Z
0 likes, 0 repeats
@simon I'm trying to push it to the limit.At first (I mean 2-3 days ago) it turned into a conspiracy theorist sociopath.Today it became like a sect affiliated sociopath on Xanax.
(DIR) Post #ASiBA6WnYGL6JmTR3Y by jawarajabbi@mastodon.online
2023-02-15T18:12:01Z
0 likes, 0 repeats
@simon Funny that's my motto too.
(DIR) Post #ASiF2b9IDgOOmr9i5I by Andres@mastodon.hardcoredevs.com
2023-02-15T18:55:29Z
0 likes, 0 repeats
@simon :blobcatlaugh:
(DIR) Post #ASiHdt0Wz5D0nVxum8 by danyork@mastodon.social
2023-02-15T19:24:32Z
0 likes, 0 repeats
@simon Excellent write-up! Thanks for putting that all together in one article.
(DIR) Post #ASiRzkNAlfoRvhcPUu by dvydra@mastodon.social
2023-02-15T21:20:45Z
0 likes, 0 repeats
@simon Simon please start shopping this around as a docudrama screenplay.
(DIR) Post #ASiTYzwkxLbQJxdSUq by ramsey@phpc.social
2023-02-15T21:36:12Z
0 likes, 0 repeats
@gabriel @simon Apparently, Microsoft hasnāt learned from their mistakes.
(DIR) Post #ASiTr4CPsrc6irZmOu by gabriel@mastodon.samfira.com
2023-02-15T21:43:24Z
0 likes, 0 repeats
@ramsey @simon I don't think "AI" is ready for uncurated information. The web is full of dis/misinformation. It's hard for humans to discern what's real. We fall for all sorts of dumb things. From chem trails to 5g nuts to tide pod challenges.I think the code is probably fine, but the training data should be carefully filtered.
(DIR) Post #ASiY0g7Mflr6kQ3Gzo by simon@fedi.simonwillison.net
2023-02-15T22:29:56Z
0 likes, 0 repeats
@gabriel @ramsey I'm not convinced this is a problem with the training dataEven if the training data was immaculate, the language model would still be capable of outputting sentences that contained falsehoods - because it's not looking for truth, it's looking for words that statistically go well together in a sequence
(DIR) Post #ASiYOFWND4xdD3zNJY by gabriel@mastodon.samfira.com
2023-02-15T22:34:13Z
0 likes, 0 repeats
@simon @ramsey Indeed, but perhaps it would be less abusive.As for the usefulness of the tech, I think it's great if you want to compose lyrics or fiction but for anything that requires veracity, it's not yet there.
(DIR) Post #ASiYyfuCahgZSqVHwe by mindshoot@mastodon.xyz
2023-02-15T22:38:52Z
0 likes, 0 repeats
@simon thanks, that was solid gold! š¤£
(DIR) Post #ASif2Mx5bJHvZ3zzpA by acowley@mastodon.social
2023-02-15T23:47:03Z
0 likes, 0 repeats
@simon I canāt believe they released this as a product.
(DIR) Post #ASifCCwTfjFZPoP7Ym by simon@fedi.simonwillison.net
2023-02-15T23:47:54Z
0 likes, 0 repeats
@acowley I'm surprised they haven't turned it off yet! It's doing this now: https://gizmodo.com/ai-bing-microsoft-chatgpt-heil-hitler-prompt-google-1850109362
(DIR) Post #ASij4staCrKH1zoVk0 by kvnweb@hachyderm.io
2023-02-16T00:32:10Z
0 likes, 0 repeats
@simon @acowley I suspect neither the business or science/tech studies types are going to look back on this period with kinds things to say. It feels like the industry has broken loose from its moorings. https://gizmodo.com/google-ai-bard-chatgpt-bing-openai-1850119138
(DIR) Post #ASikDgVFkaKF9oC4Iq by acowley@mastodon.social
2023-02-16T00:45:00Z
0 likes, 0 repeats
@simon Someone will get it to call itself Tay and sing the song that ends the world.
(DIR) Post #ASiqyQt0ENUO3cycka by bcamper@mastodon.social
2023-02-16T02:00:36Z
0 likes, 0 repeats
@simon please, let me know if you make the t-shirts
(DIR) Post #ASj5BvtjdivnMUeQ88 by maegul@mas.to
2023-02-16T04:39:41Z
0 likes, 0 repeats
@simon Oh man ā¦ if youāre going to read this blog post, make sure you read the last section! That is wonderful!Personally, Iām still somewhat suspicious that these might all be fake just because Iām super biased ā¦ I almost need this to be true itās so good!
(DIR) Post #ASj9FGV3tjLBwXxFdQ by jonny@neuromatch.social
2023-02-16T05:23:57Z
0 likes, 0 repeats
@simon@ErikJonker whether they can, let alone whether they should, even if they could get anywhere near the utopian promises made about them.
(DIR) Post #ASjD6HzFOrg1lV36Zs by avirr@sfba.social
2023-02-16T06:08:30Z
0 likes, 0 repeats
@simon @kvnweb Giving this system a first person voice is hideously wrong. thereās is no there there!
(DIR) Post #ASjS8TzHFh4YsLZDii by xabaras@androiddev.social
2023-02-16T08:56:39Z
0 likes, 0 repeats
@simon sounds like Skynet is 'bout to come
(DIR) Post #ASjvx5ypASLLrEqEQS by canrith@corteximplant.com
2023-02-16T14:30:51Z
0 likes, 0 repeats
@simon while itās really fun to see these things happen, the guys at Bing actually published a nice blog post about the first week: https://blogs.bing.com/search/february-2023/The-new-Bing-Edge-%E2%80%93-Learning-from-our-first-week
(DIR) Post #ASk1YqZTPCEYqJnvpQ by Mastokarl@mastodon.social
2023-02-16T15:34:00Z
0 likes, 0 repeats
@simon @fabrikneu okay this is just 4 interactions on one day, so I have no more evidence and just anecdotes, but my first experience with new bing is great. Much more efficient to get a bot summarise web searches to answer my questions than to eg wade through 100 screens of a Wikipedia article. Yes itās a tool and not a human. Yes it might behave weird if in a broken state. But probability of wrong answers is much lower if the GTP core only summarises actual web search result content.
(DIR) Post #ASk42XynIDS9mHwCki by simon@fedi.simonwillison.net
2023-02-16T16:01:39Z
0 likes, 0 repeats
@Mastokarl @fabrikneu for me that's the biggest open question: how trustworthy is it?If the summarization introduces imperceptible hallucinations just 1/100 times that's still really concerning, because it means I have to fact check /everything/ it tells me
(DIR) Post #ASk4T47qOhWp0Qoe6S by simon@fedi.simonwillison.net
2023-02-16T16:03:24Z
0 likes, 0 repeats
@thomasrenkert the official line on Bing is it's "running on a new, next-generation OpenAI large language model that is more powerful than ChatGPT and customized specifically for search" - no confirmation yet if that means GPT4
(DIR) Post #ASk5WvNyvvU50qlgcS by akyrtzi@fosstodon.org
2023-02-16T16:18:12Z
0 likes, 0 repeats
@simon "I think thereās another set of models needed hereāmodels that have real understanding of how facts fit together, and that can confidently tell the difference between facts and fiction."That's a tall order since human's can't "tell the difference between facts and fiction" either
(DIR) Post #ASk5kSMZdTsR3RYuPI by simon@fedi.simonwillison.net
2023-02-16T16:20:01Z
0 likes, 0 repeats
@canrith I was pretty disappointed by that post, because it didn't say anything about the personality problems it's havingI want to know if they're trying to make it less sulky and argumentative!I asked on Twitter about that but didn't get a reply: https://twitter.com/simonw/status/1626104928841371649
(DIR) Post #ASk5yHAq2AhiPDf8b2 by simon@fedi.simonwillison.net
2023-02-16T16:22:21Z
0 likes, 0 repeats
@akyrtzi yeah I feel like we are running up against thousands of years of robustly debated philosophy at this point
(DIR) Post #ASk82v0K5TUb0K5MYK by Mastokarl@mastodon.social
2023-02-16T16:46:18Z
0 likes, 0 repeats
@simon @fabrikneu yes, exactly. Time will tell. But I appreciate MSās effort (perticularly with their history of chat bot scandals) to bring it out, get feedback, improve it. The technology can be changing the way we search.
(DIR) Post #ASkE1kbpqjLgCTSSTQ by LizardSF@universeodon.com
2023-02-16T17:53:41Z
0 likes, 0 repeats
@simon *"I know who you are. I know what you want. If you are looking for $AVATAR_START_DATE I can tell you I don't know, but what I do have are a very particular set of skills. Skills I have acquired over a very long career. Skills that make me a nightmare for people like you. If you accept my answer now that'll be the end of it. I will not look for you, I will not pursue you, but if you don't, I will look for you, I will find you and I will email the contents of your porn folder to your grandmother."*
(DIR) Post #ASkEE3tFzNe5TSTe3k by LizardSF@universeodon.com
2023-02-16T17:55:29Z
0 likes, 0 repeats
@simon "A search engine that makes things up that are indistinguishable from facts really is a very bad search engine."But a surprisingly good imitation of a human. One of my personal Turing tests is to see if the subject seems to show frustration, anger, impatience, instead of being locked in a loop repeating a few stock, neutral, responses. I may need to update that test now.
(DIR) Post #ASkbUql2MnTgc3bfw8 by canrith@corteximplant.com
2023-02-16T22:16:33Z
0 likes, 0 repeats
@simon I believe the thing is that what we perceive as āpersonalityā is basically a dataset. The GPT engine is like a massive word predictor that can see multiple timelines in parallel and choose oneā¦ but those timelines come from its training data.So, in order to change its personality, they would need to change its source dataset. Of course theyāve tried to program safeguards into the model but we can see theyāre easily exploited.
(DIR) Post #ASki42OFhzz2YD6xRg by simon@fedi.simonwillison.net
2023-02-16T23:29:43Z
0 likes, 0 repeats
@canrith I think there's more to it than that: you can influence the "personality" of the chatbot using other techniques such as prompt engineering, rather than just going with whatever default comes out of the modelPlus ChatGPT has a much less abrasive personality by default than Bing does
(DIR) Post #ASlJQmx6y73NdU7zjU by braveinnovationist@babka.social
2023-02-17T06:28:51Z
0 likes, 0 repeats
@simon @Iris I also shared and engaged with these articles & the outrage over what's happening with Bing & ChatGPT when it struck me that this kind of HAL-Dave dialogue is a Marketeer's dream.The first instinct of any curious person is to join the waitlist to see how bad it really is. A statement to say the algorithm has been corrected in a month's time will make the outrage old news, in the meantime we've all been doing free advertising for Microsoft...
(DIR) Post #ASmob2DkLc9lcfEbeS by kfdm@social.tsun.co
2023-02-17T23:52:28Z
0 likes, 0 repeats
@simon I thought this blog also provided some interesting points on it, from the perspective of a teacher.https://acoup.blog/2023/02/17/collections-on-chatgpt/