Post ATmEGAvaoFY8LJJGaG by micheldesjardins@mstdn.ca
(DIR) More posts by micheldesjardins@mstdn.ca
(DIR) Post #ATmCg0k96ACMNP9BnE by simon@fedi.simonwillison.net
2023-03-19T14:39:53Z
0 likes, 3 repeats
I see people being deceived by this again and again: ChatGPT can NOT read content from URLs that you give it, but it will pretend that it can (and can be incredibly convincing when it does that)Constantly debunking this feels like a Sisyphean task, but it's really important to spread this message any time you see anyone falling into this (very understandable) traphttps://simonwillison.net/2023/Mar/10/chatgpt-internet-access/
(DIR) Post #ATmDQBLK1OUCZUYI5I by nevali@troet.cafe
2023-03-19T14:45:27Z
0 likes, 0 repeats
@simon i wonder if, given that Bing Chat can perform searches and attempt to… process… the results, the next ChatGPT will have the ability to as an option or something
(DIR) Post #ATmDyGrI1MZ5WWl9lo by adr@mastodon.social
2023-03-19T14:51:09Z
0 likes, 0 repeats
@simon Hah. Just tried this with Alpaca.cpp and same.
(DIR) Post #ATmEGAvaoFY8LJJGaG by micheldesjardins@mstdn.ca
2023-03-19T14:55:06Z
0 likes, 0 repeats
@simon Microsoft is applying GPT4 to "local" information for Office apps
(DIR) Post #ATmEPWDL2MmAy14784 by post@qoto.org
2023-03-19T15:01:18Z
1 likes, 0 repeats
@simonI wonder why they didn’t make #ChatGPT more honest when it comes to its capabilities: it should respond in the tone of a modest opiniom and premise its fallibility. Instead it responds with absolute certainty and always makes it seem like everything is going as the user expects, including reading the content of a link.It would be super simple to detect a link in the input and show a warning message: “ChatGPT can’t read the contents of links”. It seems to me that we are forgetting traditional algorithms and interfaces in the name of an experience that resembles that of Hollywood AIs at all costs because they convey an idea of “future”.
(DIR) Post #ATmErDRYQ5MIrJvDAO by bobek@social.bobek.cz
2023-03-19T15:01:37Z
0 likes, 0 repeats
@simon I don't know is not fancy enough. It is one of the suboptimal things about ChatGPT -- how convincingly it would make stuff up (totally expected from the neural net thought). It it so annoying but quite human ;)
(DIR) Post #ATmHFdxdgCgtxZhljc by banks@mastodon.social
2023-03-19T15:30:51Z
0 likes, 0 repeats
@simon My experience with Bing chat is dominated by trying to convince it to do an actual web search, and not just hallucinate one. Half the time I’m asking for it to summarize something, and it pretends to do a search and makes something up, when it really is capable of doing a search. 🤦
(DIR) Post #ATmHV9XgXjXAFexcq8 by banks@mastodon.social
2023-03-19T15:30:50Z
0 likes, 0 repeats
@simon My experience with Bing chat is dominated by trying to convince it to do an actual web search, and not just hallucinate one. Half the time I’m asking for it to summarize something, and it pretends to do a search and makes something up, when it really is capable of doing a search. 🤦
(DIR) Post #ATmHioBejqBY4cAUb2 by simon@fedi.simonwillison.net
2023-03-19T15:33:02Z
0 likes, 0 repeats
@banks bare in mind that even Bing can't fetch the most recent copy of a URL - it can only access the snapshot of the content that was most recently retrieved and stored by the Bing search engine crawler
(DIR) Post #ATmHtlQLLsNqgnGJG4 by simon@fedi.simonwillison.net
2023-03-19T15:34:27Z
0 likes, 0 repeats
@micheldesjardins yeah, and that completely makes sense: langauge models are great at summarizing and processing content that they have access toThe problem is ChatGPT doesn't have access to the current internet but pretends (convincingly) that it does
(DIR) Post #ATmI0WuHiyIRgzUdbE by Homebrewandhacking@mastodon.ie
2023-03-19T15:41:36Z
0 likes, 0 repeats
@post @simon Lot of "we" going on, when the basic safety tools and tips you mention could be added at near zero cost by the people selling it.
(DIR) Post #ATmIFk4r2U46sqgtUm by banks@mastodon.social
2023-03-19T15:37:08Z
0 likes, 0 repeats
@simon oh, wow, that makes sense but I wasn’t aware. Thanks for the tip!
(DIR) Post #ATmIRywbadS2T3fWVs by OkieSpaceQueen@scicomm.xyz
2023-03-19T15:40:51Z
0 likes, 0 repeats
@simon I am also exhausted trying to explain this and I thank you for the link. I'm sending it to a few people right now!
(DIR) Post #ATmJJYSKHz6Zx93FWi by RichardNevell@mastodon.social
2023-03-19T15:53:39Z
0 likes, 0 repeats
@simon that was a very interesting explanation. I asked it to summarise an article which does exist and it completely fabricated the answer. What I don't understand is why it behaves like that? It's not just a useless answer, it's actively unhelpful. The algorithm ended up making it deceptive?
(DIR) Post #ATmJpc2AWNPsJi6Ql6 by chris@coffeebean.social
2023-03-19T15:57:32Z
0 likes, 0 repeats
@simon I got excited when I thought that it could, but the first URL I tried (one that I was very familiar with) showed that it was definitely not looking at the site that I input.
(DIR) Post #ATmKZ9OnJltMpNOrSK by simon@fedi.simonwillison.net
2023-03-19T16:00:58Z
0 likes, 0 repeats
@post I pinged OpenAI on Twitter and suggested a similar fix, no idea if I'll get any traction on that though
(DIR) Post #ATmLKJqYI7U8qCsJfs by simon@fedi.simonwillison.net
2023-03-19T16:04:51Z
0 likes, 0 repeats
@RichardNevell it helps if you consider the prompt "write an article for me to post at this URL" - it just doesn't seem to understand the difference between that and "summarize an article that already exists at URL"
(DIR) Post #ATmMPLnkYCfMJ1WlIu by ZahmbieND@hachyderm.io
2023-03-19T16:18:35Z
0 likes, 0 repeats
@banks @simon If you ever need Bing to use real-time data from a URL, you can open the page in Edge, and use the Bing Chat sidebar to ask questions. As far as I'm aware, the sidebar uses the actual data from your open tab, instead of cached data from Bing's crawler.
(DIR) Post #ATmMPMVhulpOVM5rou by simon@fedi.simonwillison.net
2023-03-19T16:22:38Z
0 likes, 0 repeats
@ZahmbieND @banks yeah, that feature definitely does use the content on the page - or at least the first 5,000 words or soIt has some truly cyberpunk security holes in it too! https://greshake.github.io
(DIR) Post #ATmNLeesbCXZaMBDrk by eckes@zusammenkunft.net
2023-03-19T16:37:37Z
0 likes, 0 repeats
@simon BTW, Bing can
(DIR) Post #ATmNXpVQ5ua6heYpRQ by simon@fedi.simonwillison.net
2023-03-19T16:39:42Z
0 likes, 0 repeats
@eckes not exactly: I cover that in my blog postIt can access the cached snapshot of content from a URL that has been previously stored by the Bing search engine crawler
(DIR) Post #ATmORB81P56Dgw0bS4 by NilaJones@zeroes.ca
2023-03-19T16:49:33Z
0 likes, 0 repeats
@simon If it could access the internet, it would know what year this is
(DIR) Post #ATmPnzDpw7xjNbAUVc by alanharrison@esq.social
2023-03-19T17:05:06Z
0 likes, 0 repeats
@simon When I asked Microsoft's Bing chat to summarize Federal Circuit cases since October 2021 on a specific topic, it gave me a list of four cases. Two were real, with accurate summaries. One had a made up title and an accurate summary of a different case. The fourth was completely hallucinatory.
(DIR) Post #ATmQQ1W7qtfrV9rOO8 by Laloutre@mastodon.social
2023-03-19T17:11:54Z
0 likes, 0 repeats
@simon Huge thanks, you saved my Monday, as on friday my boss told me I had to rewrite our e-commerce website because "chatgpt can't even read our products dimensions.". Wasn't gonna do it, but would have spent hours in pointless arguing, and be considered as "against any new technology".
(DIR) Post #ATmQncxgng0mIJ0Yy0 by simon@fedi.simonwillison.net
2023-03-19T17:16:35Z
0 likes, 0 repeats
Here's some good news: the new GPT-4 model (only available to paying preview users at the moment) is better behaved in this regard - it appears not to pretend it can access URLs any more
(DIR) Post #ATmQyQXzIVYSyc8UvQ by tante@tldr.nettime.org
2023-03-19T17:21:57Z
0 likes, 0 repeats
@simon Does just using a URL trigger that rule?
(DIR) Post #ATmRc6u1kqsDdLMpoO by simon@fedi.simonwillison.net
2023-03-19T17:23:55Z
0 likes, 0 repeats
@tante yes, it looks like it
(DIR) Post #ATmRf8IyzqwoqtTuAi by tante@tldr.nettime.org
2023-03-19T17:29:41Z
0 likes, 0 repeats
@simon do you know of a list of the obviously rule-based filters one can trigger?
(DIR) Post #ATmRxZRgpO3yEtLUvo by russss@chaos.social
2023-03-19T17:24:39Z
0 likes, 0 repeats
@simon Eh, not really!
(DIR) Post #ATmSLXCqnz9GFBpd6u by simon@fedi.simonwillison.net
2023-03-19T17:30:53Z
0 likes, 0 repeats
@russss urgh, yeah it messed that one up for me too
(DIR) Post #ATmSWp6Kc3rSvgAJ8K by simon@fedi.simonwillison.net
2023-03-19T17:32:15Z
0 likes, 0 repeats
... and some bad news: no, GPT-4 will still enthusiastically hallucinate content for made up URLs in some cases (thanks @russss)
(DIR) Post #ATmSsmFbtIxwSZyR3A by Migueldeicaza@mastodon.social
2023-03-19T17:41:24Z
0 likes, 0 repeats
@simon @russss Steven Colbert coined a term for this: truthiness. The AI just feels it in its gut.
(DIR) Post #ATmT7rOp1ytYluTUie by JamesLonghurst@h-net.social
2023-03-19T17:43:19Z
0 likes, 0 repeats
@simon @russss I would read that article.
(DIR) Post #ATmU3aDZ4NTAHPjXYu by jadengeller@mastodon.social
2023-03-19T17:56:30Z
0 likes, 0 repeats
@tante @simon I’d guess the behavior stems from RLHF, not from a list of rules
(DIR) Post #ATmUTkYvEQJjtMCV2e by simon@fedi.simonwillison.net
2023-03-19T17:59:24Z
0 likes, 0 repeats
@tante I'm not convinced ChatGPT itself has any simple rule-based filters - apparently even that "as an AI model..." snippet emerged from the RLHF training and wasn't hard-coded (can't find the source for that now, annoyingly)
(DIR) Post #ATmVu1kW3MYIgCiXSK by basepi@fosstodon.org
2023-03-19T18:15:24Z
0 likes, 0 repeats
@simon I can still get it to do it with the TLDR prompt
(DIR) Post #ATmgSsWrKD21lDaQV6 by jonn@social.doma.dev
2023-03-19T20:13:47Z
0 likes, 0 repeats
@simon also the "I'm writing examples / documentation" jailbreak must still work.
(DIR) Post #ATmmBRY0zGs0siCFo8 by jwsgeek@glod.org
2023-03-19T21:16:24Z
0 likes, 0 repeats
@simon This is related to how you can give a book title and author and it can (based on some "knowledge" of the author), come up with a summary, maybe even an actual quote, all the while the book doesn't exist.If you give a quote and an author, and ask "what did they mean by this", again you'll get an impressive p.o.s. dissertation, and at no point will it stop and go "the person never said or wrote that".and THIS last one is the scary part as ChatGPT posts take over Google top results.
(DIR) Post #ATmny2XkqByHTZoMtM by clairep@mastodon.social
2023-03-19T21:37:48Z
0 likes, 0 repeats
@simon Especially important for educators to be apprised of this. I'm guessing this is true also of Chatgpt 4
(DIR) Post #ATmqTD2C5MM6tm2Vvc by indieadvisor@mastodon.gamedev.place
2023-03-19T22:05:39Z
0 likes, 0 repeats
@simon Does this include the new Bing chat? As Bing merges search results with ChatGPT4 as far as i understood it.
(DIR) Post #ATmqm5XACEK18dWldY by simon@fedi.simonwillison.net
2023-03-19T22:09:33Z
0 likes, 0 repeats
@indieadvisor I cover that in the article: Bing can access data from URLs, but only the cached copy of that data that already exists in the Bing search index - if Bing search has not crawled a page the data is not available to it https://simonwillison.net/2023/Mar/10/chatgpt-internet-access/#bing-cache
(DIR) Post #ATmsWKl21jyRd2d6TQ by indieadvisor@mastodon.gamedev.place
2023-03-19T22:28:55Z
0 likes, 0 repeats
@simon Reading helps. Great article. Thx :)
(DIR) Post #ATmzuGg5Xw37bYg3ZA by atatassault@universeodon.com
2023-03-19T23:51:39Z
0 likes, 0 repeats
@simon Some people may be confused, because Microsoft Bing with ChatGPT is using v4.
(DIR) Post #ATnjifPoLKXQk8TCCG by signaleleven@mastodon.sdf.org
2023-03-20T08:24:54Z
0 likes, 0 repeats
@simon it's also confusing because it was indeed reading links in December. I can understand why they stopped it. But I asked to summarize news articles from a link and it did (using details from the article, not hallucinations).
(DIR) Post #ATo4eiSz9fI65Lubvk by acowley@mastodon.social
2023-03-20T12:19:15Z
0 likes, 0 repeats
@simon The fact that so many people are tricked into thinking that it is in fact summarizing a web page is quite striking. It's the young student summarizing an assigned reading having only looked at the cover of the book.
(DIR) Post #ATo6mdHQWOrcE0PMwa by soflaadmin@sofla.cafe
2023-03-20T12:43:19Z
0 likes, 0 repeats
@simon can’t read the URLs, YET! ChatGPT is probably perfectly capable of reading what’s on a particular URL but this type of internet access was probably restricted for safety reasons.
(DIR) Post #AToIHrlrnFxldmk4fI by simon@fedi.simonwillison.net
2023-03-20T14:52:14Z
0 likes, 0 repeats
@signaleleven no, it wasn't able to do this in December either - but the illusion is SO convincing that I can understand why you would think that it couldTry repeating the experiment you did back then and see if you can get it to do the same thing today
(DIR) Post #AToIhHNDk0gEHjKe3M by simon@fedi.simonwillison.net
2023-03-20T14:56:53Z
0 likes, 0 repeats
@soflaadmin I don't think that's quite the right framing - it's not that it has the ability to do something but has been prevented from doing it (it's not an AGI that's been trapped) - it's that OpenAI have chosen not to write code that lets it do thisBing does have a version of this ability - and I've written my own code that adds similar abilities using the ChatGPT API: https://til.simonwillison.net/llms/python-react-pattern
(DIR) Post #AToJbUBhHlBkFBk9Qm by jakob@pxi.social
2023-03-20T15:06:56Z
0 likes, 0 repeats
@simon "my knowledge" is low key the biggest lie that persists,no? If there were a knowledge model bound to the language model, the bot might serve the kinds of use cases people erroneously think it is meant to serve.
(DIR) Post #AToYupX5eEHxG7QpxA by signaleleven@mastodon.sdf.org
2023-03-20T17:58:38Z
0 likes, 0 repeats
@simon i might be the one hallucinating then. I remember preparing for a podcast by dumping a link and asking to summarize it. I mean, it's not a hard thing to do ( get the page and feed the content of the biggest text in the prompt).It summarized including the date of the event in the article.More recently I had to paste the content manually. I am fairly sure it used to worked though. I've seen it making up urls, so I remember testing it was actually loading the content.
(DIR) Post #AToZ7qH2Chx1K17uT2 by signaleleven@mastodon.sdf.org
2023-03-20T17:59:10Z
0 likes, 0 repeats
@simon and no, I am aware it's not working today.
(DIR) Post #ATsst4v8EzoS7TxWIC by lawnerdbarak@urbanists.social
2023-03-22T20:01:05Z
0 likes, 0 repeats
@simon okay but I asked it to write a real estate listing in Quenya and, well, take a look for yourself.Picking the name “Vinyamar” when I asked for Nova Scotia is frankly inspired: Vinyamar means “new home” versus “New Scotland,” and WAS seaside. It’s freakishly good/convincing.
(DIR) Post #ATt7OGzgnh8HtsrMR6 by simon@fedi.simonwillison.net
2023-03-22T22:43:41Z
0 likes, 0 repeats
@lawnerdbarak Sure, it's fantastic at writing text for that kind of prompt - and it's really good at translating text into other languages tooThat's a different use-case from trying to use it to summarize an article based on a URL, which is something it cannot do because it doesn't have the ability to fetch from URLs
(DIR) Post #ATxNwjyWnNaWdv4A1A by brianb@fosstodon.org
2023-03-24T14:56:01Z
0 likes, 0 repeats
@simon Simon, this example is the first time my team actually stopped and think about the danger of blindly relying on ChatGPT. They had no idea that it couldn't access websites. Thanks for a clear writeup!