Post AYuD8UTuSJauov2Fpg by pjbrunet@noagendasocial.com
(DIR) More posts by pjbrunet@noagendasocial.com
(DIR) Post #AYu3Okl0i9pmOH9Mci by simon@fedi.simonwillison.net
2023-08-19T22:54:14Z
0 likes, 0 repeats
Do you use GPT-4 (via ChatGPT or through the OpenAI API)?
(DIR) Post #AYu3rhJ4cT2q5Qstmq by simon@fedi.simonwillison.net
2023-08-19T22:57:37Z
0 likes, 0 repeats
(Posting this because it's been dawning on me that those of us who are most convinced by the utility of these AI models might be stood alone on our GPT-4 island, while everyone else throws a few moderately sophisticated questions at GPT-3.5 et al, gets obviously wrong results and writes the entire thing off as hype)
(DIR) Post #AYu46mMG7Y9a64Z93w by acdha@code4lib.social
2023-08-19T22:59:56Z
0 likes, 0 repeats
@simon Only casually for personal use - my employer needs copyright so I avoid the coding tools where it’d be most interesting
(DIR) Post #AYu4VN27PnAgnDs5tQ by MattHodges@mastodon.social
2023-08-19T23:02:07Z
0 likes, 0 repeats
@simon I use it mostly via Bing
(DIR) Post #AYu4iUj87ZeAQ3FQJ6 by mattb@mastodon.nz
2023-08-19T23:04:19Z
0 likes, 0 repeats
@simon It's not just quality/hype stopping use of GPT-4/ChatGPT/OpenAPI. I'm absolutely convinced about the utility of the models, I just fundamentally distrust OpenAI and have zero desire to put more data / usage into their ecosystem.
(DIR) Post #AYu5M8waWqueRaRsXY by rweir@mastodon.social
2023-08-19T23:09:24Z
0 likes, 0 repeats
@simon fwiw the openai website said they’d open it up to all api users in July sometime, but they’ve now changed that to “soon.” So, still waiting.
(DIR) Post #AYu5k3enpwrIzsX7D6 by jamesturk@mastodon.social
2023-08-19T23:14:58Z
0 likes, 0 repeats
@simon definitely seen this divide in convo among some peers too, until people started discussing specifics (copilot vs gpt4 vs 3.5) it was confusing how varied the takes were.
(DIR) Post #AYu66fIcZ49w74wCHY by simon@fedi.simonwillison.net
2023-08-19T23:16:37Z
0 likes, 0 repeats
@kjrsten The Llama 2 models I've been able to run locally - 7B and 13B - aren't even at ChatGPT 3.5 quality (though pretty astonishing how good they are considering they run on my laptop)I've not played enough with Llama 70B to have a good feel for it, but my current hunch is it's around about 3.5 levelI don't think anything out there comes close to GPT-4 yet
(DIR) Post #AYu6iBrSSm5toh7rVo by mborus@mastodon.social
2023-08-19T23:24:08Z
0 likes, 0 repeats
@simon I think by now most “normal” people have played with it, and are a bit bored by the hype. And they have not found a usage for it, where the occasional wrong answer isn’t a problem. I found mine (language learning and simple beginners CSS, JavaScript problems) and still am amazed on a daily basis.
(DIR) Post #AYu6tJvCpp4u08Bb9M by nighthawk@aus.social
2023-08-19T23:25:53Z
0 likes, 0 repeats
@simon Yep, via @witkit.
(DIR) Post #AYu7PGkOAmV7QFMQoy by simon@fedi.simonwillison.net
2023-08-19T23:33:38Z
0 likes, 0 repeats
@mborus I wouldn't expect most "normal" people to have played with GPT-4 directly yet because you generally still have to pay for access
(DIR) Post #AYu7dzec0VMukigD4a by pjbrunet@noagendasocial.com
2023-08-19T23:28:28Z
0 likes, 0 repeats
@simon Tried the Google one, it's OK. Will I ever need bland, unoriginal writing? Probably not. I read enough books such that I never require a thesaurus, and never feel like I cannot verbalize my thoughts without outside help.I don't think LLMs are useless. I can see it saving time in the CLI, processing text, instead of sed, awk, regex.
(DIR) Post #AYu7e0Q79tMl82u97A by simon@fedi.simonwillison.net
2023-08-19T23:35:52Z
0 likes, 0 repeats
@pjbrunet I have been deeply unimpressed with Bard so far, I'm really hoping it gets betterGetting LLMs write for me is one of the things I'm most interested in - but I use them to help me understand the world a whole bunch, on top of all the coding assistance https://simonwillison.net/2023/Aug/3/weird-world-of-llms/#tips-for-using-them
(DIR) Post #AYu82teJTCXzMuPly4 by mborus@mastodon.social
2023-08-19T23:43:27Z
0 likes, 0 repeats
@simon that’s true. They will only have played with the free one. But if you play with gpt-4 you can get a lot of nonsense. Just did some random tests on trivia (https://chat.openai.com/share/83bebce4-32ad-49da-a573-65eaaae6a26d) and it’s hit and miss. A normal person would probably concentrate on what it gets wrong.
(DIR) Post #AYuAzDOYPga3JTynnE by aiefel@mastodon.social
2023-08-20T00:19:18Z
0 likes, 0 repeats
@simon have less trust in how the product is applied than the product itself.
(DIR) Post #AYuD8UTuSJauov2Fpg by pjbrunet@noagendasocial.com
2023-08-20T00:05:43Z
0 likes, 0 repeats
@simon Sorry, I meant Claude. I was impressed by its ability to do charts, like "remove the 2nd column and sort by..." For code, I'm thinking if you're doing something common and predictable, then wonderful, the LLM has the answer. Like for some mindless CRUD job, you saved time. But if that code is so predictable an LLM can write it, chances are you're not really innovating. It only regurgitates what others have commonly done already.
(DIR) Post #AYuD8VHtSTZpJwQAk4 by simon@fedi.simonwillison.net
2023-08-20T00:42:20Z
0 likes, 0 repeats
@pjbrunet I've been founding that outsourcing even the simplest coding stuff to it has given me an enormous boost in terms of the scope of coding challenges I can take on - if I don't have to waste any time constantly reviewing how trivial common coding pattern works I can spend much more time on the interesting problems
(DIR) Post #AYuDKVOgsP5N8TK6d6 by smach@masto.machlis.com
2023-08-20T00:42:29Z
0 likes, 0 repeats
@simon I coded an internal app at work where people can choose 3.5 or 4. For generating SQL, 4 is noticeably superior.
(DIR) Post #AYuI3v3mi1jNZjSHj6 by xk3@mastodon.social
2023-08-20T01:36:28Z
0 likes, 0 repeats
@simon how do you feel about Claude? I've been using it instead of ChatGPT and it does seem a bit better but I haven't used GPT4
(DIR) Post #AYuJbUrciD8285LFVQ by dws@mastodon.cloud
2023-08-20T01:54:42Z
0 likes, 0 repeats
@simon Not yet because 3.5 and the others have been meeting my feeble needs.
(DIR) Post #AYuJyWeuClRqMANFwG by llimllib@hachyderm.io
2023-08-20T01:57:19Z
0 likes, 0 repeats
@simon No, I tried and didn’t find it useful (doesn’t really fit any of your categories)
(DIR) Post #AYuMJv7ce8NcDX8T9k by simon@fedi.simonwillison.net
2023-08-20T02:26:19Z
0 likes, 0 repeats
@xk3 I'm using Claude a lot myself - I really love the 100,000 token context limit, and it appears to have slightly more recent training data than the OpenAI September 2021 cut-offClaude did flat-out refuse to summarize something for me today because it happened to mention a bomb threat somewhere in the conversation though
(DIR) Post #AYuMhDHpHEXdDL10XQ by eichin@mastodon.mit.edu
2023-08-20T02:26:50Z
0 likes, 0 repeats
@simon I'm fascinated by what you've been able to get out of it, but (even ignoring the copyright problems/threats) there's nothing I've seen it do so far that I personally want done :-) (Now, when it gets to the point of doing code review, or anything even vaguely security related, it'll be another story...)
(DIR) Post #AYuMwjDqp2xubaYPtw by simon@fedi.simonwillison.net
2023-08-20T02:32:51Z
0 likes, 0 repeats
@eichin I used it for some inexcusable Shakespeare analysis today! https://chat.openai.com/share/3464ba30-d95b-4f29-a525-2324015af49b
(DIR) Post #AYvgqx9uHqAeKeC0tE by raiderrobert@mastodon.social
2023-08-20T17:49:38Z
0 likes, 0 repeats
@simon I had a thought today to try out ChatGPT 4 asking it about finding references to its summaries.So I asked it to summarize the Baked Data Pattern, which it did a pretty good job of. When I asked for references, it linked to several articles that don't exist, and it made comments about how the Jamstack is the same thing.So suffice to say the hallucinations are still very real (phrasing?)
(DIR) Post #AYwH09A4cwL6VuxCng by simon@fedi.simonwillison.net
2023-08-21T00:36:07Z
0 likes, 0 repeats
I thought it would be interesting to run the same poll on Twitter in order to compare the resultsOf the people who follow me and are active and vote in polls... the Mastodon crowd are about three times as likely to not want to use GPT-4, and just under half as likely to have tried it than the Twitter crowdThis more-or-less matches my intuition, which is that Mastodon users are a lot more actively resistant to weird ethically complicated hosted LLMs
(DIR) Post #AYwLLqHftQkG9c6sbY by simon@fedi.simonwillison.net
2023-08-21T01:24:40Z
0 likes, 0 repeats
@dbreunig Yeah, that's very much part of my intuition about what's going on here
(DIR) Post #AYwPh7eGvXhJuI5AAq by Biggles@qoto.org
2023-08-21T02:13:32Z
0 likes, 0 repeats
@simon To LLMs in general, tbh. There are some tools that have hidden issues, gotchas or legal issues, and shouldn't be used by people who aren't deeply aware of the drawbacks and limitations. You are aware of them. I don't see you making assumptions and coming to great harm because you, say, trusted the LLM not to hallucinate entire legal filings. You are in the minority, sadly - I know otherwise sane people who think they can replace actual humans with just a LLM. And its causing harm to those real people. Until someone comes up with a LLM that can readily tell me "I'm sorry, I don't have the data I need to give you a good, correct answer." - I fear they'll do more harm than good. And i don't see that coming any time soon.
(DIR) Post #AYwX8vYECLUqAtScs4 by j2kun@mathstodon.xyz
2023-08-21T03:36:57Z
0 likes, 0 repeats
@simon to be fair, "do you use" is a very different question from "have you tried"