Post AZIKUGFPQ6FmWCcRhQ by soviut@hachyderm.io
(DIR) More posts by soviut@hachyderm.io
(DIR) Post #AZ9yTBx09qs4e7Pm6a by simon@fedi.simonwillison.net
2023-08-27T15:14:21Z
0 likes, 0 repeats
Here's the video, full set of slides and annotated transcript for the talk I gave at WordCamp US #WCUS on Friday: "Making Large Language Models work for you"https://simonwillison.net/2023/Aug/27/wordcamp-llms/
(DIR) Post #AZ9ydwNyMuAfYd9ibo by simon@fedi.simonwillison.net
2023-08-27T15:15:18Z
0 likes, 0 repeats
... including a colophon detailing how I built the annotated version of the talk https://simonwillison.net/2023/Aug/27/wordcamp-llms/#wordcamp-colophon
(DIR) Post #AZA3DNSs54etUfbGUK by nick@amok.recoil.org
2023-08-27T16:07:24Z
0 likes, 0 repeats
@simon perfect timing. I just got hold of a 128GB M1 and have got the basic large llama models working, but need to formalise it a bit more now
(DIR) Post #AZA3QJIItijwK8p9MW by simon@fedi.simonwillison.net
2023-08-27T16:08:02Z
0 likes, 0 repeats
@nick I only have 64GB and I'm beginning to regret not going larger!
(DIR) Post #AZA4CXAuanQtH4MSxM by webology@mastodon.social
2023-08-27T16:18:46Z
0 likes, 0 repeats
@simon > You should always share your prompts: I asked it for a “black background illustration alien UFO delivering a thumb drive by beam”.As I do more and more with LLMs and this tech, I have thought about this quite a bit. Does it make sense to embed some of these in HTML via a comment or something else? This feels potentially like a "view source" opportunity.
(DIR) Post #AZA4NeQJz09OwNs08G by watchie@mathstodon.xyz
2023-08-27T16:19:15Z
0 likes, 0 repeats
@simon how long did that process take in total?
(DIR) Post #AZA4YuQ6qeijjr8gi0 by grauhausen@mastodon.online
2023-08-27T16:20:16Z
0 likes, 0 repeats
@simon really interesting talk!
(DIR) Post #AZA4kV7Al7n2jKoDVw by osma@sigmoid.social
2023-08-27T16:24:47Z
0 likes, 0 repeats
@simonThanks again for a fantastic talk and a superb transcript! You're doing an awesome job!One question: you have a wide repertoire of skills and tricks for using pretrained LLMs, but you didn't mention fine-tuning, why? To me this is perhaps the single most exciting way of applying LLMs to solve practical problems - teaching them new skills just by example. And with techniques like PEFT and QLoRA it's very cheap and easy. Of course many things are possible without fine-tuning as well...
(DIR) Post #AZABJx8NAuzLUVesd6 by simon@fedi.simonwillison.net
2023-08-27T17:37:01Z
0 likes, 0 repeats
@watchie about three hours for this talk, mostly on a plane
(DIR) Post #AZABWmsyabdrTGZp8y by simon@fedi.simonwillison.net
2023-08-27T17:38:11Z
0 likes, 0 repeats
@osma mainly because I don't feel like fine-tuning has quite proveD itself yet as something most developers need in their toolbox - you can get a long way with just prompt engineering and tricks like RAG
(DIR) Post #AZABWrSvYsmBfvQajg by simon@fedi.simonwillison.net
2023-08-27T17:39:31Z
0 likes, 0 repeats
@osma I think that might change over the next few months though, as the community gains more experience fine-tubing things on top of Llama 2 (already starting to happen)I want there to be a good cookbook of proven fine-tuning recipes I can point people to
(DIR) Post #AZACKPJEWKLmpSWE7s by simon@fedi.simonwillison.net
2023-08-27T17:49:18Z
0 likes, 1 repeats
I finished on a positive note: the application of LLMs that excites me most is the possibility that they just might be able to give way more people the ability to take control of their computers, without having to spend months learning to program first!
(DIR) Post #AZACxCg5Pwdiwmhw92 by lewiscowles1986@phpc.social
2023-08-27T17:55:59Z
0 likes, 0 repeats
@simon it would be nice if we could achieve the same goals without mass theft of work without consent, but I think you're right that it could be a nice start
(DIR) Post #AZADKg0GYIJiYiL66S by simon@fedi.simonwillison.net
2023-08-27T18:00:52Z
0 likes, 0 repeats
@lewiscowles1986 yeah I talked a bunch about that in the presentation https://simonwillison.net/2023/Aug/27/wordcamp-llms/#how-they-are-trained
(DIR) Post #AZAGwnq8AEyBjKTDYO by matt@toot.cafe
2023-08-27T18:41:16Z
0 likes, 0 repeats
@simon Tip: Next time, it would be good to proofread the alt text using text-to-speech. Whether it's a screen reader or something else doesn't really matter. I caught some occurrences of "Al" rather than "AI" in the alt text. Thanks.
(DIR) Post #AZAHXw60QnTc3BbOtc by osma@sigmoid.social
2023-08-27T18:48:20Z
0 likes, 0 repeats
@simonVery good points!I've tried fine-tuning with the GPT-3 API, and it was surprisingly easy, but a bit costly. Lately I've also fine-tuned Llama 2 locally, and it was a lot more difficult to put together all the pieces in the right way. Even ChatGPT is of limited help because the tools and libraries are so new. A good cookbook would certainly help! There are of course many examples in blog posts etc. but it's hard to adapt those to your own situation.
(DIR) Post #AZAOJuGb8PH00CoWO0 by simon@fedi.simonwillison.net
2023-08-27T20:04:08Z
0 likes, 0 repeats
@matt Oh that's annoying! I tried to catch those in my edit but they're hard. to visually distinguish. I've gone through and fixed them now.
(DIR) Post #AZAPtXCeShheqoWL6O by leonoverweel@mastodon.social
2023-08-27T20:21:30Z
0 likes, 0 repeats
@simon yess this was my take too when OpenAI launched Codex two years ago :) https://dynamicallytyped.com/stories/2021/talking-with-codex/
(DIR) Post #AZAQV4ruG2MjycidyS by simon@fedi.simonwillison.net
2023-08-27T20:28:20Z
0 likes, 0 repeats
@earthshine I was really worried about that six months this ago. I'm not worried about that at all today.You may need $1m to train a model from scratch, but you can run capable models now on a consumer device - I even have ones that work OK on my iPhone!And fine-tuning a custom model on top of an openly licensed foundation model costs hundreds of dollars, not millions
(DIR) Post #AZAQk9sbuLUPrZEuNk by simon@fedi.simonwillison.net
2023-08-27T20:31:12Z
0 likes, 0 repeats
@earthshine I have a series of posts where I've been tracking the ability to run LLMs on personal devices here https://simonwillison.net/series/llms-on-personal-devices/
(DIR) Post #AZAWeyEEOngO0KGISm by lewiscowles1986@phpc.social
2023-08-27T21:37:25Z
0 likes, 0 repeats
@simon reading it now. As far as the alien right now. Have you tries OnnyxStream?https://github.com/vitoplantamura/OnnxStreamIt's stable diffusion in a few hundred MB, rather than gigabytes. Almost small enough to deploy.I Kick the tyres with this stuff even if I am firmly in the "this is [currently] hype [as well as other things]"You are one of the folks I listen to a lot about this though. You are prolific, indomitable!
(DIR) Post #AZAWqTxwjyu1ca8NCy by simon@fedi.simonwillison.net
2023-08-27T21:38:48Z
0 likes, 0 repeats
@lewiscowles1986 wow, I had not seen that one! Stable Diffusion on a Raspberry Pi is impressive
(DIR) Post #AZAX2UUGd0pOiGygsa by lewiscowles1986@phpc.social
2023-08-27T21:41:33Z
0 likes, 0 repeats
@simon I'm, running it on an M1 mac, but it works for x86 too.My pal Andy prior to me finding this was linking me 4GB in-browser models (which led to the learning that Chrome needs to run Chrome Canary to allocate 4GB without crashing).
(DIR) Post #AZAnLi361pxZXiSGum by simon@fedi.simonwillison.net
2023-08-28T00:44:37Z
0 likes, 0 repeats
@lewiscowles1986 I'm so impressed by that in-browser MLC one - wow, it supports Llama 70B now if you have 64GB of RAM! https://github.com/mlc-ai/web-llm/commit/473a0f57863fc3e0b44a9179192fcfd310b1762b
(DIR) Post #AZB4n7HA2gMep1ginI by lewiscowles1986@phpc.social
2023-08-28T03:59:45Z
0 likes, 0 repeats
@simonI do have more than 64GB RAM on some boxes. Might give it a go. 😂
(DIR) Post #AZBLSn2BHyLlTNZh5c by davidvasandani@social.coop
2023-08-28T07:05:47Z
0 likes, 0 repeats
@simon As someone with no formal programming experience, LLMs have allowed me to add functionality and tests to an open-source Go app, write a simple Rust app for Windows that provided substantial business value, and build a Node app that solved a personal problem. The words to describe this new ability are difficult to express.
(DIR) Post #AZDGDREhwLAu4MDLTk by soviut@hachyderm.io
2023-08-29T05:17:14Z
0 likes, 0 repeats
@simon the jaded part of me says "until the models they're given are all trained on ads".
(DIR) Post #AZDGlAJzKqpnSFlaVs by simon@fedi.simonwillison.net
2023-08-29T05:23:19Z
0 likes, 0 repeats
@soviut another reason I'm so keen on openly licensed models, in particular ones with transparency as to what they were trained on
(DIR) Post #AZGdVeBtOWbDsbQeem by johnadams@mastodon.social
2023-08-30T20:22:12Z
0 likes, 0 repeats
@simon Probably best no hype LLM presentation I have seen. Thank you. One question is when I ask something like “Find me the best all-road, flat-bar, no suspension, crmo steel bikes available today” I do not get a single good answer that match the criteria (i.e 10 garbage ones). The data is definitely there. After correcting it multiple times like these are “drop bars ones”,etc. I get maybe 1-2 good answers. Why do you think that is? Isnt it a very simple straightforward query?
(DIR) Post #AZGfiXr8STPtuCzvjU by simon@fedi.simonwillison.net
2023-08-30T20:47:06Z
0 likes, 0 repeats
@johnadams I wouldn't expect an LLM on its own to have enough depth of quality data to answer that, even the really good ones like GPT-4The solution to that will eventually lie in tools: AI search assistants like Bing and Bard that can run their own searches will hopefully be able to handle that kind of thing reliably some day, but they're not there yet in my experience
(DIR) Post #AZHcF3RSLwvOTXC15s by soviut@hachyderm.io
2023-08-31T07:43:01Z
0 likes, 0 repeats
@simon I'm interested in paid licensing models so all the artists I know who's work has been scraped can get royalties when those models are used. And it should be opt-in; register your artwork to be used for training and get paid when it gets used.
(DIR) Post #AZIAu1XbioLQtWmFTU by simon@fedi.simonwillison.net
2023-08-31T14:11:13Z
0 likes, 0 repeats
@soviut I'm not convinced that's technically possible with the way these models work - my understanding is that every image generated is influenced by all billion+ images in the training set at onceI believe we would need to invent entirely new model architectures to enable this. I'd love to be proved wrong about that!
(DIR) Post #AZIKUGFPQ6FmWCcRhQ by soviut@hachyderm.io
2023-08-31T15:58:34Z
0 likes, 0 repeats
@simon perhaps an adversarial model that identifies features from an artist's training data. I'm joking, but who knows.Until anything like that, it'd be more likely that trainers will pay to get access to sets of artist data. The artist registers their works with a guild/group and are paid each time the entire guild dataset is licensed to be used for training. Sort of like blanket licenses for stock images.