Post AU8E4rO2GJ5V8umzia by markus@hachyderm.io
(DIR) More posts by markus@hachyderm.io
(DIR) Post #AU7EfqA1aZ0utfuRVY by simon@fedi.simonwillison.net
2023-03-29T18:11:01Z
0 likes, 1 repeats
If you haven't tried running an open-ish alternative to ChatGPT on your own computer yet gpt4all is the new easiest way to do so: checkout the repo, download a 3.9GB model file and you can run the compiled binaries in the chat/ folder directly on Intel/M1 Mac, Windows or Linux: https://github.com/nomic-ai/gpt4allIt's not too bad for something that runs locally!
(DIR) Post #AU7ErhFmWzLKkNFv1c by msprout@fosstodon.org
2023-03-29T18:12:45Z
0 likes, 0 repeats
@simon Llama.cpp absolutely SMACKS. it is very scriptable, too. i've engineered a few fun prompts for myself, for some smaller tasks.
(DIR) Post #AU7F4szQCwIWuVtP5k by pjbrunet@noagendasocial.com
2023-03-29T18:17:44Z
0 likes, 0 repeats
@simon Hmm can it scrape the live Internet? Can I prompt it from the terminal? Sounds cool.
(DIR) Post #AU7G52KDbBJnomFuwC by simon@fedi.simonwillison.net
2023-03-29T18:27:07Z
0 likes, 0 repeats
Added a new tag to my blog covering Homebrew Large Language Models - turns out I've written 12 posts about those alread! https://simonwillison.net/tags/homebrewllms/
(DIR) Post #AU7G6B6sQFpjLxYlV2 by simon@fedi.simonwillison.net
2023-03-29T18:28:28Z
0 likes, 0 repeats
@pjbrunet You can prompt it from the terminal, but it can't produce anything that's not in its 3.9GB model file... although it wouldn't be very hard to provide it tools to let it scrape things from the internet using a version of this pattern: https://til.simonwillison.net/llms/python-react-pattern
(DIR) Post #AU7GhNF6HCb1hF8tTE by osma@sigmoid.social
2023-03-29T18:32:39Z
0 likes, 0 repeats
@simonI already tried alpaca.cpp running Alpaca-LoRA and it was very easy, gotta try this one next!#GPT4all seems like it could be a great base model for finetuning it for even more specific instructed tasks - I wonder if it's easy to re-finetune a model that is already finetuned with #LoRA? Just fold in the LoRA weights somehow?
(DIR) Post #AU7GuAO3NtwdaeQk1Q by osma@sigmoid.social
2023-03-29T18:34:11Z
0 likes, 0 repeats
@simon@pjbrunet How many days until we see a ReActGPT4All project announcement? ;)
(DIR) Post #AU7HdmGNPO9U5maXgG by PlaneSailingGames@chirp.enworld.org
2023-03-29T18:44:37Z
0 likes, 0 repeats
@simon interesting, thanks for sharing. Must not let the urge to try it interrupt my real work!
(DIR) Post #AU7HppFq72ZoqR5A3M by shajith@mastodon.social
2023-03-29T18:46:23Z
0 likes, 0 repeats
@simon I just wanted to thank you for your AI coverage, it’s been indispensable for me and I’m sure for a lot of people. How far away do you think we are from commoditized LLMs running on consumer hardware? The Llama based experiments make it sound like it is definitely plausible, but I can’t tell if there’s some insurmountable last mile hurdle.
(DIR) Post #AU7I0ZiaDVIJqReWJs by mcc@mastodon.social
2023-03-29T18:48:32Z
0 likes, 0 repeats
@simon Something I noticed, I don't see a single word in this github repo about the licensing of either the model or the scripts. I assume the model licensing is inherited from llama but distributing code with no posted license or copyright release of any kind seems pretty strange.
(DIR) Post #AU7ID6CBIC3W2xT7Bo by misc@mastodon.social
2023-03-29T18:51:03Z
0 likes, 0 repeats
@simon Afraid that my MacBook Air will start showing sparks of general intelligence.
(DIR) Post #AU7IYKsXtZt5fRPWvg by devdevcharlie@hachyderm.io
2023-03-29T18:54:38Z
0 likes, 0 repeats
@simon thanks for sharing!!
(DIR) Post #AU7JWu7G0roKKSAngW by jedfox@mastodon.social
2023-03-29T19:05:26Z
0 likes, 0 repeats
@simon @pjbrunet I tried using that pattern with alpaca and it kept on guessing instead of trying to run external code :(I think someone needs to train a model specifically to use these sorts of external resources.
(DIR) Post #AU7MC1FMbbwQE3hK4W by tfardet@fediscience.org
2023-03-29T19:35:22Z
0 likes, 0 repeats
@simon you might also be interested in this post: https://floss.social/@alcinnz/110107929126671106
(DIR) Post #AU7OdTlcsL7cPJ0g1A by oldwired@hachyderm.io
2023-03-29T20:02:37Z
0 likes, 0 repeats
@simon is there anything yet I could easily (additionally) train with my own Text corpus so it can answer questions relating to it?(The text corpus would be a 800p manual)
(DIR) Post #AU7XHbMQrRazAurucK by simon@fedi.simonwillison.net
2023-03-29T21:39:04Z
0 likes, 0 repeats
@shajith Honestly it feels SO close to me. I'd be shocked if we didn't have LLMs on consumer hardware (outside of just us early adopter hacker types) within a yearI can run Stable Diffusion on my phone and laptop really easily already
(DIR) Post #AU7XfI5CBhf6Kr6OCO by simon@fedi.simonwillison.net
2023-03-29T21:41:08Z
0 likes, 0 repeats
@oldwired Maybe! Replicate fine-tuned LLaMA on the Simpsons: https://replicate.com/blog/fine-tune-llama-to-speak-like-homer-simpson - I'm not 100% convinced the same trick would work for something like a software manual, but it wouldn't surprise me if it didAlternatively you can use this pattern, which doesn't require any fine-tuning at all: https://simonwillison.net/2023/Jan/13/semantic-search-answers/
(DIR) Post #AU7YER3mnoG4JUCX1U by simon@fedi.simonwillison.net
2023-03-29T21:44:56Z
0 likes, 0 repeats
@mcc It's not clean at all from a license POVLLaMA is licensed by Facebook for non-commercial use only, and apparently they've started sending out DMCA take-downs: https://twitter.com/theshawwn/status/1638925249709240322And the fine-tuning data was generated using OpenAI's GPT, which has terms that say you can't use it to help train a competing modelCerebras-GPT is much more interesting - it appears to be cleanly Apache 2 licensed. It's not instruction-tuned yet though: https://simonwillison.net/2023/Mar/28/cerebras-gpt/
(DIR) Post #AU7YmMI12sCrMc25fk by mcc@mastodon.social
2023-03-29T21:54:42Z
0 likes, 0 repeats
@simon I don't know the details of LlaMA's development, but it's constantly mindblowing to me that a corporation will perform mass scraping of copyrighted works, create a derivative work (the model) in violation of the plain reading of the law and relevant licenses, and then try to enforce copyright and license terms on a work that's already made entirely of infringement and ignoring license terms.
(DIR) Post #AU7ZDput08Bx4KJgC8 by shajith@mastodon.social
2023-03-29T21:59:19Z
0 likes, 0 repeats
@simon That sounds right to me too. I’m very interested in how the nature of applications will change when the LLM is client side instead of server side. Eg: a product that’s a collection of langchain style agent pipelines, but the LLM bits of that run on clients? Built-in LLMs in web browsers that allow web apps to use that as a native capability? Like, say, how AJAX changed things. Market structure would change too: Such a world doesn’t look like an OpenAI takes all market, for instance.
(DIR) Post #AU7cEvaPhqKgnzx7lg by sri@floss.social
2023-03-29T22:35:16Z
0 likes, 0 repeats
@simon You know, I'm curious - it would be interesting to train a gpt4 AI the ethical way with every protection - and I'm wondering could you use it as validator like in a GAN training? That way you could track unethical use of AI against these corporate ones? Is that out there idea?
(DIR) Post #AU7caUAQKYxu8Z4kNc by pjbrunet@noagendasocial.com
2023-03-29T22:41:09Z
0 likes, 0 repeats
@simon Maybe I'm lazy but it seems totally useless, like Eliza from the 1980s. It can't remember anything and it has no access to the Internet. > I'm giving you a number to remember: 12345. Do you remember the number?Yes, I do!> What is the number I told you to remember?It was actually "67890". Sorry for any confusion or misunderstanding caused by my original statement.Meh. It cannot even tell me today's date.
(DIR) Post #AU7d2Xw3uhM3vtjqZE by simon@fedi.simonwillison.net
2023-03-29T22:46:12Z
0 likes, 0 repeats
@pjbrunet If you want it to remember numbers you'll need to copy the previous conversation in with each new prompt - which is what ChatGPT does, it just does it invisiblyIf you want it to know today's date, you'll need to tell it that as part of your prompt tooThis is a raw language model - things like ChatGPT have extra layers of trickery going on to help work around these kinds of issues
(DIR) Post #AU7dV3mz4mbkpj5YsS by nick@amok.recoil.org
2023-03-29T22:49:38Z
0 likes, 0 repeats
@simon Huh. I wonder to what degree LoRA is introducing new possible output vs preventing lower quality responses. I guess it's more the latter?
(DIR) Post #AU7eDeIQhk58nf3hdA by simon@fedi.simonwillison.net
2023-03-29T22:54:55Z
0 likes, 0 repeats
Worth noting gpt4all has a bunch of of licensing problemIt's built on top of LLaMA which Facebook say is for non-commercial use only, and apparently they've started sending out DMCA take-downs: https://twitter.com/theshawwn/status/1And the fine-tuning data was generated using GPT3, which has terms that say you can't use it to help train a competing modelCerebras-GPT is more interesting - it appears to be cleanly Apache 2 licensed. though, not yet instruction tuned: https://simonwillison.net/2023/Mar/28/cerebras-gpt/
(DIR) Post #AU7evOkXrpzjKD4RyS by vitriolix@mastodon.social
2023-03-29T23:05:20Z
0 likes, 0 repeats
@simon this is great. they need to mirror their gpt4all-lora-quantized.bin. download though.
(DIR) Post #AU7fVihZADi7d6zrzE by faassen@fosstodon.org
2023-03-29T23:12:03Z
0 likes, 1 repeats
@simon It's rather rich that the license of GPT-3 should be respected when training something, even though it is itself trained on all sorts of text with all sorts of licenses.So if I want to prevent any competing LLM to use my blog content all I need to do is put some GPT-3 output in it?
(DIR) Post #AU7fszjE1bRX67s4PY by bplein@bvp.me
2023-03-29T23:16:03Z
0 likes, 0 repeats
@simon Related: doesn’t most code generated by these systems suffer the same problem? It will offer code based on licensed code, without any mention of the licenses involved. People who use these trained systems to come up with code are actually then using code that has been stripped of all licenses and attribution.
(DIR) Post #AU7gCDgyKfuFuJz6OG by simon@fedi.simonwillison.net
2023-03-29T23:19:50Z
0 likes, 0 repeats
@bplein I'm not convinced that the licensed code thing is a material problem for the way I'm using these systems: the snippets they generate for me are clearly recombined from literally millions of examples they've seenI believe it's possible to get it to spit out a chunk of "copied" code if you ask a very specific question that can only be answered by one fragment in the training set, but I don't think my usage patterns risk triggering that edge-case - not without me noticing at least
(DIR) Post #AU7h2TqvUd1J5RYnjM by pjbrunet@noagendasocial.com
2023-03-29T23:31:03Z
0 likes, 0 repeats
@simon Well, I hope it evolves but for now it just seems like random useless words. I'm writing this from memory...> HelloWorld> What's your name?My name is [name]. Can we talk about something else? <3> Don't use emojis.OK, I'm capable of GIFs blah blah (Control-C)> List my preferences.I want to communicate with mutual understanding. <3WTF hearts?! It's like talking to a retarded Nigerian spammer.
(DIR) Post #AU7sAxNigibuTv8xiy by zellyn@hachyderm.io
2023-03-30T01:33:57Z
0 likes, 0 repeats
@simon This one didn't work terribly well for me 😞> What is the mathematician Hamming known for?Hammond was a mathematician who made significant contributions to number theory, specifically in his work on prime numbers and Fermat's Last Theorem (which he proved). He also worked extensively with cryptography.
(DIR) Post #AU8E4rO2GJ5V8umzia by markus@hachyderm.io
2023-03-30T05:37:54Z
0 likes, 0 repeats
@simon @bplein GitHub’s copilot docs suggested it would spit out code fragments from public code about 1 percent of the time, and there’s an option to filter those out: https://github.com/features/copilot (in the faq section)
(DIR) Post #AU8T2AqcwqIDwbL8oS by lcfd@fosstodon.org
2023-03-30T08:25:33Z
0 likes, 0 repeats
@simon I didn't know about gpt4all, thanks!
(DIR) Post #AU8cuRobOERNKufhM8 by quinn@social.circl.lu
2023-03-30T10:17:37Z
0 likes, 0 repeats
@simon oooooohhhh
(DIR) Post #AU8eCFhFHkZSuxHgqO by jzitt@c.im
2023-03-30T10:31:44Z
0 likes, 0 repeats
@simon I'm looking forward to a tool that would work completely offline from a local collection of text. Does this tool or others do that yet?
(DIR) Post #AU8l7nM9PXxvXiEtPc by discgolfer@uniflydiscs.com
2023-03-30T11:48:49Z
0 likes, 0 repeats
@simon This is cool. I'm going to check it out. Thank you.
(DIR) Post #AU8xcvorC89ODXHPHM by simon@fedi.simonwillison.net
2023-03-30T14:09:48Z
0 likes, 0 repeats
@jzitt everyone wants that! There are tricks: you can implement something like this for example, but it still requires quite a bit of work: https://simonwillison.net/2023/Jan/13/semantic-search-answers/I'm still waiting for clear evidence that fine-tuning against local docs can produce great results - though it does at least work for Simpsons scripts: https://replicate.com/blog/fine-tune-llama-to-speak-like-homer-simpson
(DIR) Post #AU9Fw8uQY7x9UYCB4S by simon@fedi.simonwillison.net
2023-03-30T17:34:39Z
0 likes, 0 repeats
@sri apparently OpenAI used something like that for ChatGPT https://www.technologyreview.com/2023/03/03/1069311/inside-story-oral-history-how-chatgpt-built-openai/"The researchers are using a technique called adversarial training to stop ChatGPT from letting users trick it into behaving badly (known as jailbreaking). This work pits multiple chatbots against each other: one chatbot plays the adversary"
(DIR) Post #AU9GBwqPAr2ChrqnrM by simon@fedi.simonwillison.net
2023-03-30T17:37:17Z
0 likes, 0 repeats
@o @faassen apparently Google considered using ChatGPT output to train Bard! https://news.ycombinator.com/item?id=35370915
(DIR) Post #AU9HjguZ1DQd5jRwSu by sri@floss.social
2023-03-30T17:55:17Z
0 likes, 0 repeats
@simon huh, so not far fetched at all - neat.Of course, now chatGPT should be the validator for commercial ones - I don't think they should be able to run an AI without such a mechanism.
(DIR) Post #AU9JAZawQfGwAleaUy by faassen@fosstodon.org
2023-03-30T18:11:09Z
0 likes, 0 repeats
@simon @o Ah so my notion of someone posting openai output and then someone else using it as training data to get around the license already maybe happened!
(DIR) Post #AU9asddMTJ9Wf6Np8y by bplein@bvp.me
2023-03-30T21:29:34Z
0 likes, 0 repeats
@simon I am not an IP lawyer or student, but isn’t that saying that one can just change variable names, tabbing, and other inconsequential (to how the code runs) items, and get around IP laws? Seems pretty smart to me that software companies are working hard to be able to crush the little guy by simply remixing their code through a tumbler (my terminology, stolen from the crypto community, for effect). Truly interested in this, I’m not trolling. I am cynical at times, however. 😀
(DIR) Post #AUEA25dD9s4KoVWLSq by natedog@fosstodon.org
2023-04-02T02:21:38Z
0 likes, 0 repeats
@simon all the gpt neo and neox models from eleuther are apache 2.0 and should probably have better performance than these