Which huggingface model is best for creative writing and copywriting? Or is there a better one? reddit / selfhosted ---------------------------------------------------------------------- kaszebe · 2023-10-09 13:11 I was using Claude until they gimped the hell out of it. Claude USED to be amazing before they turned it into a medicated drooling mental patient in a straight jacket. So, I'm looking into self-hosting. I have a 4090 (24gb RAM) and i7-10700k and 32gb of RAM. Can you tell me if my above stats are good enough for self hosting—and if so, which huggingface (or other) model is best for creative writing and copywriting? This is for work and not play. Would it help if I double the RAM to 64GB? Also, I'm a bit new to AI...still learning. Is it possible to "train" my own self-hosted AI to remember to not use passive voice when writing and write in a certain way? (e.g. *"Write at a 9th-grade reading level. Avoid awkward phrasing and aim for for natural language and fluidity. etc*) I continually have to remind Claude and ChatGPT of the way I want them to write certain things. They will forget what I told them just 2 minutes and 1 message ago (e.g. "Don't use passive voice"). thanks ---------------------------------------------------------------------- pelic4n · 2023-10-09 14:44 While I get this is technically a Self Hosted Question, since it is so much more nuanced, I would suggest asking in the LocalLLaMa sub for a better answer overall: https://www.reddit.com/r/localLLaMA/ > So, I'm looking into self-hosting. I have a 4090 (24gb RAM) and i7-10700k and 32gb of RAM. Already I can tell you that a local LLM is going to run great on a system like this. > Would it help if I double the RAM to 64GB? It isn't "needed" but it would only benefit you to have more RAM. - If you're running it on a dedicated rig for just the LLM, then you will always have more memory to work with. - If you are running it a machine that is used as a daily driver for other tasks, the extra overhead in RAM allows you to still use the LLM to high capacity without impacting usage of the machine for general use. > Can you tell me if my above stats are good enough for self hosting—and if so, which huggingface (or other) model is best for creative writing and copywriting? This is for work and not play. >I continually have to remind Claude and ChatGPT of the way I want them to write certain things. They will forget what I told them just 2 minutes and 1 message ago (e.g. "Don't use passive voice"). Every model is going to limit you in some way to the data that was used in Training. Having said that, and having not delved deep into LORAs outside of for Stable Diffusion, it is my understanding that: - Finding a large model that fits your use case and adjusting each individual instance/interaction with a LORA to fit your specific needs is a must. - This is going to take time to find one that you like and works for you, potentially also meaning you are using multiple for different use cases. - Adjust a LORA to "Train" the model for your specific needs on a per task basis. The only suggestion I would have is a model like Falcon 180B, though I doubt most individuals can run a model locally like this: https://huggingface.co/tiiuae/falcon-180B And here are some useful links in case you need some more info: https://huggingface.co/ https://github.com/oobabooga/text-generation-webui https://www.reddit.com/r/PygmalionAI/ https://www.reddit.com/r/LocalLLaMA/comments/16y95hk/a_starter_guide_f or_playing_with_your_own_local_ai/ https://www.reddit.com/r/LocalLLaMA/comments/16zpkve/what_exactly_does _a_lora_do/ ---------------------------------------------------------------------- kaszebe · 2023-10-09 15:40 Wow, thanks for the detailed response! So...ok. More RAM (for the PC and not the graphics card) can be a benefit? And do you think that a 4090 will be able to power more powerful models over the next 12 months and as self-hosted LLMs get better? I'm wondering if the 5090 will come out in 2024. However, it will probably be a bitch to get and be upwards of $3k or more when you can find one for sale. ---------------------------------------------------------------------- SynySynson · 2023-10-09 18:43 It is pretty speculative but I think the last rumors I heard were that next-gen nvidia will be 2025. ---------------------------------------------------------------------- kaszebe · 2023-10-10 16:30 Cool so the 4090 I just bought should have some shelf life then.