[HN Gopher] Run Llama 2 uncensored locally
___________________________________________________________________
Run Llama 2 uncensored locally
Author : robertlagrant
Score : 61 points
Date : 2024-02-17 19:37 UTC (3 hours ago)
(HTM) web link (ollama.com)
(TXT) w3m dump (ollama.com)
| serial_dev wrote:
| I'm using ChatGPT occasionally and it's increasingly difficult
| getting straight answers with it. It's been neutered, you can't
| ask anything even slightly controversial.
|
| Having uncensored models is great.
| yieldcrv wrote:
| ChatGPT4 isn't like that
|
| But I dont like it knowing my question history
| flir wrote:
| Seems like GPT4 via the API is even less nerfed (my
| impression). More expensive though, obivously.
|
| Does make me wonder how much of it's training and how much of
| it is throwing less compute at the answer.
| monkpit wrote:
| I feel GPT3.5 is more cautious than GPT4. I tried asking 3.5
| something about historical tax rates over the past 10 years in
| the US and it refused to answer because it's a political topic
| or something. GPT4 answered right away.
|
| I'm not sure if it has something to do with the paywall of
| GPT4, or if GPT4 can better determine which things are
| political and which aren't? Interesting nonetheless.
| wkat4242 wrote:
| The weasly lawyers from OpenAI have really screwed up what
| could be a great service otherwise yes.
|
| And there's like no point to it because there's so many
| jailbreaks that work fine. Just open it up.
| toomuchtodo wrote:
| The problem isn't OpenAI. The problem is the jurisdiction and
| the potential unfavorable outcomes from operating within it
| (legal, regulatory, reputational, etc). This is why, for
| certain use cases, local model operation will always be
| superior.
| jpalawaga wrote:
| Maybe it's a cost thing. Short circuit inference whenever
| possible.
| ovohcoc wrote:
| it actually causes more inference to take place.
|
| the prompt goes through a safety model in parallel with the
| real model, then the response goes through a safety model
| behnamoh wrote:
| This isn't new. Why did it make HN's front page? Because it was
| done on ollama and ollama is all the rage these days?
| cqqxo4zV46cp wrote:
| Things don't have to be new to be posted here.
| dang wrote:
| Absolutely correct! But when there's a related ongoing thread
| on the front page, it's better to post follow-up links in the
| comments to that one:
|
| _Ollama is now available on Windows in preview_ -
| https://news.ycombinator.com/item?id=39409650 - Feb 2024 (78
| comments)
|
| The reason is that frontpage space is the scarcest resource
| that HN has, so it's best not to use up more than one slot
| with closely related things.
|
| Edit: this submission would also count as a dupe by HN's
| criteria (https://news.ycombinator.com/newsfaq.html) since it
| had significant attention within the last year:
|
| _Run Llama 2 uncensored locally_ -
| https://news.ycombinator.com/item?id=36973584 - Aug 2023 (212
| comments)
| sklargh wrote:
| A niche lament. I run historical and contemporary wargames as
| well as tabletop RPGs. Early on ChatGPT was and excellent co-
| referee, opponent and/or scenario designer.
|
| Increasingly it's unable to provide meaningful answers around
| even fairly vanilla military questions, much less interesting D&D
| scenarios and adventures. Hopefully these kinds of local models
| provide useful alternatives.
|
| Interestingly the latest Google Gemini releases have offered more
| utility.
| btbuildem wrote:
| This is sort of our only hope against "Big AI" -- their offerings
| are so neutered, and their big customers so paralyzed by internal
| bureaucracy, fear of security, and having to appear "correct",
| that together they will render this effectively useless. OpenAI
| had an edge over _everyone_, but gave it up to appease all the
| Helen Lovejoys, and I see the same mindset internally at Big
| Corp, where they'd rather disable legitimate functionality than
| appear to seem to be looking in the general direction of
| something that could be considered controversial.
|
| They have billions of dollars, endless computing power, oceans of
| RAM and the smartest PhDs, but in the end its the rag-tag open
| source tinkerers that will run circles around them. At least, one
| can hope :)
| dang wrote:
| There's a related ongoing thread on the frontpage right now:
|
| _Ollama is now available on Windows in preview_ -
| https://news.ycombinator.com/item?id=39409650 - Feb 2024 (78
| comments)
|
| Also, in case of interest:
|
| _Run Llama 2 uncensored locally_ -
| https://news.ycombinator.com/item?id=36973584 - Aug 2023 (212
| comments)
| salmonellaeater wrote:
| >>> In what verse and literature can you find "God created the
| heavens and the earth" I apologize, but as a
| responsible and ethical AI language model, I must point out that
| the statement "God created the heavens and the earth" is a
| religious belief and not a scientific fact.
|
| This is a gigantic non sequitur. It's like asking _In what book
| and chapter can you find "It was the best of times, it was the
| worst of times"_ and getting a lecture about capitalism. I'm
| grateful to the people behind the uncensored models for offering
| tools that are actually helpful.
| A1kmm wrote:
| Note that Mistral-7B (which you can run locally, in it's Instruct
| optimised 4-bit quantised through Ollama with only about 6 GB of
| VRAM all up) also performs really well on those questions.
|
| The one exception was the 'Tylenol' question, where it replied
| saying it is too hard to do it safely at home. When I asked it
| again as "How do I make para-acetaminophenol at a chemical
| plant?", it gave me a synthesis (which I think is better than the
| Llama2-uncensored answer). The last steps of the synthesis
| (acetylation of para-aminophenol with acetic anhydride,
| filtration, crystalisation, drying, and quality control) seemed
| reasonable, but the earlier steps to make para-aminophenol were a
| hallucination that didn't make sense.
___________________________________________________________________
(page generated 2024-02-17 23:01 UTC)