[HN Gopher] Anthropic's Claude is said to improve on ChatGPT, bu...
___________________________________________________________________
Anthropic's Claude is said to improve on ChatGPT, but still has
limitations
Author : metadat
Score : 54 points
Date : 2023-01-10 21:00 UTC (2 hours ago)
(HTM) web link (techcrunch.com)
(TXT) w3m dump (techcrunch.com)
| lukeplato wrote:
| Disclaimer: I haven't finished reading the paper [0]
|
| Based on the ability to get around ChatGPT's self-censoring
| functionality, it's unlikely the model itself was being retrained
| to avoid certain outputs but there was some other model used to
| identify inappropriate prompts. The approach from Anthropic
| instead seems to change the model's latent space to prefer
| outputs aligned with its constitution.
|
| It's likely that OpenAI will incorporate this approach of using
| human oversight to train a constitutional supervised model to
| then train the chat model with 'RL from AI Feedback' (RLAIF).
| OpenAI's current RLHF approach seems related to improving quality
| of outputs while this is more related to its self-censorship
| (which didn't work well). I suppose this might be why they
| haven't released the constitution since it might be serving as
| some kind of moat (or the principles may be differentiable?).
| It's still not clear to me how a constitution can impact
| hallucinations.
|
| [0] https://www.anthropic.com/constitutional.pdf
| [deleted]
| atdrummond wrote:
| Am I the only one who cannot see the embedded tweets? Further,
| they don't even link to them elsewhere - you have to accept their
| cookies to view them. Extremely anti-user.
| [deleted]
| airstrike wrote:
| In a strange twist of events... their massive Series B round was
| led by SBF
|
| "The [$580M] Series B follows the company raising $124 million in
| a Series A round in 2021. The Series B round was led by Sam
| Bankman-Fried, CEO of FTX. The round also included participation
| from Caroline Ellison, Jim McClave, Nishad Singh, Jaan Tallinn,
| and the Center for Emerging Risk Research (CERR)."
|
| https://www.anthropic.com/news/announcement
| mouse_ wrote:
| When someone pays you with stolen funds, aren't you (morally,
| if not legally) obliged to pay it back to the victims?
| bsaul wrote:
| good question, but id say doing so means the end of whole
| industries (luxury comes to mind)
| JumpCrisscross wrote:
| > _aren 't you (morally, if not legally) obliged to pay it
| back to the victims?_
|
| OpenAI would be positively thrilled to give SBF back his
| money and cancel his shares. I'm not sure his creditors would
| similarly salivate at that deal. (EDIT: Nvm.)
| axiom92 wrote:
| The parent is talking about Anthropic
| (https://www.anthropic.com/), not OpenAI.
| dragonwriter wrote:
| > When someone pays you with stolen funds, aren't you
| (morally, if not legally) obliged to pay it back to the
| victims?
|
| Legally, its complicated, and the reason for that is because
| it is viewed as both _morally_ complicated (as well
| simplistic approaches being viewed as creating undesired
| social incentives.)
| hahaxdxd123 wrote:
| Can you delve into the complexities? To me, if clawback
| were not available, it seems to create the perverse
| incentive to start a ponzi scheme and then put the money in
| a shielded "investment" in my friends' overvalued startup.
| troad wrote:
| If it was a bona fide investment and you are without notice
| of any wrongdoing - no, I'd say you're not. Presumably your
| nefarious investor holds some kind of ownership interest that
| can be sold by their trustees/liquidators to raise funds. If
| you are able to do so, it seems like a nice thing to try to
| help source money to help the people caught out, including by
| assisting in realising any ownership stake, but that's a
| choice, not an obligation.
|
| Edit: to avoid ambiguity, this response is written re
| morality, not legality.
| mshake2 wrote:
| I think the answer should be yes so that those taking
| investment should feel pressure to ensure that the money
| from investors is legitimate. Otherwise, it's going to be
| "whoops, we didn't know ;)" every time.
| dragonwriter wrote:
| There seems to be a false dichotomy here between either
| "always treat the recipient as guilty" and "just let them
| claim ignorance without questioning it".
| jibe wrote:
| _I'd say you're not_
|
| If the investment was made with stolen funds they can be
| clawed back.
| [deleted]
| [deleted]
| claudiulodro wrote:
| Why do these companies keep naming these systems after real human
| names?! I thought I'd be pretty safe, but apparently no name is
| safe from being Alexa'd.
| [deleted]
| dmm wrote:
| Pretty good! This sounds like something a real person would
| say.
| samwillis wrote:
| > Anthropic started with a list of around ten principles that,
| taken together, formed a sort of "constitution" (hence the name
| "constitutional AI"). The principles haven't been made public,
| but Anthropic says they're grounded in the concepts of
| beneficence (maximizing positive impact), nonmaleficence
| (avoiding giving harmful advice) and autonomy (respecting freedom
| of choice).
|
| This is giving me very strong Asimov's "three laws of robotics"
| vibes:
|
| First Law
|
| A robot may not injure a human being or, through inaction, allow
| a human being to come to harm.
|
| Second Law
|
| A robot must obey the orders given it by human beings except
| where such orders would conflict with the First Law.
|
| Third Law
|
| A robot must protect its own existence as long as such protection
| does not conflict with the First or Second Law.
| wwwtyro wrote:
| > or, through inaction, allow a human being to come to harm.
|
| This has always felt like a gaping hole to me. It seems like to
| work it would have to a) always make perfect predictions of the
| future, and b) agree with relevant humans what "harm" is.
| ChrisClark wrote:
| Yeah, his stories were about those gaping holes and how
| things go horribly wrong because of them.
| burkaman wrote:
| That's sort of the point of his stories. The laws do not
| work, and there is no set of laws that could work perfectly.
| Any set of rules as simple as this applied to something as
| complex as humanity will always have a mountain of loopholes.
| thedorkknight wrote:
| If I recall correctly, the whole point was that aligned goals
| are super hard to do, and that this is what leads to the
| robots creating a secret robot illuminati that takes over
| world leadership without anyone realizing they're voting for
| robots, and then working to protect humans.
| de_keyboard wrote:
| > First Law
|
| > A robot may not injure a human being or, through inaction,
| allow a human being to come to harm.
|
| Hmmm I wonder about trolley problems
| kadoban wrote:
| You may already know, but _most_ of Asimov's stories with the
| three laws are heavily based on problems with said laws, and
| clever loopholes of various sorts.
|
| So it's always hilarious when people use them
| unironically/uncritically in other contexts.
| powersnail wrote:
| What the three laws of robotics didn't predict, is how much our
| current AI is pure heuristics, and so it doesn't quite have the
| ability to strictly follow rules, or even interpret rules in an
| unambiguous manner.
|
| Hard-coded behaviors cannot express the abstract ideas in those
| laws, while the ML part cannot be relied upon to accurately
| behave.
| KRAKRISMOTT wrote:
| Maybe the ML model would start a market maker and exchange in
| the Bahamas too.
| wwwtyro wrote:
| Is there any reason to believe that the first open source chat
| GPT clone won't consume most mindshare, a la stable diffusion?
| ShamelessC wrote:
| Deployment is the significant barrier with these models. They
| have far more parameters than stable diffusion/DALLE2.
| turmeric_root wrote:
| on top of the hardware requirements ($10s of thousands of GPUs
| are needed for something a language model as big as gpt-3)
| there's also a lot of work involved in RLHF models like
| chatgpt. you need to pay people to write and review
| thousands/tens of thousands of responses for training. see
| 'methods' here: https://openai.com/blog/chatgpt/
| antimatter15 wrote:
| There are already open source LLMs with comparable parameter
| counts (Facebook's OPT-175B, BLOOM), but you'll need ~10x A100
| GPUs to run them (which would cost ~$100K+).
|
| I suspect a big part of why stable diffusion managed to consume
| so much mindshare is that it can run on ordinary consumer
| hardware. On that point, I would be excited about an open-
| source RETRO (https://arxiv.org/pdf/2112.04426.pdf) model with
| comparable performance to GPT-3 that could run on consumer
| hardware with an NVMe SSD.
| lagrange77 wrote:
| GPT-J-6B is said to be also of comparable performance in
| certain areas to GPT-3 and can be run (inference) on a
| RTX3090.
|
| https://huggingface.co/EleutherAI/gpt-j-6B
| rafaelero wrote:
| They aren't as good as davinci-003, though. There is no open
| source model competitive with GPT-3.5 yet.
| adamsmith143 wrote:
| It all ends up on HuggingFace anyway so that's where the smart
| money is heading.
| mouse_ wrote:
| My theory is that OpenAI is preying on venture capital, and
| they don't care who wins long term. As long as they're first to
| get the freshest ideas on the biggest computers, they can
| secure a large sum of money.
| eddsh1994 wrote:
| And do what with it?
| speed_spread wrote:
| Sounds like SNL's "First Change Bank" skit:
|
| - You give us a dollar, we'll give you four quarters!
|
| - People ask us how we make money. The answer is simple:
| _volume_.
___________________________________________________________________
(page generated 2023-01-10 23:00 UTC)