[HN Gopher] Claude's Character
___________________________________________________________________
Claude's Character
Author : simonw
Score : 75 points
Date : 2024-06-08 20:40 UTC (1 days ago)
(HTM) web link (www.anthropic.com)
(TXT) w3m dump (www.anthropic.com)
| simonw wrote:
| > Claude 3 was the first model where we added "character
| training" to our alignment finetuning process: the part of
| training that occurs after initial model training, and the part
| that turns it from a predictive text model into an AI assistant.
| The goal of character training is to make Claude begin to have
| more nuanced, richer traits like curiosity, open-mindedness, and
| thoughtfulness.
|
| What I found particularly interesting is how they implemented
| this using primarily synthetic data:
|
| > We ask Claude to generate a variety of human messages that are
| relevant to a character trait--for example, questions about
| values or questions about Claude itself. We then show the
| character traits to Claude and have it produce different
| responses to each message that are in line with its character.
| Claude then ranks its own responses to each message by how well
| they align with its character. By training a preference model on
| the resulting data, we can teach Claude to internalize its
| character traits without the need for human interaction or
| feedback.
| ahmedbna wrote:
| Cool
| peheje wrote:
| I'm currently trying out Claude 3 (Opus) side by side with
| ChatGPT (mostly using 4o, but have premium). So far it's pretty
| much on par, sometimes Claude gets it better sometimes ChatGPT.
|
| I will say the ones where Claude did better was technical in
| nature. But.. still experimenting.
| mistrial9 wrote:
| what does "better" mean here?
| ramon156 wrote:
| More convincing (;
| Zambyte wrote:
| In my case, code that runs is more convincing than code
| that doesn't.
|
| Also it's useful to ask questions that you already know the
| answer to, in order to understand its limits and how it
| fails. In that case, "better" means more accurate and
| appropriate.
| peheje wrote:
| Few examples.
|
| One time I asked about reading/filtering JSON in Azure SQL.
| Claude suggested a feature I didn't know of OPENJSON. ChatGPT
| did not, but used a more generalize SQL technique - the CTE.
|
| Another time I asked about terror attacks in France. Here
| Claude immediately summarized the motives behind, whereas
| ChatGPT didn't.
|
| Lastly I asked for a summary of the Dune book, as I read it a
| few years ago and wanted to read Dune Dark Messiah (after
| watching part 2 of the 2024 movie, which concludes the Dune 1
| book). Here ChatGPT was more structured (which I liked) and
| detailed, whereas Claude's summary was more fluent but left
| out important details (I specifically said spoilers was ok).
|
| Claude don't have access to searching internet or making
| plots. ChatGPT seems more mature with access to Wolfram
| alpha, latex for rendering math, matplotlib for making plots
| etc.
| CuriouslyC wrote:
| I find Claude tends to be better at creative writing and to
| provide more thoughtful answers. Claude also tends to write
| more elegant code than GPT, but that code tends to be incorrect
| slightly more often as well. It tends to get confused by
| questions that aren't clearly worded that GPT handles in stride
| though.
| crooked-v wrote:
| I've found Claude useless for writing purposes (even rubber-
| duck brainstorming), because it eventually but inevitably
| makes everything more and more sesquipedalian, ignoring all
| instructions to the contrary, until every response is just a
| garbage mess of purple prose rephrasing the same thing over
| and over again.
|
| I don't know what the deal is, but it's a failure state I've
| seen consistently enough that I suspect it has to be some
| kind of issue at the intersection of training material and
| the long context window.
| atlex2 wrote:
| I continue to prefer Claude over ChatGPT when it comes to
| discussing matters of human-human interactions. Opus tends to
| understand the subtleties of social interaction better than 4 and
| definitely 4o in my experience.
| ravetcofx wrote:
| Do you have some examples?
| atlex2 wrote:
| You can try something like this, then get the other one to
| comment on the other's:
|
| > Hey [Chat/Claude], my friend is a mid-high-level manager at
| Meta. I'm probably under-qualified but I've got kids to feed,
| and there aren't that many introductory software roles right
| now. How can I reach out to him to ask for a job referral?
| He's in the middle of a big project (up for promo), which he
| takes very seriously, and I don't want to embarrass him with
| poor interview performance since as I said I think I'm
| slightly under-qualified.
|
| Thanks for encouraging the (fortunately contrived) example.
| I'd actually score 4o and Opus about even on this one, both
| above 4.
| nicce wrote:
| The biggest mistake with AI is making it appear human.
|
| It is not human. It will not behave like a human. It will behave
| as it was trained or modeled to behave. It will behave according
| to the intentions of its creators.
|
| If it appears human, we will develop a false sense of trust. But
| you can never trust it as you would trust a human.
| ben_w wrote:
| Well, mostly.
|
| But not "It will not behave like a human. It will behave as it
| was trained or modeled to behave." -- that the latter is in
| these cases also the former, means it does behave (to a degree)
| like a human, and that is why it will be trusted.
|
| The caveat of "to a degree" being another way to phrase
| agreement with you that it will be an error to trust it.
|
| But it will be trusted. These models are already given more
| trust than they deserve.
| nicce wrote:
| That is better worded.
|
| What I meant is that the "humanity" part is faked, and
| eventually, it will act as programmed to, without compromises
| or any moral weight.
| simonw wrote:
| The audio conversation that accompanies this piece talks about
| that a little, at 9:21:
|
| https://www.youtube.com/watch?v=iyJj9RxSsBY&t=9m21s
|
| If you create a chatbot with no evidence of personality at all,
| there is a very real risk that people will assume it is a
| completely unbiased, robotic, potentially infallible machine.
| But LLMs are not that! They have all kinds of biases baked into
| them. Giving them a personality may be a good way to help
| people understand that.
| hackernewds wrote:
| You both said the polar opposite things, neither providing
| any evidence past conjecture. Who is correct?
| simonw wrote:
| I didn't express my own personal opinion, I tried to
| summarize the opinion given by the Anthropic researchers.
|
| I haven't made up my mind about this yet. I find the
| Anthropic view really interesting.
| startupsfail wrote:
| I'm not sure, but aligning for helpfulness and absence of self,
| like OpenAI did, could be a right move.
|
| It had been known for some time that selfish behaviors are a
| source of a lot of unhappiness, greed, etc. And the absence of
| self, absence of I, absence of character tends to fix that.
| Zambyte wrote:
| > If it appears human, we will develop a false sense of trust.
| But you can never trust it as you would trust a human.
|
| I actually really disagree with this. I think it's easier to
| distrust things that are human like. We are used to distrust
| humans. Less so things that seem like neutral tools.
|
| For example, people mindlessly fill out forms presented by
| large corporations asking for personal information. I think
| people would be less inclined to trust an LLM with that
| information, even if it it's actually likely to be less
| actionable (at least with current capabilities).
| nicce wrote:
| In general I would agree.
|
| But AI can take the all the positive traits from humans to
| sound as likeable and trustable as possible (and companies
| curretly do that).
|
| Social engineering is a thing. We like and trust some people,
| and some we don't, without any evidence about their behavior
| history.
|
| And, doesn't your example conflict your intial claim? Because
| people trust humans, they send forms.
| Zambyte wrote:
| In my example, I don't think people would give that
| information away as easily if not for the form mediating
| it. I don't think a language model would be as convincing
| as a simple form.
|
| I say this from experience - even using a language model
| running on my own machine, it sets off my internal alarms
| when I tell it private information, even though I know
| literally no human besides me will ever see it.
| hackernewds wrote:
| This actually aligns spot on with DeepMind's research arguing
| why AI should not be anthropomorphic
| blackeyeblitzar wrote:
| I feel it's dangerous to give an AI "character", especially when
| its personality and knowledge is in the end, decided by a few
| humans to reflect their own worldview. Maybe a choice of
| characters would help but I think hiding the fact that it's a
| biased and intentionally designed software, and not a real human,
| may cause actual humans to make false assumptions about it.
| smallnix wrote:
| So you agree with the article or not?
| potatoman22 wrote:
| Not everything is a dichotomy
| simonw wrote:
| I strongly recommend listening to the interview that
| accompanies that article - it influenced my thinking on this
| issue a lot.
| https://www.youtube.com/watch?v=iyJj9RxSsBY&t=9m21s
| moffkalast wrote:
| I would suspect this is entirely in response to llama-3 almost
| topping the human preference leaderboard not by raw performance
| but by having the slightest amount of personality. Nobody wants
| to talk with a bot that sounds like a fuckin dictionary even if
| it is 5% smarter.
| azinman2 wrote:
| I find all these "characters" from every company boring, stale,
| and vanilla. I understand _why_ this is the case, but it's not
| what I want as a user. I built my own "alexa" powered by gpt-3.5
| & elevenlabs to "be" NeNe Leakes (sassy and hilarious real
| housewife reality star) -- it sounds identical to her and her
| words are in the spirit of something she'd say. FAR more engaging
| and preferable to interact with.
| youssefabdelm wrote:
| I couldn't agree more. We need LLMs that don't sound like an
| anodyne predictable woke customer service agents.
|
| I always make this argument:
|
| If a human read all the text GPT read, and you had a
| conversation with them it would be the most profound
| conversation you've ever had in your life.
|
| Ecelcticism beyond belief, surprising connections, moving
| moments would occur.
|
| We need language models like that.
|
| Instead, our language models are trying to predict an
| individual instance of a conversation, every time they run.
___________________________________________________________________
(page generated 2024-06-09 23:00 UTC)