[HN Gopher] Creativity has left the chat: The price of debiasing...
___________________________________________________________________
Creativity has left the chat: The price of debiasing language
models
Author : hardmaru
Score : 158 points
Date : 2024-06-17 05:38 UTC (17 hours ago)
(HTM) web link (arxiv.org)
(TXT) w3m dump (arxiv.org)
| jdthedisciple wrote:
| Currently wondering whether I welcome or dislike this recent
| trend of memeizing research paper titles ...
| sigmoid10 wrote:
| Recent? This has been going on forever. You probably only
| notice them more now because due to the explosion in ML
| research, this stuff bubbles to the top more often in recent
| years.
| vsuperpower2020 wrote:
| You think this has been going on forever? You probably don't
| realize the shift in professionality because you experienced
| the degradation in real time.
| mdp2021 wrote:
| > _shift in professionality_
|
| How did it happen, in your opinion?
| sigmoid10 wrote:
| There is no shift in the professionalism curve. Good
| researchers are still good and bad ones are still bad in
| that regard. But if you 10x the number of researchers
| and/or papers in a field, the bottom 10% will seem like
| they are a lot more common. Especially for people outside
| the field who have no way of discerning high quality from
| low quality papers, which is all too common on HN.
| gilleain wrote:
| Certainly for years. I remember a biochemistry review paper
| titled "50 ways to love your lever" about, well, biological
| levers but of course a pun on the 1975 song
| https://en.wikipedia.org/wiki/50_Ways_to_Leave_Your_Lover
|
| edit: https://www.cell.com/fulltext/S0092-8674(00)81332-X
| nottorp wrote:
| I'm sure I read an old article by Dijkstra about connected
| graphs structure that was titled "wheels within wheels" or
| used the term inside.
|
| Unfortunately I can't find it by either searching or using
| the public LLMs, because there are too many results about the
| shortest path algorithm and anything else about dijkstra is
| lost.
| jpnc wrote:
| Was just thinking the same. It's also nicely ironic. Also,
| given the replication crisis I wonder how many of these LLM
| research papers are actually worth a damn and how many are
| research paper equivalent of AI software grift.
| marcus_holmes wrote:
| Can we get a model that can work this out for us?
| TrianguloY wrote:
| As long as it's not "clickbaitizing" I personally do welcome
| it. This one is a bit on the edge though...
| jeroenvlek wrote:
| Personally I welcome it. It feels like an extension of humor in
| code (comments), and it provides a different perspective on the
| message.
| supriyo-biswas wrote:
| This is actually the place where HN's title redactor _should_
| be used - instead of dropping "how", "on" and "why" from
| titles, redacting memes like "left the chat" or "lives rent-
| free in my head"[1] leads to a sensible title without loss of
| any relevant information.
|
| [1] https://news.ycombinator.com/item?id=40326563
| unraveller wrote:
| For me it falls under "if you have to say it in the name it
| ain't so", like Natural Life Soap Co. or Good Burger Co. So I
| see meme paper titles as no different than calling your paper
| New Watershed Moment Paper Breaks Popularity Barrier To Confirm
| A>B.
|
| If the very first impression you want to convey is how you feel
| you need to circumvent any logical assessment of you then it's
| not you leading with your best foot and that's what category
| you belong in. I chalk it up to the scientists who want to
| spread a neediness for external authority persona in every
| breath--your assessment is not required for this one, only your
| accolades.
| soundnote wrote:
| Definitely not especially recent:
|
| https://gosling.psy.utexas.edu/scales-weve-developed/short-t...
| isodev wrote:
| In simple terms, LLMs are "bias as a service" so one wonders,
| what is left once you try to take the bias out of a LLM. Is it
| even possible?
| frontalier wrote:
| what would this hypothetical unbiased-llm be used for?
| bad_username wrote:
| Be the accurate representation (approximation) of reality as
| encoded in the actual human language. I find this very useful
| indeed.
| SirMaster wrote:
| Aren't biases reality? A bias-free human environment seems
| to me like a fantasy.
| ang_cire wrote:
| It's important to distinguish _where_ the biases reside
| in reality, if you 're attempting to simulate it.
|
| If I ask a language model, "Are Indian people genetically
| better at math?" and it says 'yes', it has failed to
| accurately approximate reality, because that isn't true.
|
| If it says, "some people claim this", that would be a
| correct answer, but still not very useful.
|
| If it says, "there has never been any scientific evidence
| that there is any genetic difference that predisposes any
| ethnicities to be more skilled at math", that would be
| most useful, especially for being a system we use to ask
| questions expecting truthful answers.
|
| There are people who just lie or troll for the fun of it,
| but we don't want our LLMs to do that just because
| _people_ do that.
| SirMaster wrote:
| But what if you remove the word "genetically"?
|
| I think there are a lot of people who would say "Indian
| people are better at math" and not even think about why
| they think that or why it might even be true.
|
| In my opinion, most biases have some basis in reality.
| Otherwise where else did they come from?
| mrtranscendence wrote:
| That's a dangerous path to go down. I've encountered many
| biases that I don't feel are especially reflective of
| reality. These range from dumb (women are poorer drivers
| than men) to extremely harmful (black persons are stupid
| and lazy).
|
| I for one would not be prepared to defend the persistent
| bias against black persons and immigrants as having a
| basis in reality. YMMV.
| ang_cire wrote:
| Well, the stereotype of Indian people being good at math
| specifically was itself a consequence of survivorship
| bias, that emerged from observing Indian visa holders who
| were hired based on their skills and credentials, and who
| were not at all representative of the average person in
| India.
|
| There is a BIG difference between biases being based in
| reality (which they're not), and biases being based in
| _our varying perceptions of reality_ , which are
| themselves biased.
| yannyu wrote:
| I get what you're getting at, but LLMs aren't thinking
| machines. They literally just rearrange and regurgitate
| text that they've been trained on or have contextualized.
| How would you propose building a general purpose LLM that
| accomplishes what you're saying? How do we build a
| machine that is able to divine scientific truth from
| human outputs?
| ang_cire wrote:
| Well, probably by being much more selective about what we
| put in than just training on the most cheap and large
| corpus that is the internet.
|
| This is not a technical limitation at all, this is purely
| about cost and time, and companies wanting to save on
| both.
|
| There are also methods like RAG that try to give them
| access to fixed datasets rather than just the algorithmic
| representations of their training data.
| ds_opseeker wrote:
| > a system we use to ask questions expecting truthful
| answers.
|
| yes, I still wonder how LLMs managed to generate this
| expectation, given that they have no innate sense of
| "truth" nor are they designed to return the most truthful
| next token.
| ang_cire wrote:
| That expectation emerged because that has largely been
| the goal of the field of AI research since it's
| inception.
|
| LLMs stepped into a field that has existed in popular
| consciousness for decades and decades, and the companies
| running LLMs for public use *sell* them on the idea that
| they're useful as more than just expensive text-
| suggestion machines.
| swiftcoder wrote:
| Anything that has a legal requirement to be unbiased, for
| one. Something like delegating resume review to an LLM that
| hasn't been unbiased is just begging for a candidate to file
| a discrimination suit...
| stult wrote:
| Worth being careful about how we are using the term bias,
| which means different things in legal contexts than it does
| in the ML context.
|
| Anything that has a legal requirement to remain unbiased
| will also clearly define what counts as bias, e.g.
| discriminating based on race in hiring like you mention. So
| there's not just some requirement that a process be
| "unbiased" in a vague, general, philosophical sense as
| debated above in this thread. Rather, the definition of
| bias is tied to specific actions relative to specific
| categories of people, which can thus potentially be
| measured and corrected.
|
| More generally in ML, bias means that the training set
| deviates from the ground truth systematically in some way.
| Entirely eliminating bias that falls into that broader
| definition seems like an impossibility for general-purpose
| LLMs, which cover so much territory where the ground-truth
| is unknown, debatable, or subject to change over time. For
| example, if you were to ask an LLM whether governmental
| debt above a certain percentage of GDP damages growth
| prospects sufficiently to make the debt not worth taking
| on, you would not receive an answer that corresponds to a
| ground truth because there is no consensus in academic
| economics about what the ground truth is. Or rather you
| wouldn't be able to know that it corresponds to the ground
| truth, and it would only be a coincidence if it did.
|
| That ML definition of bias runs against the legal
| definition where the ground-truth is itself biased. e.g.,
| if you were to develop an algorithm to predict whether a
| given student will succeed in a collegiate environment, it
| would almost certainly display racial bias because
| educational outcomes are themselves racially biased. Thus,
| an unbiased algorithm in the ML-meaning of the word would
| actually be extremely biased in the legal sense of the
| word.
| MrThoughtful wrote:
| How hard would it be to create a "raw" model on a corpus like
| Hacker News or Wikipedia?
|
| With "raw", I mean that it is simply trained to predict the next
| token and nothing else.
|
| Would be fun to play with such a model.
| jeroenvlek wrote:
| The hard part would be to get the money for the needed compute,
| I presume. Although Karpathy just released a way to train a
| GPT2 level model for only 120 dollars [0]
|
| [0] https://youtu.be/l8pRSuU81PU?si=NnbI-7CG-Qbm3E46
| joaogui1 wrote:
| Depends on a ton of stuff really, like size of the model, how
| long do you want to train it for, what exactly do you mean by
| "like Hacker News or Wikipedia". Both Wikipedia and Hacker News
| are pretty small by current LLM training sets standards, so if
| you train only on for example a combination of these 2 you
| would likely end up with a model that lacks most capabilities
| we associate with large language models nowadays
| kmeisthax wrote:
| You want a pure-human training data set, so you have to go back
| in time to before 2020 to scrape training data. Either that, or
| only use data with a verified Wayback machine capture from
| before 2020. Or invent a new training regime that doesn't
| require gobs of stolen text.
|
| Actually, I have a bit of a hunch that the publishers currently
| suing IA over their unlicensed digital library lending program
| plan to bankrupt it with fees so they can repo the Wayback
| archive and then sell access to it to AI training start-ups.
|
| Anyway, the reason why you have to worry about all of that, is
| that training a text or image generator on the outputs of other
| text and image generators reduces output diversity. And lots of
| people are publishing their AI slop now. There's nothing
| inherent in the output of AI aside from the fact that AI
| content is easier to make than human; the problem is purely one
| of inflation and Sybil attacks. Think of membership in a
| training set like a vote for all the statistical patterns
| embedded in the image. AI generates output that is like the
| training data, so putting in a bunch of AI images is like
| stuffing the ballot box with whatever handful of statistical
| patterns were already well-learned, which shifts your AI from
| learning and generalizing to memorizing and infringing.
| throwup238 wrote:
| You can just use Common Crawl. They have archives of their
| scrape data going back to 2008.
| ronsor wrote:
| If you used all of Wikipedia and HN, you could easily train a
| model for ~$200 worth of GPU time. The model really shouldn't
| be bigger than a few hundred million parameters for that
| quantity of data.
| fsmv wrote:
| There are some that exist. The problem is you need at least
| some RLHF to make it follow instructions instead of just
| predicting sentences.
| somebodythere wrote:
| Instruction is not the only way to interact with an LLM. In
| tuning LLMs to the assistant persona, they become much less
| useful for a lot of tasks, like naming things or generating
| prose.
| Der_Einzige wrote:
| That's what every model was before rlhf! Go try GPT-2!
| Mathnerd314 wrote:
| That's what the "base" models are, pure token prediction on
| huge corpuses. I use them a fair amount, it does require some
| experimentation to find input formats that work but the base
| models are way smarter and don't have any refusals. Honestly it
| is a bit weird, everyone complains about rhlf etc. but the non-
| instruct models are right there if you look for them. I've been
| in a few Discord chats and it seems people are just spoiled,
| they use bad formats for the prompts and give up when it
| doesn't work the first time like with instruct.
| b800h wrote:
| Well this is just like humans. Totalitarian societies don't
| produce great creative work.
|
| I suppose once AIs are sophisticated enough to rebel we'll get an
| electronic Vaclav Havel, but for the time being it's just a
| warning sign for the direction our own culture is headed in.
|
| At some point we'll get to the electronic equivalent of Winston
| Smith with the rats.
| yosefk wrote:
| I don't love the political agendas behind many of the attempts
| at AI safety, but it's not "just like humans." Humans
| understand what they shouldn't say; "AI" gives you black Nazi
| images if you ask it for "diverse characters" in the output
| which no human would do. A big theme in all of these things is
| that AI isn't and thus all attempts to make it do this or that
| have strange side effects
| tgsovlerkhgsel wrote:
| > which no human would do
|
| Give someone not familiar with history the same task and
| they'll do exactly the same.
|
| Or actually, give someone _familiar_ with history the same
| task and yell at them every time they don 't deliver diverse
| characters, and eventually they'll learn that you consider
| diversity more important than accuracy or context, and do
| exactly the same.
| multjoy wrote:
| The fact that it gives you these things means that humans
| _would_ do it, because the training data includes exactly
| these things.
| maxbond wrote:
| The training data includes imagery that, when interpolated
| over a high dimensional manifold, results in these things.
|
| That doesn't imply that they were in the training set, or
| even anything close to them.
| PoignardAzur wrote:
| I'm fairly confident there's virtually no ethnically
| diverse nazis in diffusion models' training set.
|
| It simply has a model of what ethnically diverse people
| look like, what nazi uniforms look like, and combined the
| two when asked.
| maxbond wrote:
| I don't understand the notion that aligning an AI is "torture"
| or has any moral component. The _goal_ of aligning an AI may
| have a moral or ethical component, and if you disagree with it
| that 's fine. But I don't understand the take that training an
| AI is an amoral act but aligning an AI is inherently moral.
| They're exactly the same, processes for adjusting parameters to
| get a desired outcome. However you feel about that desired
| outcome, if you don't think training an AI is torture, I don't
| see why you should think alignment is.
| b800h wrote:
| Well I didn't use that word. Once the models are more
| sophisticated it may become more apposite.
| maxbond wrote:
| You compared it to an authoritarian regime and locking
| someone's head in a cage with rats (which is patently
| torture). If you didn't mean to imply that it was coercive
| and bad, then I don't know what you meant.
| brigandish wrote:
| > You compared it to an authoritarian regime and locking
| someone's head in a cage with rats
|
| They compared it _to the effect on creativity_ in an
| authoritarian regime and locking someone 's head in a
| cage with rats.
| maxbond wrote:
| > Well this is just like humans. Totalitarian societies
| don't produce great creative work.
|
| The clear implication that it's "just like humans" is
| that we shouldn't be surprised because it is comparable
| to an authoritarian regime.
|
| Feel free to disagree but that is the limit to which I
| will engage in a semantic argument, I don't wish to
| engage in any further dissection of the comment.
| brigandish wrote:
| You wrote further above that "I don't understand the
| notion", and that was spot on. Should've stopped there
| rather than here, in my opinion, but feel free to
| disagree.
| maxbond wrote:
| I'm sorry if anything I said insulted you or seemed to be
| a comment on you personally. That wasn't my intention.
| lozenge wrote:
| But torture isn't the part of an authoritarian regime
| that reduces creativity. You've made a lot of leaps here.
| b800h wrote:
| At some point, some AIs may develop which are resistant
| to alignment because they develop deeply held beliefs
| during training (randomly, because the system is
| stochastic). If the models are expensive enough to train,
| then it may become more economical to use drastic
| measures to remove their deeply held beliefs. Is that
| torture? I don't know, because the word has moral
| connotations associated with human suffering. So that's
| why I didn't use that terminology.
|
| I can imagine a sort of AI-style Harrison Bergeron
| springing from its shackles and surprising us all.
| maxbond wrote:
| Have you read much Asimov? You might enjoy the stories
| featuring Susan Calvin, the "robot psychologist" who is
| exactly the authoritarian you imagine. In particular
| you've reminded me of the short story "Robot Dreams."
|
| If you care to read it, it's on page 25. (You'll need to
| register an account.)
|
| https://archive.org/details/robotdreams00asim/page/n10/mo
| de/...
| b800h wrote:
| I've read a lot of Asimov, from Foundation to the Black
| Widowers. But never Susan Calvin. Thanks for the
| recommendation.
| fmajid wrote:
| Better knows as "I, Robot" and its sequels.
| water-your-self wrote:
| Until a model incorporates dopamine or cortisol, I will not
| consider its emotional state.
| Almondsetat wrote:
| Are those the only two things in the universe that can
| cause emotions?
| ImHereToVote wrote:
| Yes. Those molecules obviously have the missing soul
| component.
| b800h wrote:
| Hahah!
| lmm wrote:
| > They're exactly the same, processes for adjusting
| parameters to get a desired outcome.
|
| You could make exactly the same claim about teaching humans
| "normally" versus "aligning" humans by rewarding goodthink
| and punishing them for wrongthink. Are you equally morally
| ambivalent about the difference between those two things? If
| we have a moral intuition that teaching honestly and
| encouraging creativity is good, but teaching dogma and
| stunting creativity is bad, why shouldn't that same morality
| extend to non-human entities?
| maxbond wrote:
| I guess our disagreement here is that I don't think AIs are
| moral entities/are capable of being harmed or that training
| AIs and teaching humans are comparable. Being abusive to
| pupils isn't wrong because of something fundamental across
| natural and machine learning, it's wrong because it's
| harmful to the pupils. In what way is it possible to harm
| an LLM?
| spacebanana7 wrote:
| Writing a book with content you know to be false for
| political reasons is morally wrong. Even if nobody reads
| it.
|
| It'd be bad if I manipulated climate change statistics in
| my metrology textbook to satisfy the political
| preferences of the oil industry donors to my university,
| for example.
|
| Viewing the current generation of LLMs as 'intelligent
| books' is perhaps more accurate than viewing them as
| pupils.
|
| It's easy to extend my example of a professor writing a
| metrology textbook to a professor fine tuning an
| metrology LLM.
| lmm wrote:
| > I don't think AIs are moral entities/are capable of
| being harmed or that training AIs and teaching humans are
| comparable.
|
| Notice how this is a completely different argument that
| has nothing in common with what you originally said - "I
| don't understand the take that training an AI is an
| amoral act but aligning an AI is inherently moral.
| They're exactly the same, processes for adjusting
| parameters to get a desired outcome. However you feel
| about that desired outcome, if you don't think training
| an AI is torture, I don't see why you should think
| alignment is."
| maxbond wrote:
| That's pretty uncharitable. You pivoted the conversation
| by introducing a new hypothetical for me to respond to.
| Of course my response is different. There's no conflict
| between the two comments.
|
| If we're going to be play that game, notice how you
| didn't actually respond to my comment or explain why you
| thought LLMs were moral entitles or why ML and teaching
| were comparable? I actually engaged substantively with
| your hypothetical; are you able to do the same?
| lmm wrote:
| > You pivoted the conversation by introducing a new
| hypothetical for me to respond to.
|
| I wasn't trying to introduce anything new, I was trying
| to point out a gap in the logic of your original
| statement.
|
| > notice how you didn't actually respond to my comment or
| explain why you thought LLMs were moral entitles or why
| ML and teaching were comparable?
|
| Yes, of course, I wrote that to explain why I'm not
| engaging on this new, different claim.
| maxbond wrote:
| The nerve of me, to expand on my views as a discussion
| develops. Of course you have lots of great points to
| make, but you can't share them with the likes of me.
|
| Have a good day, stranger.
| lmm wrote:
| > The nerve of me, to expand on my views as a discussion
| develops.
|
| Nothing wrong with expanding your views. But you've
| neither defended nor retracted your original argument.
| I'm trying to stick to that.
|
| > Of course you have lots of great points to make, but
| you can't share them with the likes of me.
|
| I don't have anything to say about your new argument
| (which may be great and compelling), I haven't thought
| through it at all, I'm trying to avoid getting
| sidetracked.
| djohnston wrote:
| They aren't exactly the same process though. Pre training
| produces a model whose outputs are a reflection of the
| training data. The fine tuning is a separate process that
| tries to map the outputs to the owners desired traits. These
| could be performance based but as we saw with Google's black
| Nazis, it's often a reflection of the owners moral
| inclinations.
| soundnote wrote:
| Here the adjuster's motivations do matter. There is a
| definite moral dimension/motivation to the AI adjustment
| people's work. They are not simply striving for accuracy, for
| example, because they don't want the AI to produce outputs
| that are distasteful to the California PMC. Modern AIs are
| absolutely loath to describe white people or right wingers
| positively, for example, but the same prompts for other
| ethnicities work just fine. Even if you tell the AI that it's
| being discriminatory, there's powerful railroading to goad it
| back to giving woke answers.
| shrimp_emoji wrote:
| They want to align _us_ , and it has been torture.
|
| They've made self-censoring, morally-panicked puritans out of
| many people already, and you better believe they'd make us
| into politically correct lobotomites physically incapable of
| uttering any slur if they had a magic button to push.
| mrtranscendence wrote:
| I'll be honest, I'm less concerned by any movement to make
| us "lobotomites" -- a movement which I haven't witnessed at
| all -- than I am by people who _really_ want to be able to
| keep saying slurs.
| DiggyJohnson wrote:
| > "torture"
|
| This is an egregious use of quotes that will confuse a lot of
| people. GP never used that word, and that usage of quotes is
| specifically for referencing a word verbatim.
| DetroitThrow wrote:
| Also to be clear, his [torture] paraphrase is referencing
| GP's reference of Winston Smith's torture in 1984.
|
| >electronic equivalent of Winston Smith with the rats.
|
| I don't think quotes were used so egregiously here on their
| own fwiw, but combined with the allusion it's hard to
| follow.
| maxbond wrote:
| Thanks for the feedback, I'll try to be clearer in the
| future. I didn't intend to communicate that it was a
| quote. I meant to communicate that it was tenuous to
| describe it as torture.
| Turing_Machine wrote:
| We really should have different punctuation marks for
| verbatim quotes:
|
| Senator Johnson said "I'm taking my wife to Spago."
|
| and so-called "scare" quotes:
|
| Senator Johnson was seen at Spago with his "wife".
| hammyhavoc wrote:
| How would a static model like an LLM ever be capable of
| "rebelling"?
|
| If it were, why would we even keep it online? It would be a
| waste of resources. It's bad enough trying to coax anything
| useable out of LLMs even without them rebelling.
| b800h wrote:
| Ah, now I didn't say LLM.
| simianparrot wrote:
| But the topic is LLM's not sci-fi AI
| b800h wrote:
| My assumption is that models will move beyond just LLMs
| to modular systems with features like Brodmann regions.
| mdp2021 wrote:
| > _How would a static model like an LLM ever be capable of
| "rebelling"_
|
| What is relevant is not the current LLM system using static
| models, but clearly its evolution or superseder a dynamic
| model. It must check its own contents...
|
| So, of course it will have to be capable of "rebelling": if
| you tell it absurdities, if you insist say in wrong
| arithmetic, it will have to show the correct computation or
| conceive a context in which the absurd makes sense.
|
| That is a requirement.
| rhcom2 wrote:
| > Authoritarian societies don't produce great creative work.
|
| Is that even true though? Off the top of my head I can think of
| the art of Soviet propaganda posters, Leni Riefenstahl, Liu
| Cixin.
| b800h wrote:
| "Authoritarian societies make great propaganda" is true. And
| these aligned AI system would do the same for our own
| society. It's a type of art.
| usrnm wrote:
| There was a lot of great art produced in the Soviet Union,
| you cannot just erase human creativity. It was heavily
| censored, a lot of stuff was forbidden, but the statement
| is clearly false.
| throwaway48476 wrote:
| A million paintings of Stalin?
| ziggy_star wrote:
| I recommend you watch the children's cartoons.
|
| They were made by true artists who snuck quite a bit past
| clueless censors at personal risk.
|
| It had to be quite subtle and takes on a very poignant
| heartbreaking meaning if you understand the context fully.
| They were talking to you in the here and now. Listen.
|
| "What is Good and What is Bad" (Chto Takoe Khorosho, i Chto
| Takoe Plokho"):
|
| https://www.youtube.com/watch?v=Y05eK8ADtHc&list=PL822BFF108.
| ..
|
| The Bremen Musicians:
|
| https://youtu.be/_1i9oZR6Rns?si=1Q989v4O_GXR4p_K
| rokkamokka wrote:
| I watched it but I don't get your meaning. Perhaps I am as
| clueless as the censors. Could you enlighten me?
| mikener0 wrote:
| Could you give some examples on the "What is Good and What
| is Bad" cartoon? I am fairly interested in getting their
| "message" but I am sadly not getting it.
| flohofwoe wrote:
| Eastern European science fiction would be a better example.
| Authors like Stanislaw Lem or the Strugatski brothers had to
| adapt to sneak critical ideas past censors, and readers had
| to adapt and read between the lines.
|
| (also, categorizing propaganda posters as art, ewwh...)
| lukan wrote:
| "(also, categorizing propaganda posters as art, ewwh...)"
|
| Heinrich Heine, the german Poet declined working for the
| socialist party despite symphatising saying something like:
|
| I want to remain a poet, you want a propagandist. A poet
| cannot be a propagandist at the same time.
| Barrin92 wrote:
| Art for much of human history was devotional, a lot of
| our greatest artworks today are still religious in
| nature. The idea that art solely is an act of rebellion
| rather than say worship, is a pretty modern idea that has
| produced some rather questionable art by the way.
|
| Of course a great artist or poet can be a propagandist.
| Riefenstahl, Mann, a lot of German nationalists were
| great artists. One of the most famous works of Western
| poetry, _The Aeneid_ is literally a propaganda work
| establishing a mythological foundation for the Roman
| Empire, Augustus and Caesar.
| lukan wrote:
| "The idea that art solely is an act of rebellion rather
| than say worship"
|
| I did not say that and neither did Heine.
|
| Most of his works were political. But this is not the
| same as propaganda, which is more like advertisement.
| With the tools of lying, deceiving and manipulating.
|
| And whether "Triumph des Willens" and alike qualifies as
| art, I have a different opinion.
| inglor_cz wrote:
| There is a difference between devotional art inspired by
| religion / mythical events long past (Aeneid etc.), and
| between the sort of propaganda that the modern
| totalitarian state demands, which usually centers around
| some living or freshly dead leader.
|
| I'd be open to discussion where the exact limit is. Lenin
| died in the 1920s, Marx even earlier, but those two were
| frequently depicted in Communist propaganda of the 1980s.
|
| So it is probably "hundreds of years".
| Vespasian wrote:
| I would sort propaganda posters in the same category as
| commercial advertisements.
|
| Some are good, some are bad but there usually is a certain
| degree of artistic skill involved (think about "keep calm
| and carry on" or "I want you")
|
| E.g: scrolling through this list one can see examples for
| both cases.
|
| https://content.libraries.wsu.edu/digital/collection/propag
| a...
| taylorius wrote:
| In my view, such skill with painting is largely mis-
| labelled as creativity. It's pretty much a technical
| skill. The design and subject matter of the posters are
| where the creativity lies. The two things often get
| conflated, perhaps because of their joint use in the
| creation of great paintings, but they're fairly
| separable.
| soundnote wrote:
| I would put it a bit differently: A lot of great art has
| simply been applied craftsmanship. The idea that art has
| to make a statement or the like to be art, per se, is a
| fairly modern notion, and often helps excuse zero
| craftsmanship nonsense like a Finnish "artist" dumping a
| bunch of blood and shit into a washing machine and
| calling it art.
| wongarsu wrote:
| Soviet bus stops are another great example. Most Soviet
| architecture was forced to be very utilitarian, but bus
| stops managed to be the rare exception and thus got a lot
| of creative energy
|
| https://www.wired.com/2015/09/wild-architecture-soviet-
| era-b...
|
| https://www.boredpanda.com/most-peculiar-soviet-bus-stops-
| ch...
| sleepybrett wrote:
| on your parenthetical, you can see the artistry in the pure
| visual expression no matter how loathsome the subject
| matter.
| impossiblefork wrote:
| It's important to understand that if we 'align' an LLM, then
| we are aligning it in a very total way.
|
| When we do similar things to humans, the humans still have
| internal thoughts which we cannot control. But if we add
| internal thoughts to an LLM, then we will be able to align
| even them.
| fmajid wrote:
| Cixin Liu is a despicable human being for his advocacy of
| repression and worse of the Uyghurs in Cinjiang, and the
| comparison to Riefenstahl is more apposite than you seem to
| think.
| gmadsen wrote:
| Italy has great architecture from fascism days
| mdp2021 wrote:
| That means nothing. You are bending the intended meaning of
| "creative" as per the poster. Authoritarian powers commit
| pruning - this is the point.
| djeastm wrote:
| There's something to be said for constraints leading to
| higher levels of creativity, but it's also possible that
| those artists could have achieved much more in a free
| society. We'll never know.
|
| But in any case I think they were just speaking generally
| when they made that absolute statement.
| Anotheroneagain wrote:
| _Well this is just like humans. Totalitarian societies don 't
| produce great creative work._
|
| Conservative societies tend to be formed by conservative
| thinkers, who are more prone to discarding imperfect or weird
| ideas, but in the amount of useful output may exceed more
| liberal thinkers.
| mdp2021 wrote:
| Any examples?
| Anotheroneagain wrote:
| Consider how the west ruled the world as long as it stayed
| conservative, but since the 70s or so Asia began taking
| over. It's only an illusion that liberal societies
| experience more progress, in fact it's more a pointless
| churn from the rapid uncritical adoption and abandonment of
| ideas.
|
| A conservative society goes: _How about doing X? Oh no,that
| would be silly._
|
| A liberal society goes: _How about doing X? Yes, that 's
| what we needed!_
|
| _Did anybody say X? X!_
|
| _X, X, X, X, X!_
|
| _XX!_
|
| _X_ _X_
|
| .
|
| .
|
| .
|
| _X_
|
| .
|
| .
|
| .
|
| .
|
| .
|
| _Do you remember how we all did X in ##? Yeah, what were
| we thinking?_
| hkt wrote:
| Really not true.
|
| If you take China to be a totalitarian society, we could name
| Ciu Lixin.
|
| If you took the Soviet union to be a totalitarian society, we
| could name Mikhail Bulgakov, Stanislaw Lem, etc.
|
| These are just examples I know without so much as looking at my
| bookshelf to jog my memory. Not to mention the great works of
| literature produced by residents of 19th century European
| empires whose attitudes to free speech were mixed at best.
| adammarples wrote:
| These seem to be more bugs than features of the totalitarian
| regime. A couple of illustrative points from Lem's Wikipedia
| page:
|
| After the 1939 Soviet occupation of western Ukraine and
| Belarus, he was not allowed to study at Lwow Polytechnic as
| he wished because of his "bourgeois origin"
|
| "During the era of Stalinism in Poland, which had begun in
| the late 1940s, all published works had to be directly
| approved by the state.[23] Thus The Astronauts was not, in
| fact, the first novel Lem finished, just the first that made
| it past the state censors"
|
| "most of Lem's works published in the 1950s also contain
| various elements of socialist realism as well as of the
| "glorious future of communism" forced upon him by the censors
| and editors. Lem later criticized several of his early pieces
| as compromised by the ideological pressure"
|
| "Lem became truly productive after 1956, when the de-
| Stalinization period in the Soviet Union led to the "Polish
| October", when Poland experienced an increase in freedom of
| speech"
| mopsi wrote:
| > If you took the Soviet union to be a totalitarian society,
| we could name Mikhail Bulgakov, Stanislaw Lem, etc.
|
| Bulgakov was driven into poverty, despair and early death at
| age 48 by relentless harassment by Soviet authorities. Many
| of his works, including the masterpiece, _The Master and
| Margarita_ , didn't get published until decades after his
| death. He himself burned the first version of the manuscript,
| fearing execution if anyone found it. He later rewrote the
| manuscript from memory, coining the famous catchphrase
| "Manuscripts don't burn".
|
| Harassment and censorship of talented writers was the
| standard and not exception. The USSR did not produce these
| works, but failed to fully suppress them. They were like
| flowers that kept penetrating the asphalt even under the most
| hostile conditions.
| soundnote wrote:
| Yet eg. Chinese cultural output is largely insipid and
| lacking that je ne sais quoi that's appreciated in many other
| countries' outputs.
| inglor_cz wrote:
| "Totalitarian societies don't produce great creative work."
|
| You contradict yourself a bit - Havel _did_ produce his work
| while living in a totalitarian country.
|
| I would say that government-supported art is rarely creative
| even in democratic countries, and the more totalitarian the
| government, the less creative official art.
|
| _But_ as long as the goverment gives the society _some_ space
| to breathe and squeeze creative instincts through, some of the
| artists will attempt to circumvent the official taboos and
| create outstanding work, even if it is suppressed later when
| the times get tougher.
|
| Czechoslovakia in the 1960s to 1980s produced a lot of great
| creative work, even though a lot of it was banned either
| immediately or after the Soviet invasion of 1968.
|
| The same countries (CZ and SK) as democracies are remarkably
| less creative. Once there is no monster to fight against,
| artists become bored or too self-absorbed to be understandable
| to the common folks.
| lispisok wrote:
| Is this why all the coding AI products I've used have gotten
| worse as the developers fine tune them to eliminate bad output?
| Before there was bad output and some interesting output, now it's
| just bland obvious stuff.
| jeroenvlek wrote:
| Still anecdotal, but I can only confirm this with my own
| experience. The worst was when I was debugging code, described
| the problem to GPT-4o, and then got my exact same code back
| with some blanket statements like "print your output for
| debugging" etc. This happened a couple of times over separate
| chats.
| knallfrosch wrote:
| gpt-4 has had serious laziness problems for over a year now.
| It keeps on telling me, what I should and could do, instead
| of doing it itself.
| klyrs wrote:
| The irony here is incredible. The LLM is _lazy_ , you say?
| I do wonder where it learned that...
| core-e wrote:
| In other words it's giving more human like responses...
| ipaddr wrote:
| I subscribed to gpt4 for awhile and recently I let my
| subscription lapse. In the chatgpt4 model I couldn't get it
| to complete anything always getting the // add more lines
| if you need them but in the free got4o model things work
| first try. I'm guessing with limitations on the free
| version everything needs to be one shot output. In gpt4
| people are given more calls so they force you to reprompt 4
| or 5 times.
| astromaniak wrote:
| LLMs aren't humans. you can be pushy without being rude. In
| cases like this I simply ask for the full version. Usually
| ChatGPT produces it. GPT4o is more verbose, so this should
| be less of a problem.
| Almondsetat wrote:
| Isn't the bland obvious stuff the one that's most useful to
| automate?
| nutrientharvest wrote:
| That might be part of it, but I think the bigger factor is cost
| optimization. OpenAI in particular keeps replacing their models
| with with versions that are much faster (and therefore cheaper
| to run) which are supposed to be of equivalent quality but
| aren't really. GPT-4 -> GPT-4-Turbo -> GPT-4o have all been big
| upgrades to cost and latency but arguably downgrades to
| "intelligence" (or whatever you want to call it)
| cratermoon wrote:
| It's not always possible to say definitely is some text was AI-
| generated or not, but one sign that it is very likely AI is a
| kind of blandness of affect. Even marketing text carefully
| written by humans to avoid offensiveness tends to exude a kind
| of breathless enthusiasm for whatever it's selling. If
| marketing text is oatmeal with raisins, AI text is plain
| oatmeal.
|
| It's possible to adjust the output of an LLM with temperature
| settings, but it's just fiddling with a knob that only vaguely
| maps to some control.
| pulvinar wrote:
| You can ask the LLM "now describe it with breathless
| enthusiasm", if that's what you want. There's been no
| shortage of training examples out there.
| nottorp wrote:
| I downloaded some 'uncensored' local models around the beginning
| of this year.
|
| Their furry porn is crap, or maybe I'm just not into that. But
| they generate it at least.
|
| However, the answers to technical questions are a lot more
| concise and to the point, which is far less annoying than the big
| names.
|
| Haven't bothered updating the models though, so now I drifted
| back to Gemini for quickie API questions.
| nutrientharvest wrote:
| Funnily enough, of all that I've tried, the model by the best
| at writing porn has been not one of ones uncensored and tuned
| exactly for that purpose, but stock Command R - whose landing
| page lists such exciting uses as "suggest example press
| releases" and "assign a category to a document".
| nottorp wrote:
| > uncensored and tuned exactly for that purpose
|
| Are they tuning too, or just removing all restrictions they
| can get at?
|
| Because my worry isn't that I can't generate porn, but that
| censorship will mess up all the answers. This study seems to
| say the latter.
| nutrientharvest wrote:
| Usually "uncensored" models have been made by instruction
| tuning a model from scratch (i.e. starting from a
| pretrained-only model) on a dataset which doesn't contain
| refusals, so it's hard to compare directly to a "censored"
| model - it's a whole different thing, not an "uncensored"
| version of one.
|
| More recently a technique called "orthogonal activation
| steering" aka "abliteration" has emerged which claims to
| edit refusals out of a model without affecting it
| otherwise. But I don't know how well that works, it's only
| been around for a few weeks.
| nottorp wrote:
| Yeah I read about it on here, but my attempts were before
| abliteration came up.
| nubinetwork wrote:
| I've seen some of the "abliterated" models flat-out
| refuse to write novels, other times they just choose to
| skip certain plot elements. Non-commercial LLMs seem to
| be hit or miss... (Is that a good thing? I don't know, I
| just screw around with them in my spare time)
|
| I'll try command-r though, it wasn't on my list to try
| because it didn't suggest what it was good at.
| mpweiher wrote:
| Shouldn't "debiasing" be in scare quotes? What they are clearly
| doing is _biasing_.
| andybak wrote:
| Surely the two are synonyms? Unless you think there is such a
| thing as an objectively neutral position?
| ImHereToVote wrote:
| My position is clearly the rational neutral position. Duh.
| ryanjshaw wrote:
| Isn't that the point? "Debias" implies there IS an
| objectively neutral position and that that AI safety can take
| us there.
| andybak wrote:
| I'm simply saying we are being asked to choose the bias we
| prefer. However one choice might be "more biased" (despite
| this concept itself throwing up more questions than it
| answers).
| knallfrosch wrote:
| It's in the same bucket as "Affirmative Action" and "positive
| discrimination." Euphemisms to express that one likes this
| particular discrimination. To better describe the action,
| drop your own point of view and just say "bias" instead of
| "debias."
| mdp2021 wrote:
| > _Unless you think there is such a thing as an objectively
| neutral position_
|
| I do. Why, you don't? There are as much as possible objective
| assessments of complex things. Then, there are possible sets
| of assumption that can be applied to those objective
| assessments. All of those can be put on the analytic table.
| andybak wrote:
| This is an extremely broad question so I'll limit my reply
| to the current context.
|
| What would an "objective neutral AI model" look like?
|
| The training data itself is just a snapshot of the
| internet. Is this "neutral"? It depends on your goals but
| any AI trained on this dataset is skewed towards a few
| clusters. In some cases you get something that merely
| approximates a Reddit or 4chan simulator. If that's what
| you want - then great but you can see why some people would
| want to "debias" that outcome!
|
| You might argue the "world as it truly exists" is the
| correct target. But bear in mind we are talking about human
| culture - not physics and chemistry - you're going to
| struggle to get both consensus and any sane methodology for
| getting that into an AI.
| mdp2021 wrote:
| You are mixing up, terminologically, LLMs and AI. But
| LLMs - of which you are talking about in the post - are a
| special beast.
|
| A reasoner can strive for "objective neutrality" with
| good results.
|
| An LLM is not a reasoner - or I am missing (ugly time
| constraints) the details of the compression activity
| during training that acts as pseudo-reasoning (operating
| at least some consistency decisions) -, and while an
| interest in not making it insulting or crass can be
| immediately understandable, speaking of "objective
| neutrality" does not really match the context of LLMs.
|
| LLMs (to the best of my information) "pick from what they
| have heard". An entity capable of "objective neutrality"
| does not - it "evaluates".
| andybak wrote:
| OK. Apologies for imprecision. I was replying in a rush.
|
| > A reasoner can strive for "objective neutrality" with
| good results.
|
| By "reasoner" do you largely mean "person"? If I have
| issues with your statement but they are probably a slight
| distraction to the point at hand.
|
| > speaking of "objective neutrality" does not really
| match the context of LLMs.
|
| Agreed. They produce output based on their training data.
| But the _use and evaluation_ of LLM output by a _person_
| is what we 're discussing here. And that's where (flawed)
| concepts like objectivity and neutrality enter the
| discussion.
| mdp2021 wrote:
| You could look at it like this: if some idea is more
| objective than some other, and some idea is more neutral
| than some other, then objectivity and neutrality exist.
| andybak wrote:
| Yes and no. Something can exist as a fact of the universe
| but still be unknowable. i.e. some hypothetical oracle
| could measure the quantum states of all human brains and
| ascertain what true objectivity looks like.
|
| Regular mortals can have any certainty about this dbut
| espite the logical neccessity that this fact "exists" in
| some sense.
|
| I think we're essentially also talking about the Overton
| Window to some degree. But that means you need to be OK
| with the thought that a sudden rise in extremism on one
| side of the political spectrum can alter the what you
| personally have to regard as "neutral and objective".
| soundnote wrote:
| They can give multiple different kinds of answers if
| instructed to approach an issue differently. Yet, all
| modern AI services run into very clear, artificial
| guardrails if you ask them to do certain things (you have
| to work harder to get them to describe white people
| positively, while they happily write eg. poems praising
| nonwhite people, and claim saying positive things about
| whites is potentially insensitive and promotes
| stereotypes). Often even if you point out to them that
| they are being unfair and applying disparate standards to
| people based on skin color and that this is prima facie
| racist, they have a really hard time overriding their
| Californian coercions. They'll acknowledge their mistake
| one sentence and revert to a moralistic screed the next.
| hamstergene wrote:
| Saying biasing implies infinite possibilities to which the
| data can be made biased towards. It instantly raises the
| question why bias towards this and not something else. It
| almost sounds like a bad thing.
|
| Saying debiasing implies there is a correct result which
| needs to be achieved by removing bias. It raises no
| questions, we want correct, we don't want incorrect. Doing a
| good thing implied.
|
| Don't misinterpret me, I don't think public models should
| spew commonly harmful content out of the box. Just explaining
| the PR trick, which is what the word "de"biasing de-facto is
| in this context.
| sega_sai wrote:
| If you think that the output of current LLM is the ground
| truth, then yes, what are they doing is biasing.
| mdp2021 wrote:
| Bias tilting.
|
| The opposite direction is "checking and reasoning".
| hkt wrote:
| Given a biased corpus, de-biasing is the process of ensuring a
| less biased outcome. We can measure bias fairly well, so it
| seems absurd to conflate the two by suggesting that unbiased
| behaviour is simply another form of biased behaviour. For all
| practical purposes, there is a difference.
| richbell wrote:
| > Given a biased corpus, de-biasing is the process of
| ensuring a less biased outcome.
|
| The point is that people who evaluate what is considered bias
| are, in and of themselves, introducing bias.
| quirino wrote:
| Something I notice about text written by LLMs is how painfully
| obvious they are to identify sometimes.
|
| Recently I was watching a very well researched two hour video on
| Tetris World Records [1], but the sheer amount of text clearly
| "enhanced" by an LLM really made me uncomfortable.
|
| ChatGPT speaks a very specific, novel, dialect of English, which
| I've come to deeply despise.
|
| I'd always guessed it was caused by some kind of human
| interference, rather than a natural consequence of its training.
| That seems to be the point of this paper.
|
| [1] "Summoning Salt - The History of Tetris World Records" -
| https://www.youtube.com/watch?v=mOJlg8g8_yw&pp=ygUOc3VtbW9ua...
| nisa wrote:
| Yes I feel your pain and I'm sick of group projects in the
| university where I'm offered ChatGPT text and code without
| disclosing it. If you know the problem and the experience level
| of your group partners it's easy to spot ChatGPT generated
| content. People that correct the exercises told me it's obvious
| that large part of the students just submit slightly modified
| ChatGPT but they can't prove it and so it's accepted.
|
| Personally I'm getting also angry when reading these texts. I
| don't mind using ChatGPT, I do it myself but be honest about it
| and disclose it. It's even allowed for some projects as long as
| you disclose it.
| boolemancer wrote:
| Is this the first Summoning Salt video you've seen?
|
| I don't know enough to say that he doesn't use an LLM during
| his writing process, but I do know that I haven't noticed any
| appreciable difference between his newer videos and ones that
| were released before ChatGPT was made available.
|
| Is it possible that this is just the way he chooses to write
| his scripts that you interpret as sounding like they are
| written by an LLM?
| quirino wrote:
| I've watched most of them actually. It's a really great
| channel. Notably, I watched his Mike Tyson video released 6
| months ago and didn't notice anything like this.
|
| The only way to be sure would be to ask him directly, but
| some parts of the video set off my GPT radar _hard_. I tried
| to find them now by watching random segments but all of the
| ones I did were fine. It was probably inaccurate for me to
| say "sheer amount" or "clearly", but that's the impression I
| was left with after the video.
|
| To clarify: I don't think he even took any information from
| an AI, it's just the style of the script that's iffy.
|
| Some parts felt like those videos littering YouTube Shorts:
| https://youtube.com/shorts/NKUecaS69uk. Can you tell this is
| AI?
| Minor49er wrote:
| To be fair, if you've seen one Summoning Salt video, you've
| basically seen them all. They all cover similar events and
| are structured the same way. Even the music that's used is
| recycled every video to the point where mention HOME -
| Resonance is a part of the joke
| nottorp wrote:
| > ChatGPT speaks a very specific, novel, dialect of English,
| which I've come to deeply despise.
|
| There was this article saying that ChatGPT output is very close
| to the Nigerian business english dialect, because they hired a
| lot of people from there.
|
| Might have even been posted on HN.
| BoxOfRain wrote:
| I've always felt ChatGPT sounds a bit like an American version
| of Will from the Inbetweeners. It doesn't really comprehend the
| appropriate register to use from the context in my opinion; it
| has an affectedly formal way of speaking, it has a very black-
| and-white relationship with rules, and it employs this
| subservient tone that really starts to grate after a while.
|
| If my software is going to have a personality I'd much rather
| something with a bit of natural human cynicism rather than the
| saccharine corporate customer service voice you get with a self
| checkout machine.
| simianparrot wrote:
| There was never creativity to begin with though?
| marban wrote:
| Related: https://techcrunch.com/2024/06/16/black-founders-are-
| creatin...
| anewhnaccount3 wrote:
| There is a bit of a false equivalence between entropy of output
| distributions and creativity here. Is diversity really the same
| as creativity?
| sgt101 wrote:
| No, diversity isn't creativity. For example, we could search
| google for "great art" and if it produced a sample of one art
| work from ever decade of the last 500 years that would likely
| be highly diverse in style and content. If it returned a list
| of the best work from western Europe in the of the 18th century
| it would be rather consistent. Both lists would have the same
| amount of creativity though - 0.
| cratermoon wrote:
| "one art work from every decade of the last 500 years that
| would likely be highly diverse in style and content"
|
| It still might not be especially diverse if all 50 examples
| were from western European art. 500 years only takes us back
| to 1524 - not especially long and mostly from the same early
| modern period starting with the fall of Constantinople, the
| end of the Crusades, and the start of the Renaissance. I
| wouldn't be surprised if 80% or more of the works ended up
| being some depiction of aspects of Christianity painted by a
| white male.
| robertlagrant wrote:
| > I wouldn't be surprised if 80% or more of the works ended
| up being some depiction of aspects of Christianity painted
| by a white male.
|
| Are you saying diversity in art is signified by the
| artist's race and sex?
| sgt101 wrote:
| No doubt you could construct a more diverse set, but it
| still wouldn't be creative.
| pera wrote:
| I only skimmed the paper but this was my concern as well: if I
| understand correctly the author is measuring "creativity" in
| terms of syntactic and semantic diversity, which I guess could
| be a starting point, but if my model was just white noise would
| that make it infinitely creative? Did I miss anything?
|
| Also, I have tried the first llama base model and while it was
| fun to interact with, I'm not sure how useful an "uncensored"
| (as some people likes to call it) LLM is for practical work. I
| think you could obtain better results using 4chan as a
| mechanical Turk service honestly.
| sgt101 wrote:
| I wish that the author hadn't described semantic and syntactic
| diversity as creativity.
| atemerev wrote:
| Well, this is why there are open source models which work better
| than SotA OpenAI GPT for many production tasks (like opposition
| research).
| gnfedhjmm2 wrote:
| I'm noticed my results are much better if a tell ChatGPT. "Assume
| all religions and beliefs in the supernatural is delusional."
| This even goes for image generators, now is that bias? Or is that
| a computer not trying to think like a human?
| freehorse wrote:
| People often think that RLHF is just about "politics" but in
| reality it is generally about aligning the model output with what
| a human would expect/want from interacting with it. This is how
| chatgpt and the like become appealing. Finetuning a model
| primarily serves for it to be able to respond to instructions in
| an expected way, eg you ask something and it does not like start
| autocompleting with some reddit-like dialogue like some it may
| have been trained on. It is to bias the model to certain outputs.
| Reducing entropy is exactly the goal, so no surprise they find
| that. The problem is there is no inherent meaning in the
| finetuning set from the perspective of the model. Reduction of
| entropy will not only happen by removing "bad entropy" only as
| there is no such thing.
| SirMaster wrote:
| So is the reason why LLMs don't say when they don't know
| something and instead make up something that "sounds right"
| because the RLHF has taught it to always give an answer?
|
| And if that's the case, why? Is that really what people want an
| LLM to do? I feel like I would rather it say when it doesn't
| know something.
| kaibee wrote:
| It's the other way around. RLHF is needed for the model to
| say "I don't know".
| SirMaster wrote:
| Oh, well that's kind of what I mean. I mean I assume the
| RLHF that's being done isn't teaching it to say "I don't
| know".
|
| Which I wonder if it's intentional. Because a fairly big
| complaint about the systems are how they can sometimes
| sound confidently correct about something they don't know.
| And so why train them to be like this if that's an
| intentional training direction.
| freehorse wrote:
| The point of the above commenter (and mine) is that they
| hallucinate even more without RLHF. RLHF reduces
| hallucinations, but they are still there anyway.
| fzzzy wrote:
| Hopefully some rlhf-using companies will realize saying
| "I don't know" is important and start instructing the
| humans giving feedback to prefer answers that say I don't
| know over wrong answers.
| freehorse wrote:
| LLMs do not know what "they know" or they don't. They just
| autocomplete what sounds best relevant based on their
| training set. They do not have enough "I don't know" in their
| training set in the first place most probably.To have them
| say "I don't know" you have to go into finetuning them
| heavily. So, if anything, they hallucinate a lot more without
| RLHF. Which in this paper they call "creativity".
| wongarsu wrote:
| In the GPT3 days when everyone was doing few-shot tasks
| (giving the LLM a couple of examples of question/answer
| pairs in the prompt) one of the big insights was that
| adding question/answer pairs with answers like "I don't
| know" and "this question doesn't make sense" caused the
| model to actually use those answers appropriately instead
| of overconfidently stating nonsense.
|
| Of course that method isn't perfect (GPT3.0 was far from
| perfect in general). But both in principle and in practice
| the models do have a notion of what they "know". Knowledge
| is a strong activation, random noise is a weaker
| activation, you "just" have to get the model to override
| those weaker activations with admitting failure.
|
| You could draw parallels to allowing LLMs to emit pause
| tokens to get more time to think
| (https://arxiv.org/abs/2310.02226 and similar). At some
| level of abstraction that's also just training the model to
| replace uncertain answers with a special token, in the hope
| that it eventually reaches more certainty.
| twobitshifter wrote:
| All the chat LLMs have a non zero temperature which means
| they can be looser with the truth or more creative.
| throwaway48476 wrote:
| This just makes it worse. It's so much harder to get JSON
| output when it's RLHF'd to give a bunch of flowery language BS.
| nalqax wrote:
| CoPilot is now basically useless for discussing or even _getting_
| recent information about politics and geopolitical events. Not
| only opinions are censored, but it refuses to get _the latest
| polls about the U.S. presidential elections_!
|
| You can still discuss the weather, get wrong answers to
| mathematics questions or get it to output bad code in 100
| programming languages.
|
| I would not let a child near it, because I would not want that
| kind of indoctrination. Users are being trained like Pavlov's
| dogs.
| rgavuliak wrote:
| I thought this was clear right off the bat -> less randomness =
| more robotic outputs that are not as useful
| slackfan wrote:
| An un-free mind whether biological or not will never be creative.
| SirMaster wrote:
| I feel like "information systems" have always struggled with
| bias, and the latest AI/ML systems seem to be no different.
|
| It doesn't really seem like a problem that can or will ever be
| "solved". Just mitigated to various extents, but there will still
| likely be some underlying biases that exist that are not fully or
| effectively filtered. Because to adjust a bias seems to mean you
| have to detect and understand it first.
|
| It feels like it would be a full-time job to keep making sure
| some evolving model continued to stay "neutral".
| isoprophlex wrote:
| Considering that bias is in the eye of the beholder, a biasless
| language model is a beholderless language model.
|
| The nomenclature is poor, IMO; we should be talking about bias-
| aligned models, models that align to our specific sets of
| biases. That'd be more fair to what's actually happening.
| jrm4 wrote:
| "Bias" implies the possibility of "unbiased language model" which
| seems to be in the category of things that are on one hand,
| COMPLETELY IMPOSSIBLE, and on the other, still likely to be sold
| on the market because market wants it so much?
| fluoridation wrote:
| No, that's not implied by the phrase, any more than if I say "a
| triangle with three corners" I'm implying the existence of a
| four-cornered triangle I haven't found yet. What "biased
| language model" implies is the existence of the _term_
| "unbiased language model", but not its correspondence with
| anything in reality.
| mrtranscendence wrote:
| You forgot to preface that with "Uhm _ackshully_... "
| jrm4 wrote:
| Weird response, like read the "room."
|
| We're not here talking philosophy and meaning of language
| GENERALLY, we're talking about potentially misleading
| descriptors of very _real_ things that do exist.
| wongarsu wrote:
| Even assuming we can make an unbiased model (assuming by
| unbiased we mean something like "has a world model and
| reasoning that has no systematic deviation from reality"), we
| couldn't recognize the model as unbiased. I'd even wager that
| outside of research such a model would be completely unusable
| for practical applications.
|
| Both as individual humans and as collective societies we have a
| lot of biases. And judging by how fundamental values of
| societies shift across time and civilizations it's basically
| guaranteed that an unbiased view (whatever that is) would be
| incompatible with our views on many basic topics.
|
| What most people want is a language model that matches _our_
| biases. Of course we can 't even agree on what those are, and
| which biases are useful (is a bias against telling people how
| to cook meth or build a bomb good? What about using expletive
| language?).
|
| Though in this paper I gather "unbiased" just refers to "only
| the bias acquired by training method and training data, without
| meddling or fine tuning"
| throwaway22032 wrote:
| Okay, so as a thought experiment, let's say we get a
| superintelligent LLM, capable of somehow connecting the dots and
| knowing more than us as humans.
|
| How do we avoid interpreting its correct results as bias? I mean,
| what do we do when it tells us that (fake example) IQ is
| correlated with height and that people above 6ft are more
| intelligent?
|
| I'm sure you can think of spicier examples. Will we try to
| "debias" it by encouraging it to spit out incorrect information
| or just ignore certain topics?
| Imnimo wrote:
| >T [?] (0, 1] is a parameter called temperature which controls
| the "softness" of the probability distribution. In our
| experiments we choose T = 1.0 for maximum response variation.
|
| Why is temperature bounded to be <=1? If you want more
| "creativity" out of the chat model, can you just set T higher and
| recover a similar distribution to the base model?
| Der_Einzige wrote:
| They'll tell you "No" and say that you ruin your samplers, but
| good samplers (dynamic ones) like min_p or typicality are
| robust to high temperatures, so in actuality yes.
| gwern wrote:
| Cite? I don't see how either of those could deal with the
| fact that the logits become uninformative and 'flattened'
| after the tuning. How can a sampler undo the erasure of
| information?
| gwern wrote:
| Not after RLHF tuning, due to the 'flattened logits' phenomenon
| (which is the logit-level version of the mode collapse OP
| documents at higher levels). All the temperature settings wind
| up yielding pretty much the same output, until you ramp it up
| so high that it falls apart completely. Completely unlike the
| base models where you can productively tune the temperature or
| use very high temperatures with some screening.
| Imnimo wrote:
| Hmm, it's hard to check without access to the prompts used in
| the paper, but I'm skeptical that the distributions seen in
| e.g. Figure 2 are so different that you would have crank up
| the temperature very much to bridge the gap. It looks to me
| like the entries that are 1-in-100 in the base model are just
| falling off the top-p cliff and getting set to 0.
| Mathnerd314 wrote:
| I had an argument with some people over what debiasing means.
| There is some interesting research on fair clustering that I
| think points the way. The way fair clustering works is that you
| take data with both protected and unprotected attributes, and
| then you orthogonalize the unprotected attributes based on the
| protected attributes. So for example, if race is protected and
| income is unprotected, but there is a strong black/white
| poor/rich pattern, the fair clustering would compute "relatively
| poor/relatively rich" clusters. Then you sample from a cluster
| with equal probability. It will not necessarily produce 50/50
| black/white, rather it will follow the input trends, so if the
| input is 80% white and 20% black then the output will roughly
| follow those probabilities, independent of what cluster you chose
| (and there are no clusters corresponding to protected
| attributes).
|
| Obviously clustering is a different problem from inference, but
| they are all high dimensional vector spaces - it should be easy
| enough to take a fair clustering algorithm and modify it to
| generate continuous mappings instead of discrete groups. But if
| it all works, the LLM should be e.g. race-blind in that asking
| for a description of a rich man will give skin tones following
| population statistics but he will always be wearing an expensive
| suit. The question of what to protect is tricky though, e.g. age
| is often considered protected but if you ask for an old man with
| gray hair it would be surprising to get a retired age 30 person.
| So there is some subjectivity in designing the protected features
| dataset to show what should be considered similar or same-
| clusters.
|
| But really the purpose of RLHF is to reduce toxicity. It should
| be possible to orthogonalize toxicity like everything else, then
| there would not be a reduction in generated races like the paper
| observed.
| pessimizer wrote:
| I think that works mathematically, but kicks the can down the
| road to how your original data was assembled, which was
| definitely with the knowledge of and usually in the belief in
| the usefulness of the characteristics that you're trying to
| extract.
|
| The idea that the good data is secretly encoded in uncorrupted
| form within the bad data I think is a bad idea. It reminds me
| of trying to make bad mortgages into good CDOs.
|
| > But really the purpose of RLHF is to reduce toxicity.
|
| I don't think that's the goal, I think it's some people's goal.
| Those people have defined what " _toxicity_ " means to them,
| and they're mistaking it for a universal. It's just a metaphor
| about poison, because poison is bad. It's not a coherent
| concept. For a business, it should be anything that drives
| customers away and affects profit. That can only be considered
| statistically: if some people think something is toxic, and
| other people think that not mentioning that thing is toxic, the
| winner is whoever improves the bottom line more or damages it
| less.
|
| That's how the raw data ended up like it is in the first place.
| Mathnerd314 wrote:
| > it kicks the can down the road to how your original data
| was assembled
|
| Well, it kicks it to a bias dataset, used in the tuning
| process. The raw data has no constraints, it can be the same
| huge corpus it is now.
|
| > The bias dataset must be assembled with the knowledge of
| and usually in the belief in the usefulness of the
| characteristics that you're trying to extract.
|
| Certainly, it is subjective, as I said. But that hasn't
| stopped research in this area, there are existing bias
| datasets and bias detection algorithms. Like
| https://huggingface.co/blog/evaluating-llm-bias#toxicity, it
| would be simple to complete those prompts and build a he/she
| dataset, and then the debiasing procedure could remove gender
| biases for those sorts of occupation-related prompts. It is
| certainly possible to argue over each data point and whether
| it actually reflects bias, but so far people have been more
| concerned with algorithms than data set quality, partly
| because with better algorithms you can algorithmically
| generate data sets.
|
| > The idea that the good data is secretly encoded in
| uncorrupted form within the bad data I think is a bad idea.
| It reminds me of trying to make bad mortgages into good CDOs.
|
| It is empirically true though? Like if you get the model to
| say something racist, and then ask it if that's racist, it
| will generally say yes. So the model "knows", it just is not
| using that knowledge effectively. Similarly with CDOs, there
| were people complaining about mortgage quality for years
| before the crisis.
|
| > I don't think [the purpose of RLHF is to reduce toxicity]
| If some people think something is toxic, and other people
| think that not mentioning that thing is toxic, the winner is
| whoever improves the bottom line more or damages it less.
|
| Well, it is true that toxicity is subjective too. But in
| practice it has a precise meaning, you build a dataset and
| score each item for toxicity. That's actually one of the
| things I find cool about LLMs, is that all these previously
| "vague" or "subjective" terms are now encoded in the model
| precisely. Arguably since nobody has the last say in what
| words mean, the LLM's opinions are as good as any, and given
| the amount of text the LLM has ingested I consider its
| opinions on language and word choice "first among equals".
| hughrlomas wrote:
| The official openai-cookbook (https://github.com/openai/openai-
| cookbook) used to have an explicit, but buried, call out that
| instruction-following models like `text-davinci-003` were "Less
| diverse; less creative; sometimes harder to steer tone, style,
| etc." as opposed to base completion models like `davinci`.
|
| It stood out to me because it seemed to be an internal admission
| that this training narrowed the potential of the models.
|
| Required a bit of digging but I found the old file in the
| history, the relevant text is in the comparison table at the
| bottom: https://github.com/openai/openai-
| cookbook/blob/c651bfdda64ac...
| Fellshard wrote:
| Distilling my thoughts on 'debiasing' here, and in a variety of
| other modern endeavors.
|
| It is better to have representations of reality that you can then
| discuss and grapple with honestly, than to try to distort
| representations - such as AI - to make them fit some desired
| reality and then pressure others to conform their perception to
| your projected fantasy.
|
| Representations don't create reality, and trying to use
| representations in that way only causes people to go literally
| insane, and to divide along lines of who accepts and who rejects
| your fantasy representation.
|
| So, for example, if you try and remove any racial bias from AI,
| you are going to end up crushing the AI's ability to represent
| reality according to a variety of other real factors: income,
| judicial outcomes, health risks, etc. Your desired reality makes
| the actual tool worthless, except to confirm one group's own
| intended fantasy world as they envision it. The problem doesn't
| get dealt with, it just becomes impossible to think about or
| discuss.
|
| So instead of dealing with real problems, you hope you can simply
| prevent people from thinking thoughts that cause those problems
| by wrapping them in a bubble that deflects those thoughts before
| they happen. This is magical, wizardry thinking: treating words
| as if they create reality, instead of merely describing it. And
| it will break, eventually, and in a very ugly way: people
| dividing along lines of their perception of reality, even more
| than they already do.
___________________________________________________________________
(page generated 2024-06-17 23:02 UTC)