Subj : Re: Writing with LLMs
To   : tenser
From : Bob Worm
Date : Thu Sep 18 2025 21:08:33

  Re: Re: Writing with LLMs
  By: tenser to Nightfox on Thu Sep 18 2025 23:49:07

Hi, tenser.

 > alternated confidently between asserting that the observed
 > behavior was intended or a bug.

I heard that this is a result of the way the models are trained. Essentially the model gets no "points" for saying that it doesn't know the answer to something, whereas a confidently stated answer with some grain of truth in it (something you'd call bluffing if a person did it) would score at least some points. So you get what's incentivised.

BobW
--- SBBSecho 3.29-Linux
 * Origin: >>> Magnum BBS <<< - magnumbbs.net (21:1/205)

.