[HN Gopher] Some thoughts on autoregressive models
___________________________________________________________________
Some thoughts on autoregressive models
Author : Wonderfall
Score : 70 points
Date : 2025-03-03 16:40 UTC (4 days ago)
(HTM) web link (wonderfall.dev)
(TXT) w3m dump (wonderfall.dev)
| CamperBob2 wrote:
| The author (and Chomsky) fail to understand that LLMs (as well as
| human brains) are not just autoregressive models, but _nonlinear_
| autoregressive models. Put a slightly different way, you can
| describe LLMs as autoregressive, but only by taking liberties
| with the classical definition of 'autoregressive.'
|
| _The human mind is not, like ChatGPT and its ilk, a lumbering
| statistical engine for pattern matching, gorging on hundreds of
| terabytes of data and extrapolating the most likely
| conversational response or most probable answer to a scientific
| question. On the contrary, the human mind is a surprisingly
| efficient and even elegant system that operates with small
| amounts of information; it seeks not to infer brute correlations
| among data points but to create explanations. - Noam Chomsky_
|
| It's as if Chomsky has either never heard of transformers, or
| doesn't understand what they do.
|
| _Before speaking a sentence, we have a general idea of what
| we're going to say; we don't really choose what to say next based
| on the last word. That kind of planning isn't something that can
| be represented sequentially._
|
| It's as if the author (and Chomsky) has never seen a CoT model in
| action.
| Wonderfall wrote:
| Author here and I welcome the feedback, but I don't really
| understand your point. My post is clearly not dismissive of
| efforts to make LLMs reason using CoT prompting techniques and
| post-training, and I think such efforts are even mentioned. The
| model remains autoregressive either way, and this reasoning is
| not some kind of magic that makes them behave differently -
| these improvements only make them perform (much) better on
| given tasks.
|
| Additionally, I'm not dismissive of the non-linear nature of
| transformers which I'm familiar with. Attention mechanism is a
| lot more complex than a linear relationship between the
| prediction and the past inputs, yes. But the end result remains
| sequential prediction. Ironically, diffusion models are kind of
| the opposite: sequential internally, parallel prediction at
| each step.
|
| (Note: added note on terminology since the confusion arised by
| my use of "linearity", which was not referring to the attention
| mechanism itself. I've read so many papers that are perfectly
| fine with the use of "autoregressive" for this paradigm that I
| forgot some people coming from traditional statistics may be
| confused. Also "based on the last word" was wrong and meant
| "last words" or "previous words", obviously.)
|
| All that being said, I don't think it's fair to say one doesn't
| understand how transformers work solely because of semantic
| interpretation. I appreciate the feedback though!
| nikhilsimha wrote:
| Not saying that our current approaches will lead to
| intelligence. No one can know.
|
| It could very well be that the internal mechanism of our
| thought has an auto-regressive reasoning component.
|
| With the full system effectively "combining" short term
| memory (what just happened) and "pruned" long-term memory
| (what relevant things i know from the past) and pushing that
| into a RAW autoregressive reasoning component.
|
| It is also possible that another specialized auto regressive
| reasoning component is driving the "prune" and "combine"
| operations. This whole system could be solely represented in
| the larger network.
|
| The argument that "intelligence cannot be auto-regressive"
| seems to be without basis to me.
|
| > there is strong evidence that not all thinking is
| linguistic or sequential.
|
| It is possible that a system wrapping a core auto-regressive
| reasoner can produce non-sequential thinking - even if you
| don't allow for weight updates.
| Wonderfall wrote:
| I completely agree. I never said that "intelligence cannot
| be auto-regressive", I just questioned whether this can be
| achieved or not this way. And I don't actually have
| answers, I just wrote down some thoughts so it would
| sparkle some interesting discussions about that, and I'm
| glad it did work (a little) in the end.
|
| I also mentioned that I'm supportive of architectures that
| will integrate autoregressive components. Totally agree
| with that.
| eldenring wrote:
| > The context window can be compared to working memory in humans:
| it's fast, efficient but gets rapidly overloaded. Humans manage
| this limitation by offloading previously learned information into
| other memory forms, whereas LLMs can only mimic this process
| superficially at best.
|
| This is just silly. Humans forget things all the time! If I want
| to remember something I write it down.
|
| > The nature of hallucination is very different between AR models
| and humans, as one has a world model and the other doesn't.
|
| I stopped reading at this point. There's not much signal here,
| just basic facts about LLMs and then leaps to very bold
| statements.
|
| Here is an interesting experiment I use to help people understand
| next token prediction. Think of a simple math problem in your
| head, maybe 3 digit by 2 digit multiplication. Then speak out
| every single thought you have while solving it.
| raylad wrote:
| Do you think in words when you do a 3 x 2 digit multiplication?
|
| I do it all in images and I think many other people do too.
| throwing_away wrote:
| We think of LLMs as not math-proficient (because they aren't
| yet) but what about those multimodal models?
|
| I wonder if anyone has tried getting them to "imagine" math
| in a way to "visually compute".
| Wonderfall wrote:
| > There's not much signal here, just basic facts about LLMs and
| then leaps to very bold statements.
|
| The article wasn't supposed to be informative for people who
| already know how LLMs work. Like the title said, just wanted to
| write down some thoughts.
|
| > This is just silly. Humans forget things all the time! If I
| want to remember something I write it down.
|
| The opposite was never stated. Human memory is of course
| selective.
|
| > Here is an interesting experiment I use to help people
| understand next token prediction. Think of a simple math
| problem in your head, maybe 3 digit by 2 digit multiplication.
| Then speak out every single thought you have while solving it.
|
| Now a point I'm happy to discuss! The process of solving it is
| actually quite autoregressive-like, but this is also an example
| of a common pitfall with LLMs: they purely rely on pattern
| matching because they don't have the internal representation of
| what they really deal with (algebra). But we all know that.
|
| The main question is whether LLMs taught to reason actually
| show that they have this kind of representation. They still
| work very differently I'd say; even for tasks that seem trivial
| to humans, reasoning LLMs will make a lot of mistakes before
| arriving at a plausible-sounding result. Because it was trained
| to reason, there's a higher chance now that the plausible-
| sounding result is actually correct. But this property is
| actually quite interesting once applied to complex tasks that
| would take too much time and overwhelming for humans, and
| that's where they shine as powerful tools.
| kevmo314 wrote:
| > even for tasks that seem trivial to humans, reasoning LLMs
| will make a lot of mistakes before arriving at a plausible-
| sounding result.
|
| Like a lot of my coworkers analyzing a production bug? I
| would agree if the statement were that LLMs were underpowered
| compared to a human brain today but I'm not seeing evidence
| that humans do reasoning in a way that can't be correctly
| modeled.
|
| From your article and comments, it sounds like the take is
| something like "humans don't actually reason
| autoregressively" which could be true, I don't know enough to
| know, but sort of like saying physics models aren't _really_
| how nature works: ultimately LLMs are executable models of
| the world, it 's even in the name.
| Wonderfall wrote:
| > From your article and comments, it sounds like the take
| is something like "humans don't actually reason
| autoregressively" which could be true, I don't know enough
| to know, but sort of like saying physics models aren't
| really how nature works: ultimately LLMs are executable
| models of the world, it's even in the name.
|
| The conclusion states "Language and thought are not purely
| autoregressive in humans".
|
| Which doesn't mean humans don't have autoregressive
| components in their thinking. At least that's my opinion. I
| don't make this bold statement and I don't know enough to
| know, too.
|
| > Like a lot of my coworkers analyzing a production bug? I
| would agree if the statement were that LLMs were
| underpowered compared to a human brain today
|
| Clearly not in the same way and that was what I was trying
| to explain with regards to the hallucination issue too.
| Humans are also learning from proofs, can apply frameworks,
| etc. there's no denying that. But the internal process of
| an LLM remains pattern matching and sequential prediction
| whereas there's more to the human's thinking process.
|
| LLMs are underpowered in some aspects that can't be
| replicated with autoregressive modeling, but are already
| stronger in other aspects. That is what I think.
|
| > but I'm not seeing evidence that humans do reasoning in a
| way that can't be correctly modeled.
|
| Me neither, this is not what my stance is, and I'm actually
| optimistic about it. I just don't think we should be
| satisfied with only autoregressive modeling if the ambition
| is to reach or comprehend human-level intelligence.
| fergal_reid wrote:
| Similar arguments to LeCun.
|
| People are going to keep saying this about autoregressive models,
| how small errors accumulate and can't be corrected, while we
| literally watch reasoning models say things like "oh that's not
| right, let me try a different approach".
|
| To me, this is like people saying "well NAND gates clearly can't
| sort things so I don't see how a computer could".
|
| Large transformers can clearly learn very complex behavior, and
| the limits of that are not obvious from their low level building
| blocks or training paradigms.
| Tostino wrote:
| I think recurrent training approaches like those discussed in
| COCONUT and similar papers show promising potential. As these
| techniques mature, models could eventually leverage their
| recurrent architecture to perform tasks requiring precise
| sequential reasoning, like odd/even bit counting that current
| architectures struggle with.
| yorwba wrote:
| As the number of self-corrections increases, it also increases
| the likelihood that it will say "oh that's not right, let me
| try a different approach" _after finding the correct solution_.
| Then you can get into a second-guessing loop that never arrives
| at the correct answer.
|
| If the self-check is more reliable than the solution-generating
| process, that's still an improvement, but as long as the model
| makes small errors when correcting itself, those errors will
| still accumulate. On the other hand, if you can have a reliable
| external system do the checking, you can actually guarantee
| correctness.
| solveit wrote:
| Error correction is possible even if the error correction is
| itself noisy. The error does not need to accumulate, it can
| be made as small as you like at the cost of some efficiency.
| This is not a new problem, the relevant theorems are
| incredibly robust and have been known for decades.
| yorwba wrote:
| Can you link me to a proof demonstrating that the error can
| be made arbitrarily small? (Or at least a precise statement
| of the theorem you have in mind.) I would think that if the
| last step of error correction turns a correct intermediate
| result into an incorrect final result with probability _p_
| , that puts a lower bound of _p_ on the overall error rate.
| dartos wrote:
| > while we literally watch reasoning models say things like "oh
| that's not right, let me try a different approach".
|
| Not saying I disagree with your premise that errors can't be
| corrected by using more and more tokens, but this argument is
| weird to me.
|
| The model isn't intentionally generating text. The kinds of "oh
| let me try a different approach" lines I see are often followed
| by the same approach just taken. I wouldn't say most of the
| time, but often enough that I notice.
|
| Just because a model generates text doesn't mean that the text
| actually represents anything at all, let alone a reflection of
| an internal process.
| TeMPOraL wrote:
| > _Just because a model generates text doesn't mean that the
| text actually represents anything at all, let alone a
| reflection of an internal process._
|
| What does it represent then? What are all these billion
| weights for? It's not a bag full of NULLs that just pulls
| next words from a look-up table. _Obviously_ there is some
| kind of internal process.
|
| Also I don't get why people ignore the temporal aspect.
| Humans too generate thoughts in sequence, and can't
| arbitrarily mutate what came before. _Time_ and memory is
| what forces sequential order - we too just keep piling on
| more thoughts to correct previous thoughts while they are
| still in working memory (context).
| _heimdall wrote:
| The text represents a prediction of how a human may
| respond, one word(ish) at a time, that's it.
|
| With "reasoning" models, the reasoning layer is basically
| another LLM instructed to specifically predict how a human
| may respond to the underlying LLM's answer, fake prompt
| engineering if you will.
|
| There of course is some kind of internal process, but we
| can't prove any kind of reasoning. We ask a question, the
| main LLM responds, and we see how the reasoning layer LLM
| itself responds to that.
| _0ffh wrote:
| Please don't confuse people with wrong information, the
| reasoning part in reasoning models is the exact same LLM
| that produces the final answer. For example o1 uses
| special "thinking" tokens to demarcate between reasoning
| and answer sections of it's output.
| _heimdall wrote:
| Sure, that's a great clarification though maybr a bit of
| an implementation detail in this context.
|
| Functionally my argument stands in this context - just
| because we can see one stream of LLM responses responding
| to the primary response stream says nothing of reasoning
| or what is going on internally in the reasoning layer.
| dartos wrote:
| > what is going on internally in the reasoning layer.
|
| We literally know exactly what is going on with every
| layer.
|
| It's well defined. There are mathematical proofs for
| everything.
|
| Moreover it's all machine instructions which can be
| observed.
|
| The emergent properties we see in LLMs are surprising and
| impressive, but not magic. Internally what is happening
| is a bunch of matrix multiplications.
|
| There's no internal thought or process or anything like
| that.
|
| It's all "just" math.
|
| To assume anything else is personification bias.
|
| To look at LLMs outputting text and a human writing text
| and think "oh these two things must be working in the
| same way" is just... not a very critical line of thought.
| dartos wrote:
| > Humans too generate thoughts in sequence,
|
| You don't know this. I don't feel like I generate thoughts
| in sequence, for me it feels hierarchical.
|
| > can't arbitrarily mutate what came before
|
| Uhh... what?
|
| Do you remember your memories as a child? Or what you ate
| for breakfast 3 weeks ago?
|
| Have you ever misremembered an event or half remembered a
| solution to a problem?
|
| The information in human minds are entirely mutable. They
| are not like computers...
|
| > It's not a bag full of NULLs that just pulls next words
| from a look-up table.
|
| Funny enough, the attention mechanism that's popular right
| now is effectively lots and lots of stacked look up tables.
| That's how it's taught as well (what with the Q K and V)
|
| Tho I don't think that's a requirement for LLMs in general.
|
| I find a lot of people who half understand cognition and
| understand computing look at LLMs and work backwards to
| convince themselves that it's "thinking" or doing more
| cognitive functions like we humans do. It's personification
| bias.
| Tostino wrote:
| Not OP.
|
| > Do you remember your memories as a child? Or what you
| ate for breakfast 3 weeks ago?
|
| For me, this seems like conjuring up and thinking about a
| childhood event is like putting what came out of my
| nebulous 'memory' fresh into context at the point in time
| you are thinking about it, along with whatever thoughts I
| had about it (how embarrassed I was, how I felt proud
| because of X, etc). As that context fades into the past,
| some of those thoughts may get mixed back into that
| region of my 'memory' associated with that event.
| nyrikki wrote:
| Subbarao Kambhampati, who seems to only use X is a good
| resource. He points out how the CoT text is not of semantic
| importantce.
|
| This work from his team shows how few 'reasoning' traces
| are valid.
|
| https://atharva.gundawar.com/searchformer_response_analysis
| ....
|
| This paper shows how the scratch space gets transformers to
| PTIME from TC0 without it.
|
| https://arxiv.org/abs/2502.02393
|
| OpenAI may be able to do more in the long term because they
| don't show the <think> and can spend more of that scratch
| space on improving answers vs appeasing users, but time
| will show.
|
| Remember that probabilistic checkable proofs show how
| random data can improve computation.
|
| The AI field has always had a problem with wishful mnomics.
|
| But it is probably not a binary choice, if we could get the
| scratch space to reliably simulate Dykstra' shunting and
| convert to postfix as an example, that would be great.
| naasking wrote:
| > The model isn't intentionally generating text.
|
| What's the mechanistic model of "intention" that you're using
| to claim that there is no intention in the model's operation?
|
| > Just because a model generates text doesn't mean that the
| text actually represents anything at all, let alone a
| reflection of an internal process.
|
| Generating text is the trace of an internal process in an
| LLM.
| energy123 wrote:
| Yann LeCun's prediction was empirically refuted. He says that
| the longer LLMs run, the less accurate they get. OpenAI showed
| the opposite is true.
| mrfox321 wrote:
| They didn't show this, they just increased the length where
| accuracy breaks down.
| energy123 wrote:
| Explain? OpenAI showed the new scaling law in December 2024
| that performance keeps increasing proportional to ln(N
| reasoning tokens)
| mentalgear wrote:
| link?
| Wonderfall wrote:
| LeCun is for sure a source of inspiration, and I think he has a
| fair critique that still holds true despite what people think
| when they see reasoning models in action. But I don't think
| like him that autoregressive models are a doomed path or
| whatever. I just like to question things (and don't have
| absolute answers).
|
| I-JEPA and V-JEPA have recently shown promising results as
| well.
| PartiallyTyped wrote:
| I'd argue that humans are by definition autoregressive
| "models", and we can change our minds mid thought as we process
| logical arguments. The issue around small errors accumulating
| makes sense if there is no sense of evaluation and recovery,
| but clearly, both evaluation and recovery is done.
|
| Of course, this usually requires the human to have some sense
| of humility and admit their mistakes.
|
| I wonder, what if we trained more models with data that self-
| heals or recovers mid sentence?
| ziofill wrote:
| Is anyone aware of a formalization of the idea that to get
| "symbols" out of fuzzy probability distributions one needs
| distributions whose value goes exactly to zero over some regions
| of the domain? I.e. Gaussian mixtures won't cut it. And they will
| need very high Fourier frequencies.
|
| I have the gut feeling that until a model allows for a small
| probability that 2x3 is 7, there will always be hallucinations.
| Probabilities need to be clamped to zero to emulate symbolic
| behaviour.
| toxik wrote:
| We do clamp probabilities to zero. Look into top-p sampling or
| nucleus sampling.
| TeMPOraL wrote:
| Symbolic behavior is artificial and not how humans think
| either. 0 is not a probability (neither is 1) - a value of 0 or
| 1 basically breaks calculations by dragging everything along to
| the limit, the same way infinity does, or 0 in the denominator
| (in fact, that's what 1 and 0 translate to if you switch to
| logprobs or other equivalent ways to calculate probabilities).
|
| Consider: if you clamp the probability distribution of answers
| to 2x3, so that it's 0 everywhere else and 1 at 6, you're
| basically saying that it is fundamentally impossible for you to
| misunderstand the question, or make mistake in the answer, or
| that you're dreaming, or hallucinating, or that you've
| momentarily forgotten that the question was preceded by "In
| base 4, what is ", or any number of other things that
| absolutely are _possible_ , even if highly unlikely, in the
| real world.
| DeathArrow wrote:
| >Most generative AI models nowadays are autoregressive.
|
| There are also diffusion based models which don't rely on next
| token prediction.
| Wonderfall wrote:
| Yeah, this is mentioned in the article. (The LLaDa paper is
| even what triggered its writing!)
| suddenlybananas wrote:
| >Isn't language by itself linear.
|
| We've known that language is hierarchal, not linear for hundreds
| of years at this point.
| Wonderfall wrote:
| I guess semantics matter. Language is primarily hierarchical,
| but its presentation is what's linear. And LLMs mainly learn
| and work from this presentation; the question is, and one of
| the main points, whether emerging patterns is enough evidence
| to show that there's hierarchical thinking.
| mxwsn wrote:
| > But what is the original purpose of AI research? I will speak
| for myself here, but I know many other AI researchers will say
| the same: the ultimate goal is to understand how humans think.
| And we think the best (or the funniest) way to understand how
| humans think is to try to recreate it.
|
| Eh. To riff on Dijkstra, this is like submarine engineers saying
| their ultimate goal is to understand how fish swim.
| Wonderfall wrote:
| I come from a medical science background, where I studied the
| brain from a "traditional" neuroscience perspective (biology,
| pathology, anatomy, psychology and whatnot). That the best way
| is actually to try to recreate it is honestly how I feel
| whenever I read about AI advancements where the clear goal is
| to achieve/surpass human intelligence, something we don't fully
| understand yet.
|
| "What I cannot create, I do not understand." someone clever
| once said.
| UncleEntity wrote:
| It doesn't really follow that we (humans) have to replicate
| how we (humans) gained intelligence, there very well could be
| a shortcut that doesn't involve millions of years of getting
| eaten by tigers.
| aoeusnth1 wrote:
| I think the author is projecting significantly when he says the
| goal of AI researchers is to understand and replicate how humans
| think. If you start from that wrong assumption of course it looks
| silly for them to be doing anything other than neuroscience
| research, the author's field.
|
| It's like saying the stockfish developers should stop researching
| mixed NN and search methods because they don't understand how
| humans play chess yet.
| Wonderfall wrote:
| This is mainly a misunderstanding due to the way I phrased it.
| This is what I think. I know for a fact that is the case for
| other AI researchers having watched many conferences - "all of
| them" is not what I meant (I wrote "many other") and we
| certainly need people to approach problems from different
| perspectives and backgrounds, since they will benefit from each
| other in the end. Not going to lie I'm a bit disappointed to
| see these kind of comments.
| aoeusnth1 wrote:
| Fair enough about your motivation, however you also to
| further in saying that the _best_ way to achieve and exceed
| human intelligence is to first understand it. That didn 't
| pan out for chess, it hasn't contributed much to our current
| SOTA approaches to many other problems where LLMs are king,
| and I'm not sure why neuroscientists are so confident in some
| future where their field is the key to intelligence when
| their track record of breakthroughs is so poor.
| Wonderfall wrote:
| I admit my phrasing was poor there, and I got too excited.
| I will clarify since I don't really disagree with you or
| what others said (claiming it's the best way is an
| overstatement).
|
| Well, one could say that neural networks pioneers modeled
| their ideas on simplified brain structures representations.
| Modern neural networks have little in common with an actual
| biological brain, however, the inspiration remains there
| (even for modern NNs like CNNs). I recall the intent was
| there too, originally: providing a framework to study
| biological cognition in the 50s. Then it evolved to become
| a new paradigm in computer science so that we have programs
| able to learn and adapt for problems that are formally too
| complicated for deterministic solutions.
| aithrowawaycomm wrote:
| > You can say LLMs are fundamentally dumb because of their
| inherent linearity. Are they? Isn't language by itself linear
| (more precisely, the presentation of it)?
|
| Any linearity (or at least partial ordering) of intelligence
| comes from time and causality, not language - in fact the
| linearity of language is a limitation human cognition struggles
| to fight against.
|
| I think this is where "chimpanzees are intelligent" comes to the
| rescue - AI has a nasty habit of focusing too much on humans. It
| is vacuous to think that chimpanzee intelligence can be reduced
| to a linear sequence of oohs-and-aahs, although I suspect a
| transformer trained on thousands of hours of chimp vocalizations
| could keep a real chimp busy for a long time. Ape cognition is
| much deeper and more mysterious: imperfect "axioms" and
| "algorithms" about space, time, numbers, object-ness, identifying
| other intelligences, etc, seem to be somehow built-in, and all
| apes seem to share deep cognitive tools like self-reflection,
| estimating the cognitive complexity of a task, robust
| quantitative reasoning, and so on. Nor does it really make sense
| to hand-wave about "evolutionary training data" - there are stark
| micro- and macro-architectural differences between primate brains
| and squirrel brains. Not to mention that all species have the
| exact same _amount_ of data - if it was just about millions of
| years, why are bees and octopi uniquely intelligent among
| invertebrates? Why aren 't there any chimpanzee-level squirrels?
| Rather than twisting into knots about "high quality evolutionary
| data," it makes a lot more sense to point towards evolution
| pressuring the development of different brain architectures with
| stronger cognitive abilities. (Especially considering how rapidly
| modern human intelligence seems to evolved - much more easily
| explained by sudden favorable mutations vs stumbling into an East
| African data treasure trove.)
|
| Human intelligence uses these "algorithms" + the more modern tool
| of language to reason about the world. I believe any AI system
| which starts with language and sensory input[1], then hopes to
| get causality/etc via Big Data is doomed to failure: it might be
| an exceptionally useful text generator/processor but there will
| be infinite families of text-based problems that toddlers can
| solve but the AI cannot.
|
| [1] I also think sight-without-touch is doomed to failure,
| especially with video generation, but that's a different
| discussion. And AIs can somewhat cheat "touch" if they train
| extensively on a good video game engine (I see RDR2 is used a
| lot).
| hansvm wrote:
| > By design, AR models lack planning and reasoning capabilities.
| If you generate one word at a time, you don't really have a
| general idea of where you're heading.
|
| I have one minor quibble here, which is that the limitation
| described isn't a criticism of AR models (whose outputs are only
| "backward-looking" for their inputs), but just a subset of AR
| models in popular use. An AR model is fully capable of generating
| a large state space and doing many computations (even doing many
| full-connected diffusion steps) before generating the first
| output token.
|
| That quibble wouldn't be worth mentioning unless AR models had
| some sort of advantage, but they do, and it's incredibly
| important. AR factorization of the conditional probabilities
| allows you to additively consider the loss contribution from each
| output token -- you can blindly shove whatever data you want into
| the thing, add up all the errors, and backpropagate, all while
| guaranteeing that the distribution you're learning is the same
| distribution from your training data.
|
| If you're not careful, via some mechanism (like AR), the
| distribution you learn will have almost nothing to do with the
| distribution you're training on -- a common failure mode being a
| tendancy to predict "average-looking" sub-tiles in a composite
| image and only predict images which can be comprised out of those
| smaller, averge-looking sub-tiles. Imagine (as an example, with
| low enough model capacity), you had a model generating people and
| everyone was vaguely 5'10", ambiguously gendered, and a bit tan,
| contrasted with that same model trained using AR where you'd
| expect the outputs to be bad in other ways if you had
| insufficient capacity but to at least have a mix of colors,
| heights, and genders. Increasing capacity can help, but why
| bother when something like AR solves it by definition?
___________________________________________________________________
(page generated 2025-03-07 23:01 UTC)