[HN Gopher] The illusion of state in state-space models
___________________________________________________________________
The illusion of state in state-space models
Author : canjobear
Score : 55 points
Date : 2024-06-07 11:53 UTC (11 hours ago)
(HTM) web link (arxiv.org)
(TXT) w3m dump (arxiv.org)
| optimalsolver wrote:
| >Figure 1: We prove that SSMs, like transformers, cannot solve
| inherently sequential problems like permutation com- position
| (S5), which lies at the heart of state-tracking prob- lems like
| tracking chess moves in source-target notation (see Section 3.2),
| evaluating Python code, or entity tracking. Thus, SSMs cannot, in
| general, solve these problems either
|
| Do Microsoft & friends who are about to build trillion dollar AI
| data centers know about these proven limitations of transformer-
| based architectures?
| mjburgess wrote:
| Yes. The solution is to hoover up all the queries people are
| currently putting to ChatGPT, save how people have "prompt
| engineer"ed the solution (ie., answered the question
| themselves); and then hope that you can just feed this back to
| people without them noticing.
|
| The only open question is whether people have common-enough
| queries for this charade to work out. It seems there's quite a
| lot at least. But this number will decrease over time for
| various reasons. So it's a game of building a system that can
| be retrained on the answers people are giving it fast enough
| that people don't notice where the answers are coming from.
| FeepingCreature wrote:
| Counterpoint: the more networks internalize the patterns
| behind engineered prompts, the closer they get to being
| general reasoners.
| mjburgess wrote:
| Statistical patterns in text tokens have nothing to do with
| reasoning. They lack such a property to learn in the first
| place. This would be more obvious, I guess, if the tokens
| were in an alien language. Consider a translation to 1,000
| different alien languages of any given novel.. there is no
| distributional property they would share.
|
| Statistical AI is just a way of sampling from a historical
| dataset with a similarity metric. It only works to answer
| questions if you're sampling from a (Q, A) database in the
| same language the user already understands. The question
| was answered by a reasoner, it is now answered by a system
| which replays answers.
| FeepingCreature wrote:
| This is just chinese room. But also, I disagree that the
| languages would not share properties. A novel is too
| small. Consider a network trained on GPT-3 scale datasets
| in 1000 alien languages. The shared structures behind the
| sentences will be the same, even if the grammar is
| completely different. Stars will be stars, moons will be
| moons. I'd bet you that the model would be able to
| translate shared concepts between those languages even if
| it had _not_ been trained on the same novels, same as GPT
| can translate terms that it has not seen in dictionary
| pairs.
|
| (I have no idea if it can! But I'm confident enough that
| I'm willing to just say it can, at risk of being proven
| wrong. If GPT can't do that, I'm fundamentally
| misunderstanding how it works - that is, I don't have a
| paper offhand showing that GPT shares concept neurons
| between languages, but I'm willing to bet I could find
| one if I went looking.)
|
| In other words, if you co-trained GPT-4 on Earth Internet
| and Alien Internet, there's a good chance it'd end up
| able to translate English to Alienese, purely as an
| emergent ability, if it had the concept of translation at
| all.
|
| Intelligence is compression. With sufficient abstraction
| (layers) and sufficient volume (dataset), any description
| of the same reality will assume the same structure. And
| no learning algo worth its salt will keep two identical
| structures around.
| mjburgess wrote:
| Almost every predicate in natural languages is an
| arbitrary association of properties in the world. There
| isn't any property a person has "bald", nor are there
| "tree"s in the world. What bundles of properties the vast
| majority of language names are a product of historical
| and contingent associations we've made for practical
| reasons.
|
| Likewise, languages do not have the same distributional
| structure. There is no reason an alien language would use
| discrete tokens to name properties, nor co-locate tokens
| by linear position in a 'sentence'. Historical human
| languages did not; using, eg., the full 2D structure of
| the clay tablet.
|
| To suppose that it is the glyphs and their colocation
| which somehow bare meaning is a nonesensical
| superstition. The world is what our words mean, and it is
| we, in that world, who provide them their meaning. We can
| do so with arbitrary linguistic structures.
| FeepingCreature wrote:
| Right, but neural networks at a certain level of scale
| begin to compress _the perception described by the text_
| rather than the text per se. The lexical features are
| abstracted away and the network begins to understand
| meaning directly. Of course, you 'd still have
| differentiation by different perceptions and focus on
| different parts of reality, but I see no reason why that
| should not likewise abstract out at even higher levels.
| Ultimately, the whole point of language is that it's
| about reality, and there is only one reality no matter
| how many different cultures you feed in. The network
| isn't doing magic - it's just doing exactly the same
| thing that we're doing with language. The reason why
| language is useful is the same reason it's learnable.
| mjburgess wrote:
| If you want to understand how statistical AI systems
| work, then you should understand the basics of formalism.
| They are curve-fitting algorithms that approximate the
| probability distribution of a hisotrical dataset. That's
| all they are.
|
| By analogising to any animal or human mind, you aren't
| describing anythign that acutally exists. A neutral
| network algorithm isnt neural and it isnt a network. It's
| a statistical curve-fitting algorith. One oughtnt study
| trees to understand a decision tree either.
|
| This language is entirely metaphorical. There are no
| neurones in an NN, there are just summation entires in a
| matrix. This matrix comprises weights, which define the
| orientation and scale of the line pieces which form the
| curve being fit to the data.
| FeepingCreature wrote:
| They're, in particular, _multi-level_ curve-fitting
| algorithms. A formalism which, I 'm sure you are aware,
| can express any bounded computation. In particular,
| depending on how you set up the tape, neural networks are
| comfortably Turing complete.
|
| We're not talking about 2-layer perceptrons anymore here.
|
| In my opinion, you should look less at the formalism and
| more at the empirically demonstrated performance.
| mjburgess wrote:
| I'm not aware that it can express any bounded
| computation. But such is irrelevant anyway: there is no
| function from the distributional structure of text tokens
| to reasoning. There is nothing to approximate, and this
| is a system trained just to approximate some presumed
| function: our mental capacities arent patterns in glyphs.
|
| Whatever property you might imagine a transformer
| architecture to have (and it is vastly fewer than the set
| needed for general computation), the problem here is that
| it's being applied to approximate the structure of
| historical text data which isnt being generated from such
| a function.
|
| Indeed, there is no function which generates text data,
| there's a very large number of independent generating
| processes that give rise to the distribution of text. The
| phrase "the war in ukraine" acquires a different
| semantics over 2010-2030 in a radically different way
| than, "I liked that film" does.
|
| The capacities which produce distributions of text tokens
| are highly varied, complex, invovle a vast array of our
| mental processes, and so on. There's literally almost
| nothing in the distribution of text tokens that
| corresponds to any features of these processes.
|
| The structure of language is conventional, and rests on
| our familiarity with such conventions. Otherwise, let's
| end all science, everything to be known about the world
| derives from how "e" occurs alongside "lectron"
| stevenhuang wrote:
| Computability is of critical relevance.
|
| Our mental capacities _can_ be patterns.
| https://en.m.wikipedia.org/wiki/Predictive_coding
|
| > There's literally almost nothing in the distribution of
| text tokens that corresponds to any features of these
| processes.
|
| There's nothing in evolutionary fitness that necessitates
| intelligence or reasoning ability either. Yet here we
| are.
|
| Sorry but if you don't see the connections then you need
| to do some reading on theory of mind, cognition,
| information theory, physics, philosophy. All of the
| fundamental basis are met to allow reasoning to emerge in
| LLMs.
|
| It is clear now where your confusion lies, and why you
| are led to believe so strongly that LLMs cannot reason:
| it is because you are an ML practitioner you overweight
| your expertise yet you don't know what you don't know,
| and have foundational gaps in your knowledge. You lack
| the context in these other fields. If you had them, your
| position should be closer to agnostic than this strong
| belief of yours that LLMs in their current form cannot
| reason.
| mjburgess wrote:
| lol, well when I've written my PhD on those areas we can
| return to whether I'm an expert on them or not.
|
| One does not form "connections" between them as in some
| wide-eye conspiracy theorist.. "predictive coding" has
| little to do with "prediction" in the ML sense. The
| latter concerned with making a quantitive estimate of
| some variable by summarising historical data.
|
| What we are doing when we revise a "mental model" is done
| by counter-factual simulation of possible future states.
| Statistical AI models conditional probability structures,
| and computes predictions as an expectation over weighted
| summarised historical data. This is not a means of
| performing counter-factual simulation.
|
| One trivial, sadly empirical, way to see this is to note
| that each marginal token generated is of constant time
| and energy use. Yet trivially, reasoning and a variety of
| other mental capacities should require abitarily
| different time to run. Eg., simulating a complex scenario
| is necessarily more intensive than a simple one, and so
| on.
|
| Yet I am quite annoyed that we need such dumb
| observations to make this point. It speaks of a profound
| ignorance of "theory of mind, cognition, information
| theory, physics, philosophy" and especially neurology and
| zoology which are your most significant missing terms.
|
| It is no real mystery what the structure of various
| mental capacities involves; nor any mystery what
| s(Ws(Ws(WX+B)+B...)...)...) computes. Even involving
| anything beyond trivial applied statistics and trivial
| results in science shouldnt be required here. This stuff
| is very obvious.
| FeepingCreature wrote:
| > Indeed, there is no function which generates text data,
| there's a very large number of independent generating
| processes that give rise to the distribution of text. The
| phrase "the war in ukraine" acquires a different
| semantics over 2010-2030 in a radically different way
| than, "I liked that film" does.
|
| But those semantics _are_ revealed in the greater context
| of the phrase! That 's why it's so important that
| transformers can attend to large context ranges; that's
| what lets them learn the greater semantic patterns to
| begin with. And at the limit, at a scale greater than
| phrases, I simply reject the idea that the same article -
| the same _book_ - can have totally different meanings
| depending on context. Language isn 't just shaped by
| context, it _shapes_ context itself. Because language
| cannot be considered without context, it reveals
| information about that context, and in fact any
| compression of language ultimately requires modelling the
| person and even society that produced it. That 's what
| the network learns.
|
| If your words don't have meaning beyond themselves, what
| are you even talking about?
| mjburgess wrote:
| > If your words don't have meaning beyond themselves,
| what are you even talking about?
|
| ...err... of course? That's the whole point.
|
| "Context" here isnt other _words_. It 's the _world_. The
| meaning of words is the world.
|
| When I say, "I like what you're wearing" i'm not
| summarising a history of prior texts; it has nothnig to
| do with any statistical operation over historical
| documents. It has entirely to do with what you're
| wearing.
|
| Langauge use is a side-effect of being embedded in
| reality, directly attentive to it, and so on. Words are
| mere symptoms of how we are situated in the world.
|
| LLMs merely replay these back to us. They are not in the
| world. They cannot, in principle, ever _mean_ , "I like
| what you're wearing"
| FeepingCreature wrote:
| I'd say exactly the opposite. It's because human speech
| is about the world, and LLM speech is about human speech,
| that LLM speech is about the world, by transitivity. "The
| ball fell to the <floor>", the LLM predicts, ultimately,
| not because of any feature of the human brain, or any
| feature of the English language, but because of
| _gravity._ That the causal arrow passes through a human
| brain does not make this any less true! Because human
| speech is inextricable from physical reality, language
| models likewise learn to model reality. They learn this
| poorly, incrementally, making missteps on the way -
| granted! But that they learn it at all shows that there
| 's more going on than statistical modelling - or that
| statistical modelling _of language_ is more than it
| sounds like.
| canjobear wrote:
| Languages do in fact have similar distributional
| structure, so that it is possible to learn how to
| translate words without supervision:
| https://arxiv.org/abs/2203.04863
|
| In principle languages are arbitrary, but in practice
| they're describing the same world and the same concepts
| end up being useful.
| mjburgess wrote:
| Those are on embedding vectors, not on words. They use
| embeddings created from translated sources, eg.,
| wikipedia articles.
|
| Yes, if you construct an embedding vector on texts-A, and
| another on texts-B where (A, B) are translations of each
| other, then an "unsupervised" algorithm really will give
| you the dizzying heights of a little above coin-flip
| accuracy on highly engineered self-selected benchmarks.
|
| They evaluate by by looking at the most in-use words in
| each vocab.. so you take the most in-use words on
| translations of Wikipedia, whose frequency is decided by
| the need of translation.. and then you use that to
| evaluate.
|
| It is blindingly obvious that the structure and frequency
| of heirglphys on tombs, Chinese glyphs in poetry, and
| latin in medieval liturgical literature are not
| distributed by Reality.. written in this order by God so
| that the Langauge of Reality is what places "d" alongside
| "oor". We already know this to be the case. The
| assumption of its opposite is rank pseudoscience.
| FeepingCreature wrote:
| > It is blindingly obvious that the structure and
| frequency of heirglphys on tombs, Chinese glyphs in
| poetry, and latin in medieval liturgical literature are
| not distributed by Reality.. written in this order by God
| so that the Langauge of Reality is what places "d"
| alongside "oor". We already know this to be the case. The
| assumption of its opposite is rank pseudoscience.
|
| I mean, on the first level, of course they're _entirely_
| determined by reality in the sense that the human brain
| is a real, physical object. But also on a second level
| they 're _still_ entirely determined by reality because
| the shape of the human _mind_ is also entirely determined
| by reality. What is the mind _for_ except reflecting
| reality? What is language for except communicating it?
| Sure that reality is warped, filtered, reduced and
| biased, but the data is still in there. That 's in large
| part why LLMs need such ludicrously large training runs.
|
| I don't think letter frequency is objectively determined,
| but we know for a fact (many studies!) that the features
| that large language models learn are far, far above the
| scale of letters. Even arguing about phrases isn't
| engaging with the current state of the art.
|
| We're not talking about Markov chains here.
| empath75 wrote:
| Have you ever actually bothered testing your assumption
| that chatgpt can't reason. And I don't mean that it fails
| to reason properly about certain questions, that's
| trivially easy to show, but is less interesting than many
| people think it is, because humans can't reason properly
| about many questions (see, for example, the Monty Hall
| problem). Test your assumption that it can never reason
| properly about any question not in its data set. It's way
| easier, IME, to find examples where it does give the
| correct answer to novel problems than it is to trip it up
| with a difficult problem.
| mjburgess wrote:
| It's data set is, approximately, everything ever written.
| We have no access to it. And this is widely studied. You
| can find trivial mistakes in apparent reasoning.
|
| There are two hypotheses: H1, the structure of a response
| from any given prompt is computed using distributional
| properties of historical data; H2: the response is
| computed via deduction from premises to conclusions of
| agent employing the semantics of the terms, their logical
| connections, and connections of relevance.
|
| In many cases a prompt/reply will confirm both
| hypotheses, hence confirmation bias and why we dont
| bother "confirming" any hypothesis. Rather to choose
| betweeen them, if you wanted to use data, you just find
| cases where reasoning fails in such a way that H1 is the
| more plausible answer. Such cases are easy to find, and
| across the literature.
|
| This whole thing is pointless however, because it's
| blindingly obvious from what a statistical AI algorithm
| does, which is empirical function fiting on historical
| datasets to approximate conditional probability
| distributions. This process is extremely well-understood,
| and we know _necessarily_ that it is just approximating a
| historical distribution.
| stevenhuang wrote:
| > This process is extremely well-understood, and we know
| necessarily that it is just approximating a historical
| distribution.
|
| Prove that this precludes formation of mind. Prove that
| human minds do not work this way.
|
| Until such premises are established, your argument is not
| sound. This leaves open the possibility what we're seeing
| is in fact a degree of reasoning ability in LLMs.
| mjburgess wrote:
| As I said in my other comment. Statistical models
| generate output with constant time/energy. So for _any_
| input y, and _any_ output x, the computation is constant
| time. eg., generating _any_ 100 tokens of output takes
| the same time as any other.
|
| If you reflect on that carefully, you'll note the only
| way this is possible is if the algorithm does not use any
| semantic features of the input. Indeed, if it has no
| relevant capacities at all.
|
| Consider e.g., the prompt, "imagine a world where..., and
| then infer..., and then what would be... ?" for various
| "..." we can choose, the output ought take arbitrarily
| longer to compute; but its constant.
|
| For any word: imagine, reason, suppose, simulate,
| believe, remember... denoting any alleged mental
| operation, we can trivially construct cases where
| actually performing this operation would take more or
| less time.
|
| "Given premises A, B, C, D,...; infer conclusion..." has
| a prompt reply which is constant in time regardless of
| how many premises we input. "Recall memories A, B,C,.."
| likewise.
|
| The only way that answering "yes" to a question, say,
| _regardless of what that question is_ , taking the same
| time/energy/etc. to answer.. would be if no semantic
| feature of that question was being used in its answering.
|
| This is an LLM, it's all statistical AI: the number of
| operations performed per prediction is constant _for all
| predictions_. Almost no alleged capacities are consistent
| with this fact.
| FeepingCreature wrote:
| The human brain computes precisely the same amount of
| information every millisecond. That is, the computational
| power of a single neuron is necessarily capped by its
| switching speed.
|
| This proves humans cannot think.
| stevenhuang wrote:
| The guy you're responding to can be seen routinely
| interjecting with these kinds of unfounded assertions
| that LLMs can't reason everywhere LLMs are discussed. He
| can't help himself it seems. You'd think he'd engage with
| the responses he receives, maybe down-weight his
| convictions through exposure to reasonable arguments, but
| nope.
|
| An LLM would have the wherewithal to consider the
| possibility they're wrong, or to be agnostic in their
| beliefs until we ourselves understand better what it
| means to reason.
| empath75 wrote:
| Not being able to solve a problem "in general" does not mean
| that it can't solve specific and useful instances of the
| problem. There are lots of SAT solvers that can't solve SAT
| problems in general (in any reasonable amount of time), but
| nevertheless can solve many useful categories of SAT problems.
| canjobear wrote:
| People at Microsoft Research certainly do. But the bet is that,
| with scale, these limitations won't matter in practice. For
| example Transformers can't recognize well-nested brackets (like
| {[()]}) to infinite depth; the depth that Transformers can
| recognize is limited by how many self-attention layers they
| have. But in practice, you rarely need much depth.
| toxik wrote:
| Notably, humans also cannot track this to infinite depth.
| Somehow we know this and reach for algorithmic solutions
| (like pen and paper).
| logicchains wrote:
| Interestingly LLMs also do better with pen and paper
| reasoning: https://arxiv.org/abs/2310.07923 .
| gaudat wrote:
| Another name hijacked by AI/ML... I was hoping to see some
| control thoery...
| throwaway42668 wrote:
| This is what they actually mean by AI will be in everything.
| Just a relentless campaign of appropriating names that used to
| have meanings.
| nico wrote:
| Interesting. This is how I sometimes feel about physics
|
| The field completely appropriated and redefined so many terms
| of common language, that now it's hard to talk in plain
| language with someone formally trained in physics about
| physical phenomena
|
| For example, everyone has some sort of intuitive idea about
| what Energy is. But if you use that word with a physicist,
| watch out, for them it means something super specific within
| the context of assumptions and mathematical models and they
| will assume you don't know what you are talking about because
| you are not using their definitions from their models
|
| Same thing happens with infinite in math
| aerospace_guy wrote:
| This has been happening for years now. Just adding "AI" or
| "LLM" gets the views and $$ these days.
| szvsw wrote:
| I mean, if you like state space models, then you should read
| the paper on Mamba if you haven't already! Because it quite
| literally uses state spaces... and you will probably think it's
| a really cool application of state spaces!
|
| Apologies if you know the following already, but maybe others
| reading your comment feeling similarly will not be familiar and
| might be interested.
|
| At least intuitively, I like to motivate it this way- pick your
| favorite simple state space problem. Say a coupled spring
| system of two masses, maybe with some driving forces. Set it
| up. Perturb it. Make a bunch of observations at various points
| in time. Now use your observations to figure out the state
| space matrices.
|
| There's fundamentally not really anything different (in my
| opinion) with using Mamba (or another state space model) as a
| function approximation of whatever phenomenon you are
| interested in. Okay Mamba has more moving parts, but the core
| idea is the same: you are saying that on some level, a state
| space is an appropriate prior for approximation of the dynamics
| of the quantities of interest. It turns out being pretty
| remarkable the number of things this can work out quite well
| for. For instance, I use it to model the 15-min interval data
| for heating, cooling, and electricity usage of a whole building
| given 15 min weather data, occupancy schedules, and
| descriptions of the building characteristics (eg building
| envelope construction, equipment types, number of occupants,
| etc).
| duped wrote:
| It's not hijacked, the formulation is the same. For any layer
| there is a state-space formulation h = Ah + Bx
| y = Ch + Dx
|
| where x is the input, y is the output, and h is the state. They
| use "h" instead of "s" for the state variables because they're
| called "hidden states" in the literature. edit: it is obnoxious
| they've flipped the convention for A/B/C/D which is the one
| thing controls people agree on (we can't even agree on the
| signs and naming of transfer function coefficients!).
|
| Where this diverges from dynamical systems/controls is that
| they're proving that when x/h/y are represented with finite
| precision numbers, the model is limited in the problems it can
| represent (no surprise from controls perspective), and they
| prove this by using an equivalence to the state-space formulae
| that's consistent with evaluating it on massively parallel
| hardware.
|
| The classical controls theory is not super applicable here,
| because what controls people care about (is the system stable,
| is its rise/fall time in bounds, what about overshoot, etc) is
| _not_ what ML researchers care about (what classes of AI
| problems can be modeled and evaluated using this computational
| architecture).
| 3abiton wrote:
| To be fair state-space models come originally from physics
| briandw wrote:
| The early research in neural networks was hampered by proofs that
| perceptrons could never solve certain functions, like XOR. DNNs
| could have been developed much sooner otherwise. I view these
| proof papers with some skepticism, since they can be
| unnecessarily dismissive of good ideas.
| nickpsecurity wrote:
| I've only read the abstract. It says that they have experiments
| to back their claims. So, that's a proof claim with
| experimental data. It's in a field where most learning happens
| by experimental exploration, too.
|
| I don't think it will hold us back. If anything, it's very
| exciting to see how many people in the ML field are challenging
| the status quo from many, different angles.
| logicchains wrote:
| >The early research in neural networks was hampered by proofs
| that perceptrons could never solve certain functions, like XOR.
| DNNs could have been developed much sooner otherwise.
|
| These proofs still hold; pure MLPs (without a modern activation
| function) aren't very useful, both in theory and practice. What
| made them useful was the realisation that combining them with a
| proper activation function makes them much more useful, both
| theoretically and practically; this discovery took time.
| nyrikki wrote:
| How did papers that demonstrated single layer neural networks
| can't do XOR, but multi layer neural networks can, hamper
| development?
|
| XOR is simply not linearly serperatable, requiring an MLP or
| kernel trick still holds.
|
| It is a similar reason that attention works for majority gates
| but not parity gates in the general case.
|
| Acknowledging that reality resulted in new developments, but is
| still a limitation.
|
| Perceptrons are binary classifiers.
| empath75 wrote:
| There's nothing wrong with this paper and it's important to
| understand exactly what LLMs (and related technologies) are
| capable of and what they aren't, but I do have a problem with the
| general argument that <ai technology> can't do <some task> and so
| that means it's useless and won't lead to AGI. I feel like a lot
| of people think that if a single neural network architecture
| can't achieve something on it's own that building a system that
| supplements the AI with other tools is somehow _cheating_.
| General AI if it ever happens is going to very likely be a system
| of specialized components working together, and there's not going
| to be a single piece of it that you can point to and say "that is
| where the intelligence lives" -- it's sort of liking digging
| through the brain looking for the homunculus that's piloting the
| ship. If an LLM can't track state, hook it up to something that
| can. If it doesn't have long term memory and retrieval, hook it
| up to something that does.
| andoando wrote:
| I think a way to connect these different systems as a
| homogenous system is the difficult part.
| szvsw wrote:
| Very well said. It absolutely drives me crazy - the focus on
| the model architecture for autoregressive text generation as
| the "key" (or blocker) to AGI seems so misguided as a debate.
| It seems clear to me (and maybe I'm wrong!) that a much larger
| system of modules interacting is essential (actuators, sensors,
| short term and long term memory, working memory, etc), and I
| completely fail to understand why it feels like the debate
| revolves around neural network architectures as opposed to the
| larger system architecture for agency. The language and
| decision making component is obviously extremely important and
| necessary, but it feels to me like it will always just be a
| necessary but not sufficient component for AGI.
|
| The fact that we have already obviously developed a technology
| which has an excellent grasp of language, can select between
| tools, perform knowledge retrieval from external databases,
| perform some form of simplistic reasoning and planning... it
| really feels (again, only intuitively!) like everything outside
| the perimeter of LLMs is where the actual frontier of AGI will
| be.
|
| It feels like with enough sensorimotor affordances and
| mechanisms for saving and accessing memory, all that is left is
| the (obviously complex) framework for connecting all of these
| together, with potentially many instances of each module
| working together... especially when you consider that one such
| system can potentially use human actors as tools (eg "hello
| fellow human, please complete this captcha for me, as I am low-
| vision").
|
| I guess more succinctly - I do not believe that GPT-4 et al are
| remotely close to AGI on their own, _but_ I do feel (again,
| this is just "vibes," and obviously people smarter than me
| disagree!) that GPT-4 could be a component combined with lots
| of other tech we already have to essentially achieve AGI
| already. Perhaps what I am describing would just be a pale
| imitation of what others mean by AGI and be reductively called
| another mechanical Turk. I am probably the misguided one, but
| nonetheless, I can't escape the feeling that "the one model to
| rule them all" is a red herring vis-a-vis a complex network
| (ha) of modules interacting together to achieve meaningful
| agency. Maybe it's just that I am more interested in
| "meaningful agency" as a guiding principle over "AGI."
| genevra wrote:
| Exactly, we're already adding basic memory functions to GPT
| models
|
| Text generation can only get so much better but the other
| modules required to simulate intelligence have tons of room
| for improvement
| logicchains wrote:
| >the general argument that <ai technology> can't do <some task>
| and so that means it's useless and won't lead to AGI
|
| The paper isn't making that argument. The point it makes is
| that parallelisable SSMs are theoretically no more powerful
| than transformers, contrary to some people's assumption that
| they'd be theoretically equivalent to RNNs and hence more
| efficient at certain kinds of problems.
| Lerc wrote:
| Reading that abstract, I'm not sure what is being claimed here.
| They say "the "state" in common SSMs is an illusion:" Is this a
| claim that the activations within a state space model contain no
| state or that it cannot contain an arbitrary state. The former
| seems trivially false and the latter seems completely
| uncontroversial.
| logicchains wrote:
| If you read the rest of the paper it's clear what they mean.
| SSMs are theoretically no more powerful than transformers,
| meaning without chain of thought they can't efficiently solve
| state-tracking problems, unlike RNNs:
| https://arxiv.org/abs/2207.00729 .
| opprobium wrote:
| Not just efficiently, can't solve.
| logicchains wrote:
| They can solve it if you keep adding layers to the
| transformer, it's just not efficient; you'd need
| exponentially more layers than a similarly sized RNN.
| yobbo wrote:
| The difference to RNNs is that the state in SSMs is a linear
| combination of previous inputs, which makes it possible to
| parallelise training in various ways.
|
| Gates and non-linearities in RNNs allow the state to be "any"
| function of previous inputs.
| duped wrote:
| I disagree with the editorialized abstract. State is not an
| illusion. There is a limit to what stateful LTI systems are
| capable of modeling. It is very interesting that they've been
| able to prove what that limit is in the context of computation.
|
| From my reading, "non gated SSMs" are LTI systems. They're linear
| (the paper ignores the activation function) and the "does not
| depend on input" criteria is the controls way of saying "time
| invariant" (replace time with the domain of the input vector). It
| is not surprising that a system that cannot adapt to changing
| input conditions (by construction) is fundamentally limited to
| the problems it can model. This is why adaptive filters are
| studied - LTI systems are useful computationally, but limited
| because they cannot adapt. Relaxing the "TI" constraint greatly
| expands the domains of problems that can be solved.
|
| When the feedback matrix is diagonal you have an FIR system.
| While "finite impulse response" has as specific mathematical
| definition, conceptually it means the state at time n depends
| solely on past inputs and states n - N where N is the size of the
| state vector. So of course, if the feedback matrix is diagonal,
| the system is limited in the kinds of stateful problems it can
| handle.
|
| What this paper is missing is that connection to the limits of
| LTI and connection to TC0, which is very interesting indeed.
|
| In other words, the state of the network is still infinitely long
| and definitely not an illusion. However if the problem _requires_
| the network to adapt to its input then a non-gated SSM is not
| going to be sufficient. That 's an interesting research space.
___________________________________________________________________
(page generated 2024-06-07 23:01 UTC)