[HN Gopher] General Theory of Neural Networks
___________________________________________________________________
General Theory of Neural Networks
Author : rdlecler1
Score : 70 points
Date : 2024-07-11 14:34 UTC (3 days ago)
(HTM) web link (robleclerc.substack.com)
(TXT) w3m dump (robleclerc.substack.com)
| rdlecler1 wrote:
| Despite vast implementation constraints spanning diverse
| biological systems, a clear pattern emerges the repeated and
| recursive evolution of Universal Activation Networks (UANs).
| These networks consist of nodes (Universal Activators) that
| integrate weighted inputs from other units or environmental
| interactions and activate at a threshold, resulting in an action
| or an intentional broadcast. Minimally, Universal Activator
| Networks include gene regulatory networks, cell networks, neural
| networks, cooperative social networks, and sufficiently advanced
| artificial neural networks.
|
| Evolvability and generative open-endedness define Universal
| Activation Networks, setting them apart from other dynamic
| networks, complex systems or replicators. Evolvability implies
| robustness and plasticity in both structure and function,
| differentiable performance, inheritable replication, and
| selective mechanisms. They evolve, they learn, they adapt, they
| get better and their open-enedness lies in their capacity to form
| higher-order networks subject to a new level of selection.
| RaftPeople wrote:
| Thoughts:
|
| > _2-UANs operate according to either computational principles
| or magic._
|
| Given that quantum effects do exist, does this mean that the
| result of quantum activity is still just another physical input
| into the UAN and does not change the analysis of what the UAN
| computes? It seems difficult to think that what a UAN computes
| is not impacted by those lower level details (meaning
| specifically quantum effects, I'm not thinking of just
| alternate implementations).
|
| > _4-A UANs critical topology, and its implied gating logic,
| dictate its function, not the implementation details._
|
| Dynamic/short term networks in brain:
|
| Neurons in the brain are dynamically inhibited+excited due to
| various factors including brain waves, which seems like they
| are dynamically shifting between different networks on the fly.
| I assume when you say topology, you're not really thinking in
| terms of static physical topology, but more of the current
| logical topology that may be layered on top of the physical?
|
| Accounting for Analog:
|
| A neurons function is heavily influenced by current analog
| state, how is that accounted for in the formula for the UAN?
|
| For example, activation at the same synapse can either trigger
| an excitatory post synaptic action potential or an inhibitory
| post synaptic action potential depending on the concentration
| of permeant ions inside and outside the cell at that moment.
|
| I'm assuming a couple possible responses might be:
|
| 1-Even though our brain has analog activity that influence the
| operation of cells, there is still an equivalent UAN that does
| not make use of analog.
|
| or
|
| 2-Analog activity is just a lower level UAN (e.g. atom/molecule
| level)
|
| I don't think either of those are strong responses. The first
| triggers the question: "How do you know and how do you find
| that UAN?". The second one seems to push the problem down to
| just needing to simulate physics within +/- some error.
| AndrewKemendo wrote:
| This is another example of Markov Chains in the wild - so that's
| what he's seeing
|
| The general nn is a discrete implementation of that
|
| https://en.m.wikipedia.org/wiki/Markov_chain
| AIorNot wrote:
| What's wild to me is that Donald Hoffman is also proposing a
| similar foundation for his metaphysical theory of consciousness,
| ie that it is a fundamental property and that it exists outside
| of spacetime and leads via a markov chain of conscious agents (in
| a Network as described above)
|
| Ie everything that exists may be the result of some kind of Uber
| Network existing outside of space and time
|
| It's a wild theory but the fact that these networks keep popping
| up and recurring at level upon level when agency and intelligence
| is needed is crazy
|
| https://youtu.be/yqOVu263OSk?si=SH_LvAZSMwhWqp5Q
| mistermann wrote:
| I don't think one even needs "supernatural" explanations.
|
| 1. Consider the base hardware of each agent:
|
| http://neuropathologyblog.blogspot.com/2017/06/shannon-curra...
|
| 2. Consider that (according to science anyways) there is no
| central broadcaster of reality (it is at least plausible)
|
| 3. Consider each agent (often/usually) "knows" all of reality,
| or at least any point you query them about (for sure: all
| agents claim to know the unknowable, regularly; I have yet to
| encounter one who can stop a "powerful" invocation of #3 (or
| even try: the option seems literally unavailable), _though
| minor ones can be overridden fairly trivially_ (I can think of
| two contrasting paths of interesting consideration based on
| this detail, one of them being extremely optimistic, _and
| trivially plausible_ ))
|
| Simplified: what is known to be, _is_ (locally).
|
| 4. Consider the possibility (or assume as a premise of a
| thought experiment) that reality and the universe are not
| _exactly_ the very same thing ( "it exists outside of
| spacetime"), though it may appear that they are (see #3)
|
| Is it not fairly straightforward what is going on?
|
| A big part of the problem is that #3 is ~inevitably[1] invoked
| if such things are analyzed, screwing up the analysis, thus
| rendering the theory _necessarily_ "false" (it "is"
| false...though, it will typically not be asserted as such
| explicitly, and direct questions will be ignored/dodged).
|
| [1] which is... _weird_ (the inevitable part...like, it is as
| if consciousness is ~hardwired to disallow _certain_ inspection
| (highly predictable evasive actions are invoked in response),
| something which can _easily be tested /demonstrated_).
| cscurmudgeon wrote:
| Why is #3 obvious? How can agents know all of reality? May be
| a subset?
| mistermann wrote:
| knowledge = knowledge (true belief)
|
| "knowledge" = belief (possibly true but not necessarily,
| but _sincerely_ perceived as "true")
|
| (I'm considering this from an abstract / autistic / "That's
| _pedantic!_ [so stop doing it] " perspective, so I include
| quotation marks to note the technical distinction...in
| phenomenological analysis, perhaps they'd be left out, to
| better illustrate the local experience of reality, the true
| "is-ness" _as it is_. In normative discussions ( "anything
| that good hackers would find interesting"), these things
| are generally rather taboo.)
|
| There's lots of nuance I'm leaving out, but that's the
| general idea.
|
| A popular though terminating description for the phenomenon
| is "that's just people expressing their opinion, _everyone_
| does it, that 's what _everything_ boils down to " (which
| can make it not only not obvious, but damn near _invisible_
| )...but consider the semantic differences of that with and
| without the inclusion of the word "just". (Also: watch out
| for #3, it's recursively self-referential, and has
| substantial cloaking / shape-shifting abilities. It is
| _almost_ always and everywhere.)
|
| An alternate perspective: consider what an uneducated
| person "sees" in "reality" (aka: what "is", and "is not")
| as they go about their day, compared to highly educated (as
| opposed to _knowledgeable_ ) people _from very distinct
| disciplines_.
| AIorNot wrote:
| Can you explain #3 and 4# more clearly?
|
| in #2 you are claiming there is no objective reality or no
| 'broadcaster' of reality
|
| We must assume some things as being objective such as a
| rational universe in order to make any claims at all.
|
| -if you are saying in #3 that humans as conscious agents make
| subjective claims about reality but that those claims are in
| fact 'the reality' for that agent or person, that is a
| subjective claim. (I'm not saying that that subjective
| reality isn't true for that person)
|
| Also, Hoffman doesn't make a 'supernatural' claim per se, his
| claim is simply that reality as 'we all see it' is NOT the
| whole story, and that it is in fact only the projection of a
| vast, infinitely complex network of conscious agents that
| creates what we perceive as the material universe and time.
| He starts with the idea that consciousness as a property is
| fundamental, existing outside of space and time and that if
| you apply reasoning and mathematics that networks of agents
| acting as UANs in a sense project that material universe into
| being, with that assumption, ie that it extrapolates to our
| entire universe.
|
| I'm not sure I'm (or anyone for that matter) is really
| qualified to answer that claim..it's so big that it does
| verge on mysticism. that's why I said its such a wild idea,
| but I found the article above another interesting piece of
| evidence for Hoffman, because it talks about a general theory
| underlying such networks:
|
| whose "repeated and recursive evolution of Universal
| Activation Networks (UANs). These networks consist of nodes
| (Universal Activators) that integrate weighted inputs from
| other units or environmental interactions and activate at a
| threshold, resulting in an action or an intentional
| broadcast"
|
| ie this is very similar to Hoffmans system of Conscious
| Agents -which is an extreme theory of such networks that I
| described above
|
| https://evolutionnews.org/2023/10/eccentric-theories-of-
| cons...
| optimalsolver wrote:
| >it exists outside of spacetime
|
| So I guess this theory won't be subject to empirical testing
| any time soon?
| jungturk wrote:
| Perhaps "outside" means "resident on the boundary of", a la
| holograms and Maldecena's AdS/CFT work, and so still within
| reach of experiment?
| 29athrowaway wrote:
| In the biology there are families of neurons, each one with
| different morphologies.
| smokel wrote:
| People seem to be obsessed with finding fundamental properties in
| neural networks, but why not simply marvel at the more basic
| incredible operations of addition and multiplication, and stop
| there?
| falcor84 wrote:
| Evolutionary pressure is such that, generally speaking,
| individuals who "stop there" are less successful than ones who
| always crave more. We are all descendants of those who were
| "obsessed" with: mating, hoarding, conquering and yes, finding
| patterns and fundamental properties.
| smokel wrote:
| My point exactly, but I obviously failed to communicate that
| :)
|
| Multiplication and addition are more fundamental than neural
| networks.
| sixo wrote:
| God this grandiose prose style is insufferable. Calm down.
|
| Anyway, this doesn't even try to make the case that that equation
| is universal, only that "learning" is a general phenomena of
| living systems, which can be modeled probably in many different
| ways.
| ai4ever wrote:
| architecture astronauts let loose on unified field theories..
| talking warm and fuzzy - big bold ideas.
|
| let them, i say, until, the tide shifts to something else
| tomorrow, and a new generation of big-picture thought leaders
| take over dumping their insufferable text on the populace.
| proof_by_vibes wrote:
| The excitement of new horizons is necessary for innovation, and
| a substack article is a safe way to express that excitement.
| It's clearly understood by the choice of medium that this is
| meant to be speculation, so there aren't any significant risks
| in engaging with the text on its own terms.
| cfgauss2718 wrote:
| Agreed, I can't help but feel there is some overcompensation
| driving the style of writing. It was difficult to finish.
| cfgauss2718 wrote:
| There are some interesting parallels to ideas in this article and
| IIT. The focus on parsimony in networks, and pruning connections
| that are redundant to reveal the minimum topology (and the
| underlying computation)is reminiscent of parts of IIT: I'm
| thinking of the computation of the maximally irreducible concept
| structure via searching for a network partition which minimizes
| the integrated cause-effect information in the system. Such
| redundant connections are necessarily severed by the partition.
| zaven wrote:
| What's IIT?
| diego898 wrote:
| I'm guessing Integrated Information Theory
|
| https://en.wikipedia.org/wiki/Integrated_information_theory
| t_serpico wrote:
| "Topology is all that matters" --> bold statement, especially
| when you read the paper. The original authors were much more
| reserved in terms of their conclusions.
___________________________________________________________________
(page generated 2024-07-14 23:00 UTC)