[HN Gopher] Generative Flow Networks
___________________________________________________________________
Generative Flow Networks
Author : selimonder
Score : 127 points
Date : 2022-03-07 11:24 UTC (11 hours ago)
(HTM) web link (yoshuabengio.org)
(TXT) w3m dump (yoshuabengio.org)
| awelkie wrote:
| Having just skimmed the linked tutorial, it looks like GFlowNets
| could be useful for automated theorem proving. Anyone with more
| knowledge know if this is possible?
| andrewnc wrote:
| I don't see why not. I _think_ they can be used for anything a
| GNN / Transformer could be used for, but with better
| probabilistic qualities.
| mcbuilder wrote:
| What do you see as the application to GNN / Transformer
| tasks? Both of those networks are best suited for different
| tasks, and GFlowNetworks seem well suited to quickly
| approximating the partition function of say an Deep RL reward
| distribution, something that MCMC is ill suited for because
| of the incredibly high dimensional space. I see these all as
| different techniques, perhaps the ideas will combine at some
| point but GFlowNetworks are not an evolution of GNNs or
| Transformers.
| manux wrote:
| Yeah the "net" in GFlowNet refers to how the underlying
| state is interpreted, not to an architecture. It is a way
| to train generative models, on any kind of discrete data
| (like graphs).
|
| Source: am first author of original GFlowNet paper.
| mcbuilder wrote:
| Thanks for your comment! I originally finished my PhD in
| computational neuroscience and have been riding the deep
| learning wave for the past five years. I listened to the
| audio Machine Learning Street Talk episode on GFlowNets,
| and that was my general introduction. It wasn't until I
| looked at the image of the GFlowNet on your blog post a
| connection to actually biological plausible architectures
| became apparent. The individual firing of a neuron can be
| interpreted as a Poisson process, but with populations of
| neurons we often would approximate the firing rate of the
| entire population as a Gaussian distribution. That would
| be one patch of cortical column, but these populations
| are linked together to form larger scale functional brain
| networks. Anyway, thinking about that scale you could
| imagine the neural activity communicated between these
| patches of activity as a type of flow from one region to
| another. I think your paper raises interesting questions
| about how this inference engine in our brain might be
| doing some sort of MCMC like sampling and constructing
| different belief hypothesis.
| manux wrote:
| Yes, although they may not be ideal unless you're able to
| define a "distance" between a proof that the agent is proposing
| and whether that proof is correct (i.e. unless you're able to
| define a reward or energy function).
|
| It may be possible to infer/learn a score from existing proofs
| though. We have a paper that manages to both learn a flow and
| an energy function (the score) from data:
| https://arxiv.org/abs/2202.01361
|
| I don't know much about theorem proving though. Can some value
| be attributed to partial proofs?
| mark_l_watson wrote:
| I started reading through the linked tutorial, but I think I need
| to first back up and review reinforcement learning (I took 2
| Coursera classes in RL, and helped with one project at a previous
| job, so I am at a beginner stage).
|
| I love deep learning, a practical way to solve many problems (and
| most of my work over the last 6 years) but DL seems incomplete as
| far as explainability, causation, etc. Anyway, I just retired
| last week (mostly to take care of my wife who is having a health
| crisis) so I have plenty of time to study GFN, probably starting
| with re-taking the 2 Coursera classes on RL and then finishing
| the RL specialization.
| cs702 wrote:
| For a quick, easy-to-digest introduction to the topic, there's a
| fantastic video with explanations and an interview with Bengio
| that uses Galton peg boards as a starting point for understanding
| generative flow networks. Highly recommended if you're a
| researcher or practitioner interested in learning about this:
|
| https://www.youtube.com/watch?v=M49TMqK5uCE
| ramesh31 wrote:
| Correct me if I'm wrong, but it seems like SOTA for AI now is
| entirely in developing more and more complex sets of convolutions
| for deep learning?
| MathYouF wrote:
| And what is the SoTA for biological intelligence? While not
| exactly this, it seems to be pretty similar (even between
| members of the same species).
|
| Snark aside, with a few nuances, you're mostly correct.
| soraki_soladead wrote:
| Here's the referenced tutorial:
| https://milayb.notion.site/GFlowNet-Tutorial-919dcf0a0f0c4e9...
|
| (Some of the links appear to require signing in to Notion but
| this one doesn't.)
| manux wrote:
| Hi, first author of the original NeurIPS paper here! Someone just
| shared this HN post with me. I'll go through the comments but
| happy to try to answer questions as well.
| cs702 wrote:
| I'm still wrapping my head around this (it seems rather
| magical; can it really be just _that_?)... but I find the idea
| of "sampling from a distribution of trajectories that reach a
| terminal state whose probability is proportional to a positive
| reward function, by minimizing the difference between the flow
| coming into the trajectories and the flow coming out of them,
| which by construction must be equal" to be both beautiful and
| elegant -- like all great ideas.
|
| How did you and your coauthors come up with this? Trial and
| error? Or was there a moment of serendipitous creative insight?
|
| --
|
| To the moderators: My comment is now at the top of the page,
| but manux's comment above is more deserving of the top spot. I
| just upvoted it. Please consider pinning it at the top.
| manux wrote:
| I think the inspiration came to me from looking at SumTrees
| and from having worked on Temporal Difference learning for a
| long time. The idea of flows came to Yoshua and I from the
| realization that we wanted some kind of energy
| conservation/preservation mechanism from having multiple
| paths lead to the same state.
|
| But, yes in the moment it felt like some very serendipitous
| insight!
| cs702 wrote:
| Thank you. It's nice to hear that you had one of those
| shower/bathtub Eureka moments!
|
| > ...we wanted some kind of energy
| conservation/preservation mechanism from having multiple
| paths lead to the same state
|
| Makes sense. FWIW, to me this looks like a _Conservation
| Law_ -- as in Physics. I mean, it 's not that the flows
| "must be" conserved, but that they _are_ conserved (or go
| into sinks). Any physicists interested in AI should be all
| over this; it 's right up their alley.
| igorkraw wrote:
| Could you highlight the difference between this and training a
| permutation invariant or equivariant policy network using
| standard supervised or RL methods? Assuming I also have a way
| of having an invariant/equivariant loss function
| manux wrote:
| What the permutation invariance gets you is that the model
| doesn't arbitrarily prefer one (graph) configuration over
| another, but this seems tangential. The difference between
| this and RL is in what we do with the reward:
|
| - RL says, give me a reward and I'll give you its max.
|
| - GFlowNet says, give me a reward and I'll give you all its
| modes (via p(x) \propto R(x)).
|
| Yes you would ideally have a loss (well, a reward/energy)
| that is invariant and operates e.g. directly on the molecule
| rather than on some arbitrary ordering of the nodes.
| [deleted]
___________________________________________________________________
(page generated 2022-03-07 23:01 UTC)