[HN Gopher] Normalizing Flows Are Capable Generative Models
___________________________________________________________________
Normalizing Flows Are Capable Generative Models
Author : danboarder
Score : 162 points
Date : 2025-06-27 20:50 UTC (1 days ago)
(HTM) web link (machinelearning.apple.com)
(TXT) w3m dump (machinelearning.apple.com)
| tiahura wrote:
| https://github.com/bayesiains/nflows
| imoverclocked wrote:
| It's pretty great that despite having large data centers capable
| of doing this kind of computation, Apple continues to make things
| work locally. I think there is a lot of value in being able to
| hold the entirety of a product in hand.
| xnx wrote:
| Google has a family of local models too!
| https://ai.google.dev/gemma/docs
| ivape wrote:
| Gemma and Llama can't be bundled commercially, which sucks
| because they make two of the leading small llms. Qwen3 might
| be the last one with an Apache license.
| nolist_policy wrote:
| You can bundle and use Gemma commercially[1].
|
| [1] https://ai.google.dev/gemma/terms
| ivape wrote:
| I'll have to read that, thanks.
| coliveira wrote:
| It's very convenient for Apple to do this: less expenses on
| costly AI chips, and more excuses to ask customers to buy their
| latest hardware.
| nine_k wrote:
| Users have to pay for the compute somehow. Maybe by paying
| for models run in datacenters. Maybe paying for hardware
| that's capable enough to run models locally.
| Bootvis wrote:
| I can upgrade to a bigger LLM I use through an API with one
| click. If it runs on my device device I need to buy a new
| phone.
| nine_k wrote:
| I* can run the model on my device, no matter if I have an
| internet connection, nor if I have a permission from
| whoever controls the datacenter. I can run the model
| against highly private data while being certain that the
| private data never leaves my device.
|
| It's a different set of trade-offs.
|
| * Theoretically; I don't own an iPhone.
| eru wrote:
| Well, unless it's open source, you can't be so certain.
| But more certain than when processing in the cloud,
| that's true.
| lostlogin wrote:
| But also: if Apple's way works, it's incredibly wasteful.
|
| Server side means shared resources, shared upgrades and
| shared costs. The privacy aspect matters, but at what cost?
| shakna wrote:
| Server side means an excuse to not improve model handling
| everywhere you can, and increasing global power usage by
| noticable percentage point, at a time when we're
| approaching "point of no return" with burning out the
| only planet we can live on.
|
| The cost, so far, is greater.
| hu3 wrote:
| > Server side means an excuse to not improve model
| handling everywhere you can...
|
| How so if efficiency is key for datacenters to be
| competitive? If anything it's the other way around.
| coliveira wrote:
| The previous commenter is right in that server-side
| companies have little incentive to do less, especially
| when they're backed by investors money. Client-side AI
| will be bound by device capabilities and customer
| investment in new devices.
| shakna wrote:
| Or, instead of improving efficiency, they go ahead and
| just deploy more generators [0]. Stop gap measures are
| cheaper.
|
| [0] https://interestingengineering.com/innovation/elon-
| musk-xai-...
| eru wrote:
| Well, if it were easier to build power stations, they'd
| do so.
| thfuran wrote:
| More like squinting to see if it's still visible in the
| rear view mirror.
| eru wrote:
| How does running AI workloads on end user devices
| magically make them use less energy?
| gessha wrote:
| With the wave of enshitiffication that's surrounding
| everything tech or tech-adjacent, the privacy cost is
| pretty~ high.
| zamadatix wrote:
| If iPhones were the efficient/smart way to pay for compute
| then Apple's datacenter would be built with those instead
| of servers.
| v5v3 wrote:
| With no company having a clear lead in everyday ai for the non
| technical mainstream user, there is only going to be a race to
| the bottom for subscription and API pricing.
|
| Local doesn't cost the company anything, and increases the
| minimum hardware customers need to buy.
| eru wrote:
| > Local doesn't cost the company anything, [...]
|
| Not completely true: those models are harder to develop. The
| logistics are a hassle.
| ivape wrote:
| It takes about a $400 dollar graphics card to comfortably run
| something like a 3b-8b model. Comfortable as in fast inference,
| good sized context. 3b-5b models are what devices can somewhat
| fit. That means for us to get good running local models, we'd
| have to shrink one of those $400 dollar graphics cards down to
| a phone.
|
| I don't see this happening in the next 5 years.
|
| The Mac mini being shrunk down to phone size is probably the
| better bet. We'd have to bring down the power consumption
| requirements too by a lot. Edge hardware is a ways off.
| nolist_policy wrote:
| Gemma 3n E4B runs at 35tk/s prompt processing and 7-8 tk/s
| decode on my last last last gen flagship Android.
| ivape wrote:
| I doubt this. What kind of t/s are you getting once your
| context window is reasonably saturated? Probably slows down
| to a crawl making it not good enough yet (the hardware that
| is).
| MBCook wrote:
| I wonder if it's noticeably faster or slower than the common way
| on the same set of hardware.
| yorwba wrote:
| Figure 10 in https://arxiv.org/pdf/2506.06276 has a speed
| comparison. You need fairly large batch sizes for this method
| to come out ahead. The issue is that the architecture is very
| sequential, so you need to be generating several images at the
| same time to make good use of GPU parallelism.
| b0a04gl wrote:
| flows make sense here not just for size but cuz they're fully
| invertible and deterministic. imagine running same gen on 3
| iphones, same output. means apple can kinda ensure same input
| gives same output across devices, chips, runs. no weird variance
| or sampling noise. good for caching, testing, user trust all
| that. fits apple's whole determinism dna and more of predictable
| gen at scale
| yorwba wrote:
| Normalizing flows generate samples by starting from Gaussian
| noise and passing it through a series of invertible
| transformations. Diffusion models generate samples by starting
| from Gaussian noise and running it through an inverse diffusion
| process.
|
| To get deterministic results, you fix the seed for your
| pseudorandom number generator and make sure not to execute any
| operations that produce different results on different
| hardware. There's no difference between the approaches in that
| respect.
| GenerocUsername wrote:
| Agree. I am a image gen laymen, but when I was running stable
| diffusion in 2022 it seemed like I could get the same image
| if I used the same seed and parameters. Seemed easy to get
| same image when you have full control of the inputs. The
| randomness is a choice
| lnyan wrote:
| normalizing flow might be unpopular but definitely not a
| forgotten technique
| layer8 wrote:
| Earlier discussion: https://news.ycombinator.com/item?id=44358535
| tomhow wrote:
| Thanks. I looked at that thread and it wasn't great, with most
| of the comments being meta-commentary related to the article
| and Apple's AI progress rather than the actual research paper.
|
| I've decided to keep this thread on the front page, move the
| on-topic comments from that other thread to this one, and leave
| the rest of it in the past.
| jc4p wrote:
| i've been trying to keep up with this field (image generation) so
| here's quick notes I took:
|
| Claude's Summary: "Normalizing flows aren't dead, they just
| needed modern techniques"
|
| My Summary: "Transformers aren't just for text"
|
| 1. SOTA model for likelihood on ImageNet 64x64, first ever sub
| 3.2 (Bits Per Dimension) prev was 2.99 by a hybrid diffusion
| model
|
| 2. Autoregressive (transformers) approach, right now diffusion is
| the most popular in this space (it's much faster but a diff
| approach)
|
| tl;dr of autoregressive vs diffusion (there's also other
| approaches)
|
| Autoregression: step based, generate a little then more then more
|
| Diffusion: generate a lot of noise then try to clean it up
|
| The diffusion approach that is the baseline for sota is Flow
| Matching from Meta: https://arxiv.org/abs/2210.02747 -- lots of
| fun reading material if you throw both of these into an LLM and
| ask it to summarize the approaches!
| godelski wrote:
| You have a few minor errors and I hope I can help out.
| > Diffusion: generate a lot of noise then try to clean it up
|
| You could say this about Flows too. The history of them is
| shared with diffusion and goes back to the Whitening Transform.
| Flows work by a coordinate transform so we have an isomorphism
| where diffusion works through, for easier understanding, a
| hierarchical mixture of gaussians. Which is a lossy process
| (more confusing when we get into latent diffusion models, which
| are the primary type used). The goal of a Normalizing Flow is
| to turn your sampling distribution, which you don't have an
| explicit representation of, into a probability distribution
| (typically Normal Noise/Gaussian). So in effect, there are a
| lot of similarities here. I'd highly suggest learning about
| Flows if you want to better understand Diffusion Models.
| > The diffusion approach that is the baseline for sota is Flow
| Matching from Meta
|
| To be clear, Flow Matching is a Normalizing Flow. Specifically,
| it is a Continuous and Conditional Normalizing Flow. If you
| want to get into the nitty gritty, Ricky has a really good
| tutorial on the stuff[0]
|
| [0] https://arxiv.org/abs/2412.06264
| jc4p wrote:
| thank you so much!!! i should've put that final sentence in
| my post!
| godelski wrote:
| Happy to help and if you have any questions just ask, this
| is my jam
| godelski wrote:
| As far as I'm aware, this is the largest Normalizing Flow that
| exists, and I think they undermined their work by not mentioning
| this...
|
| Their ImageNet model (4_1024_8_8_0.05[0]) is ~820M while AFHQ is
| ~472M. Prior to that there is DenseFlow[1] and MaCow[2], which
| are both <200M parameters. For more comparison, that makes
| DenseFlow and MaCow smaller than iDDPM[3] (270M params) and
| ADM[4] (553M for 256 unconditional). And now, it isn't uncommon
| for modern diffusion models to have several billion
| parameters![5] (from this we get some numbers on ImageNet-256,
| which allows a direct comparison, making TarFlow closer to
| MaskDiT/2 and much smaller than SimpleDiffusion and VDM++, both
| of which are in billions. But note that this is 128 vs 256!)
|
| Essentially, the argument here is that you can scale (Composable)
| Normalizing Flows just as well as diffusion models. There's a lot
| of extra benefits you get too in the latent space, but that's a
| much longer discussion. Honestly, the TarFlow method is simple
| and there's probably a lot of improvements that can be made. But
| don't take that as a knock on this paper! I actually really
| appreciated it and it really set out to show what they tried to
| show. The real thing is just no one trained flows at this scale
| before and this really needs to be highlighted.
|
| The tldr: people have really just overlooked different model
| architectures
|
| [0] Used a third party reproduction so might be different but
| their AFHQ-256 model matches at 472M params
| https://github.com/encoreus/GS-Jacobi_for_TarFlow
|
| [1] https://arxiv.org/abs/2106.04627
|
| [2] https://arxiv.org/abs/1902.04208
|
| [3] https://arxiv.org/abs/2102.09672
|
| [4] https://arxiv.org/abs/2105.05233
|
| [5] https://arxiv.org/abs/2401.11605
|
| [Side note] Hey, if the TarFlow team is hiring, I'd love to work
| with you guys
| yorwba wrote:
| In the follow-up, they go all the way to 3.8 billion
| parameters: https://machinelearning.apple.com/research/starflow
| godelski wrote:
| Thanks! Idk how I missed that one. Really glad they put that
| extra information in
| kleskling wrote:
| I've been working on a JAX implementation for my own projects.
| I've implemented everything in the paper except guidance.
|
| See here: https://github.com/homerjed/transformer_flow
|
| I'm happy to see the return of normalising flows - exact
| likelihood models have many benefits. I found the model needed
| soft-clipping on some operations to ensure numerical stability.
|
| I wonder if adding transformers can be done for the GLOW
| algorithm since attention and 1x1 convolutions could be made to
| do the same operation.
___________________________________________________________________
(page generated 2025-06-28 23:01 UTC)