[HN Gopher] Physics Informed Neural Networks
___________________________________________________________________
Physics Informed Neural Networks
Author : nchagnet
Score : 89 points
Date : 2025-02-16 21:14 UTC (1 days ago)
(HTM) web link (nchagnet.pages.dev)
(TXT) w3m dump (nchagnet.pages.dev)
| westurner wrote:
| From "Physics-Based Deep Learning Book" (2021)
| https://news.ycombinator.com/item?id=28510010 :
|
| > _Physics-informed neural
| networks:https://en.wikipedia.org/wiki/Physics-
| informed_neural_networ... _
| Jordanpomeroy wrote:
| Very well explained to a lay person.
|
| Are PINNs the current state of the art in ML methods for solving
| PDEs? What are their limitations?
| nchagnet wrote:
| Thank you!
|
| As far as I can tell, PINNs are promising and an active
| research area, but they are also young and far from being as
| widely adopted as finite element methods (at least that's my
| experience academic environments).
|
| I do see great improvements are being made both on the
| performance level but also on the applications.
|
| One aspect I didn't discuss in the post is the use for inverse
| solution search, where you fit experimental data to your
| equation, and where your parameters and your initial conditions
| can also be trainable parameters. This has great potential to
| improve the methodology of experimental results analysis.
| cherryteastain wrote:
| > Are PINNs the current state of the art in ML methods for
| solving PDEs? What are their limitations?
|
| I guess in a way they are. They aren't new, they have been
| around since the 90s [1]. The problem with them is, you
| typically need to train them on a specific problem (boundary
| conditions, domain, equation, PDE coefficients etc). Compared
| to a traditional solver, the training is much slower, and on
| top of that the results are typically much less accurate. The
| PDE + NN community has a bit of a problem dealing with this in
| general [2], there are tons of papers that make NNs look much
| better at solving PDEs than they are compared to traditional
| solvers.
|
| [1] https://www.cs.uoi.gr/~lagaris/papers/TNN-LLF.pdf
|
| [2] https://www.nature.com/articles/s42256-024-00897-5
| hansvm wrote:
| It depends on the PDE and what you want to do with it. A PINN
| requires:
|
| 1. Some example data or other way to add boundary conditions
|
| 2. Autograd over PDE constraints
|
| 3. A training loop incorporating both of those
|
| And it produces
|
| a. An approximate, differentiable, mesh-free solution
|
| PINNs are most applicable when (1) is expensive (since that
| expense will apply more to traditional solvers, especially with
| fine meshes) and when the error in (a) is acceptable.
|
| Regarding the error, PINNs are still extremely useful in
| generating an initial state to pass to a traditional solver
| even when the error is not tolerable, so that's not _really_ a
| concern. The main consideration is how expensive a particular
| problem is to solve classically. If it's too cheap, the PINN
| will never beat it.
|
| You have a secondary consideration with (2) and (3). The
| training loop is a fixed cost which you can amortize over many
| executions, but you have to use the network enough times for
| that to actually pay off.
|
| The last point I want to bring up is that you can sometimes get
| value from the extra features in (a). Perhaps you want to use
| the PINN to figure out where your mesh should be finer, or you
| have a derived field you want to inspect. Neural net gradients
| in general tend to poorly approximate real gradients if you
| only train on the function itself, but PINNs have the gradients
| you're likely to care about baked into their definition (and
| can thus approximate them well), and they'll model those much
| more cheaply than traditional solvers will.
|
| We used them for a few things at my last job, and they were
| definitely worth it. We erred toward smaller (faster) nets with
| higher errors just to accelerate convergence with a classical
| solver.
| quanto wrote:
| I get that PINN is a less expensive approximate solution
| method. If so, how does it perform superior to many
| approximate, coarse numerical methods?
| hansvm wrote:
| 1. Those methods are coarse. The interpolation they provide
| is worse than what a PINN provides, meaning that
| equivalently performing PINNs (compard to coarse numerical
| methods) can easily and cheaply serve as better
| initializations for your finer numerical methods.
|
| 2. Go back to (1) from my previous message. For some
| intuition, fiddly solutions take a long time to optimize.
| Your only options (aside from spending more time and money)
| are tailoring the initial conditions and the algorithm for
| your particular problem. You see that a lot in, e.g., 1-3
| atom quantum chemistry, where a good choice of basis
| functions is worth several papers. A neural network allows
| you to automagically bake everything that's hard about your
| problem into the training step and amortize those hard
| calculations across many experiments. It's not superior to
| enough man-centuries of human intuition, but it's dead
| simple to deploy, and for those sorts of hard problems it
| definitely beats a single human century of effort. Once you
| have a neural network output, the problem is well
| conditioned and suitable for refinement by a classical
| solver.
|
| For a somewhat concrete example, imagine a problem where
| the space is largely uninteresting but there are a few
| tight swirls here and there. Coarse numerical methods can't
| really do anything with those. Adaptive-precision numerical
| methods can, but they're slow, and you have to re-run an
| intensive solving step for every new input. The PINN
| solution bakes everything that's hard about that into the
| neural net structure, and it solution will have
| approximately the right swirls in approximately the right
| places. If you want to refine them further, the fact that
| your solver doesn't have to dynamically handle resolution
| anymore and doesn't have to deal with any major phase
| shifts makes it much easier to iterate on via the normal
| classical methods.
| hazrmard wrote:
| Good read! I am developing PINNs at work and this certainly
| helped me recall important concepts. This post used deepxde
| library [2] to compose the PINN. Can anyone comment on how
| NVIDIA's modulus [2] compares to this? Modulus appears to be much
| more verbose and poorly documented.
|
| [1]: https://github.com/lululxvi/deepxde [2]:
| https://github.com/nvidia/modulus
| joshpopelka20 wrote:
| It's from the future. Must be really good :)
|
| Physics informed neural networks 16 Feb 2026
| SnooSux wrote:
| Around a month ago there was a PINN post[1] on here and there was
| a healthy amount of skepticism in the comments. Even in the toxic
| positivity of LinkedIn, commentors say they're overhyped when a
| ML "Influencer" posts that one GIF with a MLP and PINN fitting to
| an oscillator. I would be interested to see what they're actively
| being used for.
|
| [1] https://news.ycombinator.com/item?id=42769623
| nchagnet wrote:
| I fully agree with the comments in that post. I started
| studying them because, well, they sound really cool, but my
| first impression was definitely that this sounded like a lot of
| effort (computationally) to solve a single equation. But then I
| only tested on simple equations where traditional solvers have
| no difficulties.
|
| In my previous position, we studied the behaviour of black
| holes with exotic geometries, and we never could make our
| solvers work with the added time dependence. I would be very
| curious to see how a PINN would have fared on this (given
| enough compute time of course).
| lagrange77 wrote:
| Neural ODEs are also interesting.
| lagrange77 wrote:
| I can also recommend Steve Brunton's playlist [0] on the topic of
| physics informed machine learning, as well as the book 'Data-
| driven Science & Engineering' [1] by him and Nathan Kutz.
|
| [0]
| https://www.youtube.com/playlist?list=PLMrJAkhIeNNQ0BaKuBKY4...
|
| [1] https://databookuw.com
| quanto wrote:
| The key intuition is calculating the loss function without
| actually knowing the exact solution ("labels" in supervised
| learning parlance). Note that this is not unique to PINN: there
| are existing numerical methods that do exactly this.
|
| I used to solve PDEs for a living; and my academic background is
| in numerical solutions to PDEs before going into ML. In my
| industry and academic experience, PINN is a novel curiosity with
| perhaps niche applications that I am not as familiar with. Yes, I
| am aware of works of Bruton, Duvenaud et al (and was even in the
| same lab group with some of them). I am happy to be corrected and
| learn if PINN has found a strong application.
|
| A better introduction to this approach and its critique here:
| https://arxiv.org/pdf/2206.02016
| sundarurfriend wrote:
| I've never clearly understood the relationship/difference between
| PINNs and SciML (Scientific Machine Learning). The "how do they
| work" section here sounds pretty similar to how I've heard SciML
| described in the past.
|
| From some searcing around, it sounds like maybe SciML is a
| broader concept with PINNs being a particular implementation of
| it? Maybe SciML started with PINN related ideas, but has
| broadened beyond that over time? Would appreciate an explanation
| from someone who's actively in this field.
___________________________________________________________________
(page generated 2025-02-17 23:01 UTC)