[HN Gopher] How to solve computational science problems with AI:...
___________________________________________________________________
How to solve computational science problems with AI: PINNs
Author : mertkavi
Score : 32 points
Date : 2025-01-20 15:26 UTC (2 days ago)
(HTM) web link (mertkavi.com)
(TXT) w3m dump (mertkavi.com)
| patrickkidger wrote:
| FWIW - I used to do research in this area - PINNs are a terribly
| overhyped idea.
|
| See for example
| https://www.nature.com/articles/s42256-024-00897-5
|
| Classical solvers are very very good at solving PDEs. In contrast
| PINNs solve PDEs by... training a neural network. Not once, that
| can be used again later. But every single time you solve a new
| PDE!
|
| You can vary this idea to try to fix it, but it's still really
| hard to make it better than any classical method.
|
| As such the main use cases for PINNs -- they do have them! -- is
| to solve awkward stuff like high-dimensional PDEs or nonlocal
| operators or something. Here it's not that the PINNs got any
| better, it's just that all the classical solvers fall off a
| cliff.
|
| ---
|
| Importantly -- none of the above applies to stuff like neural
| differential equations or neural closure models. These are
| genuinely really cool and have wide-ranging applications.! The
| difference is that PINNs are _numerical solvers_ , whilst
| NDEs/NCMs are _techniques for modelling data_.
|
| /rant ;)
| __mmd wrote:
| I believe a lot of this hype is purely attributable to
| Karniadakis and how bad a lot of the methods in many areas of
| engineering are. The methods coming out of CRUNCH (PINNs chief
| among them) seem, if they are not just actually, more
| intelligent in comparison, since engineers are happy to take a
| solution to inverse or model selection problems by pure brute
| force as "innovative" haha.
| mnky9800n wrote:
| I love karniadakis energy. I invited him to give a talk in my
| research center ands his talk was fun and really targeted at
| physicists who understand numerical computing. He gave a good
| sell and was highly opinionated which was super welcomed. His
| main argument was that these are just other ways to arrive
| optimisation and they worked very quickly with only a bit of
| data. I am sure he would correct me greatly at this point.
| I'm not an expert on this topic but he knew the field very
| well and talked at length about the differences between one
| iterative method he developed and the method that Yao lai at
| Stanford developed after I had her work on my mind because
| she talked in an ai conference I organised in Oslo. I liked
| that he seemed to be willing to disagree with people about
| his own opinions because he simply believed he is correct.
|
| Edit: this is the Yao lai paper I'm talking about:
|
| https://www.sciencedirect.com/science/article/pii/S002199912.
| ..
| anon389r58r58 wrote:
| The general rule of thumb to go by is that whatever
| Karniadakis proposes, doesn't actually work outside of his
| benchmarks. PINNs don't really work, and _his flavor_ of
| neural operators also don't really work.
|
| PINNs have serious problems with the way the "PDE-component"
| of the loss function needs to be posed, and outside of
| throwing tons of, often Chinese, PhD students, and postdocs
| at it, they usually don't work for actual problems. Mostly
| owed to the instabilities of higher order automatic
| derivatives, at which point PINN-people begin to go through a
| cascade of alternative approaches to obtain these higher-
| order derivatives. But these are all just hacks.
| mnky9800n wrote:
| What do you do now?
| Matthyze wrote:
| Me and a friend were discussing PINNs, and he made an argument
| against them: The Bitter Lesson. PINNs are a way of incorporating
| domain knowledge into ML models. The Bitter Lesson, for those
| unaware, is a famous essay by Rich Sutton that states that the
| history of AI is full of attempts of methods guided by
| domain/expert knowledge, but that ultimately, all such methods
| were overtaken by methods that simply scaled data and/or
| computation.
|
| http://www.incompleteideas.net/IncIdeas/BitterLesson.html
|
| I would love to hear HN's take on this argument.
| add-sub-mul-div wrote:
| Computer science is currently subservient to an economic
| climate in which the only viable business is one that scales
| revenue without scaling labor. That's the bitter lesson.
| PaulHoule wrote:
| But isn't that the story of technology and civilization?
| Hunter-Gatherers produced no surplus and couldn't support a
| complex and unequal society. Early agriculture could support
| a pyramid, but not very high.
|
| It used to be almost everyone worked in agriculture, now
| about 1% does, so the others are free to _do something else_.
| Prior to the microprocessor making a computer required manual
| assembly of thousands of parts, early microprocessors
| contained thousands of parts manufactured by a small number
| of photographic and chemical steps, and the number of parts
| has grown into the billions without the number of steps
| expanding millions of times.
| rangestransform wrote:
| Improving labour productivity is good, actually
| esafak wrote:
| It works when data is cheap, and model size is not an issue.
| nicoco wrote:
| Scaling data is not always possible. It's really hard to get
| your hands on good labelled medical imaging data, for instance.
| Maybe it makes sense to try to incorporate insights from
| biology and physiological instead of hoping that the neural net
| will "get it" from seeing enough data.
| roger_ wrote:
| This articles seems like it was at least partially written by AI.
| Lots of fluff and no clear explanation of what PINNs are and how
| they work (other than the code).
| getnormality wrote:
| Agreed. Stylistic hints include the heavy use of bulleted lists
| with bold headings, and the general lack of concern with
| justifying any vaguely plausible-sounding assertion ("The PINN
| approach ensures physical consistency, efficient computation,
| and accurate generalization from limited data.")
|
| I think someone who cared about the specific content would at
| least note that linear PDEs like the heat equation often have
| closed-form solutions and/or efficient algorithms for solving
| any particular problem, so aren't likely to be usefully solved
| with PINNs.
___________________________________________________________________
(page generated 2025-01-22 23:00 UTC)