[HN Gopher] Graph-based AI model maps the future of innovation
___________________________________________________________________
Graph-based AI model maps the future of innovation
Author : laurex
Score : 53 points
Date : 2024-11-13 18:40 UTC (4 hours ago)
(HTM) web link (news.mit.edu)
(TXT) w3m dump (news.mit.edu)
| drawnwren wrote:
| Is it just me or does this read like complete word soup?
|
| > The application could lead to the development of innovative
| sustainable building materials, biodegradable alternatives to
| plastics, wearable technology, and even biomedical devices.
|
| That a transform from materials to a 19th century Russian painter
| somehow is applicable to what just so happens to be the zeitgeist
| of materials science beggars belief.
| gran_colombia wrote:
| > One comparison revealed detailed structural parallels between
| biological materials and Beethoven's 9th Symphony, highlighting
| shared patterns of complexity through isomorphic mapping.
|
| This is not serious.
| gran_colombia wrote:
| > The resulting material integrates an innovative set of
| concepts that include a balance of chaos and order, adjustable
| porosity, mechanical strength, and complex patterned chemical
| functionalization. We uncover other isomorphisms across
| science, technology and art, revealing a nuanced ontology of
| immanence that reveal a context-dependent heterarchical
| interplay of constituents.
|
| The article itself seems generated.
| uoaei wrote:
| I encounter this take more and more, where jargony sciencey
| language is dismissed as "generated". We forget that actual
| people do write like this, and self-satisfied researchers
| especially so.
|
| More likely, this author read a bit too much Deleuze and is
| echoing that language to make the discovery feel more
| important than incidental.
| equestria wrote:
| Paste it into any AI detector (e.g.,
| https://quillbot.com/ai-content-detector). They're not
| perfect, but they're pretty good in the aggregate. This
| text is almost certainly generated by an LLM.
| sdesol wrote:
| I ran this across my AI spelling and Grammar checker at
|
| https://app.gitsense.com/?doc=4715cf6d95689&other-
| models=Cla...
|
| Note, sentences highlighted in yellow means one or more
| models disagree.
|
| The sentence that makes me think this might not be AI
| generated is
|
| "Researchers can use this framework to answer complex
| questions, find gaps in current knowledge, suggest new
| designs for materials, and predict how materials might
| behave, and link concepts that had never been connected
| before."
|
| The use of "and" before "predict how materials" was
| obviously unnecessary and got caught by both gpt-4o and
| claude 3.5 sonnet and when I questioned Llama 3.5 about
| it, it also agreed.
|
| For AI generated, it seems like there are too many
| imperfections, which makes me believe it might well be
| written by a human.
| yifanl wrote:
| If you write in a manner that gets you dismissed as a
| chatbot, then you've still failed to communicate, even if
| you physically typed the characters in the keyboard. The
| essence of communication isn't how nice the handwriting is,
| its how usefully you've conveyed the information.
| abeppu wrote:
| Skimming the actual paper ... it seems pretty bad?
|
| The thing about Beethoven's 9th and biological materials which is
| mentioned in the OP is just that, out of a very large knowledge
| graph, they found small subgraph isomorphic to a subgraph created
| from a text about the symphony. But they seem not to cover the
| fact that a sufficiently large graph with some high-level
| statistical properties would have small subgraphs isomorphic to a
| 'query' graph. Is this one good or meaningful in some way, or is
| it just an inevitable outcome of having produced such a large
| knowledge graph at the start? The reader can't really tell,
| because figure 8 which presents the two graphs has such a poor
| resolution that one cannot read any of the labels. We're just
| expected to see "oh the nodes and their degrees match so it has
| the right shape", but that doesn't really tell us that their
| system had any insight through this isomorphism-based mining
| process.
|
| For the stuff about linking art (e.g. a Kandinsky painting) with
| material design ... they used an LLM to generate a description of
| a material for DALL-E where the prompt includes information about
| the painting, and then they show the resulting image and the
| painting. But there's no measure of what a "good" material
| description is, and there certainly is no evaluation of the
| contribution of the graph-based "reasoning". In particular an
| obvious comparison would be to "Describe this painting." ->
| "Construct a prompt for DALL-E to portray a material whose
| structure has properties informed by this description of a
| painting ..." -> render.
|
| It really seems like the author threw a bunch of stuff against
| the wall and didn't even look particularly closely to see if it
| stuck.
|
| Also, the only equation in the paper is the author giving the
| definition of cosine similarity, before 2 paragraphs justifying
| its use in constructing their graph. Like, who is the intended
| audience?
|
| https://iopscience.iop.org/article/10.1088/2632-2153/ad7228#...
| refulgentis wrote:
| Thank you for taking the time to read and write this up,
| something was "off" in the quotes describing the materials that
| had me at 4 of 5 alarm bells ringing. Now I can super skim
| confidently and giggle.
|
| - real output here is text, using a finetuned Mixtral provided
| leading Qs
|
| - the initial "graph" with the silly beethoven-inspired
| material is probably hand constructed, they don't describe its
| creation process at all
|
| - later, they're constructing graphs with GPT-3.5 (!?) (they
| say rate limits, but somethings weird with the whole thing,
| they're talking about GPT-4 vision preview etc., which was
| roughly a year before the paper was released)
|
| - Whole thing reads like someone had a long leash to spend a
| year or two exploring basic consumer LLMs, finetune one LLM,
| and sorta just published whatever they got 6 months to a year
| later.
| bbor wrote:
| Great writeup, thanks! That Kadinsky quote is what set off
| alarm bells for me, as it seems like a quintessential failure
| case for laypeople understanding LLMs -- they take some basic,
| vague insights produced by a chatbot as profound discoveries.
| It seems the reviewers may have agreed, to some extent; note
| that it was received by _Machine Learning_ 24-03-26, but only
| accepted (after revisions) on 24-08-21.
|
| I wrote more below with a quote, but re: "who's the intended
| audience?" I think the answer is the same kind of people Gary
| Marcus writes for: other academic leaders, private investors,
| and general technologists. Definitely not engineers looking to
| apply their work immediately, nor the vast majority of
| scientists that are doing the long, boring legwork of
| establishing facts.
|
| In that context, I would defend the paper as evocative and
| creative, even though your criticisms all ring true. Like, take
| a look at their (his?) HuggingFace repo:
| https://huggingface.co/lamm-mit It seems clear that they're
| doing serious work with real LLMs, even if it's scattershot.
|
| Honestly, if I was a prestigious department head with millions
| at my disposal in an engineering field, I'm not sure I would
| act any differently!
|
| ETA: Plus, I'll defend him purely on the basis of having a
| gorgeous, well-documented Git repo for the project:
| https://github.com/lamm-mit/GraphReasoning?tab=readme-ov-fil...
| Does this constitute scientific value on its own? Not really.
| Does it immediately bias me in his favor? Absolutely!
| dacox wrote:
| ...k
| CatWChainsaw wrote:
| "The Future of Innovation" sounds exactly like freshly squeezed
| GPT drivel I'd expect to read from a vapid "hustler" on LinkedIn.
| quataran wrote:
| Wow, what's happened to MIT?
| bbor wrote:
| Well... Markus J. Buehler is the McAfee
| Professor of Engineering and former Head of the MIT Department
| of Civil and Environmental Engineering at the Massachusetts
| Institute of Technology. He directs the Laboratory for
| Atomistic and Molecular Mechanics (LAMM), leads the MIT-Germany
| program, and is Principal Investigator on numerous national and
| international research program... [he] is a founder of the
| emerging research area of materiomics. He has appeared on
| numerous TV and radio outlets to explain the impact of his
| research to broad audiences.
|
| I think this guy's just playing political/journalistic games
| with his research, and tailoring it for impact rather than
| rigor. I'm not sure I _endorse_ it necessarily, but I don 't
| think we should write this off as "dumb article from MIT", but
| rather "the explorations of a media-savvy department head".
| That doesn't excuse the occasional overselling of results of
| course, as that's dangerous to science no matter the
| motivation.
| youoy wrote:
| I think this article marks the "peak of inflated expectations" of
| AI for HN posts.
| quantadev wrote:
| Since all humans alive today have undergone the sum total of all
| human evolution, and are the ultimate creation of millions of
| years of evolution, it makes sense that the kinds of things we
| find "artistically pleasing" (both visually and thru sound) could
| have many patterns that apply to reality in deeper ways than any
| of us know, and so letting AI use art as it's inspiration for
| using those patterns in it's search for knew knowledge seems like
| a good idea.
|
| Also there are also certain aspects of physical geometric
| relationships and even sound relationships that would not be able
| to be conveyed to an AI by any other means than thru art and
| music. So definitely using art to inspire science is a good
| approach.
|
| Even the great Physicists throughout history have often
| appreciated how there is indeed beauty in the mathematical
| symmetries and relationships exhibited in the mathematics of
| nature, and so there is definitely a connection even if not quite
| tangible nor describable by man.
| 331c8c71 wrote:
| Can we call this "Deep Trolling"?
| kubb wrote:
| The very fact that some people are trying to take this
| seriously is probably the point he's trying to make.
| woozyolliew wrote:
| One to save for April 1st
| fudged71 wrote:
| Oh I'm glad that I'm not the only one who has gotten lost in the
| sauce by asking LLMs to recursively synthesize from data towards
| some grand insights--we want to see results when there is none
| apparent. What you end up getting is some bizarre theories
| overfit on the data with zero causal relationships. LLMs are
| fundamentally pattern matching systems and they will find
| "connections" between any two domains if prompted. It just reeks
| of confirmation bias; researchers looking for connections between
| art and science will find them.
|
| The simpler explanation makes more sense: knowledge graphs
| naturally show certain structural properties, and these
| properties appear across domains due to basic mathematical
| constraints, common organizational principles, and human
| cognitive patterns reflected in data. Sure, LLMs trained on human
| knowledge can identify these patterns, generate plausible
| narratives, and create appealing connections - but this doesn't
| necessarily indicate novel scientific insights, predictive power,
| or practical utility.
|
| If you find yourself going down a rabbit hole like this (and
| trust me, we've all been there), my advice is to ask "is there a
| simpler explanation that I'm missing?" Then start from square
| one: specific testable hypotheses, rigorous controls, clear
| success metrics, practical demonstrations, and independent
| validation. And maybe add a "complexity budget" - if your
| explanation requires three layers of recursive AI analysis to
| make sense, you're probably way too deep in the sauce.
| nnurmanov wrote:
| Since the article mentions graphs, I'd like to ask what would be
| the advantages of graph databases over relational? Graph
| databases have become popular in RAG related topics, maybe mainly
| GraphRag related work by MS. So I wonder if the same accuracy
| with RAG could be achieved by traditional databases. Or if graph
| databases are an absolute must, then what are their limitations?
| Are there any successful production usage cases of graph
| databases?
___________________________________________________________________
(page generated 2024-11-13 23:00 UTC)