[HN Gopher] Guide to Machine Learning with Geometric, Topologica...
___________________________________________________________________
Guide to Machine Learning with Geometric, Topological, and
Algebraic Structures
Author : johmathe
Score : 108 points
Date : 2024-07-15 16:19 UTC (6 hours ago)
(HTM) web link (www.arxiv.org)
(TXT) w3m dump (www.arxiv.org)
| dpflan wrote:
| The paper's references have some good ones for getting more
| acquainted with these subjects; this one being a nice dense one
| to start with:
|
| - Geometric Deep Learning Grids, Groups, Graphs, Geodesics, and
| Gauges: https://geometricdeeplearning.com/
| funnygiraffe wrote:
| Is geometric, topological, and algebraic ML/data analysis
| actually used in the industry? It is certainly beautiful math.
| However, during grad school I met a few pure math PhD students
| who were saying that after finishing their PhD they will just go
| into industry to do topological data analysis (this was about 10
| years ago and ML wasn't yet as hyped up). However, I have never
| heard of anybody actually having success on that plan.
| dpflan wrote:
| I believe a use-case(s) receiving attention is drug design,
| protein design, chemical design, etc.
|
| Here is a summer school by the London Geometry and Machine
| Learning group where research topics are shared and discussed.
| - https://www.logml.ai/
|
| Here is another group, a weekly reading group on graphs and
| geometry: https://portal.valencelabs.com/logg
| funnygiraffe wrote:
| Thanks. That's certainly very interesting. Albeit it seems to
| me that the number of jobs doing geometric and topological
| ML/AI work in the drug or protein design space would be quite
| limited, because any discovery ultimately has to be validated
| through a wet lab process (or perhaps phase 1-3 clinical
| trials for drugs) which is expensive and time-consuming.
| However, I'm very uninformed and perhaps there is indeed a
| sizable job market here.
| dpflan wrote:
| I think the job market in general for this kind of stuff is
| "small"; but you can find jobs. Look at Isomoprhic Labs for
| example. There are new AI/ML companies that have emerged in
| recent years, helped by success of things like AlphaFold. I
| think your question is really: does this research actually
| creates tangible results? If it did, it would be able to
| create more jobs to support it by virtue of being
| economically successfully and therefore growing?
| heyitsguay wrote:
| As someone who did an applied math PhD before drifting
| towards ML, it's worth pointing out that these applied math
| groups typically talk about applications, but the real
| question is whether they are actually used for the stated
| application in practice due to outperforming methods that use
| less pretty math. Typically (in every case i have seen) the
| answer is "no", and the mathematicians don't even really care
| about solving the applied problems nor fully understand what
| it would mean to do so. It's just a source of grant-
| justifiable abstract problems.
|
| I would love to be proven wrong though!
| dpflan wrote:
| Indeed, the ivory tower has nice chats and ideas and is a
| cool place to hang out, but does application actually
| occur.
| llm_trw wrote:
| I've had some success using hyperbolic embeddings for bert like
| models.
|
| It's not something that the companies I've worked for
| advertised or wrote papers about.
| jqgatsby wrote:
| Hyperbolic embeddings have been an interest of mine ever
| since the Max Nickel paper. Would love to connect directly to
| discuss this topic if you're open. here's my email:
| https://photos.app.goo.gl/1khCwXBsVBuEP6xF7
| fjork wrote:
| I don't think there's much use currently. But I kinda like the
| direction of the paper anyway. Most mathematical objects in ML
| have geometric or topological structure, implicitly defined. By
| making that structure explicit, we at worst have a fresh new
| perspective on some ML thing. Like how viewing the complex
| numbers on a 2d cartesian plane often clicks more for students
| compared to the dry algebraic perspective. So even in the worst
| case I think there's some pedagogical clarity here.
| itissid wrote:
| One common theme I see in the paper(e.g. in protein folding) is:
|
| "Identify what properties are important (geometry, algebra, topo)
| and which one is an useful prior and then "use" the guide to
| select an initial struct. This is probably harder than it
| sounds(unlike bayesian priors which are more forgiving for one to
| select, but quite like them in that they both require special
| assumptions)."
|
| I wonder: could one use it to bring together certain multimodal
| data and a proposed network for a task? Like could one bring in
| sensor, map topology, urban topology, pictures which have certain
| properties and that help me use this guide to make a statement
| like : "Street data could be embedded with Sensor data to do ABC
| kind of inference using XYZ NNetwork structure because this paper
| suggests that is a reasonable thing to do"?
| uoaei wrote:
| All machine learning is just embedding of various forms. If you
| have a way to translate disparate types of data into a common
| space, in ways that preserve inductive bias and information
| content, you can then combine them for downstream tasks.
| mjhay wrote:
| I am 100% convinced that these kind of approaches will be what
| delivers ML research from the current resource-hungry and
| ungeneralizable status quo. Low-dimensional Euclidean geometry is
| special. Higher-dimensional Euclidean spaces are less special.
| Most real-life data is high-dimensional, not at all smooth, and
| possessing a structure you cannot call Euclidean with a straight
| face. Look at what works with tabular data (which is probably
| most of what practitioners work with in the wild). It's gradient
| boosted trees, not neural networks.
|
| There is a fundamental mismatch between the data we usually work
| with and the spaces we shove it into. Tools from algebraic
| topology and geometry are old hat in physics. If anything, they
| should be even more useful in ML.
| Davidzheng wrote:
| I heavily disagree with the statement "tools of algebraic
| topology is old hat in physics"
| BenoitP wrote:
| Well, I consider Lorentz' work to be old hat. I can't find an
| older example though.
|
| https://en.m.wikipedia.org/wiki/Lorentz_group
| OutOfHere wrote:
| Comment is heavily exaggerated in every way.
| mistrial9 wrote:
| some people on this thread are asking about jobs. The bigger
| picture here is that previously intractable problems are going to
| be solved with a new combination of math, data and compute..
| there are lots of commercial cases that will change dramatically.
| How can individual people or small groups benefit from serious
| problem solving, economically?
| OutOfHere wrote:
| Note that the GPU hardware is setup for Euclidean matrix
| operations. Even if you had a deep structure learner, it won't
| necessarily help you if you have to go back to emulating it on
| Euclidean hardware.
___________________________________________________________________
(page generated 2024-07-15 23:00 UTC)