[HN Gopher] Jurgen Schmidhuber:the Father of Generative AI Witho...
___________________________________________________________________
Jurgen Schmidhuber:the Father of Generative AI Without Turing Award
Author : kleiba
Score : 55 points
Date : 2025-06-20 19:00 UTC (3 hours ago)
(HTM) web link (www.jazzyear.com)
(TXT) w3m dump (www.jazzyear.com)
| nharada wrote:
| Doesn't he know the Turing Award is really just a generalization
| of the Fields Medal, an award that actually came years earlier?
| logicchains wrote:
| I'm sure he wouldn't object to a Fields Medal either.
| triceratops wrote:
| I chuckled but I also maybe didn't understand. Is the joke that
| computer science is a generalization of math? That can't be
| rigth.
| dgacmu wrote:
| No, the joke is that schmidhuber is known for (rightly or
| wrongly) pointing to modern contributions in deep neural
| networks and saying they're just a trivial
| generalization/adaptation/etc. of work he did 30 years ago.
| belval wrote:
| Every so often Schmidhuber is brought back to the front-page of
| HN, people will argue that he "invented it all" while others will
| say that he's a-posteriori claiming all the good ideas were his.
|
| Relativity Priority Dispute:
| https://en.wikipedia.org/wiki/Relativity_priority_dispute
|
| We all stand on the shoulders of giants, things can be invented
| and reinvented and ideas can appear twice in a vacuum.
| kleiba wrote:
| But as far as I understand, Schmidhuber's claim is more severe:
| namely that Bengio, Hinton and LeCun _intentionally_ failed to
| cite prior work by others (including himself) but instead only
| cited each other in order to boost their respective scientific
| reputation.
|
| I personally think that he's not doing himself or his argument
| of favor by presenting it the way he does. While he basically
| argues that science should be totally objective and neutral,
| there's no denying that if you put yourself in a less likeable
| light, you're not going to make any friends.
|
| On the other hand, he's gone at length with compiling detailed
| references to support his points. I can appreciate that because
| it makes his argument a lot less hand-wavey: you can go to his
| blog and compare the cited references yourself. Except that I
| couldn't because I'm not an ML expert.
| jll29 wrote:
| I have seen many cases where people -- accidentally as well
| as intentionally -- copied or re-invented the work of others
| (a friend posted on LinkedIn that someone else plagiarized
| his whole Ph.D. thesis, including the title, so only the name
| was changed; only the references to seperately published
| papers on which the individual chapters were based at the end
| still had my friend's name in it, so you could see it was a
| fake thesis).
|
| If a bona fide scientist makes a mistake about missing
| attribution, they would correct it as soon as possible. Many,
| however, would not correct such a re-discovery, because it's
| embarrassing.
|
| But the worst is when people don't even imagine anything like
| what they are working on could already exist, and they don't
| even bother finding and reading related work -- in other
| words, ignorance. Science deserves better, but there are more
| and more ignorant folks around that want to ignore all work
| before them.
| godelski wrote:
| > Many, however, would not correct such a re-discovery,
| because it's embarrassing.
|
| This is a culture thing that needs to change.
|
| I'm a pretty big advocate of open publishing and avoiding
| the "review process" as it stands today. The reason is
| because we shouldn't be chasing these notions of novelty
| and "impact". They are inherently subjective and lead to
| these issues of credit. Your work isn't diminished because
| you independently invented it, rather, that _strengthens_
| your work. There 's more evidence! Everything is
| incremental and so all this stuff does is makes us focus
| more on trying to show our uniqueness rather than showing
| our work. The point of publishing is to communicate. The
| peer review process only happens post communicating: when
| people review, replicate, build on, or build against. We're
| just creating an overly competitive environment. It is only
| "embarrassing" because it "undermines" the work. It only
| "undermines" the work because how we view credit.
|
| Consider this as a clear example. Suppose you want to
| revisit a work but just scale it up and use on modern
| hardware. You could get state of the art results but if you
| admit to such a thing with no claimed changes (let's say
| you literally just increase number of layers) you'll never
| get published. You'll get responses about how we "already
| knew this" and "obviously it scales". But no one tested
| it... right? That's just bad for science. It's bad if we
| can't do mundane boring shit.
| Lerc wrote:
| I can see how someone could feel like that if they looked at
| the world in a particular way.
|
| I have had plenty of ideas in the last few years that I have
| played with that I have seen published in papers in the
| following months. Rather than feeling like "I did it first" I
| feel gratified that not only was I on the right track, but
| someone else has done the hard slog.
|
| Most papers are not published by people who had the idea the
| day before. Their work goes back further than that. Refining
| the idea, testing it and then presenting the results takes
| time, sometimes years, occasionally decades.
|
| If this happens to you, don't think "Hey! That idea belongs to
| me!". Thank them for proving you right.
|
| Now if they patent it, that's a different story. I don't think
| the ideas that sometimes float through my brain belong to me,
| but I'm not keen on them belonging to someone else either.
| kleiba wrote:
| I think that's slightly mispresenting Schmidhuber's case
| though, because he does not just say "oh, I already had that
| same idea before you, I just follow up on it". He is usually
| referring to work that he or members of his group (or third-
| party researchers for that amatter) did, in fact, already
| publish.
| Mond_ wrote:
| Oh boy, I sure can't wait to see the comments on this one!
|
| Schmidhuber sure seems to be a personality, and so far I've
| mostly heard negative things about his "I invented this" attitude
| to modern research.
| kleiba wrote:
| A lot of this is because nobody likes braggers - however, in
| all fairness, his argument is that a lot of what is considered
| modern ML is based on _many_ previous results, including but
| not limited to his own research.
| goldemerald wrote:
| No discussion with Schmidhuber is complete without the infamous
| debate at NIPS 2016 https://youtu.be/HGYYEUSm-0Q?t=3780 . One of
| my goals as a ML researcher is to publish something and have
| Schmidhuber claim he's already done it.
|
| But more seriously, I'm not a fan of Schmidhuber because even if
| he truly did invent all this stuff early in the 90s, he's
| inability to see its application to modern compute held the field
| back by years. In principle, we could have had GANs and self-
| supervised models' years earlier if he had "revisited his early
| work". It's clear to me no one read his early paper's when
| developing GANs/self-supervision/transformers.
| andy99 wrote:
| It's very common in science for people to have had results they
| didn't understand the significance of that later were
| popularized by someone else.
|
| There is the whole thing with Damadian claiming to have
| invented MRI (he didn't) when the Nobel prize went to Mansfield
| and Lauterbur (see the Nobel prize part of the article).
| https://en.m.wikipedia.org/wiki/Paul_Lauterbur
|
| And I've seen other less prominent examples.
|
| It's a lot like the difference between ideas and execution and
| people claiming someone "stole" their idea because they made a
| successful business from it.
| nextos wrote:
| I think he did understand both the significance of his work and
| the importance of hardware. His group pioneered porting models
| to GPUs.
|
| But personal circumstances matter a lot. He was stuck at IDSIA
| in Lugano, i.e. relatively small and not-so-well funded
| academia.
|
| He could have done much better in industry, with access to lots
| of funding, a bigger headcount, and serious infrastructure.
|
| Ultimately, models matter much less than infrastructure.
| Transformers are not that important, other architectures such
| as deep SSMs or xLSTM are able to achieve comparable results.
| godelski wrote:
| > if he had "revisited his early work".
|
| Given that you're a researcher yourself I'm surprised by this
| comment. Have you not yourself experienced the harsh rejection
| of "not novel"? That sounds like a great way to get stuck in
| review hell. (I know I've experienced this even when doing
| novel things just by too closely relating it to other
| methodologies when explaining "oh, it's just ____").
|
| The other part seems weird too. Who isn't upset when their work
| doesn't get recognized and someone else gets credit. Are we not
| all human?
| mindcrime wrote:
| I'll probably get flamed to death for saying this, but I like
| Jurgen. I mean, I don't know him in person (never met him) but
| I've seen a lot of his written work and interviews and what-not
| and he seems like an alright guy to me. Yes, I get it... there's
| that whole "ooooh, Jurgen is always trying to claim credit for
| everything" thing and all. But really, to me, it doesn't exactly
| come off that way. Note that he's often pointing out the lack of
| credit assigned even to people who lived and died _centuries
| before him_.
|
| His "shtick" to me isn't just about him saying "people didn't
| give me credit" but it seems more "AI people in general haven't
| credited the history of the field properly." And in many cases he
| seems to have a point.
| noosphr wrote:
| It's a clash of cultures.
|
| He is an academic that cares for understanding where ideas came
| from. His detractors need to be the smartest people in the room
| to get paid millions and raise billions.
|
| It's not very sexy to say 'Oh yes, we are just using an old
| Soviet learning algorithm on better hardware. Turns out we
| would have lost the cold war if the USSR had access to a
| 5090.", which won't get you the billions you need to build the
| supercomputers that push the state of the art today.
| godelski wrote:
| I think you sum up my feelings about him as well. He's a bit
| much sometimes but it's hard to deny that he's made monumental
| contributions to the field.
|
| It's also funny that we laugh at him when we also have a joke
| that in AI we just reinvent what people did in the 80's. He's
| just the person being more specific as to what and who.
|
| Ironically, I think the problem is we care too much about
| credit. It ends up getting hoarded rather than shared. We then
| just oversell our contributions because if you make the
| incremental improvements that literally everyone does, you get
| your works rejected for being incremental.
|
| I don't know what it is about CS specifically, but we have a
| culture problem or attribution and hype. From building on open
| source, it's libraries all the way down, but we act like we did
| it all alone. To jumping on bandwagons as if there's a right
| and immutable truth to how to do certain things, until the
| bubbles pop and we laugh at how stupid anyone was to do such a
| thing. Yet we don't contribute back to those projects that have
| US foundation, we laugh at "theory" which we stand on, and we
| listen to the same hype train people who got it wrong last time
| instead of turning to those who got it right. Why? It goes
| directly counter to the ideas of a group who love to claim
| rationalism, "working from first principles", and "I care what
| works"
| FL33TW00D wrote:
| If you guys were the inventors of Facebook, you'd have invented
| Facebook
| noosphr wrote:
| Before people say that he is claiming credit for things he didn't
| do, or that he invented everything, please read his own paper on
| the subject:
|
| https://people.idsia.ch/~juergen/deep-learning-history.html
|
| The history section starts in 1676.
| swyx wrote:
| > 1676: The Chain Rule For Backward Credit Assignment
|
| Schmidhuber is nothing but a stickler for backward credit
| assignment
| ur-whale wrote:
| If there ever was an example of a terrible personality getting in
| the way of career, Schmidhuber is most definitely it.
|
| You may have had many brilliant ideas, but everyone makes an
| abrupt 180 when they see the tip your beard turn the corner at
| conferences, that can't be a good signal for getting awards.
| banq wrote:
| Why did Google give birth to the Transformer? Because Google
| created an ecosystem where everything could flourish, while the
| old man in Switzerland lacked such an environment--what could
| even the smartest and greatest individual do against that?
|
| As an organization, fostering an organically growing context is
| like governing a great nation with delicate care. A bottom-up
| (organic growth) environment is the core context for sustained
| innovation and development!
| esafak wrote:
| The fact that the field of machine learning keeps "discovering"
| things already established in other fields, and christening them
| with new names does lend some credence to Schmidhuber. The field
| is more industrial than academic, and cares about money more than
| credit, and industrial-scale data theft is all in a day's work.
|
| As another commenter said, his misfortune is being in a lab with
| no industrial affiliation.
___________________________________________________________________
(page generated 2025-06-20 23:00 UTC)