[HN Gopher] "Attention is all you need" paper digested (2018)
___________________________________________________________________
"Attention is all you need" paper digested (2018)
Author : binidxaba
Score : 75 points
Date : 2023-10-07 17:27 UTC (5 hours ago)
(HTM) web link (www.youtube.com)
(TXT) w3m dump (www.youtube.com)
| artninja1988 wrote:
| Love Yannic
| wanderingstan wrote:
| For a higher level shorter overview, I found this video
| informative: https://youtu.be/SZorAJ4I-sA?si=pnfzZ17PYQfV4aqq
| dzign wrote:
| Google regrets publishing this paper!
| [deleted]
| random3 wrote:
| Maybe, but it would have surfaced regardless, either directly
| or through related things. While the transformer may evolve
| into the next thing, it's equaly likely the next evolution will
| be unrelated to transformer.
|
| Moreover while the transformers and current LLMs are a leap,
| the monoculture around them is not necessarily a good thing,
| defocusing many good researchers from otherwise promising tech.
|
| Finally, cross-polination of ideas is where the magic happens.
| mensetmanusman wrote:
| It's probably Nobel worth based on its world impact.
| [deleted]
| dinvlad wrote:
| What impact?
| esafak wrote:
| It's used in all sorts of models, like LLMs and image
| generators. It's probably the most popular foundation of
| advanced ML models today.
| dzign wrote:
| You mean a Turing Award...
| whatyesaid wrote:
| Too many coauthors for a Turing or Nobel.
| numbers_guy wrote:
| World impact is not how Nobels are won. If that was the case
| Elon Musk, Jeff Bezos, Zuckerberg and others would all have
| multiple ones.
|
| I honestly do not see this paper as being in the same
| magnitude of brilliance as a typical Nobel would be. Not to
| mention that it barely counts as science (actually it
| probably does not). Don't get me wrong. It is a huge
| achievement for both the machine learning research field and
| for humanity as a whole, but putting along the achievements
| of Nobel physicists and such feels wrong.
| dvngnt_ wrote:
| objectively speaking what makes them so much better?
| abhishekjha wrote:
| Not sure if sarcasm or not.
| nothrowaways wrote:
| Why do you think it's sarcasm?
| dzign wrote:
| https://www.washingtonpost.com/technology/2023/05/04/google-.
| ..
| Alifatisk wrote:
| Here's the thing, if google didn't publish this paper, it would
| probably just collect dust somewhere at Google.
|
| OpenAi (ClosedAi) saw the potential and demonstrated its
| impressive and unbelievable capabilities!
|
| If Google never shared the transformer model with the world, we
| would probably not have what we have today.
| dzign wrote:
| https://www.washingtonpost.com/technology/2023/05/04/google-.
| ..
| hn_throwaway_99 wrote:
| Very much disagree. Google clearly saw the potential of this
| as well, and did a ton of work and created a lot of leading
| models based on this.
|
| The big difference between Google and OpenAI is that Google
| "had a ton more to lose" so to speak and went forward much
| more cautiously. See all the hullaballoo they had to deal
| with e.g. with their "Ethical AI" group and the Timnit Gebru
| fiasco, as well as cases like where that dim bulb Google
| employee claimed that LaMDA was sentient. OpenAI, on the
| other hand, was "full speed ahead" from the get-go.
|
| As a result, many of the top AI researchers left Google.
| After all, wouldn't you rather work at a place where you
| could see your work productized as fast as you could build
| it, rather than at a place where other sizable teams in your
| company were actively working in an adversarial role to put
| up roadblocks and vetoes wherever they could?
| [deleted]
| machinelearning wrote:
| https://www.askyoutube.ai/share/6521aa6077733b7c0ad24b55
___________________________________________________________________
(page generated 2023-10-07 23:00 UTC)