[HN Gopher] Andrej Karpathy: Deep Dive into LLMs Like ChatGPT [v...
___________________________________________________________________
Andrej Karpathy: Deep Dive into LLMs Like ChatGPT [video]
Author : leroman
Score : 224 points
Date : 2025-02-05 18:29 UTC (4 hours ago)
(HTM) web link (www.youtube.com)
(TXT) w3m dump (www.youtube.com)
| ipsum2 wrote:
| He's made more than 5 videos covering basically the same topic,
| of transformer architecture and training. Wonder whats different
| about this one?
| ks2048 wrote:
| From the description: I have one "Intro to
| LLMs" video already from ~year ago, but that is just a re-
| recording of a random talk, so I wanted to loop around and do a
| lot more comprehensive version.
|
| I think he has videos on building GPT2 from scratch, but this
| seems more high-level.
| karpathy wrote:
| My YouTube videos fall into two tracks:
|
| 1. technical track (all the GPT repro series)
|
| 2. general audience track
|
| For (2), I had a 1hr video from 1 year ago, but I didn't
| actually expect that video to be some kind of authoritative
| introduction to LLMs. The history is that I was invited to give
| an LLM talk (to general audience), prepared some random slides
| for a day, gave the talk, and then re-recorded the talk in my
| hotel room later in a single take, and that become the video.
| It was quite random and haphazard. So I wanted to loop back
| around more formally and do a more comprehensive intro to LLMs
| for general audience; Something I could for example give to my
| parents, or a friend who uses ChatGPT all the time and is
| interested in it, but doesn't have the technical background to
| go through my videos in (1). That's this video.
| gdjdkslslp wrote:
| I haven't watched this video yet, but do you plan to create
| any technical videos in the (1) series on RL in LLMs?
| amelius wrote:
| This would be very welcome as it brings us closer to
| understanding the secret sauce behind training a real,
| practical LLM.
| gardenhedge wrote:
| I watched that one after finding it randomly! Will give the
| new one a watch too
| gnunez wrote:
| Great work! I love your videos; they've taught me so much.
| Any plans for a Mixture of Experts (MoE) video? My
| understanding is that starting from GPT4 most advance models
| use MoE to some extent. For example, can I take the model
| from your GPT2 video and just change the feed forward layer
| to an MoE layer like the one found here (1)? I guess I can
| just try it myself but I enjoy the expert guidance you
| provide in your videos. Please don't stop! great content!
|
| 1. https://github.com/mistralai/mistral-
| inference/blob/main/src...
| sota_pop wrote:
| Really love his "let's build" series - I end up picking up cool
| Python tricks along the process, even in addition to the higher
| level content.
| demarq wrote:
| I wish there were another way to distribute video. Content
| disappears from youtube eventually, for silly reasons.
|
| I think this is important content, the more people know how ai
| works under the hood the more empowered society will be.
| IncreasePosts wrote:
| You can just make a torrent of the video. It will then survive
| as long as you/others are willing to seed it.
| m_ppp wrote:
| Do you think videos disappearing is the biggest problem with
| YouTube from a distribution perspective?
| demarq wrote:
| Durability is not a Distribution problem.
| m_ppp wrote:
| I guess more in the problem of whether or not you consider
| it a good distribution platform, if you can't ascertain its
| durability.
| Dinux wrote:
| YouTube is known for not deleting videos, and so far the never
| have (with some obvious exceptions)
| layer8 wrote:
| If accessible knowledge about how LLMs work ever disappears, it
| won't be due to YouTube.
| tmp111111 wrote:
| Andrej, I like you much more now than when you were at Tesla. You
| have been adding real value to my life and many others. Thank
| you.
| Dinux wrote:
| Thanks Andrej. I have a pretty good understanding of how LLMs
| work and how they are trained, but alot of my friends don't.
| These videos/talks give them 'some' idea.
| behnamoh wrote:
| I'm a simple man, I see a Karpathy video, I click, I watch, I
| enjoy. :)
___________________________________________________________________
(page generated 2025-02-05 23:00 UTC)