[HN Gopher] Let's Build the GPT Tokenizer [video]
___________________________________________________________________
Let's Build the GPT Tokenizer [video]
Author : davidbarker
Score : 269 points
Date : 2024-02-20 17:20 UTC (5 hours ago)
(HTM) web link (www.youtube.com)
(TXT) w3m dump (www.youtube.com)
| threesevenths wrote:
| Andrej's video on building GPT nano is an excellent tutorial of
| all of the steps involved in a modern LLM.
| xdennis wrote:
| https://www.youtube.com/watch?v=kCc8FmEb1nY
| sabareesh wrote:
| Even if you pay it is hard to get such a high quality content!
| progbits wrote:
| I've been learning a few new CS things recently and honestly I
| mostly find inverse correlation between cost and quality.
|
| There are books from oreilly and paid MOOC courses that are
| just padded with lots of unnecessary text or silly "concept
| definition" quizzes to make them seem worth the price.
|
| And there are excellent free YT video lectures, free books or
| blog posts.
|
| Andrej's YT videos are one great example.
| https://course.fast.ai is another.
| simmanian wrote:
| Do you have recommendations for other high quality courses
| teaching CS things?
| davidbarker wrote:
| I can highly recommend CS50 from Harvard
| (https://www.youtube.com/@cs50). Even after being involved
| in tech for 25+ years, I learnt a lot from just the first
| lecture alone.
|
| Disclosure: Professor Malan is a friend of mine, but I was
| a fan of CS50 long before that!
| GaneshSuriya wrote:
| - operating system in three easy pieces
| (https://pages.cs.wisc.edu/~remzi/OSTEP) is incredible for
| learning OS internals
|
| - beej's networking guide is the best thing for network
| layer stuff https://beej.us/guide/
|
| - explained from first principles great too
| https://explained-from-first-principles.com/
|
| - pintos from Stanford https://web.stanford.edu/class/cs140
| /projects/pintos/pintos_...
| diimdeep wrote:
| Build an 8-bit computer from scratch
| https://eater.net/8bit/ https://www.youtube.com/playlist?li
| st=PLowKtXNTBypGqImE405J2...
|
| Andreas Kling. OS hacking: Making the system boot with
| 256MB RAM https://www.youtube.com/watch?v=rapB5s0W5uk
|
| MIT 6.006 Introduction to Algorithms, Spring 2020 https://w
| ww.youtube.com/playlist?list=PLUl4u3cNGP63EdVPNLG3T...
|
| MIT 6.824: Distributed Systems
| https://www.youtube.com/@6.824
|
| MIT 6.172 Performance Engineering of Software Systems, Fall
| 2018 https://www.youtube.com/playlist?list=PLUl4u3cNGP63VIB
| QVWguX...
|
| CalTech cs124 Operating Systems
| https://duckduckgo.com/?t=ffab&q=caltech+cs124&ia=web
|
| try searching here at HN for recommendations
| https://hn.algolia.com
| nojvek wrote:
| Thank you a ton for the links.
| thfuran wrote:
| >And there are excellent free YT video lectures, free books
| or blog posts.
|
| There's also a tremendous amount of extremely low quality
| YouTube and blog content.
| progbits wrote:
| Sure. I don't claim the free content is all good.
|
| But from my limited sample size, the best free content is
| better than the best paid content.
| GaneshSuriya wrote:
| It's not only about the cost, though. There's an inverse
| correlation with the glossiness of the content as well.
|
| If the web page /content is too polished, they're most likely
| optimizing for wooing users.
|
| Unlike a lot of the examples I gave in the sibling comments.
| Where the optimization is only on the love for the topic
| being discussed
| rahimnathwani wrote:
| There's an inverse correlation with the glossiness of the
| content as well.
|
| This is probably due to survivorship bias. Sites that have
| poor content and poor visual appeal (glossiness) never get
| on your radar.
|
| i.e. Berkson's Paradox:
| https://en.wikipedia.org/wiki/Berkson%27s_paradox
| danielmarkbruce wrote:
| There are some extremely good CS textbooks which cost money.
| That being said, many good ML/AI texts are free. But it's not
| easy reading.
| 3abiton wrote:
| His previous video onLLM tramsformer foundation is extremely
| useful.
| mrtksn wrote:
| I can't recommend enough the whole series, zero to hero:
| https://karpathy.ai/zero-to-hero.html
|
| No metaphors trying explain "complex" ideas, making them scary
| and seem overly complex. Instead, hands on implementations with
| analogy explainers where you can actually understand the ideas
| and see how simple it is.
|
| Steeper learning curve at first but it is much more satisfying
| and you actually earn the ability to reason about this stuff
| instead of writing over the top influencer BS.
| MPSimmons wrote:
| Thanks for this link - I have some free time coming up, and
| this seems like a great use of it!
| moffkalast wrote:
| > you see when it's a space egg, it's a single token
|
| I'm not sure if the crew of the Nostromo would agree ;)
___________________________________________________________________
(page generated 2024-02-20 23:00 UTC)