[HN Gopher] A (Long) Peek into Reinforcement Learning
       ___________________________________________________________________
        
       A (Long) Peek into Reinforcement Learning
        
       Author : Brysonbw
       Score  : 138 points
       Date   : 2025-03-23 07:40 UTC (3 days ago)
        
 (HTM) web link (lilianweng.github.io)
 (TXT) w3m dump (lilianweng.github.io)
        
       | fancyfredbot wrote:
       | Very nice! I also recommend the CS885 course from university of
       | Waterloo which is all on YouTube. It covers Bayesian
       | reinforcement learning which is missing from most courses
       | (including this one).
        
         | wiesbadener wrote:
         | Here a link to the lecture playlist:
         | https://www.youtube.com/playlist?list=PLdAoL1zKcqTXFJniO3Tqq...
        
       | yamrzou wrote:
       | (2018)
       | 
       | Has RL changed since then?
        
         | fancyfredbot wrote:
         | This page covers the basics which are pretty timeless. You'd
         | want to understand this before learning about any more recent
         | advances.
         | 
         | It doesn't get into state of the art algorithms, for example
         | proximal policy optimisation isn't mentioned although the paper
         | on this was published in 2017 and is probably considered the
         | best algorithm for at least some applications.
        
       | antman wrote:
       | Is anyone aware of a course that is beyond the basics and is
       | focused on modern algorithms?
        
         | calebkaiser wrote:
         | It sort of depends on which direction you want to go in. If
         | you're interested in deep RL as applied specifically to LLMs,
         | I'd second another commenter's recommendation of Spinning Up
         | from OpenAI. It hasn't been updated for a while and is a little
         | outdated, but it provides a really nice introduction to some
         | key ideas like PPO:
         | https://spinningup.openai.com/en/latest/user/introduction.ht...
         | 
         | If you want to get more of a bird's eye view of modern deep RL
         | in general, then HuggingFace's course is a good place to start.
         | The course itself is kind of "all over the place"--not
         | necessarily a bad thing, just maybe not what you want if you're
         | looking to go super deep on a single topic. But if you want to
         | get a look at some robotics stuff, deep Q-learning, multi-agent
         | stuff, etc., it'll give you a nice sort of "tasting menu." As
         | with most HuggingFace stuff, the format is really nice, it does
         | a good job of introducing you to key ideas/projects in the
         | ecosystem, and it has some cool project components:
         | https://huggingface.co/learn/deep-rl-course/en/unit0/introdu...
        
           | avandekleut wrote:
           | I used SpinningUp as one of my main sources for my thesis. I
           | also wrote some downloadable-and-runnable blog posts here:
           | https://avandekleut.github.io/ppo/
        
         | joelthelion wrote:
         | This one looks pretty good: https://arxiv.org/pdf/2412.05265v2,
         | although I've only skimmed it so far.
        
       | jxjnskkzxxhx wrote:
       | If you want a (long) peek at RL read Sutton and barto. Amazing
       | book.
        
       | amelius wrote:
       | Curious what people here have built using RL?
        
         | dcrimp wrote:
         | A mate of mine built a works scheduler using RL + MCTS. It was
         | interesting seeing the scheduler get smarter as they added in
         | reward for real life constraints. For example, certain types of
         | work couldn't happen on a tuesday - they add that in to the
         | reward calculation, retrain, it now avoids Tuesdays. Build up
         | that reward calculation based on available data, and it got to
         | be super capable at making a workable schedule. Also orders of
         | magnitude faster than linear solvers (albeit without guarantee
         | of "optimality").
        
         | phyalow wrote:
         | Highly profitable trading systems.
        
       | farhanhubble wrote:
       | Lillian's writing is really crisp without compromising the
       | technical details.
       | 
       | I learnt reinforcement learning from Udacity course. It was one
       | of the finest on the internet. Whenever I couldn't find some
       | thing in the course I went to this website. I created my own
       | small introductory course for RL, a pure hands-on experience for
       | learners.
        
         | ultrasounder wrote:
         | I found this in the Authors GitHub repo. Looks interesting.
         | Will use this to complement my learning resources
         | https://github.com/farhanhubble/discover-drl/blob/master/Red...
        
           | farhanhubble wrote:
           | If you're interested, this is the course that I created:
           | https://github.com/KnowchowHQ/rl-in-
           | action/tree/master/C1-RL...
           | 
           | The other one you found is just one of the notebooks from the
           | course.
        
       ___________________________________________________________________
       (page generated 2025-03-26 23:02 UTC)