Reinforcement Learning

https://bsky.app/profile/natolambert.bsky.social/post/3lh5jih226k2k

Anyone interested in learning about RLHF? This text isn't complete yet, but looks to be a pretty useful resource as is already.

An overview of RL published just a few days ago. 144 pages of goodies covering everything from basic RL theory to modern deep RL algorithms and various related niches.

This manuscript gives a big-picture, up-to-date overview of the field of (deep) reinforcement learning and sequential decision making, covering value-based RL, policy-gradient methods, model-based methods, and various other topics (including a very brief discussion of RL+LLMs).

Recordings for the RLC keynote talks have been released.

Keynote speakers:

• David Silver
• Doina Precup (Not recorded)
• Peter Stone
• Finale Doshi-Velez
• Sergey Levine
• Emma Brunskill
• Andrew Barto

OpenAI just put out a blog post about a new model trained via RL (I'm assuming this isn't the usual RLHF) to perform chain of thought reasoning before giving the user its answer. As usual, there's very little detail about how this is accomplished so it's hard for me to get excited about it, but the rest of you might find this interesting.