A recommendation on Berkeley's Deep Reinforcement Learning Bootcamp
Aug 27, 2017
I just spent the weekend at the Berkeley Reinforcement Learning Bootcamp. Overall I found it to be an intense but rewarding experience but also expensive if you're paying out of pocket. Read on for more of my thoughts and my recommendations if you should attend.
Recommendations
I'm guessing most people will care about this so I'll just put it at the top. My recommendation is basically split into three categories.
I'm at least lightly interested in AI/RL and my employer will pay
Then you should definitely come. It's definitely worth the weekend because no matter what you'll be walking out of here with more knowledge than when you started. The information has been condensed by the experts to maximize your learning in a short time.
I'm interested but I'm paying myself
Sort of a toss up here. The bootcamp is pretty expensive but it still might be worthwhile. Read on for more details to see if it's worth it for you.
An approximate schedule and what was covered
The bootcamp was split over two days, with approximately 12 hours of activity each day. In a day specifically there about
- 2 hours of social/networking time, which included things like lunch and coffee breaks
- 3 hours of working on provided labs with PhD students available to help
- 7 hours of lectures with topics ranging from basics to cutting edge research
Lecture topics
Specifically the lectures covered topics starting from V and Q learning, through DQN, and then branched to focus on specific RL focus areas such as various types of Model Based Learning, Model Free Learning, Evolutionary methods, as well as lectures focused on general tips for learning how to learn Reinforcement Learning (say that three times fast). Overall the lectures were wellrstructured, given by a variety of speakers, and very informative. However, the target audience skillset varied greatly, the difficultly ranged from basic to extremely intense (literally cutting edge) pretty fast.
Labs
The labs were pretty basic, just implementations of Q and V learning, and the "Fill in the two magic lines" type of labs. The neat part where that there was plenty of expert TAs around to help with questions. Unfortunately the labs were not released ahead of time, so if you came with little knowledge of the basics you would spend most of your time stuck in the shallow end of the pool, and wouldn't really be able to really utilize the specialized help.
Networking
Everyone here was pretty enthusiastic to be here so overall a great crowd. The people certainly made the bootcamp more fun.
So If I'm paying myself is it worth it?
Well you'll definitely learn something, but "Is it worth it?" is a different question. If you're just starting out in RL then you really won't be able to grok most of the knowledge unfortunately. You'd be better off reading books and following tutorials online.
However, if you have some experience and practice using RL you're in the sweet spot, as the advanced topics will push your knowledge, and there are lots of friendly experts around to answer detailed questions.
A Final Note
With AI and Deep RL being a hot topic there's the risk that some of these conferences are just cashing in on the hype without providing much content. That is not the case with this bootcamp, lecturers and TAs were very knowledgeable, just be sure you're prepared to understand what they're teaching you!