Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/brianspiering/rl-course

Applied Reinforcement Learning course
https://github.com/brianspiering/rl-course

artificial-intelligence deep-learning deep-reinforcement-learning machine-learning reinforcement-learning

Last synced: about 1 year ago
JSON representation

Applied Reinforcement Learning course

Awesome Lists containing this project

README 

          
Applied Reinforcement Learning Course






----

Course Description

----



This course covers the fundamental techniques in applied Reinforcement Learning (RL). 



Reinforcement Learning is one the most successful subdomains within Artificial Intelligence (AI). The core idea in Reinforcement Learning is agents learn to which actions are successful in a given environment. 



Reinforcement Learning extends the usefulness of Machine Learning into new domains, such as video games, board games, robotics, and using human feedback to improve machine learning models (i.e., training GPTChat). The course will focus on contemporary, practical applications of Reinforcement Learning, such as multi-arm bandits, q-learning, and Deep Reinforcement Learning. 



----

Logistics

----



__Instructor:__ Brian Spiering   

__Website__: [github.com/brianspiering/

rl-course](https://github.com/brianspiering/rl-course)    



Learning Outcomes

----



By the end of the course, you should be able to:  



1. Apply Reinforcement Learning techniques to solve complex, applied problems.

1. Define and identify examples of common Reinforcement Learning terms (e.g., agent, environment, state, and rewards).

1. Implement common Reinforcement Learning techniques (e.g., multi-arm bandits and q-learning) from scratch.

1. Create end-to-end Deep Reinforcement Learning applications.



Prerequisites

----



- Working knowledge of probability, statistics, machine learning, and deep learning.

- Intermediate level of Python (e.g., ability to create to classes).



----

Topics

----



1.  Welcome 

2.  Intro to RL 

3.  Markov Decision Processes (MDP) 

4.  Policy 

5.  Value Iteration

6.  Multi-armed Bandits for AB testing

7.  Contextual Bandits for Recommendation Systems

8.  Dynamic Programming 

9.  Monte Carlo 

10. Temporal Difference Learning

11. Q-Learning

12. Value Function Approximation 

13. Deep Reinforcement Learning (DRL)

14. Deep Q-learning

15. Deep Reinforcement Learning Applications

16. Proximal Policy Optimization (PPO)

17. Alpha Go 

18. Future of RL 

19. Student's Choice



The Deep Learning section will be in [Keras](https://www.tensorflow.org/guide/keras).



----

Textbooks

----



1. Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto. The standard textbook for RL [pdf](http://incompleteideas.net/book/bookdraft2017nov5.pdf)  

1. Algorithms for Reinforcement Learning by Csaba Szepesvari. A delightful walk-through of the most common RL algorithms [pdf](https://sites.ualberta.ca/~szepesva/papers/RLAlgsInMDPs.pdf)  

1. Artificial Intelligence: A Modern Approach 4ᵗʰ Edition by Stewart Russell and Peter Norvig. The standard textbook for AI. [website](https://aima.cs.berkeley.edu/)  

 

----

Grading

----



| Item           | Weight   |

|:---------------|:--------:|

| Prework Check  | 10%      |

| Participation  | 10%      |

| Exam           | 20%      |

| Labs           | 30%      |

| Final Project  | 30%      |

| __Total__      | __100%__ |



Each item's contribution is capped its respective percentage. The total course percentage is capped at 100%.



Currently, there is no extra credit. If there is any extra credit, it entirely at the discretion of the instructor.



### Participation



I try to create an active learning environment in my classroom, which is incentivized with the _Participation_ grade. Attendance is mandatory, you can't participate if you don't attend. It is the responsibility of the student to attend all classes. If you have to miss class, due to sickness, job related activities, or other circumstances, please notify your instructor by Slack in advance. 



Tardiness negatively impacts an active learning environment, thus will impact your participation grade.



You must show up to each session prepared. Each person is important to the dynamic of the class, and therefore students are required to participate in class activities. Expect to be "cold called". I call on students at random not to put you on the spot but to keep you engaged in the material at all times.



I expect you to be fully present and engaged at all times.  



I except you follow the [etiquette guidelines](https://github.com/brianspiering/teaching_materials/blob/master/online_class_etiquette_guidelines.md) throughout the entire course. This is your warning. Every violation will result in a loss of participation points, negatively impacting your total grade. 



### Prework Check



There will be a required prework __before__ each class. The prework will check that you have watch the videos, read the reading, and are prepared for the in-class activities. The prework check also allows be to get a sense for the current  understanding of the class. I know which material I can skim/skip and which material to over in greater detail.



The prework will be online in Canvas.



### Exam



The single exam will be a take-home exam. 



### Labs



The labs will be hands-on activities. The focus will be implementing algorithms from scratch or applying common libraries.



1. Tower of Hanoi   

2. Grid World       

3. Pyramid Escape   

4. Tic, Tac, Toe    

5. Basket Catch     

6. Cart Pole        



### Final Project 



In lieu of a Final Exam, there will be a Final Project. The Final Project will be implement a Reinforcement Learning solution for a __novel__ task. By novel, I mean a task that has no existing Reinforcement Learning solutions. The task could be a game, a dataset, or a business problem.



Details and due dates are in Final Project Folder.