You should take this course if you have an interest in machine learning and the desire to engage with it from a theoretical perspective. Through a combination of classic papers and more recent work, you will explore automated decision-making from a computer-science perspective. You will examine efficient algorithms, where they exist, for single-agent and multi-agent planning as well as approaches to learning near-optimal decisions from experience. At the end of the course, you will replicate a result from a published paper in reinforcement learning.
Why Take This Course?
This course will prepare you to participate in the reinforcement learning research community. You will also have the opportunity to learn from two of the foremost experts in this field of research, Profs. Charles Isbell and Michael Littman.
Prerequisites and Requirements
Before taking this course, you should have taken a graduate-level machine-learning course and should have had some exposure to reinforcement learning from a previous course or seminar in computer science.
Additionally, you will be programming extensively in Java during this course. If you are not familiar with Java, we recommend you review Udacity's Object Oriented Programming in Java course materials to get up to speed beforehand.
What Will I Learn?
P4: Train a Smartcab to Drive
In the not-so-distant future, taxicab companies across the United States no longer employ human drivers to operate their fleet of vehicles. Instead, the taxicabs are operated by self-driving agents — known as smartcabs — to transport people from one location to another within the cities those companies operate. In major metropolitan areas, such as Chicago, New York City, and San Francisco, an increasing number of people have come to rely on smartcabs to get to where they need to go as safely and efficiently as possible. Although smartcabs have become the transport of choice, concerns have arose that a self-driving agent might not be as safe or efficient as human drivers, particularly when considering city traffic lights and other vehicles. To alleviate these concerns, your task as an employee for a national taxicab company is to use reinforcement learning techniques to construct a demonstration of a smartcab operating in real-time to prove that both safety and efficiency can be achieved.
- Reinforcement Learning Basics
- Introduction to BURLAP
- TD Lambda
- Convergence of Value and Policy Iteration
- Reward Shaping
- Partially Observable MDPs
- Topics in Game Theory
- Further Topics in RL Models