https://github.com/renan-siqueira/reinforcement-learning-frozen-lake
This project aims to explore the basic concepts of Reinforcement Learning using the FrozenLake environment from the OpenAI Gym library.
https://github.com/renan-siqueira/reinforcement-learning-frozen-lake
artificial-intelligence bellman-equation frozenlake gym machine-learning openai python q-learning reinforcement-learning sarsa
Last synced: 6 months ago
JSON representation
This project aims to explore the basic concepts of Reinforcement Learning using the FrozenLake environment from the OpenAI Gym library.
- Host: GitHub
- URL: https://github.com/renan-siqueira/reinforcement-learning-frozen-lake
- Owner: renan-siqueira
- License: mit
- Created: 2023-10-30T17:04:10.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2023-10-30T22:34:02.000Z (almost 2 years ago)
- Last Synced: 2024-01-29T00:32:44.379Z (over 1 year ago)
- Topics: artificial-intelligence, bellman-equation, frozenlake, gym, machine-learning, openai, python, q-learning, reinforcement-learning, sarsa
- Language: Python
- Homepage:
- Size: 89.8 KB
- Stars: 0
- Watchers: 1
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Reinforcement Learning with FrozenLake
This project aims to explore the basic concepts of Reinforcement Learning using the `FrozenLake` environment from the OpenAI Gym library.
## Concepts covered:
- Introduction to reinforcement learning: agents, environments, actions, rewards.
- Policies and value functions.
- Bellman's equation.
---
## GIF Comparison
See each agent in action (all trained for 10,000 episodes):
Random Agent
Random with Bellman
Q-Learning Agent
SARSA Agent
*The best performance after 10,000 episodes was from Q-Learning Agent
---
## Instructions:
### Environment Setup:
It's recommended to create a virtual environment to install the necessary dependencies and maintain the project's consistency:
```bash
python -m venv env
source env/bin/activate # On Windows use: env\Scripts\activate
```
After activating the virtual environment, install the dependencies through the `requirements.txt` file:
```bash
pip install -r requirements.txt
```
### Execution:
There are four main scripts to run:
- `random_agent.py`: Initial random agent implementation.
- `random_agent_bellman_function.py`: Random agent implementation with Bellman's function.
- `qlearning_agent.py`: Agent implemented using the Q-Learning algorithm.
- `sarsa_agent.py`: Agent implemented using the SARSA algorithm.
To run any of these scripts, use:
```bash
python .py
```
In addition, the project also contains an auxiliary script test_pygame.py that can be used to validate the installation of pygame:
```bash
python test_pygame.py
```
---
## Implemented Algorithms:
### Q-Learning:
A reinforcement learning technique where the agent learns to act in a way that maximizes the expected reward over time.
### SARSA (State-Action-Reward-State-Action):
An on-policy control technique where the agent learns to evaluate actions in the environment based on actual rewards received.
---
## Environment:
### FrozenLake (Stochastic):
The agent must navigate a frozen lake and reach the goal without falling into holes. In stochastic mode, there's a chance the agent might slip even when given a clear movement instruction.
---
## Credits:
This project was developed with the help of the OpenAI platform and based on tutorials and documentation from the OpenAI Gym library.