https://github.com/bergel/reinforcementlearning

Implementation of Reinforcement Learning (Q-Learning) in Pharo Smalltalk. The code is described in a chapter of the Agile Visualization book.
https://github.com/bergel/reinforcementlearning

machine-learning pharo pharo-smalltalk reinforcement-learning visualization

Last synced: 7 days ago
JSON representation

Implementation of Reinforcement Learning (Q-Learning) in Pharo Smalltalk. The code is described in a chapter of the Agile Visualization book.

• Host: GitHub
• URL: https://github.com/bergel/reinforcementlearning
• Owner: bergel
• License: mit
• Created: 2021-05-26T07:15:41.000Z (over 3 years ago)
• Default Branch: main
• Last Pushed: 2021-08-31T06:28:57.000Z (over 3 years ago)
• Last Synced: 2024-10-31T06:27:45.321Z (about 2 months ago)
• Topics: machine-learning, pharo, pharo-smalltalk, reinforcement-learning, visualization
• Language: Smalltalk
• Homepage:
• Size: 1.32 MB
• Stars: 0
• Watchers: 2
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Reinforcement Learning in Pharo

[![Tests](https://github.com/bergel/ReinforcementLearning/actions/workflows/runTests.yml/badge.svg)](https://github.com/bergel/ReinforcementLearning/actions/workflows/runTests.yml)

[![UML Class diagram](https://github.com/bergel/ReinforcementLearning/actions/workflows/visualizeClassDiagram.yml/badge.svg)](https://github.com/bergel/ReinforcementLearning/blob/main/ci_data/uml.png)

[![Coverage](https://raw.githubusercontent.com/bergel/ReinforcementLearning/main/ci_data/coverageBadge.svg)](https://github.com/bergel/ReinforcementLearning/blob/main/ci_data/coverage.png)

Reinforcement learning is a machine learning algorithm that (i) explores a graph made of states and actions, and (ii) identifies an optimal route in this graph based on a reward mechanism. The code contained in this repository implements the Q-Learning algorithm. Its implementation is simple and provides some visualizations. The UML Class diagram of this project is [available online](https://github.com/bergel/ReinforcementLearning/blob/main/ci_data/uml.png).

The content of this repository is designed to run on the [Pharo programming language](https://pharo.org). The code provided in this repository is part of the book titled _Agile Visualization with Pharo -- Crafting Interactive Visual Support Using Roassal_, published by APress.

-----

## Installation

The project is known to work on Pharo 9 and Pharo 10. Open a workspace and run the following:

```Smalltalk

[ Metacello new

    baseline: 'ReinforcementLearning';

    repository: 'github://bergel/ReinforcementLearning:main';

    load ] on: MCMergeOrLoadWarning do: [ :warning | warning load ]

```

------

## Screenshots

Here are some screenshots of the visualization to illustrates the execution of the Q-Learning algorithm. The example describes the classical scenario where a knight need to exit a map while avoiding monsters. Here is the solution that shows the path from the starting point (large blue dot) toward the exit (yellow cell), while avoiding monsters (light red cells):



Here is the Q-Table that indicates the rewards of each actions for all the states:



The culumated reward along the episodes:



Q-Learning, as any Reinforcement Learning algorithm, explores a graph composed of nodes (states) and edges (actions leading to a state transition). The graph is also visually represented: