https://github.com/markub3327/dueling-dqn-with-autoencoder

Signal novelty detection as an intrinsic reward for robotics
https://github.com/markub3327/dueling-dqn-with-autoencoder

anomaly-detection autoencoder deep-neural-networks dueling-dqn keras reinforcement-learning robotics tensorflow

Last synced: 9 months ago
JSON representation

Signal novelty detection as an intrinsic reward for robotics

• Host: GitHub
• URL: https://github.com/markub3327/dueling-dqn-with-autoencoder
• Owner: markub3327
• License: mit
• Created: 2023-03-06T10:30:14.000Z (over 3 years ago)
• Default Branch: main
• Last Pushed: 2023-04-20T04:54:23.000Z (about 3 years ago)
• Last Synced: 2024-05-01T18:52:29.266Z (about 2 years ago)
• Topics: anomaly-detection, autoencoder, deep-neural-networks, dueling-dqn, keras, reinforcement-learning, robotics, tensorflow
• Language: Jupyter Notebook
• Homepage:
• Size: 1.62 MB
• Stars: 4
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# Signal novelty detection as an intrinsic reward for robotics

Paper: [Signal Novelty Detection as an Intrinsic Reward for Robotics](https://doi.org/10.3390/s23083985)

## AutoEncoder Architecture

AutoEncoder is one of the deep neural network methods used to detect anomalies in the signals received from the sensors on the robot. The principle is to train on the previously observed states from the experience replay buffer (RB), while the prediction is done over the observed state. In this way, early detection of new previously unobserved states is possible, which will help the agent to explore the environment and try to access previously unvisited states in the agent's state space.

![AutoEncoder](imgs/Figure_1.svg)

## Results

The results were obtained by the Dueling Deep Q Network (DQN) algorithm, which represented the agent. All tested tasks were learned without extrinsic reward from the environment using only this novelty intrinsic reward method.

* Acrobot-v1

![AutoEncoder](imgs/Figure_2.svg)

* CartPole-v1

![AutoEncoder](imgs/Figure_4.svg)

* LunarLander-v2

![AutoEncoder](imgs/Figure_6.svg)

* MountainCar-v0

![AutoEncoder](imgs/Figure_8.svg)