https://github.com/tesslerc/gac
Code accompanying NeurIPS 2019 paper: "Distributional Policy Optimization - An Alternative Approach for Continuous Control"
https://github.com/tesslerc/gac
deep-reinforcement-learning neurips-2019 nips-2019 reinforcement-learning-algorithms
Last synced: 6 months ago
JSON representation
Code accompanying NeurIPS 2019 paper: "Distributional Policy Optimization - An Alternative Approach for Continuous Control"
- Host: GitHub
- URL: https://github.com/tesslerc/gac
- Owner: tesslerc
- License: gpl-3.0
- Created: 2019-05-09T17:40:28.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2019-12-17T20:24:32.000Z (over 6 years ago)
- Last Synced: 2024-09-06T21:46:29.233Z (almost 2 years ago)
- Topics: deep-reinforcement-learning, neurips-2019, nips-2019, reinforcement-learning-algorithms
- Language: Python
- Homepage: https://rlrl.net.technion.ac.il/2019/10/28/distributional-policy-optimization/
- Size: 764 KB
- Stars: 21
- Watchers: 2
- Forks: 6
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
This repo contains the code for the implementation of [Distributional Policy Optimization: An Alternative Approach for Continuous Control](https://arxiv.org/abs/1905.09855) (NeurIPS 2019). The theoretical framework is named DPO (Distributional Policy Optimization), whereas the Deep Learning approach to attaining it is named GAC (Generative Actor Critic).
# How to run
An example of how to run the code is provided below. The exact hyper-parameters per each domain are provided in the appendix of the paper.
main.py --visualize --env-name Hopper-v2 --training_actor_samples 32 --noise normal --batch_size 128 --noise_scale 0.2 --print --num_steps 1000000 --target_policy exponential --train_frequency 2048 --replay_size 200000
# Visualizing
You may visualize the run by adding the flag --visualize and starting a visdom server as follows:
python3.6 -m visdom.server
# Requirements
- mujoco - see explanation here: https://github.com/openai/mujoco-py
- gym
- numpy
- tqdm - for tracking experiment time left
- visdom - for visualization of the learning process
# Performance
The graphs below are taken from the paper and compare the performance of our proposed method to various baselines. The best performing method is the Autoregressive network.
