https://github.com/tristandeleu/pytorch-maml-rl

Reinforcement Learning with Model-Agnostic Meta-Learning in Pytorch
https://github.com/tristandeleu/pytorch-maml-rl

Last synced: 2 days ago
JSON representation

Reinforcement Learning with Model-Agnostic Meta-Learning in Pytorch

• Host: GitHub
• URL: https://github.com/tristandeleu/pytorch-maml-rl
• Owner: tristandeleu
• License: mit
• Created: 2018-04-28T17:11:19.000Z (almost 7 years ago)
• Default Branch: master
• Last Pushed: 2022-12-27T08:13:00.000Z (about 2 years ago)
• Last Synced: 2025-02-12T07:15:08.491Z (9 days ago)
• Language: Python
• Size: 4.72 MB
• Stars: 838
• Watchers: 15
• Forks: 163
• Open Issues: 29
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• Awesome-pytorch-list-CNVersion - pytorch-maml-rl - Agnostic Meta-Learning)。 (Pytorch & related libraries｜Pytorch & 相关库 / Other libraries｜其他库:)
• Awesome-pytorch-list - pytorch-maml-rl - Agnostic Meta-Learning in Pytorch. (Pytorch & related libraries / Other libraries:)

README

        
# Reinforcement Learning with Model-Agnostic Meta-Learning (MAML)

![HalfCheetahDir](https://raw.githubusercontent.com/tristandeleu/pytorch-maml-rl/master/_assets/halfcheetahdir.gif)

Implementation of Model-Agnostic Meta-Learning (MAML) applied on Reinforcement Learning problems in Pytorch. This repository includes environments introduced in ([Duan et al., 2016](https://arxiv.org/abs/1611.02779), [Finn et al., 2017](https://arxiv.org/abs/1703.03400)): multi-armed bandits, tabular MDPs, continuous control with MuJoCo, and 2D navigation task.

## Getting started

To avoid any conflict with your existing Python setup, and to keep this project self-contained, it is suggested to work in a virtual environment with [`virtualenv`](http://docs.python-guide.org/en/latest/dev/virtualenvs/). To install `virtualenv`:

```

pip install --upgrade virtualenv

```

Create a virtual environment, activate it and install the requirements in [`requirements.txt`](requirements.txt).

```

virtualenv venv

source venv/bin/activate

pip install -r requirements.txt

```

#### Requirements

 - Python 3.5 or above

 - PyTorch 1.3

 - Gym 0.15

## Usage

#### Training

You can use the [`train.py`](train.py) script in order to run reinforcement learning experiments with MAML. Note that by default, logs are available in [`train.py`](train.py) but **are not** saved (eg. the returns during meta-training). For example, to run the script on HalfCheetah-Vel:

```

python train.py --config configs/maml/halfcheetah-vel.yaml --output-folder maml-halfcheetah-vel --seed 1 --num-workers 8

```

#### Testing

Once you have meta-trained the policy, you can test it on the same environment using [`test.py`](test.py):

```

python test.py --config maml-halfcheetah-vel/config.json --policy maml-halfcheetah-vel/policy.th --output maml-halfcheetah-vel/results.npz --meta-batch-size 20 --num-batches 10  --num-workers 8

```

## References

This project is, for the most part, a reproduction of the original implementation [cbfinn/maml_rl](https://github.com/cbfinn/maml_rl/) in Pytorch. These experiments are based on the paper

> Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-Agnostic Meta-Learning for Fast Adaptation of Deep

Networks. _International Conference on Machine Learning (ICML)_, 2017 [[ArXiv](https://arxiv.org/abs/1703.03400)]

If you want to cite this paper

```

@article{finn17maml,

  author    = {Chelsea Finn and Pieter Abbeel and Sergey Levine},

  title     = {{Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks}},

  journal   = {International Conference on Machine Learning (ICML)},

  year      = {2017},

  url       = {http://arxiv.org/abs/1703.03400}

}

```

If you want to cite this implementation:

```

@misc{deleu2018mamlrl,

  author = {Tristan Deleu},

  title  = {{Model-Agnostic Meta-Learning for Reinforcement Learning in PyTorch}},

  note   = {Available at: https://github.com/tristandeleu/pytorch-maml-rl},

  year   = {2018}

}

```