{"id":13492848,"url":"https://github.com/openai/baselines","last_synced_at":"2025-05-14T07:07:09.633Z","repository":{"id":38838043,"uuid":"92238803","full_name":"openai/baselines","owner":"openai","description":"OpenAI Baselines: high-quality implementations of reinforcement learning algorithms","archived":false,"fork":false,"pushed_at":"2024-08-01T21:31:33.000Z","size":6620,"stargazers_count":16257,"open_issues_count":503,"forks_count":4905,"subscribers_count":645,"default_branch":"master","last_synced_at":"2025-05-07T06:18:20.406Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/openai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-05-24T01:58:13.000Z","updated_at":"2025-05-07T04:57:30.000Z","dependencies_parsed_at":"2023-02-13T18:35:17.879Z","dependency_job_id":"d7e016d0-3ba1-49a5-9022-e254df99d637","html_url":"https://github.com/openai/baselines","commit_stats":{"total_commits":294,"total_committers":120,"mean_commits":2.45,"dds":0.7653061224489796,"last_synced_commit":"ea25b9e8b234e6ee1bca43083f8f3cf974143998"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openai%2Fbaselines","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openai%2Fbaselines/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openai%2Fbaselines/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openai%2Fbaselines/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/openai","download_url":"https://codeload.github.com/openai/baselines/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254092648,"owners_count":22013290,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T19:01:09.836Z","updated_at":"2025-05-14T07:07:09.572Z","avatar_url":"https://github.com/openai.png","language":"Python","readme":"**Status:** Maintenance (expect bug fixes and minor updates)\n\n\u003cimg src=\"data/logo.jpg\" width=25% align=\"right\" /\u003e [![Build status](https://travis-ci.org/openai/baselines.svg?branch=master)](https://travis-ci.org/openai/baselines)\n\n# Baselines\n\nOpenAI Baselines is a set of high-quality implementations of reinforcement learning algorithms.\n\nThese algorithms will make it easier for the research community to replicate, refine, and identify new ideas, and will create good baselines to build research on top of. Our DQN implementation and its variants are roughly on par with the scores in published papers. We expect they will be used as a base around which new ideas can be added, and as a tool for comparing a new approach against existing ones. \n\n## Prerequisites \nBaselines requires python3 (\u003e=3.5) with the development headers. You'll also need system packages CMake, OpenMPI and zlib. Those can be installed as follows\n### Ubuntu \n    \n```bash\nsudo apt-get update \u0026\u0026 sudo apt-get install cmake libopenmpi-dev python3-dev zlib1g-dev\n```\n    \n### Mac OS X\nInstallation of system packages on Mac requires [Homebrew](https://brew.sh). With Homebrew installed, run the following:\n```bash\nbrew install cmake openmpi\n```\n    \n## Virtual environment\nFrom the general python package sanity perspective, it is a good idea to use virtual environments (virtualenvs) to make sure packages from different projects do not interfere with each other. You can install virtualenv (which is itself a pip package) via\n```bash\npip install virtualenv\n```\nVirtualenvs are essentially folders that have copies of python executable and all python packages.\nTo create a virtualenv called venv with python3, one runs \n```bash\nvirtualenv /path/to/venv --python=python3\n```\nTo activate a virtualenv: \n```\n. /path/to/venv/bin/activate\n```\nMore thorough tutorial on virtualenvs and options can be found [here](https://virtualenv.pypa.io/en/stable/) \n\n\n## Tensorflow versions\nThe master branch supports Tensorflow from version 1.4 to 1.14. For Tensorflow 2.0 support, please use tf2 branch.\n\n## Installation\n- Clone the repo and cd into it:\n    ```bash\n    git clone https://github.com/openai/baselines.git\n    cd baselines\n    ```\n- If you don't have TensorFlow installed already, install your favourite flavor of TensorFlow. In most cases, you may use\n    ```bash \n    pip install tensorflow-gpu==1.14 # if you have a CUDA-compatible gpu and proper drivers\n    ```\n    or \n    ```bash\n    pip install tensorflow==1.14\n    ```\n    to install Tensorflow 1.14, which is the latest version of Tensorflow supported by the master branch. Refer to [TensorFlow installation guide](https://www.tensorflow.org/install/)\n    for more details. \n\n- Install baselines package\n    ```bash\n    pip install -e .\n    ```\n\n### MuJoCo\nSome of the baselines examples use [MuJoCo](http://www.mujoco.org) (multi-joint dynamics in contact) physics simulator, which is proprietary and requires binaries and a license (temporary 30-day license can be obtained from [www.mujoco.org](http://www.mujoco.org)). Instructions on setting up MuJoCo can be found [here](https://github.com/openai/mujoco-py)\n\n## Testing the installation\nAll unit tests in baselines can be run using pytest runner:\n```\npip install pytest\npytest\n```\n\n## Training models\nMost of the algorithms in baselines repo are used as follows:\n```bash\npython -m baselines.run --alg=\u003cname of the algorithm\u003e --env=\u003cenvironment_id\u003e [additional arguments]\n```\n### Example 1. PPO with MuJoCo Humanoid\nFor instance, to train a fully-connected network controlling MuJoCo humanoid using PPO2 for 20M timesteps\n```bash\npython -m baselines.run --alg=ppo2 --env=Humanoid-v2 --network=mlp --num_timesteps=2e7\n```\nNote that for mujoco environments fully-connected network is default, so we can omit `--network=mlp`\nThe hyperparameters for both network and the learning algorithm can be controlled via the command line, for instance:\n```bash\npython -m baselines.run --alg=ppo2 --env=Humanoid-v2 --network=mlp --num_timesteps=2e7 --ent_coef=0.1 --num_hidden=32 --num_layers=3 --value_network=copy\n```\nwill set entropy coefficient to 0.1, and construct fully connected network with 3 layers with 32 hidden units in each, and create a separate network for value function estimation (so that its parameters are not shared with the policy network, but the structure is the same)\n\nSee docstrings in [common/models.py](baselines/common/models.py) for description of network parameters for each type of model, and \ndocstring for [baselines/ppo2/ppo2.py/learn()](baselines/ppo2/ppo2.py#L152) for the description of the ppo2 hyperparameters. \n\n### Example 2. DQN on Atari \nDQN with Atari is at this point a classics of benchmarks. To run the baselines implementation of DQN on Atari Pong:\n```\npython -m baselines.run --alg=deepq --env=PongNoFrameskip-v4 --num_timesteps=1e6\n```\n\n## Saving, loading and visualizing models\n\n### Saving and loading the model\nThe algorithms serialization API is not properly unified yet; however, there is a simple method to save / restore trained models. \n`--save_path` and `--load_path` command-line option loads the tensorflow state from a given path before training, and saves it after the training, respectively. \nLet's imagine you'd like to train ppo2 on Atari Pong,  save the model and then later visualize what has it learnt.\n```bash\npython -m baselines.run --alg=ppo2 --env=PongNoFrameskip-v4 --num_timesteps=2e7 --save_path=~/models/pong_20M_ppo2\n```\nThis should get to the mean reward per episode about 20. To load and visualize the model, we'll do the following - load the model, train it for 0 steps, and then visualize: \n```bash\npython -m baselines.run --alg=ppo2 --env=PongNoFrameskip-v4 --num_timesteps=0 --load_path=~/models/pong_20M_ppo2 --play\n```\n\n*NOTE:* Mujoco environments require normalization to work properly, so we wrap them with VecNormalize wrapper. Currently, to ensure the models are saved with normalization (so that trained models can be restored and run without further training) the normalization coefficients are saved as tensorflow variables. This can decrease the performance somewhat, so if you require high-throughput steps with Mujoco and do not need saving/restoring the models, it may make sense to use numpy normalization instead. To do that, set 'use_tf=False` in [baselines/run.py](baselines/run.py#L116). \n\n### Logging and vizualizing learning curves and other training metrics\nBy default, all summary data, including progress, standard output, is saved to a unique directory in a temp folder, specified by a call to Python's [tempfile.gettempdir()](https://docs.python.org/3/library/tempfile.html#tempfile.gettempdir).\nThe directory can be changed with the `--log_path` command-line option.\n```bash\npython -m baselines.run --alg=ppo2 --env=PongNoFrameskip-v4 --num_timesteps=2e7 --save_path=~/models/pong_20M_ppo2 --log_path=~/logs/Pong/\n```\n*NOTE:* Please be aware that the logger will overwrite files of the same name in an existing directory, thus it's recommended that folder names be given a unique timestamp to prevent overwritten logs.\n\nAnother way the temp directory can be changed is through the use of the `$OPENAI_LOGDIR` environment variable.\n\nFor examples on how to load and display the training data, see [here](docs/viz/viz.ipynb).\n\n## Subpackages\n\n- [A2C](baselines/a2c)\n- [ACER](baselines/acer)\n- [ACKTR](baselines/acktr)\n- [DDPG](baselines/ddpg)\n- [DQN](baselines/deepq)\n- [GAIL](baselines/gail)\n- [HER](baselines/her)\n- [PPO1](baselines/ppo1) (obsolete version, left here temporarily)\n- [PPO2](baselines/ppo2) \n- [TRPO](baselines/trpo_mpi)\n\n\n\n## Benchmarks\nResults of benchmarks on Mujoco (1M timesteps) and Atari (10M timesteps) are available \n[here for Mujoco](https://htmlpreview.github.com/?https://github.com/openai/baselines/blob/master/benchmarks_mujoco1M.htm) \nand\n[here for Atari](https://htmlpreview.github.com/?https://github.com/openai/baselines/blob/master/benchmarks_atari10M.htm) \nrespectively. Note that these results may be not on the latest version of the code, particular commit hash with which results were obtained is specified on the benchmarks page. \n\nTo cite this repository in publications:\n\n    @misc{baselines,\n      author = {Dhariwal, Prafulla and Hesse, Christopher and Klimov, Oleg and Nichol, Alex and Plappert, Matthias and Radford, Alec and Schulman, John and Sidor, Szymon and Wu, Yuhuai and Zhokhov, Peter},\n      title = {OpenAI Baselines},\n      year = {2017},\n      publisher = {GitHub},\n      journal = {GitHub repository},\n      howpublished = {\\url{https://github.com/openai/baselines}},\n    }\n\n","funding_links":[],"categories":["Frameworks and Libraries","Uncategorized","Python","Reinforcement Learning","AI Framework","Deep Reinforcement Learning","Algorithm Repos","时间序列","Benchmark Results","Projects","PyTorch Models","💻 Codebases \u0026 Benchmarks","强化学习","RL Frameworks \u0026 Implementations \u003ca name=\"frameworks\"\u003e\u003c/a\u003e","Codes","Meta-Learning"],"sub_categories":["Uncategorized","NLP","Quant","DCGAN","Others","网络服务_其他","Global AI Projects","Reinforcement Learning","Recent Advances (2023–2025)","Multi-task learning"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenai%2Fbaselines","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenai%2Fbaselines","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenai%2Fbaselines/lists"}