{"id":24811873,"url":"https://github.com/instadeepai/fastpbrl","last_synced_at":"2025-11-03T18:29:25.533Z","repository":{"id":37917852,"uuid":"506139976","full_name":"instadeepai/fastpbrl","owner":"instadeepai","description":"Vectorization techniques for fast population-based training.","archived":false,"fork":false,"pushed_at":"2022-08-12T09:22:14.000Z","size":109,"stargazers_count":55,"open_issues_count":1,"forks_count":3,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-04-06T11:24:28.734Z","etag":null,"topics":["jax","population-based-training","reinforcement-learning","vectorization"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2206.08888","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/instadeepai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-06-22T07:20:20.000Z","updated_at":"2024-12-16T21:31:38.000Z","dependencies_parsed_at":"2022-07-11T23:47:41.875Z","dependency_job_id":null,"html_url":"https://github.com/instadeepai/fastpbrl","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/instadeepai/fastpbrl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/instadeepai%2Ffastpbrl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/instadeepai%2Ffastpbrl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/instadeepai%2Ffastpbrl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/instadeepai%2Ffastpbrl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/instadeepai","download_url":"https://codeload.github.com/instadeepai/fastpbrl/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/instadeepai%2Ffastpbrl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279015312,"owners_count":26085684,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-13T02:00:06.723Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["jax","population-based-training","reinforcement-learning","vectorization"],"created_at":"2025-01-30T13:16:38.534Z","updated_at":"2025-10-13T13:31:49.048Z","avatar_url":"https://github.com/instadeepai.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Fast Population-Based Reinforcement Learning\n\n[![PyPI Python Version](https://img.shields.io/badge/python-3.8-blue.svg)](https://www.python.org/downloads/release/python-380/)\n[![Jax 0.2.26](https://img.shields.io/badge/jax-0.2.26-informational?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAAABGdBTUEAALGPC/xhBQAAACBjSFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAABmJLR0QA/wD/AP+gvaeTAAAAB3RJTUUH5gQYCxw3PnucPAAABvVJREFUWMPtlFuMnWUVhp/1ff+//32Y6cx0OsdOB9pOCwVHii0kiKByNMRABYzFGNMIeiGGkxEagomIViRiMeoFkYAgSJWGU1IVBC22FVI7gXYYaSsz0043s6e7M7NPs/f+j9/nRb3xjnJLn2Rdrqwnb9ZacJrTfNyRj9r4wPYSjrYd5aa+0BhxrFiuChXnVR2sQEhoCs38iBhVXDF3NuGq+moJ9GoBGxMz40/vT9lU3vkow5/9xww3bWznjp8sfLPqqweMFWkTS3fVQTUViDB3vCiTh/Nbu19ZuCd4Zmm7k/d+LaG6TESS2bl5PfHe1OOtF3vf0ac63No9/G7PAJdd4587X3d+HhpZAuhLQ0evq7larNJ+FOh3942pWrE+FL2a2TXUPXS51PW3saLjJNZj77wnpXx5halz6JQTeOyvq7l2uOK8sL/tVhGWe44lh0+uUmK/BZQQTFcoz1SwxvZo330wiUyvcpQjAoX8NLPHZrEJ7eFsdO8pCUzkx3jolQzjs6nPVn29MTaCiOGdxt/ZXtrFQqOJFcsF0TK+4gyjYyFcCD83XyjRvbiHhl/n6MQUVuzJBXRkSp2KwB/39bGmN2gtN/WdQSwdiVHMh0cZmd9LVRsa1qca1nmzdZKDfSdw0g5+JeT90QmiKGRq4hilqTI6rdFZNZnp8370oQX+si/PW5M5jledG4zlSldbUk7A4fouFsIavkB7a1uUTaUxLuweOkLQGkNomZue59D4Iab+k8fEFgw20+89duP2e/Z9aIE3J1pYP9gYKDX0bXEiKUFRDP9NMZgAUYhIZWnH4q0prY9bYxl1Z3ircwpBsNaSn/yAMAgQBDQjucH0b1/c9DM+lMATrxf5wZfbKdacmxuhOj82ikZSZWT+DRpRQMZxaEt5L286/8L70477+zCOaEZNXu0+yGxHAyetiSox4gjKU4G32P3F0W3Hp5dd14UDMH18jM3Pr+DcvmZnM1IdAK62ttNrFBIrjfETHj9+vrQ2SuRmR1uUwEw4Ss3O4bgpurK5yoqunh1va5acf865r00cPHpTEES9CEyuqdIz2ooJLYgl3Zt6rf2TLS+2rMqw7kufOCnwzFt9rB+stxwqpn/pR3KZCIkJmjI6X39420vtD2/b6qf+OZG9LUxkGVaIUmXKuRn6F61Ai9Dqetkiduv0iYJdme7hic4NHS3NFCgo+HkK5P8/Uvu/Apw/78vz6K4cK5cEG+qBuj6xeApLsbBApRLf+rUNszvezg+cWQvUDXEiKGWZdg9RkjpojacdFqxxA5P0KYGrykOsbvQCQqNR5+CBeWxiUSlBpQR/JryiLAsb5v5VfXrkhXdRB/JZLh1a6PdjdbsS66U0EDVpNAKUUssjUvf5kXxPYFFKC0l6hhNM4CQWz1pS1mLiCGUS1kZdXF1didUW4yRMHJmkMlvFBBa3zcHGFhMYL5iPbj9jY0//sZdP4NyzoYO7nqptCmNZJwKuNjhpl8EVPQiQ9djYDEWUCFaajJf+xlRxP4kxOEqTdV0qgU9Oe1zfdhEdURajLKWFEtMfFBArJBj29h1j6MhiWpopMKyrT/mbNu64d4v6/h+q66u+vsWPRaJEMFbwjYs4Hq6XIhZXN2OlokRxtPEu++fGiKKISqXCXGmeUqVCY6HGRfFyLvbPwsZgIsPE6CR+xcfxNFPdZZ7uHmHvQB7lKhCkOR3csv3Gn65X5bq9L07McmUTPJ1gkgRMgiLGlZg4NihriJhjrLqLRhJA2iOXzviu49Rja+pDrb3+zZkr0IkiJKJQLDBzpIgNLMkiyxsrJ6lon939k5T6faxvSRpmebMQ3OdMHT42OF9JsEDWE+IEwtiiFWQ8od60gBB7JWrZCiBYpafW9C/9btD0C7EkXGnP6Tu8e+LhA9XDgyKQFcjGIBbinNk51VXuTTfds0/oOjv7xrn2gzUnjyG2g07UaGyplJPfhJFtd7SQ8YSFpsFayKYVibEEoUXrFF39Q/jZAyzOZB7bd/f92ze/9BzL9njc+tC13L3uyTW1Gf+HxliymRSrWtrJ4Bzvz7Vutg5rwzj+VZwkzs7c+wz39LKq2FlOLXG36DvuunN8PB+fGSWsRYhTrjKOI8aCATG5tDLGihHEZGwrmcX1kYEud/PAF66uPfv1b/GnPc/y4HXbWdSdGfcr0SXW0GusTZyctp1duUdv2HHFk28cHBsv1moXRMacGagkNjkr57H0meFLBh+Ra27fS2ebHijVzHnGgKMh5QoN/+SnyKaFMLLECWhlpWNZ+ehTd105+vjunXzjM58H4PVtI1y+cR1bvvjccFCLz7DW2mxLyqw8q3MkCpPiV8uP8OnhT60uNRurDdZ2qCwb9PD+QlzNc5rTfOz5LxIAojXu7EOvAAAAJXRFWHRkYXRlOmNyZWF0ZQAyMDIyLTA0LTI0VDExOjI4OjU1KzAwOjAw2lOe1QAAACV0RVh0ZGF0ZTptb2RpZnkAMjAyMi0wNC0yNFQxMToyODo1NSswMDowMKsOJmkAAABXelRYdFJhdyBwcm9maWxlIHR5cGUgaXB0YwAAeJzj8gwIcVYoKMpPy8xJ5VIAAyMLLmMLEyMTS5MUAxMgRIA0w2QDI7NUIMvY1MjEzMQcxAfLgEigSi4A6hcRdPJCNZUAAAAASUVORK5CYII=)](https://jax.readthedocs.io/en/latest/)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit\u0026logoColor=white)](https://github.com/pre-commit/pre-commit)\n\nThis repository contains the code for the paper \"Fast Population-Based Reinforcement Learning on a Single Machine paper from\nInstaDeep\",\n[(Flajolet et al., 2022)](https://arxiv.org/pdf/2206.08888.pdf)\n:computer::zap:.\n\n## First-time setup\n\n### Install Docker\nThis code requires docker to run. To install docker please follow the online instructions \n[here](https://docs.docker.com/engine/install/ubuntu/). To enable the code to run on GPU, please\ninstall [Nvidia-docker](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html) (as well as\nthe latest nvidia driver available for your GPU).\n\n### Build and run a docker image\nOnce docker and docker Nvidia are installed, you can simply build the docker image with the following command:\n```bash\nmake build\n```\nand, once the image is built, start the container with:\n```bash\nmake dev_container\n```\nInside the container, you can run the `nvidia-smi` command to verify that your GPU is found.\n\n## Run preconfigured scripts\n\n### Replicate the experiments from the paper\n\nWe provide scripts and commands to replicate the experiments discussed in the paper. All these commands are\ndefined in the Makefile at the root of the repository.\n\nTo replicate the experiments corresponding to Figure 2 (where we measure the runtime of a population-wide\nupdate step with various implementations), run:\n```\nmake run_timing_sactd3\nmake run_timing_dqn\n```\n\nTo replicate the experiments discussed in Section 5 (which correspond to full training runs), run the following:\n```\nmake run_td3_cemrl\nmake run_td3_dvd\nmake run_td3_pbt\nmake run_sac_pbt\n```\n\nNote that dvd training runs are unstable and sometimes crash early on due to NaNs.\n\nWe use `tensorboard` to log metrics during the training run. The tensorboard command\nto run to visualize them is printed when the experiment starts.\n\n### Launch a test script\n\nRun the following command to start a short test which validates that the code in the training scripts is working\nas expected.\n```\nmake test_training_scripts\n```\n\n## Contributors\n\n\u003ca href=\"https://github.com/thomashirtz\" title=\"Thomas Hirtz\"\u003e\u003cimg src=\"https://github.com/thomashirtz.png\" height=\"auto\" width=\"50\" style=\"border-radius:50%\"\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/flajolet\" title=\"Arthur Flajolet\"\u003e\u003cimg src=\"https://github.com/flajolet.png\" height=\"auto\" width=\"50\" style=\"border-radius:50%\"\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/cibeah\" title=\"Claire Bizon Monroc\"\u003e\u003cimg src=\"https://github.com/cibeah.png\" height=\"auto\" width=\"50\" style=\"border-radius:50%\"\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/ranzenTom\" title=\"Thomas Pierrot\"\u003e\u003cimg src=\"https://github.com/ranzenTom.png\" height=\"auto\" width=\"50\" style=\"border-radius:50%\"\u003e\u003c/a\u003e\n\n## Citing this work\n\nIf you use the code or data in this package, please cite:\n\n```bibtex\n@inproceedings{flajolet2022fast,\n  title={Fast Population-Based Reinforcement Learning on a Single Machine},\n  author={Flajolet, Arthur and Monroc, Claire Bizon and Beguir, Karim and Pierrot, Thomas},\n  booktitle={International Conference on Machine Learning},\n  pages={6533--6547},\n  year={2022},\n  organization={PMLR}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finstadeepai%2Ffastpbrl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Finstadeepai%2Ffastpbrl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finstadeepai%2Ffastpbrl/lists"}