{"id":20749048,"url":"https://github.com/sash-a/es_pytorch","last_synced_at":"2025-04-28T12:10:21.083Z","repository":{"id":68173031,"uuid":"277367786","full_name":"sash-a/es_pytorch","owner":"sash-a","description":"High performance implementation of Deep neuroevolution in pytorch using mpi4py. Intended for use on HPC clusters","archived":false,"fork":false,"pushed_at":"2022-01-24T10:24:42.000Z","size":6665,"stargazers_count":26,"open_issues_count":7,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-30T09:31:26.204Z","etag":null,"topics":["ai","deep-neuroevolution","evolutionary-strategy","gym","mpi","mpi4py","neuroevolution","pytorch","reinforcement-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sash-a.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-07-05T19:16:24.000Z","updated_at":"2025-01-17T10:39:07.000Z","dependencies_parsed_at":null,"dependency_job_id":"71326636-8690-44cb-a468-60d985f22531","html_url":"https://github.com/sash-a/es_pytorch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sash-a%2Fes_pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sash-a%2Fes_pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sash-a%2Fes_pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sash-a%2Fes_pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sash-a","download_url":"https://codeload.github.com/sash-a/es_pytorch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251311332,"owners_count":21569009,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","deep-neuroevolution","evolutionary-strategy","gym","mpi","mpi4py","neuroevolution","pytorch","reinforcement-learning"],"created_at":"2024-11-17T08:20:45.064Z","updated_at":"2025-04-28T12:10:21.076Z","avatar_url":"https://github.com/sash-a.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## *Depreciated in faviour of my [much faster implementation in Julia](https://github.com/sash-a/ScalableES.jl/)*\n\n# Evolutionary strategies (deep neuroevolution) in pytorch using MPI\n\nThis implementation was made to be as simple and efficient as possible.  \nReference implementation can be found [here](https://github.com/uber-research/deep-neuroevolution) (in tensorflow using redis).  \nBased on two papers by uber AI labs [here](https://arxiv.org/abs/1712.06567) and [here](https://arxiv.org/abs/1712.06560).\n\n### Implementation\nThis was made for use on a cluster using MPI (however it can be used on a single machine). With regards to efficiency it \nonly scatters the positive fitness, negative fitness and noise index, per policy evaluated, to all other processes each generation. The noise is placed in a block \nof shared memory (on each node) for fast access and low memory footprint.\n\n### How to run\n* conda install: `conda install -n es_env -f env.yml`\n* example usages: `simple_example.py` `obj.py` `nsra.py`\n* example configs are in `config/`\n\n```\nconda activate es_env\nmpirun -np {num_procs} python simple_example.py configs/simple_conf.json\n```\n\nMake sure that you insert this line before you create your neural network as the initial creation sets the \ninitial parameters, which must be deterministic across all threads\n```\ntorch.random.manual_seed({seed})\n```\n\n### General info\n* In order to define a policy create a `src.nn.nn.BaseNet` (which is a simple extension of a `torch.nn.Module`) and \npass it to a `Policy` along with an `src.nn.optimizers.Optimizer` and float value for the noise standard deviation, an \nexample of this can be seen in `simple_example.py`.\n* If you wish to share the noise using shared memory and MPI, then instantiate the `NoiseTable` using \n`NoiseTable.create_shared(...)`, otherwise if you wish to use your own method of sharing noise/running \nsequentially then simply create the noise table using its constructor and pass your noise to it like this: \n`NoiseTable(my_noise, n_params)`\n* `NoiseTable.create_shared(...)` will throw an error if less than 2 MPI procs are used\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsash-a%2Fes_pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsash-a%2Fes_pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsash-a%2Fes_pytorch/lists"}