{"id":13528444,"url":"https://github.com/cpnota/autonomous-learning-library","last_synced_at":"2025-09-11T13:45:04.526Z","repository":{"id":40005620,"uuid":"156611625","full_name":"cpnota/autonomous-learning-library","owner":"cpnota","description":"A PyTorch library for building deep reinforcement learning agents.","archived":false,"fork":false,"pushed_at":"2024-03-17T21:46:51.000Z","size":6539,"stargazers_count":650,"open_issues_count":24,"forks_count":72,"subscribers_count":21,"default_branch":"develop","last_synced_at":"2025-09-02T18:54:33.065Z","etag":null,"topics":["a2c","advantage-actor-critic","ddpg","deep-deterministic-policy-gradient","deep-q-learning","deep-reinforcement-learning","dqn","dqn-pytorch","ppo","proximal-policy-optimization","reinforcement-learning","reinforcement-learning-algorithms","sac","soft-actor-critic"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cpnota.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-11-07T21:39:56.000Z","updated_at":"2025-09-02T16:56:39.000Z","dependencies_parsed_at":"2023-11-09T13:05:50.862Z","dependency_job_id":"46dbd4c9-f9ff-40bb-9834-6ea96c9075e1","html_url":"https://github.com/cpnota/autonomous-learning-library","commit_stats":{"total_commits":242,"total_committers":10,"mean_commits":24.2,"dds":"0.11157024793388426","last_synced_commit":"3d4c258be6d05a338991f8baed622816e19b2121"},"previous_names":[],"tags_count":25,"template":false,"template_full_name":null,"purl":"pkg:github/cpnota/autonomous-learning-library","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cpnota%2Fautonomous-learning-library","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cpnota%2Fautonomous-learning-library/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cpnota%2Fautonomous-learning-library/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cpnota%2Fautonomous-learning-library/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cpnota","download_url":"https://codeload.github.com/cpnota/autonomous-learning-library/tar.gz/refs/heads/develop","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cpnota%2Fautonomous-learning-library/sbom","scorecard":{"id":306682,"data":{"date":"2025-08-11","repo":{"name":"github.com/cpnota/autonomous-learning-library","commit":"f8073e51eb2462b8425dcb740f06e97ff10e2b1b"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.8,"checks":[{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Code-Review","score":0,"reason":"Found 0/24 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/python-package.yml:1","Info: topLevel 'contents' permission set to 'read': .github/workflows/python-publish.yml:11","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/python-package.yml:21: update your workflow using https://app.stepsecurity.io/secureworkflow/cpnota/autonomous-learning-library/python-package.yml/develop?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/python-package.yml:23: update your workflow using https://app.stepsecurity.io/secureworkflow/cpnota/autonomous-learning-library/python-package.yml/develop?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/python-publish.yml:20: update your workflow using https://app.stepsecurity.io/secureworkflow/cpnota/autonomous-learning-library/python-publish.yml/develop?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/python-publish.yml:22: update your workflow using https://app.stepsecurity.io/secureworkflow/cpnota/autonomous-learning-library/python-publish.yml/develop?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/python-publish.yml:34: update your workflow using https://app.stepsecurity.io/secureworkflow/cpnota/autonomous-learning-library/python-publish.yml/develop?enable=pin","Warn: pipCommand not pinned by hash: .github/workflows/python-package.yml:28","Warn: pipCommand not pinned by hash: .github/workflows/python-package.yml:29","Warn: pipCommand not pinned by hash: .github/workflows/python-publish.yml:27","Warn: pipCommand not pinned by hash: .github/workflows/python-publish.yml:28","Warn: pipCommand not pinned by hash: .github/workflows/python-publish.yml:29","Info:   0 out of   4 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   1 third-party GitHubAction dependencies pinned","Info:   0 out of   5 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 30 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-17T22:13:41.576Z","repository_id":40005620,"created_at":"2025-08-17T22:13:41.576Z","updated_at":"2025-08-17T22:13:41.576Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274648319,"owners_count":25324299,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-11T02:00:13.660Z","response_time":74,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["a2c","advantage-actor-critic","ddpg","deep-deterministic-policy-gradient","deep-q-learning","deep-reinforcement-learning","dqn","dqn-pytorch","ppo","proximal-policy-optimization","reinforcement-learning","reinforcement-learning-algorithms","sac","soft-actor-critic"],"created_at":"2024-08-01T07:00:19.257Z","updated_at":"2025-09-11T13:45:04.503Z","avatar_url":"https://github.com/cpnota.png","language":"Python","readme":"# The Autonomous Learning Library: A PyTorch Library for Building Reinforcement Learning Agents\n\nThe `autonomous-learning-library` is an object-oriented deep reinforcement learning (DRL) library for PyTorch.\nThe goal of the library is to provide the necessary components for quickly building and evaluating novel reinforcement learning agents,\nas well as providing high-quality reference implementations of modern DRL algorithms.\nThe full documentation can be found at the following URL: [https://autonomous-learning-library.readthedocs.io](https://autonomous-learning-library.readthedocs.io).\n\n## Tools for Building New Agents\n\nThe primary goal of the `autonomous-learning-library` is to facilitate the rapid development of new reinforcement learning agents by providing common tools for building and evaluation agents, such as:\n\n* A flexible function `Approximation` API that integrates features such as target networks, gradient clipping, learning rate schedules, model checkpointing, multi-headed networks, loss scaling, logging, and more.\n* Various memory buffers, including prioritized experience replay (PER), generalized advantage estimation (GAE), and more.\n* A `torch`-based `Environment` interface that simplies agent implementations by cutting out the `numpy` middleman.\n* Common wrappers and agent enhancements for replicating standard benchmarks.\n* [Slurm](https://slurm.schedmd.com/documentation.html) integration for running large-scale experiments.\n* Plotting and logging utilities including `tensorboard` integration and utilities for generating common plots.\n\nSee the [documentation](https://autonomous-learning-library.readthedocs.io) guide for a full description of the functionality provided by the `autonomous-learning-library`.\nAdditionally, we provide an [example project](https://github.com/cpnota/all-example-project) which demonstrates the best practices for building new agents.\n\n## High-Quality Reference Implementations\n\nThe `autonomous-learning-library` separates reinforcement learning agents into two modules: `all.agents`, which provides flexible, high-level implementations of many common algorithms which can be adapted to new problems and environments, and `all.presets` which provides specific instansiations of these agents tuned for particular sets of environments, including Atari games, classic control tasks, and MuJoCo/Pybullet robotics simulations. Some benchmark results showing results on-par with published results can be found below:\n\n![atari40](benchmarks/atari_40m.png)\n![atari40](benchmarks/mujoco_v4.png)\n![pybullet](benchmarks/pybullet_v0.png)\n\nAs of today, `all` contains implementations of the following deep RL algorithms:\n\n- [x] Advantage Actor-Critic (A2C)\n- [x] Categorical DQN (C51)\n- [x] Deep Deterministic Policy Gradient (DDPG)\n- [x] Deep Q-Learning (DQN) + extensions\n- [x] Proximal Policy Optimization (PPO)\n- [x] Rainbow (Rainbow)\n- [x] Soft Actor-Critic (SAC)\n\nIt also contains implementations of the following \"vanilla\" agents, which provide useful baselines and perform better than you may expect:\n\n- [x] Vanilla Actor-Critic\n- [x] Vanilla Policy Gradient\n- [x] Vanilla Q-Learning\n- [x] Vanilla Sarsa\n\n## Installation\n\nFirst, you will need a new version of [PyTorch](https://pytorch.org) (\u003e1.3), as well as [Tensorboard](https://pypi.org/project/tensorboard/).\nThen, you can install the core `autonomous-learning-library` through PyPi:\n\n```\npip install autonomous-learning-library\n```\n\nYou can also install all of the extras (such as Gym environments) using:\n\n```\npip install autonomous-learning-library[all]\n```\n\nFinally, you can install directly from this repository including the dev dependencies using:\n\n```\ngit clone https://github.com/cpnota/autonomous-learning-library.git\ncd autonomous-learning-library\npip install -e .[dev]\n```\n\n## Running the Presets\n\nIf you just want to test out some cool agents, the library includes several scripts for doing so:\n\n```\nall-atari Breakout a2c\n```\n\nYou can watch the training progress using:\n\n```\ntensorboard --logdir runs\n```\n\nand opening your browser to http://localhost:6006.\nOnce the model is fully trained, you can watch the trained model play using:\n\n```\nall-watch-atari Breakout \"runs/a2c_[id]/preset.pt\"\n```\n\nwhere `id` is the ID of your particular run. You should should be able to find it using tab completion or by looking in the `runs` directory.\nThe `autonomous-learning-library` also contains presets and scripts for classic control and PyBullet environments.\n\nIf you want to test out your own agents, you will need to define your own scripts.\nSome examples can be found in the `examples` folder).\nSee the [docs](https://autonomous-learning-library.readthedocs.io) for information on building your own agents!\n\n## Note\n\nThis library was built in the [Autonomous Learning Laboratory](http://all.cs.umass.edu) (ALL) at the [University of Massachusetts, Amherst](https://www.umass.edu).\nIt was written and is currently maintained by Chris Nota (@cpnota).\nThe views expressed or implied in this repository do not necessarily reflect the views of the ALL.\n\n## Citing the Autonomous Learning Library\n\nWe recommend the following citation:\n\n```\n@misc{nota2020autonomous,\n  author = {Nota, Chris},\n  title = {The Autonomous Learning Library},\n  year = {2020},\n  publisher = {GitHub},\n  journal = {GitHub repository},\n  howpublished = {\\url{https://github.com/cpnota/autonomous-learning-library}},\n}\n```\n","funding_links":[],"categories":["Libraries","Pytorch \u0026 related libraries｜Pytorch \u0026 相关库","Python","Pytorch \u0026 related libraries"],"sub_categories":["Other libraries｜其他库:","Other libraries:"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcpnota%2Fautonomous-learning-library","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcpnota%2Fautonomous-learning-library","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcpnota%2Fautonomous-learning-library/lists"}