{"id":24625472,"url":"https://github.com/seungjaeryanlee/rldb","last_synced_at":"2025-10-06T17:31:27.600Z","repository":{"id":57462135,"uuid":"176241314","full_name":"seungjaeryanlee/rldb","owner":"seungjaeryanlee","description":"Performances of Reinforcement Learning Agents","archived":false,"fork":false,"pushed_at":"2019-12-19T08:32:39.000Z","size":12791,"stargazers_count":53,"open_issues_count":1,"forks_count":2,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-09-18T18:46:29.875Z","etag":null,"topics":["database","python3","reinforcement-learning","state-of-the-art"],"latest_commit_sha":null,"homepage":"https://www.endtoend.ai/envs/gym/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/seungjaeryanlee.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-03-18T08:54:24.000Z","updated_at":"2025-04-27T13:03:18.000Z","dependencies_parsed_at":"2022-09-10T04:03:41.956Z","dependency_job_id":null,"html_url":"https://github.com/seungjaeryanlee/rldb","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/seungjaeryanlee/rldb","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seungjaeryanlee%2Frldb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seungjaeryanlee%2Frldb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seungjaeryanlee%2Frldb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seungjaeryanlee%2Frldb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/seungjaeryanlee","download_url":"https://codeload.github.com/seungjaeryanlee/rldb/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seungjaeryanlee%2Frldb/sbom","scorecard":{"id":813542,"data":{"date":"2025-08-11","repo":{"name":"github.com/seungjaeryanlee/rldb","commit":"8c471c4666d6210c68f3cb468e439a2b168c785d"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.5,"checks":[{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Code-Review","score":0,"reason":"Found 0/30 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 1 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-23T13:41:22.813Z","repository_id":57462135,"created_at":"2025-08-23T13:41:22.813Z","updated_at":"2025-08-23T13:41:22.813Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278648761,"owners_count":26021968,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-06T02:00:05.630Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","python3","reinforcement-learning","state-of-the-art"],"created_at":"2025-01-25T04:36:11.178Z","updated_at":"2025-10-06T17:31:25.864Z","avatar_url":"https://github.com/seungjaeryanlee.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"docs/rldb.png\" align=\"right\" width=\"20%\"/\u003e\n\n# rldb\n\n[![Build Status](https://travis-ci.com/seungjaeryanlee/rldb.svg?branch=master)](https://travis-ci.com/seungjaeryanlee/rldb)\n[![PyPI version](https://badge.fury.io/py/rldb.svg)](https://pypi.org/project/rldb/)\n![GitHub repo size](https://img.shields.io/github/repo-size/seungjaeryanlee/rldb.svg?color=brightgreen)\n\n![Environments tracked in rldb](https://img.shields.io/badge/environments-114-blue.svg)\n![Papers tracked in rldb](https://img.shields.io/badge/papers-23-blue.svg)\n![Repos tracked in rldb](https://img.shields.io/badge/repos-2-blue.svg)\n![Algorithms tracked in rldb](https://img.shields.io/badge/algorithms-60-blue.svg)\n![Entries tracked in rldb](https://img.shields.io/badge/entries-3380-blue.svg)\n\nDatabase of RL algorithms\n\n| Atari Space Invaders Scores | MuJoCo Walker2d Scores |\n|:-:|:-:|\n| ![Atari Space Invaders Scores](/docs/atari-space-invaders.png) | ![MuJoCo Walker2d Scores](/docs/mujoco-walker2d.png) |\n\n## Examples\n\nYou can use `rldb.find_all({})` to retrieve all existing entries in `rldb`.\n\n```python\nimport rldb\n\n\nall_entries = rldb.find_all({})\n```\n\nYou can also filter entries by specifying key-value pairs that the entry must match:\n\n```python\nimport rldb\n\n\ndqn_entries = rldb.find_all({'algo-nickname': 'DQN'})\nbreakout_noop_entries = rldb.find_all({\n    'env-title': 'atari-breakout',\n    'env-variant': 'No-op start',\n})\n```\n\nYou can also use `rldbl.find_one(filter_dict)` to find one entry that matches the key-value pair specified in `filter_dict`:\n\n```python\nimport rldb\nimport pprint\n\n\nentry = rldb.find_one({\n    'env-title': 'atari-pong',\n    'algo-title': 'Human',\n})\npprint.pprint(entry)\n```\n\n\n\u003cdetails\u003e\u003csummary\u003eOutput\u003c/summary\u003e\n\u003cp\u003e\n\n```python\n{\n    'algo-nickname': 'Human',\n    'algo-title': 'Human',\n    'env-title': 'atari-pong',\n    'env-variant': 'No-op start',\n    'score': 14.6,\n    'source-arxiv-id': '1511.06581',\n    'source-arxiv-version': 3,\n    'source-authors': [   'Ziyu Wang',\n                          'Tom Schaul',\n                          'Matteo Hessel',\n                          'Hado van Hasselt',\n                          'Marc Lanctot',\n                          'Nando de Freitas'],\n    'source-bibtex': '@article{DBLP:journals/corr/WangFL15,\\n'\n                     '    author    = {Ziyu Wang and\\n'\n                     '                 Nando de Freitas and\\n'\n                     '                 Marc Lanctot},\\n'\n                     '    title     = {Dueling Network Architectures for Deep '\n                     'Reinforcement Learning},\\n'\n                     '    journal   = {CoRR},\\n'\n                     '    volume    = {abs/1511.06581},\\n'\n                     '    year      = {2015},\\n'\n                     '    url       = {http://arxiv.org/abs/1511.06581},\\n'\n                     '    archivePrefix = {arXiv},\\n'\n                     '    eprint    = {1511.06581},\\n'\n                     '    timestamp = {Mon, 13 Aug 2018 16:48:17 +0200},\\n'\n                     '    biburl    = '\n                     '{https://dblp.org/rec/bib/journals/corr/WangFL15},\\n'\n                     '    bibsource = {dblp computer science bibliography, '\n                     'https://dblp.org}\\n'\n                     '}',\n    'source-nickname': 'DuDQN',\n    'source-title': 'Dueling Network Architectures for Deep Reinforcement '\n                    'Learning'\n}\n```\n\n\u003c/p\u003e\n\u003c/details\u003e\n\n## Entry Structure\n\nHere is the format of every entry:\n\n```python3\n{\n    # BASICS\n    \"source-title\": \"\",\n    \"source-nickname\": \"\",\n    \"source-authors\": [],\n\n    # MISC.\n    \"source-bibtex\": \"\",\n\n    # ALGORITHM\n    \"algo-title\": \"\",\n    \"algo-nickname\": \"\",\n    \"algo-source-title\": \"\",\n\n    # SCORE\n    \"env-title\": \"\",\n    \"score\": 0,\n}\n```\n\n- `source-title` is the full title of the source of the score: it can be the title of the paper or GitHub repository title. `source-nickname` is a popular nickname or acronym for that title if it exists, otherwise it is the same as `source-title`. \n- `source-authors` are a list of authors or contributors.\n- `source-bibtex` is a BibTeX-format citation.\n- `algo-title` is the full title of the algorithm used. `algo-nickname` is the nickname or acronym for that algorithm if it exists, otherwise it is the same as `algo-nickname`.\n- `algo-source-title` is the title of the source of the **algorithm**. It can and often is different from `source-title`.\n\nFor example, the **Space Invaders** score of **Asynchronous Advantage Actor Critic (A3C)** algorithm in the **Noisy Networks for Exploration (NoisyNet)** paper is represented by the following entry:\n\n```python3\n{\n    #  BASICS\n    \"source-title\": \"Noisy Networks for Exploration\",\n    \"source-nickname\": \"NoisyNet\",\n    \"source-authors\": [\n        \"Meire Fortunato\",\n        \"Mohammad Gheshlaghi Azar\",\n        \"Bilal Piot\",\n        \"Jacob Menick\",\n        \"Ian Osband\",\n        \"Alex Graves\",\n        \"Vlad Mnih\",\n        \"Remi Munos\",\n        \"Demis Hassabis\",\n        \"Olivier Pietquin\",\n        \"Charles Blundell\",\n        \"Shane Legg\",\n    ],\n\n    #  ARXIV\n    \"source-arxiv-id\": \"1706.10295\",\n    \"source-arxiv-version\": 2,\n\n    #  MISC.\n    \"source-bibtex\": \"\"\"\n@article{DBLP:journals/corr/FortunatoAPMOGM17,\n    author    = {Meire Fortunato and\n                 Mohammad Gheshlaghi Azar and\n                 Bilal Piot and\n                 Jacob Menick and\n                 Ian Osband and\n                 Alex Graves and\n                 Vlad Mnih and\n                 R{\\'{e}}mi Munos and\n                 Demis Hassabis and\n                 Olivier Pietquin and\n                 Charles Blundell and\n                 Shane Legg},\n    title     = {Noisy Networks for Exploration},\n    journal   = {CoRR},\n    volume    = {abs/1706.10295},\n    year      = {2017},\n    url       = {http://arxiv.org/abs/1706.10295},\n    archivePrefix = {arXiv},\n    eprint    = {1706.10295},\n    timestamp = {Mon, 13 Aug 2018 16:46:11 +0200},\n    biburl    = {https://dblp.org/rec/bib/journals/corr/FortunatoAPMOGM17},\n    bibsource = {dblp computer science bibliography, https://dblp.org}\n}\"\"\",\n\n    # ALGORITHM\n    \"algo-title\": \"Asynchronous Advantage Actor Critic\",\n    \"algo-nickname\": \"A3C\",\n    \"algo-source-title\": \"Asynchronous Methods for Deep Reinforcement Learning\",\n\n    # HYPERPARAMETERS\n    \"algo-frames\": 320 * 1000 * 1000,  # Number of frames\n\n    # SCORE\n    \"env-title\": \"atari-space-invaders\",\n    \"env-variant\": \"No-op start\",\n    \"score\": 1034,\n    \"stddev\": 49,\n}\n```\n\nNote that, as shown here, the entry can contain additional information.\n\n## Sources\n\n### Papers\n\n#### Deep Q-Networks\n\n- [x] [Playing Atari with Deep Reinforcement Learning (Mnih et al., 2013)](https://arxiv.org/abs/1312.5602)\n- [x] [Human-level control through deep reinforcement learning (Mnih et al., 2015)](https://deepmind.com/research/dqn/)\n- [x] [Deep Recurrent Q-Learning for Partially Observable MDPs (Hausknecht and Stone, 2015)](https://arxiv.org/abs/1507.06527)\n- [x] [Massively Parallel Methods for Deep Reinforcement Learning (Nair et al., 2015)](https://arxiv.org/abs/1507.04296)\n- [x] [Deep Reinforcement Learning with Double Q-learning (Hasselt et al., 2015)](https://arxiv.org/abs/1509.06461)\n- [x] [Prioritized Experience Replay (Schaul et al., 2015)](https://arxiv.org/abs/1511.05952)\n- [x] [Dueling Network Architectures for Deep Reinforcement Learning (Wang et al., 2015)](https://arxiv.org/abs/1511.06581)\n- [x] [Noisy Networks for Exploration (Fortunato et al., 2017)](https://arxiv.org/abs/1706.10295)\n- [x] [A Distributional Perspective on Reinforcement Learning (Bellemare et al., 2017)](https://arxiv.org/abs/1707.06887)\n- [x] [Rainbow: Combining Improvements in Deep Reinforcement Learning (Hessel et al., 2017)](https://arxiv.org/abs/1710.02298)\n- [x] [Distributional Reinforcement Learning with Quantile Regression (Dabney et al., 2017)](https://arxiv.org/abs/1710.10044)\n- [x] [Implicit Quantile Networks for Distributional Reinforcement Learning (Dabney et al., 2018)](https://arxiv.org/abs/1806.06923)\n- [x] [Distributed Prioritized Experience Replay (Horgan et al., 2018)](https://arxiv.org/abs/1803.00933)\n\n\n#### Policy Gradients\n\n- [x] [Asynchronous Methods for Deep Reinforcement Learning (Mnih et al., 2016)](https://arxiv.org/abs/1602.01783)\n- [x] [Trust Region Policy Optimization (Schulman et al., 2015)](https://arxiv.org/abs/1502.05477)\n- [x] [Proximal Policy Optimization Algorithms (Schulman et al., 2017)](https://arxiv.org/abs/1707.06347)\n- [x] [Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (Wu et al., 2017)](https://arxiv.org/abs/1708.05144)\n- [x] [Addressing Function Approximation Error in Actor-Critic Methods (Fujimoto et al., 2018)](https://arxiv.org/abs/1802.09477)\n- [x] [IMPALA: Importance Weighted Actor-Learner Architectures (Espeholt et al., 2018)](https://arxiv.org/abs/1802.01561)\n- [x] [The Reactor: A fast and sample-efficient Actor-Critic agent for Reinforcement Learning (Gruslys et al., 2017)](https://arxiv.org/abs/1704.04651)\n\n#### Exploration\n\n- [x] [Exploration by Random Network Distillation (Burda et al., 2018)](https://arxiv.org/abs/1810.12894)\n\n#### Misc.\n\n- [x] [Trust-PCL: An Off-Policy Trust Region Method for Continuous Control (Nachum et al., 2017)](https://arxiv.org/abs/1707.01891)\n\n### Repositories\n\n- [x] [OpenAI Baselines](https://github.com/openai/baselines)\n- [x] [RL Baselines Zoo](https://github.com/araffin/rl-baselines-zoo)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fseungjaeryanlee%2Frldb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fseungjaeryanlee%2Frldb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fseungjaeryanlee%2Frldb/lists"}