{"id":13679699,"url":"https://github.com/takuseno/d3rlpy","last_synced_at":"2026-10-19T11:51:29.830Z","repository":{"id":37881070,"uuid":"266369147","full_name":"takuseno/d3rlpy","owner":"takuseno","description":"An offline deep reinforcement learning library","archived":false,"fork":false,"pushed_at":"2025-09-10T23:06:16.000Z","size":22309,"stargazers_count":1686,"open_issues_count":60,"forks_count":269,"subscribers_count":23,"default_branch":"master","last_synced_at":"2026-10-18T18:18:45.291Z","etag":null,"topics":["deep-learning","deep-reinforcement-learning","offline-rl","pytorch"],"latest_commit_sha":null,"homepage":"https://takuseno.github.io/d3rlpy","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/takuseno.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":"ROADMAP.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"claude":null,"gemini":null,"cursor":null,"copilot":null,"dco":null,"cla":null,"disclosure":null}},"created_at":"2020-05-23T15:51:51.000Z","updated_at":"2026-10-10T07:02:45.000Z","dependencies_parsed_at":"2026-08-25T02:42:11.231Z","dependency_job_id":"6e10daf4-cc73-48f7-904d-a4ecfe0e2121","html_url":"https://github.com/takuseno/d3rlpy","commit_stats":{"total_commits":1213,"total_committers":15,"mean_commits":80.86666666666666,"dds":0.03462489694971149,"last_synced_commit":"a11b15db1e60d47296fdc7e7880fe1165803138a"},"previous_names":[],"tags_count":34,"template":false,"template_full_name":null,"purl":"pkg:github/takuseno/d3rlpy","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/takuseno%2Fd3rlpy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/takuseno%2Fd3rlpy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/takuseno%2Fd3rlpy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/takuseno%2Fd3rlpy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/takuseno","download_url":"https://codeload.github.com/takuseno/d3rlpy/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/takuseno%2Fd3rlpy/sbom","scorecard":{"id":866603,"data":{"date":"2025-08-11","repo":{"name":"github.com/takuseno/d3rlpy","commit":"4f0956ba8db469ebeeb617bb28060a8401a7dfa6"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":2.8,"checks":[{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Maintained","score":2,"reason":"1 commit(s) and 2 issue activity found in the last 90 days -- score normalized to 2","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Code-Review","score":1,"reason":"Found 3/30 approved changesets -- score normalized to 1","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/format_check.yml:1","Warn: no topLevel permission defined: .github/workflows/test.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/format_check.yml:9: update your workflow using https://app.stepsecurity.io/secureworkflow/takuseno/d3rlpy/format_check.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/format_check.yml:11: update your workflow using https://app.stepsecurity.io/secureworkflow/takuseno/d3rlpy/format_check.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/format_check.yml:15: update your workflow using https://app.stepsecurity.io/secureworkflow/takuseno/d3rlpy/format_check.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/test.yml:14: update your workflow using https://app.stepsecurity.io/secureworkflow/takuseno/d3rlpy/test.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/test.yml:16: update your workflow using https://app.stepsecurity.io/secureworkflow/takuseno/d3rlpy/test.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/test.yml:20: update your workflow using https://app.stepsecurity.io/secureworkflow/takuseno/d3rlpy/test.yml/master?enable=pin","Warn: containerImage not pinned by hash: docker/Dockerfile:1: pin your Docker image by updating pytorch/pytorch:2.5.1-cuda12.4-cudnn9-devel to pytorch/pytorch:2.5.1-cuda12.4-cudnn9-devel@sha256:14611869895df612b7b07227d5925f30ec3cd6673bad58ce3d84ed107950e014","Warn: pipCommand not pinned by hash: docker/Dockerfile:7-38","Warn: pipCommand not pinned by hash: docker/Dockerfile:7-38","Warn: pipCommand not pinned by hash: .github/workflows/format_check.yml:24","Warn: pipCommand not pinned by hash: .github/workflows/format_check.yml:25","Warn: pipCommand not pinned by hash: .github/workflows/format_check.yml:26","Warn: pipCommand not pinned by hash: .github/workflows/format_check.yml:27","Warn: pipCommand not pinned by hash: .github/workflows/test.yml:37","Warn: pipCommand not pinned by hash: .github/workflows/test.yml:38","Warn: pipCommand not pinned by hash: .github/workflows/test.yml:39","Warn: pipCommand not pinned by hash: .github/workflows/test.yml:40","Warn: downloadThenRun not pinned by hash: .github/workflows/test.yml:50","Info:   0 out of   6 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   1 downloadThenRun dependencies pinned","Info:   0 out of   1 containerImage dependencies pinned","Info:   0 out of  10 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Vulnerabilities","score":0,"reason":"17 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: PYSEC-2024-48 / GHSA-fj7x-q9j7-g6q6","Warn: Project is vulnerable to: GHSA-6rq9-53c3-f7vj","Warn: Project is vulnerable to: PYSEC-2023-38 / GHSA-ffxj-547x-5j7c","Warn: Project is vulnerable to: PYSEC-2025-10 / GHSA-h36j-8vv3-cj52","Warn: Project is vulnerable to: PYSEC-2024-223 / GHSA-h8wv-9h96-m4hr","Warn: Project is vulnerable to: PYSEC-2024-222 / GHSA-whh8-fjgc-qp73","Warn: Project is vulnerable to: PYSEC-2018-34 / GHSA-2fc2-6r4j-p65h","Warn: Project is vulnerable to: PYSEC-2021-856 / GHSA-5545-2q6w-2gh6","Warn: Project is vulnerable to: PYSEC-2019-108 / GHSA-9fq2-x9r6-wfmf","Warn: Project is vulnerable to: PYSEC-2018-33 / GHSA-cw6w-4rcx-xphc","Warn: Project is vulnerable to: PYSEC-2021-857 / GHSA-f7c7-j99h-c22f","Warn: Project is vulnerable to: GHSA-fpfv-jqm9-f5jm","Warn: Project is vulnerable to: PYSEC-2017-1 / GHSA-frgw-fgh6-9g52","Warn: Project is vulnerable to: GHSA-3749-ghw9-m3mg","Warn: Project is vulnerable to: PYSEC-2025-41 / GHSA-53q9-r3pm-6pq6","Warn: Project is vulnerable to: GHSA-887c-mr87-cxwp","Warn: Project is vulnerable to: GHSA-g7vv-2v7x-gj9p"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 3 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-24T02:49:08.063Z","repository_id":37881070,"created_at":"2025-08-24T02:49:08.063Z","updated_at":"2025-08-24T02:49:08.063Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":344872684,"owners_count":38356429,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-10-03T21:59:58.778Z","status":"online","status_checked_at":"2026-10-19T02:00:06.414Z","response_time":53,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","deep-reinforcement-learning","offline-rl","pytorch"],"created_at":"2024-08-02T13:01:08.488Z","updated_at":"2026-10-19T11:51:29.161Z","avatar_url":"https://github.com/takuseno.png","language":"Python","funding_links":[],"categories":["Reinforcement Learning","Python","Open Source Software/Implementations","Industry Strength Reinforcement Learning","💻 Codebases \u0026 Benchmarks","Libraries"],"sub_categories":["Others","Off-Policy Evaluation and Learning: Applications","Recent Advances (2023–2025)"],"readme":"\u003cp align=\"center\"\u003e\u003cimg align=\"center\" width=\"300px\" src=\"assets/logo.png\"\u003e\u003c/p\u003e\n\n# d3rlpy: An offline deep reinforcement learning library\n\n![test](https://github.com/takuseno/d3rlpy/workflows/test/badge.svg)\n[![Documentation Status](https://readthedocs.org/projects/d3rlpy/badge/?version=latest)](https://d3rlpy.readthedocs.io/en/latest/?badge=latest)\n[![codecov](https://codecov.io/gh/takuseno/d3rlpy/branch/master/graph/badge.svg?token=AQ02USKN6Y)](https://codecov.io/gh/takuseno/d3rlpy)\n[![Maintainability](https://api.codeclimate.com/v1/badges/c9162eb736d0b0f612d8/maintainability)](https://codeclimate.com/github/takuseno/d3rlpy/maintainability)\n![MIT](https://img.shields.io/badge/license-MIT-blue)\n\nd3rlpy is an offline deep reinforcement learning library for practitioners and researchers.\n\n```py\nimport d3rlpy\n\ndataset, env = d3rlpy.datasets.get_dataset(\"hopper-medium-v0\")\n\n# prepare algorithm\nsac = d3rlpy.algos.SACConfig(compile_graph=True).create(device=\"cuda:0\")\n\n# train offline\nsac.fit(dataset, n_steps=1000000)\n\n# train online\nsac.fit_online(env, n_steps=1000000)\n\n# ready to control\nactions = sac.predict(x)\n```\n\n- Documentation: https://d3rlpy.readthedocs.io\n- Paper: https://arxiv.org/abs/2111.03788\n\n\u003e [!IMPORTANT]\n\u003e v2.x.x introduces breaking changes. If you still stick to v1.x.x, please explicitly install previous versions (e.g. `pip install d3rlpy==1.1.1`).\n\n## Key features\n\n### :zap: Most Practical RL Library Ever\n- **offline RL**: d3rlpy supports state-of-the-art offline RL algorithms. Offline RL is extremely powerful when the online interaction is not feasible during training (e.g. robotics, medical).\n- **online RL**: d3rlpy also supports conventional state-of-the-art online training algorithms without any compromising, which means that you can solve any kinds of RL problems only with `d3rlpy`.\n\n### :beginner: User-friendly API\n- **zero-knowledge of DL library**: d3rlpy provides many state-of-the-art algorithms through intuitive APIs. You can become a RL engineer even without knowing how to use deep learning libraries.\n- **extensive documentation**: d3rlpy is fully documented and accompanied with tutorials and reproduction scripts of the original papers.\n\n### :rocket: Beyond State-of-the-art\n- **distributional Q function**: d3rlpy is the first library that supports distributional Q functions in the all algorithms. The distributional Q function is known as the very powerful method to achieve the state-of-the-performance.\n- **data-prallel distributed training**: d3rlpy is the first library that supports data-parallel distributed offline RL training, which allows you to scale up offline RL with multiple GPUs or nodes. See [example](examples/distributed_offline_training.py).\n\n\n## Installation\nd3rlpy supports Linux, macOS and Windows.\n\n### Dependencies\nInstalling d3rlpy package will install or upgrade the following packages to satisfy requirements:\n- torch\u003e=2.5.0\n- tqdm\u003e=4.66.3\n- gym\u003e=0.26.0\n- gymnasium==1.0.0\n- click\n- colorama\n- dataclasses-json\n- h5py\n- structlog\n- typing-extensions\n- scikit-learn\n\n### PyPI (recommended)\n[![PyPI version](https://badge.fury.io/py/d3rlpy.svg)](https://badge.fury.io/py/d3rlpy)\n![PyPI - Downloads](https://img.shields.io/pypi/dm/d3rlpy)\n```\n$ pip install d3rlpy\n```\n### Anaconda\n[![Anaconda-Server Badge](https://anaconda.org/conda-forge/d3rlpy/badges/version.svg)](https://anaconda.org/conda-forge/d3rlpy)\n[![Anaconda-Server Badge](https://anaconda.org/conda-forge/d3rlpy/badges/downloads.svg)](https://anaconda.org/conda-forge/d3rlpy)\n```\n$ conda install conda-forge/noarch::d3rlpy\n```\n\n### Docker\n![Docker Pulls](https://img.shields.io/docker/pulls/takuseno/d3rlpy)\n```\n$ docker run -it --gpus all --name d3rlpy takuseno/d3rlpy:latest bash\n```\n\n## Supported algorithms\n| algorithm | discrete control | continuous control |\n|:-|:-:|:-:|\n| Behavior Cloning (supervised learning) | :white_check_mark: | :white_check_mark: |\n| [Neural Fitted Q Iteration (NFQ)](https://link.springer.com/chapter/10.1007/11564096_32) | :white_check_mark: | :no_entry: |\n| [Deep Q-Network (DQN)](https://www.nature.com/articles/nature14236) | :white_check_mark: | :no_entry: |\n| [Double DQN](https://arxiv.org/abs/1509.06461) | :white_check_mark: | :no_entry: |\n| [Deep Deterministic Policy Gradients (DDPG)](https://arxiv.org/abs/1509.02971) | :no_entry: | :white_check_mark: |\n| [Twin Delayed Deep Deterministic Policy Gradients (TD3)](https://arxiv.org/abs/1802.09477) | :no_entry: | :white_check_mark: |\n| [Soft Actor-Critic (SAC)](https://arxiv.org/abs/1812.05905) | :white_check_mark: | :white_check_mark: |\n| [Batch Constrained Q-learning (BCQ)](https://arxiv.org/abs/1812.02900) | :white_check_mark: | :white_check_mark: |\n| [Bootstrapping Error Accumulation Reduction (BEAR)](https://arxiv.org/abs/1906.00949) | :no_entry: | :white_check_mark: |\n| [Conservative Q-Learning (CQL)](https://arxiv.org/abs/2006.04779) | :white_check_mark: | :white_check_mark: |\n| [Advantage Weighted Actor-Critic (AWAC)](https://arxiv.org/abs/2006.09359) | :no_entry: | :white_check_mark: |\n| [Critic Reguralized Regression (CRR)](https://arxiv.org/abs/2006.15134) | :no_entry: | :white_check_mark: |\n| [Policy in Latent Action Space (PLAS)](https://arxiv.org/abs/2011.07213) | :no_entry: | :white_check_mark: |\n| [TD3+BC](https://arxiv.org/abs/2106.06860) | :no_entry: | :white_check_mark: |\n| [Policy Regularization with Dataset Constraint (PRDC)](https://arxiv.org/abs/2306.06569) | :no_entry: | :white_check_mark: |\n| [Implicit Q-Learning (IQL)](https://arxiv.org/abs/2110.06169) | :no_entry: | :white_check_mark: |\n| [Calibrated Q-Learning (Cal-QL)](https://arxiv.org/abs/2303.05479) | :no_entry: | :white_check_mark: |\n| [ReBRAC](https://arxiv.org/abs/2305.09836) | :no_entry: | :white_check_mark: |\n| [Decision Transformer](https://arxiv.org/abs/2106.01345) | :white_check_mark: | :white_check_mark: |\n| [Q-learning Decision Transformer (QDT)](https://arxiv.org/abs/2209.03993) | :no_entry: | :white_check_mark: |\n| [Transformer Actor-Critic with Regularization (TACR)](https://www.ifaamas.org/Proceedings/aamas2023/pdfs/p2815.pdf) | :no_entry: | :white_check_mark: |\n| [Gato](https://arxiv.org/abs/2205.06175) | :construction: | :construction: |\n\n## Supported Q functions\n- [x] standard Q function\n- [x] [Quantile Regression](https://arxiv.org/abs/1710.10044)\n- [x] [Implicit Quantile Network](https://arxiv.org/abs/1806.06923)\n\n## Benchmark results\nd3rlpy is benchmarked to ensure the implementation quality.\nThe benchmark scripts are available [reproductions](https://github.com/takuseno/d3rlpy/tree/master/reproductions) directory.\nThe benchmark results are available [d3rlpy-benchmarks](https://github.com/takuseno/d3rlpy-benchmarks) repository.\n\n## Examples\n### MuJoCo\n\u003cp align=\"center\"\u003e\u003cimg align=\"center\" width=\"160px\" src=\"assets/mujoco_hopper.gif\"\u003e\u003c/p\u003e\n\n```py\nimport d3rlpy\n\n# prepare dataset\ndataset, env = d3rlpy.datasets.get_d4rl('hopper-medium-v0')\n\n# prepare algorithm\ncql = d3rlpy.algos.CQLConfig(compile_graph=True).create(device='cuda:0')\n\n# train\ncql.fit(\n    dataset,\n    n_steps=100000,\n    evaluators={\"environment\": d3rlpy.metrics.EnvironmentEvaluator(env)},\n)\n```\n\nSee more datasets at [d4rl](https://github.com/rail-berkeley/d4rl).\n\n### Atari 2600\n\u003cp align=\"center\"\u003e\u003cimg align=\"center\" width=\"160px\" src=\"assets/breakout.gif\"\u003e\u003c/p\u003e\n\n```py\nimport d3rlpy\n\n# prepare dataset (1% dataset)\ndataset, env = d3rlpy.datasets.get_atari_transitions(\n    'breakout',\n    fraction=0.01,\n    num_stack=4,\n)\n\n# prepare algorithm\ncql = d3rlpy.algos.DiscreteCQLConfig(\n    observation_scaler=d3rlpy.preprocessing.PixelObservationScaler(),\n    reward_scaler=d3rlpy.preprocessing.ClipRewardScaler(-1.0, 1.0),\n    compile_graph=True,\n).create(device='cuda:0')\n\n# start training\ncql.fit(\n    dataset,\n    n_steps=1000000,\n    evaluators={\"environment\": d3rlpy.metrics.EnvironmentEvaluator(env, epsilon=0.001)},\n)\n```\n\nSee more Atari datasets at [d4rl-atari](https://github.com/takuseno/d4rl-atari).\n\n\n### Online Training\n```py\nimport d3rlpy\nimport gym\n\n# prepare environment\nenv = gym.make('Hopper-v3')\neval_env = gym.make('Hopper-v3')\n\n# prepare algorithm\nsac = d3rlpy.algos.SACConfig(compile_graph=True).create(device='cuda:0')\n\n# prepare replay buffer\nbuffer = d3rlpy.dataset.create_fifo_replay_buffer(limit=1000000, env=env)\n\n# start training\nsac.fit_online(env, buffer, n_steps=1000000, eval_env=eval_env)\n```\n\n## Tutorials\nTry cartpole examples on Google Colaboratory!\n\n- offline RL tutorial: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/takuseno/d3rlpy/blob/master/tutorials/cartpole.ipynb)\n- online RL tutorial: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/takuseno/d3rlpy/blob/master/tutorials/online.ipynb)\n\nMore tutorial documentations are available [here](https://d3rlpy.readthedocs.io/en/stable/tutorials/index.html).\n\n## Contributions\nAny kind of contribution to d3rlpy would be highly appreciated!\nPlease check the [contribution guide](CONTRIBUTING.md).\n\n## Community\n| Channel | Link |\n|:-|:-|\n| Issues | [GitHub Issues](https://github.com/takuseno/d3rlpy/issues) |\n\n\u003e [!IMPORTANT]\n\u003e Please do NOT email to any contributors including the owner of this project to ask for technical support. Such emails will be ignored without replying to your message. Use GitHub Issues to report your problems.\n\n## Projects using d3rlpy\n| Project | Description |\n|:-:|:-|\n| [MINERVA](https://github.com/takuseno/minerva) | An out-of-the-box GUI tool for offline RL |\n| [SCOPE-RL](https://github.com/hakuhodo-technologies/scope-rl) | An off-policy evaluation and selection library |\n\n## Roadmap\nThe roadmap to the future release is available in [ROADMAP.md](ROADMAP.md).\n\n## Citation\nThe paper is available [here](https://arxiv.org/abs/2111.03788).\n```\n@article{d3rlpy,\n  author  = {Takuma Seno and Michita Imai},\n  title   = {d3rlpy: An Offline Deep Reinforcement Learning Library},\n  journal = {Journal of Machine Learning Research},\n  year    = {2022},\n  volume  = {23},\n  number  = {315},\n  pages   = {1--20},\n  url     = {http://jmlr.org/papers/v23/22-0017.html}\n}\n```\n\n## Acknowledgement\nThis work started as a part of [Takuma Seno](https://github.com/takuseno)'s Ph.D project at Keio University in 2020.\n\nThis work is supported by Information-technology Promotion Agency, Japan\n(IPA), Exploratory IT Human Resources Project (MITOU Program) in the fiscal\nyear 2020.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftakuseno%2Fd3rlpy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftakuseno%2Fd3rlpy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftakuseno%2Fd3rlpy/lists"}