{"id":27944881,"url":"https://github.com/devslem/offlinerl","last_synced_at":"2025-05-07T12:54:59.419Z","repository":{"id":183078017,"uuid":"666817159","full_name":"DevSlem/OfflineRL","owner":"DevSlem","description":"A collection of notebooks to describe offline reinforcement learning algorithms and their implementations.","archived":false,"fork":false,"pushed_at":"2023-07-23T18:11:48.000Z","size":902,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-07T12:54:53.556Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DevSlem.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-07-15T17:11:28.000Z","updated_at":"2024-10-31T02:21:44.000Z","dependencies_parsed_at":"2023-07-22T18:44:50.347Z","dependency_job_id":null,"html_url":"https://github.com/DevSlem/OfflineRL","commit_stats":null,"previous_names":["devslem/offlinerl"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DevSlem%2FOfflineRL","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DevSlem%2FOfflineRL/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DevSlem%2FOfflineRL/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DevSlem%2FOfflineRL/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DevSlem","download_url":"https://codeload.github.com/DevSlem/OfflineRL/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252882791,"owners_count":21819154,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-05-07T12:54:56.224Z","updated_at":"2025-05-07T12:54:59.413Z","avatar_url":"https://github.com/DevSlem.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# OfflineRL\n\nA collection of notebooks to describe offline reinforcement learning algorithms and their implementations. \n\n\u003e This repository is based on: Levine, Sergey, et al. \"[Offline reinforcement learning: Tutorial, review, and perspectives on open problems.](https://arxiv.org/abs/2005.01643)\" arXiv preprint arXiv:2005.01643 (2020). \n\nI'm a beginner in offline RL, so I'm not sure if the descriptions and implementations are correct. But, I'm trying to make it as accurate as possible and describe in details. If you find an error or have any suggestions, please let me know by opening an issue.\n\n## Contents\n\n1. [Introduction](/chap1_introduction.ipynb) - Introduction to offline RL and off-policy methods.\n2. Coming soon...\n\n### Preliminary\n\nWe assume that the reader is familiar with the basic concepts of reinforcement learning, such as the Markov decision process (MDP), the Bellman equation, and the Q-learning algorithm, and with deep reinforcement learning, such as the deep Q-network (DQN), Policy Gradient. \n\nFurthermore, we assume that the reader is familiar with machine learning and deep learning concepts, such as the gradient descent method, the backpropagation algorithm, and the neural network.\n\n\u003e Note that if you're not familiar with off-policy methods, refer to the [1. Introduction](/chap1_introduction.ipynb) notebook.\n\n## Setup\n\nFirst of all, install [Python 3.10](https://www.python.org/downloads/) or higher. Then, install dependencies:\n\n```bash\npip install torch==2.0.1\npip install matplotlib==3.7.2\npip install gymnasium==0.29.0\npip install swig\npip install \"gymnasium[all]\"\npip install ipykernel\npip install ipywidgets\n```\n\n\u003e Note that the code is executed in Linux (Ubuntu 22.04). Dependencies may be different depending on your OS.\n\n\u003e Note that The above dependencies may change in the future as the code is updated.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevslem%2Fofflinerl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdevslem%2Fofflinerl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevslem%2Fofflinerl/lists"}