{"id":13401181,"url":"https://github.com/markusdumke/reinforcelearn","last_synced_at":"2025-05-07T16:28:02.465Z","repository":{"id":214207519,"uuid":"81085431","full_name":"markusdumke/reinforcelearn","owner":"markusdumke","description":"R Package for Reinforcement Learning","archived":false,"fork":false,"pushed_at":"2022-11-15T19:56:42.000Z","size":2397,"stargazers_count":32,"open_issues_count":7,"forks_count":6,"subscribers_count":7,"default_branch":"master","last_synced_at":"2024-07-21T07:31:56.024Z","etag":null,"topics":["machine-learning","r","r-package","reinforcement-learning"],"latest_commit_sha":null,"homepage":"https://markusdumke.github.io/reinforcelearn/","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/markusdumke.png","metadata":{"files":{"readme":"README.Rmd","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2017-02-06T12:34:42.000Z","updated_at":"2024-02-28T20:04:08.000Z","dependencies_parsed_at":"2023-12-26T17:31:45.775Z","dependency_job_id":"c6cfa927-fba5-42ab-868f-2f7c9f7d6999","html_url":"https://github.com/markusdumke/reinforcelearn","commit_stats":null,"previous_names":["markusdumke/reinforcelearn"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/markusdumke%2Freinforcelearn","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/markusdumke%2Freinforcelearn/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/markusdumke%2Freinforcelearn/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/markusdumke%2Freinforcelearn/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/markusdumke","download_url":"https://codeload.github.com/markusdumke/reinforcelearn/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252914828,"owners_count":21824438,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["machine-learning","r","r-package","reinforcement-learning"],"created_at":"2024-07-30T19:00:59.564Z","updated_at":"2025-05-07T16:28:02.445Z","avatar_url":"https://github.com/markusdumke.png","language":"R","funding_links":[],"categories":["R"],"sub_categories":[],"readme":"---\noutput: github_document\n---\n\n```{r, echo = FALSE}\nknitr::opts_chunk$set(\n  collapse = TRUE,\n  comment = \"#\u003e\",\n  message = FALSE,\n  fig.path = \"README-\"\n)\n```\n\n# Reinforcement Learning in R \u003cimg src=\"man/figures/logo.png\" align=\"right\" height=\"36\"/\u003e\n\n[![Travis-CI Build Status](https://travis-ci.org/markusdumke/reinforcelearn.svg?branch=master)](https://travis-ci.org/markusdumke/reinforcelearn)\n[![CRAN_Status_Badge](http://www.r-pkg.org/badges/version/reinforcelearn)](https://cran.r-project.org/package=reinforcelearn)\n[![Coverage Status](https://img.shields.io/codecov/c/github/markusdumke/reinforcelearn/master.svg?maxAge=600)](https://codecov.io/github/markusdumke/reinforcelearn?branch=master)\n\n```{r, include = FALSE}\nwriteLines(capture.output(devtools::session_info()), \"session_info.txt\")\n```\n\n\n### Documentation\n\n[Website](https://markusdumke.github.io/reinforcelearn)\n\n----\n\n### Installation\n\n```{r, eval = FALSE}\n# Install from CRAN.\ninstall.packages(\"reinforcelearn\")\n\n# Install development version from github.\ndevtools::install_github(\"markusdumke/reinforcelearn\")\n```\n\n----\n\n### Get started\n\nReinforcement Learning with the package `reinforcelearn` is as easy as\n\n```{r}\nlibrary(reinforcelearn)\n\nenv = makeEnvironment(\"windy.gridworld\")\nagent = makeAgent(\"softmax\", \"table\", \"qlearning\")\n\n# Run interaction for 10 episodes.\ninteract(env, agent, n.episodes = 10L)\n```\n\n----\n\n### Environments\n\nWith `makeEnvironment` you can create reinforcement learning environments.\n\n```{r}\n# Create environment.\nstep = function(self, action) {\n  state = list(mean = action + rnorm(1), sd = runif(1))\n  reward = rnorm(1, state[[1]], state[[2]])\n  done = FALSE\n  list(state, reward, done)\n}\n\nreset = function(self) {\n  state = list(mean = 0, sd = 1)\n  state\n}\n\nenv = makeEnvironment(\"custom\", step = step, reset = reset)\n```\n\nThe environment is an `R6` class with a set of attributes and methods.\nYou can interact with the environment via the `reset` and `step` method.\n\n```{r}\n# Reset environment.\nenv$reset()\n\n# Take action.\nenv$step(100)\n```\n\nThere are some predefined environment classes, e.g. `MDPEnvironment`, which allows you to create a Markov Decision Process by passing on state transition array and reward matrix, or `GymEnvironment`, where you can use toy problems from [OpenAI Gym](https://gym.openai.com/).\n\n```{r, eval = FALSE}\n# Create a gym environment.\n# Make sure you have Python, gym and reticulate installed.\nenv = makeEnvironment(\"gym\", gym.name = \"MountainCar-v0\")\n\n# Take random actions for 200 steps.\nenv$reset()\nfor (i in 1:200) {\n  action = sample(0:2, 1)\n  env$step(action)\n  env$visualize()\n}\nenv$close()\n```\n\nThis should open a window showing a graphical visualization of the environment during interaction.\n\nFor more details on how to create an environment have a look at the vignette: [Environments](https://markusdumke.github.io/reinforcelearn/articles/environments.html)\n\n----\n\n### Agents\n\nWith `makeAgent` you can set up a reinforcement learning agent to solve the environment, i.e. to find the best action in each time step.\n\nThe first step is to set up the policy, which defines which action to choose. For example we could use a uniform random policy.\n\n```{r}\n# Create the environment.\nenv = makeEnvironment(\"windy.gridworld\")\n\n# Create agent with uniform random policy.\npolicy = makePolicy(\"random\")\nagent = makeAgent(policy)\n\n# Run interaction for 10 steps.\ninteract(env, agent, n.steps = 10L)\n```\n\nIn this scenario the agent chooses all actions with equal probability and will not learn anything from the interaction. Usually we want the agent to be able to learn something. Value-based algorithms learn a value function from interaction with the environment and adjust the policy according to the value function. For example we could set up Q-Learning with a softmax policy.\n\n```{r}\n# Create the environment.\nenv = makeEnvironment(\"windy.gridworld\")\n\n# Create qlearning agent with softmax policy and tabular value function.\npolicy = makePolicy(\"softmax\")\nvalues = makeValueFunction(\"table\", n.states = env$n.states, n.actions = env$n.actions)\nalgorithm = makeAlgorithm(\"qlearning\")\nagent = makeAgent(policy, values, algorithm)\n\n# Run interaction for 10 steps.\ninteract(env, agent, n.episodes = 10L)\n```\n\n----\n\n### Vignettes\n\nAlso have a look at the vignettes for further examples.\n\n- [Environments](https://markusdumke.github.io/reinforcelearn/articles/environments.html)\n- [Agents](https://markusdumke.github.io/reinforcelearn/articles/agents.html)\n\n----\n\nLogo is a modification of https://www.r-project.org/logo/.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarkusdumke%2Freinforcelearn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarkusdumke%2Freinforcelearn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarkusdumke%2Freinforcelearn/lists"}