{"id":30957269,"url":"https://github.com/mvrahden/reinforce-js","last_synced_at":"2025-09-11T13:45:11.647Z","repository":{"id":57352592,"uuid":"112175454","full_name":"mvrahden/reinforce-js","owner":"mvrahden","description":"[INACTIVE] A collection of various machine learning solver. The library is an object-oriented approach (baked with Typescript) and tries to deliver simplified interfaces that make using the algorithms pretty simple.","archived":false,"fork":false,"pushed_at":"2018-06-29T10:50:50.000Z","size":173,"stargazers_count":31,"open_issues_count":3,"forks_count":7,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-09-08T05:55:49.179Z","etag":null,"topics":["ai","artificial-intelligence","deep-learning","deep-q-network","deep-reinforcement-learning","deepmind","deterministic-policy-gradients","dqn","dqn-solver","learning-agents","machine-learning-solver","neural-network","npm","reinforce-js","reinforcement","reinforcement-learning","solver","td-solver","temporal-differencing-learning","typescript"],"latest_commit_sha":null,"homepage":"https://npmjs.com/package/reinforce-js","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mvrahden.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-11-27T09:27:41.000Z","updated_at":"2024-08-22T07:57:39.000Z","dependencies_parsed_at":"2022-09-16T08:10:50.163Z","dependency_job_id":null,"html_url":"https://github.com/mvrahden/reinforce-js","commit_stats":null,"previous_names":["mvrahden/reinforce-ts"],"tags_count":47,"template":false,"template_full_name":null,"purl":"pkg:github/mvrahden/reinforce-js","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mvrahden%2Freinforce-js","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mvrahden%2Freinforce-js/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mvrahden%2Freinforce-js/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mvrahden%2Freinforce-js/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mvrahden","download_url":"https://codeload.github.com/mvrahden/reinforce-js/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mvrahden%2Freinforce-js/sbom","scorecard":{"id":669618,"data":{"date":"2025-08-11","repo":{"name":"github.com/mvrahden/reinforce-js","commit":"7cb04a67e662cd435cec43063e89b933c282fdc9"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":2.8,"checks":[{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Code-Review","score":0,"reason":"Found 0/30 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"SAST","score":0,"reason":"no SAST tool detected","details":["Warn: no pull requests merged into dev branch"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":8,"reason":"2 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-v6h2-p8h4-qcjw","Warn: Project is vulnerable to: GHSA-f8q6-p94x-37v3"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-21T19:30:18.243Z","repository_id":57352592,"created_at":"2025-08-21T19:30:18.244Z","updated_at":"2025-08-21T19:30:18.244Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274648319,"owners_count":25324299,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-11T02:00:13.660Z","response_time":74,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","artificial-intelligence","deep-learning","deep-q-network","deep-reinforcement-learning","deepmind","deterministic-policy-gradients","dqn","dqn-solver","learning-agents","machine-learning-solver","neural-network","npm","reinforce-js","reinforcement","reinforcement-learning","solver","td-solver","temporal-differencing-learning","typescript"],"created_at":"2025-09-11T13:45:09.002Z","updated_at":"2025-09-11T13:45:11.632Z","avatar_url":"https://github.com/mvrahden.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# reinforce-js\n[![Build Status](https://travis-ci.org/mvrahden/reinforce-js.svg?branch=master)](https://travis-ci.org/mvrahden/reinforce-js)\n[![Build status](https://ci.appveyor.com/api/projects/status/wpi6tohcfap8iei7/branch/master?svg=true)](https://ci.appveyor.com/project/mvrahden/reinforce-js/branch/master)\n[![js-google-style](https://img.shields.io/badge/code%20style-google-blue.svg)](https://google.github.io/styleguide/jsguide.html)\n\n[dqn-solver]: https://github.com/mvrahden/reinforce-js/blob/master/examples/dqn-solver.md\n[dqn-solver-src]: https://github.com/mvrahden/reinforce-js/blob/master/examples/dqn-solver-src.md\n[td-solver-src]: https://github.com/mvrahden/reinforce-js/blob/master/examples/td-solver-src.md\n\n**Call For Volunteers:** Due to my lack of time, I'm desperately looking for voluntary help. Should you be interested in building reinforcement agents (even though you're a newbie) and willing to develop this educational project a little further, please contact me :) There are some points on the agenda, that I'd still like to see implemented to make this project a nice library for abstract educational purposes.\n\n\u003e INACTIVE: Due to lack of time and help\n\n**reinforce-js** \u0026ndash; a collection of various simple reinforcement learning solver. This library is for **educational purposes** only. The library is an object-oriented approach and tries to deliver simplified interfaces that make using the algorithms pretty easy (baked with [Typescript](https://github.com/Microsoft/TypeScript)). More over it is an extension of _Andrej Karpathy's_ reinforcement learning library that implements several common RL algorithms.\nIn particular, the library currently includes:\n\n* **Deep Q-Learning** for Q-Learning with function approximation with Neural Networks ([DQNSolver Details][dqn-solver] and related [Google DeepMind Paper](https://www.nature.com/articles/nature14236))\n* **Dynamic Programming** methods\n* (Tabular) **Temporal Difference Learning** (SARSA/Q-Learning)\n* **Stochastic/Deterministic Policy Gradients** and Actor Critic architectures for dealing with continuous action spaces. (_very alpha, likely buggy or at the very least finicky and inconsistent_)\n\n## For Production Use\n\n### What does the Library offer?\n\nCurrently exposed Classes:\n\n#### DQN-Solver\n\n[Code-Example][dqn-solver-src] and [General Information][dqn-solver]\n\n* `DQNSolver` - Concrete **Deep Q-Learning Solver**\n  * This class is containing the main Deep Q-Learning algorithm of the *DeepMind* paper. On instantiation it needs to be configured with two configuration objects. It is an algorithm, which has minimum knowledge of its environment. The behavior of the algorithm can be tuned via its hyperparamters (`DQNOpt`).\n  * The Deep Q-Learning algorithm is designed to have a certain universality, since its reasoning is just depending on a environmental perception and an environmental feedback.\n  * The [learning-agents](https://mvrahden.github.io/learning-agents)-implementation shows that the DQNSolver can also be designed in such a way, that its agent has a maximum autonomy by establishing its own reward-scheme.\n* `DQNOpt` - Concrete options of `DQNSolver`\n  * This class is for the configuration of the DQNSolver. It holds all the **hyperparameter** for the DQNSolver. For the detailed initialization please see the [General Information][dqn-solver].\n* `DQNEnv` - Concrete environment of `DQNSolver`\n  * This class is for the configuration of the DQNSolver. It holds the boundary-measures of the environment, in which the DQNSolver should operate. For the detailed initialization please see the [General Information][dqn-solver].\n* **Example Application**: [Learning Agents](https://mvrahden.github.io/learning-agents) (GitHub Page)\n\n#### TD-Solver (not tested)\n\n[Code-Example][td-solver-src]\n\n* **TDSolver** - Concrete *Temporal Difference* Solver\n* **TDOpt** - Concrete *Options* for TDSolver creation\n* **TDEnv** - Concrete *Environment* for TDSolver creation\n\nPlanned to be implemented:\n\n* **DPSolver** - Concrete *Dynamic Programming* Solver\n* **DPOpt** - Concrete *Options* for DPSolver creation\n* **SimpleReinforcementSolver** - Concrete *Simple Reinforcement* Solver\n* **SimpleReinforcementOpt** - Concrete *Options* for SimpleReinforcementSolver creation\n* **RecurrentReinforcementSolver** - Concrete *Recurrent Reinforcement* Solver\n* **RecurrentReinforcementOpt** - Concrete *Options* for RecurrentReinforcementSolver creation\n* **DeterministPGSolver** - Concrete *Deterministic Policy Gradient* Solver\n* **DeterministPGOpt** - Concrete *Options* for DeterministPGSolver creation\n\n### How to install as a dependency:\n\nDownload available `@npm`: [reinforce-js](https://www.npmjs.com/package/reinforce-js)\n\nInstall via command line:\n\n```\nnpm install --save reinforce-js@latest\n```\n\nThe project directly ships with the transpiled Javascript code.\nAnd for TypeScript development it also contains Map-files and Declaration-files.\n\n### How to import?\n\nThese classes can be imported from this `npm` module, e.g.:\n```typescript\nimport { DQNSolver, DQNOpt, DQNEnv } from 'reinforce-js';\n```\n\nFor JavaScript usage `require` classes from this `npm` module as follows:\n```javascript\nconst DQNSolver = require('reinforce-js').DQNSolver;\nconst DQNOpt = require('reinforce-js').DQNOpt;\nconst DQNEnv = require('reinforce-js').DQNEnv;\n```\n\n### Example Application\n\nFor the DQN-Solver please visit [Learning Agents](https://mvrahden.github.io/learning-agents) (GitHub Page).\n\n## Community Contribution\n\nEverybody is more than welcome to contribute and extend the functionality!\n\nPlease feel free to contribute to this project as much as you wish to.\n\n1. clone from GitHub via `git clone https://github.com/mvrahden/reinforce-js.git`\n2. `cd` into the directory and `npm install` for initialization\n3. Try to `npm run test`. If everything is green, you're ready to go :sunglasses:\n\nBefore triggering a pull-request, please make sure that you've run all the tests via the *testing command*:\n\n```\nnpm run test\n```\n\nThis project relies on Visual Studio Codes built-in Typescript linting facilities. Let's follow primarily the [Google TypeScript Style-Guide](https://github.com/google/ts-style) through the included *tslint-google.json* configuration file.\n\n### Dependencies\n\nThis Library relies on the object-oriented _Deep Recurrent Neural Network_ library:\n\n* **GitHub**: [recurrent-js](https://github.com/mvrahden/recurrent-js)\n* **npm**: [recurrent-js](https://www.npmjs.com/package/recurrent-js)\n\n### Work in Progress\nPlease be aware that this repository is still _under construction_. Changes are likely to happen.\nThere are still classes to be added, e.g. *DPSolver*, *SimpleReinforcementSolver*, *RecurrentReinforcementSolver*, *DeterministPG* and their individual *Opts* and *Envs*\n\n## License\n\nAs of License-File: [MIT](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmvrahden%2Freinforce-js","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmvrahden%2Freinforce-js","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmvrahden%2Freinforce-js/lists"}