{"id":13689299,"url":"https://github.com/sawcordwell/pymdptoolbox","last_synced_at":"2026-04-05T01:31:27.470Z","repository":{"id":15879569,"uuid":"18620622","full_name":"sawcordwell/pymdptoolbox","owner":"sawcordwell","description":"Markov Decision Process (MDP) Toolbox for Python","archived":false,"fork":false,"pushed_at":"2023-06-04T08:37:14.000Z","size":759,"stargazers_count":550,"open_issues_count":32,"forks_count":252,"subscribers_count":23,"default_branch":"master","last_synced_at":"2025-12-15T05:29:26.075Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sawcordwell.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2014-04-10T02:08:50.000Z","updated_at":"2025-11-12T23:26:09.000Z","dependencies_parsed_at":"2022-07-20T22:54:50.837Z","dependency_job_id":"f8909dff-bdb6-4b40-9959-cae951f19068","html_url":"https://github.com/sawcordwell/pymdptoolbox","commit_stats":{"total_commits":300,"total_committers":3,"mean_commits":100.0,"dds":"0.010000000000000009","last_synced_commit":"7c96789cc80e280437005c12065cf70266c11636"},"previous_names":[],"tags_count":20,"template":false,"template_full_name":null,"purl":"pkg:github/sawcordwell/pymdptoolbox","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sawcordwell%2Fpymdptoolbox","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sawcordwell%2Fpymdptoolbox/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sawcordwell%2Fpymdptoolbox/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sawcordwell%2Fpymdptoolbox/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sawcordwell","download_url":"https://codeload.github.com/sawcordwell/pymdptoolbox/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sawcordwell%2Fpymdptoolbox/sbom","scorecard":{"id":802180,"data":{"date":"2025-08-11","repo":{"name":"github.com/sawcordwell/pymdptoolbox","commit":"7c96789cc80e280437005c12065cf70266c11636"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3,"checks":[{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Code-Review","score":0,"reason":"Found 2/30 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE.txt:0","Info: FSF or OSI recognized license: BSD 3-Clause \"New\" or \"Revised\" License: LICENSE.txt:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 2 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-23T10:47:33.141Z","repository_id":15879569,"created_at":"2025-08-23T10:47:33.142Z","updated_at":"2025-08-23T10:47:33.142Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31421869,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-05T00:25:07.052Z","status":"ssl_error","status_checked_at":"2026-04-05T00:25:05.923Z","response_time":60,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T15:01:41.850Z","updated_at":"2026-04-05T01:31:27.445Z","avatar_url":"https://github.com/sawcordwell.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"Markov Decision Process (MDP) Toolbox for Python\n================================================\n\n.. image:: https://travis-ci.org/sawcordwell/pymdptoolbox.svg?branch=master\n    :target: https://travis-ci.org/sawcordwell/pymdptoolbox\n    :alt: Build Status\n.. image:: https://coveralls.io/repos/sawcordwell/pymdptoolbox/badge.png\n    :target: https://coveralls.io/r/sawcordwell/pymdptoolbox\n    :alt: Code Coverage\n.. image:: https://pypip.in/py_versions/pymdptoolbox/badge.svg\n    :target: https://pypi.python.org/pypi/pymdptoolbox/\n    :alt: Supported Python versions\n.. image:: https://pypip.in/implementation/pymdptoolbox/badge.svg\n    :target: https://pypi.python.org/pypi/pymdptoolbox/\n    :alt: Supported Python implementations\n.. image:: https://pypip.in/license/pymdptoolbox/badge.svg\n    :target: https://pypi.python.org/pypi/pymdptoolbox/\n    :alt: License\n\nThe MDP toolbox provides classes and functions for the resolution of\ndescrete-time Markov Decision Processes. The list of algorithms that have been\nimplemented includes backwards induction, linear programming, policy iteration,\nq-learning and value iteration along with several variations.\n\nThe classes and functions were developped based on the\n`MATLAB \u003chttp://www.mathworks.com/products/matlab/\u003e`_\n`MDP toolbox \u003chttp://www.inra.fr/mia/T/MDPtoolbox/\u003e`_ by the\n`Biometry and Artificial Intelligence Unit \u003chttp://mia.toulouse.inra.fr/\u003e`_ of\n`INRA Toulouse \u003chttp://www.toulouse.inra.fr/\u003e`_ (France). There are editions\navailable for MATLAB, GNU Octave, Scilab and R.\nThe suite of MDP toolboxes are described in Chades I, Chapron G, Cros M-J,\nGarcia F \u0026 Sabbadin R (2014) 'MDPtoolbox: a multi-platform toolbox to solve\nstochastic dynamic programming problems', *Ecography*, vol. 37, no. 9, pp.\n916–920, doi `10.1111/ecog.00888 \u003chttp://dx.doi.org/10.1111/ecog.00888\u003e`_.\n\nFeatures\n--------\n  - Eight MDP algorithms implemented\n  - Fast array manipulation using `NumPy \u003chttp://www.numpy.org\u003e`_\n  - Full sparse matrix support using\n    `SciPy's sparse package \u003chttp://www.scipy.org/SciPyPackages/Sparse\u003e`_\n  - Optional linear programming support using\n    `cvxopt \u003chttp://abel.ee.ucla.edu/cvxopt/\u003e`_\n\nPLEASE NOTE: the linear programming algorithm is currently unavailable except\nfor testing purposes due to incorrect behaviour.\n\nInstallation\n------------\nNumPy and SciPy must be on your system to use this toolbox. Please have a\nlook at their documentation to get them installed. If you are installing\nonto Ubuntu or Debian and using Python 2 then this will pull in all the\ndependencies:\n\n  ``sudo apt-get install python-numpy python-scipy python-cvxopt``\n\nOn the other hand, if you are using Python 3 then cvxopt will have to be\ncompiled (pip will do it automatically). To get NumPy, SciPy and all the\ndependencies to have a fully featured cvxopt then run:\n\n  ``sudo apt-get install python3-numpy python3-scipy liblapack-dev libatlas-base-dev libgsl0-dev fftw-dev libglpk-dev libdsdp-dev``\n\nThe two main ways of downloading the package is either from the Python Package\nIndex or from GitHub. Both of these are explained below.\n\nPython Package Index (PyPI)\n~~~~~~~~~~~~~~~~~~~~~~~~~~~\n.. image:: https://pypip.in/download/pymdptoolbox/badge.svg\n    :target: https://pypi.python.org/pypi//pymdptoolbox/\n    :alt: Downloads\n.. image:: https://pypip.in/version/pymdptoolbox/badge.svg\n    :target: https://pypi.python.org/pypi/pymdptoolbox/\n    :alt: Latest Version\n.. image:: https://pypip.in/status/pymdptoolbox/badge.svg\n    :target: https://pypi.python.org/pypi/pymdptoolbox/\n    :alt: Development Status\n.. image:: https://pypip.in/wheel/pymdptoolbox/badge.svg\n    :target: https://pypi.python.org/pypi/pymdptoolbox/\n    :alt: Wheel Status\n.. image:: https://pypip.in/egg/pymdptoolbox/badge.svg\n    :target: https://pypi.python.org/pypi/pymdptoolbox/\n    :alt: Egg Status\n.. image:: https://pypip.in/format/pymdptoolbox/badge.svg\n    :target: https://pypi.python.org/pypi/pymdptoolbox/\n    :alt: Download format\n\nThe toolbox's PyPI page is https://pypi.python.org/pypi/pymdptoolbox/ and there\nare both zip and tar.gz archive options available that can be downloaded.\nHowever, I recommend using `pip \u003chttps://pip.pypa.io/en/latest/\u003e`_ to install\nthe toolbox if you have it available. Just type\n\n  ``pip install pymdptoolbox``\n\nat the console and it should take care of downloading and installing everything\nfor you. If you also want cvxopt to be automatically downloaded and installed\nso that you can help test the linear programming algorithm then type\n\n  ``pip install \"pymdptoolbox[LP]\"``\n\nIf you want it to be installed just for you rather than system wide then do\n\n  ``pip install --user pymdptoolbox``\n\nIf you downloaded the package manually from PyPI\n\n  1. Extract the `*.zip` or `*.tar.gz` archive\n\n     ``tar -xzvf pymdptoolbox-\u003cVERSION\u003e.tar.gz``\n\n     ``unzip pymdptoolbox-\u003cVERSION\u003e``\n\n  2. Change to the PyMDPtoolbox directory\n\n     ``cd pymdptoolbox``\n\n  3. Install via Setuptools, either to the root filesystem or to your home\n     directory if you don't have administrative access.\n\n     ``python setup.py install``\n\n     ``python setup.py install --user``\n\n     Read the\n     `Setuptools documentation \u003chttps://pythonhosted.org/setuptools/\u003e`_ for\n     more advanced information.\n\nOf course you can also use virtualenv or simply just unpack it to your working\ndirectory.\n\nGitHub\n~~~~~~\n\nClone the Git repository\n\n    ``git clone https://github.com/sawcordwell/pymdptoolbox.git``\n\nand then follow from step two above. To learn how to use Git then I reccomend\nreading the freely available `Pro Git book \u003chttp://git-scm.com/book\u003e`_ written\nby Scott Chacon and Ben Straub and published by Apress.\n\nQuick Use\n---------\nStart Python in your favourite way. The following example shows you how to\nimport the module, set up an example Markov decision problem using a discount\nvalue of 0.9, solve it using the value iteration algorithm, and then check the\noptimal policy.\n\n.. code:: python\n\n  import mdptoolbox.example\n  P, R = mdptoolbox.example.forest()\n  vi = mdptoolbox.mdp.ValueIteration(P, R, 0.9)\n  vi.run()\n  vi.policy # result is (0, 0, 0)\n\nDocumentation\n-------------\nDocumentation is available at http://pymdptoolbox.readthedocs.org/\nand also as docstrings in the module code.\nIf you use `IPython \u003chttp://ipython.scipy.org\u003e`_ to work with the toolbox,\nthen you can view the docstrings by using a question mark ``?``. For example:\n\n.. code:: python\n\n    import mdptoolbox\n    mdptoolbox?\u003cENTER\u003e\n    mdptoolbox.mdp?\u003cENTER\u003e\n    mdptoolbox.mdp.ValueIteration?\u003cENTER\u003e\n\nwill display the relevant documentation.\n\nContribute\n----------\nIssue Tracker: https://github.com/sawcordwell/pymdptoolbox/issues\n\nSource Code: https://github.com/sawcordwell/pymdptoolbox\n\nSupport\n-------\nUse the issue tracker.\n\nLicense\n-------\nThe project is licensed under the BSD license. See `\u003cLICENSE.txt\u003e`_ for details.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsawcordwell%2Fpymdptoolbox","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsawcordwell%2Fpymdptoolbox","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsawcordwell%2Fpymdptoolbox/lists"}