{"id":15540327,"url":"https://github.com/rm-hull/markov-chains","last_synced_at":"2025-09-01T01:32:30.343Z","repository":{"id":62434500,"uuid":"57884712","full_name":"rm-hull/markov-chains","owner":"rm-hull","description":"A library (and application examples) of stochastic discrete-time Markov Chains (DTMC) in Clojure","archived":false,"fork":false,"pushed_at":"2020-09-10T18:59:07.000Z","size":740,"stargazers_count":32,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-07-30T08:31:19.824Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://www.destructuring-bind.org/markov-chains","language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rm-hull.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-05-02T11:08:47.000Z","updated_at":"2025-04-28T04:34:15.000Z","dependencies_parsed_at":"2022-11-01T21:16:07.307Z","dependency_job_id":null,"html_url":"https://github.com/rm-hull/markov-chains","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/rm-hull/markov-chains","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rm-hull%2Fmarkov-chains","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rm-hull%2Fmarkov-chains/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rm-hull%2Fmarkov-chains/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rm-hull%2Fmarkov-chains/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rm-hull","download_url":"https://codeload.github.com/rm-hull/markov-chains/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rm-hull%2Fmarkov-chains/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273064229,"owners_count":25039259,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-31T02:00:09.071Z","response_time":79,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-02T12:13:32.145Z","updated_at":"2025-09-01T01:32:30.205Z","avatar_url":"https://github.com/rm-hull.png","language":"Clojure","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Markov Chains\n\n[![Build Status](https://travis-ci.org/rm-hull/markov-chains.svg?branch=master)](http://travis-ci.org/rm-hull/markov-chains)\n[![Coverage Status](https://coveralls.io/repos/rm-hull/markov-chains/badge.svg?branch=master)](https://coveralls.io/r/rm-hull/markov-chains?branch=master)\n[![Dependencies Status](https://versions.deps.co/rm-hull/markov-chains/status.svg)](https://versions.deps.co/rm-hull/markov-chains)\n[![Downloads](https://versions.deps.co/rm-hull/markov-chains/downloads.svg)](https://versions.deps.co/rm-hull/markov-chains)\n[![Clojars Project](https://img.shields.io/clojars/v/rm-hull/markov-chains.svg)](https://clojars.org/rm-hull/markov-chains)\n[![Maintenance](https://img.shields.io/maintenance/yes/2020.svg?maxAge=2592000)]()\n\nA library (and application examples) of stochastic discrete-time Markov Chains\n(DTMC) in Clojure: a random process that undergoes state transitions according\nto a pre-determined probability distribution matrix describing a state space.\n\n**A. K. Dewdney** describes the process succinctly in _The Tinkertoy Computer,\nand other machinations_:\n\n\u003e Simple in principle, Mark V. Shaney consists of two parts, a table builder\n\u003e and a text generator.  After scanning an input text and constructing the\n\u003e table of follower probabilities, Mark V. Shaney is ready to \"talk\". It begins\n\u003e with a single pair of words. The generating algorithm is simple:\n\n    repeat\n        r ← random\n        determine pair follower\n        output follower\n        first ← second\n        second ← word\n    until someone complains\n\n\u003e When a random number _r_ is selected, it determines a follower by the process\n\u003e of adding together the probabilities store for each of the words that follow\n\u003e the given pair until those probabilities first equal or exceed _r_. In this\n\u003e way, each follower word will be selected, in the long run, with a frequency\n\u003e that reflects its frequency in the original text. And in this way, the text\n\u003e so generated bears an eerie resemblance to the original:\n\u003e\n\u003e \"When I meet someone on a professional basis, I want them to shave their\n\u003e arms. While at a conference a few weeks back, I spent an interesting evening\n\u003e with a grain of salt. I wouldn't dare take them seriously! This brings me\n\u003e back to the brash people who dare others to do so or not. I love a good flame\n\u003e argument, probably more than anyone...\n\nAside from text generation, there are many other applications for using\nmarkov-chains for generating superficially realistic data sets, some of which\nwill be explored in this project:\n\n* **algorithmic music generation**, using [supercollider](http://supercollider.github.io/) and\n  [overtone](http://overtone.github.io/) where the states of the system represent notes, and the\n  probability matrix derived from an existing corpus of traditional music.\n\n* **stochastic L-systems**, modelling the growth processes of plants with probalistic\n  state rather than formalized rules.\n\n* **machine code generation**, using a simple [redcode VM](https://github.com/rm-hull/corewar),\n  we might be able to \"grow\" core-war contestants, find the most resilient and strongest contenders,\n  breed them with genetic algorithms, and eventually they will take over.\n\n### Pre-requisites\n\nYou will need [Leiningen](https://github.com/technomancy/leiningen) 2.8.1 or\nabove installed.\n\n### Building\n\nTo build and install the library locally, run:\n\n    $ cd markov-chains\n    $ lein test\n    $ lein install\n\n### Including in your project\n\nThere is a version hosted at [Clojars](https://clojars.org/rm-hull/markov-chains).\nFor leiningen include a dependency:\n\n```clojure\n[rm-hull/markov-chains \"0.1.1\"]\n```\n\nFor maven-based projects, add the following to your `pom.xml`:\n\n```xml\n\u003cdependency\u003e\n  \u003cgroupId\u003erm-hull\u003c/groupId\u003e\n  \u003cartifactId\u003emarkov-chains\u003c/artifactId\u003e\n  \u003cversion\u003e0.1.1\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n## Basic Usage\n\nThe library consists of two functions `markov-chains.core/collate` and\n`markov-chains.core/generate`.\n\n### Table building\n\n_collate_ takes a list of tokens and a number representing the desired\nremembered state. It will produce a map of state transitions and their\nprobabilities.  **IMPORTANT NOTE:** rather than representing the probabilities\nin the range 0..1, _collate_ just returns a tally of counts.  The _generate_\nfunction is aware of tally counts and makes the necessary adjustments to turn\nthem into probabilities.\n\n```clojure\n(use 'markov-chains.core)\n\n(def prob-matrix\n  (collate (seq \"AEAEAAAAAEAAAAAAEEAEEAEAEEEAAAEAAAA\") 1))\n\n(clojure.pprint/pprint prob-matrix)\n; =\u003e {(\\A) {\\E 8, \\A 14},\n;     (\\E) {\\A 8, \\E 4}}\n```\n\n### Output generator\n\n_generate_ takes a probability matrix, and produces an infinite stream of\ntokens, taking into consideration any remembered state. The stream is lazily\nevaluated, so it is important to use it with `take` or some other similar sink\nfunction.\n\n```clojure\n(apply str\n  (take 40\n    (generate prob-matrix)))\n; =\u003e \"EAAAAAAEAEAAEAEAEEAAAEEEEEEAAAAAAAAEEAAE\"\n```\n\n## Applications / Examples\n\nThe full code for these examples can be found in the [test/examples](https://github.com/rm-hull/markov-chains/blob/master/test/examples) directory.\n\n### Text Generation\n\nUsing some common (free) literature from [project gutenberg](https://www.gutenberg.org/):\n\n```clojure\n(use 'markov-chains.core)\n\n(def three-men-in-a-boat\n  (-\u003e\n    (slurp \"test/examples/resources/308.txt\")\n    (clojure.string/split #\"\\s+\")\n    (collate 2)))\n\n(-\u003e\u003e\n  (generate three-men-in-a-boat)\n  (take 60)\n  (clojure.string/join \" \"))\n; =\u003e \"it oozed over the blackness, and heard Harris’s sleepy voice asking\n;     where we drew near it, so they spread their handkerchiefs on the back\n;     of Harris and Harris’s friend as to avoid running down which, we managed\n;     to get out of here while this billing and cooing is on. We’ll go down\n;     to eat vegetables. He said they were demons.\"\n```\n\n### Algorithmic Music\n\nThe [Wikipedia page](https://en.wikipedia.org/wiki/Markov_chain#Music) on\nMarkov chains shows a simple 1st-order matrix as follows:\n\n| Note   | A    | C♯   | E♭   |\n|--------|-----:|-----:|-----:|\n| **A**  | 0.1  | 0.6  | 0.3  |\n| **C♯** | 0.25 | 0.05 | 0.7  |\n| **E♭** | 0.7  | 0.3  | 0    |\n\nWe can easily represent this in code as (using the same MIDI notation as used\nwith `overtone.live/note`):\n\n```clojure\n(use 'markov-chains.core)\n\n(def first-order-prob-matrix {\n  [:A4]  { :A4 0.1  :C#4 0.6  :Eb4 0.3 }\n  [:C#4] { :A4 0.25 :C#4 0.05 :Eb4 0.7 }\n  [:Eb4] { :A4 0.7  :C#4 0.3  :Eb4 0 }})\n\n(take 12 (generate first-order-prob-matrix))\n; =\u003e (:Eb4 :A4 :C#4 :Eb4 :A4 :Eb4 :A4 :C#4 :Eb4 :A4 :C#4 :Eb4)\n```\n\nObviously, the random nature means that the sequence will be different on each\nevaluation. _(Note to self: maybe should use a state monad for the random\nfunction)_\n\nThe keys on the matrix represent the current system state, along with any\nprevious state that also needs to be remembered: as this is a first-order\nmatrix, only the current state is used. As shown in the next example,\nhigher-order systems will need to consider past state as well in order to\nproduce results with a sense of phrasal structure, rather than the aimless\nwandering of a first-order system.\n\nThis generated MIDI sequence can be then fed into overtone (make sure you have\ninstalled supercollider first):\n\n```clojure\n(use 'overtone.live)\n\n(definst saw-wave [freq 440 attack 0.01 sustain 0.4 release 0.1 vol 0.4]\n  (* (env-gen (env-lin attack sustain release) 1 1 0 1 FREE)\n     (saw freq)\n     vol))\n\n(defn saw2 [music-note]\n  (saw-wave (midi-\u003ehz (note music-note))))\n\n(defn play-chord [a-chord]\n  (doseq [note a-chord] (saw2 note)))\n\n(defn chord-progression-time [notes]\n  (doseq [[time note] (map list (iterate (partial + 400) (now)) notes)]\n    (at time (play-chord (chord note :major)))))\n\n(chord-progression-time (take 24 (generate first-order-prob-matrix)))\n```\n\nFor a 2nd-order matrix, we take the previous state into consideration as well\nas the current state:\n\n| Note   | A    | D    | G    |\n|--------|-----:|-----:|-----:|\n| **AA** | 0.18 | 0.6  | 0.22 |\n| **AD** | 0.5  | 0.5  | 0    |\n| **AG** | 0.15 | 0.75 | 0.1  |\n| **DD** | 0    | 0    | 1    |\n| **DA** | 0.25 | 0    | 0.75 |\n| **DG** | 0.9  | 0.1  | 0    |\n| **GG** | 0.4  | 0.4  | 0.2  |\n| **GA** | 0.5  | 0.25 | 0.25 |\n| **GD** | 1    | 0    | 0    |\n\nRepresenting this as follows:\n\n```clojure\n(def second-order-prob-matrix {\n  [:A4 :A4] { :A4 0.18 :D4 0.6  :G4 0.22 }\n  [:A4 :D4] { :A4 0.5  :D4 0.5  :G4 0    }\n  [:A4 :G4] { :A4 0.15 :D4 0.75 :G4 0.1  }\n  [:D4 :D4] { :A4 0    :D4 0    :G4 1    }\n  [:D4 :A4] { :A4 0.25 :D4 0    :G4 0.75 }\n  [:D4 :G4] { :A4 0.9  :D4 0.1  :G4 0    }\n  [:G4 :G4] { :A4 0.4  :D4 0.4  :G4 0.2  }\n  [:G4 :A4] { :A4 0.5  :D4 0.25 :G4 0.25 }\n  [:G4 :D4] { :A4 1    :D4 0    :G4 0    }})\n\n(take 12 (generate second-order-prob-matrix))\n; =\u003e (:D4 :A4 :G4 :G4 :D4 :A4 :G4 :D4 :A4 :A4 :A4 :D4)\n```\n\nPlaying in overtone, gives a slighly less jarring rendition than the\nfirst-order system:\n\n```clojure\n(chord-progression-time (take 40 (generate second-order-prob-matrix)))\n```\n\n\u003e TODO: Row-row-row your boat example\n\n### Artificial Botany\n\nA realistic shrub can be generated with a formal\n[L-system](https://github.com/rm-hull/lindenmayer-systems) grammar by\nrepeatedly iterating the following rule and running the resultant operations\nthrough a [logo-style interpreter](https://github.com/rm-hull/turtle):\n\n    F → F[+F]F[-F][F]\n\nThis results in a fairly uniform shrub structure as follows:\n\n![SVG](https://rawgithub.com/rm-hull/markov-chains/master/test/examples/resources/formal_grammar_shrub.svg)\n\nBy taking the stream of operations that the L-system outputs (`:fwd`, `:left`,\n`:right`, etc.) and collating them into a probability matrix, we are now able\nto get the markov chains generator to emit a realistic stream of operations\nto pass to the logo drawing system.\n\n```clojure\n(def shrub\n  (-\u003e\u003e\n    (l-system \"F\" (\"^\") (\"F=F[+F]F[-F][F]\"))\n    (drop 5)\n    first\n    flatten))\n\n(def second-order-prob-matrix (collate shrub 2))\n\n(clojure.pprint/pprint second-order-prob-matrix)\n; =\u003e {(:right :fwd) {:restore 625, :save 156},\n;     (:save :fwd) {:restore 625, :save 156},\n;     (:restore :save) {:fwd 781, :left 156, :right 156},\n;     (:restore :restore) {:fwd 156, :save 218, :restore 93},\n;     (:restore :fwd) {:save 781},\n;     (:fwd :save) {:left 625, :right 625},\n;     (:left :fwd) {:restore 625, :save 156},\n;     (:save :right) {:fwd 781},\n;     (:save :left) {:fwd 781},\n;     (:fwd :restore) {:fwd 625, :save 875, :restore 375}}\n\n(spit \"markov_shrub.svg\"\n  (draw!\n    -\u003esvg\n    (augment (take 10000 (generate second-order-prob-matrix)))\n    [800 600]))\n```\n\nSee the [source](https://github.com/rm-hull/markov-chains/blob/master/test/examples/botany.clj)\nfor details on the `l-system` implementation. The generated SVG is random of\ncourse, but would look something like this:\n\n![SVG](https://rawgithub.com/rm-hull/markov-chains/master/test/examples/resources/markov_shrub.svg)\n\n### Population Modelling\n\nIn _Mathematische Abenteuer mit dem Computer_, **L. Råde** \u0026 **R.D. Nelson**\npose exercise 15.1 as follows:\n\n![SVG](https://rawgithub.com/rm-hull/markov-chains/master/doc/population-modelling.svg)\n\n\u003e In a certain country the movement of people between cities\n\u003e and the countryside is such that, in any one year, 20 per cent of\n\u003e those living in the countryside move to the cities, while 10 per\n\u003e cent of those living in the cities move to live in the countryside.\n\u003e\n\u003e Suppose that at the beginning of a year 5 million people live in the\n\u003e cities and 4 million live in the countryside. Write a program so that\n\u003e you can study the development year by year of the numbers of\n\u003e inhabitants in the cities and the countryside.\n\nWriting this program directly, might yield a solution as follows:\n\n```clojure\n(defn population-modelling [town country]\n  (cons\n    [town country]\n    (lazy-seq\n      (population-modelling\n        (+ (* town 0.9) (* country 0.2))\n        (+ (* town 0.1) (* country 0.8))))))\n\n(first (drop 1000 (population-modelling 5 4)))\n; =\u003e [6.000000000000029 3.000000000000017]\n```\n\nClearly the combined population of 9 million settles quite quicky into 6\nmillion townsfolk and 3 million in the countryside. What is interesting is\nthat any pair combination that adds upto 9 million ends up with the same\nvalue.\n\n```clojure\n(first (drop 1000 (population-modelling 8.5 0.5)))\n; =\u003e [6.000000000000021 3.0000000000000115]\n```\n\nActual population migration will never of course be exactly as the model\nsuggests, but we can use the probabilities in the exercise and create a\ncorresponding probability matrix:\n\n```clojure\n(def model {\n  [:town]    { :town 0.9 :country 0.1 }\n  [:country] { :town 0.2 :country 0.8 }})\n```\n\nAnd then some machinery to run the model and graph the results:\n\n```clojure\n(use '(incanter core stats datasets charts))\n\n(defn accumulator [data]\n  (reductions\n    (fn [acc value] (update acc value inc))\n    (into {} (map vector (distinct data) (repeat 0)))\n    data))\n\n(defn ratio [kw]\n (fn [m]\n  (double (/ (kw m) (apply + (vals m))))))\n\n(view\n  (line-chart\n    (iterate inc 0)\n    (-\u003e\u003e\n      (generate model)\n      (take 5000)\n      accumulator\n      next\n      (map (ratio :town))\n      (map (partial * 9)))\n    :x-label \"time\"\n    :y-label \"town dwellers\"))\n```\n\nRunning this a few times illustrates that whatever starting point, the ratio of town\ndwellers to countryfolk settles to 2 to 1 over time:\n\n![PNG](https://rawgithub.com/rm-hull/markov-chains/master/doc/pop1.png) ![PNG](https://rawgithub.com/rm-hull/markov-chains/master/doc/pop2.png) ![PNG](https://rawgithub.com/rm-hull/markov-chains/master/doc/pop3.png)\n\nUsing an even longer timeframe:\n\n```clojure\n(-\u003e\u003e\n  (generate model)\n  (take 1000000)\n  frequencies\n  ((ratio :town))\n  (* 9))\n; =\u003e 5.9997419999999995\n```\n\n## References\n\n* [The Algorithmic Beauty of Plants](http://algorithmicbotany.org/papers/abop/abop.pdf) [PDF]\n* [The Tinkertoy Computer and other machinations](https://www.amazon.co.uk/Tinkertoy-Computer-Other-Machinations-Recreations/dp/B019L537LS/)\n* [Row, row, row your boat...](https://gist.github.com/ctford/2877443)\n\n## License\n\n### The MIT License (MIT)\n\nCopyright (c) 2016 Richard Hull\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frm-hull%2Fmarkov-chains","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frm-hull%2Fmarkov-chains","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frm-hull%2Fmarkov-chains/lists"}