{"id":13482293,"url":"https://github.com/mccallofthewild/markov","last_synced_at":"2025-04-21T18:31:23.453Z","repository":{"id":54513664,"uuid":"104426925","full_name":"mccallofthewild/markov","owner":"mccallofthewild","description":"⛓ A Crystal library for building Markov Chains and running Markov Processes.","archived":false,"fork":false,"pushed_at":"2021-03-22T09:18:03.000Z","size":142,"stargazers_count":21,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-01T15:10:28.344Z","etag":null,"topics":["markov","markov-chain","markov-model","markovian-processes","natural-language-processing"],"latest_commit_sha":null,"homepage":"https://mccallofthewild.github.io/markov/","language":"Crystal","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mccallofthewild.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-09-22T03:24:41.000Z","updated_at":"2023-04-29T06:35:30.000Z","dependencies_parsed_at":"2022-08-13T18:20:27.208Z","dependency_job_id":null,"html_url":"https://github.com/mccallofthewild/markov","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mccallofthewild%2Fmarkov","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mccallofthewild%2Fmarkov/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mccallofthewild%2Fmarkov/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mccallofthewild%2Fmarkov/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mccallofthewild","download_url":"https://codeload.github.com/mccallofthewild/markov/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250110948,"owners_count":21376561,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["markov","markov-chain","markov-model","markovian-processes","natural-language-processing"],"created_at":"2024-07-31T17:01:00.610Z","updated_at":"2025-04-21T18:31:23.070Z","avatar_url":"https://github.com/mccallofthewild.png","language":"Crystal","funding_links":[],"categories":["Algorithms and Data structures","Crystal"],"sub_categories":[],"readme":"# ⛓ Markov\n\nA Crystal library for building Markov Chains and running Markov Processes.\n\n[![Build Status](https://travis-ci.org/mccallofthewild/markov.svg?branch=master)](https://travis-ci.org/mccallofthewild/markov) [![Docs](https://img.shields.io/badge/docs-available-brightgreen.svg)](https://mccallofthewild.github.io/markov/) [![GitHub release](https://img.shields.io/github/release/mccallofthewild/markov.svg)](https://github.com/mccallofthewild/markov/releases)\n\n### _What is a Markov Chain?_\n\nA Markov Chain is essentially a mechanism for guessing probable future events based on a sample of past events.\nFor a great explanation, watch [this Khan Academy video](https://www.khanacademy.org/computing/computer-science/informationtheory/moderninfotheory/v/markov_chains).\n\n### Visit the [API Documentation](https://mccallofthewild.github.io/markov/) for a more in-depth look at the library's functionality.\n\n## Installation\n\nAdd this to your application's `shard.yml`:\n\n```yaml\ndependencies:\n  markov:\n    github: mccallofthewild/markov\n```\nIn your terminal, install Crystal dependencies with:\n```bash\n$ shards install\n```\nor \n```bash\n$ crystal deps\n```\n\n## Usage\nBegin by requiring the `Markov` module:\n```crystal\nrequire \"markov\"\n```\n### Basic -- Hello Markov\nA classic Markov text generator. This example will work well for small (array-sized) data sets.\n\nNOTE: `Markov::Chain` is a generic type which contains, receives and generates elements of `LinkType`.\n\nWe'll start with the sample text:\n```crystal \nexample_string = \"how much wood would a woodchuck chuck if a woodchuck could chuck wood\"\n```\nThere are several `Markov::Chain` constructors to choose from. The simplest one takes in a `LinkType` array of elements as `sample` and a `seed` of `LinkType`. `seed` is the element in `sample` you want to start the chain with. If not provided, a random element will be chosen.\n```crystal\nexample_arr = example_string.split(\" \") #=\u003e [\"how\",\"much\",\"wood\",\"would\",\"a\",\"woodchuck\",\"chuck\",\"if\",\"a\",\"woodchuck\",\"could\",\"chuck\",\"wood\"]\nseed = example_arr[0] #=\u003e \"how\"\n\nexample_chain = Markov::Chain(String).new sample: example_arr, seed: seed\n```\nFinally, we'll generate a probable sequence of elements with the `Markov::Chain#generate` method:\n```crystal\nputs example_chain.generate(10)\n```\nOutput:\n```bash\n[\"much\", \"wood\", \"would\", \"a\", \"woodchuck\", \"could\", \"chuck\", \"if\", \"a\", \"woodchuck\"]\n```\nThat's it! \n\nIf we wanted to get the elements one at a time, we could use the `Markov::Chain#next` method instead:\n```crystal\nputs example_chain.next #=\u003e \"much\"\nputs example_chain.next #=\u003e \"wood\"\nputs example_chain.next #=\u003e \"would\"\n```\n\n### Advanced \nThis implementation was built for larger data sets, with asynchronous input in mind.\n\nIn this example, we will create a `Markov::Chain` which can generate realistic movie titles.\n\nTo begin, we instantiate a `Markov::TransitionTable`. A `TransitionTable` is a mechanism for training and implementing Markov processes.\n\n```crystal \nexample_table = Markov::TransitionTable(String).new\n```\n\n#### `Markov::TransitionTable#add`\nNow we'll add a movie title using the `Markov::TransitionTable#add` method:\n\n```crystal\nmovie_one = %w(the great gatsby) # shortcut syntax for [\"the\",\"great\",\"gatsby\"]\n\nmovie_one.each do |word|\n  example_table.add(word)\nend\n```\n`Markov::TransitionTable#add` adds elements one at a time. At a deeper level, it's adding each new word to the previous word's [Transition Matrix](https://en.wikipedia.org/wiki/Stochastic_matrix) (`Markov::TransitionMatrix`).\n\n#### `Markov::TransitionTable#fill`\nFor syntactic sugar, if we have an array of elements, we can avoid looping through and `#add`-ing them by using the `Markov::TransitionTable#fill` method instead:\n\n```crystal\nmovie_one = %w(the great gatsby) # shortcut syntax for [\"the\",\"great\",\"gatsby\"]\n\nexample_table.fill table_with: movie_one\n```\n\n#### `Markov::TransitionTable#reset`\nA problem arises at this point:\n```crystal\nmovie_two = %w(great expectations)\nexample_table.fill table_with: movie_two\n```\nThe above code sequentially adds each word to the `TransitionTable`. But _The Great Gatsby_ and _Great Expectations_ are two separate movie titles; the \"Great\" at the beginning of _Great Expectations_ is not a probable transition from the \"Gatsby\" at the end of _The Great Gatsby_.\n\nTo solve this, use `Markov::TransitionTable#reset`. `#reset` clears the `TransitionTable`'s last added key, allowing us to separate titles like so:\n\n```crystal \nmovie_one = %w(the great gatsby)\nexample_table.fill table_with: movie_one\n\nexample_table.reset\nmovie_two = %w(great expectations)\nexample_table.fill table_with: movie_two\n\nexample_table.reset\nmovie_three = %w(the great escape)\nexample_table.fill table_with: movie_three\n```\n\n#### Implementing the `TransitionTable` with a `Markov::Chain`\nFinally, we can put the `TransitionTable` to use by passing it to a `Markov::Chain` constructor as `transition_table`:\n\n```crystal\nexample_chain = Markov::Chain(String).new transition_table: example_table, seed: \"great\"\n```\n\n#### Handling Dead Ends\nWith small and/or unique data sets, Markov chains are fallible to reaching dead ends. That is, they can often reach a point where there is nothing to transition to.\n\nWhen this happens in the `Markov` module, `Markov::Exceptions::EmptyTransitionMatrixException` is raised.\n\nFor example:\n\n```crystal\ndead_end_array = %w(some say the world will end in fire)\ndead_end_chain = Markov::Chain(String).new sample: dead_end_array, seed: \"fire\"\n# nothing comes after \"fire\", so the chain is at a dead end.\ndead_end_chain.next # raises `EmptyTransitionMatrixException`\n```\n\nTo prevent this, use the `Markov::Chain#on_dead_end` exception handler. \n\nThis method takes in a callback block with arguments of: the `Markov::Chain`'s `@transition_table`, the `Markov::Chain` instance, and the `EmptyTransitionMatrixException` raised.\n\nThe block's return value of `LinkType` fills in as the next item in the chain.\n\n```crystal\ndead_end_array = %w(some say the world will end in fire)\ndead_end_chain = Markov::Chain(String).new sample: dead_end_array, seed: \"fire\"\n\ndead_end_chain.on_dead_end do |transition_table, chain, exception|\n  \"some\"\nend\n\ndead_end_chain.next #=\u003e \"some\"\ndead_end_chain.next #=\u003e \"say\"\ndead_end_chain.next #=\u003e \"the\"\n```\n\n## Contributing\n\n1. Fork it ( https://github.com/mccallofthewild/markov/fork )\n2. Create your feature branch (git checkout -b my-new-feature)\n3. Commit your changes (git commit -am 'Add some feature')\n4. Push to the branch (git push origin my-new-feature)\n5. Create a new Pull Request\n\n## Contributors\n\n- [McCall Alexander](https://github.com/mccallofthewild) mccallofthewild - creator, maintainer\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmccallofthewild%2Fmarkov","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmccallofthewild%2Fmarkov","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmccallofthewild%2Fmarkov/lists"}