{"id":13686195,"url":"https://github.com/czekster/markov","last_synced_at":"2025-05-01T09:30:45.180Z","repository":{"id":37787513,"uuid":"470928531","full_name":"czekster/markov","owner":"czekster","description":"Materials for book: \"Markov Chains for programmers\"","archived":false,"fork":false,"pushed_at":"2022-06-24T12:58:45.000Z","size":21995,"stargazers_count":273,"open_issues_count":0,"forks_count":20,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-04-09T21:52:46.691Z","etag":null,"topics":["markov-chain","numerical-methods","programming-challenges","simulation"],"latest_commit_sha":null,"homepage":"https://czekster.github.io/markov/","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/czekster.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-03-17T09:33:20.000Z","updated_at":"2024-03-15T17:11:30.000Z","dependencies_parsed_at":"2022-08-28T22:40:59.059Z","dependency_job_id":null,"html_url":"https://github.com/czekster/markov","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/czekster%2Fmarkov","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/czekster%2Fmarkov/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/czekster%2Fmarkov/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/czekster%2Fmarkov/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/czekster","download_url":"https://codeload.github.com/czekster/markov/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251852609,"owners_count":21654436,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["markov-chain","numerical-methods","programming-challenges","simulation"],"created_at":"2024-08-02T15:00:22.649Z","updated_at":"2025-05-01T09:30:44.386Z","avatar_url":"https://github.com/czekster.png","language":"C","funding_links":[],"categories":["C"],"sub_categories":[],"readme":"# Book: \"Markov Chains for programmers\"\n[\u003cimg align=\"right\" src=\"cover-imgs/cover-thumb.png\"\u003e](MC-for-programmers2022.pdf)\nThis book was published by Self Publishers Worldwide (another publication by the _lazy panda_ collection), and authored by Ricardo M. Czekster.\n\nThe first edition was made public in April/2022. \n\n\"Markov Chains for programmers\" is devoted to programmers at any level wanting to understand more about the underpinnings of Markov Chains (MC) and basic solution methods.\n\nYou will find here supporting materials for the book such as **C programming code** and solutions, MATLAB scripts, PRISM models (CTMC/DTMC) for the examples provided in the book, which you may also [download here](MC-for-programmers2022.pdf).\n\nThis book is a companion to this [paper about Performance Evaluation](https://www.researchgate.net/publication/337623607_Introduction_to_Performance_Evaluation_of_Systems), published as a pre-print in [ResearchGate](https://www.researchgate.net/profile/Ricardo-Czekster).\n\n## License\nThe book is under Creative Commons (CC BY 4.0) and the code is under GPLv3.\n\nFeel free to use it and attribute the original source.\n\nDOI link: [http://dx.doi.org/10.6084/m9.figshare.19714384.v2](http://dx.doi.org/10.6084/m9.figshare.19714384.v2)\n\n## Folders\nThe structure of the repository is as follows:\n\n- `challenges`: proposed challenges throughout the book;\n- `matlab`: MATLAB\u0026reg; scripts for some solutions; \n- `models`: a list of DTMC and CTMC models (in the format that we explored in the book, with code);\n- `prism`:  [PRISM](https://www.prismmodelchecker.org/) models (in CTMC and DTMC);\n- `spreadsheets`: MS-Excel spreadsheets reproducing the experiments (CTMC/DTMC solution, simulation);\n\n## Challenges\nThe book proposes a number of _challenges_ throughout its chapters. The code for all solutions is available for download in this repository.\n\nOne remark: to avoid repetitions in the code, there is a folder called `project00` that has a `Makefile` and all the challenges (you may want to check the file `markov.c` to access auxiliary functions). In this project, each challenge has a `main()` function, so they work independently.\n\nThe `project01` folder contains a full project with one `main()` function and several built-in command-line options. Consult \"Challenge 07\" for more information about this project.\n\nPlease, let me know if you find something odd or something that needs changing. Thank you in advance.\n\n[Click here for more information about individual challenges](challenges.md)\n\n\n## Models\nThe book offers many models for evaluating your projects in a textual matrix format (.txt), prism (.sm or .pm), and MATLAB (.m).\n\n### CTMC/DTMC models to run internally\nThe following models are available (they were named **S**N where N is the number of states):\n\n- `models/S2-lighting-ctmc.txt`: The Lighting model, using rates;\n- `models/S2-lighting-dtmc.txt`: The Lighting model, using probabilities;\n- `models/S2-pushkin-dtmc.txt`: The pushkin model developed by A. A. Markov (according to Grinstead and Snell's book _Introduction to Probability_);\n- `models/S3-land-of-oz-dtmc.txt`: The land of oz model;\n- `models/S3-weather-belfast-dtmc.txt`: The Belfast weather model (by Stewart);\n- `models/S4-aging-rejuvenation-ctmc`: Software aging and rejuvenation model, CTMC;\n- `models/S4-aging-rejuvenation-dtmc`: Same (aging \u0026 rej), DTMC version;\n- `models/S4-generic-ctmc.txt`: The Generic model;\n- `models/S4-lilypad-ctmc.txt`: The Lily Pad model (frog in the pond);\n- `models/S4-maze-model-dtmc.txt`: The Mouse Maze model;\n- `models/S5-ehrenfest-dtmc.txt`: [The Ehrenfest model](https://en.wikipedia.org/wiki/Ehrenfest_model);\n- `models/S6-birth-and-death-ctmc.txt`: A six state Birth and Death model;\n- `models/S9-maze-model-dtmc.txt`: A larger Mouse Maze model, with 9 states;\n- `models/S26-shakespeare-ctmc.txt`: A brief analysis of one of Shakespeare's Sonnets with MC;\n\n### PRISM models\n- `prism/belfast-dtmc.pm` for the Belfast weather model;\n- `prism/generic-ctmc.sm` for the Generic model;\n- `prism/lily-ctmc.sm` and `lily.dtmc.pm` for the Lily pad model;\n- `prism/aging.sm` and accompanying `prism/aging.props` model and property file, respectively;\n\n### MATLAB models\n- `matlab/generic_model_dtmc.m`;\n- `matlab/matlabsolution_dtmc.m`;\n\n## Challenges\nYou may try to run the challenges code for these models (beware that the type should match, either CTMC or DTMC).\n[Click here for more information on models](models.md)\n\n## Numerical methods, models and tools\nThe methods, models and tools explored in the book are Power Matrix, Vector-Matrix Multiplication, [PRISM](https://www.prismmodelchecker.org/) models, some MATLAB\u0026reg; scripting, _forward_ simulation, and some MS-Excel spreadsheets (with basic models).\n\n## How to cite the work\nI thank you for your attribution, and for recognising the work.\nTo cite the book, please follow this suggestion:\n- Czekster, Ricardo M. **\"Markov Chains for programmers\"**, Available at: https://czekster.github.io/markov/. DOI: http://dx.doi.org/10.6084/m9.figshare.19714384.v2, April, 2022.\n\n## Author\n\u003cimg align=\"right\" width=\"200\" src=\"images/rmc.png\"\u003e\nRicardo M. Czekster\nrczekster at gmail\n\nRicardo is an active researcher in dependability and cyber-security.\nPlease, check out his virtual presence on the following platforms:\n- [ORCID](https://orcid.org/0000-0002-6636-4398)\n- [ResearchGate](https://www.researchgate.net/profile/Ricardo-Czekster)\n- [DBLP](https://dblp.uni-trier.de/pers/hd/c/Czekster:Ricardo_M=)\n- [Google Scholar profile](https://scholar.google.com.br/citations?user=M4SZcHsAAAAJ)\n- [LinkedIn](https://www.linkedin.com/in/ricardoczekster/)\n- [Publons](https://publons.com/researcher/1440444/ricardo-m-czekster/)\n- [GitHub](https://github.com/czekster/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fczekster%2Fmarkov","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fczekster%2Fmarkov","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fczekster%2Fmarkov/lists"}