{"id":17862695,"url":"https://github.com/stevana/property-based-testing-stateful-systems-tutorial","last_synced_at":"2025-03-21T00:30:37.807Z","repository":{"id":65738311,"uuid":"587191439","full_name":"stevana/property-based-testing-stateful-systems-tutorial","owner":"stevana","description":" A tutorial about how to apply property-based testing to stateful systems.","archived":false,"fork":false,"pushed_at":"2023-02-10T13:09:58.000Z","size":431,"stargazers_count":70,"open_issues_count":0,"forks_count":4,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-03-17T19:11:35.222Z","etag":null,"topics":["fault-injection","haskell","linearizability","property-based-testing","simulation-testing","state-machines","tutorial"],"latest_commit_sha":null,"homepage":"","language":"Haskell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stevana.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":".github/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-01-10T07:06:13.000Z","updated_at":"2024-12-04T02:36:39.000Z","dependencies_parsed_at":"2023-02-19T16:45:47.313Z","dependency_job_id":null,"html_url":"https://github.com/stevana/property-based-testing-stateful-systems-tutorial","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stevana%2Fproperty-based-testing-stateful-systems-tutorial","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stevana%2Fproperty-based-testing-stateful-systems-tutorial/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stevana%2Fproperty-based-testing-stateful-systems-tutorial/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stevana%2Fproperty-based-testing-stateful-systems-tutorial/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stevana","download_url":"https://codeload.github.com/stevana/property-based-testing-stateful-systems-tutorial/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244717176,"owners_count":20498280,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fault-injection","haskell","linearizability","property-based-testing","simulation-testing","state-machines","tutorial"],"created_at":"2024-10-28T08:54:42.614Z","updated_at":"2025-03-21T00:30:37.452Z","avatar_url":"https://github.com/stevana.png","language":"Haskell","readme":"Property-based testing stateful systems: a tutorial\n================================================================\n\n[![GitHub\nCI](https://github.com/stevana/property-based-testing-stateful-systems-tutorial/workflows/CI/badge.svg)](https://github.com/stevana/property-based-testing-stateful-systems-tutorial/actions)\n[![Hackage](https://img.shields.io/hackage/v/property-based-testing-stateful-systems-tutorial.svg)](https://hackage.haskell.org/package/property-based-testing-stateful-systems-tutorial)\n\nProperty-based testing (PBT), i.e. generating random inputs and checking some\nproperty of the output, of pure programs is an established practice by now. It's\ntaught in introductory university classes and it's part of test suites in\nindustry.\n\nMost real world programs are not pure though, they are stateful. While it's\noften possible to structure your program in such a way that the impure stuff is\ndone in `main`, e.g. read the contents of a file, and then passed on to a pure\nfunction, e.g. a parser, it's not always possible. Consider a long-running\nprogram that interacts with the filesystem and with other programs over the\nnetwork, e.g. some kind of web service or a distributed database. It's difficult\nto split such a program up into doing a little bit of impure stuff at the start,\nthen hand it over to a pure function (which we can apply PBT on).\n\nGiven this it's perhaps a bit surprising that there are relatively few resources\nabout applying PBT to stateful systems. This repository is an attempt to close\nthat gap and try to make PBT stateful systems more common.\n\nThe goals we'd like to achieve are:\n\n  - Show how to test stateful (i.e. impure/monadic) programs using\n    property-based testing;\n\n  - Show how we can do concurrent testing to help uncover problems such as race\n    conditions;\n\n  - Show how we can build bigger systems in a modular way by applying the\n    property-based testing equivalent of integration and contract tests;\n\n  - Show how to use fault injection and so called simulation testing to\n    \"end-to-end\" test distributed systems;\n\n  - Introduce the reader to related work and open problems in the area along the\n    way.\n\nIn the interest of brevity, we assume that the reader already has:\n\n  - Enough familiarity with Haskell to be able to read simple programs, for\n    example if you can follow along in the *Learn You a Haskell for Great Good!*\n    [tutorial](http://learnyouahaskell.com/chapters), then you should be fine;\n\n  - Some experience with property-based testing of non-stateful (i.e. pure)\n    programs. For example as explained in the official QuickCheck\n    [manual](http://www.cse.chalmers.se/~rjmh/QuickCheck/manual.html) or in the\n    following\n    [tutorial](https://begriffs.com/posts/2017-01-14-design-use-quickcheck.html);\n\n  - Basic knowledge of state machines (i.e.\n    [Mealy](https://en.wikipedia.org/wiki/Mealy_machine) / [Moore\n    machines](https://en.wikipedia.org/wiki/Moore_machine) and\n    [transducers](https://en.wikipedia.org/wiki/Finite-state_transducer)).\n\nOther than that this tutorial is striving to be as self-contained as possibly as\nwell as accessible to non-Haskell programmers.\n\nStructure\n---------\n\nThe tutorial is split up into five parts (so far), and each part has the\nfollowing structure:\n\n- Motivation: explains why we are doing what we are about to do;\n- Plan: how we will do it;\n- Code: a concrete implementation of the idea (in case you get stuck when trying\n  to implement it yourself);\n- Discussion: common questions or objections;\n- Exercises: things the authors were to lazy to do, but they know how to;\n- Problems: things the authors don't know how to do (yet);\n- See also: links to further reading about the topic or related topics;\n- Summary: the most important take away.\n\nThe parts build upon each other. We start by modelling and testing a simple\ncounter using a state machine in part 1, we then reuse the same state machine\nmodel to test the counter for thread-safety using linearisability in part 2. In\npart 3 we will implement a queue and a web service that uses said queue, the\nstate machine model for the queue and the real implementation of the queue will\nbe contract tested to ensure that the model is faithful to the implementation,\nsubsequently while testing the web service we will use the model in place of the\nreal queue. In part 4 we introduce fault injection to the queue allowing us to\ntest how the web service performs when its dependency fails. Finally, in part 5,\nwe combine all the above ideas in what, sometimes is called simulation testing,\nto test a distributed system that uses replicated state machines.\n\nTable of contents\n-----------------\n\n1. [State machine testing](./docs/Part01SMTesting.md#readme)\n2. [Concurrent state machine testing with\n   linearisability](./docs/Part02ConcurrentSMTesting.md#readme)\n3. [Integration tests against state machine fakes and consumer-driven contract\n   tests for the fakes](./docs/Part03SMContractTesting.md#readme)\n4. [Fault-injection](./docs/Part04FaultInjection.md#readme)\n5. Simulation testing\n\nUsage\n-----\n\nThis repository contains literate Haskell code in `src`. If you want to interact\nwith it, install [`ghcup`](https://www.haskell.org/ghcup/install/) and then type\n`cabal repl`. Alternatively, if you are using the\n[`nix`](https://nixos.org/download.html) package manager, then running\n`nix-shell` in the root directory should give you the right `ghc` version and\nall other dependencies you might need.\n\nThe literate code is transformed into markdown using\n[`pandoc`](https://pandoc.org/) in\n[`tools/generate_markdown.sh`](./tools/generate_markdown.sh) and the markdown is\nput inside the [`docs`](./docs) directory for easier browsing.\n\nThe following is a link to the [first part](./docs/Part01SMTesting.md#readme) of the\ngenerate markdown, at the end it will link to the second part and so on. Or you\ncan use the table of contents above or the `docs` directory to jump to desired\npart straight away.\n\nContributing\n------------\n\nAny feedback, suggestions for improvement or questions are most welcome via the\nissue tracker!\n\nSee the [`CONTRIBUTING.md`](./.github/CONTRIBUTING.md) file for more detailed\nguidelines regarding contributing.\n\nLicense\n-------\n\nSee the [`LICENSE`](./LICENSE) file.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstevana%2Fproperty-based-testing-stateful-systems-tutorial","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstevana%2Fproperty-based-testing-stateful-systems-tutorial","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstevana%2Fproperty-based-testing-stateful-systems-tutorial/lists"}