{"id":30803760,"url":"https://github.com/kaeedo/scrutiny","last_synced_at":"2025-09-05T23:04:27.168Z","repository":{"id":40484326,"uuid":"227371702","full_name":"kaeedo/Scrutiny","owner":"kaeedo","description":"Randomly test state machines (such as your UI) by randomly navigating through transitions","archived":false,"fork":false,"pushed_at":"2024-12-16T20:23:22.000Z","size":11353,"stargazers_count":91,"open_issues_count":8,"forks_count":3,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-08-01T01:43:05.002Z","etag":null,"topics":["automated-testing","csharp","dotnet-core","dotnet-library","fsharp","ui-testing"],"latest_commit_sha":null,"homepage":"","language":"F#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kaeedo.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null},"funding":{"liberapay":"kaeedo","custom":["#donations"]}},"created_at":"2019-12-11T13:25:48.000Z","updated_at":"2025-01-14T09:45:49.000Z","dependencies_parsed_at":"2023-02-15T08:00:58.914Z","dependency_job_id":null,"html_url":"https://github.com/kaeedo/Scrutiny","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/kaeedo/Scrutiny","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaeedo%2FScrutiny","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaeedo%2FScrutiny/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaeedo%2FScrutiny/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaeedo%2FScrutiny/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kaeedo","download_url":"https://codeload.github.com/kaeedo/Scrutiny/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaeedo%2FScrutiny/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273833395,"owners_count":25176291,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-05T02:00:09.113Z","response_time":402,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automated-testing","csharp","dotnet-core","dotnet-library","fsharp","ui-testing"],"created_at":"2025-09-05T23:04:21.728Z","updated_at":"2025-09-05T23:04:27.148Z","avatar_url":"https://github.com/kaeedo.png","language":"F#","funding_links":["https://liberapay.com/kaeedo","#donations"],"categories":[],"sub_categories":[],"readme":"![Header](header.svg)\n\nF# and C# library for testing state machines by randomly choosing available states and valid transitions. Designed for\nusage with UI tests\n\n[![Nuget](https://img.shields.io/nuget/vpre/scrutiny?color=blue\u0026style=for-the-badge)](https://www.nuget.org/packages/Scrutiny/) ![Build](https://github.com/kaeedo/Scrutiny/workflows/Build/badge.svg?branch=master)\n\n## Description\n\nDescribe your UI as a state machine, and then use Scrutiny to simulate a \"User\" that randomly clicks around on your\nsite.\nScrutiny will attempt to create a Directed Adjacency Graph of your states, and then randomly choose an unvisited state\nto navigate to.\nIt will repeat this process until all states have been visited.\nDuring each state, Scrutiny will attempt to run any defined actions within that state.\nOnce all states have been visited, if an exit action has been defined it will then navigate there and quit.\nScrutiny will then also generate an HTML file which visualizes the State Machine as a graph.\n\nScrutiny was designed to run UI tests, but using e.g. CanopyUI or Selenium is only an implementation detail. In theory,\nany state machine can be tested with Scrutiny.\n\n---\n\nThere are several usage example projects in the `usageExamples` directory, implemented using different technologies. The\nfirst two are implemented in F#, and the third one in C#.\n\n* [Canopy UsageExample](usageExamples/UsageExample.Canopy) for a sample test implemented\n  with [CanopyUI](https://github.com/lefthandedgoat/canopy)\n* [Playwright UsageExample](usageExamples/UsageExample.Playwright) for a sample test implemented\n  with [PlaywrightSharp](https://github.com/microsoft/playwright-sharp)\n* [C# UsageExample](usageExamples/UsageExample.CSharp) for a sample test implementation also using Playwright, but this\n  time with C#\n\nA tiny sample site exists in the [Usage Example directory](usageExamples/Web). This is the website that the usage\nexamples are testing. It features three pages, a home page, comment page, and a sign in page. A user can only leave a\ncomment if they are signed in.\nThe usage examples showcase a certain approach a developer can take as to how to model their web site as a state\nmachine. In this case, the home and comment page are each listed twice, once as logged out, and once as logged in.\nThis is only one way to handle this case, and the developer could choose to model it in any other way.\n\nScrutiny will also draw a diagram representing the system under test as has been modeled by the various `page`s.\nThe [Sample Web site](usageExamples/Web) looks like this:\n\n![SUT sample report](images/scrutinyDemo.gif)\n\n# Documentation\n\n\u003cdetails\u003e\n  \u003csummary\u003e\u003ci\u003eClick\u003c/i\u003e for F# documentation\u003c/summary\u003e\n\nDefine one `page` object for each state in your UI. A state can be anything from a page, or an individual modal, or the\nsame page as a different state, but altered, for example a logged in user.\n\nThe possible custom operations are:\n\n- `name`: Name of the state. Required\n- `onEnter`: Function to run when entering this page. Only one allowed\n- `onExit`: Function to run when exiting this page. Only one allowed\n- `transition`: Possible transition. Define how to transition to the next state, as well as which state to navigate to.\n  Any number of transitions allowed\n- `action`: Possible action. Define function to run while in this page state. Any number of actions allowed\n\nA `page` looks like this:\n\n    let loggedInComment = fun (globalState: GlobalState) -\u003e\n        page {\n            name \"Logged In Comment\"\n\n            onEnter (fun ls -\u003e\n                printfn \"Checking on page comment\"\n                \"#header\" == \"Comments\"\n            )\n\n            onExit (fun _ -\u003e\n                printfn \"Exiting comment\"\n            )\n\n            transition {\n                via (fun ls -\u003e click ls.HomeLink)\n                destination home\n            }\n            transition {\n                via (fun _ -\u003e click \"#signin\")\n                destination signIn\n            }\n\n            action {\n                fn (fun _ -\u003e () (*do something on the page*))\n            }\n            action {\n                fn (fun _ -\u003e () (*do something else on the page*))\n            }\n            action {\n                isExit\n                fn (fun _ -\u003e () (*final action to perform before exiting the test*))\n            }\n        }\n\nThe `name` must be unique. Any number of `transition`s and any number of `action`s can be defined. The `onEnter` function will be executed everytime scrutiny transitions to this state, and `onExit` will execute everytime scrutiny leaves this state. `name`, `onEnter`, and `onExit` must be defined before any `transition`s and `action`s.\n\n\nAny `action` can be be marked as `isExit`, and multiple `page`s can have an `action` that is the exit action. If\nmultiple are defined, Scrutiny will randomly choose one to perform.\nThe `GlobalState` in the example is any type defined in your test that you can use to pass data between states,\ne.g. `Username` or `IsLoggedIn`\n\n`action`s are defined as follow within a page CE:\n\n    page {\n        name \"something\"\n\n        action {\n            name \"Name of action\"\n            dependantActions [ \"Other action\" ]\n            isExit\n            fn (fun _ -\u003e (*This is the function that gets run*))\n        }\n    }\n\nThe `name` defines a name for this `action`. Optional. This is how this action is reffered to when another action or\ntransition depends on it\nThe `dependantActions` list defines any actions that will be run before this action is run. Optional\nThe `isExit` marks this action as a potential exit action. Optional\nThe `fn` is the actual function to run as this action. Required\n\n`transition`s are defined as follows within a page CE:\n\n    page {\n        name \"something\"\n\n        transition {\n            dependantActions [ \"Other action\" ]\n            via (fun _ -\u003e (*how to transition to the next state*))\n            destination otherPage\n        }\n    }\n\nThe `dependantActions` list defines any actions that will be run before this action is run. Optional\nThe `via` function is executed that will actually transition the state machine to the next state. Required\nThe `destination` is the state that will be transitioned to. Required\n\n### Configuration\n\nSome things can be configured via `ScrutinyConfig`. The default config is:\n\n    { ScrutinyConfig.Seed = Environment.TickCount\n      MapOnly = false\n      ComprehensiveActions = true\n      ComprehensiveStates = true\n      ScrutinyResultFilePath = Directory.GetCurrentDirectory() + \"/ScrutinyResult.html\"\n      Logger = printfn \"%s\" }\n\n`Seed` is printed during each test to be able to recreate a specific test run.\n`MapOnly` won't run the test at all, but only generate the HTML Graph report.\n`ComprehensiveActions` will run ALL defined actions anytime it enters a state with actions defined. If false, it will\nrun a random subset of actions.\n`ComprehensiveStates` will visit ALL states in the state machine. If this is false, then it will visit at least half of\nall states before randomly quitting.\n`ScrutinyResultFilePath` is the directory and specified file name that the generated HTML report will be saved in\n`Logger` is how individual messages from scrutiny will be logged. The signature is `string -\u003e unit`. This is useful for\nthings like XUnit that bring their own console logging mechanism, or if you wanted to integrate a larger logging\nframework.\n\nTo actually run the test, call the `scrutinize` function with your entry state, config, and global state object. e.g.\n\n    // Sample Global State. This can be anything, and all page states will receive the same instance\n    type GlobalState() =\n        member val IsSignedIn = false with get, set\n        member val Username = \"MyUsername\" with get, set\n        member val Number = 42\n\n    [\u003cEntryPoint\u003e]\n    let main argv =\n        let options = FirefoxOptions()\n        do options.AddAdditionalCapability(\"acceptInsecureCerts\", true, true)\n\n        use ff = new FirefoxDriver(options)\n        let currentDirectory = DirectoryInfo(Directory.GetCurrentDirectory())\n\n        let config =\n            { ScrutinyConfig.Default with\n                  Seed = 553931187\n                  MapOnly = false\n                  ComprehensiveActions = true\n                  ComprehensiveStates = true\n                  ScrutinyResultFilePath = currentDirectory.Parent.Parent.Parent.FullName + \"/myResult.html\" }\n\n        // Start tests. In this case we're using CanopyUI, but can be any test runner e.g. XUnit or Expecto\n        // Start CanopyUI tests\n        \"Scrutiny\" \u0026\u0026\u0026 fun _ -\u003e\n            printfn \"opening url\"\n            url \"https://localhost:5001/home\"\n\n            let gs = GlobalState()\n\n            // The call to start Scrutiny, and construct a graph and \"click\" through all states\n            scrutinize config gs home\n            // or\n            // scrutinizeWithDefaultConfig gs home\n\n        switchTo ff\n        pin canopy.types.direction.Right\n\n        run()\n        quit ff\n\n        0\n\nAt the end of the run, Scrutiny will return an object which contains the generated adjacency graph, as well as a list of\nindividual steps taken, along with the actions performed in each state.\n\n#### Important note for F# users\n\nAs the transitions ultimately depict a cyclic graph, it is necessary to declare module or namespace as recursive so that\npages defined later can be referenced by pages earlier. Note the usage of the `rec` keyword.\ne.g.:\n\n    module rec MyPages =\n        let firstPage = fun (globalState: GlobalState) -\u003e\n            page {\n                name \"First Page\"\n                transition {\n                    via (fun _ -\u003e click \"#second\")\n                    destination secondPage\n                }\n            }\n\n        let secondPage = fun (globalState: GlobalState) -\u003e\n            page {\n                name \"Second Page\"\n                transition {\n                    via (fun _ -\u003e click \"#first\")\n                    destination firstPage\n                }\n            }\n\n\u003cdetails\u003e\n  \u003csummary\u003eMigration v1 to v2\u003c/summary\u003e\n\n  * Within a `page` computation expression, ensure that `name` is first, and that any `onEnter` and `onExit` functions are defined before any `transition`s and `action`s.\n  * `transition`s are now defined using a `transition` computation expression:\n    * Before: `transition ((fun _ -\u003e click \"#signin\") ==\u003e signIn)`\n    * After:\n        ```\n        transition {\n            via (fun _ -\u003e click \"#signin\")\n            destination signIn\n        }\n        ```\n  * `action`s are now defined using an `action` computation expression:\n    * Before: `action (fun _ -\u003e () /*do something on the page*/)`\n    * After:\n        ```\n        action {\n            fn (fun _ -\u003e () /*do something on the page*/)\n        }\n        ```\n  * `exitAction`s are now defined as a regular action, but with the `isExit` property set:\n    * Before: `exitAction (fun _ -\u003e () /*final action to perform before exiting the test*/)`\n    * After:\n        ```\n        action {\n            isExit\n            fn (fun _ -\u003e () /*final action to perform before exiting the test*/)\n        }\n        ```\n\n\u003c/details\u003e\n\n\u003c/details\u003e\n\n---\n\n\u003cdetails\u003e\n  \u003csummary\u003e\u003ci\u003eClick\u003c/i\u003e for C# documentation\u003c/summary\u003e\n\nDefine one class for each state in your UI, and decorate it with the `PageState` attribute. A state can be anything from\na page, or an individual modal, or the same page as a different state, but altered, for example a logged in user.\n\nThe possible attributes are:\n\n- `PageState`: Define a class as a Page state.\n- `OnEnter`: Function to run when entering this page. Only one allowed\n- `OnExit`: Function to run when exiting this page. Only one allowed\n- `TransitionTo`: Possible transition. Define how to transition to the next state, as well as which state to navigate\n  to. Any number of transitions allowed\n- `Action`: Possible action. Define function to run while in this page state. Any number of actions allowed. Optionally\n  can be configured to be an exit action via the property `IsExit`\n- `DependantAction`: Takes a string as a parameter. Only valid on Transitions and Actions. References an action that\n  should be run before this action/transition. Multiple dependant actions can be referenced per action/transition\n\nA `PageState` could look like this:\n\n    using Scrutiny.CSharp;\n\n    [PageState]\n    public class LoggedInComment\n    {\n        private readonly GlobalState globalState;\n\n        public LoggedInComment(GlobalState globalState)\n        {\n            // Construct anything necessary.\n            // The constructor is called everytime Scrutiny navigates to this state\n        }\n\n        [OnEnter]\n        public void OnEnter()\n        {\n            // Do something when scrutiny enters this state\n            // Can optionally be async/await\n            // Can only define one\n        }\n\n        [Action]\n        public async Task WriteComments()\n        {\n            // Do something on the page\n            // Can optionally be non-async\n            // Define any number of these\n        }\n\n        [Action(IsExit = true)]\n        public async Task ExitAction()\n        {\n            // One exit actions amongst all page states is chosen\n            // Define any number of these\n            // Can optionally be non-async\n        }\n\n        [ExitAction]\n        public async Task ExitAction()\n        {\n\n        }\n\n        [TransitionTo(nameof(AnotherState))]\n        [DependantAction(nameof(WriteComments))] // Optioanlly run the WriteComments action before executing this transition\n        public void TransitionToAnotherState()\n        {\n            // Code to perform state transition\n            // Define any number of these\n            // Can optionally be async/await\n        }\n    }\n\n### Configuration\n\nSome things can be configured via the `Scrutiny.CSharp.Configuration.Configuration` POCO. The default config is:\n\n    Seed = Environment.TickCount\n    MapOnly = false\n    ComprehensiveActions = true\n    ComprehensiveStates = true\n    ScrutinyResultFilePath = Directory.GetCurrentDirectory() + \"/ScrutinyResult.html\"\n    Logger = (Action\u003cstring\u003e)((s) =\u003e Console.WriteLine(s))\n\n`Seed` is printed during each test to be able to recreate a specific test run.\n`MapOnly` won't run the test at all, but only generate the HTML Graph report.\n`ComprehensiveActions` will run ALL defined actions anytime it enters a state with actions defined. If false, it will\nrun a random subset of actions.\n`ComprehensiveStates` will visit ALL states in the state machine. If this is false, then it will visit at least half of\nall states before randomly quitting.\n`ScrutinyResultFilePath` is the directory and specified file name that the generated HTML report will be saved in\n`Logger` is how individual messages from scrutiny will be logged. This is useful for things like XUnit that bring their\nown console logging mechanism, or if you wanted to integrate a larger logging framework.\n\nTo actually run the test, call the `Scrutiny.CSharp.Scrutinize.Start\u003cHome\u003e(gs, config)` method. It takes your entry\nstate as a generic type argument, and a constructed global state object as well as your config as parameters.\n\n    using Scrutiny.CSharp;\n\n    [Fact]\n    public async Task WithAttrs()\n    {\n        var browser = await playwright.Firefox.LaunchAsync(headless: false);\n        var context = await browser.NewContextAsync(ignoreHTTPSErrors: true);\n        var page = await context.NewPageAsync();\n\n        await page.GoToAsync(\"https://127.0.0.1:5001/home\");\n\n        var config = new Configuration\n        {\n            Seed = 553931187,\n            MapOnly = false,\n            ComprehensiveActions = true,\n            ComprehensiveStates = true\n        };\n\n        var gs = new GlobalState(page, outputHelper);\n        var result = Scrutinize.Start\u003cHome\u003e(gs, config);\n\n        Assert.Equal(7, result.Steps.Count());\n        Assert.Equal(5, result.Graph.Count());\n    }\n\nThe global state can be any class you want it to be. Scrutiny will pass the instance that is passed into the start\naround to each `PageState` it visits.\nAt the end of the run, Scrutiny will return an object which contains the generated adjacency graph, as well as a list of\nindividual steps taken, along with the actions performed in each state.\n\n\u003cdetails\u003e\n  \u003csummary\u003eMigration v1 to v2\u003c/summary\u003e\n\n  * `[ExitAction]` attribute removed. Set the `IsExit` property on an `Action` isntead\"\n    * Before:\n        ```\n        [ExitAction]\n        public async Task ExitAction()\n        ```\n    * After:\n        ```\n        [Action(IsExit = true)]\n        public async Task ExitAction()\n        ```\n\n\u003c/details\u003e\n\n\u003c/details\u003e\n\n---\n\n## Development\n\nTo run the usage examples, you must start the [web project](usageExamples/Web).\n\nThe HTML report is a single file with all javascript written inline\n\n---\n\n### Donations\n\nDonations are greatly appreciated, but not needed at all. Please only donate if you are in a position to be able to\nafford it, and only if you truly believe in the gift of giving.\n\nLiberapay: [![Liberapay](https://liberapay.com/assets/widgets/donate.svg)](https://liberapay.com/kaeedo)\n\u003cdetails\u003e\n  \u003csummary\u003e\u003ci\u003eClick\u003c/i\u003e for cryptocurrency links\u003c/summary\u003e\n\nEthereum: `0x05f231D19c19A2111fe03c923F26813Bad43B57f`\n\nCardano ADA: `addr1qx35nmy62dfp3n5tqgga92gxcnq5vkvflw963yg7fm5e5my68x9frc2qq0r8nstjtnjcrcnpmtpzwvp0sqz46y4ykrmqrd4dg9`\n\u003c/details\u003e\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkaeedo%2Fscrutiny","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkaeedo%2Fscrutiny","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkaeedo%2Fscrutiny/lists"}