{"id":15411431,"url":"https://github.com/haf/expecto","last_synced_at":"2025-05-13T21:12:13.566Z","repository":{"id":41534464,"uuid":"71635809","full_name":"haf/expecto","owner":"haf","description":"A smooth testing lib for F#. APIs made for humans! Strong testing methodologies for everyone!","archived":false,"fork":false,"pushed_at":"2025-03-30T23:57:46.000Z","size":40149,"stargazers_count":705,"open_issues_count":58,"forks_count":98,"subscribers_count":28,"default_branch":"main","last_synced_at":"2025-04-29T13:58:37.555Z","etag":null,"topics":["performance-testing","property-based-testing","stress-testing","testing","unit-testing"],"latest_commit_sha":null,"homepage":"","language":"F#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/haf.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":["haf"]}},"created_at":"2016-10-22T11:49:30.000Z","updated_at":"2025-04-28T18:50:49.000Z","dependencies_parsed_at":"2023-01-25T18:16:28.751Z","dependency_job_id":"f3afb4f2-7bfe-47c3-9023-bd7c65a89034","html_url":"https://github.com/haf/expecto","commit_stats":{"total_commits":1341,"total_committers":77,"mean_commits":"17.415584415584416","dds":0.7844891871737509,"last_synced_commit":"d597b1088bb41983bbdac2be79a54382aa99c4f2"},"previous_names":[],"tags_count":106,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haf%2Fexpecto","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haf%2Fexpecto/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haf%2Fexpecto/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/haf%2Fexpecto/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/haf","download_url":"https://codeload.github.com/haf/expecto/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254029008,"owners_count":22002284,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["performance-testing","property-based-testing","stress-testing","testing","unit-testing"],"created_at":"2024-10-01T16:49:06.105Z","updated_at":"2025-05-13T21:12:08.553Z","avatar_url":"https://github.com/haf.png","language":"F#","readme":"\u003ch1 align=\"center\"\u003eExpecto\u003c/h1\u003e\n\u003cp align=\"center\"\u003eAn advanced testing library for F#\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n\u003ca href=\"https://travis-ci.org/haf/expecto\"\u003e\u003cimg alt=\"Linux Build\" src=\"https://travis-ci.org/haf/expecto.svg?branch=master\"\u003e\u003c/a\u003e\n\u003ca href=\"https://www.nuget.org/packages/expecto\"\u003e\u003cimg alt=\"Nuget\" src=\"https://img.shields.io/nuget/dt/Expecto\"\u003e\u003c/a\u003e\n\u003cimg src=\"https://img.shields.io/github/languages/top/haf/expecto?color=%23b845fc\"\u003e\n\u003ca href=\"https://opensource.org/licenses/Apache-2.0\"\u003e\u003cimg alt=\"License - Apache 2.0\" src=\"https://img.shields.io/badge/License-Apache%202.0-blue.svg\"\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/sponsors/haf\" title=\"Sponsor this\"\u003e\u003cimg src=\"https://img.shields.io/static/v1?label=Sponsor\u0026message=%E2%9D%A4\u0026logo=GitHub\u0026color=red\" /\u003e\u003c/a\u003e\n\u003c/p\u003e\n\u003cbr\u003e\n\nExpecto aims to make it easy to test CLR based software; be it with unit tests, stress tests, regression tests or\nproperty based tests. Expecto tests are parallel and async by default, so that you can use all your cores for\ntesting your software. This also opens up a new way of catching threading and memory issues for free using stress\ntesting.\n\n![Parallel by default](./docs/stress.jpg)\n\nWith Expecto you write tests as values. Tests can be composed, reduced, filtered, repeated and passed as values, because\nthey are values. This gives the programmer a lot of leverage when writing tests.  Setup and teardown are just simple\nfunctions, no need for attributes.\n\nExpecto comes with batteries included with an integrated test runner, but it's still open for extension due to its\ncompositional model.\n\nExpecto comes with performance testing, making statistically sound performance comparison simple.\n\nExpecto also provides a simple API for property based testing using FsCheck.\n\n## Quickstart\n\n    dotnet new install \"Expecto.Template::*\"\n    dotnet new expecto -n PROJECT_NAME -o FOLDER_NAME\n    \nFollow [Smooth Testing](https://www.youtube.com/channel/UC2SN9CUu9LlOBukXXv_bT5Q) on YouTube to learn the basics.\n\nWhat follows is the Table of Contents for this README, which also serves as the documentation for the project.\n\n\u003c!-- toc --\u003e\n\n- [Quickstart](#quickstart)\n- [Installing](#installing)\n- [IDE integrations](#ide-integrations)\n- [.NET integration](#net-integration)\n  - [Prettify stacktraces/ship test logs](#prettify-stacktracesship-test-logs)\n  - [TestResults file](#testresults-file)\n- [.NET support](#net-support)\n- [Testing \"Hello world\"](#testing-hello-world)\n- [Running tests](#running-tests)\n  - [`runTestsWithCLIArgs`](#runtestswithcliargs)\n  - [`runTestsWithCLIArgsAndCancel`](#runtestswithcliargsandcancel)\n  - [`runTestsInAssemblyWithCLIArgs`](#runtestsinassemblywithcliargs)\n  - [`runTestsInAssemblyWithCLIArgsAndCancel`](#runtestsinassemblywithcliargsandcancel)\n  - [Filtering with `filter`](#filtering-with-filter)\n  - [Shuffling with `shuffle`](#shuffling-with-shuffle)\n  - [Stress testing](#stress-testing)\n- [Writing tests](#writing-tests)\n  - [Normal tests](#normal-tests)\n  - [`testList` for grouping](#testlist-for-grouping)\n  - [Test fixtures](#test-fixtures)\n  - [Theory tests](#theory-tests)\n  - [Pending tests](#pending-tests)\n  - [Focusing tests](#focusing-tests)\n  - [Sequenced tests](#sequenced-tests)\n  - [Parameterised tests with `testParam`](#parameterised-tests-with-testparam)\n  - [Setup and teardown](#setup-and-teardown)\n  - [Property based tests](#property-based-tests)\n    - [Link collection](#link-collection)\n    - [Code from FsCheck](#code-from-fscheck)\n- [Expectations with `Expect`](#expectations-with-expect)\n  - [`Expect` module](#expect-module)\n  - [`Performance` module](#performance-module)\n    - [Example](#example)\n  - [Performance.findFastest](#performancefindfastest)\n- [`main args` and command line – how to run console apps examples](#main-args-and-command-line--how-to-run-console-apps-examples)\n- [Contributing and building](#contributing-and-building)\n- [BenchmarkDotNet usage](#benchmarkdotnet-usage)\n- [You're not alone!](#youre-not-alone)\n  - [Testing hardware](#testing-hardware)\n- [Sending e-mail on failure – custom printers](#sending-e-mail-on-failure--custom-printers)\n- [About test parallelism](#about-test-parallelism)\n  - [What does 'expected to have type TestCode' mean?](#what-does-expected-to-have-type-testcode-mean)\n  - [My tests are hanging and I can't see why](#my-tests-are-hanging-and-i-cant-see-why)\n- [Migration notes](#migration-notes)\n  - [11.0.0](#1100)\n\n\u003c!-- tocstop --\u003e\n\n![Sample output](docs/sample-output-2.png)\n\n\n## Installing\n\nIn your `paket.dependencies`:\n\n    nuget Expecto\n    nuget Expecto.BenchmarkDotNet\n    nuget Expecto.FsCheck\n    nuget Expecto.Hopac\n\nTests should be first-class values so that you can move them around and execute\nthem in any context that you want.\n\nLet's have look at what an extensive unit test suite looks like when running\nwith Expecto:\n\n![Sample output from Logary](docs/sample-output-logary.png)\n\n\n## IDE integrations\n\nThere's a NuGet `Expecto.VisualStudio.TestAdapter` for Visual Studio integration.\n\n## .NET integration\n\nYou can use `dotnet run` or `dotnet watch` from the command line.\n\n    dotnet watch -p MyProject.Tests run -f net6.0\n\n\n### Prettify stacktraces/ship test logs\n\nTo get a [complete logging solution][logary] and stacktrace highlighting, parsing and\nthe ability to ship your build logs somewhere, also add these:\n\n    nuget Logary.Adapters.Facade prerelease\n\nAnd in your tests:\n\n```fsharp\nopen Hopac\nopen Logary\nopen Logary.Configuration\nopen Logary.Adapters.Facade\nopen Logary.Targets\n\n[\u003cEntryPoint\u003e]\nlet main argv =\n  let logary =\n    Config.create \"MyProject.Tests\" \"localhost\"\n    |\u003e Config.targets [ LiterateConsole.create LiterateConsole.empty \"console\" ]\n    |\u003e Config.processing (Events.events |\u003e Events.sink [\"console\";])\n    |\u003e Config.build\n    |\u003e run\n  LogaryFacadeAdapter.initialise\u003cExpecto.Logging.Logger\u003e logary\n\n  // Invoke Expecto:\n  runTestsInAssemblyWithCLIArgs [] argv\n```\n\nNow, when you use Logary in your app, you can see your log messages\ntogether with the log output/summary/debug printing of Expecto,\nand the output won't be interlaced due to concurrency.\n\n### TestResults file\n\nUse `--nunit-summary TestResults.xml` or `--junit-summary TestResults.junit.xml` (JUnit support is incomplete).\n\n## .NET support\n\n[Expecto has its own .NET template](https://github.com/MNie/Expecto.Template)! You could create a base .NET\nproject with expecto.  How to do that? Simply write following lines:\n\n    dotnet new install 'Expecto.Template::*'\n    dotnet new expecto -n PROJECT_NAME -o FOLDER_NAME\n\nHow to run it?\n\n    dotnet restore\n    dotnet run\n\n![How to create expecto template](docs/expecto.gif)\n\n## Testing \"Hello world\"\n\nThe test runner is the test assembly itself. It's recommended to compile your test assembly as a console application.\nYou can run a test directly like this:\n\n```fsharp\nopen Expecto\n\nlet tests =\n  test \"A simple test\" {\n    let subject = \"Hello World\"\n    Expect.equal subject \"Hello World\" \"The strings should equal\"\n  }\n\n[\u003cEntryPoint\u003e]\nlet main args =\n  runTestsWithCLIArgs [] args tests\n```\n\nNo magic is involved here. We just created a single test and hooked it\ninto the assembly entry point.\n\nThe `Expect` module contains functions that you can use to assert with.\nA testing library without a good assertion library is like love without kisses.\n\nNow compile and run! `xbuild Sample.fsproj \u0026\u0026 mono --debug bin/Debug/Sample.exe`\n\n## Running tests\n\nHere's a simple test:\n\n```fsharp\nopen Expecto\n\nlet simpleTest =\n  testCase \"A simple test\" \u003c| fun () -\u003e\n    let expected = 4\n    Expect.equal expected (2+2) \"2+2 = 4\"\n```\n\nThen run it like this, e.g. in the interactive or through a console app.\n\n```fsharp\nrunTestsWithCLIArgs [] [||] simpleTest\n```\n\nwhich returns 1 if any tests failed, otherwise 0. Useful for returning to the\noperating system as error code.\n\nIt's worth noting that `\u003c|` is just a way to change the associativity of the\nlanguage parser. In other words; it's equivalent to:\n\n```fsharp\ntestCase \"A simple test\" (fun () -\u003e\n  Expect.equal 4 (2+2) \"2+2 should equal 4\")\n```\n\n### `runTestsWithCLIArgs`\n\nSignature `CLIArguments seq -\u003e string[] -\u003e Test -\u003e int`. Runs the passed tests\nand also overrides the passed `CLIArguments` with the command line parameters.\n\n### `runTestsWithCLIArgsAndCancel`\n\nSignature `CancellationToken -\u003e ExpectoConfig -\u003e Test -\u003e int`. Runs the passed tests\nand also overrides the passed `CLIArguments` with the command line parameters.\n\n### `runTestsInAssemblyWithCLIArgs`\n\nSignature `CLIArguments seq -\u003e string[] -\u003e int`. Runs the tests in the current\nassembly and also overrides the passed `CLIArguments` with the command line\nparameters. All tests need to be marked with the `[\u003cTests\u003e]` attribute.\n\n### `runTestsInAssemblyWithCLIArgsAndCancel`\n\nSignature `CancellationToken -\u003e CLIArguments seq -\u003e string[] -\u003e int`. Runs the tests in the current\nassembly and also overrides the passed `CLIArguments` with the command line\nparameters. All tests need to be marked with the `[\u003cTests\u003e]` attribute.\n\n### Filtering with `filter`\n\nYou can single out tests by filtering them by name (e.g. in the\ninteractive/REPL). For example:\n\n```fsharp\nopen Expecto\nopen MyLib.Tests\nintegrationTests // from MyLib.Tests\n|\u003e Test.filter defaultConfig.joinWith.asString (fun z -\u003e (defaultConfig.joinWith.format z).StartsWith \"another test\" ) // the filtering function\n|\u003e runTestsWithCLIArgs [] [||]\n```\n\n### Shuffling with `shuffle`\n\nYou can shuffle the tests randomly to help ensure there are no run order dependencies.\nFor example:\n\n```fsharp\nopen Expecto\nopen MyLib.Tests\nmyTests // from MyLib.Tests\n|\u003e Test.shuffle defaultConfig.joinWith.asString\n|\u003e runTestsWithCLIArgs [] [||]\n```\n\n### Stress testing\n\nTests can also be run randomly for a fixed length of time.\nThe idea is that this will catch the following types of bugs:\n\n- Memory leaks.\n- Threading bugs running same test at same time.\n- Rare threading bugs.\n- Rare property test fails.\n\nThe default config will run FsCheck tests with a higher end size than normal.\n\n## Writing tests\n\nExpecto supports the following test constructors:\n\n- normal test cases with `testCase`, `testCaseAsync` and `testCaseTask`\n- lists of tests with `testList`\n- test fixtures with `testFixture`, `testFixtureAsync`, `testFixtureTask`\n- pending tests (that aren't run) with `ptestCase`, `ptestCaseAsync` and `ptestCaseTask`\n- focused tests (that are the only ones run) with `ftestCase`,\n   `ftestCaseAsync` and `ftestCaseTask`\n- sequenced tests with `testSequenced` and `testSequencedGroup` (tests inside a\n   group are run in sequence w.r.t each other)\n- parametised tests with `testParam`\n- testCases with the workflow builder `test`, `ptest`, `ftest` supporting\n   deterministic disposal, loops and such\n- property based tests with `testProperty`, `testPropertyWithConfig` and\n  `testPropertyWithConfigs`, `testPropertyWithConfigsStdGen`,\n  `testPropertyWithConfigStdGen` from `Expecto.FsCheck`\n- performance tests with `Expecto.BenchmarkDotNet` and `benchmark\u003cTBench\u003e :\n   string -\u003e Test`.\n- wrapping your test with a label with `testLabel`. If your root label is the\n  same across your test project, you'll have an easier time filtering tests.\n\nAll of the above compile to a `Test` value that you can compose. For example,\nyou can compose a `test` and a `testCaseAsync` in a `testList` which you wrap in\n`testSequenced` because all tests in the list use either `Expect.fasterThan` or\nthey are using `Expecto.BenchmarkDotNet` for performance tests.  You have to\nremember that **the fully qualified names of tests need to be unique across your\ntest project.**\n\n### Normal tests\n\n- `test : string -\u003e TestCaseBuilder` -  Builds a test case in a computation expression.\n- `testAsync : string -\u003e TestAsyncBuilder` - Builds an async test case in a computation expression.\n- `testTask : string -\u003e TestTaskBuilder` - Builds a task test case in a computation expression.\n- `testCase : string -\u003e (unit -\u003e unit) -\u003e Test` - Builds a test case from a test function.\n- `testCaseAsync : string -\u003e Async\u003cunit\u003e -\u003e Test` - Builds an async test case from an async expression.\n- `testCaseTask : string -\u003e (unit -\u003e Task\u003cunit\u003e) -\u003e Test` - Builds an async test case from a function returning a task. Unlike async, tasks start right away and thus must be wrapped in a function so the task doesn't start until the test is run.\n\n### `testList` for grouping\n\nTests can be grouped (with arbitrary nesting):\n\n```fsharp\nlet tests =\n  testList \"A test group\" [\n    test \"one test\" {\n      Expect.equal (2+2) 4 \"2+2\"\n    }\n\n    test \"another test that fails\" {\n      Expect.equal (3+3) 5 \"3+3\"\n    }\n\n    testAsync \"this is an async test\" {\n      let! x = async { return 4 }\n      Expect.equal x (2+2) \"2+2\"\n    }\n\n    testTask \"this is a task test\" {\n      let! n = Task.FromResult 2\n      Expect.equal n 2 \"n=2\"\n    }\n  ]\n  |\u003e testLabel \"samples\"\n```\n\nAlso have a look at [the\nsamples](https://github.com/haf/expecto/blob/master/Expecto.Sample/Expecto.Sample.fs).\n\n### Test fixtures\n\n- `testFixture : ('a -\u003e unit -\u003e unit) -\u003e (seq\u003cstring * 'a\u003e) -\u003e seq\u003cTest\u003e`\n\nThe test fixture takes a factory and a sequence of partial tests. The `'a`\nparameter will be inferred to the *function type*, such as\n`MemoryStream -\u003e 'a -\u003e unit -\u003e 'a`.\n\nExample:\n\n```fsharp\ntestList \"Setup \u0026 teardown 3\" [\n  let withMemoryStream f () =\n    use ms = new MemoryStream()\n    f ms\n  yield! testFixture withMemoryStream [\n    \"can read\",\n      fun ms -\u003e ms.CanRead ==? true\n    \"can write\",\n      fun ms -\u003e ms.CanWrite ==? true\n  ]\n]\n```\n\n- `testFixtureAsync : ('a -\u003e unit -\u003e Async\u003cunit\u003e) -\u003e (seq\u003cstring * 'a\u003e) -\u003e seq\u003cTest\u003e`\n\nThe test fixture async takes a factory and a sequence of partial\ntests. The `'a` parameter will be inferred to the *function type*,\nsuch as `MemoryStream -\u003e 'a -\u003e 'a`.\n\nExample:\n\n```fsharp\ntestList \"Setup \u0026 teardown 4\" [\n  let withMemoryStream f = async {\n    use ms = new MemoryStream()\n    do! f ms\n  }\n  yield! testFixture withMemoryStream [\n    \"can read\",\n      fun ms -\u003e async { return ms.CanRead ==? true }\n    \"can write\",\n      fun ms -\u003e async { return ms.CanWrite ==? true }\n  ]\n]\n```\n\n\n- `testFixtureTask : ('a -\u003e unit -\u003e Task\u003cunit\u003e) -\u003e (seq\u003cstring * 'a\u003e) -\u003e seq\u003cTest\u003e`\n\nThe test fixture task takes a factory and a sequence of partial tests.\nThe `'a` parameter will be inferred to the *function type*, such as\n`MemoryStream -\u003e 'a -\u003e 'a`.\n\nExample:\n\n```fsharp\ntestList \"Setup \u0026 teardown 5\" [\n  let withMemoryStream f = task {\n    use ms = new MemoryStream()\n    do! f ms\n  }\n  yield! testFixture withMemoryStream [\n    \"can read\",\n      fun ms -\u003e task { return ms.CanRead ==? true }\n    \"can write\",\n      fun ms -\u003e task { return ms.CanWrite ==? true }\n  ]\n]\n```\n\n### Theory tests\n\n- `testTheory : string -\u003e seq\u003c'a\u003e -\u003e ('a -\u003e 'b) -\u003e Test`\n\nThe test theory takes a name and a sequence of cases to test against.\nThe `'a` parameter will be inferred to the *sequence type*, such as `string -\u003e seq\u003cint\u003e -\u003e (int -\u003e 'b) -\u003e Test`.\n\nExample:\n\n```fsharp\ntestList \"theory testing\" [\n  testTheory \"odd numbers\" [1; 3; 5] \u003c| fun x -\u003e\n    Expect.isTrue (x % 2 = 1) \"should be odd\"\n]\n```\n\nTheory tests can simulate multiple parameters via tuples. For example, passing input with an expected result\n\nExample:\n\n```fsharp\ntestList \"theory testing with an expected result\" [\n  testTheory \"sum numbers\" [(1,1),2; (2,2),4] \u003c| fun ((a,b), expected) -\u003e\n    Expect.equal (a+b) expected \"should be equal\"\n]\n```\n\n### Pending tests\n\n- `ptestCase`\n- `ptest`\n- `ptestAsync`\n- `ptestTask`\n- `ptestCaseAsync`\n- `ptestCaseTask`\n- `ptestTheory`\n- `ptestTheoryAsync`\n- `ptestTheoryTask`\n\nYou can mark an individual spec or container as Pending. This will prevent the spec (or specs within the list) from\nrunning.  You do this by adding a `p` before *testCase* or *testList* or `P` before *Tests* attribute (when reflection\ntests discovery is used).\n\n```fsharp\nopen Expecto\n\n[\u003cPTests\u003e]\nlet skippedTestFromReflectionDiscovery = testCase \"skipped\" \u003c| fun () -\u003e\n    Expect.equal (2+2) 4 \"2+2\"\n\n[\u003cTests\u003e]\nlet myTests =\n  testList \"normal\" [\n    testList \"unfocused list\" [\n      ptestCase \"skipped\" \u003c| fun () -\u003e Expect.equal (2+2) 1 \"2+2?\"\n      testCase \"will run\" \u003c| fun () -\u003e Expect.equal (2+2) 4 \"2+2\"\n      ptest \"skipped\" { Expect.equal (2+2) 1 \"2+2?\" }\n      ptestAsync \"skipped async\" { Expect.equal (2+2) 1 \"2+2?\" }\n    ]\n    testCase \"will run\" \u003c| fun () -\u003e Expect.equal (2+2) 4 \"2+2\"\n    ptestCase \"skipped\" \u003c| fun () -\u003e Expect.equal (2+2) 1 \"2+2?\"\n    ptestList \"skipped list\" [\n      testCase \"skipped\" \u003c| fun () -\u003e Expect.equal (2+2) 1 \"2+2?\"\n      ftest \"skipped\" { Expect.equal (2+2) 1 \"2+2?\" }\n    ]\n  ]\n```\n\nOptionally, in the `TestCode` (function body):\n\n- `Tests.skiptest`\n- `Tests.skiptestf`\n\n### Focusing tests\n\nFocusing can be done with\n\n- `ftestCase`\n- `ftestList`\n- `ftestCaseAsync`\n- `ftestCaseTask`\n- `ftest`\n- `ftestAsync`\n- `ftestTask`\n- `ftestTheory`\n- `ftestTheoryAsync`\n- `ftestTheoryTask`\n\nIt is often convenient, when developing to be able to run a subset of specs.  Expecto allows you to focus specific test\ncases or tests list by putting `f` before *testCase* or *testList* or `F` before attribute *Tests*(when reflection tests\ndiscovery is used).\n\n```fsharp\nopen Expecto\n[\u003cFTests\u003e]\nlet someFocusedTest = test \"will run\" { Expect.equal (2+2) 4 \"2+2\" }\n[\u003cTests\u003e]\nlet someUnfocusedTest = test \"skipped\" { Expect.equal (2+2) 1 \"2+2?\" }\n```\n\nor\n\n```fsharp\nopen Expecto\n\n[\u003cTests\u003e]\nlet focusedTests =\n  testList \"unfocused list\" [\n    ftestList \"focused list\" [\n      testCase \"will run\" \u003c| fun () -\u003e Expect.equal (2+2) 4 \"2+2\"\n      ftestCase \"will run\" \u003c| fun () -\u003e Expect.equal (2+2) 4 \"2+2\"\n      test \"will run\" { Expect.equal (2+2) 4 \"2+2\" }\n    ]\n    testList \"unfocused list\" [\n      testCase \"skipped\" \u003c| fun () -\u003e Expect.equal (2+2) 1 \"2+2?\"\n      ftestCase \"will run\" \u003c| fun () -\u003e Expect.equal (2+2) 4 \"2+2\"\n      test \"skipped\" { Expect.equal (2+2) 1 \"2+2?\" }\n      ftest \"will run\" { Expect.equal (2+2) 4 \"2+2\" }\n    ]\n    testCase \"skipped\" \u003c| fun () -\u003e Expect.equal (2+2) 1 \"2+2?\"\n  ]\n```\n\nExpecto accepts the command line argument `--fail-on-focused-tests`, which checks if focused tests exist.  This\nparameter can be set in build scripts and allows CI servers to reject commits that accidentally included focused tests.\n\n### Sequenced tests\n\nYou can mark an individual spec or container as Sequenced.  This will make sure these tests are run sequentially.  This\ncan be useful for timeout and performance testing.\n\n```fsharp\n[\u003cTests\u003e]\nlet timeout =\n  testSequenced \u003c| testList \"Timeout\" [\n    test \"fail\" {\n      let test = TestCase(Test.timeout 10 (TestCode.Sync (fun _ -\u003e Thread.Sleep 100)), Normal)\n      async {\n        let! eval = Impl.evalTests defaultConfig test\n        let result = Impl.TestRunSummary.fromResults eval\n        result.failed.Length ==? 1\n      } |\u003e Async.RunSynchronously\n    }\n    test \"pass\" {\n      let test = TestCase(Test.timeout 1000 (TestCode.Sync ignore), Normal)\n      async {\n        let! eval = Impl.evalTests defaultConfig test\n        let result = Impl.TestRunSummary.fromResults eval\n        result.passed.Length ==? 1\n      } |\u003e Async.RunSynchronously\n    }\n  ]\n```\n\nYou can also mark a test list as a Sequenced Group.  This will make sure the tests in this group are not run at the same\ntime.\n\n```fsharp\n[\u003cTests\u003e]\nlet timeout =\n  let lockOne = obj()\n  let lockTwo = obj()\n  testSequencedGroup \"stop deadlock\" \u003c| testList \"possible deadlock\" [\n    testAsync \"case A\" {\n      lock lockOne (fun () -\u003e\n        Thread.Sleep 10\n        lock lockTwo (fun () -\u003e\n          ()\n        )\n      )\n    }\n    testAsync \"case B\" {\n      lock lockTwo (fun () -\u003e\n        Thread.Sleep 10\n        lock lockOne (fun () -\u003e\n          ()\n        )\n      )\n    }\n  ]\n```\n\n### Parameterised tests with `testParam`\n\n- `testParam`\n\n```fsharp\ntestList \"numberology 101\" (\n  testParam 1333 [\n    \"First sample\",\n      fun value () -\u003e\n        Expect.equal value 1333 \"Should be expected value\"\n    \"Second sample\",\n      fun value () -\u003e\n        Expect.isLessThan value 1444 \"Should be less than\"\n] |\u003e List.ofSeq)\n```\n\n### Setup and teardown\n\nA simple way to perform setup and teardown is by using `IDisposable` resources:\n\n```fsharp\nlet simpleTests =\n    testList \"simples\" [\n        test \"test one\" {\n            use resource = new MyDatabase()\n            // test code\n        }\n    ]\n```\n\nFor more complex setup and teardown situations we can write one or more setup\nfunctions to manage resources:\n\n```fsharp\nlet clientTests setup =\n    [\n        test \"test1\" {\n            setup (fun client store -\u003e\n                // test code\n            )\n        }\n        test \"test2\" {\n            setup (fun client store -\u003e\n                // test code\n            )\n        }\n        // other tests\n    ]\n\nlet clientMemoryTests =\n    clientTests (fun test -\u003e\n        let client = memoryClient()\n        let store = memoryStore()\n        test client store\n    )\n    |\u003e testList \"client memory tests\"\n\nlet clientIntegrationTests =\n    clientTests (fun test -\u003e\n        // setup code\n        try\n            let client = realTestClient()\n            let store = realTestStore()\n            test client store\n        finally\n            // teardown code\n    )\n    |\u003e testList \"client integration tests\"\n```\n\n### Property based tests\n\nReference [FsCheck](https://github.com/fscheck/FsCheck) and Expecto.FsCheck to test properties.\n\n```fsharp\nmodule MyApp.Tests\n\n// the ExpectoFsCheck module is auto-opened by this\n// the configuration record is in the Expecto namespace in the core library\nopen Expecto\n\nlet config = { FsCheckConfig.defaultConfig with maxTest = 10000 }\n\nlet properties =\n  testList \"FsCheck samples\" [\n    testProperty \"Addition is commutative\" \u003c| fun a b -\u003e\n      a + b = b + a\n\n    testProperty \"Reverse of reverse of a list is the original list\" \u003c|\n      fun (xs:list\u003cint\u003e) -\u003e List.rev (List.rev xs) = xs\n\n    // you can also override the FsCheck config\n    testPropertyWithConfig config \"Product is distributive over addition\" \u003c|\n      fun a b c -\u003e\n        a * (b + c) = a * b + a * c\n  ]\n\nTests.runTestsWithCLIArgs [] [||] properties\n```\n\nYou can freely mix testProperty with testCase and testList. The config looks\nlike the following.\n\n```fsharp\ntype FsCheckConfig =\n    /// The maximum number of tests that are run.\n  { maxTest: int\n    /// The size to use for the first test.\n    startSize: int\n    /// The size to use for the last test, when all the tests are passing. The size increases linearly between Start- and EndSize.\n    endSize: int\n    /// If set, the seed to use to start testing. Allows reproduction of previous runs.\n    replay: (uint64 * uint64) option\n    /// The Arbitrary instances on this class will be merged in back to front order, i.e. instances for the same generated type at the front\n    /// of the list will override those at the back. The instances on Arb.Default are always known, and are at the back (so they can always be\n    /// overridden)\n    arbitrary: Type list\n    /// Callback when the test case had input parameters generated.\n    receivedArgs: FsCheckConfig\n               -\u003e (* test name *) string\n               -\u003e (* test number *) int\n               -\u003e (* generated arguments *) obj list\n               -\u003e Async\u003cunit\u003e\n    /// Callback when the test case was successfully shrunk\n    successfulShrink: FsCheckConfig\n                   -\u003e (* test name *) string\n                   -\u003e (* shrunk new arguments *) obj list\n                   -\u003e Async\u003cunit\u003e\n    /// Callback when the test case has finished\n    finishedTest: FsCheckConfig\n               -\u003e (* test name *) string\n               -\u003e FsCheckTestData\n               -\u003e Async\u003cunit\u003e\n  }\n```\n\nHere is another example of testing with custom generated data:\n\n```fsharp\nmodule MyApp.Tests\n\n// the ExpectoFsCheck module is auto-opened by this\n// the configuration record is in the Expecto namespace in the core library\nopen Expecto\nopen FsCheck\n\ntype User = {\n    Id : int\n    FirstName : string\n    LastName : string\n}\n\ntype UserGen() =\n   static member User() : Arbitrary\u003cUser\u003e =\n       let genFirsName = Gen.elements [\"Don\"; \"Henrik\"; null]\n       let genLastName = Gen.elements [\"Syme\"; \"Feldt\"; null]\n       let createUser id firstName lastName =\n           {Id = id; FirstName = firstName ; LastName = lastName}\n       let getId = Gen.choose(0,1000)\n       let genUser =\n           createUser \u003c!\u003e getId \u003c*\u003e genFirsName \u003c*\u003e genLastName\n       genUser |\u003e Arb.fromGen\n\nlet config = { FsCheckConfig.defaultConfig with arbitrary = [typeof\u003cUserGen\u003e] }\n\nlet properties =\n  testList \"FsCheck samples\" [\n\n    // you can also override the FsCheck config\n    testPropertyWithConfig config \"User with generated User data\" \u003c|\n      fun x -\u003e\n        Expect.isNotNull x.FirstName \"First Name should not be null\"\n  ]\n\nTests.runTestsWithCLIArgs [] [||] properties\n```\n\nAnd a further example of creating constraints on generated values\n\n```fsharp\nopen System\nopen Expecto\nopen FsCheck\n\nmodule Gen =\n    type Float01 = Float01 of float\n    let float01Arb =\n        let maxValue = float UInt64.MaxValue\n        Arb.convert\n            (fun (DoNotSize a) -\u003e float a / maxValue |\u003e Float01)\n            (fun (Float01 f) -\u003e f * maxValue + 0.5 |\u003e uint64 |\u003e DoNotSize)\n            Arb.from\n    type 'a ListOf100 = ListOf100 of 'a list\n    let listOf100Arb() =\n        Gen.listOfLength 100 Arb.generate\n        |\u003e Arb.fromGen\n        |\u003e Arb.convert ListOf100 (fun (ListOf100 l) -\u003e l)\n    type 'a ListOfAtLeast2 = ListOfAtLeast2 of 'a list\n    let listOfAtLeast2Arb() =\n        Arb.convert\n            (fun (h1,h2,t) -\u003e ListOfAtLeast2 (h1::h2::t))\n            (function\n                | ListOfAtLeast2 (h1::h2::t) -\u003e h1,h2,t\n                | e -\u003e failwithf \"not possible in listOfAtLeast2Arb: %A\" e)\n            Arb.from\n    let addToConfig config =\n        {config with arbitrary = typeof\u003cFloat01\u003e.DeclaringType::config.arbitrary}\n\n[\u003cAutoOpen\u003e]\nmodule Auto =\n    let private config = Gen.addToConfig FsCheckConfig.defaultConfig\n    let testProp name = testPropertyWithConfig config name\n    let ptestProp name = ptestPropertyWithConfig config name\n    let ftestProp name = ftestPropertyWithConfig config name\n    let etestProp stdgen name = etestPropertyWithConfig stdgen config name\n\nmodule Tests =\n    let topicTests =\n        testList \"topic\" [\n            testProp \"float between 0 and 1\" (fun (Gen.Float01 f) -\u003e\n                () // test\n            )\n            testProp \"list of 100 things\" (fun (Gen.ListOf100 l) -\u003e\n                () // test\n            )\n            testProp \"list of at least 2 things\" (fun (Gen.ListOfAtLeast2 l) -\u003e\n                () // test\n            )\n            testProp \"list of at least 2 things without gen\" (fun h1 h2 t -\u003e\n                let l = h1::h2::t\n                () // test\n            )\n        ]\n```\n\nIt will be translated to the FsCheck-specific configuration at runtime. You can pass your own callbacks and use\n`Expecto.Logging` like shown in the\n[Sample](https://github.com/haf/expecto/blob/master/Expecto.Sample/Expecto.Sample.fs#L23) to get inputs for tests and\ntests printed.\n\nIf a property fails, the output could look like this.\n\n    [11:06:35 ERR] samples/addition is not commutative (should fail) failed in 00:00:00.0910000.\n    Failed after 1 test. Parameters:\n      2 1\n    Shrunk 2 times to:\n      1 0\n    Result:\n      False\n    Focus on error:\n      etestProperty (1865288075UL, 296281834UL) \"addition is not commutative (should fail)\"\n\nThe output that Expecto gives you, lets you recreate the exact test (that's from the 18..., 29... seed numbers). It's\nalso a good idea to lift inputs and the test-case/parameter combination that failed into its *own* test (which isn't a\nproperty based test).\n\nFsCheck `Arb.Register` can't be used with Expecto because it is thread local and Expecto runs multithreaded by default.\nThis could be worked around but `Arb.Register` is being deprecated by FsCheck. The recommended way to register and use\ncustom generators is to define `testPropertyWithConfig` functions like `testProp` above for each area with common\ngenerator use. This ensures the library will always be used in a thread safe way.\n\n#### Link collection\n\nThese are a few resources that will get you on your way towards fully-specified systems with property-based testing.\n\n- [An introduction to property-based testing](http://fsharpforfunandprofit.com/posts/property-based-testing/) with [slides and video](http://fsharpforfunandprofit.com/pbt/)\n- [Choosing properties for property-based testing](http://fsharpforfunandprofit.com/posts/property-based-testing-2/)\n- [(video) Race conditions, distribution and interactions](https://vimeo.com/68383317)\n- [Test data: generators, shrinkers and Arbitrary instances](https://fscheck.github.io/FsCheck/TestData.html)\n- [Model based testing](https://fscheck.github.io/FsCheck/StatefulTesting.html)\n- [Testing and quality assurance in Haskell](http://book.realworldhaskell.org/read/testing-and-quality-assurance.html)\n- [Property-based testing for better code](https://www.youtube.com/watch?v=shngiiBfD80)\n\n#### Code from FsCheck\n\nThese code snippets show a bit of the API usage and how to create Arbitrary\ninstances (which encapsulate generation with Gen instances and shrinkage),\nrespectively.\n\n- [FsCheck Examples.fs](https://github.com/fscheck/FsCheck/blob/master/examples/FsCheck.Examples/Examples.fs)\n- [FsCheck Arbitrary.fs](https://github.com/fscheck/FsCheck/blob/master/src/FsCheck/Arbitrary.fs#L26)\n\n## Expectations with `Expect`\n\nAll expect-functions have the signature `actual -\u003e expected -\u003e string -\u003e unit`,\nleaving out `expected` when obvious from the function.\n\n### `Expect` module\n\nThis module is your main entry-point when asserting.\n\n- `throws`\n- `throwsC`\n- `throwsT`\n- `throwsAsync`\n- `throwsAsyncC`\n- `throwsAsyncT`\n- `isNone`\n- `isSome`\n- `isChoice1Of2`\n- `isChoice2Of2`\n- `isOk` - Expect the value to be a Result.Ok value\n- `isError` - Expect the value to be a Result.Error value\n- `isNull`\n- `isNotNull`\n- `isNotNaN`\n- `isNotPositiveInfinity`\n- `isNotNegativeInfinity`\n- `isNotInfinity`\n- `isLessThan`\n- `isLessThanOrEqual`\n- `isGreaterThan`\n- `isGreaterThanOrEqual`\n- `notEqual`\n- `isFalse`\n- `isTrue`\n- `exists` - Expect that some element from `actual` sequence satisfies the given `asserter`\n- `all` - Expect that all elements from `actual` satisfy the given `asserter`\n- `allEqual` - Expect that all elements from `actual` are equal to `equalTo`\n- `sequenceEqual`\n- `floatClose : Accuracy -\u003e float -\u003e float -\u003e string -\u003e unit` - Expect the\n   floats to be within the combined absolute and relative accuracy given by\n   `abs(a-b) \u003c= absolute + relative * max (abs a) (abs b)`. Default accuracy\n   available are: `Accuracy.low = {absolute=1e-6; relative=1e-3}`,\n   `Accuracy.medium = {absolute=1e-8; relative=1e-5}`,\n   `Accuracy.high = {absolute=1e-10; relative=1e-7}`,\n   `Accuracy.veryHigh = {absolute=1e-12; relative=1e-9}`.\n- `floatLessThanOrClose : Accuracy -\u003e float -\u003e float -\u003e string -\u003e unit` - Expect actual\n   to be less than expected or close.\n- `floatGreaterThanOrClose : Accuracy -\u003e float -\u003e float -\u003e string -\u003e unit` - Expect actual\n   to be greater than expected or close.\n- `sequenceStarts` - Expect the sequence `subject` to start with `prefix`. If\n   it does not then fail with `format` as an error message together with a\n   description of `subject` and `prefix`.\n- `sequenceContainsOrder` - Expect the sequence `actual` to contains elements from sequence `expected` in the right order.\n- `isAscending` - Expect the sequence `subject` to be ascending. If it does not\n   then fail with `format` as an error message.\n- `isDescending` - Expect the sequence `subject` to be descending. If it does\n   not then fail with `format` as an error message.\n- `stringContains` – Expect the string `subject` to contain `substring` as part\n   of itself.  If it does not, then fail with `format` and `subject` and\n   `substring` as part of the error message.\n- `isMatch` - Expect the string `actual` to match `pattern`\n- `isRegexMatch` - Expect the string `actual` to match `regex`\n- `isMatchGroups` - Expects the string `actual` that matched groups (from a `pattern` match) match with `matchesOperator`\n- `isMatchRegexGroups` - Expects the string `actual` that matched groups (from a `regex` match) match with `matchesOperator`\n- `isNotMatch` - Expect the string `actual` to not match `pattern`\n- `isNotRegexMatch` - Expect the string `actual` to not match `regex`\n- `stringStarts` – Expect the string `subject` to start with `prefix` and if it\n   does not then fail with `format` as an error message together with a\n   description of `subject` and `prefix`.\n- `stringEnds` - Expect the string `subject` to end with `suffix`. If it does\n   not then fail with `format` as an error message together with a description\n   of `subject` and `suffix`.\n- `stringHasLength` - Expect the string `subject` to have length equals\n   `length`. If it does not then fail with `format` as an error message together\n   with a description of `subject` and `length`.\n- `isNotEmpty` - Expect the string `actual` to be not null nor empty\n- `isNotWhitespace` - Expect the string `actual` to be not null nor empty nor whitespace\n- `isEmpty` - Expect the sequence `actual` to be empty\n- `isNonEmpty` - Expect the sequence `actual` to be not empty\n- `hasCountOf` - Expect that the counts of the found value occurrences by `selector` in `actual` equals the `expected`.\n- `contains : 'a seq -\u003e 'a -\u003e string -\u003e unit` – Expect the sequence to contain\n   the item.\n- `containsAll: 'a seq -\u003e 'a seq -\u003e string -\u003e unit` - Expect the sequence\n   contains all elements from second sequence (not taking into account an order\n   of elements).\n- `distribution: 'a seq -\u003e Map\u003c'a, uint32\u003e -\u003e string -\u003e unit` - Expect the sequence contains all elements from map (first element in tuple is an item expected to be in sequence, second is a positive number of its occurrences in a sequence). Function is not taking into account an order of elements.\n- `streamsEqual` – Expect the streams to be byte-wise identical.\n- `isFasterThan : (unit -\u003e 'a) -\u003e (unit -\u003e 'a) -\u003e string -\u003e unit` – Expect the\n    first function to be faster than the second function with the passed string\n    message, printed on failure. See the next section on [Performance](#performance-module) for example\n    usage.\n- `isFasterThanSub` – Like the above but with passed function signature of\n   `Performance.Measurer\u003cunit,'a\u003e -\u003e 'a`, allowing you to do setup and teardown\n   of your subject under test (the function) before calling the `measurer`. See\n   the next section on [Performance](#performance-module) for example usage.\n- `wantOk` - Expect the result to be `Ok` and returns its value, otherwise fails.\n- `wantError` - Expect the result to be `Error` and returns its value, otherwise fails.\n\nAlso note, that there's a \"fluent\" API, with which you can pipe the test-subject\nvalue into the expectation:\n\n```fsharp\nopen Expecto\nopen Expecto.Flip\n\nlet compute (multiplier: int) = 42 * multiplier\n\ntest \"yup yup\" {\n  compute 1\n    |\u003e Expect.equal \"x1 = 42\" 42\n\n  compute 2\n    |\u003e Expect.equal \"x2 = 82\" 84\n}\n|\u003e runTestsWithCLIArgs [] [||]\n```\n\n### `Performance` module\n\nExpecto supports testing that an implementation is faster than another. Use it\nby calling `Expect.isFasterThan` wrapping your `Test` in `testSequenced`.\n\n![Sample output](./docs/half-is-faster.png)\n\nThis function makes use of a statistical test called [Welch's\nt-test](https://en.wikipedia.org/wiki/Welch's_t-test).  It starts with the null\nhypothesis that the functions mean execution times are the same.  The functions\nare run alternately increasing the sample size to test this hypothesis.\n\nOnce the probability of getting this result based on the null hypothesis goes\nbelow 0.01% it rejects the null hypothesis and reports the results.  If the\nperformance is very close the test will declare them equal when there is 99.99%\nconfidence they differ by less than 0.5%.  0.01%/99.99% are chosen such that if\na test list has 100 performance tests a false test failure would be reported\nonce in many more than 100 runs.\n\nThis results in a performance test that is very quick to run (the greater the\ndifference the quicker it will run).  Also, because it is a relative test it can\nnormally be run across all configurations as part of unit testing.\n\nThe functions must return the same result for same input. Note that since\nExpecto also has a FsCheck integration, your outer (sequenced) test could be\nthe property test, generating random data, and your TestCode/function body/\nactual test could be an assertion that for the same (random instance) of test\ndata, one function should be faster than the other.\n\nFrom `Expect.isFasterThanSub`, these results are possible (all of which generate\na test failure, except the `MetricLessThan` case):\n\n```fsharp\n  type 'a CompareResult =\n    | ResultNotTheSame of result1:'a * result2:'a\n    | MetricTooShort of sMax:SampleStatistics * machineResolution:SampleStatistics\n    | MetricLessThan of s1:SampleStatistics * s2:SampleStatistics\n    | MetricMoreThan of s1:SampleStatistics * s2:SampleStatistics\n    | MetricEqual of s1:SampleStatistics * s2:SampleStatistics\n```\n\nYou can explore these cases yourself with `Expecto.Performance.timeCompare`,\nshould you wish to.\n\n#### Example\n\nAll of the below tests pass.\n\n```fsharp\n[\u003cTests\u003e]\nlet performance =\n  testSequenced \u003c| testList \"performance\" [\n\n    testCase \"1 \u003c\u003e 2\" \u003c| fun () -\u003e\n      let test () =\n        Expect.isFasterThan (fun () -\u003e 1) (fun () -\u003e 2) \"1 equals 2 should fail\"\n      assertTestFailsWithMsgContaining \"same\" (test, Normal)\n\n    testCase \"half is faster\" \u003c| fun () -\u003e\n      Expect.isFasterThan (fun () -\u003e repeat10000 log 76.0)\n                          (fun () -\u003e repeat10000 log 76.0 |\u003e ignore; repeat10000 log 76.0)\n                          \"half is faster\"\n\n    testCase \"double is faster should fail\" \u003c| fun () -\u003e\n      let test () =\n        Expect.isFasterThan (fun () -\u003e repeat10000 log 76.0 |\u003e ignore; repeat10000 log 76.0)\n                            (fun () -\u003e repeat10000 log 76.0)\n                            \"double is faster should fail\"\n      assertTestFailsWithMsgContaining \"slower\" (test, Normal)\n\n    ptestCase \"same function is faster should fail\" \u003c| fun () -\u003e\n      let test () =\n        Expect.isFasterThan (fun () -\u003e repeat100000 log 76.0)\n                            (fun () -\u003e repeat100000 log 76.0)\n                            \"same function is faster should fail\"\n      assertTestFailsWithMsgContaining \"equal\" (test, Normal)\n\n    testCase \"matrix\" \u003c| fun () -\u003e\n      let n = 100\n      let rand = Random 123\n      let a = Array2D.init n n (fun () _ -\u003e rand.NextDouble())\n      let b = Array2D.init n n (fun () _ -\u003e rand.NextDouble())\n      let c = Array2D.zeroCreate n n\n\n      let reset() =\n        for i = 0 to n-1 do\n            for j = 0 to n-1 do\n              c.[i,j] \u003c- 0.0\n\n      let mulIJK() =\n        for i = 0 to n-1 do\n          for j = 0 to n-1 do\n            for k = 0 to n-1 do\n              c.[i,k] \u003c- c.[i,k] + a.[i,j] * b.[j,k]\n\n      let mulIKJ() =\n        for i = 0 to n-1 do\n          for k = 0 to n-1 do\n            let mutable t = 0.0\n            for j = 0 to n-1 do\n              t \u003c- t + a.[i,j] * b.[j,k]\n            c.[i,k] \u003c- t\n      Expect.isFasterThanSub (fun measurer -\u003e reset(); measurer mulIKJ ())\n                             (fun measurer -\u003e reset(); measurer mulIJK ())\n                             \"ikj faster than ijk\"\n\n    testCase \"popcount\" \u003c| fun () -\u003e\n      let test () =\n        Expect.isFasterThan (fun () -\u003e repeat10000 (popCount16 \u003e\u003e int) 987us)\n                            (fun () -\u003e repeat10000 (popCount \u003e\u003e int) 987us)\n                            \"popcount 16 faster than 32 fails\"\n      assertTestFailsWithMsgContaining \"slower\" (test, Normal)\n  ]\n```\n\nA failure would look like this:\n\n```\n[13:23:19 ERR] performance/double is faster failed in 00:00:00.0981990.\ndouble is faster. Expected f1 (0.3067 ± 0.0123 ms) to be faster than f2 (0.1513 ± 0.0019 ms) but is ~103% slower.\n```\n\n### Performance.findFastest\n\nExpecto can use `isFasterThan` to find the fastest version of a function for a given `int` input.\nThis can be useful for optimising algorithm constants such as buffer size.\n\n```fsharp\n[\u003cTests\u003e]\nlet findFastest =\n  testSequenced \u003c| testList \"findFastest\" [\n\n    testCase \"different values gives an error\" (fun _ -\u003e\n      Performance.findFastest id 10 20 |\u003e ignore\n    ) |\u003e assertTestFailsWithMsgStarting \"Expected results to be the same.\"\n\n    testCase \"find fastest sleep\" (fun _ -\u003e\n      let f i = Threading.Thread.Sleep(abs(i-65)*10)\n      let result = Performance.findFastest f 0 100\n      Expect.equal result 65 \"find min\"\n    )\n\n    testCase \"find fastest hi\" (fun _ -\u003e\n      let f i = Threading.Thread.Sleep(abs(i-110)*10)\n      let result = Performance.findFastest f 0 100\n      Expect.equal result 100 \"find min\"\n    )\n\n    testCase \"find fastest lo\" (fun _ -\u003e\n      let f i = Threading.Thread.Sleep(abs(i+10)*10)\n      let result = Performance.findFastest f 0 100\n      Expect.equal result 0 \"find min\"\n    )\n  ]\n```\n\n## `main args` and command line – how to run console apps examples\n\nFrom code you can run:\n\n```fsharp\nTests.runTestsInAssemblyWithCLIArgs [Stress 0.1;Stress_Timeout 0.2] [||]\n```\n\nFrom the command line you can run:\n\n```\ndotnet run -p Expecto.Tests -f net6.0 -c release -- --help\ndotnet watch -p Expecto.Tests run -f net6.0 -c release -- --colours 256\n```\n\n## Contributing and building \n\nPlease review the [guidelines for contributing](./CONTRIBUTING.md) to Expecto;\nthis document also includes instructions on how to build.\n\nWe'd specifically like to call out the following people for their great contributions to Expecto in the past:\n\n- @mausch — for building Fuchu which became the foundation of Expecto\n- @AnthonyLloyd — for maintaining Expecto for some years and drastically improving it\n\n## BenchmarkDotNet usage\n\nThe integration with\n[BenchmarkDotNet](http://benchmarkdotnet.org/).\n\n```fsharp\nopen Expecto\n\ntype ISerialiser =\n  abstract member Serialise\u003c'a\u003e : 'a -\u003e unit\n\ntype MySlowSerialiser() =\n  interface ISerialiser with\n    member __.Serialise _ =\n      System.Threading.Thread.Sleep(30)\n\ntype FastSerialiser() =\n  interface ISerialiser with\n    member __.Serialise _ =\n      System.Threading.Thread.Sleep(10)\n\ntype FastSerialiserAlt() =\n  interface ISerialiser with\n    member __.Serialise _ =\n     System.Threading.Thread.Sleep(20)\n\ntype Serialisers() =\n  let fast, fastAlt, slow =\n    FastSerialiser() :\u003e ISerialiser,\n    FastSerialiserAlt() :\u003e ISerialiser,\n    MySlowSerialiser() :\u003e ISerialiser\n\n  [\u003cBenchmark\u003e]\n  member __.FastSerialiserAlt() = fastAlt.Serialise \"Hello world\"\n\n  [\u003cBenchmark\u003e]\n  member __.SlowSerialiser() = slow.Serialise \"Hello world\"\n\n  [\u003cBenchmark(Baseline = true)\u003e]\n  member __.FastSerialiser() = fast.Serialise \"Hello world\"\n\n[\u003cTests\u003e]\nlet tests =\n  testList \"performance tests\" [\n    test \"three serialisers\" {\n      benchmark\u003cSerialisers\u003e benchmarkConfig (fun _ -\u003e null) |\u003e ignore\n    }\n  ]\n```\n\nIn the current code-base I'm just printing the output to the console; and by\ndefault all tests are run in parallel; so you'll need to use `--sequenced` as\ninput to your exe, or set parallel=false in the config to get valid results.\n\nTo read more about how to benchmark with BenchmarkDotNet, see its [Getting\nstarted](http://benchmarkdotnet.org/GettingStarted.htm) guide.\n\nHappy benchmarking!\n\n## You're not alone!\n\nOthers have discovered the beauty of tests-as-values in easy-to-read F#.\n\n* [Logary](https://github.com/logary/logary)\n* [Unquote has built-in support](https://github.com/SwensenSoftware/unquote/pull/128)\n* [Visual Studio Plugin for\n  Expecto](https://www.nuget.org/packages/Expecto.VisualStudio.TestAdapter/) –\n  just add the `nuget Expecto.VisualStudio.TestAdapter version_in_path: true` to\n  your Paket file and you're off to the races!\n* [FsCheck supports it](https://fscheck.github.io/FsCheck/QuickStart.html#Integration-with-Expecto)\n\n![Expecto VS Test Plugin](./docs/expecto-vs-addon.jpeg \"Easy to get started even\nfor Enterprise Developers\")\n\n### Testing hardware\n\nPeople have been testing hardware with Expecto.\n\n![Expecto Hardware Testing](./docs/hw-testing.jpg \"Testing Hardware with\nExpecto, photo by Roman Provazník\")\n\n## Sending e-mail on failure – custom printers\n\nThe printing mechanism in Expecto is based on the [Logary\nFacade](https://github.com/logary/logary#the-logary-facade-adapter), which\ngrants some privileges, like being able to use **any** Logary target to print.\nJust follow the above link to learn how to initialise Logary. Then if you wanted\nto get notified over e-mail whenever one of your tests fail, configure Logary\nwith `Logary.Targets.Mailgun`:\n\n```fsharp\nopen Logary\nopen Logary.Configuration\nopen Logary.Adapters.Facade\nopen Logary.Targets\nopen Hopac\nopen Mailgun\nopen System.Net.Mail\n\nlet main argv =\n  let mgc =\n    MailgunLogaryConf.Create(\n      MailAddress(\"travis@example.com\"),\n      [ MailAddress(\"Your.Mail.Here@example.com\") ],\n      { apiKey = \"deadbeef-2345678\" },\n      \"example.com\", // sending domain of yours\n      Error) // cut-off level\n\n  use logary =\n    withLogaryManager \"MyTests\" (\n      withTargets [\n        LiterateConsole.create LiterateConsole.empty \"stdout\"\n        Mailgun.create mgc \"mail\"\n      ]\n      \u003e\u003e withRules [\n        Rule.createForTarget \"stdout\"\n        Rule.createForTarget \"mail\"\n      ])\n    |\u003e run\n\n  // initialise Logary Facade with Logary proper:\n  LogaryFacadeAdapter.initialise\u003cExpecto.Logging.Logger\u003e logary\n\n  // run all tests\n  Tests.runTestsInAssemblyWithCLIArgs [] argv\n```\n\n## About test parallelism\n\nSince the default is to run all of your tests in parallel, it's important that\nyou don't use global variables, global singletons or mutating code. If you do,\nyou'll have to slow down all of your tests by sequencing them (or use locks in\nyour testing code).\n\nFurthermore, `printfn` and sibling functions aren't thread-safe, i.e. a given\nstring may be logged in many passes and concurrent calls to printfn and\nConsole.X-functions have their outputs interleaved. If you want to log from\ntests, you can use code like:\n\n```fsharp\nopen Expecto.Logging\nopen Expecto.Logging.Message\n\nlet logger = Log.create \"MyTests\"\n\n// stuff here\n\ntestCase \"reading prop\" \u003c| fun () -\u003e\n  let subject = MyComponent()\n  // this will output to the right test context:\n  logger.info(\n    eventX \"Has prop {property}\"\n    \u003e\u003e setField \"property\" subject.property)\n  Expect.equal subject.property \"Goodbye\" \"Should have goodbye as its property\"\n```\n\n### What does 'expected to have type TestCode' mean?\n\nIf you get an error message like this:\n\n```\nThis expression was expected to have type    'TestCode'    but here has type    'unit'\n```\n\nIt means that you have code like `testCase \"abc\" \u003c| Expect.equal ...`. Instead\nyou should create a function like so: `testCase \"abc\" \u003c| fun () -\u003e Expect.equal\n...`.\n\n### My tests are hanging and I can't see why\n\nThis might be due to how terminals/the locking thereof work: try running your tests with `--no-spinner` and see if that works.\n\n![Expecto expecto](./docs/expecto-patronus-2000x1126.png)\n\n\n[logary]: https://github.com/logary/logary#using-logary-in-a-library\n\n## Migration notes\n\n### 11.0.0\n- Expecto 11.0.0-alpha5 breaks compatibility with YoloDev.Expecto.TestSdk \u003c= 0.14.3. YoloDev.Expecto.TestSdk \u003e= 0.15 is required for use with VisualStudio, Rider, dotnet test, and anything that uses the vstest adapter system.\n- Any usages of the `replay` (a.k.a `stdGen` with `etestProperty*` functions) config with FsCheck tests will need to be updated to use `uint64` by appending `UL` to the literals. They will also now require a third item indicating the size. E.g. from `(1865288075, 296281834)` to `(1865288075UL, 296281834UL, Some 3)`. With FsCheck 2 the size is ignored and `None` can be used.\n- FsCheck 2 is no longer supported, so we're switching Expecto.FsCheck to use FsCheck 3 by default, even though FsCheck 3 is still in release candidate state. If you still want FsCheck2, we will continue to release FsCheck2 support for the time being using a version suffix, e.g. [11.0.0-fscheck2](https://www.nuget.org/packages/Expecto.FsCheck/11.0.0-alpha1-fscheck2)  \n","funding_links":["https://github.com/sponsors/haf"],"categories":["Testing","测试","Identifiers"],"sub_categories":["Performance Analysis","GUI - other"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhaf%2Fexpecto","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhaf%2Fexpecto","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhaf%2Fexpecto/lists"}