{"id":15555755,"url":"https://github.com/unkindpartition/tasty","last_synced_at":"2025-12-11T23:29:41.506Z","repository":{"id":8169476,"uuid":"9592114","full_name":"UnkindPartition/tasty","owner":"UnkindPartition","description":"Modern and extensible testing framework for Haskell","archived":false,"fork":false,"pushed_at":"2025-03-28T23:16:44.000Z","size":942,"stargazers_count":648,"open_issues_count":26,"forks_count":110,"subscribers_count":15,"default_branch":"master","last_synced_at":"2025-03-31T12:56:16.051Z","etag":null,"topics":["golden-tests","haskell","hunit","quickcheck","smallcheck","testing","unit-testing"],"latest_commit_sha":null,"homepage":"","language":"Haskell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/UnkindPartition.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2013-04-22T06:14:16.000Z","updated_at":"2025-03-28T23:16:45.000Z","dependencies_parsed_at":"2024-04-09T11:39:40.402Z","dependency_job_id":"a1a9eff2-6b59-4640-b26c-f0eb1a702c9d","html_url":"https://github.com/UnkindPartition/tasty","commit_stats":{"total_commits":767,"total_committers":78,"mean_commits":9.833333333333334,"dds":0.2790091264667536,"last_synced_commit":"7d61d9887df5180abf69d6427655d3e71331217d"},"previous_names":["feuerbach/tasty"],"tags_count":118,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UnkindPartition%2Ftasty","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UnkindPartition%2Ftasty/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UnkindPartition%2Ftasty/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UnkindPartition%2Ftasty/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/UnkindPartition","download_url":"https://codeload.github.com/UnkindPartition/tasty/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248208739,"owners_count":21065218,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["golden-tests","haskell","hunit","quickcheck","smallcheck","testing","unit-testing"],"created_at":"2024-10-02T15:10:30.692Z","updated_at":"2025-12-11T23:29:41.470Z","avatar_url":"https://github.com/UnkindPartition.png","language":"Haskell","readme":"# Tasty\n\n**Tasty** is a modern testing framework for Haskell.\n\nIt lets you combine your unit tests, golden tests, QuickCheck/SmallCheck\nproperties, and any other types of tests into a single test suite.\n\nFeatures:\n\n* Run tests in parallel but report results in a deterministic order\n* Filter the tests to be run using patterns specified on the command line\n* Hierarchical, colored display of test results\n* Reporting of test statistics\n* Acquire and release resources (sockets, temporary files etc.) that can be\n  shared among several tests\n* Extensibility: add your own test providers and ingredients (runners) above and\n  beyond those provided\n\nTo find out what's new, read the **[change log][]**.\n\n[change log]: https://github.com/UnkindPartition/tasty/blob/master/core/CHANGELOG.md\n\n## Example\n\nHere's how your `test.hs` might look like:\n\n```haskell\nimport Test.Tasty\nimport Test.Tasty.SmallCheck as SC\nimport Test.Tasty.QuickCheck as QC\nimport Test.Tasty.HUnit\n\nimport Data.List\nimport Data.Ord\n\nmain = defaultMain tests\n\ntests :: TestTree\ntests = testGroup \"Tests\" [properties, unitTests]\n\nproperties :: TestTree\nproperties = testGroup \"Properties\" [scProps, qcProps]\n\nscProps = testGroup \"(checked by SmallCheck)\"\n  [ SC.testProperty \"sort == sort . reverse\" $\n      \\list -\u003e sort (list :: [Int]) == sort (reverse list)\n  , SC.testProperty \"Fermat's little theorem\" $\n      \\x -\u003e ((x :: Integer)^7 - x) `mod` 7 == 0\n  -- the following property does not hold\n  , SC.testProperty \"Fermat's last theorem\" $\n      \\x y z n -\u003e\n        (n :: Integer) \u003e= 3 SC.==\u003e x^n + y^n /= (z^n :: Integer)\n  ]\n\nqcProps = testGroup \"(checked by QuickCheck)\"\n  [ QC.testProperty \"sort == sort . reverse\" $\n      \\list -\u003e sort (list :: [Int]) == sort (reverse list)\n  , QC.testProperty \"Fermat's little theorem\" $\n      \\x -\u003e ((x :: Integer)^7 - x) `mod` 7 == 0\n  -- the following property does not hold\n  , QC.testProperty \"Fermat's last theorem\" $\n      \\x y z n -\u003e\n        (n :: Integer) \u003e= 3 QC.==\u003e x^n + y^n /= (z^n :: Integer)\n  ]\n\nunitTests = testGroup \"Unit tests\"\n  [ testCase \"List comparison (different length)\" $\n      [1, 2, 3] `compare` [1,2] @?= GT\n\n  -- the following test does not hold\n  , testCase \"List comparison (same length)\" $\n      [1, 2, 3] `compare` [1,2,2] @?= LT\n  ]\n```\n\nAnd here is the output of the above program:\n\n![](https://raw.github.com/UnkindPartition/tasty/master/screenshot.png)\n\n(Note that whether QuickCheck finds a counterexample to the third property is\ndetermined by chance.)\n\n## Packages\n\n[tasty][] is the core package. It contains basic definitions and APIs and a\nconsole runner.\n\n[tasty]: https://hackage.haskell.org/package/tasty\n\nIn order to create a test suite, you also need to install one or more «providers» (see\nbelow).\n\n### Providers\n\nThe following providers exist:\n\n* [tasty-hunit](https://hackage.haskell.org/package/tasty-hunit) — for unit tests\n  (based on [HUnit](https://hackage.haskell.org/package/HUnit))\n* [tasty-golden][] — for golden\n  tests, which are unit tests whose results are kept in files\n* [tasty-smallcheck](https://hackage.haskell.org/package/tasty-smallcheck) —\n  exhaustive property-based testing\n  (based on [smallcheck](https://hackage.haskell.org/package/smallcheck))\n* [tasty-quickcheck](https://hackage.haskell.org/package/tasty-quickcheck) — for randomized\n  property-based testing (based on [QuickCheck](http://hackage.haskell.org/package/QuickCheck))\n* [tasty-hedgehog](https://hackage.haskell.org/package/tasty-hedgehog) — for randomized\n  property-based testing (based on [Hedgehog](https://hackage.haskell.org/package/hedgehog))\n* [tasty-hspec](https://hackage.haskell.org/package/tasty-hspec) — for\n  [Hspec](https://hspec.github.io/) tests\n* [tasty-leancheck](https://hackage.haskell.org/package/tasty-leancheck) — for\n  enumerative property-based testing\n  (based on [LeanCheck](https://hackage.haskell.org/package/leancheck))\n* [tasty-program](https://hackage.haskell.org/package/tasty-program) — run\n  external program and test whether it terminates successfully\n* [tasty-wai](https://hackage.haskell.org/package/tasty-wai) —\n  for testing [wai](https://hackage.haskell.org/package/wai) endpoints\n* [tasty-inspection-testing](https://hackage.haskell.org/package/tasty-inspection-testing) —\n  for compile-time testing of code properties\n  (based on [inspection-testing](http://hackage.haskell.org/package/inspection-testing))\n* [tasty-flaky](https://hackage.haskell.org/package/tasty-flaky) — add delay and\n  retry logic to any test that is known to fail intermittently\n\n[tasty-golden]: https://hackage.haskell.org/package/tasty-golden\n\nIt's easy to create custom providers using the API from `Test.Tasty.Providers`.\n\n### Ingredients\n\nIngredients represent different actions that you can perform on your test suite.\nOne obvious ingredient that you want to include is one that runs tests and\nreports the progress and results.\n\nAnother standard ingredient is one that simply prints the names of all tests.\n\nIt is possible to write custom ingredients using the API from `Test.Tasty.Runners`.\n\nSome ingredients that can enhance your test suite are:\n\n* [tasty-ant-xml](https://hackage.haskell.org/package/tasty-ant-xml) adds a\n  possibility to write the test results in a machine-readable XML format, which\n  is understood by various CI systems and IDEs\n* [tasty-rerun](https://hackage.haskell.org/package/tasty-rerun) adds support for\n  minimal test reruns by recording previous test runs and using this information\n  to filter the test tree. For example, you can use this ingredient to only run\n  failed tests, or only run tests that threw an exception.\n* [tasty-html](https://hackage.haskell.org/package/tasty-html) adds the\n  possibility to write the test results as a HTML file\n* [tasty-stats](https://hackage.haskell.org/package/tasty-stats) adds the\n  possibility to collect statistics of the test suite in a CSV file.\n\n### Test discovery\n\n`tasty` by itself forces you to explicitly write out the `TestTree` yourself.\nThe packages listed below allow you to write tests at the top-level, and will\nautomatically collect them into a single `TestTree`.\n\n* [tasty-th](https://hackage.haskell.org/package/tasty-th)\n* [tasty-discover](https://hackage.haskell.org/package/tasty-discover)\n* [tasty-autocollect](https://hackage.haskell.org/package/tasty-autocollect)\n\n### Other packages\n\n* [tasty-hunit-adapter](https://hackage.haskell.org/package/tasty-hunit-adapter)\n  converts existing HUnit test suites into tasty test suites\n* [tasty-expected-failure](https://hackage.haskell.org/package/tasty-expected-failure) provides\ntest markers for when you expect failures or wish to ignore tests.\n* [tasty-bench](https://hackage.haskell.org/package/tasty-bench) covers performance\nregression testing and extends `tasty` to a benchmark framework\nsimilar to `criterion` and `gauge`.\n\n\n## Options\n\nOptions allow one to customize the run-time behavior of the test suite, such\nas:\n\n* mode of operation (run tests, list tests, run tests quietly etc.)\n* which tests are run (see «Patterns» below)\n* parameters of individual providers (like depth of search for SmallCheck)\n\n### Setting options\n\nThere are two main ways to set options:\n\n#### Runtime\n\nWhen using the standard console runner, the options can be passed on the\ncommand line or via environment variables. To see the available options, run\nyour test suite with the `--help` flag. The output will look something like this\n(depending on which ingredients and providers the test suite uses):\n\n```\n% ./test --help\nMmm... tasty test suite\n\nUsage: test [-p|--pattern PATTERN] [-t|--timeout DURATION] [--no-progress]\n            [-l|--list-tests] [-j|--num-threads NUMBER] [-q|--quiet]\n            [--hide-successes] [--color never|always|auto] [--ansi-tricks ARG]\n            [--smallcheck-depth NUMBER] [--smallcheck-max-count NUMBER]\n            [--quickcheck-tests NUMBER] [--quickcheck-replay SEED]\n            [--quickcheck-show-replay] [--quickcheck-max-size NUMBER]\n            [--quickcheck-max-ratio NUMBER] [--quickcheck-verbose]\n            [--quickcheck-shrinks NUMBER]\n\nAvailable options:\n  -h,--help                Show this help text\n  -p,--pattern PATTERN     Select only tests which satisfy a pattern or awk\n                           expression\n  -t,--timeout DURATION    Timeout for individual tests (suffixes: ms,s,m,h;\n                           default: s)\n  --no-progress            Do not show progress\n  -l,--list-tests          Do not run the tests; just print their names\n  -j,--num-threads NUMBER  Number of threads to use for tests execution\n                           (default: # of cores/capabilities)\n  -q,--quiet               Do not produce any output; indicate success only by\n                           the exit code\n  --hide-successes         Do not print tests that passed successfully\n  --min-duration-to-report DURATION\n                           The minimum amount of time a test can take before\n                           tasty prints timing information (suffixes: ms,s,m,h;\n                           default: s)\n  --color never|always|auto\n                           When to use colored output (default: auto)\n  --ansi-tricks ARG        Enable various ANSI terminal tricks. Can be set to\n                           'true' or 'false'. (default: true)\n  --smallcheck-depth NUMBER\n                           Depth to use for smallcheck tests\n  --smallcheck-max-count NUMBER\n                           Maximum smallcheck test count\n  --quickcheck-tests NUMBER\n                           Number of test cases for QuickCheck to generate.\n                           Underscores accepted: e.g. 10_000_000\n  --quickcheck-replay SEED Random seed to use for replaying a previous test run\n                           (use same --quickcheck-max-size)\n  --quickcheck-show-replay Show a replay token for replaying tests\n  --quickcheck-max-size NUMBER\n                           Size of the biggest test cases quickcheck generates\n  --quickcheck-max-ratio NUMBER\n                           Maximum number of discared tests per successful test\n                           before giving up\n  --quickcheck-verbose     Show the generated test cases\n  --quickcheck-shrinks NUMBER\n                           Number of shrinks allowed before QuickCheck will fail\n                           a test\n```\n\nEvery option can be passed via environment. To obtain the environment variable\nname from the option name, replace hyphens `-` with underscores `_`, capitalize\nall letters, and prepend `TASTY_`. For example, the environment equivalent of\n`--smallcheck-depth` is `TASTY_SMALLCHECK_DEPTH`.\n\nNote on boolean options: by convention, boolean (\"on/off\") options are specified\nusing a switch on the command line, for example `--quickcheck-show-replay`\ninstead of `--quickcheck-show-replay=true`. However, when\npassed via the environment, the option value needs to be `True` or `False`\n(case-insensitive), e.g. `TASTY_QUICKCHECK_SHOW_REPLAY=true`.\n\nIf you're using a non-console runner, please refer to its documentation to find\nout how to configure options during the run time.\n\n#### Compile-time\n\nYou can also specify options in the test suite itself, using\n`localOption`. It can be applied not only to the whole test tree, but also to\nindividual tests or subgroups, so that different tests can be run with\ndifferent options.\n\nIt is possible to combine run-time and compile-time options, too, by using\n`adjustOption`. For example, make the overall testing depth configurable\nduring the run time, but increase or decrease it slightly for individual\ntests.\n\nThis method currently doesn't work for ingredient options, such as `--quiet` or\n`--num-threads`. You can set them by setting the corresponding environment\nvariable before calling `defaultMain`:\n\n\u003ca id=\"num_threads_example\"\u003e\n\n```haskell\nimport Test.Tasty\nimport System.Environment\n\nmain = do\n  setEnv \"TASTY_NUM_THREADS\" \"1\"\n  defaultMain _\n```\n\n### Patterns\n\nIt is possible to restrict the set of executed tests using the `-p/--pattern`\noption.\n\nTasty patterns are very powerful, but if you just want to quickly run tests containing `foo`\nsomewhere in their name or in the name of an enclosing test group, you can just\npass `-p foo`. If you need more power, or if that didn't work as expected, read\non.\n\nA pattern is an [awk expression][awk]. When the expression is evaluated, the field `$1`\nis set to the outermost test group name, `$2` is set to the next test group\nname, and so on up to `$NF`, which is set to the test's own name. The field `$0`\nis set to all other fields concatenated using `.` as a separator.\n\n[awk]: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/awk.html#tag_20_06_13_02\n\nAs an example, consider a test inside two test groups:\n\n```\n  testGroup \"One\" [ testGroup \"Two\" [ testCase \"Three\" _ ] ]\n```\n\nWhen a pattern is evaluated for the above test case, the available fields and variables are:\n\n    $0 = \"One.Two.Three\"\n    $1 = \"One\"\n    $2 = \"Two\"\n    $3 = \"Three\"\n    NF = 3\n\nHere are some examples of awk expressions accepted as patterns:\n\n* `$2 == \"Two\"` — select the subgroup `Two`\n* `$2 == \"Two\" \u0026\u0026 $3 == \"Three\"`  — select the test or subgroup named `Three` in the subgroup named `Two`\n* `$2 == \"Two\" || $2 == \"Twenty-two\"` — select two subgroups\n* `$0 !~ /skip/` or `! /skip/` — select tests whose full names (including group names) do not contain the word `skip`\n* `$NF !~ /skip/` — select tests whose own names (but not group names) do not contain the word `skip`\n* `$(NF-1) ~ /QuickCheck/` — select tests whose immediate parent group name\n    contains `QuickCheck`\n\nAs an extension to the awk expression language, if a pattern `pat` contains only\nletters, digits, and characters from the set `._ -` (period, underscore, space, hyphen),\nit is treated like `/pat/` (and therefore matched against `$0`).\nThis is so that we can use `-p foo` as a shortcut for `-p /foo/`.\n\nThe only deviation from awk that you will likely notice is that Tasty\ndoes not implement regular expression matching.\nInstead, `$1 ~ /foo/` means that the string `foo` occurs somewhere in `$1`,\ncase-sensitively. We want to avoid a heavy dependency of `regex-tdfa` or\nsimilar libraries; however, if there is demand, regular expression support could\nbe added under a cabal flag.\n\nThe following operators are supported (in the order of decreasing precedence):\n\n\u003ccenter\u003e\n\u003ctable\u003e\n\u003ctr\u003e\n\u003cth\u003e\n\u003cp\u003e\u003cb\u003eSyntax\u003c/b\u003e\u003c/p\u003e\n\u003c/th\u003e\n\u003cth\u003e\n\u003cp\u003e\u003cb\u003eName\u003c/b\u003e\u003c/p\u003e\n\u003c/th\u003e\n\u003cth\u003e\n\u003cp\u003e\u003cb\u003eType of Result\u003c/b\u003e\u003c/p\u003e\n\u003c/th\u003e\n\u003cth\u003e\n\u003cp\u003e\u003cb\u003eAssociativity\u003c/b\u003e\u003c/p\u003e\n\u003c/th\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\n\u003cp\u003e\u003ccode\u003e(expr)\u003c/code\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eGrouping\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eType of \u003ccode\u003eexpr\u003c/code\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eN/A\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\n\u003cp\u003e\u003ccode\u003e$expr\u003c/code\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eField reference\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eString\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eN/A\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\n\u003cp\u003e\u003ccode\u003e!expr\u003c/code\u003e\u003c/p\u003e\n\u003cp\u003e\u003ccode\u003e-expr\u003c/code\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eLogical not\u003c/p\u003e\n\u003cp\u003eUnary minus\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eNumeric\u003c/p\u003e\n\u003cp\u003eNumeric\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eN/A\u003c/p\u003e\n\u003cp\u003eN/A\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\n\u003cp\u003e\u003ccode\u003eexpr + expr\u003c/code\u003e\u003c/p\u003e\n\u003cp\u003e\u003ccode\u003eexpr - expr\u003c/code\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eAddition\u003c/p\u003e\n\u003cp\u003eSubtraction\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eNumeric\u003c/p\u003e\n\u003cp\u003eNumeric\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eLeft\u003c/p\u003e\n\u003cp\u003eLeft\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\n\u003ctr\u003e\n\u003ctd\u003e\n\u003cp\u003e\u003ccode\u003eexpr expr\u003c/code\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eString concatenation\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eString\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eRight\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\n\u003cp\u003e\u003ccode\u003eexpr \u0026lt; expr\u003c/code\u003e\u003c/p\u003e\n\u003cp\u003e\u003ccode\u003eexpr \u0026lt;= expr\u003c/code\u003e\u003c/p\u003e\n\u003cp\u003e\u003ccode\u003eexpr != expr\u003c/code\u003e\u003c/p\u003e\n\u003cp\u003e\u003ccode\u003eexpr == expr\u003c/code\u003e\u003c/p\u003e\n\u003cp\u003e\u003ccode\u003eexpr \u0026gt; expr\u003c/code\u003e\u003c/p\u003e\n\u003cp\u003e\u003ccode\u003eexpr \u0026gt;= expr\u003c/code\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eLess than\u003c/p\u003e\n\u003cp\u003eLess than or equal to\u003c/p\u003e\n\u003cp\u003eNot equal to\u003c/p\u003e\n\u003cp\u003eEqual to\u003c/p\u003e\n\u003cp\u003eGreater than\u003c/p\u003e\n\u003cp\u003eGreater than or equal to\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eNumeric\u003c/p\u003e\n\u003cp\u003eNumeric\u003c/p\u003e\n\u003cp\u003eNumeric\u003c/p\u003e\n\u003cp\u003eNumeric\u003c/p\u003e\n\u003cp\u003eNumeric\u003c/p\u003e\n\u003cp\u003eNumeric\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eNone\u003c/p\u003e\n\u003cp\u003eNone\u003c/p\u003e\n\u003cp\u003eNone\u003c/p\u003e\n\u003cp\u003eNone\u003c/p\u003e\n\u003cp\u003eNone\u003c/p\u003e\n\u003cp\u003eNone\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\n\u003ctr\u003e\n\u003ctd\u003e\n\u003cp\u003e\u003ccode\u003eexpr ~ pat\u003c/code\u003e\u003c/p\u003e\n\u003cp\u003e\u003ccode\u003eexpr !~ pat\u003c/code\u003e\u003c/p\u003e\n\u003cp\u003e(\u003ccode\u003epat\u003c/code\u003e must be a literal, not an expression, e.g. \u003ccode\u003e/foo/\u003c/code\u003e)\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eSubstring match\u003c/p\u003e\n\u003cp\u003eNo substring match\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eNumeric\u003c/p\u003e\n\u003cp\u003eNumeric\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eNone\u003c/p\u003e\n\u003cp\u003eNone\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\n\u003cp\u003e\u003ccode\u003eexpr \u0026amp;\u0026amp; expr\u003c/code\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eLogical AND\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eNumeric\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eLeft\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\n\u003cp\u003e\u003ccode\u003eexpr || expr\u003c/code\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eLogical OR\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eNumeric\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eLeft\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\n\u003cp\u003e\u003ccode\u003eexpr1 ? expr2 : expr3\u003c/code\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eConditional expression\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eType of selected\u003cbr\u003e\u003ccode\u003eexpr2\u003c/code\u003e or \u003ccode\u003eexpr3\u003c/code\u003e\u003c/p\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n\u003cp\u003eRight\u003c/p\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003c/table\u003e\n\u003c/center\u003e\n\nThe following built-in functions are supported:\n\n```\nsubstr(s, m[, n])\n```\nReturn the at most `n`-character substring of `s` that begins at\nposition `m`, numbering from 1. If `n` is omitted, or if `n` specifies\nmore characters than are left in the string, the length of the substring\nwill be limited by the length of the string `s`.\n\n```\ntolower(s)\n```\n\nConvert the string `s` to lower case.\n\n```\ntoupper(s)\n```\n\nConvert the string `s` to upper case.\n\n```\nmatch(s, pat)\n```\n\nReturn the position, in characters, numbering from 1, in string `s` where the\npattern `pat` occurs, or zero if it does not occur at all.\n`pat` must be a literal, not an expression, e.g. `/foo/`.\n\n```\nlength([s])\n```\n\nReturn the length, in characters, of its argument taken as a string, or of the whole record, `$0`, if there is no argument.\n\n### Running tests in parallel\n\nIn order to run tests in parallel, you have to do the following:\n\n* Compile (or, more precisely, *link*) your test program with the `-threaded`\n  flag;\n* Launch the program with `+RTS -N -RTS`.\n\n### Timeout\n\nTo apply timeout to individual tests, use the `--timeout` (or `-t`) command-line\noption, or set the option in your test suite using the `mkTimeout` function.\n\nTimeouts can be fractional, and can be optionally followed by a suffix `ms`\n(milliseconds), `s` (seconds), `m` (minutes), or `h` (hours). When there's no\nsuffix, seconds are assumed.\n\nExample:\n\n    ./test --timeout=0.5m\n\nsets a 30 seconds timeout for each individual test.\n\n### Options controlling console output\n\nThe following options control behavior of the standard console interface:\n\n\u003cdl\u003e\n\u003cdt\u003e\u003ccode\u003e-q,--quiet\u003c/code\u003e\u003c/dt\u003e\n\u003cdd\u003e\n  Run the tests but don't output anything. The result is indicated only by the\n  exit code, which is 1 if at least one test has failed, and 0 if all tests\n  have passed. Execution stops when the first failure is detected, so not all\n  tests are necessarily run.\n  This may be useful for various batch systems, such as commit hooks.\n\u003c/dd\u003e\n\u003cdt\u003e\u003ccode\u003e--hide-successes\u003c/code\u003e\u003c/dt\u003e\n\u003cdd\u003eReport only the tests that has failed. Especially useful when the\nnumber of tests is large.\u003c/dd\u003e\n\u003cdt\u003e\u003ccode\u003e-l,--list-tests\u003c/code\u003e\u003c/dt\u003e\n\u003cdd\u003eDon't run the tests; only list their names, in the format accepted by\n\u003ccode\u003e--pattern\u003c/code\u003e.\u003c/dd\u003e\n\u003cdt\u003e\u003ccode\u003e--color\u003c/code\u003e\u003c/dt\u003e\n\u003cdd\u003eWhether to produce colorful output. Accepted values: \u003ccode\u003enever\u003c/code\u003e,\n\u003ccode\u003ealways\u003c/code\u003e, \u003ccode\u003eauto\u003c/code\u003e. \u003ccode\u003eauto\u003c/code\u003e means that colors will\nonly be enabled when output goes to a terminal and is the default value.\u003c/dd\u003e\n\u003c/dl\u003e\n\n### Custom options\n\nIt is possible to add custom options, too.\n\nTo do that,\n\n1. Define a datatype to represent the option, and make it an instance of\n   `IsOption`\n2. Register the options with the `includingOptions` ingredient\n3. To query the option value, use `askOption`.\n\nSee the [Custom options in Tasty][custom-options-article] article for some examples.\n\n## Project organization and integration with Cabal\n\nThere may be several ways to organize your project. What follows is not\nTasty's requirements but my recommendations.\n\n### Tests for a library\n\nPlace your test suite sources in a dedicated subdirectory (called `tests`\nhere) instead of putting them among the main library sources.\n\nThe directory structure will be as follows:\n\n    my-project/\n      my-project.cabal\n      src/\n        ...\n      tests/\n        test.hs\n        Mod1.hs\n        Mod2.hs\n        ...\n\n`test.hs` is where your `main` function is defined. The tests may be\ncontained in `test.hs` or spread across multiple modules (`Mod1.hs`, `Mod2.hs`,\n...) which are then imported by `test.hs`.\n\nAdd the following section to the cabal file (`my-project.cabal`):\n\n    test-suite test\n      default-language:\n        Haskell2010\n      type:\n        exitcode-stdio-1.0\n      hs-source-dirs:\n        tests\n      main-is:\n        test.hs\n      build-depends:\n          base \u003e= 4 \u0026\u0026 \u003c 5\n        , tasty \u003e= 0.7 -- insert the current version here\n        , my-project   -- depend on the library we're testing\n        , ...\n\n### Tests for a program\n\nAll the above applies, except you can't depend on the library if there's no\nlibrary. You have two options:\n\n* Re-organize the project into a library and a program, so that both the\n  program and the test suite depend on this new library. The library can be\n  declared in the same cabal file.\n* Add your program sources directory to the `Hs-source-dirs`. Note that this\n  will lead to double compilation (once for the program and once for the test\n  suite).\n\n## Dependencies\n\nTasty executes tests in parallel to make them finish faster.\nIf this parallelism is not desirable, you can declare *dependencies* between\ntests, so that one test will not start until certain other tests finish.\n\nDependencies are declared using the `after` or `sequentialTestGroup` combinator:\n\n* `after AllFinish \"pattern\" my_tests` will execute the test tree `my_tests` only after all\n    tests that match the pattern finish.\n* `after AllSucceed \"pattern\" my_tests` will execute the test tree `my_tests` only after all\n    tests that match the pattern finish **and** only if they all succeed. If at\n    least one dependency fails, then `my_tests` will be skipped.\n* `sequentialTestGroup groupName dependencyType [tree1, tree2, ..]` will execute all tests\n   in `tree1` first, after which it will execute all tests in `tree2`, and so forth. Like\n   `after`, `dependencyType` can either be set to `AllFinish` or `AllSucceed`.\n\nThe relevant types are:\n\n``` haskell\nafter\n  :: DependencyType -- ^ whether to run the tests even if some of the dependencies fail\n  -\u003e String         -- ^ the pattern\n  -\u003e TestTree       -- ^ the subtree that depends on other tests\n  -\u003e TestTree       -- ^ the subtree annotated with dependency information\n\nsequentialTestGroup\n  :: TestName       -- ^ name of the group\n  -\u003e DependencyType -- ^ whether to run the tests even if some of the dependencies fail\n  -\u003e [TestTree]     -- ^ trees to execute sequentially\n  -\u003e TestTree\n\ndata DependencyType = AllSucceed | AllFinish\n```\n\nThe pattern follows the same AWK-like syntax and semantics as described in\n[Patterns](#patterns). There is also a variant named `after_` that accepts the\nAST of the pattern instead of a textual representation.\n\nLet's consider some typical examples. (A note about terminology: here\nby \"resource\" I mean anything stateful and external to the test: it could be a file,\na database record, or even a value stored in an `IORef` that's shared among\ntests. The resource may or may not be managed by `withResource`.)\n\n1. Two tests, Test A and Test B, access the same shared resource and cannot be\n   run concurrently. To achieve this, make Test A a dependency of Test B:\n\n   ``` haskell\n   testGroup \"Tests accessing the same resource\"\n     [ testCase \"Test A\" $ ...\n     , after AllFinish \"Test A\" $\n         testCase \"Test B\" $ ...\n     ]\n   ```\n\n1. Test A creates a resource and Test B uses that resource. Like above, we make\n   Test A a dependency of Test B, except now we don't want to run Test B if Test\n   A failed because the resource may not have been set up properly. So we use\n   `AllSucceed` instead of `AllFinish`\n\n   ``` haskell\n   testGroup \"Tests creating and using a resource\"\n     [ testCase \"Test A\" $ ...\n     , after AllSucceed \"Test A\" $\n         testCase \"Test B\" $ ...\n     ]\n   ```\n\nHere are some caveats to keep in mind when using patterns to specify dependencies in Tasty:\n\n1. If Test B depends on Test A, remember that either of them may be filtered out\n   using the `--pattern` option. Collecting the dependency info happens *after*\n   filtering. Therefore, if Test A is filtered out, Test B will run\n   unconditionally, and if Test B is filtered out, it simply won't run.\n1. Tasty does not currently check whether the pattern in a dependency matches\n   anything at all, so make sure your patterns are correct and do not contain\n   typos. Fortunately, misspecified dependencies usually lead to test failures\n   and so can be detected that way.\n1. Dependencies shouldn't form a cycle, otherwise Tasty with fail with the\n   message \"Test dependencies have cycles.\" A common cause of this is a test\n   matching its own dependency pattern.\n1. Using dependencies may introduce quadratic complexity. Specifically,\n   resolving dependencies is *O(number_of_tests × number_of_dependencies)*,\n   since each pattern has to be matched against each test name. As a guideline,\n   if you have up to 1000 tests, the overhead will be negligible, but if you\n   have thousands of tests or more, then you probably shouldn't have more than a\n   few dependencies.\n\n   Additionally, it is recommended that the dependencies follow the\n   natural order of tests, i.e. that the later tests in the test tree depend on\n   the earlier ones and not vice versa. If the execution order mandated by the\n   dependencies is sufficiently different from the natural order of tests in the\n   test tree, searching for the next test to execute may also have an\n   overhead quadratic in the number of tests.\n\nUse `sequentialTestGroup` to mitigate these problems.\n\n\n## FAQ\n\n1.  **Q**: When my tests write to stdout/stderr, the output is garbled. Why is that and\n    what do I do?\n\n    **A**: It is not recommended that you print anything to the console when using the\n    console test reporter (which is the default one).\n    See [#103](https://github.com/UnkindPartition/tasty/issues/103) for the\n    discussion.\n\n    Some ideas on how to work around this:\n\n    * Use [testCaseSteps](https://hackage.haskell.org/package/tasty-hunit/docs/Test-Tasty-HUnit.html#v:testCaseSteps) (for tasty-hunit only).\n    * Use a test reporter that does not print to the console (like tasty-ant-xml).\n    * Write your output to files instead.\n\n2.  **Q**: Why doesn't the `--hide-successes` option work properly? The test headings\n    show up and/or the output appears garbled.\n\n    **A**: This can happen sometimes when the terminal is narrower than the\n    output. A workaround is to disable ANSI tricks: pass `--ansi-tricks=false`\n    on the command line or set `TASTY_ANSI_TRICKS=false` in the environment.\n\n    See [issue #152](https://github.com/UnkindPartition/tasty/issues/152).\n\n3. **Q**: Patterns with slashes do not work on Windows. How can I fix it?\n\n   **A**: If you are running Git for Windows terminal, it has a habit of\n   converting slashes to backslashes. Set `MSYS_NO_PATHCONV=1` when running the\n   Git for Windows terminal and `MSYS2_ARG_CONV_EXCL=*` when running a MinGW\n   bash directly to prevent this behaviour, or follow other suggestions from\n   [Known\n   Issues](https://github.com/git-for-windows/build-extra/blob/main/ReleaseNotes.md#known-issues).\n\n## Migration from `test-framework`\n\n`tasty` architecture is quite similar to `test-framework`, so a few mechanical changes are usually enough to migrate:\n\n* Replace packages in `build-depends`:\n  * `test-framework` -\u003e `tasty`,\n  * `test-framework-hunit` -\u003e `tasty-hunit`,\n  * `test-framework-quickcheck2` -\u003e `tasty-quickcheck`.\n* Replace module imports:\n  * `Test.Framework` -\u003e `Test.Tasty`,\n  * `Test.Framework.Providers.HUnit` -\u003e `Test.Tasty.HUnit`,\n  * `Test.Framework.Providers.QuickCheck2` -\u003e `Test.Tasty.QuickCheck`.\n* Replace in type signatures:\n  * `Test` -\u003e `TestTree`.\n* Replace `defaultMain tests` with `defaultMain (testGroup \"All\" tests)`.\n\n## Press\n\nBlog posts and other publications related to tasty. If you wrote or just found\nsomething not mentioned here, send a pull request!\n\n* [Holy Haskell Project Starter](https://yannesposito.com/Scratch/en/blog/Holy-Haskell-Starter/)\n* [First time testing, also with FP Complete](https://levischuck.com/posts/2013-11-13-first-testing-and-fpcomplete.html)\n  (tasty has been added to stackage since then)\n* [24 Days of Hackage: tasty](https://ocharles.org.uk/blog/posts/2013-12-03-24-days-of-hackage-tasty.html)\n* [Resources in Tasty](https://ro-che.info/articles/2013-12-10-tasty-resources)\n* [Custom options in Tasty][custom-options-article]\n* [Resources in Tasty (update)](https://ro-che.info/articles/2013-12-29-tasty-resources-2)\n* [Announcing tasty-rerun](https://ocharles.org.uk/blog/posts/2014-01-20-announcing-tasty-rerun.html)\n* [Code testing in Haskell revisited (with Tasty)](https://jstolarek.github.io/posts/2014-01-26-code-testing-in-haskell-revisited-with-tasty.html)\n* [New patterns in tasty][awk-patterns-article]\n* [Screencast: Dynamic Test Suites in Haskell using Hspec and Tasty](https://www.youtube.com/watch?v=PGsDvgmZF7A)\n* [Automatically generated directories for tasty tests][tasty-directories]\n\n[custom-options-article]: https://ro-che.info/articles/2013-12-20-tasty-custom-options.html\n[awk-patterns-article]: https://ro-che.info/articles/2018-01-08-tasty-new-patterns\n[tasty-directories]: https://nmattia.com/posts/2018-04-30-tasty-test-names.html\n\n## GHC version support policy\n\nWe only support the GHC/base versions [from the last 5 years](https://wiki.haskell.org/Base_package#Versions).\n\nMaintainers\n-----------\n\n[Roman Cheplyaka](https://github.com/UnkindPartition) is the primary maintainer.\n\n[Oliver Charles](https://github.com/ocharles) is the backup maintainer. Please\nget in touch with him if the primary maintainer cannot be reached.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Funkindpartition%2Ftasty","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Funkindpartition%2Ftasty","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Funkindpartition%2Ftasty/lists"}