Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/hedgehogqa/fsharp-hedgehog-xunit

Hedgehog with convenience attributes for xUnit.net
https://github.com/hedgehogqa/fsharp-hedgehog-xunit

hedgehog xunit

Last synced: 6 months ago
JSON representation

Hedgehog with convenience attributes for xUnit.net

Awesome Lists containing this project

README 

        
# Hedgehog.Xunit



[![][nuget-shield]][nuget] [![][workflow-shield]][workflow] [![Coverage Status](https://coveralls.io/repos/github/dharmaturtle/fsharp-hedgehog-xunit/badge.svg?branch=main)](https://coveralls.io/github/dharmaturtle/fsharp-hedgehog-xunit?branch=main)



[Hedgehog][hedgehog] with convenience attributes for [xUnit.net][xunit].






## Features



- Test method arguments generated with a custom [`GenX.auto`](https://github.com/hedgehogqa/fsharp-hedgehog-experimental/#auto-generation)...

- ...or with a custom [Generator](https://hedgehogqa.github.io/fsharp-hedgehog/index.html#Generators).

- `Property.check` called for each test.



## Getting Started in C#



This readme is for F#. Go [here for C# documentation.](/readmeCSharp.md)



## Getting Started in F#



Install the _Hedgehog.Xunit_ [package][nuget] from Visual Studio's Package Manager Console:



```powershell

PM> Install-Package Hedgehog.Xunit

```



Suppose you have a test that uses [Hedgehog.Experimental](https://github.com/hedgehogqa/fsharp-hedgehog-experimental) and looks similar to the following:



```f#

open Xunit

open Hedgehog

[]

let ``Reversing a list twice yields the original list`` () =

  property {

    let! xs = GenX.auto

    return List.rev (List.rev xs) = xs

  } |> Property.check

```



Then using Hedgehog.Xunit, you can simplify the above test to



```f#

open Hedgehog.Xunit

[]

let ``Reversing a list twice yields the original list, with Hedgehog.Xunit`` (xs: int list) =

  List.rev (List.rev xs) = xs

```



## Documentation



`Hedgehog.Xunit` provides the following attributes:

* [`[]`](#-property)

Extends xUnit's `Fact` to call Hedgehog's `property`.

* [`[]`](#-properties)

Configures all [`Property`](#-property) tagged tests in a module or class.

* [`GenAttribute`](#-genattribute)

Extend this attribute to set a parameter's generator.

* [`[]`](#-recheck)

Run a test with a specific `Size` and `Seed`.



### 🔖 `[]`



Methods with `[]` have their arguments generated by [`GenX.auto`](https://github.com/hedgehogqa/fsharp-hedgehog-experimental/#auto-generation).



```f#

type ``class with a test`` (output: Xunit.Abstractions.ITestOutputHelper) =

  []

  let ``Can generate an int`` (i: int) =

    output.WriteLine $"Test input: {i}"

	

=== Output ===

Test input: 0

Test input: -1

Test input: 1

...

Test input: 522317518

Test input: 404306656

Test input: 1550509078

```



`Property.check` is called.



```f#

[]

let ``This test fails`` (b: bool) =

  b



=== Output ===

Hedgehog.FailedException: *** Failed! Falsifiable (after 2 tests):

(false)

```



If the test returns `Async<_>` or `Task<_>`, then `Async.RunSynchronously` is called, _which blocks the thread._ This may have significant performance implications as tests run 100 times by default.



```f#

[]

let ``Async with exception shrinks`` (i: int) = async {

  do! Async.Sleep 100

  if i > 10 then

    failwith "whoops!"

  }



=== Output ===

Hedgehog.FailedException: *** Failed! Falsifiable (after 12 tests):

(11)

```

A test returning a `Result` in an `Error` state will be treated as a failure.



```f#

[]

let ``Result with Error shrinks`` (i: int) =

  if i > 10 then

    Error ()

  else

    Ok ()



=== Output ===

Hedgehog.FailedException: *** Failed! Falsifiable (after 13 tests and 2 shrinks):

[11]

```



Tests returning `Async>` or `Task>` are run synchronously and are expected to be in the `Ok` state.



Tests returning a `Property` or `Property` will have `Property.check` automatically called:



```f#

[]

let ``returning a failing property with an external number gen fails and shrinks`` i = property {

  let! _50 = Gen.constant 50

  return i <= _50

}



=== Output ===

System.Exception: *** Failed! Falsifiable (after 23 tests and 5 shrinks):

[51]

50

```



#### ⚙️ `[]` Configuration



`[]`'s constructor may take several arguments:

* [`AutoGenConfig` and `AutoGenConfigArgs`](#-autogenconfig-and-autogenconfigargs): Set an `AutoGenConfig` to use when generating arguments.

* [`Tests`](#-tests): Specifies the number of tests to be run.

* [`Shrinks`](#-shrinks): Specifies the maximal number of shrinks that may run.

* [`Size`](#-size): Sets the `Size` to a value for all runs.



The `Property` attribute extends `Xunit.FactAttribute`, so it may also take `DisplayName`, `Skip`, and `Timeout`.



#### 🧰 `AutoGenConfig` and `AutoGenConfigArgs`



[`GenX.defaults`](https://github.com/hedgehogqa/fsharp-hedgehog-experimental/blob/6d8719aaa6568c51478a39e2ad76afbcd43d5b8e/src/Hedgehog.Experimental/Gen.fs#L356) is the [`AutoGenConfig`](https://github.com/hedgehogqa/fsharp-hedgehog-experimental/blob/6d8719aaa6568c51478a39e2ad76afbcd43d5b8e/src/Hedgehog.Experimental/Gen.fs#L17) used by default.



Here's how to add your own generators:

1. Create a class with a single static property or method that returns an instance of `AutoGenConfig`.

2. Provide the type of this class as an argument to `[]`. (This works around the constraint that [`Attribute` parameters must be a constant.](https://stackoverflow.com/a/33007272))



```f#

type AutoGenConfigContainer =

  static member __ =

    GenX.defaults |> AutoGenConfig.addGenerator (Gen.constant 13)



[)>]

let ``This test passes`` (i: int) =

  i = 13

```



If the method takes arguments, you must provide them using `AutoGenConfigArgs`.



```f#

type ConfigWithArgs =

  static member __ a b =

    GenX.defaults

    |> AutoGenConfig.addGenerator (Gen.constant a)

    |> AutoGenConfig.addGenerator (Gen.constant b)



[, AutoGenConfigArgs = [|"foo"; 13|])>]

let ``This also passes`` s i =

  s = "foo" && i = 13

```



#### 🧰 `Tests`



Specifies the number of tests to be run, though more or less may occur due to shrinking or early failure.



```f#

[)>]

let ``This runs 3 times`` () =

  ()

```



#### 🧰 `Shrinks`



Specifies the maximal number of shrinks that may run.



```f#

[)>]

let ``No shrinks occur`` i =

  if i > 50 then failwith "oops"

```



#### 🧰 `Size`



Sets the `Size` to a value for all runs.



```f#

[]

let ``"i" mostly ranges between -1 and 1`` i =

  printfn "%i" i

```



### 🔖 `[]`



This optional attribute can decorate modules or classes. It sets default arguments for [`AutoGenConfig`, `AutoGenConfigArgs`](#-autogenconfig-and-autogenconfigargs), [`Tests`](#-tests), [`Shrinks`](#-shrinks), and [`Size`](#-size). These will be overridden by any explicit arguments on `[]`.



```f#

type Int13   = static member __ = GenX.defaults |> AutoGenConfig.addGenerator (Gen.constant 13)

type Int2718 = static member __ = GenX.defaults |> AutoGenConfig.addGenerator (Gen.constant 2718)



[, 1)>]

module ``Module with  tests`` =



  []

  let ``this passes and runs once`` (i: int) =

    i = 13



  [, 2)>]

  let ``this passes and runs twice`` (i: int) =

    i = 2718

```



### 🔖 `GenAttribute`



To assign a generator to a test's parameter, extend `GenAttribute` and override `Generator`:



```f#

type Int5() =

  inherit GenAttribute()

  override _.Generator = Gen.constant 5



[]

let ``can set parameter as 5`` ([] i) =

  Assert.StrictEqual(5, i)

```



Here's a more complex example of `GenAttribute` that takes a parameter and overrides `Property`'s `AutoGenConfig`:

  

```f#

type AutoGenConfigContainer =

  static member __ =

    GenX.defaults |> AutoGenConfig.addGenerator (Gen.constant 1)



type ConstInt(i: int)=

  inherit GenAttribute()

  override _.Generator = Gen.constant i

    

[)>]

let ``GenAttribute overrides Property's AutoGenConfig`` (one, [] two) =

  Assert.StrictEqual(1, one)

  Assert.StrictEqual(2, two)

```



### 🔖 `[]`



This optional method attribute invokes `Property.recheck` with the given `Size` and `Seed`. It must be used with `Property`.



```f#

[]

[]

let ``this passes`` i =

  i = 12345

```



## Tips



Use named arguments to select the desired constructor overload.



```f#

[, AutoGenConfig = typeof)>]

module __ =

  [, Tests = 2718, Skip = "just because")>]

  let ``Not sure why you'd do this, but okay`` () =

    ()

```



Consider extending `PropertyAttribute` or `PropertiesAttribute` to hardcode commonly used arguments.



```f#

type Int5() =

  inherit GenAttribute()

  override _.Generator = Gen.constant 5

  

type Int13 = static member __ = GenX.defaults |> AutoGenConfig.addGenerator (Gen.constant 13)



type PropertyInt13Attribute() = inherit PropertyAttribute(typeof)

module __ =

  []

  let ``this passes`` (thirteen: int) ([] five: int) =

    thirteen = 13 && five = 5



type PropertiesInt13Attribute() = inherit PropertiesAttribute(typeof)

[]

module ___ =

  []

  let ``this also passes`` (thirteen: int) ([] five: int) =

    thirteen = 13 && five = 5

```



  Known issue with generating a single tuple.



`GenX.autoWith` can generate a tuple.



```f#

[]

let ``This passes`` () =

  Property.check <| property {

      let! a, b =

        GenX.defaults

        |> AutoGenConfig.addGenerator (Gen.constant (1, 2))

        |> GenX.autoWith

      Assert.Equal(1, a)

      Assert.Equal(2, b)

  }

```



However, blindly converting the above test to `Hedgehog.Xunit` will fail.



```f#

type CustomTupleGen = static member __ = GenX.defaults |> AutoGenConfig.addGenerator (Gen.constant (1, 2))

[)>]

let ``This fails`` ((a,b) : int*int) =

  Assert.Equal(1, a)

  Assert.Equal(2, b)

```



This is because F# functions whose only parameter is a tuple will generate IL that un-tuples that parameter, yielding a function whose arity is the number of elements in the tuple. More concretely, this F#



```f#

let ``This fails`` ((a,b) : int*int) = ()

```



yields this IL (in debug mode)



```IL

.method public static 

    void 'This fails' (

        valuetype [System.Private.CoreLib]System.Int32 _arg1_0,

        valuetype [System.Private.CoreLib]System.Int32 _arg1_1

    ) cil managed 

{

    .maxstack 8

    IL_0000: ret

}

```



Due to this behavior `Hedgehog.Xunit` can't know that the original parameter was a tuple. It will therefore not use the registered tuple generator. A workaround is to pass a second (possibly unused) parameter.



```f#

type CustomTupleGen = static member __ = GenX.defaults |> AutoGenConfig.addGenerator (Gen.constant (1, 2))

[)>]

let ``This passes`` (((a,b) : int*int), _: bool) =

  Assert.Equal(1, a)

  Assert.Equal(2, b)

```



The updated F#



```f#

let ``This passes`` (((a,b) : int*int), _: bool) = ()

```



yields this IL



```IL

.method public static 

    void 'This passes' (

        class [System.Private.CoreLib]System.Tuple`2 _arg1,

        valuetype [System.Private.CoreLib]System.Boolean _arg2

    ) cil managed 

{

    .maxstack 8

    IL_0000: ret

}

```



[Source of IL.](https://sharplab.io/#v2:DYLgZgzgNALiCWwoBMQGoA+BbA9sgrsAKYAEAsgJ5l6FECwAUI8TCQHY4BOWAhsAGL42AYxjwcbEjxIAjEgF4pJNLMbMirAAaaAKgAt4EEmB6II2kgApLPKDICUJECXhsYAKlcxHiy/bUMLCTa+oYkAA48EBBE5ppW1rYOTi5unm72UCQA+s4yODjAPlb2QA)



[hedgehog]: https://github.com/hedgehogqa/fsharp-hedgehog

[xunit]: https://xunit.net/



[nuget]: https://www.nuget.org/packages/Hedgehog.Xunit/

[nuget-shield]: https://img.shields.io/nuget/v/Hedgehog.Xunit.svg

[workflow]: https://github.com/dharmaturtle/fsharp-hedgehog-xunit/actions?query=workflow%3AMain

[workflow-shield]: https://github.com/dharmaturtle/fsharp-hedgehog-xunit/workflows/Main/badge.svg