{"id":15240453,"url":"https://github.com/kputnam/forall","last_synced_at":"2025-07-21T11:32:00.799Z","repository":{"id":56847451,"uuid":"363834568","full_name":"kputnam/forall","owner":"kputnam","description":"Ruby generative property test library (ala QuickCheck)","archived":false,"fork":false,"pushed_at":"2022-03-25T00:38:06.000Z","size":97,"stargazers_count":2,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"development","last_synced_at":"2025-06-15T01:21:46.516Z","etag":null,"topics":["generative-testing","property-based-testing","rspec","testing"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kputnam.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-05-03T06:15:24.000Z","updated_at":"2023-04-21T10:30:46.000Z","dependencies_parsed_at":"2022-09-12T11:20:26.491Z","dependency_job_id":null,"html_url":"https://github.com/kputnam/forall","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/kputnam/forall","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kputnam%2Fforall","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kputnam%2Fforall/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kputnam%2Fforall/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kputnam%2Fforall/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kputnam","download_url":"https://codeload.github.com/kputnam/forall/tar.gz/refs/heads/development","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kputnam%2Fforall/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265419642,"owners_count":23761848,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["generative-testing","property-based-testing","rspec","testing"],"created_at":"2024-09-29T11:05:00.735Z","updated_at":"2025-07-21T11:32:00.780Z","avatar_url":"https://github.com/kputnam.png","language":"Ruby","readme":"# Forall [![Build Status](https://github.com/kputnam/forall/actions/workflows/build.yml/badge.svg)](https://github.com/kputnam/forall/actions/workflows/build.yml).\n\n\nProperty-based testing for Ruby (adapted from Jacob Stanley's [Hedgehog](https://github.com/hedgehogqa/haskell-hedgehog) library for Haskell, and an older project I made named [Propr](https://github.com/kputnam/propr)).\n\n## Introduction\n\nThe usual approach to testing software is to describe a set of test inputs and\ntheir expected corresponding outputs. The program is run with these inputs, and\nthe actual outputs are compared to what's expected to ensure the program\nbehaves correctly. This methodology is simple to implement and automate, but\nhas some problems like:\n\n* Writing test cases is tedious and repetitive.\n* Only edge cases that occur to the author are tested.\n* It can be difficult to see which parts of the test input are mere prerequisites rather than essential.\n* Getting 100% code coverage with trivial tests doesn't offer much assurance.\n\nProperty-based testing is an alternative and complementary approach in which\nthe binary relations between attributes of inputs and desired output are\nexpressed as functions, rather than enumerating particular inputs and outputs.\nThe properties specify things like, \"assuming the program is correct, when its\nrun with any valid inputs, the inputs and the program output are related by\n`f(input, output)`\".\n\n## Properties\n\nThe following example demonstrates testing a property with a specific input,\nthen generalizing the test for any input.\n\n```ruby\ndescribe Array do\n  include Forall::RSpecHelpers\n\n  describe \"#+(other)\" do\n    # Traditional unit test\n    it \"sums lengths\" do\n      xs = [100, 200, 300]\n      ys = [400, 500]\n      expect((xs + ys).length).to eq(xs.length + ys.length)\n    end\n\n    # Property-based test\n    it \"sums lengths\" do\n      ints = random.array(random.integer(0..999))\n\n      forall(random.sequence(ints, ints)) do |xs, ys|\n        (xs + ys).length == xs.length + ys.length\n      end\n    end\n    # property(\"sums lengths\"){|xs, ys| (xs + ys).length == xs.length + ys.length }\n    #   .check([100, 200, 300], [500, 200])\n    #   .check{ sequence [Array.random { Integer.random }, Array.random { Integer.random }] }\n  end\nend\n```\n\nThe following example is similar, but contains an error in the specification\n\n```ruby\ndescribe Array do\n  include Propr::RSpec\n\n  describe \"#|(other)\" do\n    # Traditional unit test\n    it \"sums lengths\" do\n      xs = [100, 200, 300]\n      ys = [400, 500]\n\n      # This passes\n      expect((xs | ys).length).to eq(xs.length + ys.length)\n    end\n\n    # Property-based test\n    it \"sums lengths\" do\n      ints = random.array(random.integer(0..999))\n\n      forall(random.sequence(ints, ints)) do |xs, ys|\n        (xs | ys).length == xs.length + ys.length\n      end\n    end\n\n    # property(\"sums lengths\"){|xs, ys| (xs | ys).length == xs.length + ys.length }\n    #   .check([100, 200, 300], [400, 500])\n    #   .check{ sequence [Array.random{Integer.random(min:0, max:50)}]*2 }\n  end\nend\n```\n\nWhen this specification is executed, the following error is reported.\n\n    $ rake spec\n    ..F\n\n    Failures:\n\n      1) Array#| sums lengths\n         Failure/Error: raise Falsifiable.new(counterex, m.shrink(counterex), passed, skipped)\n         Propr::Falsifiable:\n           input: [], [0, 0]\n           after: 49 passed, 0 skipped\n         # ./lib/propr/rspec.rb:29:in `block in check'\n\n    Finished in 0.22829 seconds\n    3 examples, 1 failure\n\nYou may have figured out the error is that `|` removes duplicate elements\nfrom the result. We might not have caught the mistake by writing individual\ntest cases. The output indicates Forall generated 49 sets of input before\nfinding one that failed.\n\n\u003c!--\n\nNow that a failing test case has been identified, you might write another\n`check` with those specific inputs to prevent regressions.\n\nYou could also print the initial random seed like this and when a test fails,\nexplicitly set the random seed to regenerate the same inputs for the entire\ntest suite:\n\n    $ cat spec/spec_helper.rb\n    RSpec.configure do |config|\n      srand.tap{|seed| puts \"Random seed is #{seed}\"; srand seed }\n    end\n\n    $ rake spec\n    Random seed is 146211424375622429406889408197139382303\n    ..F\n\n    Failures:\n\n      1) Array#| sums lengths\n         Failure/Error: raise Falsifiable.new(counterex, m.shrink(counterex), passed, skipped)\n         Propr::Falsifiable:\n           input:    [25, 24], [24, 27]\n           shrunken: [], [0, 0]\n           after: 49 passed, 0 skipped\n\n    Finished in 0.22829 seconds\n    3 examples, 1 failure\n\nNow change spec\\_helper.rb to explicitly set the random seed:\n\n    $ cat spec/spec_helper.rb\n    RSpec.configure do |config|\n      srand 146211424375622429406889408197139382303\n      srand.tap{|seed| puts \"Random seed is #{seed}\"; srand seed }\n    end\n\n    $ rake spec\n    Random seed is 146211424375622429406889408197139382303\n\nThe remaining output should be identical every time you run the suite, so\nlong as specs are in the same order each time.\n\n### Just Plain Functions\n\nProperties are basically just functions, they should return `true` or `false`.\n\n    p = Propr::Property.new(\"name\", lambda{|a,b| a + b == b + a })\n\nYou can invoke a property using `#check`. Like lambdas and procs, you can also\ninvoke them using `#call` or `#[]`.\n\n    p.check(3, 4)     #=\u003e true\n    p.check(\"x\", \"y\") #=\u003e false\n\nBut you can also invoke them by yielding a function that generates random inputs.\n\n    m = Propr::Random\n    p.check { m.eval(m.sequence [Integer.random, Float.random]) }  #=\u003e true\n    p.check { m.eval(m.sequence [String.random , String.random]) } #=\u003e false\n\nWhen invoked with a block, `check` will run `p` with 100 random inputs by\ndefault, but you can also pass an argument to `check` indicating how many\nexamples `p` should be tested against.\n\n## Using Propr + Test Frameworks\n\nMixing in a module magically defines the `property` singleton method, so\nyou can use it to generate test cases.\n\n```ruby\ndescribe \"foo\" do\n  include Propr::RSpec\n\n  # This defines three test cases, one per each `check`\n  property(\"length\"){|a| a.length \u003e= 0 }.\n    check(\"abc\").\n    check(\"xyz\").\n    check{ String.random }\nend\n```\n\nNote your property should still return `true` or `false`. You should *not* use\n`#should` or `#assert`, because the test generator will generate the assertion\nfor you. This also reduces visual clutter.\n\nAlternatively, to use Propr with all specification, you can add this to your\n`spec_helper.rb`\n\n```ruby\nRSpec.configure do |config|\n  include Propr::RSpec\nend\n```\n\n### Property DSL\n\nThe code block inside `property { ... }` has an extended scope that defines\na few helpful methods:\n\n* __guard__: Skip this iteration unless all the given conditions are met. This\n  can be used, for instance, to define a property only on even integers.\n  `property{|x| guard(x.even?); x \u0026 1 == 0 }`\n\n* __error?__: True if the code block throws an exception of the given type.\n  `property{|x| error? { x / 0 }}`\n\n* __m__: Short alias for `Propr::Random`, used to generate random data as described\n  below.\n  `property{|x| m.eval(m.sequence([m.unit 0] * x)).length == x }`\n\n### Check DSL\n\nThe code block inside `check { ... }` should return a generator value. The code\nblock's scope is extended with a few combinators to compose generators.\n\n* __unit__: Create a generator that returns the given value. For instance, to yield\n  `3` as an argument to the property,\n  `check { unit(3) }`\n\n* __bind__: Chain the value yielded by one generator into another. For instance, to\n  yield two integers as arguments to a property,\n  `check { bind(Integer.random){|a| bind(Integer.random){|b| unit([a,b]) }}}`\n\n* __guard__: Short-circuit the chain if the given condition is false. The entire chain\n  will be re-run until the guard passes. For instance, to generate two distinct numbers,\n  `check { bind(Integer.random){|a| bind(Integer.random){|b| guard(a != b){ unit([a,b]) }}}}`\n\n* __join__: Remove one level of generator nesting. If you have a generator `x` that\n  *yields* a number generator, then `join x` is a number generator. For instance, to yield\n  either a number or a string,\n  `check { join([Integer.random, String.random].random) }`\n\n* __sequence__: Convert a list of generator values to a list generator. For instance, to\n  yield three integers to a property,\n  `check { sequence [Integer.random]*3 }`\n\n## Generating Random Values\n\nPropr defines a `random` method that returns a generator for most standard\nRuby types. You can run the generator using the `Propr::Random.eval` method.\n\n    \u003e\u003e m = Propr::Random\n    =\u003e ...\n\n    \u003e\u003e m.eval(Boolean.random)\n    =\u003e false\n\n### Boolean\n\n    \u003e\u003e m.eval Boolean.random\n    =\u003e true\n\n### Date\n\n    \u003e\u003e m.eval(Date.random(min: Date.today - 10, max: Date.today + 10)).to_s\n    =\u003e \"2012-03-01\"\n\nOptions\n\n* `min:` minimum value, defaults to 0001-01-01\n* `max:` maximum value, defaults to 9999-12-31\n* `center:` defaults to the midpoint between min and max\n\n### Time\n\n    \u003e\u003e m.eval Time.random(min: Time.now, max: Time.now + 3600)\n    =\u003e 2012-02-20 13:47:57 -0600\n\nOptions\n\n* `min:` minimum value, defaults to 1000-01-01 00:00:00 UTC\n* `max:` maximum value, defaults to 9999-12-31 12:59:59 UTC\n* `center:` defaults to the midpoint between min and max\n\n### String\n\n    \u003e\u003e m.eval String.random(min: 5, max: 10, charset: :lower)\n    =\u003e \"rqyhw\"\n\nOptions\n\n* `min:` minimum size, defaults to 0\n* `max:` maximum size, defaults to 10\n* `center:` defaults to the midpoint between min and max\n* `charset:` regular expression character class, defaults to `/[[:print]]/`\n\n### Numbers\n\n#### Integer.random\n\n    \u003e\u003e m.eval Integer.random(min: -500, max: 500)\n    =\u003e -382\n\nOptions\n\n* `min:` minimum value, defaults to Integer::MIN\n* `max:` maximum value, defaults to Integer::MAX\n* `center:` defaults to the midpoint between min and max.\n\n#### Float.random\n\n    \u003e\u003e m.eval Float.random(min: -500, max: 500)\n    =\u003e 48.252030464134364\n\nOptions\n\n* `min:` minimum value, defaults to -Float::MAX\n* `max:` maximum value, defaults to Float::MAX\n* `center:` defaults to the midpoint between min and max.\n\n#### Rational.random\n\n    \u003e\u003e m.eval m.bind(m.sequence [Integer.random]*2){|a,b| unit Rational(a,b) }\n    =\u003e (300421843/443649464)\n\nNot implemented, as there isn't a nice way to ensure a `min` works. Instead,\ngenerate two numeric values and combine them:\n\n#### BigDecimal.random\n\n    \u003e\u003e m.eval(BigDecimal.random(min: 10, max: 20)).to_s(\"F\")\n    =\u003e \"14.934854011762374703280016489856414847259220844969789892\"\n\nOptions\n\n* `min:` minimum value, defaults to -Float::MAX\n* `max:` maximum value, defaults to Float::MAX\n* `center:` defaults to the midpoint between min and max\n\n#### Bignum.random\n\n    \u003e\u003e m.eval Integer.random(min: Integer::MAX, max: Integer::MAX * 2)\n    =\u003e 2015151263\n\nThere's no constructor specifically for Bignum. You can use `Integer.random`\nand specify `min: Integer::MAX + 1` and some even larger `max` value. Ruby\nwill automatically handle Integer overflow by coercing to Bignum.\n\n#### Complex.random\n\n    \u003e\u003e m.eval(m.bind(m.sequence [Float.random(min:-10, max:10)]*2){|a,b| m.unit Complex(a,b) })\n    =\u003e (9.806161068637833+7.523520738439842i)\n\nNot implemented, as there's no simple way to implement min and max, nor the types\nof the components. Instead, generate two numeric values and combine them:\n\n### Collections\n\nThe class method `random` returns a generator to construct a collection of\nelements, while the `#random` instance method returns a generator which returns\nan element from the collection.\n\n#### Array.random\n\nExpects a block parameter that yields a generator for elements.\n\n    \u003e\u003e m.eval Array.random(min:4, max:4) { String.random(min:4, max:4) }\n    =\u003e [\"2n #\", \"UZ1d\", \"0vF,\", \"cV_{\"]\n\nOptions\n\n* `min:` minimum size, defaults to 0\n* `max:` maximum size, defaults to 10\n* `center:` defaults to the midpoint between min and max\n\n#### Hash.random\n\nExpects a block parameter that yields generator of [key, value] pairs.\n\n    \u003e\u003e m.eval Hash.random(min:2, max:4) { m.sequence [Integer.random, m.unit(nil)] }\n    =\u003e {564854752=\u003enil, -1065292239=\u003enil, 830081146=\u003enil}\n\nOptions\n\n* `min:` minimum size, defaults to 0\n* `max:` maximum size, defaults to 10\n* `center:` defaults to the midpoint between min and max\n\n#### Hash.random_vals\n\nExpects a hash whose keys are ordinary values, and whose values are\ngenerators.\n\n    \u003e\u003e m.eval Hash.random_vals(a: String.random, b: Integer.random)\n    =\u003e {:a=\u003e\"Fi?p`g\", :b=\u003e4551738453396095365}\n\nDoesn't accept any options.\n\n#### Set.random\n\nExpects a block parameter that yields a generator for elements.\n\n    \u003e\u003e m.eval Set.random(min:4, max:4) { String.random(min:4, max:4) }\n    =\u003e #\u003cSet: {\"2n #\", \"UZ1d\", \"0vF,\", \"cV_{\"}\u003e\n\nOptions\n\n* `min:` minimum size, defaults to 0\n* `max:` maximum size, defaults to 10\n* `center:` defaults to the midpoint between min and max\n\n#### Range.random\n\nExpects __either__ a block parameter __or__ one or both of min and max.\n\n    \u003e\u003e m.eval Range.random(min: 0, max: 100)\n    =\u003e 81..58\n\n    \u003e\u003e m.eval Range.random { Integer.random(min: 0, max: 100) }\n    =\u003e 9..80\n\nOptions\n\n* `min:` minimum element\n* `max:` maximum element\n* `inclusive?:` defaults to true, meaning Range includes max element\n\n#### Elements from a collection\n\nThe `#random` instance method is defined on the above types. It takes no parameters.\n\n    \u003e\u003e m.eval([1,2,3,4,5].random)\n    =\u003e 4\n\n    \u003e\u003e m.eval({a: 1, b: 2, c: 3, d: 4}.random)\n    =\u003e [:b, 2]\n\n    \u003e\u003e m.eval((0..100).random)\n    =\u003e 12\n\n    \u003e\u003e m.eval(Set.new([1,2,3,4]).random)\n    =\u003e 4\n\n## Search Space Attenuation\n\nThe `m.eval` method has a second parameter that serves to exponentially reduce\nthe domain for generators, specified with `min:` and `max:` parameters. The scale\nvalue may range from `0` to `1`, where `1` causes no change.\n\nWhen scale is `0`, the domain is reduced to a single value, which is specified by\nthe `center:` parameter. Usually this defaults to the midpoint between `min:` and\n`max:`. Any value between `min:` and `max:` can be given for `center:`, in addition\nto the three symbolic values, `:min`, `:mid`, and `:max`.\n\nScale values beteween `0` and `1` adjust the domain exponentially, so a domain with\n10,000 elements when `scale = 1` will have 1,000 elements when `scale = 0.5` and\nonly 100 when `scale = 0.25`.\n\nWith `scale = 0`, the domain contains at most `10000^0 = 1` elements:\n\n    \u003e\u003e m.eval Integer.random(min: 0, max: 10000, center: :min), 0\n    == m.eval Integer.random(min: 0, max: 0)\n\n    \u003e\u003e m.eval Integer.random(min: 0, max: 10000, center: :mid), 0\n    == m.eval Integer.random(min: 5000, max: 5000)\n\n    \u003e\u003e m.eval Integer.random(min: 0, max: 10000, center: :max), 0\n    == m.eval Integer.random(min: 10000, max: 10000)\n\nWith `scale = 0.25`, the domain contains at most `10000^0.25 = 10` elements:\n\n    \u003e\u003e m.eval Integer.random(min: 0, max: 10000, center: :min), 0.25\n    == m.eval Integer.random(min: 0, max: 9)\n\n    \u003e\u003e m.eval Integer.random(min: 0, max: 10000, center: :mid), 0.25\n    == m.eval Integer.random(min: 4996, max: 5004)\n\n    \u003e\u003e m.eval Integer.random(min: 0, max: 10000, center: :max), 0.25\n    == m.eval Integer.random(min: 9991, max: 10000)\n\nWith `scale = 0.50`, the domain contains at most `10000^0.5 = 100` elements:\n\n    \u003e\u003e m.eval Integer.random(min: 0, max: 10000, center: :min), 0.5\n    == m.eval Integer.random(min: 0, max: 99)\n\n    \u003e\u003e m.eval Integer.random(min: 0, max: 10000, center: :mid), 0.5\n    == m.eval Integer.random(min: 4951, max: 5048)\n\n    \u003e\u003e m.eval Integer.random(min: 0, max: 10000, center: :max), 0.5\n    == m.eval Integer.random(min: 9901, max: 10000)\n\nWith `scale = 0.75`, the domain contains at most `10000^0.75 = 1000` elements:\n\n    \u003e\u003e m.eval Integer.random(min: 0, max: 10000, center: :min), 0.75\n    == m.eval Integer.random(min: 0, max: 998)\n\n    \u003e\u003e m.eval Integer.random(min: 0, max: 10000, center: :mid), 0.75\n    == m.eval Integer.random(min: 4507, max: 5499)\n\n    \u003e\u003e m.eval Integer.random(min: 0, max: 10000, center: :max), 0.75\n    == m.eval Integer.random(min: 9002, max: 10000)\n\n### Deepening of the Search Space\n\nBy default, the test framework adapters increase the scale linearly (causing\nan exponential increase of the domain size) each time the property is tested.\n\nThat is, when running 100 iterations, scale values will be 0.00, 0.01, 0.02,\n0.03, 0.04, etc. This is intended to test the simplest counterexamples first,\nand increase the complexity of generated inputs exponentially.\n\n### Simplification of Counterexamples\n\nOnce a random input has been classified as a counterexample, Propr will\nsearch for a simpler counterexample. This is done by iteratively calling\n`#shrink` on each successively smaller counterexample.\n\n    $ cat shrink.example\n    require \"rspec\"\n    require \"propr\"\n\n    RSpec.configure do |config|\n      include Propr::RSpec\n\n      srand 146211424375622429406889408197139382303\n      srand.tap{|seed| puts \"Random seed is #{seed}\"; srand seed }\n    end\n\n    describe Float do\n      property(\"assoc\"){|x,y,z| (x + y) + z == x + (y + z) }\n        .check(-382863.98514407175, 224121.21177705095, 276118.77134001954)\n    end\n\n    $ rspec shrink.example\n    Random seed is 146211424375622429406889408197139382303\n    F\n\n    Failures:\n\n      1) Float assoc\n         Propr::Falsifiable:\n           input:    -382863.98514407175, 224121.21177705095, 276118.77134001954\n           shrunken: -0.007740960460133677, 0.011895728563551701, 3.9765678826328424e-05\n           after: 0 passed, 0 skipped\n         # ./lib/propr/rspec.rb:36:in `block in check'\n\n    Finished in 10.52 seconds\n    1 example, 1 failure\n\nNotice the output shows a \"simpler\" counterexample than the inputs we explicitly\ntested. This becomes useful when testing with more complex data like trees, where\nit can be difficult to understand which aspect of the counterexample is relevant.\n\n## Installation\n\nThere are a few things I'd like to fix before publishing this as a gem. Until\nthen, you can install directly from the git repo using Bundler, with this in\nyour Gemfile:\n\n    gem \"propr\", git: \"git@github.com:kputnam/propr.git\", branch: \"rewrite\"\n\nYou'll probably want to specify the current tag, also (eg, `..., tag: \"v0.2.0\"`)\n\n\n## More Reading\n\n* [Presentation at KC Ruby Meetup Group](https://github.com/kputnam/presentations/raw/master/Property-Based-Testing.pdf)\n\n## Related Projects\n\n* [Rantly](https://github.com/hayeah/rantly)\n* [PropER](https://github.com/manopapad/proper)\n* [QuviQ](http://www.quviq.com/documents/QuviqFlyer.pdf)\n* [QuickCheck](http://www.haskell.org/haskellwiki/Introduction_to_QuickCheck)\n\n--\u003e\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkputnam%2Fforall","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkputnam%2Fforall","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkputnam%2Fforall/lists"}