Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/kputnam/forall
Ruby generative property test library (ala QuickCheck)
https://github.com/kputnam/forall
generative-testing property-based-testing rspec testing
Last synced: 25 days ago
JSON representation
Ruby generative property test library (ala QuickCheck)
- Host: GitHub
- URL: https://github.com/kputnam/forall
- Owner: kputnam
- Created: 2021-05-03T06:15:24.000Z (over 3 years ago)
- Default Branch: development
- Last Pushed: 2022-03-25T00:38:06.000Z (over 2 years ago)
- Last Synced: 2024-04-25T04:42:31.298Z (7 months ago)
- Topics: generative-testing, property-based-testing, rspec, testing
- Language: Ruby
- Homepage:
- Size: 94.7 KB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
Awesome Lists containing this project
README
# Forall [](https://github.com/kputnam/forall/actions/workflows/build.yml).
Property-based testing for Ruby (adapted from Jacob Stanley's [Hedgehog](https://github.com/hedgehogqa/haskell-hedgehog) library for Haskell, and an older project I made named [Propr](https://github.com/kputnam/propr)).
## Introduction
The usual approach to testing software is to describe a set of test inputs and
their expected corresponding outputs. The program is run with these inputs, and
the actual outputs are compared to what's expected to ensure the program
behaves correctly. This methodology is simple to implement and automate, but
has some problems like:
* Writing test cases is tedious and repetitive.
* Only edge cases that occur to the author are tested.
* It can be difficult to see which parts of the test input are mere prerequisites rather than essential.
* Getting 100% code coverage with trivial tests doesn't offer much assurance.
Property-based testing is an alternative and complementary approach in which
the binary relations between attributes of inputs and desired output are
expressed as functions, rather than enumerating particular inputs and outputs.
The properties specify things like, "assuming the program is correct, when its
run with any valid inputs, the inputs and the program output are related by
`f(input, output)`".
## Properties
The following example demonstrates testing a property with a specific input,
then generalizing the test for any input.
```ruby
describe Array do
include Forall::RSpecHelpers
describe "#+(other)" do
# Traditional unit test
it "sums lengths" do
xs = [100, 200, 300]
ys = [400, 500]
expect((xs + ys).length).to eq(xs.length + ys.length)
end
# Property-based test
it "sums lengths" do
ints = random.array(random.integer(0..999))
forall(random.sequence(ints, ints)) do |xs, ys|
(xs + ys).length == xs.length + ys.length
end
end
# property("sums lengths"){|xs, ys| (xs + ys).length == xs.length + ys.length }
# .check([100, 200, 300], [500, 200])
# .check{ sequence [Array.random { Integer.random }, Array.random { Integer.random }] }
end
end
```
The following example is similar, but contains an error in the specification
```ruby
describe Array do
include Propr::RSpec
describe "#|(other)" do
# Traditional unit test
it "sums lengths" do
xs = [100, 200, 300]
ys = [400, 500]
# This passes
expect((xs | ys).length).to eq(xs.length + ys.length)
end
# Property-based test
it "sums lengths" do
ints = random.array(random.integer(0..999))
forall(random.sequence(ints, ints)) do |xs, ys|
(xs | ys).length == xs.length + ys.length
end
end
# property("sums lengths"){|xs, ys| (xs | ys).length == xs.length + ys.length }
# .check([100, 200, 300], [400, 500])
# .check{ sequence [Array.random{Integer.random(min:0, max:50)}]*2 }
end
end
```
When this specification is executed, the following error is reported.
$ rake spec
..F
Failures:
1) Array#| sums lengths
Failure/Error: raise Falsifiable.new(counterex, m.shrink(counterex), passed, skipped)
Propr::Falsifiable:
input: [], [0, 0]
after: 49 passed, 0 skipped
# ./lib/propr/rspec.rb:29:in `block in check'
Finished in 0.22829 seconds
3 examples, 1 failure
You may have figured out the error is that `|` removes duplicate elements
from the result. We might not have caught the mistake by writing individual
test cases. The output indicates Forall generated 49 sets of input before
finding one that failed.