Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/patrickt/effects-benchmarks
Benchmarking Haskell extensible-effect libraries.
https://github.com/patrickt/effects-benchmarks
Last synced: 2 months ago
JSON representation
Benchmarking Haskell extensible-effect libraries.
- Host: GitHub
- URL: https://github.com/patrickt/effects-benchmarks
- Owner: patrickt
- License: bsd-3-clause
- Created: 2018-10-12T22:13:54.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2020-08-02T18:08:43.000Z (over 4 years ago)
- Last Synced: 2024-10-03T12:39:38.538Z (3 months ago)
- Language: Haskell
- Size: 29.3 KB
- Stars: 34
- Watchers: 5
- Forks: 8
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# effects-benchmarks
This is a set of memory and time benchmarks for the various Haskell effects libraries, including `fused-effects`, `polysemy`, `freer-simple`, and `extensible-effects`. Benchmarks for equivalent `mtl` and handwritten computations are included for comparison purposes. It uses Cabal's new Backpack feature to save a great deal of boilerplate code.
# tl;dr
`fused-effects` and `mtl` achieve roughly commensurate performance. `polysemy` is within one to three orders of magnitude from `fused-effects` and `mtl`. `freer-simple` and `extensible-effects` are considerably slower.
# Full results
On a 15-inch 2017 MacBook Pro (3.1GHz Intel i7, 16GB RAM), here are results for space benchmarks:
```
Countdown
Case Allocated
fused-effects 40
mtl 40
polysemy 1,304
freer-simple 1,096
extensible-effects 1,304
shallow 232
Countdown + exc
Case Allocated GCs
fused-effects 336
mtl 336
polysemy 792
freer-simple 864
extensible-effects 984
shallow 320
```
(The fact that `fused-effects` and `mtl` have the same allocations is not an error; `fused-effects` was designed to have almost identical performance characteristics to `mtl`.)
Time (`State`-based countdown):
```
benchmarked Countdown/Put/fused-effects
time 5.692 μs (5.570 μs .. 5.835 μs)
benchmarked Countdown/Put/mtl
time 5.500 μs (5.386 μs .. 5.719 μs)
benchmarked Countdown/Put/polysemy
time 1.903 ms (1.845 ms .. 1.954 ms)
benchmarked Countdown/Put/freer-simple
time 1.324 ms (1.282 ms .. 1.361 ms)
benchmarked Countdown/Put/extensible-effects
time 1.383 ms (1.343 ms .. 1.422 ms)
benchmarked Countdown/Put/shallow
time 275.1 μs (270.1 μs .. 281.6 μs)
```
Time (`State` + `Error` countdown):
```
benchmarked Countdown/Put+Exc/fused-effects
time 9.829 μs (9.645 μs .. 10.04 μs)
benchmarked Countdown/Put+Exc/mtl
time 9.968 μs (9.779 μs .. 10.15 μs)
benchmarked Countdown/Put+Exc/polysemy
time 418.1 μs (408.9 μs .. 427.1 μs)
benchmarked Countdown/Put+Exc/freer-simple
time 1.157 ms (1.120 ms .. 1.199 ms)
benchmarked Countdown/Put+Exc/extensible-effects
time 1.232 ms (1.212 ms .. 1.262 ms)
benchmarked Countdown/Put+Exc/shallow
time 55.12 μs (54.34 μs .. 56.29 μs)
```
Time (custom HTTP DSL):
```
benchmarked HTTP/fused-effects
time 8.311 μs (8.060 μs .. 8.592 μs)
benchmarked HTTP/polysemy
time 14.99 ms (14.26 ms .. 15.80 ms)
benchmarked HTTP/extensible-effects
time 1.612 ms (1.578 ms .. 1.654 ms)
benchmarked HTTP/Deep embedding
time 2.027 ms (1.876 ms .. 2.184 ms)
benchmarked HTTP/Shallow embedding
time 15.94 μs (15.49 μs .. 16.46 μs)
benchmarked HTTP/freer-simple
time 1.478 ms (1.443 ms .. 1.515 ms)
```
# Future work
* More benchmarks: `Writer`, logging, `grammar`, `pipes`.
* I am going to make this emit pretty graphs, or at least graphable .csv files, someday.