Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/composewell/concurrency-benchmarks
Benchmarks comparing concurrency overhead of streamly and async
https://github.com/composewell/concurrency-benchmarks
benchmarks concurrency concurrency-benchmarks haskell
Last synced: about 2 months ago
JSON representation
Benchmarks comparing concurrency overhead of streamly and async
- Host: GitHub
- URL: https://github.com/composewell/concurrency-benchmarks
- Owner: composewell
- License: other
- Created: 2018-09-06T14:48:15.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2021-05-26T07:43:30.000Z (over 3 years ago)
- Last Synced: 2024-10-02T18:38:00.686Z (3 months ago)
- Topics: benchmarks, concurrency, concurrency-benchmarks, haskell
- Language: Haskell
- Size: 107 KB
- Stars: 7
- Watchers: 7
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: Changelog.md
- License: LICENSE
Awesome Lists containing this project
README
# concurrency-benchmarks
[](https://hackage.haskell.org/package/concurrency-benchmarks)
[](https://travis-ci.org/composewell/concurrency-benchmarks)
[](https://ci.appveyor.com/project/harendra-kumar/concurrency-benchmarks)
Benchmarks to compare the concurrency overhead of
concurrent [streamly](https://github.com/composewell/streamly) streams and the
[async](https://hackage.haskell.org/package/async) package. If you are familiar
with `async`, see [this quick
guide](https://github.com/composewell/streamly/blob/master/docs/Async.md) for
equivalent features of streamly. For more detailed introduction to streamly see
[this tutorial](https://hackage.haskell.org/package/streamly/docs/Streamly-Tutorial.html).
## How to run
Run the `run.sh` script to run the benchmarks and create the charts. You can
use `cabal new-bench` or `stack bench` to run the benchmarks. To generate
charts, run the benchmarks with `--csv-raw=results.csv` option and then run
`makecharts results.csv`. Charts are generated in the `charts` directory.
IMPORTANT NOTE: THE `maxrss` FIGURES REPORTED BY GAUGE WILL NOT BE CORRECT
UNLESS YOU RUN ONE BENCHMARK AT A TIME. This is because `maxrss` is tracked per
process.
## Methodology
A total of 10,000 tasks are run for each concurrency mechanism being compared.
Two independent experiments are performed:
1. In the first experiment, each task is just a noop i.e. it takes almost 0 time
to execute.
2. In the second experiment, each task introduces a 5 second delay
The first case shows streamly's smart scheduling to automatically run the tasks
in less number of threads than the actual number of tasks. When the tasks do
not block and have a very low latency, streamly may run multiple tasks per
thread. Therefore streamly is much faster on this benchmark.
In the second case a 5 second delay is introduced to make sure that streamly
uses one thread per task which is similar to what `async` does and therefore a
fair comparison. For the `async` package, `mapConcurrently` is used which can
be compared with streamly's `ahead` style stream.
For streamly this is the code that is benchmarked, by default streamly has a
limit on the buffer size and the number of threads, we set those limits to `-1`
which means there is no limit:
```haskell
let work = (\i -> threadDelay 5000000 >> return i)
in Stream.drain
$ Stream.fromAhead
$ Stream.maxBuffer (-1)
$ Stream.maxThreads (-1)
$ Stream.fromFoldableM $ map work [1..10000]
```
For `async` this is the code that is benchmarked:
```haskell
let work = (\i -> threadDelay 5000000 >> return i)
mapConcurrently work [1..10000]
```
## Results
These charts compare
[streamly-0.7.2](https://github.com/composewell/streamly/tree/03cf686ca7e3fe1093b064c9763de8b684cce0f4)
and `async-2.2.2`.
When compiling, `-threaded -with-rtsopts "-N"` GHC options were used to enable
the use of multiple processor cores in parallel.
For streamly, results for both `async` and `ahead` style streams are shown.
### Zero delay case
#### Peak Memory Consumed
#### Time Taken
### 5 second delay case
#### Peak Memory Consumed
#### Time Taken
Note, this time shows the overhead only and not the full time taken by the
benchmark. For example the actual time taken by the `async` benchmark is
`5.135` seconds, but since 5 second in this is the delay introduced by each
parallel task, we compute the overhead of concurrency by deducting the 5
seconds from the actual time taken, so the overhead is `135 ms` in case of
`async`.
## Feedback
Feedback is welcome. Please raise an issue, send a PR or send an email to the
author.