Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/samuell/gccontent-benchmark
Benchmarking different languages for a simple bioinformatics task (Counting the GC fraction of DNA in a FASTA file)
https://github.com/samuell/gccontent-benchmark
benchmarking bioinformatics programming-languages
Last synced: 2 months ago
JSON representation
Benchmarking different languages for a simple bioinformatics task (Counting the GC fraction of DNA in a FASTA file)
- Host: GitHub
- URL: https://github.com/samuell/gccontent-benchmark
- Owner: samuell
- License: mit
- Created: 2017-07-13T12:16:15.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2023-01-28T13:27:47.000Z (almost 2 years ago)
- Last Synced: 2024-10-10T12:48:22.612Z (3 months ago)
- Topics: benchmarking, bioinformatics, programming-languages
- Language: Rust
- Homepage:
- Size: 877 KB
- Stars: 57
- Watchers: 2
- Forks: 18
- Open Issues: 8
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Comparing string processing performance of programming languages
... using a simple bioinformatics task: Computing the GC fraction of DNA. It is based on the [GC content problem at Rosalind](http://rosalind.info/problems/gc/).
## Usage
```bash
make
cat report.md
```
If you have [pandoc](http://pandoc.org/) installed, you can also create a HTML report:
```bash
make html-report
report.html
```
## More info
This is a continuation of a previous benchmarking project, covered in [this blog post](http://saml.rilspace.com/moar-languagez-gc-content-in-python-d-fpc-c-and-c).
The idea is to compare the string processing performance of different programming languages
by implementing a very small a very simple algorithm and task: Read a [specific file](http://ftp.ensembl.org/pub/release-67/fasta/homo_sapiens/dna/Homo_sapiens.GRCh37.67.dna_rm.chromosome.Y.fa.gz)
containing DNA sequence in the [FASTA format](https://en.wikipedia.org/wiki/FASTA_format),
and compute the GC content in this file.
Two requirements apply:
1. The file must be read line by line (since DNA files are in reality ofter
bigger than RAM, and this also helps make the implementations remotely
comparable)
2. For each line, the program has to check if it starts with a `>` character,
which if so means it is a header row and should be skipped.
The FASTA file can contain DNA letters (A,C,G,T) or unknowns (N), or new-lines
(Unix style `\n` ones).
This is it. Please have a look in the Makefile, and the various implementations
in the code directories, or send a pull request with your own implementation
(if the language already exists, increase the number one step, so for a new Go
implementation, you would create a `golang.001` folder, optionally with some
tag appended to it, like: `golang.001.table-optimized`, etc).
These are some results (Execution times in seconds, smaller is better) from
running some of the tests in the Makefile, on a Dell Inspiron laptop with an
Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz, with Xubuntu 18.04 Bionic LTS 64bit
as operating system.
(Below the tables are some more details about BIOS settings etc).
| Language and implementation | Execution time (s) | Compiler or interpreter version |
|--------------------------------------------------------|--------------------|-----------------------------------------------------------------------------------------------------------------------|
| [rust.003.vectorized](rust.003.vectorized/src/main.rs) | 0.442 | rustc 1.52.0-nightly (152f66092 2021-02-17) |
| [rust.004.simd](rust.004.simd/src/main.rs) | 0.445 | rustc 1.52.0-nightly (152f66092 2021-02-17) |
| [rust.002.bitshift](rust.002.bitshift/src/main.rs) | 0.695 | rustc 1.52.0-nightly (152f66092 2021-02-17) |
| [rust.001](rust.001/src/main.rs) | 0.891 | rustc 1.52.0-nightly (152f66092 2021-02-17) |
| [c.001](c.001/gc.c) | 0.970 | gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 |
| [cpp.001](cpp.001/gc.) | 1.025 | g++ (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 |
| [d](d/gc.d) | 1.215 | LDC - the LLVM D compiler (1.22.0): based on DMD v2.092.1 |
| [c](c/gc.c) | 1.226 | gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 |
| [go.001.unroll](go.001.unroll/gc.go) | 1.616 | go version go1.15 linux/amd64 |
| [nim.003.zerocopy](nim.003.zerocopy/gc.nim) | 1.660 | Nim Compiler Version 1.2.6 [Linux: amd64] |
| [nim.002](nim.002/gc.nim) | 1.703 | Nim Compiler Version 1.2.6 [Linux: amd64] |
| [julia](julia/gc.jl) | 1.926 | julia version 1.5.3 |
| [go](go/gc.go) | 1.937 | go version go1.15 linux/amd64 |
| [c.003.ril](c.003.ril/gc.c) | 1.955 | gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 |
| [nim.001](nim.001/nim.001/gc.nim) | 2.281 | Nim Compiler Version 1.2.6 [Linux: amd64] |
| [crystal.002.peek](crystal.002.peek/gc.cr) | 2.369 | Crystal 0.36.1 [c3a3c1823] (2021-02-02) LLVM: 10.0.0 |
| [pypy](pypy/gc.py) | 2.677 | Python 2.7.13 (5.10.0+dfsg-3build2, Feb 06 2018, 18:37:50) [PyPy 5.10.0 with GCC 7.3.0] |
| [cpp](cpp/gc.cpp) | 2.832 | g++ (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 |
| [nim](nim/gc.nim) | 2.976 | Nim Compiler Version 1.2.6 [Linux: amd64] |
| [rust](rust/src/main.rs) | 3.195 | rustc 1.52.0-nightly (152f66092 2021-02-17) |
| [crystal](crystal/gc.cr) | 4.054 | Crystal 0.36.1 [c3a3c1823] (2021-02-02) LLVM: 10.0.0 |
| [ada](ada/gc.adb) | 4.235 | GNAT Community 2020 (20200818-93) |
| [java](java/java/gc.java) | 4.279 | openjdk version "11.0.10" 2021-01-19 OpenJDK Runtime Environment GraalVM CE 21.0.0.2 (build 11.0.10+8-jvmci-21.0-b06) |
| [crystal.001.csp](crystal.001.csp/gc.cr) | 4.283 | Crystal 0.36.1 [c3a3c1823] (2021-02-02) LLVM: 10.0.0 |
| [java](java/gc.java) | 4.284 | openjdk version "11.0.10" 2021-01-19 OpenJDK Runtime Environment GraalVM CE 21.0.0.2 (build 11.0.10+8-jvmci-21.0-b06) |
| [cython](cython/gc.pyx) | 6.016 | Cython version 0.26.1 |
| [fpc](fpc/gc.pas) | 6.578 | Free Pascal Compiler version 3.0.4+dfsg-18ubuntu2 [2018/08/29] for x86_64 |
| [node](node/gc.js) | 6.836 | v15.9.0 |
| [perl](perl/gc.pl) | 7.323 | This is perl 5, version 26, subversion 1 (v5.26.1) built for x86_64-linux-gnu-thread-multi |
| [python](python/gc.py) | 8.855 | Python 3.7.0 |
| [graalvm](graalvm/gc.java) | 11.734 | GraalVM Version 21.0.0.2 (Java Version 11.0.10+8-jvmci-21.0-b06) |
## Results with relaxed constraints on reading line-by-line
The below contributed versions departs slightly from reading line-by-line (by
some definition of that requirement, which is clearly very hard to define):
| Language | Execution time (s) | Compiler versions |
|----------------------------------|-------------------:|---------------------------------------------|
| [rust.007.rawio](rust.007.rawio) | 0.221 | rustc 1.52.0-nightly (152f66092 2021-02-17) |
| [rust.005.rawio](rust.005.rawio) | 0.318 | rustc 1.52.0-nightly (152f66092 2021-02-17) |
| [C.002.rawio](c.002.rawio/gc.c) | 0.524 | gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0 |
| [rust.006.rawio](rust.006.rawio) | 0.539 | rustc 1.52.0-nightly (152f66092 2021-02-17) |
## More details about settings used when benchmarking
The following CPU options were turned off in BIOS, to try to avoid fluctuating
CPU clock frequencies:
- Performance > Intel SpeedStep
- Performance > C-States Control
- Performance > Intel TurboBoost
- Power Management > Intel Speed Shift Technology
Benchmarking was done with other GUI apps, networking and bluetooth turned off.
## Incomplete list of contributions before merge to GitHub
For contributors after establishing the GitHub repo, see [this page here on GitHub](https://github.com/samuell/gccontent-benchmark/graphs/contributors).
Below is additionally an incomplete list of people who contributed to the code
examples while the benchmark was only [hosted on my old blog](https://github.com/samuell/gccontent-benchmark/graphs/contributors):
- Daniel Spångberg (working at UPPMAX HPC center at the time) contributed
numerous, extremely fast implementations in C, including the one above (c),
which is constrained by the requirement to process the file line by line.
- [Roger Peppe](https://github.com/rogpeppe)
([twitter](https://twitter.com/rogpeppe)) contributed the fastest Go
implementation, including pointers in combination with a table lookup.
- [Mario Ray Mahardhika (aka leledumbo)](https://github.com/leledumbo)
contributed the fastest FreePascal implementation, which is the one above
(fpc.000).
- [Harald Achitz](https://www.linkedin.com/in/harald-achitz-860657139/)
provided the C++ implementation used above (cpp.000).
- (Who is missing here?)