https://github.com/hecrj/fast
A command-line tool to test the optimizations performed to a program.
https://github.com/hecrj/fast
Last synced: 2 months ago
JSON representation
A command-line tool to test the optimizations performed to a program.
- Host: GitHub
- URL: https://github.com/hecrj/fast
- Owner: hecrj
- License: mit
- Created: 2014-03-06T13:24:12.000Z (about 11 years ago)
- Default Branch: master
- Last Pushed: 2014-06-18T16:12:07.000Z (almost 11 years ago)
- Last Synced: 2025-01-31T05:19:48.660Z (3 months ago)
- Language: Python
- Size: 309 KB
- Stars: 3
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.mdown
- License: COPYING.txt
Awesome Lists containing this project
README
fast
============
A command-line tool to test the optimizations performed to a program.
With `fast` you can perform benchmarks to different executables in a really easy way!
Installation
------------
```
$ pip install fast-cli
```
`fast` is still in a very early stage of development. This means that lots of changes are likely to occur, and your old benchmarks could need to be changed in order to work with the new versions.
Usage
------------
`fast` tries to load a Python module `benchmarks.py` located in the current working directory. In this module you need to define the different benchmarks that you want to perform to your executables!
Here is how you can define benchmarks in `benchmarks.py`:
```python
from fast.benchmarks import benchmark, BenchmarkBase
@benchmark
class DocUsageBenchmark(BenchmarkBase):
# The original executable from which to compare the optimizations
target = 'my_executable'
# A list of optimized executables
# default: self.target but with `_fast` appended before its extension
candidates = ('my_executable_v2',)
# Number of instances that you want for your benchmark samples
# default: 20
instances = 20
# Number of executions per instance
# The average time is taken as the total time of the instance
# default: 1
executions = 2
# Main label for the generated inputs
# default: "Input"
xlabel = "Number of as"
def args(self, instnace):
"""
Arguments that should be provided to the executables for the given instance
"""
return []
def input(self, instance):
"""
Input that should be provided to the executables for the given instance
"""
return "a" * instance
def label(self, instance):
"""
Label for the given instance
"""
return instance
```
The command `benchmark` will run the benchmarks in `benchmarks.py` that are decorated with `@benchmark`, also it accepts the names of the benchmarks as arguments. The name of one benchmark is its class name but replacing CamelCase with underscores. For example, you could be able to run the previous benchmark doing: `fast benchmark doc_usage`
To make the executables, `fast` invokes `make name_of_the_executable` before performing each benchmark. Therefore, you need to have a `Makefile` with the correct rules to generate your executables.
Example
------------
Let's see how we can use `fast` in a real-world example.
Suppose that we want to optimize a program named `swap` that reads the input in pairs of bytes and writes them in the output in reverse order. If the number of bytes of the input file is odd, then the last byte it's left in the same position.
We could write `benchmarks.py` as follows:
```python
from fast.benchmarks import benchmark, BenchmarkBase
@benchmark
class DataSizeBenchmark(BenchmarkBase):
target = 'swap'
instances = 20
executions = 3
xlabel = "Data size (Mbytes)"
def input(self, instance):
return "abc" * ((instance * 1024 * 1024) / 3)
def label(self, instance):
return instance
```
We define a benchmark that is going to increase the input size from 1 Mbyte to 20 Mbytes targeting the `swap` executable. We have not defined any candidate, so by default `fast` is going to try to make a `swap_fast` executable to perform the benchmark.
Now, we perform the benchmark:
```
hector@Ubi:~/example/swap$ fast benchmark
Making executables...
make: `swap' is up to date.
gcc swap_fast.c -o swap_fast
Benchmarking data_size...
Checking differences between swap and swap_fast...
Checking input data_size_1.in with args []...
Checking input data_size_2.in with args []...
Checking input data_size_3.in with args []...
[...]
Checking input data_size_20.in with args []...
Generating stats for swap...
Data size (Mbytes) Time (s)
1 0.2670
2 0.5348
3 0.7982
[...]
20 5.4169
Generating stats for swap_fast...
Data size (Mbytes) Time (s)
1 0.0023
2 0.0024
3 0.0046
[...]
20 0.0192
Generating plots for data_size...
Generating plot: Data size (Mbytes) vs Time (s)
Generating plot: Speedup
Done.
```
`fast` creates `.stats` files for each executable of the benchmark and two plots in `.pdf`: one that shows the Input vs Time of the target and the candidates, and another that shows the Input vs Speedup.
### DataSizeBenchmark
Our optimization worked. We got huge speedups for all the inputs!