Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/tairov/lamatune
LLama implementations benchmarking framework
https://github.com/tairov/lamatune
benchmark llama2
Last synced: 7 days ago
JSON representation
LLama implementations benchmarking framework
- Host: GitHub
- URL: https://github.com/tairov/lamatune
- Owner: tairov
- License: other
- Created: 2023-10-15T23:33:05.000Z (about 1 year ago)
- Default Branch: master
- Last Pushed: 2023-11-06T20:25:45.000Z (12 months ago)
- Last Synced: 2024-10-18T22:06:30.191Z (16 days ago)
- Topics: benchmark, llama2
- Language: Shell
- Homepage: https://engiware.com/benchmark/llama2-ports-extensive-benchmarks-mac-m1-max.html
- Size: 9.85 MB
- Stars: 12
- Watchers: 2
- Forks: 2
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
- awesome-mojo - LLama implementations benchmarking framework
README
### Llama2.c ports/forks benchmarking framwork
Have you ever wanted to benchmark a baby Llama2 models in 12 programming languages? No? Well, now you can!
### How to benchmark 2 branches/PRs of llama2.mojo?
1. Clone this repo
2. Clone & build [hypertune](https://github.com/tairov/hypertune) & make it avaialble in your PATH
4. Edit `config.sh`
```shell
# set workdir that should contain directory `models` with `*.bin` models
export WORKDIR=~/opensource
# optionally set the number of benchmark rounds
export BENCHMARK_ROUNDS=30
# also it's possible to adjust the number of threads
export THREADS=4
```
5. Edit `bench/compare2.sh` change the VERSIONS definition
```shell
VERSIONS=(
"v1,https://github.com/tairov/llama2.mojo.git,master"
"v2,[email protected]:andresnowak/llama2.mojo.git,master"
)
```
5. Run `compare2.sh` with bash
**only bash supported for now, since `compare2.sh` uses exported bash functions for passing to `hypertune`**
```shell
bash bench/compare2.sh
```
It will generate report in `/tmp` directory & open it in browser.