https://github.com/evilfreelancer/llm-msnp-tests
https://github.com/evilfreelancer/llm-msnp-tests
Last synced: about 1 month ago
JSON representation
- Host: GitHub
- URL: https://github.com/evilfreelancer/llm-msnp-tests
- Owner: EvilFreelancer
- Created: 2024-09-16T22:46:09.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-06-22T17:52:34.000Z (12 months ago)
- Last Synced: 2025-06-22T18:31:13.560Z (12 months ago)
- Language: Python
- Size: 161 KB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# LLM MSNP Tests
A small and collection of results, if checks speed of GGUF model on different combination of GPUs and CPUs.
Inspired due to conversation in Telegram chat.
## MSNP Leaderboards
https://huggingface.co/spaces/evilfreelancer/msnp-leaderboard
## How to test
First you need to install [ollama](https://ollama.com/download) to server where you will make tests.
Download models:
```shell
ollama pull llama3.1:8b-instruct-q4_0
ollama pull llama3.1:70b-instruct-q4_0
```
> Please use models with __quantization__ in name
Then you need to create Python Virtual Environment, then chroot to it:
```shell
mkdir msnp-tests
cd msnp-tests
python3 -m venv venv
source venv/bin/activate
```
Then download requirements and tests.py files:
```shell
wget https://raw.githubusercontent.com/EvilFreelancer/llm-msnp-tests/refs/heads/main/requirements.txt
wget https://raw.githubusercontent.com/EvilFreelancer/llm-msnp-tests/refs/heads/main/test.py
```
Install dependencies:
```shell
pip install -r requirements.txt
```
And run test:
```shell
python3 test.py
```
In result will be something like this:
