{"id":31644555,"url":"https://github.com/peguesj/ollama-bench","last_synced_at":"2026-05-15T18:34:59.435Z","repository":{"id":315340482,"uuid":"1059099370","full_name":"peguesj/ollama-bench","owner":"peguesj","description":"Comprehensive benchmarking and optimization tool for Ollama models with TUI and CLI interfaces","archived":false,"fork":false,"pushed_at":"2025-09-18T02:03:47.000Z","size":280,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-07T04:57:40.939Z","etag":null,"topics":["benchmark","cli","modelfile","ollama","optimization","performance","python","tui"],"latest_commit_sha":null,"homepage":"https://github.com/peguesj/ollama-bench","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/peguesj.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-18T02:03:06.000Z","updated_at":"2025-09-18T02:03:22.000Z","dependencies_parsed_at":"2025-09-18T04:13:12.432Z","dependency_job_id":"968f405e-f72b-40ba-9917-80ff775f93c2","html_url":"https://github.com/peguesj/ollama-bench","commit_stats":null,"previous_names":["peguesj/ollama-bench"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/peguesj/ollama-bench","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peguesj%2Follama-bench","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peguesj%2Follama-bench/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peguesj%2Follama-bench/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peguesj%2Follama-bench/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/peguesj","download_url":"https://codeload.github.com/peguesj/ollama-bench/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peguesj%2Follama-bench/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33074877,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-15T11:35:32.926Z","status":"ssl_error","status_checked_at":"2026-05-15T11:35:31.362Z","response_time":103,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","cli","modelfile","ollama","optimization","performance","python","tui"],"created_at":"2025-10-07T04:53:58.165Z","updated_at":"2026-05-15T18:34:59.416Z","avatar_url":"https://github.com/peguesj.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Ollama Bench\n\n[![Version](https://img.shields.io/badge/version-2.0.0-blue.svg)](https://github.com/peguesj/ollama-bench)\n[![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)\n[![Python](https://img.shields.io/badge/python-3.8%2B-blue.svg)](https://www.python.org)\n\nA comprehensive benchmarking and optimization tool for Ollama models with both Terminal User Interface (TUI) and Command-Line Interface (CLI) modes.\n\n## Author\n\n**Jeremiah Pegues** \u003cjeremiah@pegues.io\u003e\n\n## Features\n\n### 🚀 Core Features\n- **Dual Interface**: Full-featured TUI with real-time graphs or simple CLI mode\n- **Model Benchmarking**: Compare performance across multiple Ollama models\n- **System Optimization**: Automatically tune models for your hardware\n- **Resource Monitoring**: Real-time CPU, GPU, and RAM usage tracking\n- **Batch Processing**: Optimize multiple models in parallel\n- **Export Results**: Save benchmark data in CSV format\n\n### 🎯 Version 2.0 Features\n- **Modelfile Optimization**: Generate optimized configurations based on system specs\n- **Batch Model Optimization**: Optimize all models with a single command\n- **Hardware Detection**: Automatic detection of CPU, RAM, and GPU capabilities\n- **Platform-Specific Tuning**: Special optimizations for Apple Silicon\n- **Performance Profiling**: Detailed metrics including tokens/sec and memory usage\n\n## Installation\n\n### From Source\n\n```bash\ngit clone https://github.com/peguesj/ollama-bench.git\ncd ollama-bench\npip install -e .\n```\n\n### Dependencies\n\n```bash\npip install psutil pynvml py-cpuinfo\n```\n\n## Quick Start\n\n### TUI Mode (Interactive)\n\n```bash\nollama-bench\n# or\npython -m ollama_bench\n```\n\n### CLI Mode (Non-Interactive)\n\n```bash\nollama-bench --cli\n```\n\n### Direct Scripts\n\n```bash\n# Optimize a single model\npython optimize_model.py llama2\n\n# Optimize all models\npython optimize_all.py --parallel\n\n# Clean up optimized models\npython optimize_all.py --cleanup\n```\n\n## Usage Guide\n\n### TUI Interface\n\nThe TUI provides a rich interactive experience with:\n\n```\n┌─────────────────────────────────────────────────────────┐\n│                    Ollama Bench v2.0.0                  │\n├─────────────────┬───────────────────────────────────────┤\n│  === Models === │      Benchmark Results                │\n│  qwen2.5-coder  │  Model: qwen2.5-coder                │\n│  llama2:7b      │  Tokens/sec: 42.3                    │\n│  codellama:34b  │  Time: 1.2s                          │\n│                 │  Peak RAM: 7.2 GB                    │\n│  === Actions ===│                                       │\n│\u003e Run Benchmark  │  ┌─Performance Graph──────┐           │\n│  Configuration  │  │ ████████████████      │           │\n│  Optimize Model │  │ CPU: 45% GPU: 80%     │           │\n│  Export Results │  └───────────────────────┘           │\n├─────────────────┴───────────────────────────────────────┤\n│ [Up/Down] Navigate  [Enter] Select  [O] Optimize  [Q] Quit │\n├─────────────────────────────────────────────────────────┤\n│ Ready                                    CPU: 12% RAM: 8GB │\n└─────────────────────────────────────────────────────────┘\n```\n\n#### Keyboard Shortcuts\n- **Arrow Keys**: Navigate menu\n- **Enter**: Select menu item\n- **Space**: Start/stop benchmark\n- **O**: Optimize selected model (or all if none selected)\n- **E**: Edit configuration\n- **M**: Edit Modelfile\n- **X**: Export results\n- **Q**: Quit\n\n### CLI Interface\n\nThe CLI provides a simple menu-driven interface:\n\n```bash\n$ ollama-bench --cli\n\n============================================================\nOllama Bench CLI - Benchmarking Tool\n============================================================\n\n=== Main Menu ===\n1. Run Benchmark\n2. List Models\n3. Show Configuration\n4. Export Results\n5. Optimize Single Model\n6. Optimize All Models\n7. Show System Info\n8. Clean Optimized Models\nQ. Quit\n\nEnter choice: \n```\n\n### Model Optimization\n\n#### System Analysis\n\n```bash\n$ ollama-bench --cli\n# Select option 7\n\nSystem Specifications\n============================================================\nPlatform: Darwin (Apple Silicon)\nCPU: 12 cores @ 3.2 GHz\nRAM: 48.0 GB total, 32.0 GB available\nGPU: Apple Silicon GPU (36.0 GB)\n\nOptimal Parameters\n============================================================\nContext Size: 4096 tokens\nBatch Size: 512\nThreads: 11\nGPU Layers: 999\n```\n\n#### Batch Optimization\n\n```bash\n# Optimize all models with parallel processing\n$ python optimize_all.py --parallel --workers 4\n\n# Optimize specific models\n$ python optimize_all.py llama2:7b codellama:13b\n\n# Generate benchmark comparison script\n$ python optimize_all.py --benchmark\n\n# Clean up when done\n$ python optimize_all.py --cleanup\n```\n\n## Optimization Parameters\n\nThe optimizer automatically configures:\n\n| Parameter | Description | Impact |\n|-----------|-------------|--------|\n| `num_ctx` | Context window size | Larger = better comprehension |\n| `num_batch` | Batch processing size | Larger = higher throughput |\n| `num_gpu` | GPU layers to offload | 999 = full GPU acceleration |\n| `num_thread` | CPU threads | Optimized for core count |\n| `use_mlock` | Memory locking | Prevents swapping |\n| `use_mmap` | Memory mapping | Efficient for large models |\n\n## Model Recommendations by RAM\n\n| Available RAM | Model Size | Example Models |\n|--------------|------------|----------------|\n| \u003c 8 GB | 3B-7B | qwen2.5:3b, tinyllama |\n| 8-16 GB | 7B | llama2:7b, mistral:7b |\n| 16-32 GB | 13B | llama2:13b, codellama:13b |\n| 32-64 GB | 34B | codellama:34b |\n| \u003e 64 GB | 70B+ | llama2:70b, mixtral:8x7b |\n\n## Benchmark Results\n\nResults are saved in CSV format with detailed metrics:\n\n```csv\nmodel,iteration,elapsed_s,tokens_per_sec,peak_rss_bytes,cpu_percent,gpu_percent\nqwen2.5-coder,1,1.234,42.3,7516192768,45.2,78.9\nllama2:7b,1,2.456,38.1,8589934592,52.1,82.3\n```\n\n## Configuration\n\nConfiguration is stored in `~/.config/ollama_bench/config.yaml`:\n\n```yaml\nbenchmark:\n  iterations: 3\n  timeout: 120\n  num_predict: 100\n  temperature: 0.7\n  seed: 42\n  workdir: ~/.ollama_bench\n\nresources:\n  max_cpu_percent: 80\n  max_gpu_percent: 90\n  max_ram_gb: null\n  throttle_enabled: false\n\nui:\n  theme: default\n  refresh_rate: 0.5\n  show_graph: true\n```\n\n## Performance Improvements\n\nTypical optimization results:\n- **Speed**: 20-70% faster token generation\n- **Memory**: 10-20% lower RAM usage  \n- **Stability**: Reduced out-of-memory errors\n- **Efficiency**: Better CPU/GPU utilization\n\n## Development\n\n### Project Structure\n\n```\nollama-bench/\n├── ollama_bench/           # Main package\n│   ├── core/              # Core functionality\n│   │   ├── benchmark.py   # Benchmarking engine\n│   │   ├── models.py      # Model management\n│   │   ├── monitor.py     # Resource monitoring\n│   │   ├── config.py      # Configuration\n│   │   ├── system_optimizer.py  # Hardware optimization\n│   │   └── batch_optimizer.py   # Batch processing\n│   ├── tui/               # Terminal UI\n│   │   ├── app.py        # Main TUI application\n│   │   ├── components/   # UI components\n│   │   └── widgets/      # Interactive widgets\n│   ├── cli.py            # CLI interface\n│   └── utils/            # Utilities\n├── optimize_model.py      # Single model optimizer\n├── optimize_all.py        # Batch optimizer\n└── setup.py              # Package setup\n```\n\n### Testing\n\n```bash\n# Run tests\npython test_optimization.py\n\n# Test TUI import\npython -c \"from ollama_bench.tui import OllamaBenchTUI\"\n\n# Test CLI\npython -m ollama_bench.cli\n```\n\n## Troubleshooting\n\n### Terminal Issues\n- If you see Unicode errors, the tool automatically falls back to ASCII\n- For best results, use a terminal that supports UTF-8\n\n### GPU Detection\n- NVIDIA: Requires nvidia-ml-py\n- Apple Silicon: Automatic Metal acceleration\n- No GPU: Falls back to CPU-only optimization\n\n### Memory Issues\n- Reduce `num_ctx` for lower memory usage\n- Enable `low_vram` mode for limited GPU memory\n- Use quantized models (q4_0, q4_K_M)\n\n## Contributing\n\nContributions are welcome! Please:\n\n1. Fork the repository\n2. Create a feature branch\n3. Run tests and benchmarks\n4. Submit a pull request\n\n## License\n\nMIT License - see [LICENSE](LICENSE) file\n\n## Acknowledgments\n\n- Ollama team for the excellent local LLM platform\n- Python curses library for terminal UI capabilities\n- psutil for cross-platform system monitoring\n\n## Changelog\n\n### Version 2.0.0 (2024)\n- Added Modelfile optimization based on system specs\n- Implemented batch model optimization\n- Added hardware detection and profiling\n- Improved TUI with optimization features\n- Added parallel processing support\n- Fixed terminal compatibility issues\n\n### Version 1.0.0 (2024)\n- Initial release with TUI and CLI interfaces\n- Basic benchmarking functionality\n- Resource monitoring\n- Model management\n\n## Contact\n\n**Author**: Jeremiah Pegues  \n**Email**: jeremiah@pegues.io  \n**GitHub**: [github.com/peguesj](https://github.com/peguesj)\n\n---\n\n*Built with ❤️ for the Ollama community*","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpeguesj%2Follama-bench","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpeguesj%2Follama-bench","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpeguesj%2Follama-bench/lists"}