{"id":49270186,"url":"https://github.com/fabianwimberger/cloud-bench","last_synced_at":"2026-05-09T12:04:49.025Z","repository":{"id":339795215,"uuid":"1163276387","full_name":"fabianwimberger/cloud-bench","owner":"fabianwimberger","description":"Benchmark CPU, memory, and disk performance across Hetzner, AWS, OVH, and OCI instances. With cost analysis.","archived":false,"fork":false,"pushed_at":"2026-04-20T17:36:50.000Z","size":826,"stargazers_count":20,"open_issues_count":4,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2026-04-20T18:38:09.567Z","etag":null,"topics":["ansible","aws","benchmark","cloud","cost-analysis","hetzner","iac","infrastructure","oci","ovh","performance-testing","terraform"],"latest_commit_sha":null,"homepage":"https://fabianwimberger.github.io/cloud-bench/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fabianwimberger.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-21T11:28:14.000Z","updated_at":"2026-04-20T17:36:38.000Z","dependencies_parsed_at":"2026-02-21T22:07:09.901Z","dependency_job_id":null,"html_url":"https://github.com/fabianwimberger/cloud-bench","commit_stats":null,"previous_names":["fabianwimberger/cloud-bench"],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/fabianwimberger/cloud-bench","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabianwimberger%2Fcloud-bench","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabianwimberger%2Fcloud-bench/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabianwimberger%2Fcloud-bench/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabianwimberger%2Fcloud-bench/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fabianwimberger","download_url":"https://codeload.github.com/fabianwimberger/cloud-bench/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabianwimberger%2Fcloud-bench/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32263111,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-25T09:15:33.318Z","status":"ssl_error","status_checked_at":"2026-04-25T09:15:31.997Z","response_time":59,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ansible","aws","benchmark","cloud","cost-analysis","hetzner","iac","infrastructure","oci","ovh","performance-testing","terraform"],"created_at":"2026-04-25T13:16:55.433Z","updated_at":"2026-05-09T12:04:48.997Z","avatar_url":"https://github.com/fabianwimberger.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cloud Bench\n\n[![CI](https://github.com/fabianwimberger/cloud-bench/actions/workflows/validate.yml/badge.svg)](https://github.com/fabianwimberger/cloud-bench/actions)\n[![codecov](https://codecov.io/gh/fabianwimberger/cloud-bench/branch/main/graph/badge.svg)](https://codecov.io/gh/fabianwimberger/cloud-bench)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\nA cloud instance benchmarking suite comparing CPU, memory, and disk performance across providers with cost analysis. 31 instance types across Hetzner Cloud, AWS EC2, OVHcloud, Oracle Cloud (OCI), Google Cloud Platform (GCP), and Microsoft Azure.\n\n## Background\n\nA €20 instance can mean very different things between providers. This runs the same sysbench and fio suite on each, tracks cost per benchmark point, and publishes the results to GitHub Pages. Infrastructure is provisioned and torn down via Terraform and Ansible so runs stay reproducible.\n\n## Features\n\n- **Multi-provider** — Hetzner Cloud, AWS EC2, OVHcloud, OCI, GCP, Azure (31 instance types)\n- **Standardized benchmarks** — CPU (sysbench), Memory (sysbench), Disk I/O (fio)\n- **Metric averaging** — scores based on all historical runs, not just the latest\n- **Cost analysis** — performance per dollar with EUR/USD toggle\n- **Interactive dashboard** — filtering, comparison, per-instance history charts\n- **Automated pricing** — live pricing from provider APIs + ECB exchange rates\n- **Security-first** — fresh SSH keys per run, firewall whitelisting, automatic cleanup\n- **Cost guards** — pre-run estimation blocks expensive configurations ($5 / 15 instance limit)\n- **Local benchmarking** — standalone script for users to compare their own hardware\n\n## Live Dashboard\n\nView the latest results at **[fabianwimberger.github.io/cloud-bench](https://fabianwimberger.github.io/cloud-bench/)**\n\nScores shown are averaged across all benchmark runs for each instance type, not just the latest run.\n\n## Pipeline\n\n```mermaid\nflowchart LR\n    TF[Terraform\u003cbr/\u003eprovision] --\u003e AN[Ansible\u003cbr/\u003esysbench, fio]\n    AN --\u003e R[Collect JSON]\n    R --\u003e TFD[Terraform\u003cbr/\u003edestroy]\n    R --\u003e P[Process\u003cbr/\u003enormalize, price]\n    P --\u003e D[GitHub Pages\u003cbr/\u003edashboard]\n```\n\n## Quick Start\n\n### Local\n\nCompare your own machine against the official results. No cloud credentials needed:\n\n```bash\ngit clone https://github.com/fabianwimberger/cloud-bench.git\ncd cloud-bench\nsudo bash scripts/run-local-bench.sh\n```\n\nRuns the same sysbench/fio benchmarks and outputs a JSON file plus a terminal summary. See [docs/local-benchmark.md](docs/local-benchmark.md) for details.\n\n### Cloud\n\n```bash\n# Clone and configure\ngit clone https://github.com/fabianwimberger/cloud-bench.git\ncd cloud-bench\n\n# Hetzner Cloud\nexport HCLOUD_TOKEN=\"your-token\"\n./scripts/run-local.sh\n\n# AWS\nexport AWS_ACCESS_KEY_ID=\"your-key\"\nexport AWS_SECRET_ACCESS_KEY=\"your-secret\"\nPROVIDER=aws ./scripts/run-local.sh\n\n# OVHcloud\nexport OVH_OPENSTACK_USERNAME=\"user-xxxxx\"\nexport OVH_OPENSTACK_PASSWORD=\"your-password\"\nexport OVH_CLOUD_PROJECT_ID=\"your-project-id\"\nPROVIDER=ovhcloud ./scripts/run-local.sh\n\n# GCP\nexport GCP_PROJECT_ID=\"your-project-id\"\nexport GCP_CREDENTIALS='{\"type\": \"service_account\", ...}'  # Service account JSON key\nPROVIDER=gcp ./scripts/run-local.sh\n\n# Azure\nexport AZURE_SUBSCRIPTION_ID=\"your-subscription-id\"\nexport AZURE_CLIENT_ID=\"your-client-id\"\nexport AZURE_CLIENT_SECRET=\"your-client-secret\"\nexport AZURE_TENANT_ID=\"your-tenant-id\"\nPROVIDER=azure ./scripts/run-local.sh\n\n# Or run via GitHub Actions: Actions \u003e Run Benchmarks\n```\n\n## How It Works\n\n```\nconfig/instances.yaml → Terraform → Ansible (sysbench/fio) → Python (scoring) → React Dashboard\n```\n\n**Methodology:**\n- 5 runs per test, median value used for consistency\n- Scores normalized 0-100 per category\n- Weights: CPU 40%, Memory 35%, Disk 25%\n- Raw metrics averaged across all historical runs per instance\n- Cost-efficiency calculated from monthly price and composite score\n- All benchmarks run on Ubuntu 24.04 in EU (Frankfurt) regions\n\n## Configuration\n\nEdit `config/instances.yaml` to add/remove instances:\n\n```yaml\nproviders:\n  hetzner:\n    currency: EUR\n    default_region: fsn1\n    instances:\n      - id: cx23\n        name: CX23\n        arch: X86\n        vcpu: 2\n        ram_gb: 4\n        disk_gb: 40\n        pricing:\n          hourly: 0.0048\n          monthly: 2.99\n```\n\nNo code changes needed — Terraform, Ansible, and the frontend all pick up config changes automatically. See [docs/configuration.md](docs/configuration.md) for details.\n\n## Security\n\n- Fresh Ed25519 SSH key generated per run, never reused\n- Firewall/security group allows SSH from runner IP only\n- Auto-cleanup via `if: always()` — infrastructure destroyed even if benchmarks fail\n- Orphan cleanup workflows as safety net (manual trigger for all providers)\n\n## Documentation\n\n- [Setup Guide](docs/setup-guide.md) — credentials and first run\n- [Configuration](docs/configuration.md) — instances, regions, providers\n- [Architecture](docs/architecture.md) — system design and scoring\n- [Data Format](docs/data-format.md) — JSON schemas\n- [Cost Analysis](docs/cost-analysis.md) — per-run costs and safety nets\n- [Runbook](docs/runbook.md) — troubleshooting\n- [Local Benchmark](docs/local-benchmark.md) — run benchmarks on your own machine\n\n## License\n\nMIT License — see [LICENSE](LICENSE) file.\n\n### Third-Party Licenses\n\n| Component | License | Source |\n|-----------|---------|--------|\n| sysbench | [GPL v2+](https://github.com/akopytov/sysbench/blob/master/COPYING) | https://github.com/akopytov/sysbench |\n| fio | [GPL v2](https://github.com/axboe/fio/blob/master/COPYING) | https://github.com/axboe/fio |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffabianwimberger%2Fcloud-bench","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffabianwimberger%2Fcloud-bench","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffabianwimberger%2Fcloud-bench/lists"}