{"id":18473929,"url":"https://github.com/ohsayan/sky-benches","last_synced_at":"2025-10-24T21:05:05.102Z","repository":{"id":41873288,"uuid":"393267260","full_name":"ohsayan/sky-benches","owner":"ohsayan","description":"Attempts at benchmarking Skytable with the others to see where we stand","archived":false,"fork":false,"pushed_at":"2023-12-05T20:56:41.000Z","size":434,"stargazers_count":16,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"next","last_synced_at":"2023-12-05T21:40:53.229Z","etag":null,"topics":["benchmark","database","key-value","key-value-store","nosql","redis","skytable"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ohsayan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-08-06T05:52:49.000Z","updated_at":"2023-11-21T04:01:46.000Z","dependencies_parsed_at":"2022-08-11T19:51:00.569Z","dependency_job_id":null,"html_url":"https://github.com/ohsayan/sky-benches","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ohsayan%2Fsky-benches","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ohsayan%2Fsky-benches/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ohsayan%2Fsky-benches/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ohsayan%2Fsky-benches/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ohsayan","download_url":"https://codeload.github.com/ohsayan/sky-benches/tar.gz/refs/heads/next","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223319347,"owners_count":17125854,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","database","key-value","key-value-store","nosql","redis","skytable"],"created_at":"2024-11-06T10:27:12.450Z","updated_at":"2025-10-24T21:05:05.081Z","avatar_url":"https://github.com/ohsayan.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Skytable Benchmarks\n\nThe goal here is to see how [Skytable](https://github.com/skytable/skytable), which is primarily in-memory but instead of simple commands uses an actual query language (BlueQL) stacks up against in-memory key/value stores like Redis (which is inherently single threaded except for I/O threads) and KeyDB (which is multi-threaded).\n\n\u003e ### IMPORTANT NOTE: Skytable has an actual query language BlueQL while others don't. This means Skytable does way more work than the key/value stores which use commands!\n\n## Results\n\n![Benchmark graphs](./benches.svg)\n\n### Results conclusion\n\n**Even with a complete query language, Skytable is as fast, or faster** than popular key/value stores (which use simple commands and do not have a query language).\n\n**Legend**:\n- Redis is red\n- KeyDB is orange\n- Skytable is blue\n\n## Notes\n\n\u003e **Future planned benchmarks**:\n\u003e - Dragonfly\n\u003e - Aerospike\n\u003e - MongoDB\n\u003e - ArangoDB\n\u003e\n\u003e **Future planned implementation**:\n\u003e - Automated benches and report generation\n\u003e - Benchmark matrix so that you can benchmark one database against another, as you choose! Compatible commands/queries would be automatically generated\n\u003e\n\u003e **Future planned VM benches**:\n\u003e - Bench against multiple AWS+GCP+Azure VM types\n\u003e - Bench across x86_64 and ARM64\n\n## Benchmark setup\n\n- **Here's what we'll do:**\n  - We'll start up each server with the ability to use multi-threads (or io-threads) and set this to the number of available logical CPU cores (for example a processor might have 6 cores but with HT can run 12 threads so we go with 12)\n  - For each CRD operation, we'll run 1,000,000 queries for a total of 4M queries.\n    \u003e **Why not `UPDATE`s?** Unfortunately, there's no SQL equivalent of an `UPDATE` in Redis/KeyDB. So, we skip it for now.\n  - We'll run a CRUD sequence:\n    - For simple key/value stores:\n      - `SET \u003c7 digit key\u003e \u003c1 byte value\u003e`\n      - `GET \u003c7 digit key\u003e`\n      - `DEL \u003c7 digit key\u003e`\n    - For Skytable:\n      - `INSERT INTO mymodel(\u003c7 digit key\u003e, \u003c1 byte value\u003e)`\n      - `SELECT pw FROM mymodel WHERE un = \u003c7 digit key\u003e`\n      - `DELETE FROM mymodel WHERE un = \u003c7 digit key\u003e`\n- **Benchmark machine**:\n  - The benchmarking machine has:\n    - a 6C/12T configuration\n    - 32GB RAM\n    - Base clock: 2.7Ghz, Turbo: 4.5 Ghz\n    - Arch: Intel x86_64 (manufacturer is also Intel)\n    - Tiger Lake CPU (11th Generation)\n  - **OS**:\n    - Ubuntu 22.04\n    - Kernel: 5.19.0-42-generic\n    - Governor: `performance` (checking `cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor`)\n  - **Database versions**:\n    - Skytable v0.8.0-beta\n    - Redis `master` (991aff1)\n    - KeyDB `master` (674d9fb)\n\n## How the benchmarks were run\n\n- Skytable:\n  - Server started with `skyd \u003crootpass\u003e`\n  - Benchmark tool run with varying values for `\u003cconnections\u003e`: `sky-bench --password \u003crootpass\u003e --connections \u003cconnections\u003e`\n- Redis and KeyDB:\n  - Server started with `redis-server --io-threads=12` and `keydb-server --io-threads=12`\n  - Benchmark tool run for different tests (`-t` each for `get` and `set`) like this: `redis-benchmark --threads 12 -n 1000000 -d 7 -t \u003ctestname\u003e -c \u003cconnections\u003e`\n\n## License\n\nThis project is distributed under the [MIT License](/LICENSE).","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fohsayan%2Fsky-benches","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fohsayan%2Fsky-benches","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fohsayan%2Fsky-benches/lists"}