{"id":19124198,"url":"https://github.com/tair-opensource/resp-benchmark","last_synced_at":"2025-04-15T11:12:41.304Z","repository":{"id":248686290,"uuid":"829340570","full_name":"tair-opensource/resp-benchmark","owner":"tair-opensource","description":"resp-benchmark is a benchmark tool for testing databases that support the RESP protocol, such as Redis, Valkey, and Tair.","archived":false,"fork":false,"pushed_at":"2025-03-12T03:28:49.000Z","size":543,"stargazers_count":18,"open_issues_count":0,"forks_count":5,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-15T11:12:23.771Z","etag":null,"topics":["benchmark","redis","tair","valkey"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tair-opensource.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-16T08:32:01.000Z","updated_at":"2025-03-27T06:30:46.000Z","dependencies_parsed_at":"2024-08-19T11:22:46.352Z","dependency_job_id":"539a15ee-8618-47a1-b640-679ce9eaa288","html_url":"https://github.com/tair-opensource/resp-benchmark","commit_stats":null,"previous_names":["tair-opensource/resp-benchmark"],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tair-opensource%2Fresp-benchmark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tair-opensource%2Fresp-benchmark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tair-opensource%2Fresp-benchmark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tair-opensource%2Fresp-benchmark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tair-opensource","download_url":"https://codeload.github.com/tair-opensource/resp-benchmark/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249058384,"owners_count":21205911,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","redis","tair","valkey"],"created_at":"2024-11-09T05:28:23.460Z","updated_at":"2025-04-15T11:12:41.277Z","avatar_url":"https://github.com/tair-opensource.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# resp-benchmark\n\n[![Python - Version](https://img.shields.io/badge/python-%3E%3D3.8-brightgreen)](https://www.python.org/doc/versions/)\n[![PyPI - Version](https://img.shields.io/pypi/v/resp-benchmark?color=%231772b4)](https://pypi.org/project/resp-benchmark/)\n[![PyPI - Downloads](https://img.shields.io/pypi/dw/resp-benchmark?color=%231ba784)](https://pypi.org/project/resp-benchmark/)\n[![License](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/tair-opensource/resp-benchmark/blob/main/LICENSE)\n\nresp-benchmark is a benchmark tool for testing databases that support the RESP protocol, \nsuch as [Redis](https://github.com/redis/redis), [Valkey](https://github.com/valkey-io/valkey), \nand [Tair](https://www.alibabacloud.com/en/product/tair). It offers both a command-line interface and a Python library.\n\n## Installation\n\nRequires Python 3.8 or higher.\n```bash\npip install resp-benchmark\n```\n\n## Usage\n\n### Command-Line Tool\n\n```bash\nresp-benchmark --help\n```\n\n### Python Library\n\n```python\nfrom resp_benchmark import Benchmark\n\nbm = Benchmark(host=\"127.0.0.1\", port=6379)\nbm.flushall()\nbm.load_data(command=\"SET {key sequence 10000000} {value 64}\", count=1000000, connections=128)\nresult = bm.bench(\"GET {key uniform 10000000}\", seconds=3, connections=16)\nprint(result.qps, result.avg_latency_ms, result.p99_latency_ms)\n```\n\n## Custom Commands\n\nresp-benchmark supports custom test commands using placeholder syntax like `SET {key uniform 10000000} {value 64}` which means the SET command will have a key uniformly distributed in the range\n0-10000000 and a value of 64 bytes.\n\nSupported placeholders include:\n\n- **`{key uniform N}`**: Generates a random number between `0` and `N-1`. For example, `{key uniform 100}` might generate `key_0000000099`.\n- **`{key sequence N}`**: Sequentially generates from `0` to `N-1`, ensuring coverage during data loading. For example, `{key sequence 100}` generates `key_0000000000`, `key_0000000001`, etc.\n- **`{key zipfian N}`**: Generates according to a Zipfian distribution (exponent 1.03), simulating real-world key distribution.\n- **`{value N}`**: Generates a random string of length `N` bytes. For example, `{value 8}` might generate `92xsqdNg`.\n- **`{rand N}`**: Generates a random number between `0` and `N-1`. For example, `{rand 100}` might generate `99`.\n- **`{range N W}`**: Generates a pair of random numbers within the range `0` to `N-1`, with a difference of `W`, used for testing `*range*` commands. For example, `{range 100 10}` might generate\n  `89 99`.\n\n## Best Practices\n\nNotes:\n1. It is recommended to clear the data each time you test to avoid interference from existing data.\n2. In actual tests, it is recommended to manually adjust the number of `connections`, such as setting it to 128, which can be achieved through `-c 128`. \n\n### Benchmarking network\n\n```shell\n# Test PING command\nresp-benchmark -s 10 \"PING\"\n# Test ECHO command\nresp-benchmark -s 10 \"ECHO {value 64}\"\n```\n\n### Benchmarking string\n\n```shell\n# Test SET command\nresp-benchmark -s 10 \"SET {key uniform 10000000} {value 64}\"\n\n# Test GET command\nresp-benchmark --load -c 256 -P 10 -n 1000000 \"SET {key sequence 10000000} {value 64}\"\nresp-benchmark -s 10 \"GET {key uniform 10000000}\"\n```\n\n### Benchmarking list\n\n```shell\n# Test LPUSH command\nresp-benchmark -s 10 \"LPUSH {key uniform 1000} {value 64}\"\n\n# Test LINDEX command\nresp-benchmark --load -c 256 -P 10 -n 10000000 \"LPUSH {key sequence 1000} {value 64}\"\nresp-benchmark -s 10 \"LINDEX {key uniform 1000} {rand 10000}\"\n```\n\n### Benchmarking set\n\n```shell\n# Test SADD command\nresp-benchmark -s 10 \"SADD {key uniform 1000} {value 64}\"\n\n# Test SISMEMBER command\nresp-benchmark --load -c 256 -P 10 -n 10007000 \"SADD {key sequence 1000} {key sequence 10007}\"\nresp-benchmark -s 10 \"SISMEMBER {key uniform 1000} {key uniform 10007}\"\n```\n\n### Benchmarking zset\n\n```shell\n# Test ZADD command\nresp-benchmark -s 10 \"ZADD {key uniform 1000} {rand 70000} {key uniform 10007}\"\n\n# Benchmark ZSCORE \u0026 ZRANGEBYSCORE\nresp-benchmark --load -P 10 -c 256 -n 10007000 \"ZADD {key sequence 1000} {rand 70000} {key sequence 10007}\"\nresp-benchmark -s 10 \"ZSCORE {key uniform 1000} {key uniform 10007}\"\nresp-benchmark -s 10 \"ZRANGEBYSCORE {key uniform 1000} {range 70000 10}\"\n```\n\n### Benchmarking Lua Scripts\n\n```shell\nredis-cli SCRIPT LOAD \"return redis.call('SET', KEYS[1], ARGV[1])\"\nresp-benchmark -s 10 \"EVALSHA d8f2fad9f8e86a53d2a6ebd960b33c4972cacc37 1 {key uniform 100000} {value 64}\"\n```\n\n## Differences with redis-benchmark\n\nWhen testing Redis with resp-benchmark and redis-benchmark, you might get different results due to:\n\n1. redis-benchmark always uses the same value when testing the set command, which does not trigger DB persistence and replication. In contrast, resp-benchmark uses `{value 64}` to generate different data for each command.\n2. redis-benchmark always uses the same primary key when testing list/set/zset/hash commands, while resp-benchmark generates different keys using placeholders like `{key uniform 10000000}`.\n3. In cluster mode, redis-benchmark sends requests to each node, but all requests target the same slot on every node.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftair-opensource%2Fresp-benchmark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftair-opensource%2Fresp-benchmark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftair-opensource%2Fresp-benchmark/lists"}