{"id":13739342,"url":"https://github.com/ekzhang/redis-rope","last_synced_at":"2025-03-17T00:31:37.901Z","repository":{"id":50608657,"uuid":"517219530","full_name":"ekzhang/redis-rope","owner":"ekzhang","description":"🪢 A fast native data type for manipulating large strings in Redis","archived":false,"fork":false,"pushed_at":"2022-09-05T04:02:59.000Z","size":189,"stargazers_count":118,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-10-13T11:24:57.208Z","etag":null,"topics":["algorithms","data-structures","redis","redis-module","rope","rust","splay-tree","zig"],"latest_commit_sha":null,"homepage":"","language":"Zig","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ekzhang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-07-24T03:44:27.000Z","updated_at":"2024-10-09T19:15:08.000Z","dependencies_parsed_at":"2022-08-23T03:01:31.793Z","dependency_job_id":null,"html_url":"https://github.com/ekzhang/redis-rope","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ekzhang%2Fredis-rope","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ekzhang%2Fredis-rope/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ekzhang%2Fredis-rope/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ekzhang%2Fredis-rope/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ekzhang","download_url":"https://codeload.github.com/ekzhang/redis-rope/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221669244,"owners_count":16860843,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["algorithms","data-structures","redis","redis-module","rope","rust","splay-tree","zig"],"created_at":"2024-08-03T04:00:32.824Z","updated_at":"2024-10-27T11:34:42.840Z","avatar_url":"https://github.com/ekzhang.png","language":"Zig","funding_links":[],"categories":["Applications"],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://i.imgur.com/j1mQdBH.png\" alt=\"redis-rope\" width=\"800\"\u003e\n\u003c/p\u003e\n\nA fast and versatile [rope](\u003chttps://en.wikipedia.org/wiki/Rope_(data_structure)\u003e) data type for large strings in [Redis](https://redis.io), distributed as a native [module](https://redis.io/docs/reference/modules/).\n\n## Overview\n\nRopes are a more efficient data structure for large strings (indexed sequences of bytes). Unlike ordinary strings, ropes let you do some operations up to exponentially faster than their counterparts:\n\n- **Add bytes** to the beginning, middle, or end — any index you want.\n- **Delete any rope substring** or move it to a different position within the rope.\n- **Splice / concatenate any substring** of a rope with any other rope.\n- **Read any substring** with random access.\n\nThe ropes in this module are backed by [splay trees](https://en.wikipedia.org/wiki/Splay_tree), which are a self-adjusting data structure that has logarithmic amortized worst-case performance, while recently-accessed indices are also quick to access in subsequent operations. Each splay tree node stores between 64 and 127 bytes of data.\n\n### Design\n\nSome data structures tend to be too theoretical. This module attempts to provide practical guarantees:\n\n- **The memory usage of a rope is proportional to its length.** It must be a small constant factor more than the number of bytes stored. (Data is stored in chunks; the constant varies based on fragmentation.)\n- **All operations should be fast in practice.** We aim to approach the speed of ordinary strings for simple operations and to be hundreds of times faster for complex operations.\n- **This module never panics.** If a memory allocation fails, it exits gracefully with an error. The database will never be left in a partially modified or inconsistent state.\n- **Stack size is limited and should not overflow.** No operations on arbitrary trees are implemented recursively. We do not create unbounded stack buffers.\n- **Micro-optimizations are not accepted if they make the code less clear.** Safety and correctness is paramount, and code needs to be easily understood by the reader.\n\n### Example / Benchmark\n\nRopes are particularly good at speeding up complex operations on large strings. The following graph shows how performance for ropes scales on 1000 random string SPLICE operations, compared to an equivalent implementation with ordinary Redis strings. (These operations are pipelined to better measure their CPU performance; see the [benchmark code in Rust](rope-bench/src/main.rs).)\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://i.imgur.com/yJHA0xj.png\" alt=\"Latency graph comparing redis-rope and native Redis strings\" width=\"600\"\u003e\n\u003c/p\u003e\n\nFor small strings, there is not much difference. However, each time the length of the string doubles, the basic type gets exponentially slower because it does not scale to large data as well, while the `redis-rope` type provided by this module stays fast.\n\n## Installation\n\nThe `redis-rope` module has been tested with Redis 7.0+. To install, download the appropriate shared library `libredisrope.so` for your platform and load the module from the command line:\n\n```sh-session\nredis-server --loadmodule path/to/libredisrope.so\n```\n\nOr by configuration directive in `redis.conf`:\n\n```\nloadmodule path/to/libredisrope.so\n```\n\nOr from the Redis CLI, using the `MODULE LOAD` command:\n\n```\n\u003e MODULE LOAD path/to/libredisrope.so\n```\n\n### Prebuilt binaries\n\nWe will build shared libraries for each version of redis-rope on Linux and macOS, using x86-64 and ARM64 architectures. These files are small, portable artifacts and are available on the [releases page](https://github.com/ekzhang/redis-rope/releases).\n\n### Building from source\n\n`redis-rope` is written in Zig, which makes building the module from source and cross-compiling very fast (\u003c10 seconds). This is a reasonable option, especially if you want to try out the latest version of the module from the main branch.\n\n```\nzig build -Drelease-fast\n```\n\nThis requires Zig 0.9, which you can install [here](https://ziglang.org/download/). The project can also be built targeting different platforms with a command-line flag, for example:\n\n```\nzig build -Drelease-fast -Dtarget=x86_64-linux-gnu\nzig build -Drelease-fast -Dtarget=aarch64-linux-gnu\n```\n\nBuild outputs are located in the `zig-out/lib` folder.\n\n## Commands\n\n### Read operations\n\nThese are fairly straightfoward: get the length of the rope, any individual byte, or a range of bytes as a string.\n\n- `ROPE.LEN` _key_: **O(1)**\n- `ROPE.GET` _key_ _index_: **O(log N)**\n- `ROPE.GETRANGE` _key_ _start_ _stop_: **O(log N + K)**, where K is the length of the returned string\n\nAll operations support negative indices, which count backward from the end of the rope.\n\n### Write operations\n\nThe append and insert operations push data to the end of the rope, or at an index in the middle of the rope, while the delrange operation deletes a byte range from the rope.\n\nThe splice operation is the most complicated and powerful. Given the keys of two ropes, `source` and `destination`, it appends `destination` to the end of `source` and deletes `destination`. If `start` is provided, the string is inserted at that index rather than appended to the end. If `stop` is provided, then the range of bytes from `start` to `stop` is also deleted from `source` and swapped with the rope at `destination`.\n\n- `ROPE.APPEND` _key_ _str_: **O(1)**\n- `ROPE.INSERT` _key_ _index_ _str_: **O(log N)**, or **O(1)** if _index_ is 0\n- `ROPE.DELRANGE` _key_ _start_ _stop_: **O(log N)**\n- `ROPE.SPLICE` _source_ _destination_ [_start_ \\[_stop_\\]]: **O(log N)**\n\nDespite being quite powerful, each operation above takes logarithmic time, so they will remain fast for arbitrarily long ropes.\n\n### Other operations\n\nThe rope data type supports exact calculations from the `MEMORY USAGE` command, both methods of [Redis persistence](https://redis.io/docs/manual/persistence/) using RDB and AOF, asynchronous `DEL` operations, and primary-replica replication.\n\n## Example usage\n\n```scala\nredis:6379\u003e ROPE.APPEND key1 \"hello\"\n(integer) 5\nredis:6379\u003e ROPE.LEN key1\n(integer) 5\nredis:6379\u003e ROPE.GET key1 2\n\"l\"\nredis:6379\u003e ROPE.APPEND key1 \" world!\"\n(integer) 12\nredis:6379\u003e ROPE.GETRANGE key1 0 -1\n\"hello world!\"\nredis:6379\u003e ROPE.INSERT key1 6 \"rope \"\n(integer) 17\nredis:6379\u003e ROPE.GETRANGE key1 0 -1\n\"hello rope world!\"\nredis:6379\u003e ROPE.DELRANGE key1 -9 -3\n(integer) 10\nredis:6379\u003e ROPE.GETRANGE key1 0 -1\n\"hello rod!\"\nredis:6379\u003e ROPE.APPEND key2 \"goodbye\"\n(integer) 7\nredis:6379\u003e ROPE.SPLICE key1 key2 0 4\n1) (integer) 12\n2) (integer) 5\nredis:6379\u003e ROPE.GETRANGE key1 0 -1\n\"goodbye rod!\"\nredis:6379\u003e ROPE.GETRANGE key2 0 -1\n\"hello\"\nredis:6379\u003e ROPE.SPLICE key1 key2\n1) (integer) 17\n2) (integer) 0\nredis:6379\u003e ROPE.GETRANGE key1 0 -1\n\"goodbye rod!hello\"\nredis:6379\u003e MEMORY USAGE key1\n(integer) 128\nredis:6379\u003e GET key2\n(nil)\nredis:6379\u003e DEL key1\n(integer) 1\nredis:6379\u003e GET key1\n(nil)\n```\n\n## Acknowledgements\n\nCreated by Eric Zhang ([@ekzhang1](https://twitter.com/ekzhang1)). Licensed under the [MIT license](LICENSE).\n\nThanks to [antirez](http://antirez.com/) for creating Redis and [Sleator \u0026 Tarjan](https://www.cs.cmu.edu/~sleator/papers/self-adjusting.pdf) for discovering splay trees.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fekzhang%2Fredis-rope","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fekzhang%2Fredis-rope","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fekzhang%2Fredis-rope/lists"}