{"id":19839414,"url":"https://github.com/redislabs/crdbmemcalc","last_synced_at":"2026-04-16T01:31:20.147Z","repository":{"id":141980347,"uuid":"155682118","full_name":"RedisLabs/crdbmemcalc","owner":"RedisLabs","description":"CRDB/Redis memory calculator for synthetic datasets","archived":false,"fork":false,"pushed_at":"2023-03-31T14:34:52.000Z","size":22,"stargazers_count":2,"open_issues_count":1,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-06-16T13:04:48.075Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RedisLabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-11-01T08:05:38.000Z","updated_at":"2022-10-21T23:47:38.000Z","dependencies_parsed_at":null,"dependency_job_id":"69d11c2d-bf18-417f-bd89-52db97cdb21a","html_url":"https://github.com/RedisLabs/crdbmemcalc","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/RedisLabs/crdbmemcalc","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RedisLabs%2Fcrdbmemcalc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RedisLabs%2Fcrdbmemcalc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RedisLabs%2Fcrdbmemcalc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RedisLabs%2Fcrdbmemcalc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RedisLabs","download_url":"https://codeload.github.com/RedisLabs/crdbmemcalc/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RedisLabs%2Fcrdbmemcalc/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31867709,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-15T15:24:51.572Z","status":"ssl_error","status_checked_at":"2026-04-15T15:24:39.138Z","response_time":63,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T12:22:20.028Z","updated_at":"2026-04-16T01:31:20.123Z","avatar_url":"https://github.com/RedisLabs.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# crdbmemcalc\n\nThis is a tool that makes it easier to calculate the Redis Enterprise\nActive-Active (CRDT) overhead for a specified dataset.\n\nDatasets are specified using a custom JSON-based spec file which describes\nvarious aspects of the dataset, such as:\n\n* Types of keys\n* Length of key name\n* Length of values\n* Similar attributes of sub-elements (e.g. hash fields, list/set elements,\n  etc.)\n\nThe tool then spins up a Redis process, populates it with the synthetic dataset\nand produces a report comparing Redis and Redis CRDT memory usage.\n\n## Getting Started\n\nAs the `crdbmemcalc` tool requires access to `redis-server` and `crdt.so`\nbinaries, it is best to deploy it on a test Redis Enterprise cluster node.\n\nFirst, clone the repository and install it, run the following commands:\n\n    # Clone\n    git clone https://github.com/redislabs/crdbmemcalc\n    cd crdbmemcalc\n    python setup.py install\n\nYou can confirm the installation is successful by running:\n\n    crdbmemcalc --help\n\nNext, you can run `crdbmemcalc` using the default sample spec file provided:\n\n    crdbmemcalc -s testspec.json\n\n## What is reported?\n\nThe reports includes a section for every dataset described in the specfile. For\nexample:\n\n```\nDATASET CONFIGURATION\n -- Sorted Sets: large sorted sets, short elements\n\nKeys:\nKey Type             : sorted_set\nLength               : 20\n# Of Elements        : 500\nElement Length       : 20\n\n\nRedis Configuration:\n                          (Configured)      CRDB-5.0.9 (Actual)    Redis-5.0.9 (Actual)\n------------------------  --------------  ---------------------  ----------------------\nset-max-intset-entries                                      512                     512\nlist-compress-depth                                           0                       0\nzset-max-ziplist-value                                       64                      64\nhash-max-ziplist-value                                       64                      64\nhash-max-ziplist-entries                                    512                     512\nzset-max-ziplist-entries                                    128                     128\n```\n\nThe section begins with a description of the keys that make up the dataset, as\nspecified in the spec file.\n\nNext, it describes Redis configuration parameters that may affect the dataset in\nmemory.\n\nAt the bottom of the report you can find the memory used by each of the datasets\nin Redis and Redis CRDT:\n\n```\n                                                  CRDB-5.0.9    Redis-5.0.9      %\n----------------------------------------------  ------------  -------------  -----\nHashes: large hashes, short keys and values          4111480        2465024    +66\nLists: long lists, short elements                   13661024        1144224  +1093\nLists: short lists, short elements                    605024          65824   +819\nSets: large sets, short elements                     4435424        2826624    +56\nSets: small sets, short elements                      224224         151424    +48\nSorted Sets: large sorted sets, short elements       8797816        5248768    +67\nSorted Sets: small sorted sets, short elements        143480          58624   +144\nStrings: short keys and values                         21880          11424    +91\nStrings: short keys, 1K values                        145880         135424     +7\n```\n\n## Creating a spec file\n\nThe spec file is a JSON document that describes datasets that are generated and\ntested.\n\nThe `datasets` field contains an array of dataset objects, each identified by a\nname and a list of keys. For example:\n\n```json\n{\n    \"datasets\": [\n        { \n            \"name\": \"Strings: short keys and values\",\n            \"keys\": [ ... ],\n        },\n        { \n            \"name\": \"Strings: short keys, 1K values\",\n            \"keys\": [ ... ],\n        }\n    ]\n}\n```\n\nThe `keys` dataset field describes a list of dataset keys. Each dataset key is\ndescribed by a name, a key name length, type and value. It is possible to create\ndatasets with different key specifications. For example:\n\n```json\n        { \n            \"name\": \"Sets: small sets, short elements\",\n            \"keys\": [\n              {\n                \"length\": 20,\n                \"value\": {\n                    \"type\": \"set\",\n                    \"elements_num\": 20,\n                    \"element_length\": 20\n                }\n              }\n          ]\n        }\n```\n\nEach `dataset` object may also contain a `redis_config_params` object which\nincludes specific Redis configuration parameters to apply.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fredislabs%2Fcrdbmemcalc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fredislabs%2Fcrdbmemcalc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fredislabs%2Fcrdbmemcalc/lists"}