{"id":13670776,"url":"https://github.com/thustorage/Sherman","last_synced_at":"2025-04-27T13:32:56.603Z","repository":{"id":41066808,"uuid":"459901046","full_name":"thustorage/Sherman","owner":"thustorage","description":"Sherman: A Write-Optimized Distributed B+Tree Index on Disaggregated Memory","archived":false,"fork":false,"pushed_at":"2024-10-05T02:40:11.000Z","size":39,"stargazers_count":101,"open_issues_count":1,"forks_count":30,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-11-11T08:43:11.333Z","etag":null,"topics":["index","memory-disaggregation","rdma","sigmod"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thustorage.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-02-16T07:43:26.000Z","updated_at":"2024-10-28T13:09:27.000Z","dependencies_parsed_at":"2024-01-18T23:04:43.445Z","dependency_job_id":"ac4dfbb0-e903-411f-9aec-940640cf08d7","html_url":"https://github.com/thustorage/Sherman","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thustorage%2FSherman","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thustorage%2FSherman/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thustorage%2FSherman/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thustorage%2FSherman/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thustorage","download_url":"https://codeload.github.com/thustorage/Sherman/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251145697,"owners_count":21543083,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["index","memory-disaggregation","rdma","sigmod"],"created_at":"2024-08-02T09:00:49.376Z","updated_at":"2025-04-27T13:32:51.584Z","avatar_url":"https://github.com/thustorage.png","language":"C++","funding_links":[],"categories":["C++"],"sub_categories":[],"readme":"# Sherman: A Write-Optimized Distributed B+Tree Index on Disaggregated Memory \n\nSherman is a B+Tree on disaggregated memory; it uses one-sided RDMA verbs to perform all index operations.\nSherman includes three techniques to boost write performance:\n\n- A hierarchical locks leveraging on-chip memory of RDMA NICs.\n- Coalescing dependent RDMA commands \n- Two-level version layout in leaf nodes\n\nFor more details, please refer to our [paper](https://dl.acm.org/doi/abs/10.1145/3514221.3517824):\n\n[**SIGMOG'22**] Sherman: A Write-Optimized Distributed B+Tree Index on Disaggregated Memory. Qing Wang and Youyou Lu and Jiwu Shu.\n\n\n## Update (2024.10)\nPlease use [Deft](https://github.com/thustorage/deft) for evaluation, which improving Sherman in performance and correct synchronization.\n\n\n## System Requirements\n\n1. Mellanox ConnectX-5 NICs and above\n2. RDMA Driver: MLNX_OFED_LINUX-4.7-3.2.9.0 (If you use MLNX_OFED_LINUX-5**, you should modify codes to resolve interface incompatibility)\n3. NIC Firmware: version 16.26.4012 and above (to support on-chip memory, you can use `ibstat` to obtain the version)\n4. memcached (to exchange QP information)\n5. cityhash\n6. boost 1.53 (to support `boost::coroutines::symmetric_coroutine`)\n\n## Setup about RDMA Network\n\n**1. RDMA NIC Selection.** \n\nYou can modify this line according the RDMA NIC you want to use, where `ibv_get_device_name(deviceList[i])` is the name of RNIC (e.g., mlx5_0)\nhttps://github.com/thustorage/Sherman/blob/9bb950887cd066ebf4f906edbb43bae8e728548d/src/rdma/Resource.cpp#L28\n\n**2. Gid Selection.** \n\nIf you use RoCE, modify `gidIndex` in this line according to the shell command `show_gids`, which is usually 3.\nhttps://github.com/thustorage/Sherman/blob/c5ee9d85e090006df39c0afe025c8f54756a7aea/include/Rdma.h#L60\n\n**3. MTU Selection.** \n\nIf you use RoCE and the MTU of your NIC is not equal to 4200 (check with `ifconfig`), modify the value `path_mtu` in `src/rdma/StateTrans.cpp`\n\n**4. On-Chip Memory Size Selection.** \n\nChange the constant ``kLockChipMemSize`` in `include/Commmon.h`, making it \u003c= max size of on-chip memory.\n\n## Getting Started\n\n- `cd Sherman`\n- `./script/hugepage.sh` to request huge pages from OS (use `./script/clear_hugepage.sh` to return huge pages)\n- `mkdir build; cd build; cmake ..; make -j`\n- `cp ../script/restartMemc.sh .`\n- configure `../memcached.conf`, where the 1st line is memcached IP, the 2nd is memcached port\n\nFor each run with `kNodeCount` servers:\n- `./restartMemc.sh` (to initialize memcached server)\n- In each server, execute `./benchmark kNodeCount kReadRatio kThreadCount`\n\n\u003e  We emulate each server as one compute node and one memory node: In each server, as the compute node, \nwe launch `kThreadCount` client threads; as the memory node, we launch one memory thread. `kReadRatio` is the ratio of `get` operations.\n\n\u003e In `./test/benchmark.cpp`, we can modify `kKeySpace` and `zipfan`, to generate different workloads.\n\u003e In addition, we can open the macro `USE_CORO` to bind `kCoroCnt` coroutine on each client thread.\n\n## Known bugs\n\n- The two-level version may induce inconsistency in some concurrent cases. Refer to [this SIGMOD'23 paper](https://dl.acm.org/doi/10.1145/3589276)\n\n## TODO\n- Re-write `delete` operations\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthustorage%2FSherman","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthustorage%2FSherman","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthustorage%2FSherman/lists"}