{"id":13458233,"url":"https://github.com/indeedeng/lsmtree","last_synced_at":"2025-03-24T15:31:01.745Z","repository":{"id":21942192,"uuid":"25266669","full_name":"indeedeng/lsmtree","owner":"indeedeng","description":"A fast key/value store that is efficient for high-volume random access reads and writes.","archived":true,"fork":false,"pushed_at":"2017-05-17T10:25:15.000Z","size":2577,"stargazers_count":355,"open_issues_count":6,"forks_count":128,"subscribers_count":30,"default_branch":"master","last_synced_at":"2025-03-15T23:36:37.918Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/indeedeng.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-10-15T17:59:19.000Z","updated_at":"2024-10-13T12:42:47.000Z","dependencies_parsed_at":"2022-09-11T17:22:09.416Z","dependency_job_id":null,"html_url":"https://github.com/indeedeng/lsmtree","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/indeedeng%2Flsmtree","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/indeedeng%2Flsmtree/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/indeedeng%2Flsmtree/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/indeedeng%2Flsmtree/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/indeedeng","download_url":"https://codeload.github.com/indeedeng/lsmtree/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245297937,"owners_count":20592506,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T09:00:48.074Z","updated_at":"2025-03-24T15:31:00.552Z","avatar_url":"https://github.com/indeedeng.png","language":"Java","funding_links":[],"categories":["Awesome Algorithms","数据库"],"sub_categories":["LSM - Log-Structured Merge Tree (LSM树)"],"readme":"# Indeed LSM Tree\n\n## About\n\nIndeed LSM Tree is a fast key/value store that is efficient for high-volume random access reads and writes.\n\nIndeed uses it for many applications including serving job data for 100s of millions of job searches each day.\n\nIndeed LSM Tree is composed of three parts:\n\n1. [lsmtree-core](https://github.com/indeedeng/lsmtree/tree/master/lsmtree-core) is an implementation of a log-structured merge-tree.\n2. [recordlog](https://github.com/indeedeng/lsmtree/tree/master/recordlog) is a library for writing data to append-only record logs optimized for replication.\n3. [recordcache](https://github.com/indeedeng/lsmtree/tree/master/recordcache) provides abstractions around writing record logs, building LSM trees, and performing LSM tree lookups.\n\nA typical use case is requiring a key/value store replicated on multiple servers. Instead of replicating the store itself, we store writes in record logs which are slaved to each server. Each server then builds its own LSM tree from the record logs.\n\n## Benchmarks\n\n10,000,000 operations using 8 byte keys and 96 byte values\n\nRandom writes:\n\n| Software        | Time           |\n| ------------- |:-------------:|\n|Indeed LSM Tree|272 seconds|\n|Google LevelDB|375 seconds|\n|Kyoto Cabinet B-Tree|375 seconds|\n\nRandom Reads:\n\n| Software        | Time           |\n| ------------- |:-------------:|\n|Indeed LSM Tree|46 seconds|\n|Google LevelDB|80 seconds|\n|Kyoto Cabinet B-Tree|183 seconds|\n\nSame benchmark using cgroups to limit page cache to 512 MB\n\nRandom Reads:\n\n| Software        | Time           |\n| ------------- |:-------------:|\n|Indeed LSM Tree|454 seconds|\n|Google LevelDB|464 seconds|\n|Kyoto Cabinet B-Tree|50 hours|\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Findeedeng%2Flsmtree","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Findeedeng%2Flsmtree","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Findeedeng%2Flsmtree/lists"}