{"id":20434760,"url":"https://github.com/dancing-ui/level_db_xy","last_synced_at":"2025-06-22T09:39:45.940Z","repository":{"id":226862406,"uuid":"769785495","full_name":"dancing-ui/level_db_xy","owner":"dancing-ui","description":null,"archived":false,"fork":false,"pushed_at":"2024-05-30T08:59:02.000Z","size":846,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-15T19:27:47.322Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dancing-ui.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-10T03:42:12.000Z","updated_at":"2024-04-21T00:54:32.000Z","dependencies_parsed_at":"2024-05-30T10:27:07.911Z","dependency_job_id":"4ede826f-c773-4962-8c42-759762ed9b45","html_url":"https://github.com/dancing-ui/level_db_xy","commit_stats":null,"previous_names":["dancing-ui/level_db_xy"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dancing-ui%2Flevel_db_xy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dancing-ui%2Flevel_db_xy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dancing-ui%2Flevel_db_xy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dancing-ui%2Flevel_db_xy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dancing-ui","download_url":"https://codeload.github.com/dancing-ui/level_db_xy/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241977309,"owners_count":20051800,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-15T08:28:45.525Z","updated_at":"2025-03-05T06:26:28.642Z","avatar_url":"https://github.com/dancing-ui.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# level_db_xy\n## 整体架构\n![Alt text](img/structure.png)\n## 基本概念\n### [LSM(Log-Structured-Merge Tree)](https://cloud.tencent.com/developer/article/1143750)\n1. 事实上，LSM树并不像B+树、红黑树一样是一颗严格的树状数据结构，它其实是一种**存储结构**，目前HBase,LevelDB,RocksDB这些NoSQL存储都是采用的LSM树。\n2. LSM 的设计目标是提供比传统的 B+ 树更好的**写性能**。\n   1. LSM 通过将磁盘的随机写转化为顺序写来提高写性能，而付出的代价就是牺牲部分读性能、写放大（B+树同样有写放大的问题）。\n3. LSM 相比 B+ 树能提高写性能的本质原因是：外存——无论磁盘还是 SSD，其随机读写都要慢于顺序读写。\n4. LevelDB 的写操作（Put/Delete/Write）主要由两步组成：\n   1. 写日志（WAL，顺序写）。\n   2. 写 MemTable（内存中的 SkipList）。\n\n所以，正常情况下，LevelDB 的写速度非常快。\n内存中的 MemTable 写满后，会转换为 Immutable MemTable，然后被后台线程 compact 成按 key 有序存储的 SSTable（顺序写）。\nSSTable 按照数据从新到旧被组织成多个层次（上层新下层旧），点查询（Get）的时候从上往下一层层查找，所以 LevelDB 的读操作可能会有多次磁盘 IO（LevelDB 通过 table cache、block cache 和 bloom filter 等优化措施来减少读操作的 I/O 次数）。\n后台线程的定期 compaction 负责回收过期数据和维护每一层数据的有序性。在数据局部有序的基础上，LevelDB 实现了数据的（全局）有序遍历。\n\n### WAL(Write Ahead Log)\n1. WAL预写日志，是数据库系统中常见的一种手段，用于保证数据操作的原子性和持久性。在使用 WAL 的系统中，所有的修改在提交之前都要先写入 log 文件中!\n\n## 版本控制\n1. googletest==\u003ev1.14.0\n## time schedule\n2024.03.10-2024.03-16\n1. 1天完成字符串视图Slice、内存分配器arena\n2. 1天完成SkipList\n3. 3天完成memtable [参考链接](https://cloud.tencent.com/developer/article/1625049)\n4. 2天完成日志读写\n5. 3天完成WriteBatch[参考链接](https://xiaobazhang.github.io/2019/01/30/leveldb%E4%B8%ADWriteBatch%E5%86%99%E6%93%8D%E4%BD%9C/)\n6. 5天完成Sorted Strings Table(SSTable)部分\n   1. 完成FilterPolicy\n      1. 完成默认提供的BloomFilter [参考链接](https://sf-zhou.github.io/leveldb/leveldb_02_data_structure.html)\n   2. 完成Cache，实际花费2天\n   3. 完成Logger，完成logger_posix，目前仅支持posix，\n   4. 完成Env\n      1. 准备实现env_posix，这玩意是真难写啊\n   5. 完成Options\n      1. 集成zstd压缩\n      2. 完成snapshot\n      3. 补齐log_reader、log_writer缺失的crc校验\n   6. 完成BlockBuilder\n   7. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdancing-ui%2Flevel_db_xy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdancing-ui%2Flevel_db_xy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdancing-ui%2Flevel_db_xy/lists"}