{"id":24038970,"url":"https://github.com/jasonbourne723/platodb","last_synced_at":"2025-04-19T16:39:50.852Z","repository":{"id":260482257,"uuid":"881388840","full_name":"Jasonbourne723/platodb","owner":"Jasonbourne723","description":"platodb 是一款高性能的基于 LSM-Tree 的键值数据库，采用模块化架构设计，支持快速的读写性能和崩溃恢复能力。适用于高吞吐量和低延迟需求的存储场景。","archived":false,"fork":false,"pushed_at":"2025-01-16T16:35:45.000Z","size":944,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-29T10:12:38.954Z","etag":null,"topics":["database","distrubuted","golang","lsm-tree"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Jasonbourne723.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-31T13:30:14.000Z","updated_at":"2025-01-16T16:35:47.000Z","dependencies_parsed_at":null,"dependency_job_id":"90e02a9d-9dac-48af-b00b-d6ec4a03aea5","html_url":"https://github.com/Jasonbourne723/platodb","commit_stats":null,"previous_names":["jasonbourne723/platodb"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Jasonbourne723%2Fplatodb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Jasonbourne723%2Fplatodb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Jasonbourne723%2Fplatodb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Jasonbourne723%2Fplatodb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Jasonbourne723","download_url":"https://codeload.github.com/Jasonbourne723/platodb/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249739902,"owners_count":21318671,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","distrubuted","golang","lsm-tree"],"created_at":"2025-01-08T20:19:31.829Z","updated_at":"2025-04-19T16:39:50.836Z","avatar_url":"https://github.com/Jasonbourne723.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"## platodb\nplatodb 是一款高性能的基于 LSM-Tree 的键值数据库，采用模块化架构设计，支持快速的读写性能和崩溃恢复能力。适用于高吞吐量和低延迟需求的存储场景。\n\n## 架构\n\n```\n+---------------------+\n|   Network Server    | (处理客户端请求)\n+---------------------+\n           |\n           ▼\n+---------------------+\n|  Command Processor  | (解析命令)\n+---------------------+\n           |\n           ▼\n+-------------------------------------------------------+\n|                         DB                            |\n|  +---------------------+   +------------------------+ |\n|  |  Memory Table       |   |  Write-Ahead Log (WAL) | |\n|  |  (可变内存存储)      |   |  (崩溃恢复)             | |\n|  +---------------------+   +------------------------+ |\n|                 |                         |           |\n|                 ▼                         ▼           |\n|  +---------------------+   +------------------------+ |\n|  |  Flusher (持久化)    |   |        SSTable         | |\n|  +---------------------+   | (持久化块存储)          | |\n|                            +------------------------+ |\n+-------------------------------------------------------+\n```\n- 网络服务器：负责接收客户端请求，通过 TCP 协议与客户端通信，解析请求后交由命令处理器处理。\n- 命令处理器：负责解析客户端发送的命令，如 Get、Set、Delete，并将其映射为具体的数据库操作。\n- 数据库引擎 (DB)\n  - MemoryTable：活跃的数据存储，所有写入操作优先存储到内存表中。当内存表达到一定阈值时，会触发持久化。\n  - WAL：在写入内存表前记录日志，确保在系统崩溃后可以恢复未持久化的数据。\n  - SSTable：磁盘上的持久化存储结构，采用分段存储，支持二分查找、布隆过滤器加速查询，以及块缓存和快照机制提升读取性能。\n\n## 数据格式\n\n```aiignore\n\n# sstable 分段\n┌────────────┐      ┌────────────┐\n│ 000001.seg │      │ 000002.seg │\n└────────────┘      └────────────┘ \n\n# segment文件分块\n┌──────────┬──────────┬───────────┬────────────┐\n│ block-1  │ block-2  |  block-3  │  block-... │\n└──────────┴──────────┴───────────┴────────────┘\n\n# block块内包含多个记录\n┌──────────┬──────────┬───────────┬────────────┐\n│ chunk-1  │ chunk-2  |  chunk-3  │  chunk-... │\n└──────────┴──────────┴───────────┴────────────┘\n\n# 正常记录的字节格式\n┌──────────┬──────┬─────────┬──────────┬─────────┬────────┐\n│ tombstone│ crc  | KeyLen  │   Key    │ ValueLen│  Value │\n└──────────┴──────┴─────────┴──────────┴─────────┴────────┘\n\n# 已删除记录的字节格式\n┌───────────┬───────┬─────────┬─────────┐\n│ tombstone |  crc  │ KeyLen  │   Key   │\n└───────────┴───────┴─────────┴─────────┘\n```\n\n## 性能表现\n\n### 写\n\n```\ngoos: windows\ngoarch: amd64\ncpu: Intel(R) Core(TM) i7-10700 CPU @ 2.90GHz\nBenchmarkSet-16    \t  224626\t      5283 ns/op\t    1204 B/op\t      14 allocs/op\n```\n\n### 读\n\n```\ngoos: windows\ngoarch: amd64\ncpu: Intel(R) Core(TM) i7-10700 CPU @ 2.90GHz\nBenchmarkGet-16    \t 1023334\t      1083 ns/op\t       0 B/op\t       0 allocs/op\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjasonbourne723%2Fplatodb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjasonbourne723%2Fplatodb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjasonbourne723%2Fplatodb/lists"}