{"id":26363351,"url":"https://github.com/zlican/engine","last_synced_at":"2026-05-19T00:36:39.977Z","repository":{"id":275633806,"uuid":"926670231","full_name":"zlican/engine","owner":"zlican","description":"基于 Go 语言实现的分布式搜索引擎","archived":false,"fork":false,"pushed_at":"2025-02-24T09:17:29.000Z","size":66,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-24T10:26:26.142Z","etag":null,"topics":["distributed","engine","golang","kvstore","search","skiplist"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zlican.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-03T17:01:15.000Z","updated_at":"2025-02-24T09:18:35.000Z","dependencies_parsed_at":"2025-02-03T18:52:22.925Z","dependency_job_id":"41e880e5-647a-4cfd-91fa-4e26e81f5d5d","html_url":"https://github.com/zlican/engine","commit_stats":null,"previous_names":["zlican/distributed_search_engine","zlican/engine"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zlican%2Fengine","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zlican%2Fengine/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zlican%2Fengine/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zlican%2Fengine/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zlican","download_url":"https://codeload.github.com/zlican/engine/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243910732,"owners_count":20367546,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["distributed","engine","golang","kvstore","search","skiplist"],"created_at":"2025-03-16T18:27:33.736Z","updated_at":"2026-05-19T00:36:39.940Z","avatar_url":"https://github.com/zlican.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 分布式搜索引擎 [API 文档](https://github.com/zlican/engine/wiki/API-%E6%96%87%E6%A1%A3)\n\n这是一个基于 Go 语言实现的分布式搜索引擎  *latest：v1.0.0*\n![Engine SVG](https://raw.githubusercontent.com/zlican/engine/main/engine.SVG)\n\n## 核心特性\n\n- **分布式架构**: 采用 etcd 作为服务注册中心，支持多节点部署和负载均衡\n- **高性能索引**: 基于跳表(Skip List)实现的倒排索引，支持高效的查询操作\n- **复杂查询**: 支持 AND/OR 逻辑组合的多条件查询\n- **并发控制**: 实现了自定义的并发 Map 结构，优化并发性能\n- **持久化存储**: 支持 Badger 和 Bolt 两种存储引擎\n\n## 系统架构\n\n### 1. 索引结构\n\n- **正排索引**: 存储完整的文档信息\n- **倒排索引**: 基于跳表实现，支持高效的关键词检索\n- **位图过滤**: 支持通过位图实现文档过滤\n\n### 2. 分布式组件\n\n- **服务注册**: 基于 etcd 实现服务注册与发现\n- **负载均衡**: 支持轮询等负载均衡策略\n- **RPC 通信**: 使用 gRPC 实现节点间通信\n\n## 主要功能\n\n### 1. 文档管理\n\n- 添加文档\n- 删除文档\n- 批量操作支持\n\n### 2. 搜索功能\n\n- 关键词搜索\n- 复杂逻辑组合查询\n- 位图过滤支持\n\n### 3. 分布式特性\n\n- 多节点部署\n- 服务自动发现\n- 负载均衡\n- 容错处理\n\n## 目录结构\n\n```bash\n├── engine/\n│ ├── internal/\n│ │ ├── kvdb/ # 存储引擎实现\n│ │ └── reverse_index/ # 倒排索引实现\n│ ├── index_service/ # 分布式服务实现\n│ └── utils/ # 工具函数\n└── types/ # 数据类型定义\n```\n\n## 快速开始\n\n### 安装\n\n```bash\ngit clone https://github.com/zlican/engine.git\ncd engine\ngo mod download\n```\n\n### 启动单机版本\n\n```go\nindexer := new(Indexer)\nindexer.Init(1000, kvdb.BADGER, \"./data\")\n```\n\n### 启动分布式节点\n\n```go\nworker := new(IndexServiceWorker)\nworker.Init(1000, kvdb.BADGER, \"./data\", []string{\"localhost:2379\"}, 8081)\n```\n\n## 使用示例\n\n### 添加文档\n\n```go\ndoc := types.Document{\n    Id: \"1\",\n    Keywords: []*types.Keyword{{Field: \"title\", Word: \"golang\"}},\n    BitsFeature: 1,\n}\nindexer.AddDoc(doc)\n```\n\n### 搜索文档\n\n```go\nquery := types.Be(\"golang\").And(types.Be(\"database\"))\nresults := indexer.Search(query, 0, 0, nil)\n```\n\n## 性能优化\n\n1. 使用跳表实现倒排索引，提供 O(log n) 的查询性能\n2. 自定义并发 Map 结构，优化并发访问性能\n3. 批量操作支持，减少网络开销\n4. 位图过滤，提供高效的文档过滤机制\n\n## 注意事项\n\n1. 确保 etcd 服务正常运行（分布式模式下）\n2. 合理配置存储引擎参数\n3. 注意并发操作的数据一致性\n\n## 贡献指南\n\n欢迎提交 Issue 和 Pull Request。在提交 PR 前，请确保：\n\n1. 代码已经格式化\n2. 测试用例已经补充\n3. 文档已经更新\n\n## 许可证\n\n本项目采用 MIT 许可证。详见 [LICENSE](LICENSE) 文件。\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzlican%2Fengine","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzlican%2Fengine","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzlican%2Fengine/lists"}