{"id":15567577,"url":"https://github.com/wgrape/esupdater","last_synced_at":"2026-03-10T02:06:34.320Z","repository":{"id":40626125,"uuid":"437779131","full_name":"WGrape/esupdater","owner":"WGrape","description":"A high-performance lightweight framework of PHP to achieve incremental update of ES documents / 一个基于Canal实现ES文档增量更新的高性能轻量框架","archived":false,"fork":false,"pushed_at":"2023-01-17T09:50:21.000Z","size":170,"stargazers_count":205,"open_issues_count":3,"forks_count":17,"subscribers_count":3,"default_branch":"v2.x","last_synced_at":"2025-03-17T12:09:39.808Z","etag":null,"topics":["binlog","canal","composer","docker","elasticsearch","es","framework","kafka","mysql","php","php7"],"latest_commit_sha":null,"homepage":"","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/WGrape.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":".github/CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-12-13T07:41:15.000Z","updated_at":"2025-03-11T08:15:20.000Z","dependencies_parsed_at":"2023-02-10T09:01:17.014Z","dependency_job_id":null,"html_url":"https://github.com/WGrape/esupdater","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WGrape%2Fesupdater","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WGrape%2Fesupdater/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WGrape%2Fesupdater/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WGrape%2Fesupdater/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/WGrape","download_url":"https://codeload.github.com/WGrape/esupdater/tar.gz/refs/heads/v2.x","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245761295,"owners_count":20667895,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["binlog","canal","composer","docker","elasticsearch","es","framework","kafka","mysql","php","php7"],"created_at":"2024-10-02T17:11:51.836Z","updated_at":"2026-03-10T02:06:34.217Z","avatar_url":"https://github.com/WGrape.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\" \u003e\n\u003cimg width=\"200\" alt=\"img\" src=\"https://user-images.githubusercontent.com/35942268/147061994-f0d5a3ec-2d5f-4d72-af1c-139289547f25.png\"\u003e\n\u003c/div\u003e\n\n\u003cdiv align=\"center\"\u003e\n    \u003cp\u003e一个基于Canal实现ES文档增量更新的高性能轻量框架\u003c/p\u003e\n\u003c/div\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://www.oscs1024.com/project/oscs/WGrape/esupdater?ref=badge_small\" alt=\"OSCS Status\"\u003e\u003cimg src=\"https://www.oscs1024.com/platform/badge/WGrape/esupdater.svg?size=small\"/\u003e\u003c/a\u003e\n    \u003cimg src=\"https://img.shields.io/badge/php-7.0+-blue.svg\"\u003e\n    \u003cimg alt=\"GitHub release (latest by date)\" src=\"https://img.shields.io/github/v/release/wgrape/esupdater\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/version-2.x-blue.svg\"\u003e\n    \u003cimg alt=\"Docker Pulls\" src=\"https://img.shields.io/docker/pulls/lvsid/phpkafka\"\u003e\n    \u003ca href=\"https://app.travis-ci.com/github/WGrape/esupdater\"\u003e\u003cimg src=\"https://app.travis-ci.com/WGrape/esupdater.svg?branch=master\"\u003e\u003ca\u003e\n    \u003ca href=\"https://wgrape.github.io/esupdater/report.html\"\u003e\u003cimg src=\"https://img.shields.io/badge/unitest-100%25-yellow.svg\"\u003e\u003c/a\u003e\n    \u003ca href=\"LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/badge/license-MIT-green.svg\"\u003e\u003c/a\u003e\n    \u003ca href=\"doc/HOWTOCODE.md\"\u003e\u003cimg src=\"https://img.shields.io/badge/doc-中文-red.svg\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n- [一、介绍](#1)\n- \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;[1、轻量级框架](#11)\n- \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;[2、全面容器化](#12)\n- \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;[3、事件驱动化](#13)\n- \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;[4、插件化扩展](#14)\n- \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;[5、高性能消费](#15)\n- [二、快速上手](#2)\n- [三、业务接入](#3)\n- [四、扩展列表](#4)\n- [五、关于项目](#5)\n- \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;[1、深入了解](#51)\n- \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;[2、欢迎参与](#52)\n- [六、贡献列表](#6)\n\n## \u003cspan id=\"1\"\u003e一、介绍\u003c/span\u003e\nESUpdater是一个基于Canal实现ES文档增量更新的高性能轻量框架。基于以下优势，可以让你快速上手和使用。\n\n\u003cimg width=\"900\" alt=\"Architecture\" src=\"https://user-images.githubusercontent.com/35942268/145793762-a23899d6-c162-4527-ae72-643edc80bb18.png\"\u003e\n\n### \u003cspan id=\"11\"\u003e1、轻量级框架\u003c/span\u003e\n无论安装使用，还是代码设计，整个框架都非常轻量，优雅的完成数据二次处理和ES增量更新。\n\n### \u003cspan id=\"12\"\u003e2、全面容器化\u003c/span\u003e\n为解决各种依赖安装的复杂麻烦问题，已实现全面容器化，只需一条命令就可以轻松安装、部署、和维护。\n\n### \u003cspan id=\"13\"\u003e3、事件驱动化\u003c/span\u003e\n基于框架内部的事件驱动设计，可以轻松地注册不同数据表的变更事件和回调，优雅地实现增量更新。\n\n### \u003cspan id=\"14\"\u003e4、插件化扩展\u003c/span\u003e\n在不影响框架内部运行的前提下，支持插件化扩展，实现对内部行为的自定义扩展。\n\n### \u003cspan id=\"15\"\u003e5、高性能消费\u003c/span\u003e\n通过一个```Consumer```进程和多个```Worker```进程的一对多通信模型，最少提高10倍的吞吐量，实现高性能消费。\n\n## \u003cspan id=\"2\"\u003e二、快速上手\u003c/span\u003e\n\u003e 预计只需要 **3分钟** 即可完成 ！\n\n以下操作中会依赖Docker，所以请先安装并启动它。如果只是试用则强烈建议你全程使用\u003ca href=\"https://labs.play-with-docker.com/\"\u003e在线Docker网站\u003c/a\u003e，按如下步骤安装即可，非常方便。\n\n### \u003cspan id=\"21\"\u003e1、获取项目\u003c/span\u003e\n通过```git clone```或下载Release包即可获取项目，如果出错请参考[获取过程帮助](doc/HELP.md#12)文档。\n\n```bash\ngit clone https://github.com/WGrape/esupdater\ncd esupdater\n```\n\n### \u003cspan id=\"22\"\u003e2、开始安装\u003c/span\u003e\n执行```install```目录下的```install.sh```安装脚本时，需要传递如下参数以实现[设置环境变量](./doc/APPLICATION.md#3)。如果出错请参考[安装过程帮助](doc/HELP.md#13)文档。\n\n- ```your_local_ip``` ：本机IP参数，通过```ifconfig```查看，通常为192.168开头，而不是127.0.0.1\n\n```bash\ncd install\nbash install.sh ${your_local_ip}\ncd ..\n```\n\n### \u003cspan id=\"24\"\u003e3、运行项目\u003c/span\u003e\n安装成功后，执行根目录下的```start.sh```启动脚本即可。如果出错请参考[运行过程帮助](doc/HELP.md#3)文档。\n\n```bash\nbash start.sh\n\n# 查看日志输出\ntail -f /home/log/esupdater/debug.log.20220111\n```\n\n### \u003cspan id=\"25\"\u003e4、测试运行\u003c/span\u003e\n在另一个窗口进入```kafkaContainer```容器中，按如下操作启动```Kafka生产者```\n\n```bash\ndocker exec -it kafkaContainer /bin/bash\ncd /opt/kafka/\n\n# 启动时可能会出现warn, 忽略即可\n./bin/kafka-console-producer.sh --broker-list localhost:9092 --topic default_topic\n```\n\n\u003cimg width=\"843\" alt=\"img1\" src=\"https://user-images.githubusercontent.com/35942268/148804272-b00483a9-3861-4aab-8b2f-aee963784694.png\"\u003e\n\n启动成功后会进入一个生产消息的命令行，发送任意消息后，查看上一步日志中的输出，如果出现如下类似日志则说明服务已经成功运行 ！\n\n\u003cimg width=\"823\" alt=\"img2\" src=\"https://user-images.githubusercontent.com/35942268/148806227-25af15b9-5609-4de3-ac13-96fc83c7c99b.png\"\u003e\n\n## \u003cspan id=\"3\"\u003e三、业务接入\u003c/span\u003e\n如果需要在你的业务中接入此项目，请参考[应用接入文档](./doc/APPLICATION.md)。\n\n## \u003cspan id=\"4\"\u003e四、扩展列表\u003c/span\u003e\n基于插件化扩展开发，项目提供了一系列开箱即用的扩展。\n\n### 1、AutoGenerateCallback\n一个自动生成```Handler```和```Service```的事件回调模块的扩展。具体使用见[使用介绍](./plugin/autogeneratecallback/README.md)\n\n\n## \u003cspan id=\"5\"\u003e五、关于项目\u003c/span\u003e\n\n### \u003cspan id=\"51\"\u003e1、深入了解\u003c/span\u003e\n如果想要深入了解本项目，在 [doc目录](./doc) 下提供了如下丰富完善的项目文档，欢迎阅读。\n\n- [APPLICATION](doc/APPLICATION.md) ：帮助你快速在业务中接入此项目\n- [HOWTOCODE](doc/HOWTOCODE.md) ：更深的了解项目，包括架构设计、底层原理\n- [HELP](doc/HELP.md) ：解决安装和部署过程中问题的帮助手册，包括镜像制作帮助、容器部署帮助等\n\n### \u003cspan id=\"52\"\u003e2、参与项目\u003c/span\u003e\n项目源码设计简单易懂，如有更好的想法，可参考[如何贡献](doc/CONTRIBUTING.md)文档，期待提出宝贵的 [Pull request](https://github.com/WGrape/esupdater/pulls)  。\n\n如果在了解和使用过程中，有任何疑问，也欢迎提出宝贵的 [Issue](https://github.com/WGrape/esupdater/issues/new) 。\n\n开源不易，如果支持本项目 **欢迎Star ！** 以激励维护和更新的动力。\n\n## \u003cspan id=\"6\"\u003e六、贡献列表\u003c/span\u003e\n所有对本项目有过重要贡献的用户，会收录在此贡献者列表中。\n\n- 感谢 [sick-cat](https://github.com/sick-cat) 提出的Issue ：[启动配置](https://github.com/WGrape/esupdater/issues/41)\n- 感谢 [onser3](https://github.com/onser3) 提出的Issue ：[自动生成handler和service层](https://github.com/WGrape/esupdater/issues/44)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwgrape%2Fesupdater","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwgrape%2Fesupdater","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwgrape%2Fesupdater/lists"}