{"id":13696194,"url":"https://github.com/larsnovikov/horgh-replicator","last_synced_at":"2026-01-12T09:44:36.409Z","repository":{"id":64304923,"uuid":"177077853","full_name":"larsnovikov/horgh-replicator","owner":"larsnovikov","description":"Golang binlog replication from MySQL to MySQL, PostgreSQL, Vertica, Clickhouse","archived":false,"fork":false,"pushed_at":"2020-02-12T06:37:36.000Z","size":34891,"stargazers_count":55,"open_issues_count":2,"forks_count":26,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-05-03T16:36:21.455Z","etag":null,"topics":["binlog","clickhouse","go","golang","master-slave","master-slave-replication","mysql","postgresql","replication","tungsten","vertica"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/larsnovikov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-03-22T05:37:27.000Z","updated_at":"2024-10-16T08:40:21.000Z","dependencies_parsed_at":"2023-01-15T10:15:37.344Z","dependency_job_id":null,"html_url":"https://github.com/larsnovikov/horgh-replicator","commit_stats":null,"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"purl":"pkg:github/larsnovikov/horgh-replicator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/larsnovikov%2Fhorgh-replicator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/larsnovikov%2Fhorgh-replicator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/larsnovikov%2Fhorgh-replicator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/larsnovikov%2Fhorgh-replicator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/larsnovikov","download_url":"https://codeload.github.com/larsnovikov/horgh-replicator/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/larsnovikov%2Fhorgh-replicator/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28337755,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-12T06:09:07.588Z","status":"ssl_error","status_checked_at":"2026-01-12T06:05:18.301Z","response_time":98,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["binlog","clickhouse","go","golang","master-slave","master-slave-replication","mysql","postgresql","replication","tungsten","vertica"],"created_at":"2024-08-02T18:00:37.205Z","updated_at":"2026-01-12T09:44:36.404Z","avatar_url":"https://github.com/larsnovikov.png","language":"Go","funding_links":[],"categories":["Integrations"],"sub_categories":["ETL and Data Processing"],"readme":"# Mysql binlog replicator\n\n##### Based on [JackShadow/go-binlog-example](https://github.com/JackShadow/go-binlog-example) \n\n### Site \n\nSee full docs [here](https://larsnovikov.github.io/horgh) \n\n### Master types\n- MySQL\n\n### Slave types\n- MySQL\n- PostgreSQL\n- Yandex ClickHouse\n- HP Vertica\n\n### Quick start\n\nSee quick start tutorial [here](https://larsnovikov.github.io/horgh#quick_start) \n\n### Testing\n- Copy `src/.env.dist` to `src/.env` and set credentials.\n- Configure your my MySQL master as `examples/master/mysql.conf`. \nDon't forget to set `binlog_do_db=\u003cmaster_db_name\u003e` and restart MySQL service.\n- Create databases and tables as `examples/sql/`.\n- Start Docker as `make start-dev`\n- Run as `cd src` and `go run main.go listen` in docker container.\n- Copy `examples/configs/user.json` and `examples/configs/post.json` to `src/system/configs`\n- Execute `cd src` and `go run main.go load`\n\n### Add tables to replicator\n\n- Use `create-model \u003ctable\u003e` to create json config for your table.\n- Create table on slave.\n- Set `\u003ctable\u003e` to `TABLE` param in `src/.env` \n- Use `build-slave` to copy table data from master and set start position of log for table listener.\n\n### Custom handlers for field value\n\n- Create `plugins/user/\u003cplugin_name\u003e/handler.go` like `plugins/system/set_value/handler.go`\n- Execute `go build -buildmode=plugin -o plugins/user/\u003cplugin_name\u003e/handler.so plugins/user/\u003cplugin_name\u003e/handler.go`\n- Add to field description in your `src/system/configs/\u003cmodel\u003e.json`\n\n```\n\"beforeSave\": {\n  \"handler\": \"user/\u003cplugin_name\u003e\",\n  \"params\": [\n    \"***\"\n  ]\n}\n```\n\n##### System handlers\n\n- If you want to set custom field value use `system/set_value` as `handler` param. Don't forget to set `params: [\"\u003cvalue\u003e\"]`\n\n### Tools\n\n- `set-position \u003ctable\u003e \u003cbinlog_name\u003e \u003cbinlog_position\u003e` set start position of log for table listener\n- `load` start loader for replication testing (for default tables user and post)\n- `create-model \u003ctable\u003e` create model json-file by master table structure\n- `build-slave` create master table dump, restore this dump in slave, set start position of log for table listener\n- `destroy-slave` truncate table, set empty position of log for table listener\n\n### Modes\n\n- Prod mode: build app and execute listener. \n  \n  Use `make build-prod` and `make start-prod` to start and `make stop-prod` to stop.\n- Dev mode: provides the opportunity for manual start and debug. \n  \n  Use `make start-dev` to start and `make stop-dev` to stop.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flarsnovikov%2Fhorgh-replicator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flarsnovikov%2Fhorgh-replicator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flarsnovikov%2Fhorgh-replicator/lists"}