{"id":21258148,"url":"https://github.com/cuongndc9/debezium-example","last_synced_at":"2025-05-12T14:08:09.817Z","repository":{"id":116044487,"uuid":"345858201","full_name":"cuongndc9/debezium-example","owner":"cuongndc9","description":"Debezium Example | Postgres ➡️ Elasticsearch","archived":false,"fork":false,"pushed_at":"2025-03-07T19:26:35.000Z","size":26885,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-07T20:28:50.485Z","etag":null,"topics":["4pet","debezium","elasticsearch","kafka","kafka-streams","postgresql"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cuongndc9.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2021-03-09T02:27:27.000Z","updated_at":"2025-03-07T19:26:38.000Z","dependencies_parsed_at":"2024-03-05T09:09:45.913Z","dependency_job_id":null,"html_url":"https://github.com/cuongndc9/debezium-example","commit_stats":null,"previous_names":["cuonginspire/debezium-example","cuongnd9/debezium-example","cuongndc9/debezium-example"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cuongndc9%2Fdebezium-example","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cuongndc9%2Fdebezium-example/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cuongndc9%2Fdebezium-example/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cuongndc9%2Fdebezium-example/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cuongndc9","download_url":"https://codeload.github.com/cuongndc9/debezium-example/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243691400,"owners_count":20331963,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["4pet","debezium","elasticsearch","kafka","kafka-streams","postgresql"],"created_at":"2024-11-21T04:07:19.095Z","updated_at":"2025-05-12T14:08:09.797Z","avatar_url":"https://github.com/cuongndc9.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# debezium-example\n\nDebezium Example | Postgres ➡️ Elasticsearch\n\n### Schema\n\n```\n                   +-------------+\n                   |             |\n                   |  PostgreSQL |\n                   |             |\n                   +------+------+\n                          |\n                          |\n                          |\n          +---------------v------------------+\n          |                                  |\n          |           Kafka Connect          |\n          |    (Debezium, ES connectors)     |\n          |                                  |\n          +---------------+------------------+\n                          |\n                          |\n                          |\n                          |\n                  +-------v--------+\n                  |                |\n                  | Elasticsearch  |\n                  |                |\n                  +----------------+\n\n\n```\nWe are using Docker Compose to deploy the following components:\n\n* PostgreSQL\n* Kafka\n  * ZooKeeper\n  * Kafka Broker\n  * Kafka Connect with [Debezium](http://debezium.io/) and [Elasticsearch](https://github.com/confluentinc/kafka-connect-elasticsearch) Connectors\n* Elasticsearch\n\n## Install Java 11 \u0026 Maven\n\n- https://jdk.java.net/\n- https://maven.apache.org/\n\n## Kafka Streams Foreign Key Joins\n\nThis example demonstrates how two Debezium change data topics can be joined via Kafka Streams,\nusing the new foreign key join feature in Apache Kafka 2.4 ([KIP-213](https://cwiki.apache.org/confluence/display/KAFKA/KIP-213+Support+non-key+joining+in+KTable)).\n\nThe source database contains two tables, `customers` and `addresses`, with a foreign key relationship from the latter to the former,\ni.e. a customer can have multiple addresses.\n\n![4](./images/4.png)\n\n```sh\ncd aggregator-service\nmvn clean package\n```\n\n## Run docker-compose.yml file\n\n```sh\n./scripts/start.sh\n```\n\n## Open http://127.0.0.1:9021/\n\n![control center](./images/1.png)\n \n## Deploy configs\n\n![configs](./images/2.gif)\n\n## Install ElasticSearch Head \u0026 query data\n\nhttps://chrome.google.com/webstore/detail/elasticsearch-head/ffmkiejjmecolpfloofpjologoblkegm\n\n![es-head](./images/3.gif)\n\n## Documents\n\n- [Debezium Tutorial](https://debezium.io/documentation/reference/tutorial.html)\n- [Confluent Hub](https://www.confluent.io/hub/)\n- [KAFKA STREAMS](https://kafka.apache.org/documentation/streams/)\n\n## License\n\nhttps://github.com/4pet-social\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcuongndc9%2Fdebezium-example","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcuongndc9%2Fdebezium-example","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcuongndc9%2Fdebezium-example/lists"}