{"id":24044968,"url":"https://github.com/oun/kafka-event-router","last_synced_at":"2026-02-14T13:02:27.129Z","repository":{"id":45527810,"uuid":"188865092","full_name":"oun/kafka-event-router","owner":"oun","description":"Kafka outbox event router for MongoDB","archived":false,"fork":false,"pushed_at":"2022-11-16T12:28:48.000Z","size":62,"stargazers_count":4,"open_issues_count":3,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-07-31T01:58:24.449Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oun.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-05-27T15:07:13.000Z","updated_at":"2023-12-29T14:06:25.000Z","dependencies_parsed_at":"2023-01-21T18:40:00.478Z","dependency_job_id":null,"html_url":"https://github.com/oun/kafka-event-router","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/oun/kafka-event-router","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oun%2Fkafka-event-router","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oun%2Fkafka-event-router/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oun%2Fkafka-event-router/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oun%2Fkafka-event-router/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oun","download_url":"https://codeload.github.com/oun/kafka-event-router/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oun%2Fkafka-event-router/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29444047,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-14T12:43:28.304Z","status":"ssl_error","status_checked_at":"2026-02-14T12:43:14.160Z","response_time":53,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-08T23:51:40.903Z","updated_at":"2026-02-14T13:02:27.110Z","avatar_url":"https://github.com/oun.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Event Router\nKafka event router for mongodb debezium connector (https://debezium.io/blog/2019/02/19/reliable-microservices-data-exchange-with-the-outbox-pattern/).\n\n### MongoDB Event Router\n\nEvent collection schema:\n\n| Field |  Type  |   Description   | \n|-------|--------|-----------------|\n| _id   | string/object id | Event id        |\n| type  | string | Event type      |\n| aggregateId  | string | Aggregate id      |\n| aggregateType  | string | Aggregate type     |\n| payload  | string | Actual event json string |\n\nExample Event document:\n```json\n{\n    \"_id\" : \"1c65b115-1124-42ed-bca1-c6a80b29f1dd\",\n    \"type\" : \"UserCreated\",\n    \"aggregateId\" : \"cdceb9bd-5065-4c58-9333-119dee03eeb5\",\n    \"aggregateType\" : \"User\",\n    \"payload\" : \"{\\\"id\\\":\\\"cdceb9bd-5065-4c58-9333-119dee03eeb5\\\",\\\"username\\\":\\\"oun\\\",\\\"aggregateId\\\":\\\"cdceb9bd-5065-4c58-9333-119dee03eeb5\\\",\\\"aggregateType\\\":\\\"User\\\"}\"\n}\n```\n\nWe would need the Debezium MongoDB connector to monitor change in this event collection and send it to Kafka topic.\nWe also need to re-route message to different topic by using aggregate type. This will ensure that all event of the same type will go into same Kafka topic.\nThe payload must be json string and will be deserialized to the message. The message key is aggregate id. \nThe type field will be put into message header, so consumer can use it to determine the class of target type for deserialization.\nFor example we use Spring Kafka JsonDeserializer with TYPE_MAPPINGS config to easily deserialize into target class.\n\n\n### Setup Confluent Kafka\n\nFollow instruction on \nhttps://docs.confluent.io/current/quickstart/ce-docker-quickstart.html\n\n```\ncd examples/cp-all-in-one\ndocker-compose -f docker-compose.yml up\n```\n\n### Setup Kafka Connector and custom SMT\n\n#### Install Debezium mongodb connector\n\nDownload connector plugin\n```\ndocker exec -it connect bash\nwget https://repo1.maven.org/maven2/io/debezium/debezium-connector-mongodb/0.9.3.Final/debezium-connector-mongodb-0.9.3.Final-plugin.tar.gz\n```\n\nExtract plugin and move to plugin directory in kafka connector container\n```\ntar -zxvf debezium-connector-mongodb-0.9.3.Final-plugin.tar.gz\nmv debezium-connector-mongodb /usr/share/java\n```\n\nConnect connector container to mongodb cluster network\n```\ndocker network connect mongo-cluster connect\n```\n\n#### Install event router\n\nBuild mongodb-event-router jar\n```\ncd mongodb-event-router\ngradle clean build\n```\n\nCreate a directory for custom smt in plugin directory\n```\ndocker exec -it connect bash\ncd /usr/share/java\nmkdir kafka-event-router\n```\n\nCopy mongodb-event-router jar to the directory in the connector container\n```\ndocker cp build/libs/mongodb-event-router-0.0.1-SNAPSHOT.jar connect:/usr/share/java/kafka-event-router\n```\n\nRestart kafka connector\n```\ndocker restart connect\n```\n\nCreate Debezium mongodb connector\n```\ncurl -X POST \\\n  http://localhost:8083/connectors \\\n  -H 'Content-Type: application/json' \\\n  -H 'cache-control: no-cache' \\\n  -d '{\n          \"name\": \"mongodb-outbox-event\",\n          \"config\": {\n              \"connector.class\": \"io.debezium.connector.mongodb.MongoDbConnector\",\n              \"tasks.max\": \"1\",\n              \"initial.sync.max.threads\": \"1\",\n              \"mongodb.hosts\": \"rs0/mongo1:27017\",\n              \"mongodb.name\": \"mongodb\",\n              \"offset.flush.interval.ms\": \"3000\",\n              \"database.whitelist\": \"users\",\n              \"collection.whitelist\": \"users[.]outbox_events\",\n              \"transforms\": \"router\",\n              \"transforms.router.type\": \"io.github.oun.kafka.transform.outbox.MongoEventRouter\",\n              \"tombstones.on.delete\": false\n          }\n      }'\n```\n\nReference\nhttps://debezium.io/docs/install/","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foun%2Fkafka-event-router","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foun%2Fkafka-event-router","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foun%2Fkafka-event-router/lists"}