{"id":21296714,"url":"https://github.com/eccenca/cmem-plugin-kafka","last_synced_at":"2025-03-15T17:26:34.140Z","repository":{"id":57935235,"uuid":"516273919","full_name":"eccenca/cmem-plugin-kafka","owner":"eccenca","description":"This eccenca Corporate Memory plugin allows for sending and receiving messages from Apache Kafka.","archived":false,"fork":false,"pushed_at":"2024-04-08T21:45:27.000Z","size":811,"stargazers_count":0,"open_issues_count":5,"forks_count":0,"subscribers_count":12,"default_branch":"main","last_synced_at":"2024-04-14T11:51:09.153Z","etag":null,"topics":["corporate-memory","eccenca","kafka","kafka-consumer","kafka-producer","plugin"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eccenca.png","metadata":{"files":{"readme":"README-public.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2022-07-21T07:42:38.000Z","updated_at":"2024-04-22T22:52:09.026Z","dependencies_parsed_at":"2024-04-22T22:51:56.952Z","dependency_job_id":"a927352d-5fe5-4c92-8b3b-e221c95fee8a","html_url":"https://github.com/eccenca/cmem-plugin-kafka","commit_stats":{"total_commits":165,"total_committers":8,"mean_commits":20.625,"dds":0.7272727272727273,"last_synced_commit":"7551d6087f0c134cef53b3120a949eb513692bf5"},"previous_names":[],"tags_count":20,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eccenca%2Fcmem-plugin-kafka","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eccenca%2Fcmem-plugin-kafka/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eccenca%2Fcmem-plugin-kafka/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eccenca%2Fcmem-plugin-kafka/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eccenca","download_url":"https://codeload.github.com/eccenca/cmem-plugin-kafka/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243764669,"owners_count":20344463,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["corporate-memory","eccenca","kafka","kafka-consumer","kafka-producer","plugin"],"created_at":"2024-11-21T14:29:10.236Z","updated_at":"2025-03-15T17:26:34.101Z","avatar_url":"https://github.com/eccenca.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# cmem-plugin-kafka\n\nSend and receive messages from Apache Kafka.\n\nThis is a plugin for [eccenca](https://eccenca.com) [Corporate Memory](https://documentation.eccenca.com).\n\nYou can install it with the [cmemc](https://eccenca.com/go/cmemc) command line\nclients like this:\n\n```\ncmemc admin workspace python install cmem-plugin-kafka\n```\n\n## Plugin supported message format\n\n### XML dataset format\n\nAn example XML document is shown below. This document will be sent as two messages\nto the configured topic. Each message is created as a proper XML document.\n\n```xml\n\u003c?xml version=\"1.0\" encoding=\"utf-8\"?\u003e\n\u003cKafkaMessages\u003e\n    \u003cMessage\u003e\n        \u003cPurchaseOrder OrderDate=\"1996-04-06\"\u003e\n            \u003cShipTo country=\"string\"\u003e\n                \u003cname\u003estring\u003c/name\u003e\n                \u003cstreet\u003estring\u003c/street\u003e\n                \u003ccity\u003estring\u003c/city\u003e\n                \u003cstate\u003estring\u003c/state\u003e\n                \u003czip\u003e9200\u003c/zip\u003e\n            \u003c/ShipTo\u003e\n            \u003cBillTo country=\"string\"\u003e\n                \u003cname\u003estring\u003c/name\u003e\n                \u003cstreet\u003estring\u003c/street\u003e\n                \u003ccity\u003estring\u003c/city\u003e\n                \u003cstate\u003estring\u003c/state\u003e\n                \u003czip\u003e2381\u003c/zip\u003e\n            \u003c/BillTo\u003e\n        \u003c/PurchaseOrder\u003e\n    \u003c/Message\u003e\n    \u003cMessage key=\"1234\"\u003e\n        \u003cSingleTagHere\u003e\n            .\n            .\n            .\n        \u003c/SingleTagHere\u003e\n    \u003c/Message\u003e\n\u003c/KafkaMessages\u003e\n```\nProducer plugin generates 2 messages with below content\n```xml\n\u003c?xml version=\"1.0\" encoding=\"utf-8\"?\u003e\n\u003cPurchaseOrder OrderDate=\"1996-04-06\"\u003e\n    \u003cShipTo country=\"string\"\u003e\n        \u003cname\u003estring\u003c/name\u003e\n        \u003cstreet\u003estring\u003c/street\u003e\n        \u003ccity\u003estring\u003c/city\u003e\n        \u003cstate\u003estring\u003c/state\u003e\n        \u003czip\u003e9200\u003c/zip\u003e\n    \u003c/ShipTo\u003e\n    \u003cBillTo country=\"string\"\u003e\n        \u003cname\u003estring\u003c/name\u003e\n        \u003cstreet\u003estring\u003c/street\u003e\n        \u003ccity\u003estring\u003c/city\u003e\n        \u003cstate\u003estring\u003c/state\u003e\n        \u003czip\u003e2381\u003c/zip\u003e\n    \u003c/BillTo\u003e\n\u003c/PurchaseOrder\u003e\n```\n```xml\n\u003c?xml version=\"1.0\" encoding=\"utf-8\"?\u003e\n\u003cSingleTagHere\u003e\n            .\n            .\n            .\n\u003c/SingleTagHere\u003e\n```\n### JSON Dataset format\n\nAn example JSON document is shown below. This document will be sent as two messages\nto the configured topic. Each message is created as a proper JSON document.\n\n```json\n[\n  {\n    \"message\": {\n      \"key\": \"818432-942813-832642-453478\",\n      \"headers\": {\n        \"type\": \"ADD\"\n      },\n      \"content\": {\n        \"location\": [\n          \"Leipzig\"\n        ],\n        \"obstacle\": {\n          \"name\": \"Iron Bars\",\n          \"order\": \"1\"\n        }\n      }\n    }\n  },\n  {\n    \"message\": {\n      \"key\": \"887428-119918-570674-866526\",\n      \"headers\": {\n        \"type\": \"REMOVE\"\n      },\n      \"content\": {\n        \"comments\": \"We can pass any json payload here.\"\n      }\n    }\n  }\n]\n```\nProducer plugin generates 2 messages with below content\n```json\n{\n  \"location\": [\n    \"Leipzig\"\n  ],\n  \"obstacle\": {\n    \"name\": \"Iron Bars\",\n    \"order\": \"1\"\n  }\n}\n```\n```json\n{\n  \"comments\": \"We can pass any json payload here.\"\n}\n```\n### Entities format\n\nRandom values plugin entities will generate below format JSON document.\n\n```json\n{\n  \"schema\": {\n    \"type_uri\": \"https://example.org/vocab/RandomValueRow\"\n  },\n  \"entity\": {\n    \"uri\": \"urn:uuid:3c68d8e7-bf17-4045-a9eb-c9c9813f717f\",\n    \"values\": {\n      \"\u003chttps://example.org/vocab/RandomValuePath0\u003e\": [\n        \"a8o4Ocsb6RZClFRUZU3b2w\"\n      ],\n      \"\u003chttps://example.org/vocab/RandomValuePath1\u003e\": [\n        \"RTICRU7JcTUVn94decelPg\"\n      ],\n      \"\u003chttps://example.org/vocab/RandomValuePath2\u003e\": [\n        \"A9r-969NjAlX0DNWftxKoA\"\n      ],\n      \"\u003chttps://example.org/vocab/RandomValuePath3\u003e\": [\n        \"FygWRy1UJ4-IzIim1qukJA\"\n      ],\n      \"\u003chttps://example.org/vocab/RandomValuePath4\u003e\": [\n        \"AJcbn-LJEs-Dif96xu2eww\"\n      ]\n    }\n  }\n}\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feccenca%2Fcmem-plugin-kafka","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feccenca%2Fcmem-plugin-kafka","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feccenca%2Fcmem-plugin-kafka/lists"}