https://github.com/eccenca/cmem-plugin-kafka

This eccenca Corporate Memory plugin allows for sending and receiving messages from Apache Kafka.
https://github.com/eccenca/cmem-plugin-kafka

corporate-memory eccenca kafka kafka-consumer kafka-producer plugin

Last synced: about 1 month ago
JSON representation

This eccenca Corporate Memory plugin allows for sending and receiving messages from Apache Kafka.

• Host: GitHub
• URL: https://github.com/eccenca/cmem-plugin-kafka
• Owner: eccenca
• License: apache-2.0
• Created: 2022-07-21T07:42:38.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2024-04-08T21:45:27.000Z (9 months ago)
• Last Synced: 2024-04-14T11:51:09.153Z (8 months ago)
• Topics: corporate-memory, eccenca, kafka, kafka-consumer, kafka-producer, plugin
• Language: Python
• Homepage:
• Size: 792 KB
• Stars: 0
• Watchers: 12
• Forks: 0
• Open Issues: 5
• Metadata Files:
  • Readme: README-public.md
  • Changelog: CHANGELOG.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# cmem-plugin-kafka

Send and receive messages from Apache Kafka.

This is a plugin for [eccenca](https://eccenca.com) [Corporate Memory](https://documentation.eccenca.com).

You can install it with the [cmemc](https://eccenca.com/go/cmemc) command line

clients like this:

```

cmemc admin workspace python install cmem-plugin-kafka

```

## Plugin supported message format

### XML dataset format

An example XML document is shown below. This document will be sent as two messages

to the configured topic. Each message is created as a proper XML document.

```xml

    

        

            

                string

                string

                string

                string

                9200

            

            

                string

                string

                string

                string

                2381

            

        

    

    

        

            .

            .

            .

        

    

```

Producer plugin generates 2 messages with below content

```xml

    

        string

        string

        string

        string

        9200

    

    

        string

        string

        string

        string

        2381

    

```

```xml

            .

            .

            .

```

### JSON Dataset format

An example JSON document is shown below. This document will be sent as two messages

to the configured topic. Each message is created as a proper JSON document.

```json

[

  {

    "message": {

      "key": "818432-942813-832642-453478",

      "headers": {

        "type": "ADD"

      },

      "content": {

        "location": [

          "Leipzig"

        ],

        "obstacle": {

          "name": "Iron Bars",

          "order": "1"

        }

      }

    }

  },

  {

    "message": {

      "key": "887428-119918-570674-866526",

      "headers": {

        "type": "REMOVE"

      },

      "content": {

        "comments": "We can pass any json payload here."

      }

    }

  }

]

```

Producer plugin generates 2 messages with below content

```json

{

  "location": [

    "Leipzig"

  ],

  "obstacle": {

    "name": "Iron Bars",

    "order": "1"

  }

}

```

```json

{

  "comments": "We can pass any json payload here."

}

```

### Entities format

Random values plugin entities will generate below format JSON document.

```json

{

  "schema": {

    "type_uri": "https://example.org/vocab/RandomValueRow"

  },

  "entity": {

    "uri": "urn:uuid:3c68d8e7-bf17-4045-a9eb-c9c9813f717f",

    "values": {

      "": [

        "a8o4Ocsb6RZClFRUZU3b2w"

      ],

      "": [

        "RTICRU7JcTUVn94decelPg"

      ],

      "": [

        "A9r-969NjAlX0DNWftxKoA"

      ],

      "": [

        "FygWRy1UJ4-IzIim1qukJA"

      ],

      "": [

        "AJcbn-LJEs-Dif96xu2eww"

      ]

    }

  }

}

```