Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/jcustenborder/kafka-connect-transform-xml
Transformation for converting XML data to Structured data.
https://github.com/jcustenborder/kafka-connect-transform-xml
kafka-connect transformation xml
Last synced: 3 months ago
JSON representation
Transformation for converting XML data to Structured data.
- Host: GitHub
- URL: https://github.com/jcustenborder/kafka-connect-transform-xml
- Owner: jcustenborder
- License: apache-2.0
- Created: 2018-03-23T20:52:06.000Z (almost 7 years ago)
- Default Branch: master
- Last Pushed: 2022-07-26T12:45:05.000Z (over 2 years ago)
- Last Synced: 2024-10-13T05:29:58.194Z (3 months ago)
- Topics: kafka-connect, transformation, xml
- Language: Java
- Size: 46.9 KB
- Stars: 22
- Watchers: 2
- Forks: 44
- Open Issues: 21
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Introduction
This project provides transformations for Kafka Connect that will convert XML text to a Kafka Connect
struct based on the configured XML schema. This transformation works by dynamically generating
JAXB source with [XJC](https://docs.oracle.com/javase/8/docs/technotes/tools/unix/xjc.html) with the
[xjc-kafka-connect-plugin](https://github.com/jcustenborder/xjc-kafka-connect-plugin) loaded. This
allows the transformation to efficiently convert XML to structured data for Kafka connect.
Use it in conjunction with a Source connector that reads XML data, such as from a [HTTP REST endpoint](https://www.confluent.io/hub/castorm/kafka-connect-http).
# Transformations
## FromXML(Key)
This transformation is used to transform XML in the Value of the input into a JSON struct based on the provided XSD.
### Configuration
| Name | Type | Importance | Default Value | Validator | Documentation |
| ------------ | ------ | ---------- | ------------- | --------- | --------------------------------------------------------------- |
| schema.path | List | High | | | Urls to the schemas to load. http and https paths are supported |
| xjc.options.automatic.name.conflict.resolution.enabled| Boolean | | False |||
| xjc.options.strict.check.enabled | Boolean | | True |||
| xjc.options.verbose.enabled | Boolean | | False |||
#### Standalone Example
```properties
transforms=xml_key
transforms.xml_key.type=com.github.jcustenborder.kafka.connect.transform.xml.FromXml$Key
# The following values must be configured.
transforms.xml_key.schema.path = http://web.address/my.xsd
```
#### Distributed Example
```json
"transforms": "xml_key",
"transforms.xml_key.type": "com.github.jcustenborder.kafka.connect.transform.xml.FromXml$Key",
"transforms.xml_key.schema.path": "http://web.address/my.xsd"
```
## FromXML(Value)
This transformation is used to transform XML in the Value of the input into a JSON struct based on the provided XSD.
### Configuration
| Name | Type | Importance | Default Value | Validator | Documentation |
| ------------ | ------ | ---------- | ------------- | --------- | --------------------------------------------------------------- |
| schema.path | List | High | | | Urls to the schemas to load. http and https paths are supported |
| xjc.options.automatic.name.conflict.resolution.enabled| Boolean | | False |||
| xjc.options.strict.check.enabled | Boolean | | True |||
| xjc.options.verbose.enabled | Boolean | | False |||
#### Standalone Example
```properties
transforms=xml_value
transforms.xml_value.type=com.github.jcustenborder.kafka.connect.transform.xml.FromXml$Value
# The following values must be configured.
transforms.xml_value.schema.path = http://web.address/my.xsd
```
#### Distributed Example
```json
"transforms": "xml_value",
"transforms.xml_value.type": "com.github.jcustenborder.kafka.connect.transform.xml.FromXml$Value",
"transforms.xml_value.schema.path": "< Configure me >"
```