Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/joker1007/fluent-plugin-avro_turf
Fluentd formatter plugin to format by confluent schema registry format.
https://github.com/joker1007/fluent-plugin-avro_turf
avro fluentd formatter-plugin
Last synced: 2 days ago
JSON representation
Fluentd formatter plugin to format by confluent schema registry format.
- Host: GitHub
- URL: https://github.com/joker1007/fluent-plugin-avro_turf
- Owner: joker1007
- License: apache-2.0
- Created: 2019-08-24T11:21:46.000Z (about 5 years ago)
- Default Branch: master
- Last Pushed: 2023-04-11T21:02:35.000Z (over 1 year ago)
- Last Synced: 2024-04-25T20:20:27.175Z (7 months ago)
- Topics: avro, fluentd, formatter-plugin
- Language: Ruby
- Homepage:
- Size: 16.6 KB
- Stars: 2
- Watchers: 4
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# fluent-plugin-avro_turf
[Fluentd](https://fluentd.org/) formatter plugin to format by confluent schema registry format.
see. https://docs.confluent.io/current/schema-registry/serializer-formatter.html
The format is based on Apache Avro, but it is not compatible.
Representative use is formatter for https://github.com/fluent/fluent-plugin-kafka
## Installation
### RubyGems
```
$ gem install fluent-plugin-avro_turf
```
### Bundler
Add following line to your Gemfile:
```ruby
gem "fluent-plugin-avro_turf"
```
And then execute:
```
$ bundle
```
## Configuration
Example.
```
@type avro_turf_messaging
schema_registry_url http://my-registry:8081/
schemas_path ./schemas
schema_name_key schema_name
default_namespace io.github.joker1007
exclude_schema_name_key true
```
| param | type | required | default | description |
| ------------------- | ------ | -------- | ------- | -------------------------------------------------------------- |
| schema_registry_url | string | true | | Schema Registry URL parameter (ex: `http://my-registry:8081/`) |
| schemas_path | string | | nil | Schemas path to find avsc from local storage |
| default_schema_name | string | | nil | Default schema name when the record doesn't have schema_name_key |
| schema_name_key | string | | `"schema_name"` | Field for schema name |
| schema | hash | | nil | Inline schema definition. If this parameter is set, `default_schema_name` and `schema_name_key` are ignored |
| subject | string | | nil | Set subject explicitly |
| subject_key | string | | nil | Field for subject |
| default_namespace | string | | nil | Default schema namespace |
| namespace_key | string | | `"namespace"` | Field for namespace |
| schema_version | string | | `"latest"` | Set schema version explicitly that is only effective when subject or subject_key is set |
| schema_version_key | string | | `"schema_version"` | Field for schema version |
| exclude_schema_name_key | bool | | false | Set true to remove schema_name_key field from data |
| exclude_namespace_key | bool | | false | Set true to remove namespace_key field from data |
| exclude_schema_version_key | bool | | false | Set true to remove schema_version_key field from data |
| exclude_subject_key | bool | | false | Set true to remove subject_key field from data |
## Copyright
* Copyright(c) 2019- joker1007
* License
* Apache License, Version 2.0