https://github.com/streamnative/logstash-output-pulsar
Logstash output plugin for pulsar
https://github.com/streamnative/logstash-output-pulsar
Last synced: 7 months ago
JSON representation
Logstash output plugin for pulsar
- Host: GitHub
- URL: https://github.com/streamnative/logstash-output-pulsar
- Owner: streamnative
- License: apache-2.0
- Created: 2021-06-26T02:33:15.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2023-06-08T10:08:06.000Z (over 2 years ago)
- Last Synced: 2024-04-17T17:13:14.014Z (almost 2 years ago)
- Language: Java
- Size: 82 KB
- Stars: 8
- Watchers: 8
- Forks: 13
- Open Issues: 11
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# Logstash Output Pulsar Plugin
This is a Java plugin for [Logstash](https://github.com/elastic/logstash).
It is fully free and fully open source. The license is Apache 2.0, meaning you are free to use it however you want.
Write events to a pulsar topic.
This plugin uses Pulsar Client 2.9.0. For broker compatibility, see the official Pulsar compatibility reference. If the compatibility wiki is not up-to-date, please contact Pulsar support/community to confirm compatibility.
If you require features not yet available in this plugin (including client version upgrades), please file an issue with details about what you need.
# Pulsar Output Configuration Options
This plugin supports these configuration options.
| Settings | Output type | Required |
|-----------------------------------|:-----------------------------------------------------------------------------:|-----------:|
| serviceUrl | string | No |
| topic | string | Yes |
| producer_name | string | Yes |
| compression_type | string, one of["NONE","LZ4","ZLIB","ZSTD","SNAPPY"] | No |
| block_if_queue_full | bool, default is true | No |
| enable_batching | bool, default is true | No |
| enable_tls | boolean, one of [true, false]. default is false | No |
| tls_trust_store_path | string, required if enable_tls is set to true | No |
| tls_trust_store_password | string, default is empty | No |
| enable_tls_hostname_verification | boolean, one of [true, false]. default is false | No |
| protocols | array, ciphers list. default is TLSv1.2 | No |
| allow_tls_insecure_connection | boolean, one of [true, false].default is false | No |
| auth_plugin_class_name | string | No |
| ciphers | array, ciphers list | No |
| enable_token | boolean, one of [true, false]. default is false | No |
| auth_plugin_params_String | string | No |
# Example
pulsar without tls & token
```
output{
pulsar{
serviceUrl => "pulsar://127.0.0.1:6650"
topic => "persistent://public/default/%{topic_name}"
producer_name => "%{producer_name}"
enable_batching => true
}
}
```
pulsar with token
```
output {
pulsar{
serviceUrl => "pulsar://localhost:6650"
topic => "persistent://public/default/%{topic_name}"
producer_name => "%{producer_name}"
enable_batching => true
enable_token => true
auth_plugin_class_name => "org.apache.pulsar.client.impl.auth.AuthenticationToken"
auth_plugin_params_String => "token:%{token}"
}
}
```
# Installation
1. Get the latest zip file from release page.
https://github.com/streamnative/logstash-output-pulsar/releases
2. Install this plugin using logstash preoffline command.
```
bin/logstash-plugin install file://{PATH_TO}/logstash-output-pulsar-2.10.0.0.zip
```