Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/streamnative/logstash-input-pulsar
https://github.com/streamnative/logstash-input-pulsar
logstash-input logstash-plugin pulsar-io
Last synced: about 1 month ago
JSON representation
- Host: GitHub
- URL: https://github.com/streamnative/logstash-input-pulsar
- Owner: streamnative
- License: apache-2.0
- Created: 2021-06-26T01:51:57.000Z (over 3 years ago)
- Default Branch: master
- Last Pushed: 2022-12-14T16:37:14.000Z (about 2 years ago)
- Last Synced: 2024-04-17T17:13:13.683Z (9 months ago)
- Topics: logstash-input, logstash-plugin, pulsar-io
- Language: Java
- Homepage:
- Size: 79.1 KB
- Stars: 12
- Watchers: 15
- Forks: 17
- Open Issues: 8
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
README
# Logstash Input Pulsar Plugin
This is a Java plugin for [Logstash](https://github.com/elastic/logstash).
It is fully free and fully open source. The license is Apache 2.0, meaning you are free to use it however you want.
This input will read events from a Pulsar topic.
This plugin uses Pulsar Client 2.10.2. For broker compatibility, see the official Pulsar compatibility reference. If the compatibility wiki is not up-to-date, please contact Pulsar support/community to confirm compatibility.
If you require features not yet available in this plugin (including client version upgrades), please file an issue with details about what you need.
# Pulsar Input Configuration Options
This plugin supports these configuration options.
| Settings | Input type | Required |
|----------------------------------|:------------------------------------------------------------:|----------:|
| serviceUrl | string | Yes |
| topics | array | Yes |
| subscriptionName | string | Yes |
| consumerName | string | Yes |
| subscriptionType | string, one of["Shared","Exclusive","Failover","Key_shared"] | No |
| subscriptionInitialPosition | string, one of["Latest","Earliest"] | No |
| codec | string, one of ["plain","json"] | No |
| protocols | array, ciphers list. default is TLSv1.2 | No |
| ciphers | array,ciphers list | No |
| enable_tls | boolean, one of [true, false]. default is false | No |
| tls_trust_store_path | string, required if enable_tls is set to true | No |
| tls_trust_store_password | string, default is empty | No |
| enable_tls_hostname_verification | boolean, one of [true, false]. default is false | No |
| allow_tls_insecure_connection | boolean, one of [true, false].default is false | No |
| auth_plugin_class_name | string | No |
| auth_params | string | No |
# Example
pulsar without tls & token
```
input{
pulsar{
serviceUrl => "pulsar://127.0.0.1:6650"
codec => "json"
topics => [
"persistent://public/default/topic1",
"persistent://public/default/topic2"
]
subscriptionName => "my_consumer"
subscriptionType => "Shared"
subscriptionInitialPosition => "Earliest"
}
}
```
pulsar with token
```
input{
pulsar{
serviceUrl => "pulsar://127.0.0.1:6650"
codec => "plain"
topics => [
"persistent://public/default/topic1",
"persistent://public/default/topic2"
]
subscriptionName => "my_subscription"
subscriptionType => "Shared"
subscriptionInitialPosition => "Earliest"
auth_plugin_class_name => "org.apache.pulsar.client.impl.auth.AuthenticationToken"
auth_params => "token:${token}"
}
}
```
# Installation
1. Get the latest zip file from release page.
https://github.com/streamnative/logstash-input-pulsar/releases
2. Install this plugin using logstash preoffline command.
```
bin/logstash-plugin install file://{PATH_TO}/logstash-input-pulsar-2.7.1.zip
```