https://github.com/gjbae1212/fluent-bit-pubsub
FluentBit output plugin for google pubsub.
https://github.com/gjbae1212/fluent-bit-pubsub
fluent-bit fluentd golang google-pubsub
Last synced: about 1 year ago
JSON representation
FluentBit output plugin for google pubsub.
- Host: GitHub
- URL: https://github.com/gjbae1212/fluent-bit-pubsub
- Owner: gjbae1212
- License: mit
- Created: 2019-04-12T08:38:45.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2024-07-12T04:39:45.000Z (almost 2 years ago)
- Last Synced: 2025-03-24T04:50:53.313Z (over 1 year ago)
- Topics: fluent-bit, fluentd, golang, google-pubsub
- Language: Go
- Homepage:
- Size: 33.2 MB
- Stars: 7
- Watchers: 2
- Forks: 19
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# fluent-bit output plugin for google pubsub
This plugin is used to publish data to queue in google pubsub.
You could easily use it.
## Build
A bin directory already has been made binaries for mac, linux.
If you should directly make binaries for mac, linux
```bash
# local machine binary
$ bash make.sh build
# Your machine is mac, and if you should do to retry cross compiling for linux.
# A command in below is required a docker.
$ bash make.sh build_linux
```
## Usage
### configuration options for fluent-bit.conf
| Key | Description | Default |
| ----------------|------------------------------------------------|----------------|
| Project | google cloud project id | NONE(required) |
| Topic | google pubsub topic name | NONE(required) |
| JwtPath | jwt file path for accessible google cloud project | NONE(required) |
| Debug | print debug log | false(optional) |
| Timeout | the maximum time that the client will attempt to publish a bundle of messages. (millsecond) | 60000 (optional)|
| DelayThreshold | publish a non-empty batch after this delay has passed. (millsecond) | 1 |
| ByteThreshold | publish a batch when its size in bytes reaches this value. | 1000000 |
| CountThreshold | publish a batch when it has been reached count of messages. | 100 |
### Example fluent-bit.conf
```conf
[Output]
Name pubsub
Match *
Project your-project(custom)
Topic your-topic-name(custom)
Jwtpath your-jwtpath(custom)
```
### Example exec
```bash
$ fluent-bit -c [your config file] -e pubsub.so
```
