https://github.com/iamazeem/fluent-plugin-json
Fluentd filter plugin for JSON events with JSON Pointer Support (RFC-6901)
https://github.com/iamazeem/fluent-plugin-json
fluentd fluentd-filter-plugin fluentd-plugin hacktoberfest json json-pointer
Last synced: 7 months ago
JSON representation
Fluentd filter plugin for JSON events with JSON Pointer Support (RFC-6901)
- Host: GitHub
- URL: https://github.com/iamazeem/fluent-plugin-json
- Owner: iamazeem
- License: apache-2.0
- Created: 2020-07-23T03:11:48.000Z (over 5 years ago)
- Default Branch: main
- Last Pushed: 2024-09-19T09:23:30.000Z (over 1 year ago)
- Last Synced: 2025-04-03T18:05:28.269Z (11 months ago)
- Topics: fluentd, fluentd-filter-plugin, fluentd-plugin, hacktoberfest, json, json-pointer
- Language: Ruby
- Homepage: https://github.com/iamAzeem/fluent-plugin-json
- Size: 38.1 KB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# fluent-plugin-json
[](https://github.com/iamazeem/fluent-plugin-json/actions/workflows/ci.yml)
[](https://github.com/iamAzeem/fluent-plugin-json/blob/master/LICENSE)
[](https://rubygems.org/gems/fluent-plugin-json)
## Overview
[Fluentd](https://fluentd.org/) filter plugin for JSON with JSON pointer support
([RFC-6901](https://tools.ietf.org/html/rfc6901)).
## Installation
### RubyGems
```shell
gem install fluent-plugin-json
```
### Bundler
Add the following line to your Gemfile:
```ruby
gem 'fluent-plugin-json'
```
And then execute:
```shell
bundle
```
## Configuration
### `` section (required) (multiple)
* `pointer` (string) (required): The JSON pointer to an element.
* `pattern` (regexp) (required): The regular expression to match the element.
The configuration may consist of one or more checks. Each check contains a
`pointer` to a JSON element and its corresponding `pattern` (regex) to test it.
The checks are evaluated sequentially. The failure of a single check results in
the rejection of the event. A rejected event is not routed for further
processing.
**NOTE**: The JSON element pointed to by the `pointer` is always converted to a
string for testing with the `pattern` (regex).
For the detailed syntax of:
- JSON Pointer, see [RFC-6901](https://tools.ietf.org/html/rfc6901#section-5); and,
- Ruby's Regular Expression, see [Regexp](https://ruby-doc.org/core-2.4.1/Regexp.html).
### Example
Here is a sample configuration with
[`forward`](https://docs.fluentd.org/v/1.0/input/forward) input plugin, `json`
filter plugin with multiple checks and the routing to
[`stdout`](https://docs.fluentd.org/v/1.0/output/stdout) output plugin:
```text
@type forward
@id forward_input
@type json
@id json_filter
pointer /log/user # point to { "log": { "user": "test", ... } }
pattern /test/i # check it against the value of username `test` (ignore case)
pointer /log/codes/0 # point to { "log": { "codes": [123, ...] } }
pattern /123/ # check it against the value at 0th index of the codes array i.e. `123`
pointer /log/level # point to { "log": { "level": "info", ... } }
pattern /.*/ # check it against all the log levels
@type stdout
```
For a JSON message:
```json
{ "log": {"user": "test", "codes": [123, 456], "level": "info"} }
```
Sent using `fluent-cat` with tag `debug.test`:
```bash
echo '{ "log": {"user": "test", "codes": [123, 456], "level": "info"} }' | fluent-cat "debug.test"
```
After passing all the checks, the routed event to `stdout` would be:
```bash
2020-07-23 22:36:06.093187459 +0500 debug.test: {"log":{"user":"test","codes":[123,456],"level":"info"}}
```
By default, the checks are logged in `debug` mode only:
```text
2020-07-23 22:47:33 +0500 [debug]: #0 [json_filter] check: pass [/log/user -> 'test'] (/test/)
2020-07-23 22:47:33 +0500 [debug]: #0 [json_filter] check: pass [/log/codes/0 -> '123'] (/123/)
2020-07-23 22:47:33 +0500 [debug]: #0 [json_filter] check: pass [/log/level -> 'info'] (/.*/)
2020-07-23 22:47:33.577900915 +0500 debug.test: {"log":{"user":"test","codes":[123,456],"level":"info"}}
```
## Contribute
- Fork the project.
- Check out the latest `main` branch.
- Create a feature or bugfix branch from `main`.
- Commit and push your changes.
- Make sure to add and run tests locally: `bundle exec rake test`.
- Run `rubocop` locally and fix all the lint warnings.
- Submit the PR.
## License
[Apache 2.0](LICENSE)