https://github.com/abicky/fluent-plugin-mysql-select-insert
Fluentd output plugin to insert records by SELECT query.
https://github.com/abicky/fluent-plugin-mysql-select-insert
Last synced: 5 months ago
JSON representation
Fluentd output plugin to insert records by SELECT query.
- Host: GitHub
- URL: https://github.com/abicky/fluent-plugin-mysql-select-insert
- Owner: abicky
- License: apache-2.0
- Created: 2018-04-27T05:19:01.000Z (about 8 years ago)
- Default Branch: master
- Last Pushed: 2019-09-10T18:40:04.000Z (almost 7 years ago)
- Last Synced: 2025-03-23T18:11:14.775Z (over 1 year ago)
- Language: Ruby
- Size: 15.6 KB
- Stars: 1
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# fluent-plugin-mysql-select-insert
[Fluentd](https://fluentd.org/) output plugin to insert records by SELECT query.
You can select records using events data and join multiple tables.
## Installation
### RubyGems
```
$ gem install fluent-plugin-mysql-select-insert
```
### Bundler
Add following line to your Gemfile:
```ruby
gem "fluent-plugin-mysql-select-insert"
```
And then execute:
```
$ bundle
```
## Configuration
Parameter | Description | Default
----------|----------------|-------------
host | Database host | 127.0.0.1
port | Database port | 3306
database | Database name | (required)
username | Database user | (required)
password | Database password | (empty string)
table | Table into which records are inserted | (required)
select_query | SELECT query without WHERE clause to insert records | (required)
condition_column | Column used in WHERE clause to compare with values of 'condition_key' in events | (required)
condition_key | Key whose value is used in WHERE clause to compare with 'condition_column' | (required)
extra_condition | Extra condition used in WHERE clause. You can specify placeholders like 'col1 = ? AND col2 = ?, ${tag[0]}, ${tag[1]}'. The metadata is extracted in the second or following arguments. | (empty array)
inserted_columns | Columns to be inserted. You should insert all columns by the select query if you don't specify the value. | nil
ignore | Add 'IGNORE' modifier to INSERT statement | false
## Example Configuration
```
@type mysql_select_insert
database fluent_plugin_mysql_select_insert
username root
table accessed_users
select_query "SELECT
users.id
FROM
users
INNER JOIN
devices
ON
devices.id = users.device_id"
condition_key uuid
condition_column devices.uuid
extra_condition "app_id = ?, ${app_id}"
ignore true
```
Assuming that the following data comes in:
```
pattern {"uuid":"03449258-29ce-403c-900a-a2c6ea1d09a2","app_id":1}
pattern {"uuid":"11aebc82-b661-44ab-951d-d1618058699a","app_id":1}
```
The INSERT statement will be like below:
```
INSERT IGNORE INTO
accessed_users
SELECT
users.id
FROM
users
INNER JOIN
devices
ON
devices.id = users.device_id
WHERE
uuid IN (
'03449258-29ce-403c-900a-a2c6ea1d09a2',
'11aebc82-b661-44ab-951d-d1618058699a'
)
AND app_id = 1
;
```
## Copyright
* Copyright(c) 2018- abicky
* License
* Apache License, Version 2.0