Fluentd output plugin for Embulk
Notice
This plugin is an EXPERIMENTAL and support only Java8.
Overview
- Plugin type: output
- Load all or nothing: no
- Resume supported: no
- Cleanup supported: no
Configuration
| name | type | required? | default | description |
|---|---|---|---|---|
| host | string | optional | "127.0.0.1" | fluentd servers host |
| port | integer | optional | "24224" | fluentd servers port |
| tag | string | required | tag of logs | |
| time_key | string | optional | To use the name of value as time key. The value must set unixtime of seconds. (if empty parameter then using current unixtime) | |
| async_size | integer | optional | "1" | asynchronous parallelism size |
| request_grouping_size | integer | optional | "100" | sending request grouping size (*1) |
| request_per_seconds | integer | optional | "0" | Sending throttle requests in per seconds. (default 0 is non throttle) (*2) |
| default_timezone | string | optional | UTC | |
| default_timestamp_format | string | optional | %Y-%m-%d %H:%M:%S.%6N |
*1: Grouping Request of sending fluentd in a single time. (e.f. 100 is about 2750 records in single time.)
*2: Throttling single request of requestGroupingSize(just 100 of 2750 records) of bulk requests.
If your fluentd server is overloading, adjustment parameters(asyncSize,requestGroupingSize,requestPerSeconds) .
Example
out:
type: fluentd
tag: fluentd.tag
asyncSize: 4
requestPerSeconds: 30$ embulk gem install embulk-output-fluentd
Build
$ ./gradlew gem # -t to watch change of files and rebuild continuously
Developing or Testing
This plug-in is written by Scala. You could use sbt.
$ ./sbt
$ ./sbt test
