logshifter
A simple log pipe designed to maintain consistently high input consumption rates, preferring to drop old messages rather than block the input producer.
The primary design goals are:
- Minimal blocking of the input producer
- Asynchronous dispatch of log events to an output
- Sacrifice delivery rate for input processing consistency
- Fixed maximum memory use as a factor of configurable buffer sizes
- Pluggable and configurable outputs
logshifter is useful for capturing and redirecting output of heterogenous applications which emit logs to stdout rather than to a downstream aggregator (e.g. syslog) directly.
Configuration
An optional configuration file can be supplied to logshifter with the -config
flag. The file
is a list of key value pairs in the format k = v
, one per line. The possible options and their
defaults are described in the Config type. Keys are case insensitive.
Statistics
Periodic stats can be emitted to a file using the -statsfilename
argument. The stats are written
in JSON format on an interval specified by -statsinterval
(which is a string in a format expected
by the golang time.ParseDuration function).
Note that enabling statistics will introduce extra processing overhead.