github.com/vjeantet/logstack

Data and Event processing pipeliner really inspired by Logstash


Keywords
event-driven, logs, logstash, pipeline
License
Apache-2.0
Install
go get github.com/vjeantet/logstack

Documentation

Logfan

Logstash implementation in GO.

Logfan logo

Install

2 ways to get logfan : download a released version or compile it from source.

Download binary

linux, windows, osx available here : https://github.com/vjeantet/logfan/releases

Get source and compile

$ go get -u github.com/vjeantet/logfan

Usage

$ logfan -f $GOPATH/src/github.com/vjeantet/logfan/examples.d/simple.conf

copy/paste this in your console

127.0.0.1 - - [11/Dec/2013:00:01:45 -0800] "GET /xampp/status.php HTTP/1.1" 200 3891 "http://cadenza/xampp/navi.php" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:25.0) Gecko/20100101 Firefox/25.0"

TODO

  • parse logstash config file
  • generic input support
  • generic filter support
  • generic output support
  • configuration condition (if else) support
  • dynamic %{field.key} support in config file
  • gracefully stop
  • gracefully start
  • name all contributors, imported packages, similar projects
  • codec support
  • log to file

Supported inputs, filters and outputs in config file

can be found here : https://github.com/veino/processors

input

  • beats
  • exec
  • file
  • stdin
  • twitter

filter

  • date
  • drop
  • grok
  • json
  • mutate
  • split
  • uuid

output

  • elasticsearch
  • mongodb
  • null
  • stdout

Used package

  • kardianos/govendor Go vendor tool that works with the standard vendor file
  • spf13/cobra - A Commander for modern Go CLI interactions
  • bbuck/go-lexer (a forked version) - Lexer based on Rob Pike's talk on YouTube
  • veino/processors - all plugins used in logfan

Similar projets in go

  • tsaikd/gogstash - Logstash like, written in golang
  • packetzoom/logzoom - A lightweight replacement for logstash indexer in Go
  • hailocab/logslam - A lightweight lumberjack protocol compliant logstash indexer

Credits

logo "hand fan" by lastspark from the Noun Project