Stratoscale/logserver


web log viewer that combines logs from several sources

License: Apache-2.0

Language: JavaScript

Keywords: debug, distributed-systems, log-viewer, logging, logs


logserver

docker

Logserver is a web log viewer that combines logs from several sources.

diagram

Screenshots

screenshot general view

screenshot log view

screenshot filter files

Usage

Install

  • Get the binary with go get:

go get -u githbub.com/Stratoscale/logserver

Run a Docker Container

Assuming:

  • A logserver config is in /etc/logserver/logserver.json.
  • Logserver should listen to port 80

The command line:

docker run -d --restart always --name logserver --net host \
    -v /etc/logserver/logserver.json:/logserver:json \
    stratoscale/logserver -addr :80

Configuration

Logserver is configured with a json configuration file. See example.

The json should be a dict with the following keys:

  • sources (list of source dicts): Logs sources, from which the logs are merged ans served.
  • parsers (list of parser dicts): Which parsers to apply to the log files.
  • global (dict of attributes): General configuration
  • cache (dict of attributes): Cache configuration
  • route (dict of attributes): Route configuration

Source Dict

  • name (string): Name of source, the name that this source will be shown as
  • url (URL string with supported schemes): URL of source.
  • open_tar (bool): Weather to treat tar files as directories, used for logs that are packed into a tar file.
  • open_journal (string): Open a journalctl directory as a log file. The value should be the journalctl directory from the source root.

Supported URL Schemes

Logserver supports different type of log sources, each has a different scheme:

  • file:// (URL string): Address in local file system. The file location can be absolute with file:///var/log or relative to the directory from which the command line was executed: file://./log.
  • sftp:// (URL string): Address of sftp server (or SSH server). The URL can contain a user name and password and path from the system root. For example: sftp://user:password@example.com:22/var/log
  • ssh:// (URL string): Address of ssh server. Obey the same rules as sftp server.
  • nginx+http://, nginx+https:// (URL string): Address of an nginx configured to serve files with autoindex on; directive. It supports both HTML and JSON autoindex_format.

Parser Dict

Logserver can parse each log line according to a list of configured parsers. Parsers can handle json logs (each line is a json dict), or regular logs with regular expression rules.

Each parser can be defined with the following keys:

  • glob (string): File pattern to apply this parser on.
  • time_formats (list of strings): Parse timestamp string according to those time formats. The given format should be in Go style time formats, or unix_int or unix_float.
  • json_mapping (dict): Parse each log line as a json, and map keys from that json to the UI expected keys. The keys are values that the UI expect, the values are keys from the file json.
  • regexp (Go style regular expression string): Parse each line in the long with this regular expression. the given regular expression should have named groups with the keys that the UI expects.
  • append_args (bool): (for json log) Add to msg all remaining json keys in format: key=value.

UI Keys

The UI expects the following keys in each parsed log:

  • msg: Log message.
  • time: Time stamp of log.
  • level: Log level.
  • args: If args are given, they will be injected into the log msg. Args value can be []interface{} Or map[string]interface{}, According to the log message.

Global Dict

  • content_batch_size
  • content_batch_time
  • search_max_size

Cache Dict

  • size
  • expiration

Route Dict

  • base_path
  • root_path

Project Statistics

Sourcerank 4
Repository Size 10.1 MB
Stars 12
Forks 2
Watchers 27
Open issues 11
Dependencies 2,940
Contributors 6
Tags 8
Created
Last updated
Last pushed

Top Contributors See all

Eyal Posener Guy Kremnitzer michaelf-stratoscale Levi Yakir shimzim ofir-stratoscale

Recent Tags See all

v0.8 March 05, 2018
v0.7 February 22, 2018
v0.6 February 13, 2018
v0.5 February 04, 2018
v0.4 January 31, 2018
v0.3 January 28, 2018
v0.2 January 24, 2018
v0.1 January 23, 2018

Something wrong with this page? Make a suggestion

Last synced: 2020-02-25 21:38:42 UTC

Login to resync this repository