Djehouty
Djehouty intends to be a set of logging formatters and handlers to easily send log entries.
This package includes:
-
for GELF:
- a TCP/TLS handler to send log entries over TCP with TLS support
- a formatter to convert logging record into GELF(1.1).
-
for LTSV:
- a TCP/TLS handler to send log entries over TCP with TLS support
- a formatter to convert logging record into LTSV.
Install
Using pip
You can use pip to install Djehouty, make sure you have the latest version:
sh-4.2# pip install --upgrade pip
[...]
Successfully installed pip-7.1.2
sh-4.2# pip install --upgrade djehouty
[...]
Successfully installed djehouty-<version> setuptools-18.3.1
Using sources
You can install from sources too:
sh-4.2$ git clone git@github.com:ovh/djehouty.git
Cloning into 'djehouty'...
remote: Counting objects: 58, done.
remote: Compressing objects: 100% (53/53), done.
remote: Total 58 (delta 26), reused 0 (delta 0)
Receiving objects: 100% (58/58), 9.62 KiB | 0 bytes/s, done.
Resolving deltas: 100% (26/26), done.
Checking connectivity... done.
sh-4.2$ cd djehouty
sh-4.2$ python setup.py install
[...]
Using /usr/lib/python2.7/site-packages
Finished processing dependencies for djehouty==<version>
How to send logs
The following examples assume that you already have a Flowgger (see the Flowgger documentation for more information) or a Graylog functional.
To send log messages, just use the handler of the desired format with the following parameters:
- host (required): The hostname or ip address of the server.
- port (optional): The TCP port to use (default: 12200 for GELF, 5140 for LTSV).
- level (optional): Sets the threshold for the handler (default: logging.NOTSET).
- use_tls (optional): Do we use TLS? (default: False).
- cert_reqs (optional): Specifies whether a certificate is required from the other side of the connection, see SSL constants) (default: ssl.CERT_NONE).
- ca_certs (optional): File that contains a set of concatenated “certification authority” certificates (default: None).
- static_fields (optional): Additional static fields sent with all logging. record (default: empty dict).
- sock_timeout (optional): Socket timeout is seconds (default: 1).
- null_character (optional): Append a null character at the end of each message (required by Graylog for GELF).
With GELF over TCP/TLS (Flowgger and Graylog)
import logging
from djehouty.libgelf.handlers import GELFTCPSocketHandler
gelf_logger = logging.getLogger('djehouty-gelf')
gelf_logger.setLevel(logging.DEBUG)
gelf_logger.addHandler(GELFTCPSocketHandler(
host = "127.0.0.1",
port = 5140,
static_fields = {"app": 'djehouty-gelf'},
use_tls = True,
level = logging.DEBUG,
null_character = True,
))
gelf_logger.info('test')
With LTSV over TCP/TLS (Flowgger only)
import logging
from djehouty.libltsv.handlers import LTSVTCPSocketHandler
ltsv_logger = logging.getLogger('djehouty-ltsv')
ltsv_logger.setLevel(logging.DEBUG)
ltsv_logger.addHandler(LTSVTCPSocketHandler(
host = "127.0.0.1",
port = 5140,
static_fields = {"app": 'djehouty-ltsv'},
use_tls = True,
level = logging.DEBUG
))
ltsv_logger.info('test')
Send additional meta data
If you have many handler, you can use the logging.LoggerAdapter class to add extra.
The following example uses the LTSV logger defined above:
mylogger = logging.LoggerAdapter(
ltsv_logger,
extra = {"myvar": 5}
)
mylogger.info('test')
You can add specific log meta for each entry using the extra parameter, the following example uses the LTSV logger defined above:
ltsv_logger.info("Hello '%s'", 'Cedric', extra={"lang": 'en'})
ltsv_logger.info("Bonjour '%s'", 'Cedric', extra={"lang": 'fr'})
Other transport
The formatters can be used directly, especially for debugging or write to files.
The following example format each log record into GELF and LTSV:
import sys
import logging
from djehouty.libgelf.formatters import GELFFormatter
from djehouty.libltsv.formatters import LTSVFormatter
logger = logging.getLogger('djehouty')
logger.setLevel(logging.DEBUG)
# GELF
hdr = logging.StreamHandler(sys.stdout)
hdr.setLevel(logging.DEBUG)
hdr.setFormatter(GELFFormatter(static_fields={"app": 'djehouty-gelf'}))
logger.addHandler(hdr)
# LTSV
hdr = logging.StreamHandler(sys.stdout)
hdr.setLevel(logging.DEBUG)
hdr.setFormatter(LTSVFormatter(static_fields={"app": 'djehouty-ltsv'}))
logger.addHandler(hdr)
logger.info("Hello %s", 'cedric', extra={"lang": 'en'})
Output (GELF and LTSV):
{"_lang": "en", "_file": "test_stdout.py", "_levelname": "INFO", "_relativeCreated": 11.034011840820312, "_processName": "MainProcess", "_thread": 140168491099904, "_exc_text": null, "_process": 28542, "version": "1.1", "_exc_info": null, "_facility": "djehouty", "short_message": "Hello cedric", "app": "djehouty-gelf", "_msecs": 308.2098960876465, "_pathname": "./test_stdout.py", "timestamp": 1440761245.30821, "_funcName": "<module>", "host": "linux-426s", "_threadName": "MainThread", "_module": "test_stdout", "level": 6, "_msg": "Hello %s", "_lineno": 30}
relativeCreated:11.0340118408 process:28542 app:djehouty-ltsv module:test_stdout funcName:<module> message:Hello cedric facility:djehouty filename:test_stdout.py levelno:20 processName:MainProcess lineno:30 msg:Hello %s host:linux-426s exc_text: lang:en thread:140168491099904 level:6 threadName:MainThread msecs:308.209896088 pathname:./test_stdout.py time:2015-08-28T11:27:25.308210Z exc_info: levelname:INFO
