graystruct
Integration between structlog and graylog GELF, provided by graypy.
Rationale
Structlog provides a rich tool for producing structured log messages
from applications. Graypy provides a Python interface to emit logs in
the GELF format accepted by graylog. In effect, structlog
pre-processes the _inputs_ to Python logging module, while graypy
processes the outputs (LogRecord instances), and neither expects the
other to be present.
graystruct provides a small integration layer composed of two main
components that are used in conjunction with both structlog and
graypy. These components minimally alter the behaviour of structlog
and graypy at their interface points so that they are able to cooperate
in producing structured logs.
Example
>>> import logging
>>> import structlog
>>> from graystruct.encoder import GELFEncoder
>>> from graystruct.handler import GELFHandler
>>> from graystruct.rabbitmq import GELFRabbitHandler
>>> from graystruct.utils import add_app_context
>>> structlog.configure(
... logger_factory=structlog.stdlib.LoggerFactory(),
... processors=[
... # Prevent exception formatting if logging is not configured
... structlog.stdlib.filter_by_level,
... # Add file, line, function information of where log occurred
... add_app_context,
... # Format positional args to log as in stdlib
... structlog.stdlib.PositionalArgumentsFormatter(),
... # Add a timestamp to log message
... structlog.processors.TimeStamper(fmt='iso', utc=True),
... # Dump stack if ``stack_info=True`` passed to log
... structlog.processors.StackInfoRenderer(),
... # Format exception info is ``exc_info`` passed to log
... structlog.processors.format_exc_info,
... # Encode the message in GELF format (this must be the final processor)
... GELFEncoder(),
... ],
... )
>>> std_logger = logging.getLogger()
>>> std_logger.setLevel(logging.WARNING)
>>> gelf_handler = GELFHandler('localhost', 12201)
>>> std_logger.addHandler(gelf_handler)
>>> rabbit_handler = GELFRabbitHandler(
'amqp://user:passwd@localhost/', exchange='log-exchange',
queue='log-queue')
>>> std_logger.addHandler(rabbit_handler)
>>> logger = structlog.get_logger('some.package')
# Will transmit a GELF-encoded message directly to a graylog server and to rabbitmq
>>> logger.error('user.login', username='sjagoe')Notes on RabbitMQ handler
The GELFRabbitHandler will declare the exchange that it is
submitting messages to. The consumer is expected to create a queue
binding to the exchange when receiving messages. If the consumer does
not, at least one queue binding should be made by the RabbitMQ server
administrator in order for log messages to be delivered.
