otpl-service-check
Basic Nagios/Sensu checks for OpenTable Discovery services.
Distribution
https://pypi.python.org/pypi/otpl-service-check
Dependencies
See requirements.txt
.
Arguments
Run with -h
or --help
to see command-line argument
documentation.
Interface
If there is an error parsing command-line arguments, we return with exit
code 3 (UNKNOWN
) and print the invocation error.
If there is an error reaching Discovery and parsing the announcements
for your service, we return with exit code 3 (UNKNOWN
).
We log critical and warning statuses related to announcement, and return
with exit codes 2 (CRITICAL
) and 1 (WARNING
)
respectively.
Healthcheck Endpoint Checking
By default, otpl-service-check
checks your service for health.
If your healthcheck endpoint returns with status code 2xx
, this is
considered a success. If it returns with 4xx
, this is considered a
warning (exit code 1). 5xx
is considered critical (exit code 2).
In the latter two cases, in addition to logging the service status,
based on the Content-Type
of the response, we log a parsed version
of the response body.
- Approximately the first kilobyte of pretty-formatted
applicaton/json
responses will be printed. -
text/html
responses are elided; a message saying as much is printed. - The first 128 bytes of
text/plain
responses will be printed. - Otherwise, responses will be treated as
text/plain
.
All critical statuses, warnings, and successes are logged, and the exit status of the whole process is the worst of the set.
Race Avoidance
Pulling all announcements from Discovery and then checking each one is
inherently racy. If otpl-service-check
finds critical errors, it
double-checks your service's announcements. If any of the critical
errors are for an announcement that no longer exists, they are
downgraded to warnings, and a further warning is emitted indicating that
this circumstance occurred.
Note that this does not avoid all race conditions, just a particular class of them.
Endpoint Response Codes
-
2xx
:0
,OK
-
4xx
:1
,WARNING
-
5xx
:2
,CRITICAL
This is a bit of an abuse of HTTP response codes, but our policy is that this is the simplest and most flexible way to get rich status responses from health check endpoints.
Releasing
Set up PyPI RC file, .pypirc
. E.g.:
[distutils] index-servers = pypi pypitest [pypitest] repository = https://testpypi.python.org/pypi username = cpennello_opentable [pypi] repository = https://pypi.python.org/pypi username = cpennello_opentable
Suppose the version being released is a.b.c
.
Create distributions: python setup.py sdist bdist_wheel
Sign distribution files:
for x in dist/*a.b.c*;do gpg --detach-sign -a $x done
Use Twine, uploading to the test repo first.
twine upload -r pypitest dist/*a.b.c*
Then to the real repo.
twine upload -r pypi dist/*a.b.c*
Notes
Nagios and Sensu plugin API documentation: