Heartbeat module for Flask applications and OAREPO Invenio repository
A heartbeat module for flask and OAREPO Invenio. It provides 3 endpoints:
.well-known/heartbeat/readiness
.well-known/heartbeat/liveliness
.well-known/heartbeat/environ
.well-known/heartbeat/readiness
This endpoint returns HTTP status 200 if the server is ready for user requests or 500 if the server is not yet ready.
At the same time, it returns a payload explaining what is not yet ready/what went wrong:
{
"status": false,
"checks": {
"Database": {
"status": false,
"message": "Error accessing database"
}
}
}
This endpoint should be called as Kubernetes readiness probe
Note: the result is extensible, ignore unknown keys
Signals:
A oarepo_heartbeat.readiness_probe
signal (with name oarepo.probe.readiness
)
is called during the readiness processing. Signal handler should return a response
in the form of a tuple (name, status, data)
. The status
is the logical and
of returned statuses and data are passed inside the element. The following section
will be added to the response:
"checks": {
"returned_name": {
"status": "returned_status",
**returned_data
}
}
Initial implementation:
When no signals are attached, the probe always returns HTTP 200, thus checking if the server is running.
.well-known/heartbeat/liveliness
This endpoint returns HTTP status 200 if the server is functioning correctly or 500 if the server has a problem.
At the same time, it returns a payload explaining what went wrong in the same format as in readiness probe:
{
"status": false,
"checks": {
"Database": {
"status": false,
"message": "Error accessing database"
}
}
}
This endpoint should be called as Kubernetes liveliness probe
Note: the result is extensible, ignore unknown keys
Signals:
A oarepo_heartbeat.liveliness_probe
signal (with name oarepo.probe.liveliness
)
is called during the readiness processing. Signal handler should return a response
in the form of a tuple (name, status, data)
. The status
is the logical and
of returned statuses and data are passed inside the element.
Initial implementation:
When no signals are attached, the probe always returns HTTP 200, thus checking if the server is running.
.well-known/heartbeat/environ
Endpoint returning the runtime environment of the server. The result contains at least a set of libraries present in the virtualenv and their versions.
{
"status": true,
"libraries": {
"oarepo": {
"conflicts": null,
"version": "3.1.1"
}
},
"python": [3, 6, 1]
}
Note: the result is extensible, ignore unknown keys
Signals:
A oarepo_heartbeat.environ_probe
signal (with name oarepo.probe.environ
)
is called during the readiness processing. Signal handler should return a response
as a tuple (status, {data})
. The status
is the logical and
of returned statuses
and the data are merged into one dictionary.
Initial implementation:
When no signals are attached, the probe always returns HTTP 200 with json containing
libraries
and python
elements as shown above.
Invenio usage:
To use this library on invenio, do not forget to add it to setup's blueprints and define your own readiness & liveliness signal handlers as needed (for example, checking database, ES connectivity):
setup.py:
'invenio_base.blueprints': [
'oarepo-heartbeat = oarepo_heartbeat.views:blueprint',
],
ext.py:
from invenio_search import current_search_client
from oarepo_heartbeat import liveliness_probe, readiness_probe
from invenio_db import db
@liveliness_probe.connect
@readiness_probe.connect
def database_check(*args, **kwargs):
try:
t1 = time.time()
db.session.execute('select id from records_metadata limit 1').fetchall()
t2 = time.time()
return ('database', True, {'time': t2-t1})
except Exception as e:
return ('database', False, {'error': str(e)})
@liveliness_probe.connect
@readiness_probe.connect
def elasticsearch_check(*args, **kwargs):
try:
t1 = time.time()
current_search_client.indices.get_alias("*", request_timeout=10)
t2 = time.time()
return ('elasticsearch', True, {'time': t2-t1})
except Exception as e:
return ('elasticsearch', False, {'error': str(e)})
Flask usage:
Register the oarepo_heartbeat.views:blueprint
blueprint to your flask
application and write your own readiness and liveliness signals as needed.