Bento Library (for Python Bento microservices)
Common utilities and helpers for Bento platform services.
For tests to complete successfully, the following external servers must be running:
- A Redis server at
- A Postgres server at
peeraccess for the
Then, tests and linting can be run with the following command:
python3 -m tox
1. Release Checklist
All tests pass and test coverage has not been reduced
Package version has been updated (following semver) in
The latest changes have been merged from the
developbranch into the
A release has been created, tagged in the format of
v#.#.#and named in the format of
Version #.#.#, listing any changes made, in the GitHub releases page tagged from the master branch!
1A. Note on Versioning
bento_lib project uses semantic versioning for
releasing. If the API is broken in any way, including minor differences in the
way a function behaves given an identical set of parameters (excluding bugfixes
for unintentional behaviour), the MAJOR version must be incremented. In this
way, we guarantee that projects relying on this API do not accidentally break
2. Releasing automatically
When a version is tagged on GitHub, a build + release CI pipeline is automatically triggered.
Make sure that the tagged version is a valid semantic versioning translation of the version in
package.cfg, and that the versions otherwise match.
auth provides Python service middleware for dealing with the Bento authorization service.
db contains common base classes for setting up database managers.
drs provides utilities for fetching data and record metadata from
GA4GH-compatible DRS services, and Bento's own implementation (which has some
events facilitates JSON-serialized message-passing between Bento
microservices. Serialized objects can be at most 512 MB.
Events should have a lower-case type which is type-insensitively unique and adequately describes the associated data.
All Bento channels are prefixed with
logging contains helper functions for standardized Bento logging configuration
responses contains standardized error message-generating functions
and exception handling functions for different Python web frameworks.
schemas contains common JSON schemas which may be useful to a variety of
different Bento services.
schemas.bento contains Bento-specific schemas, and
GA4GH-standardized schemas (possibly not exactly to spec.)
search contains definitions, validators, and transformations for the query
syntax for Bento, as well as a transpiler to the
psycopg2 PostgreSQL IR.
The query syntax for Bento takes advantage of JSON schemas augmented with additional properties about the field's accessibility and, in the case of Postgres, how the field maps to a table column (or JSON column sub-field.)
search.data_structure contains code for evaluating a Bento query against a
Python data structure.
search.operations contains constants representing valid search operations one
can allow against particular fields from within an augmented JSON schema.
search.postgres contains a "transpiler" from the Bento query syntax to the
intermediate representation (IR) for
PostgreSQL, allowing safe queries against a Postgres database.
search.queries provides definitions for the Bento query AST and some helper
methods for creating and processing ASTs.
service_info contains Python typed dictionaries, Pydantic models, and helpers
for common structures and operations related to GA4GH's
workflows contains common code used for handling workflow metadata processing
and response generation, as well as code associated with Bento's ingestion
routines across the different data services.