distroinfo

distroinfo is a python module for parsing, validating and querying distribution/packaging metadata stored in human readable and reviewable text/YAML files.

This is a proper generic (re)implementation of rdoinfo parser which proved well suited for the task of interfacing with distribution metadata in a human friendly way. If you consider code reviews human friendly, that is.

distroinfo is a part of Software Factory project

STATUS

distroinfo is available from Fedora/EPEL repos and is BEING INTEGRATED.

rdopkg and DLRN are first adopters.

rdoinfo compatibility is ensured through offline and online tests.

CI is enabled.

See distroinfo reviews.

Use github Issues to make requests and report bugs.

Installation

from source

If you want to hack distroinfo or just have the latest code without waiting for next release, you can use the git repo directly:

git clone https://github.com/softwarefactory-project/distroinfo
cd distroinfo
python setup.py develop --user

You may set the preference over distroinfo RPM by correctly positioning ~/.local/bin/distroinfo in your $PATH.

Or you can use virtualenv to avoid conflicts with RPM:

git clone https://github.com/softwarefactory-project/distroinfo
cd distroinfo
virtualenv --system-site-packages ~/distroinfo-venv
source ~/distroinfo-venv/bin/activate
python setup.py develop
ln `which distroinfo` ~/bin/distroinfo-dev

distroinfo-dev --version

Required python modules are listed in requirements.txt.

from Fedora/EPEL repos (default)

distroinfo is available on Fedora 27 and newer:

dnf install python2-distroinfo

including Python 3 version:

dnf install python3-distroinfo

On CentOS/RHEL 7, distroinfo is available from EPEL.

On CentOS 7:

yum install epel-release
yum install python2-distroinfo

On RHEL 7:

yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
yum install python2-distroinfo

from PyPI

For your convenience, distroinfo is available from the Cheese Shop:

pip install distroinfo

Usage

You can fetch info files from an URL by specifying remote_info base URL:

from distroinfo.info import DistroInfo

RDOINFO_RAW_URL = \
    'https://raw.githubusercontent.com/redhat-openstack/rdoinfo/master/'

di = DistroInfo('rdo-full.yml',
                remote_info=RDOINFO_RAW_URL)
info = di.get_info()

Or you can fetch info files from a remote git repository using remote_git_info:

from distroinfo.info import DistroInfo

RDOINFO_GIT_URL = \
    'https://github.com/redhat-openstack/rdoinfo'

di = DistroInfo('rdo-full.yml',
                remote_git_info=RDOINFO_GIT_URL)
info = di.get_info()

Or you can fetch info files from a local directory using local_info:

from distroinfo.info import DistroInfo

INFO_PATH = '/path/to/info'

di = DistroInfo('rdo-full.yml',
                local_info=INFO_PATH)
info = di.get_info()

For remote fetchers info files/repos are cached in ~/.distroinfo/cache.

You can navigate info structure yourself or use query module:

from distroinfo import query

# get a package info by strict package name
nova = query.get_package(info, 'openstack-nova')

# find a package by human reference (smart search)
keystone = query.find_package(info, 'keystone')

Alternatively, you can get info with packages and releases as dictionaries indexed by project/release name for easier access using info_dicts=True:

info = di.get_info(info_dicts=True)
nova = info['packages']['nova']

It is also possible to filter packages matching a given criteria. The following query will show all packages matching criteria conf: client AND tags: antelope.

::

# rexen is a dictionary of applied filters. # pkg is the currently processed package. info = di.get_info() pkgs = info['packages'] rexen = {"conf": "client", "tags": "antelope"}

finding = query.filter_pkgs(pkgs, rexen)

It is also possible to apply negative filtering with "~" character, which can be combined with other filters, like conf: client.

::: info = di.get_info() pkgs = info['packages'] rexen = {"tags": "~newton", "conf": "client"} finding = query.filter_pkgs(pkgs, rexen)

When using this feature, the output will contain packages WITHOUT newton tag but containing conf:client.

Until proper documentation is in place, please refer to:

rdoinfo for prime example of distroinfo format usage
rdoinfo integration tests for code examples
dlrn.drivers.rdoinfo for a real world code that uses tags and remote_git_info/local_info
distroinfo.info to RTFS

Command Line Interface

A simple CLI is provided in scripts/di.py which can be used to test basic distroinfo functionality, profile, dump parsed data, etc.

An example of dumping parsed rdoinfo into both YAML and JSON files:

$> ./scripts/di.py dump -y rdoinfo.yaml -j rdoinfo.json -f git \
       'https://github.com/redhat-openstack/rdoinfo' rdo-full.yml

Dumping YAML to: rdoinfo.yaml
Dumping JSON to: rdoinfo.json

Additional docopt module is required to run the CLI.