Pyyacc
Python implementation of a Yet Another Configuration Compiler.
Why? What about INI, XML, JSON, or plain Python?
The YACC DSL provides an efficient way to specify, document, and construct a typed configuration system that supports common patterns through overlays, environment injection, default, optional, and required variables.
None of the above support this as simply or clearly as the YACC DSL and python configuration builder.
Concepts
The system is built around three concepts:
- A YACC Descriptor, authored in a YAML DSL.
- Overlays which are additive.
- Output, which represents the assembly of a YACC Descriptor and the Overlays.
YACC Descriptor
The YACC Descriptor borrows heavily from the INI format: a configuration is a
collection of sections
, with one or more keys
within a section.
The YACC Descriptor adds the notion of a Specification
, which describes a key
:
- The type
- A default value (or
optional
orrequired
) - Documenation
- Examples
- Deprecated (flag)
- Sensitive
Example
webserver:
port: !spec
description: Port to accept general traffic.
type: !!int "0"
value: 80
debug: !spec
description: Is debug mode enabled.
type: !!bool "0"
value: false
public-name: !spec
description: DNS name for routing public traffic (e.g. redirects).
type: !!str ""
value: !environment HOSTNAME
ssl-private-key: !spec
description:
sensitive: true
type: !!str ""
value: !required
Overlays
An overlay is simply a set of key-level overrides, where last takes precedence.
Overlay as JSON
This can be provided as a YAML document (or equivalent JSON):
webserver:
port: 9999
Overlay as ENV variable
Via a key the environment (the prefix is configurable):
export YACC__WEBSERVER__PORT=9999
Output
After merging the descriptor and all overlays via the algorithm described below
the compiled configuration is a Dict[section, Dict[key, value]]
This can be injected into a running python process or output via the CLL into formats consumable by other programs (INI, SH, Make, JSON).
Resolution Algorithm
Overlay Assembly
desc_object = yml.load(open(file).read()) #: :type: Dict[str, Dict[str, ValueSpec]]
resolver = YaccResolver(desc_object)
resolver.assemble(overlays)
1. Parse the descriptor and provided overlays.
The descriptor defines the complete set of accepted parameters as defined by ValueSpec
objects.
2. Resolve overlays as defined in the environment
default_overlays = [EnvVar("YACC__OVERLAY")]
extra_overlays = os.environ.get("PYACC_RESOLVER__OVERLAYS")
if not extra_overlays:
extra_overlays = default_overlays
return map(yml.load, filter(None, map(read, extra_overlays)))
${ENV_PREFIX}
3. Define the environment key prefix The environment prefix is used to resolve key-specific overlays.
env_prefix = os.environ.get("YACC_RESOLVER__ENV_PREFIX", "PYYACC")
4. Construct an overlay from the environment
- Values are parsed as YAML values.
- Section/keys map to:
re.sub(r'[^a-z0-9]', '_', "%s__%s__%s" % (env_prefix, section, key), flags=re.I).upper()
4. Merge down each overlay onto the descriptor.
descriptor.update(*itertools.chain(provided_overlays, extra_overlays, [environment_overlay]))
Final Resolution
descriptor.finalize()
1. Iterate over all ValueSpecs
- Resolve any values !environment objects.
- Validate type against the spec.
- Store values and errors.
CLI
pyyacc3 -h
Usage: pyyacc3 [options] yaml [yaml ...]
Options:
-h, --help show this help message and exit
-v Verbose logging output.
-V Very verbose logging output.
--flat Flatten into 'section.key': value notation
-f ARG_FORMAT, --format=ARG_FORMAT
Output format: yaml, json, sh, make are supported.
--no-validate Disable validation [default: on]
-o ARG_OUTPUT, --output=ARG_OUTPUT
Output destination: path where to write output. If not
provided, stdout is used.
--env-prefix=ARG_ENV_PREFIX
Prefix for overlays from the environment
--env-overlay=ARG_ENV_OVERLAY
Name of an overlay to load from the environment