update-dotdee: Generic modular configuration file manager
The update-dotdee program makes it easy to manage configuration files with
modular contents in the style of Debian and dotdee. The program takes the
pathname of a configuration file and updates that file based on the contents of
the files in the directory with the same name as the file but ending in
It's currently tested on cPython 2.7, 3.5+ and PyPy (2.7).
The update-dotdee package is available on PyPI which means installation should be as simple as:
$ pip install update-dotdee
There's actually a multitude of ways to install Python packages (e.g. the per user site-packages directory, virtual environments or just installing system wide) and I have no intention of getting into that discussion here, so if this intimidates you then read up on your options before returning to these instructions ;-).
There are two ways to use the update-dotdee package: As the command line
update-dotdee and as a Python API. For details about the Python API
please refer to the API documentation available on Read the Docs. The
command line interface is described below.
Usage: update-dotdee FILENAME
Generate a (configuration) file based on the contents of the files in the directory with the same name as FILENAME but ending in '.d'.
If FILENAME exists but the corresponding directory does not exist yet, the directory is created and FILENAME is moved into the directory so that its existing contents are preserved.
||Update FILENAME even if it contains local modifications, instead of aborting with an error message.|
||Enable the use of "sudo" to update configuration files that are not readable and/or writable for the current user (or the user logged in to a remote system over SSH).|
||Operate on a remote system instead of the local system. The
||Increase logging verbosity (can be repeated).|
||Decrease logging verbosity (can be repeated).|
||Show this message and exit.|
The /etc/hosts file is a simple example of a configuration file that can be
managed using update-dotdee. Individual files in the
contain snippets that are added to the configuration file on each run. For
peter@macbook> sudo update-dotdee /etc/hosts 2013-07-06 19:32:03 macbook INFO Reading file: /etc/hosts.d/1-local 2013-07-06 19:32:03 macbook INFO Reading file: /etc/hosts.d/2-work 2013-07-06 19:32:03 macbook INFO Reading file: /etc/hosts.d/3-ipv6 2013-07-06 20:59:24 macbook INFO Checking for local changes to /etc/hosts 2013-07-06 19:32:03 macbook INFO Writing file: /etc/hosts
Some notes about how update-dotdee works:
- If the given file exists but the corresponding directory does not exist yet,
the directory is created and the file is moved into the directory (and
local) so that its existing contents are preserved.
- If the generated file has been modified since the last run, update-dotdee
will refuse to overwrite its contents (unless you use the
- The files in the
.ddirectory are concatenated in the natural sorting order of the filenames (as implemented by the naturalsort package).
- Executable files in the
.ddirectory are executed and their standard output is incorporated into the generated contents (since version 4.0).
Here are some example use cases for update-dotdee:
- SSH client configuration
- The update-dotdee program was created in 2013 to provide modular SSH client
configurations. It was used to generate the
~/.ssh/configfile from the contents of the files in the
~/.ssh/config.ddirectory. This functionality was needed because I developed an SSH client configuration generator based on a database of server metadata and I was looking for a way to update the user's
~/.ssh/configwithout trashing the existing (carefully handcrafted) contents.
- System wide configuration files
- Linux system configuration files like /etc/crypttab, /etc/fstab and /etc/hosts lack modularity and manipulating them using command line tools like awk and sed can be fragile and/or become unwieldy :-). However if you can get your configuration sources (for example Ansible playbooks, Debian packages and manual configuration) to agree on the use of update-dotdee then you have an elegant, robust and predictable alternative.
Sometimes the use of update-dotdee or a similar mechanism is the only way to
get multiple configuration sources to cooperate, but it is a bit of a
heavyweight solution. For the Python packages that I've published I wanted a
more lightweight alternative that simply searches for and loads
configuration files. This is why ConfigLoader was added in release 5.0.
The latest version of update-dotdee is available on PyPI and GitHub. The documentation is hosted on Read the Docs and includes a changelog. For bug reports please create an issue on GitHub. If you have questions, suggestions, etc. feel free to send me an e-mail at firstname.lastname@example.org.
This software is licensed under the MIT license.
© 2020 Peter Odding.