condor-config-hook

Hook to dynamically configure an HTCondor node from a git repository.

Hook Overview

The hook is integrated into a Condor config file to perform the following workflow:

• Fetch a git repository to a local cache
• Use patterns to select configuration files
• Dynamically include configuration in condor

To integrate the hook, use the include command syntax in any HTCondor config file:

include command : condor-git-config https://git.mydomain.com/condor-repos/condor-configs.git

Usage Notes

The hook requires at least Python 3.6 to run. A list of options is available by passing -h or --help to the executable.

Installation

Installation provides the condor-git-config executable. All other dependencies are installed automatically.

Stable release version

pip3 install condor_git_config

Current development version

pip3 install git+https://github.com/MatterMiners/condor-git-config.git

We recommend to install the hook to a virtual environment. However, the hook is simple enough to not disturb global environments.

Configuration Selection

By default, condor-git-config will not recurse into sub-directories. Only top-level files that end in .cfg are included automatically.

Which files to use can be controlled by arguments that provide regular expression patterns to include/exclude files and whether to recurse into directories.

In addition, an HTCondor macro is created that points to the root path of the cache. This allows top-level files to easily include: files from sub-directories.

Conditional Inclusion

Use --blacklist to exclude files by relative name and --whitelist to add exceptions to blacklisted names.

This allows you to have additional configuration, which is conditionally integrated. For example, consider the following git repository tree:

|- commong.cfg
|- security.cfg
|- aaaron-cloud.cfg
|- aaaron-cloud/
|  |- overwrites.cfg
|  |- proxy.cfg
|- beebee-cloud.cfg

The aaaron-cloud folder will be ignored by default. You can conditionally include the *-cloud.cfg files like this:

--blacklist '.*-cloud\.cfg' --whitelist 'aaaron-cloud\.cfg'

This allows you to further include the files in aaaron-cloud by using include in aaaron-cloud.cfg:

# aaaron-cloud.cfg
include : $(GIT_CONFIG_CACHE_PATH)/aaaron-cloud/overwrites.cfg
include : $(GIT_CONFIG_CACHE_PATH)/aaaron-cloud/proxy.cfg

This pattern is especially useful when the whitelist is set dynamically, e.g. by using an argument file that contains the domain name.

Argument Files

The condor-git-config executable can use the @ prefix character to read arguments from files. This allows you to prepare options externally

$ cat /etc/condor-git-config/branch
--branch
aaaron-cloud

and have them used dynamically to adjust configuration

include command : condor-git-config @/etc/condor-git-config/branch -- https://git.mydomain.com/condor-repos/condor-configs.git