stackql-deploy is an open-source command line utility which implements a declarative, model driven framework to deploy and manage multi cloud stacks using stackql. stackql-deploy is distributed as a Python script to be used as a CLI tool, do the following to get started:

pip install stackql-deploy

StackQL is a utility which allows you to query and interact with cloud and SaaS resources in real time using SQL grammar. StackQL supports a full set of SQL query/DML grammar, including JOIN, UNION adn subquery functionality and supports mutation operations on cloud and SaaS resources such as create, update and delete, implemented as INSERT, UPDATE and DELETE respectively. StackQL also supports grammar for performing lifecycle operations such as starting or stopping a VM using the EXEC statement.

StackQL provider definitions are defined in plaintext OpenAPI extensions to the providers specification. These definitions are then used to generate the SQL schema and the API client. The source for the provider definitions are stored in the StackQL Registry.

A stackql-deploy project is a directory containing StackQL scripts with a manifest file at the root of the directory, for example:

├── example_stack
│   ├── stackql_docs
│   │   └── monitor_resource_group.md
│   ├── stackql_resources
│   │   └── monitor_resource_group.iql
│   ├── stackql_queries
│   │   └── monitor_resource_group.iql
│   ├── stackql_manifest.yml

the stackql_manifest.yml defines the resources in the stackql with their properties which can be scoped by environments, for example:

version: 1
name: example_stack
description: oss activity monitor stack
providers:
    - azure
globals:
    - name: subscription_id
    description: azure subscription id
    value: "{{ vars.AZURE_SUBSCRIPTION_ID }}"
    - name: location
    value: eastus
    - name: resource_group_name_base
    value: "activity-monitor"
resources:
    - name: monitor_resource_group
    description: azure resource group for activity monitor
    props:
        - name: resource_group_name
        description: azure resource group name
        value: "{{ globals.resource_group_name_base }}-{{ globals.stack_env }}"
        # OR YOU CAN DO...
        # values:
        #   prd:
        #     value: "activity-monitor-prd"
        #   sit:
        #     value: "activity-monitor-sit"
        #   dev:
        #     value: "activity-monitor-dev"

use stackql-deploy init {stack_name} to create a project directory with sample files

Deployment orchestration using stackql-deploy includes:

• pre-flight checks, which are StackQL queries that check for the existence or current configuration state of a resource
• deployment scripts, which are StackQL queries to create or update resoruces (or delete in the case of de-provisioning)
• post-deployment tests, which are StackQL queries to confirm that resources were deployed and have the desired state

This process is described here:

graph TB
    A[Start] --> B{foreach\nresource}
    B --> C[preflight\ncheck]
    C --> D{resource\nexists?}
    D -- Yes --> E[run update\nor createorupdate query]
    D -- No --> F[run create\nor createorupdate query]
    E --> G[run postdeploy check]
    F --> G
    G --> H{End}

Mutation operations are defined as .iql files in the stackql_resources directory, these are templates that are rendered with properties or environment context variables at run time, for example:

/*+ create */
INSERT INTO azure.resources.resource_groups(
    resourceGroupName,
    subscriptionId,
    data__location
)
SELECT
    '{{ resource_group_name }}',
    '{{ subscription_id }}',
    '{{ location }}'

/*+ update */
UPDATE azure.resources.resource_groups
SET data__location = '{{ location }}'
WHERE resourceGroupName = '{{ resource_group_name }}'
    AND subscriptionId = '{{ subscription_id }}'

/*+ delete */
DELETE FROM azure.resources.resource_groups
WHERE resourceGroupName = '{{ resource_group_name }}' AND subscriptionId = '{{ subscription_id }}'

Test files are defined as .iql files in the stackql_queries directory, these files define the per-flight and post-deploy checks to be performed, for example:

/*+ preflight */
SELECT COUNT(*) as count FROM azure.resources.resource_groups
WHERE subscriptionId = '{{ subscription_id }}'
AND resourceGroupName = '{{ resource_group_name }}'

/*+ postdeploy, retries=5, retry_delay=5 */
SELECT COUNT(*) as count FROM azure.resources.resource_groups
WHERE subscriptionId = '{{ subscription_id }}'
AND resourceGroupName = '{{ resource_group_name }}'
AND location = '{{ location }}'
AND JSON_EXTRACT(properties, '$.provisioningState') = 'Succeeded'

Once installed, use the build, test, or teardown commands as shown here:

stackql-deploy build prd example_stack -e AZURE_SUBSCRIPTION_ID 00000000-0000-0000-0000-000000000000 --dry-run
stackql-deploy build prd example_stack -e AZURE_SUBSCRIPTION_ID 00000000-0000-0000-0000-000000000000
stackql-deploy test prd example_stack -e AZURE_SUBSCRIPTION_ID 00000000-0000-0000-0000-000000000000
stackql-deploy teardown prd example_stack -e AZURE_SUBSCRIPTION_ID 00000000-0000-0000-0000-000000000000

Note: teardown deprovisions resources in reverse order to creation

Additional options include:

• --dry-run: perform a dry run of the stack operations.
• --on-failure=rollback: action on failure: rollback, ignore or error.
• --env-file=.env: specify an environment variable file.
• -e KEY=value: pass additional environment variables.
• --log-level: logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL), defaults to INFO.

Use stackql-deploy info to show information about the package and environment, for example:

$ stackql-deploy info
stackql-deploy version: 1.0.0
pystackql version     : 3.5.4
stackql version       : v0.5.612
stackql binary path   : /mnt/c/LocalGitRepos/stackql/stackql-deploy/stackql
platform              : Linux x86_64 (Linux-5.15.133.1-microsoft-standard-WSL2-x86_64-with-glibc2.35), Python 3.10.12

Use the --help option to see more information about the commands and options available:

stackql-deploy --help

To get started with stackql-deploy, install it locally using pip:

pip install -e .

pip uninstall stackql-deploy
pip cache purge

To distribute stackql-deploy on PyPI, you'll need to ensure that you have all required files set up correctly in your project directory. This typically includes your setup.py, README.rst, LICENSE, and any other necessary files.

First, ensure you have the latest versions of setuptools and wheel installed:

# pip install --upgrade setuptools wheel
pip install --upgrade build

Then, navigate to your project root directory and build the distribution files:

rm dist/stackql_deploy*
python3 -m build
# or
# python3 setup.py sdist bdist_wheel

This command generates distribution packages in the dist/ directory.

To upload the package to PyPI, you'll need to use twine, a utility for publishing Python packages. First, install twine:

pip install twine

Then, use twine to upload all of the archives under dist/:

twine upload --config-file .pypirc dist/*

Navigate to your docs directory and build the Sphinx documentation:

cd docs
make html

Contributions are welcome and encouraged.

Distributed under the MIT License. See LICENSE for more information.

Get in touch with us via Twitter at @stackql, email us at info@stackql.io or start a conversation using discussions.