cloud-custodian/cloud-custodian


Rules engine for cloud security, cost optimization, and governance, DSL in yaml for policies to query, filter, and take actions on resources

https://cloudcustodian.io

License: Apache-2.0

Language: Python

Keywords: aws, azure, cloud, cloud-computing, compliance, gcp, lambda, management, rules-engine, serverless


Join the chat at https://gitter.im/capitalone/cloud-custodian Build Status License Coverage Requirements Status

Cloud Custodian

Cloud Custodian is a rules engine for managing public cloud accounts and resources. It allows users to define policies to enable a well managed cloud infrastructure, that's both secure and cost optimized. It consolidates many of the adhoc scripts organizations have into a lightweight and flexible tool, with unified metrics and reporting.

Custodian can be used to manage AWS, Azure, and GCP environments by ensuring real time compliance to security policies (like encryption and access requirements), tag policies, and cost management via garbage collection of unused resources and off-hours resource management.

Custodian policies are written in simple YAML configuration files that enable users to specify policies on a resource type (EC2, ASG, Redshift, CosmosDB, PubSub Topic) and are constructed from a vocabulary of filters and actions.

It integrates with the cloud native serverless capabilities of each provider to provide for real time enforcement of policies with builtin provisioning. Or it can be run as a simple cron job on a server to execute against large existing fleets.

Engineering the Next Generation of Cloud Governance” by @drewfirment

Features

  • Comprehensive support for public cloud services and resources with a rich library of actions and filters to build policies with.
  • Supports arbitrary filtering on resources with nested boolean conditions.
  • Dry run any policy to see what it would do.
  • Automatically provisions serverless functions and event sources ( AWS CloudWatchEvents, AWS Config Rules, Azure EventGrid, GCP AuditLog & Pub/Sub, etc)
  • Cloud provider native metrics outputs on resources that matched a policy
  • Structured outputs into cloud native object storage of which resources matched a policy.
  • Intelligent cache usage to minimize api calls.
  • Supports multi-account/subscription/project usage.
  • Battle-tested - in production on some very large cloud environments.

Links

Quick Install

$ virtualenv --python=python2 custodian
$ source custodian/bin/activate
(custodian) $ pip install c7n

Usage

First a policy file needs to be created in YAML format, as an example:

policies:
- name: remediate-extant-keys
  description: |
    Scan through all s3 buckets in an account and ensure all objects
    are encrypted (default to AES256).
  resource: aws.s3
  actions:
    - encrypt-keys

- name: ec2-require-non-public-and-encrypted-volumes
  resource: aws.ec2
  description: |
    Provision a lambda and cloud watch event target
    that looks at all new instances and terminates those with
    unencrypted volumes.
  mode:
    type: cloudtrail
    events:
        - RunInstances
  filters:
    - type: ebs
      key: Encrypted
      value: false
  actions:
    - terminate

- name: tag-compliance
  resource: aws.ec2
  description: |
    Schedule a resource that does not meet tag compliance policies
    to be stopped in four days.
  filters:
    - State.Name: running
    - "tag:Environment": absent
    - "tag:AppId": absent
    - or:
      - "tag:OwnerContact": absent
      - "tag:DeptID": absent
  actions:
    - type: mark-for-op
      op: stop
      days: 4

Given that, you can run Cloud Custodian with:

# Validate the configuration (note this happens by default on run)
$ custodian validate policy.yml

# Dryrun on the policies (no actions executed) to see what resources
# match each policy.
$ custodian run --dryrun -s out policy.yml

# Run the policy
$ custodian run -s out policy.yml

You can run it with Docker as well

# Download the image
$ docker pull cloudcustodian/c7n

# Run the policy
$ docker run -it \
    -v $(pwd)/output:/output \
    -v $(pwd)/policy.yml:/policy.yml \
    --env-file <(env | grep "^AWS") \
    cloudcustodian/c7n run -v -s /output /policy.yml

Custodian supports a few other useful subcommands and options, including outputs to S3, Cloudwatch metrics, STS role assumption. Policies go together like Lego bricks with actions and filters.

Consult the documentation for additional information, or reach out on gitter.

Get Involved

Additional Tools

The Custodian project also develops and maintains a suite of additional tools here https://github.com/capitalone/cloud-custodian/tree/master/tools:

Org
Multi-account policy execution.
PolicyStream
Git history as stream of logical policy changes.
Salactus
Scale out s3 scanning.
Mailer
A reference implementation of sending messages to users to notify them.
TrailDB
Cloudtrail indexing and timeseries generation for dashboarding.
LogExporter
Cloud watch log exporting to s3
Index
Indexing of custodian metrics and outputs for dashboarding
Sentry
Cloudwatch Log parsing for python tracebacks to integrate with https://sentry.io/welcome/

Contributors

We welcome Your interest in Capital One’s Open Source Projects (the “Project”). Any Contributor to the Project must accept and sign an Agreement indicating agreement to the license terms below. Except for the license granted in this Agreement to Capital One and to recipients of software distributed by Capital One, You reserve all right, title, and interest in and to Your Contributions; this Agreement does not impact Your rights to use Your own Contributions for any other purpose.

Sign the Individual Agreement

Sign the Corporate Agreement

Code of Conduct

This project adheres to the Open Code of Conduct. By participating, you are expected to honor this code.

Project Statistics

Sourcerank 11
Repository Size 109 MB
Stars 2,044
Forks 595
Watchers 148
Open issues 478
Dependencies 245
Contributors 188
Tags 65
Created
Last updated
Last pushed

Top Contributors See all

Kapil Thangavelu Joshua Root scotwk Sonny Chad Whitacre Stefan Gordon Mandeep Bal Kit Ewbank Alfred Gamulo Kirill Logachev Colin MacDonald aluong Jamison Roberts erwelch Tanner Barlow Kyle Travis David Filiatrault Rob McBroom JohnTheodore Darcy Laycock

Packages Referencing this Repo

c7n
Cloud Custodian - Policy Rules Engine
Latest release 0.8.43.1 - Updated - 2.04K stars
c7n-autodoc
Cloud Custodian - Automated Policy Documentation
Latest release 0.3 - Published - 2.04K stars
c7n-org
Cloud Custodian - Multi Account
Latest release 0.5.1 - Updated - 2.04K stars
c7n-mailer
Cloud Custodian - Reference Mailer
Latest release 0.5.0 - Updated - 2.04K stars
c7n-azure
Cloud Custodian - Azure Support
Latest release 0.5.2 - Updated - 2.04K stars
c7n_org
Cloud Custodian - Multi Account
Latest release 0.5.0 - Updated - 2.04K stars
c7n_azure
Cloud Custodian - Azure Support
Latest release 0.3 - Updated - 2.04K stars
c7n_traildb
Cloud Custodian - Cloud Trail Tools
Latest release 0.1 - Published - 2.04K stars
c7n-kube
Cloud Custodian - Multi Account
Latest release 0.1 - Published - 2.04K stars
c7n_mailer
Cloud Custodian - Reference Mailer
Latest release 0.3.3 - Updated - 2.04K stars
c7n_logexporter
Cloud Custodian - Cloud Watch Log S3 exporter
Latest release 0.2 - Updated - 2.04K stars
c7n_salactus
Cloud Custodian - Salactus S3
Latest release 0.3.0 - Updated - 2.04K stars
c7n-policystream
Cloud Custodian - Git Commits as Logical Policy Changes
Latest release 0.3.1 - Updated - 2.04K stars
c7n-gcp
Cloud Custodian - Multi Account
Latest release 0.2.2 - Updated - 2.04K stars
c7n-guardian
Cloud Custodian - Multi Account Guard Duty Setup
Latest release 0.3 - Updated - 2.04K stars
c7n_gcp
Cloud Custodian - Multi Account
Latest release 0.2 - Updated - 2.04K stars

Recent Tags See all

0.8.43.1 April 17, 2019
0.8.43.0 April 13, 2019
0.8.42.1 March 07, 2019
0.8.42.0 March 04, 2019
0.8.41.0 February 18, 2019
0.8.40.0 January 31, 2019
0.8.33.0 December 31, 2018
0.8.32.1 November 28, 2018
0.8.32.0 November 14, 2018
0.8.31.2 September 22, 2018
0.8.31.1 September 17, 2018
0.8.31.0 September 12, 2018
untagged-08b50b9d230f7ddf28de August 12, 2018
0.8.30.0 August 12, 2018
0.8.28.2 April 19, 2018

Something wrong with this page? Make a suggestion

Last synced: 2019-04-17 12:36:50 UTC

Login to resync this repository