autoscale
Overview
autoscale is a Python library for composable autoscaling functionality.
Autoscaling is built up from three primitives:
- Reporters convey the state of the world. Cluster size, request latency, free resources, etc.
- Deciders hold business logic. They evaluate the state of a Reporter to make a scaling decision
- Scalers are the doers. They perform scale-up and scale-down actions on AWS, GCE, or your custom infrastructure
Installation
Straight from the cheeseshop:
pip install autoscaleUsage
Writing your own Reporter/Decider/Scaler
See autoscale.py
Writing an autoscaling script
See examples/
Examples
mesos_ec2.py
Scale a Mesos cluster on EC2 based on free disk, memory, and/or CPU. Best run as a cron job run once every N minutes, where N is the longest expected startup or shutdown time of a Mesos slave.
$ mesos_ec2.py --help
usage: mesos_ec2.py [-h] [-l LOG_LEVEL] -u MESOS_URL [-c CPUS] [-d DISK]
[-m MEM] -r REGION -a ASG
optional arguments:
-h, --help show this help message and exit
-l LOG_LEVEL, --log-level LOG_LEVEL
Log level (debug, [default] info, warn, error)
-u MESOS_URL, --mesos-url MESOS_URL
Mesos cluster URL
-c CPUS, --cpus CPUS Comma-delimited CPU thresholds (lower,upper)
-d DISK, --disk DISK Comma-delimited disk thresholds (lower,upper)
-m MEM, --mem MEM Comma-delimited memory thresholds (lower,upper)
-r REGION, --region REGION
AWS region
-a ASG, --asg ASG AWS auto scaling group name
$ mesos_ec2.py \
--mesos-url http://mesos_master.local:5050 \
--cpus 1,3 \
--region us-west-2 \
--asg mesos-MesosSlaveStack-1AB12345ABC-ServerGroup-789XYZ789 \
--log-level info
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): mesos_master.local
INFO:autoscale:State: {'mem_free': 6353, 'disk_free': 35703, 'cpus_free': 8.5}
INFO:autoscale:Thresholds: {'cpus': {'upper': 3, 'lower': 1}}
INFO:autoscale:Should scale by -1
Scaling mesos-MesosSlaveStack-1AB12345ABC-ServerGroup-789XYZ789 in us-west-2 by -1
INFO:autoscale:Current scale: 9
INFO:autoscale:Scaling to 8
