DevOps

Tools for deployments and systems administration.

• Free software: Public Domain
• Documentation: https://devops.readthedocs.org.

TODO

• automate adding keys for server users
• use absolute path for copying local ssh and gpg keys to remote machine. currently need to copy keys to the playbook folder for the play to run properly

Getting Started

Clone this repo:

git clone https://github.com/chishaku/devops
cd devops
make install

Create a virtual env (optional):

virtualenv env
. env/bin/activate

Install ansible:

pip install ansible

Update ansible.cfg file to point to your inventory file (hosts.ini)

Then export the ANSIBLE_CONFIG environment variable

export ANSIBLE_CONFIG=/path/to/ansible.cfg

Test connections to hosts from your inventory file:

ansible my_hosts -m ping

The Playbooks

Each playbook is run the same way. First, edit the playbook and fill out the variables.

Then run the playbook:

ansible-playbook playbook.yml

If you want to keep variables separate from the tasks run in the playbook, add the variable values to a separate vars file.

ansible-playbook playbook.yml -e "@vars_file.yml"

Deploy a project

Initial Deployment

• If this project is being pushed to (a) new machine(s)...

  • create a ssh keypair for the remote machine
  • create a gpg keypair for the remote machine

• Update your project_vars.yml file.

ansible-playbook playbooks/project/copy_remote_ssh_keys.yml -e "@project_vars.yml"
ansible-playbook playbooks/project/import_remote_gpg_keys.yml -e "@project_vars.yml"
ansible-playbook playbooks/project/deploy_project_init.yml -e "@project_vars.yml"

Continuous Deployment

ansible-playbook playbooks/project/deploy_project.yml -e "@project_vars.yml"
ansible-playbook playbooks/project/update_crontab.yml -e "@project_vars.yml"

PostgreSQL

Install Postgres

ansible-playbook playbooks/postgres/install_postgres.yml

Update Postgres config

• Copy and edit playbooks/postgres/pg_hba.conf.j2 and playbooks/postgres/postgresql.conf.j2.
• Update the path to these config files in your postgres_vars.yml file.

ansible-playbook playbooks/postgres/update_config.yml -e "@postgres_vars.yml" --become-user=postgres

Add or remove users

• Edit add_postgres_users or remove_postgres_users in your postgres_vars.yml file.

ansible-playbook playbooks/postgres/add_users.yml -e "@postgres_vars.yml" --become-user=postgres
ansible-playbook playbooks/postgres/remove_users.yml -e "@postgres_vars.yml" --become-user=postgres

Add or remove databases

• Edit add_postgres_databases or remove_postgres_databases in your postgres_vars.yml file.

ansible-playbook playbooks/postgres/add_databases.yml -e "@postgres_vars.yml" --become-user=postgres
ansible-playbook playbooks/postgres/remove_databases.yml -e "@postgres_vars.yml" --become-user=postgres

Redis

• Edit values of redis.conf in your redis_vars.yml file.

ansible-playbook playbooks/redis/install_redis.yml -e "@redis_vars.yml"
ansible-playbook playbooks/redis/update_config.yml -e "@redis_vars.yml"

Supervisor

• Edit values of supervisord.conf in your supervisor_vars.yml file.
• Add supervisor_programs.conf file and add to supervisor_vars.yml file.

ansible-playbook playbooks/supervisor/install_supervisor.yml -e "@supervisor_vars.yml"
ansible-playbook playbooks/supervisor/update_config.yml -e "@supervisor_vars.yml"
ansible-playbook playbooks/supervisor/update_programs.yml -e "@supervisor_vars.yml"

Elasticsearch

Install:

ansible-playbook playbooks/elasticsearch/install_elasticsearch.yml -e "@vars.yml" -s

Test:

curl -X GET 'http://107.170.126.239:9200'

Bootstrap servers

The bootstrap_server.yml playbook is a wrapper around many tasks you would want to run after first provisioning a server.

The tasks already included in this playbook are:

• update_users.yml
• update_sshd.yml
• install_packages.yml
• create_directories.yml
• copy_ssh_keypair_to_remote.yml

To run this playbook, first edit the vars in each included playbook or add all vars for each playbook into a single vars file.

Then run one of the following:

ansible-playbook bootstrap_server.yml
ansible-playbook bootstrap_server.yml -e "@my_vars_file.yml"

Update Users and Keys

Update SSH Configuration

Create Directories

Install Packages

0. Preliminary steps

Need to automate these steps. Install ubuntu from image with python already installed and change root password in first step of boostrap playbook

` passwd apt-get update apt-get install sudo build-essential python-dev # Need python for ansible playbooks to work `

1. Update users and packages

``` ansible-playbook -i hosts.ini -vvvv -k -e '@vars/vars_file.yml' playbooks/bootstrap.yml

-k: ask for sudo pass ```

Playbook includes the following tasks:

• update users (users, groups, sudoers, keys)
• update packages

2. Manually verify passwords and keys work

3. Update ssh config

` ansible-playbook -i hosts.ini -vvvv -e '@vars/vars_file.yml' playbooks/update_sshd.yml `

• Confirm root login and password auth no longer possible
• Update inventory list with any port changes.

Note: When confident that step 2 isn't necessary, combine this into step 1.

4. More security

• lock down all ports
• configure fail2ban and iptables
• local forward for mysql

Questions:

• Is key only auth and one obscure user and pass sufficient security?
• http://security.stackexchange.com/questions/18036/creating-user-specific-authentication-methods-in-ssh

5. Configure mysql

• update mysql config
• add mysql users with strict privileges

``` - name: Setup MySQL users

mysql_user: name={{ item.0.name }} password={{ item.0.mysql.password }} host={{ item.1 }} priv={{ item.0.mysql.privs | join('/') }} with_subelements:

• users
• mysql.hosts

```

Notes

• [How to Manage SSH Keys][0]
• [Ansible Loops][1]
• [Passing variables on the command line][2]

[0]: http://blog.appliedinformaticsinc.com/how-to-manage-ssh-keys-using-ansible/ [1]: http://docs.ansible.com/ansible/playbooks_loops.html [2]:http://docs.ansible.com/ansible/playbooks_variables.html#passing-variables-on-the-command-line