Introduction

This project implements an ElasticSearch Authentication and Authorization reverse proxy, that can be put in front of any ElasticSearch installation, without the need to install a plugin into ElasticSearch. Authentication is done vie HTTP Basic AUTH.

IT consists of 5 main components:

• the elasticsearch server that we like to secure:
• el_aap: the reverse proxy that will secure our elasticsearch instance
• el_aap_api: the rest api to manage users, roles and permissions
• el_aap_cli: a command line tool, to manage users, roles and permission rules
• mongodb to store all user, role and permission related stuff

The project is in a very early stage, but all features have been tested and seem to work as expected.

TODO:

• write tests

The code has been developed and tested on python3.4

API Documentation

REST API documentation of the api part can be discovered at:

http://$HOST:$PORT/elaap/static/swagger/index.html#/

To use the api you have to login as a user that has an "admin" flag set to true, like the "default_admin"

Requirements

• Running MongoDB
• Running Elasticsearch
• Python 3.x

Basic Installation

pip install el-aap

Configuration

el_aap, and el_aap_api are both expecting there configuration in /etc/el_aap/.

Example configuration for /etc/el_aap/el_aap.ini

[main]
dlog = el_aap.dlog
port = 9201

# Elasticsearch endpoint we like to secure.
[elasticsearch]
host = 127.0.0.1
port = 9200
scheme = http

[file:logging]
acc_log = el_aap_access.log
acc_retention = 7
app_log = el_aap_error.log
app_retention = 7
app_loglevel = DEBUG

# "main" MongoDB connection pool
[main:mongopool]
hosts = mongodb://localhost:27017,localhost:27018
db = el_aap
# pass =
# user =

# the permissions collection should use the "main" MongoDB pool, and the collection name "permission"
[permissions:mongocoll]
coll = permissions
pool = main

# the roles collection should use the "main" MongoDB pool, and the collection name "roles"
[roles:mongocoll]
coll = roles
pool = main

# the users collection should use the "main" MongoDB pool, and the collection name "users"
[users:mongocoll]
coll = users
pool = main

Example configuration for /etc/el_aap/el_aap_api.ini

[main]
dlog = el_aap_api_.dlog
# if not set, static file delivery is disabled
static_path = /path/to/static
port = 9000

# if this section is configured, password recovery feature via email is enabled
# [pw_recovery]
# api_url = http://el_aap_api.example.com
# www_url = http://el_aap_www.example.com/reset_password/
# from = elaap-pwrecovery@example.com
# subject = pw_recovery@example.com
# text_tmpl = /path/to/email/template/text_templ.jinja2
# html_tmpl = /path/to/email/template/html_templ.jinja2
# smtp_host = 127.0.0.1
# smtp_port = 10025

[file:logging]
acc_log = el_aap_api_access.log
acc_retention = 7
app_log = el_aap_api_error.log
app_retention = 7
app_loglevel = DEBUG

# "main" MongoDB connection pool
[main:mongopool]
hosts = mongodb://localhost:27017,localhost:27018
db = el_aap
# pass =
# user =

# the permissions collection should use the "main" MongoDB pool, and the collection name "permission"
[permissions:mongocoll]
coll = permissions
pool = main

# the roles collection should use the "main" MongoDB pool, and the collection name "roles"
[roles:mongocoll]
coll = roles
pool = main

# the users collection should use the "main" MongoDB pool, and the collection name "users"
[users:mongocoll]
coll = users
pool = main

# the sessions collection should use the "main" MongoDB pool, and the collection name "sessions"
[sessions:mongocoll]
coll = sessions
pool = main

# the lostpw collection should use the "main" MongoDB pool, and the collection name "lostpw"
[lostpw:mongocoll]
coll = lostpw
pool = main

The only things that needs to be adjusted are the IP and port configurations of MongoDB and Elasticsearch. Everything else should stay unchanged. The MongoDB sections in el_aap.ini have to match there counterparts in el_aap_api.ini and vise versa, except for the session section, that is only needed in el_aap_api.ini

Pre Running

We assume that you already have a running Elasticsearch and MongoDB, setting up these is out of scope, and described well enough in other places.

First Run

Before the first run, you have to create a default user and all the required indexes, this is done via:

el_aap_api indicies
el_aap_api create_admin

The first command will create all the required indexes, the second one will create a default user called "default_admin" with the password "password" that has the admin flag set to True.

Running

Both executables can be found in the contrib folder. The Proxy is called el_aap, and the api/configuration interface is called el_aap_api.

Help can be discovered thru calling contrib/el_aap(_api) --help

Both can be started with:

contrib/el_aap(_api) start

Both can be stopped with:

contrib/el_aap(_api) quit

Running them in developement mode:

contrib/el_aap(_api) devel

In developement mode, both will not daemonize and write errors and access logs to stdout. The default pidfile of both daemons is written to "/var/run/el_aap/el_aap_api.pid" or "/var/run/el_aap/el_aap.pid". Make sure you have created the folder "/var/run/el_aap/" before you try to start the daemons.

Managing Users, Roles and Permissions

To manage the proxy configuration via the REST API you need a user that has the admin flag set to "True", all other user will only be able to authenticate against the reverse proxy itself via HTTP Basic AUTH.

Per default, no roles, and no permission rules are defined, so basically all access to elasticsearch via the proxy is denied!

Also notice that all user, roles, and passwords are cached in the reverse proxy for 2 Minutes, so making changes to passwords, roles, and other may take up to 2 minutes to become visible!

Managing users, roles and permissions can either be done directly via the REST API, or vie the command line tool called el_aap_cli.

Here we describe the usage of the el_aap_cli.

To use the el_aap_cli tool, you have to create a file with the following content:

~/.el_aap_cli.ini

[main]
endpoint = http://127.0.0.1:9000/
user = default_admin
pass = password

This file contains your username and password, make sure no one has access to this file. You should use the command line tool only from you local workstation, and a user where only you have access to! You have been warned!

The el_aap_cli help can be called via

el_aap_cli --help
ElasticSearch Authentication and Authorization command line utility


positional arguments:
  {permissions,roles,users}
                        commands
    permissions         manage permissions
    roles               manage roles
    users               manage users

optional arguments:
  -h, --help            show this help message and exit

The Options are structured into sub commands, that all except --help to give future details.

Manage Users

This will create a user called "adminuser" with the password "password", that has the admin flag set. this means that this admin can also manage other users, roles and permissions

el_aap_cli users add --id adminuser --admin --email admin_user@example.com --name "Administrative User" --password password

This will create a user called "test_user" with the password "password", without the admin flag set.

el_aap_cli users add --id test_user --email test_user@example.com --name "Test User" --password password

see the following command for a detailed list of available options.

el_api_cli users --help
el_api_cli users {subcommand} --help

Manage Roles

This will create a new role called "testrole1" that has the admin_user and the default_admin as member.

el_aap_cli roles add --id testrole1 --description "test role 1" --users admin_user,default_admin

This will add the "test_user to the "testrole1".

el_aap_cli roles add_users --id testrole1 --users test_user

This will remove the "default_admin" from the test_role1.

el_aap_cli roles del_users --id testrole1 --users default_admin

see the following command for a detailed list of available options.

el_api_cli roles --help
el_api_cli roles {subcommand} --help

Manage Permissions

Permissions grand Roles access to Indexes, based and the given scope, where the scope is a regular expression that is matched against the index name, and the assigned permissions to the permission Rule.

Here is a list of all available permissions that can be used:

• :
• :cluster:
• :cluster:monitor
• :index:
• :index:crud:
• :index:crud:create
• :index:crud:read
• :index:crud:update
• :index:crud:delete
• :index:crud:search
• :index:manage:
• :index:manage:monitor

The :index:crud:* permissions grant access to CRUD related Index operations, including Searches. ":index:crud:" includes all CRUD and Search related operations

The :index:manage:monitor permissions can be used to execute index monitoring related tasks, like reading the list of aliases for an index. The :index:manage: permissions allows the creation of new index aliases and the like, it also includes the :index:manage:monitor privilege.

The :index: privilige contains the index CRUD and index manage privileges.

:cluster:monitor grants acces to cluster monitoring related endpoints. :cluster: grants access to all cluster related tasks, including monitoring.

The : permissions grants access to all of the above permissions.

el_api_cli permissions --help
el_api_cli permissions {subcommand} --help

Creating a Permission Rule

Create a permission rule which grants read and search permissions to all indexes and index aliases that match apache-acces* to the role testrole1

el_aap_cli permissions add --id apache-access-read --description "Grant Read Permissions to the Apache Access Log Index" --scope 'apache-access*' --roles testrole1 --permissions :index:crud:read,:index:crud:search

Author

Stephan Schultchen stephan.schultchen@gmail.com

License

Unless stated otherwise on-file el_app uses the MIT license, check LICENSE file.

Contributing

If you'd like to contribute, fork the project, make a patch and send a pull request