Please take a note on the change log in the bottom of the document in case you had used a previous version
Quick Start
docker run -p 5001:5001 bhujay/tokenleader
to run in background
docker run -d -p 5001:5001 bhujay/tokenleader
it is installed with default user use1 and password user1
once it is running install the client in a venv and test the features . consult the client installation doc https://github.com/microservice-tsp-billing/tokenleaderclient
Manual installation steps
optional Steps:
virtualenv -p python3 venv
source venv/bin/activate
pip install --upgrade pip
installtion:
pip install tokenleader ( create virtual env if required)
required configurations
create the following directories and files under /etc folder.
- ssh-keygen < press enter to select all defaults>
- /etc/tokenleader/tokenleader_settings.ini
- /etc/tokenleader/role_to_acl_map.yml
- /etc/tokenleader/client_configs.yml
- run tokenleader-auth - u - p --url localhost:5001
sample configuration of each files
configure the /etc/tokenleader/tokenleader_settings.ini
sudo mkdir /etc/tokenleader
sudo vi /etc/tokenleader/tokenleader_settings.ini
[flask_default]
host_name = localhost
host_port = 5001
# ssl not required since the production deployment will be behind the apache with ssl
# This is required only when flask is started without apache for testing
# put enabled for enabling ssl
ssl = disabled
ssl_settings = adhoc
[token]
# default will take the id_rsa keys from the users home directory and .ssh directiry
# put the file name here if the file name is different
#also the public ley need to be copied in the client settings file under /etc/tlclient
private_key_file_location = default
public_key_file_location = default
#use full path when deployed with apache
#private_key_file_location = /home/bhujay/.ssh/id_rsa
#public_key_file_location = /home/bhujay/.ssh/id_rsa.pub
[db]
#change the database string as appripriate for your production environment
#contributors are requested to put some more example here
SQLALCHEMY_DATABASE_URI = sqlite:////tmp/auth.db
SQLALCHEMY_TRACK_MODIFICATIONS = False
/etc/tokenleader/role_to_acl_map.yml
sudo mkdir /etc/tokenleader
sudo vi /etc/tokenleader/role_to_acl_map.yml
maintain at least one role and one entry in the following format
- name: role1
allow:
- tokenleader.adminops.adminops_restapi.list_users
- name: role2
allow:
- service1.third_api.rulename3
- service1.fourthapi_api.rulename4
/etc/tokenleader/client_configs.yml which holds the non secret configs about the client and looks as
user_auth_info_file_location: <change this location to users home dir , two files will be generated and stored here> fernet_key_file: tl_public_key: copy the public key of the server <cat sh/id_rsa.id> and paste the key here
sudo vi
sudo vi /etc/tokenleader/client_configs.yml
user_auth_info_from: file # OSENV or file , leave it as file
user_auth_info_file_location: /home/bhujay/tlclient/user_settings.ini # change this location to users home dir
fernet_key_file: /home/bhujay/tlclient/prod_farnetkeys
tl_public_key: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYV9y94je6Z9N0iarh0xNrE3IFGrdktV2TLfI5h60hfd9yO7L9BZtd94/r2L6VGFSwT/dhBR//CwkIuue3RW23nbm2OIYsmsijBSHtm1/2tw/0g0UbbneM9vFt9ciCjdq3W4VY8I6iQ7s7v98qrtRxhqLc/rH2MmfERhQaMQPaSnMaB59R46xCtCnsJ+OoZs5XhGOJXJz8YKuCw4gUs4soRMb7+k7F4wADseoYuwtVLoEmSC+ikbmPZNWOY18HxNrSVJOvMH2sCoewY6/GgS/5s1zlWBwV/F0UvmKoCTf0KcNHcdzXbeDU9/PkGU/uItRYVfXIWYJVQZBveu7BYJDR bhujay@DESKTOP-DTA1VEB
ssl_verify: False # leave it as is
users authentiaction information . The file is generated using an cli
tokenleader-auth -u user1 -p user1 --url https://localhost:5001
the file , /home/bhujay/tlclient/user_settings.ini , will be auto generated and will looks like this :
[DEFAULT]
tl_user = user1
tl_url = http://localhost:5001
tl_password = gAAAAABcYnpRqet_VEucowJrE0lM1RQh2j5E-_Al4j8hm8vJaMvfj2nk7yb3zQo95lBFDoDR_CeoHVRY3QBFFG-p9Ga4bkJKBw==
note that the original password has been encrypted before saving in the file. if the keyfile is lost or the password is forgotten the file has to be deleted and recreated. Accordingly the users password in the tokenleader server also to be changed.
from the root directory of tokenleader, change the name of org , ou , dept , wfc , role and user as per your need
TO set up the tokenleqder the following entities need to be registered in sequence adminops -h provides help to understand the various options of admin function os tokenleader
adminops initdb
adminops add org -n org1
adminops add ou -n ou1
adminops add dept -n dept1
adminops addwfc -n wfc1 --wfcorg org1 --wfcou ou1 --wfcdept dept1
adminops list wfc -n wfc1
adminops add role -n role1
adminops adduser -n user1 --password user1 --emailid user1 --rolenames role1 --wfc wfc1
adminops addservice -n tokenleader --password tokenleader --urlint localhost:5001
start the service :
tokenleader-start
Test it is working
from the cli :
tokenleader gettoken
tokenleader verify -t <paste the token here>
tokenleader list user
then from python shell it works as follows:
>>> from tokenleaderclient.client.client import Client
>>> c = Client()
>>> c.get_token()
{'message': 'success', 'status': 'success', 'auth_token': 'eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzUxMiJ9.eyJpYXQiOjE1NDk5NjcxODAsImV4cCI6MTU0OTk3MDc4MCwic3ViIjp7IndmYyI6eyJvcmd1bml0Ijoib3UxIiwibmFtZSI6IndmYzEiLCJkZXBhcnRtZW50IjoiZGVwdDEiLCJpZCI6MSwib3JnIjoib3JnMSJ9LCJlbWFpbCI6InVzZXIxIiwiaWQiOjEsInVzZXJuYW1lIjoidXNlcjEiLCJyb2xlcyI6WyJyb2xlMSJdfX0.gzW0GlgR9qiNLZbR-upuzgHMw5rOm2luV-EnHZwlOSJ-0kJnHsiiT5Wk-HZaqMGZd0YJxA1e9GMroHixtj7WJsbLLjhgqQ5H1ZprCkA9um6-vdkwAFVduWIqIN7S6LbsE036bN7y4cdgVhuJAKoiV1KyxOU1-Hxid5l3inL0Hx2aDUrZ3InzFKBw7Mll86xWdfkpHSdyVjVuayKQMvH2IdT3N15k4O2tSwV3t6UhG6MO0ngHFt3LFR471QWGzJ8UyRzqyqbheuk5vwPk684MfRclCtKx33LWAMf-HXQgVA2py_NzmEiY1ROsKmZqpbIO9YKIO_aFCmzB7DQSI8dcYg', 'service_catalog': {'tokenleader': {'endpoint_url_external': 'localhost:5001', 'endpoint_url_admin': None, 'id': 2, 'endpoint_url_internal': None, 'name': 'tokenleader'}, 'micros1': {'endpoint_url_external': 'localhost:5002', 'endpoint_url_admin': None, 'id': 1, 'endpoint_url_internal': None, 'name': 'micros1'}}}
>>> c.verify_token('eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzUxMiJ9.eyJpYXQiOjE1NDk5NjcxODAsImV4cCI6MTU0OTk3MDc4MCwic3ViIjp7IndmYyI6eyJvcmd1bml0Ijoib3UxIiwibmFtZSI6IndmYzEiLCJkZXBhcnRtZW50IjoiZGVwdDEiLCJpZCI6MSwib3JnIjoib3JnMSJ9LCJlbWFpbCI6InVzZXIxIiwiaWQiOjEsInVzZXJuYW1lIjoidXNlcjEiLCJyb2xlcyI6WyJyb2xlMSJdfX0.gzW0GlgR9qiNLZbR-upuzgHMw5rOm2luV-EnHZwlOSJ-0kJnHsiiT5Wk-HZaqMGZd0YJxA1e9GMroHixtj7WJsbLLjhgqQ5H1ZprCkA9um6-vdkwAFVduWIqIN7S6LbsE036bN7y4cdgVhuJAKoiV1KyxOU1-Hxid5l3inL0Hx2aDUrZ3InzFKBw7Mll86xWdfkpHSdyVjVuayKQMvH2IdT3N15k4O2tSwV3t6UhG6MO0ngHFt3LFR471QWGzJ8UyRzqyqbheuk5vwPk684MfRclCtKx33LWAMf-HXQgVA2py_NzmEiY1ROsKmZqpbIO9YKIO_aFCmzB7DQSI8dcYg')
{'payload': {'iat': 1549967180, 'exp': 1549970780, 'sub': {'username': 'user1', 'roles': ['role1'], 'id': 1, 'email': 'user1', 'wfc': {'orgunit': 'ou1', 'id': 1, 'org': 'org1', 'department': 'dept1', 'name': 'wfc1'}}}, 'message': 'Token has been successfully decrypted', 'status': 'Verification Successful'}
>>>
What it does
tokenleader has three simple operations:
- recieves users request , autehnticates her and provides a token which carries more users informations such as
a) user's roles ( one user can have multiple roles, although most of the cases one will suffice)
b) user is also mapped with a wfc ( work function context)
wfc is a combination of organization name, organization unit name departname
A typical token request call is :
curl -X POST -d '{"username": "admin", "password": "admin"}' \
-H "Content-Type: Application/json" localhost:5001/token/gettoken
The validity period of the token can be set through the settings.ini in future , currently it is fixed as one hour.
Before a token can be recived , user need to be registered in the token leader server following the steps shown later section of this docuement.
-
receives a token from user , can validate and unencrypt the users information.
-
maintains a catalog for all the microservies . The entry for services , it includes service name , servie account password ( we have to see if this is required at all) , url for the service endpoint. A client can query tokenleader by service name and will thus get the url for the service .
For each service end point three url can be registered , one for internal , this is the default url . External url , when you want to segregate the users network from service network and another is admin network , which can be further separated from the above two network
token can be used for authenticating an user wiithout the need for user to enter password
To verify token:
curl -H "X-Auth-Token:<paste toekn here>" localhost:5001/token/verify_token
tokenleader has a client which is automatically installed with the server , this provides a python api for making hte call and verifying the token. The client also has the RBAC enforcer for authorization. read more about the client here -
https://pypi.org/project/tokenleaderclient
https://github.com/microservice-tsp-billing/tokenleaderclient
Why token service and how it works
in situtaions where a service or a client need to make several http /REST call to an application/service(microservice)/server or to multiple applications/services/servers, sending the user name and password repeatedly over the http traffic is not desiarable, neither it is good to store the user name and password in servers session for a stateless application. In thses cases token based authentication helps.
Once an user or service obtain a token, subsequent calls to the server or even to different servers can be made using the token instead of username and password. The server then will make a validation call to tokenleader for authentication and also will retrieve role name and wfc information.
The information retrieved from the token leader then can be used by the server for granting proper authorization to the server resources . Therefore authentication is handled by the tokenleader application whereas the authorization is handled by the applicaion being served to the user.
each application uses a local role to acl map. For each api route there is one acl name which either deny or permits the http call to the api route . further to control how much data to be given access to the user , the wfc details will be used for filtering the data query ( mainly data persistance and query)
For the developer
For authorization , there is a enforcer decorator to be used by each microservice .
A sample microsdervice with this decoraator has been shown in micros1 repo . Any api route which is bind
with this decorator will retrieve role and wfc from the tokenleader service.
The role will be used by the decorator to compare with the local acl map yml file for allowing or denying the
access to api route url.
The wfc will be passed to the api route function for later usage by the function for database query filtering.
The api route function must have a keyword argument 'wfc' for the enforcer decorator to work.
Example :
@bp1.route('/test1', methods=[ 'POST'])
@authclient.enforce_access_rule_with_token(<'rulename'> )
def acl_enforcer_func_for_test(wfc=None):
'''
the rule name in this case should be :
'pkgname.modulename.classname.acl_enforcer_func_for_test'
for each api route functions the parameter wfc must be present
'''
msg = ("enforcer decorator working ok with wfc org = {},"
"orgunit={}, dept={}".format(wfc.org, wfc.orgunit, wfc.department))
return msg
In the above example, the decorator impose aceess control on the route /test1 .
role name for the user is retrived from the token leader , compared with the rule to acl map yml file (/etc/tokenleaderclient/role_acl_map_file.yml) which is maintained locally in the server where the service is running .
the role_to_acl_map file maps the api route function names to and looks like :
- name: role1
allow:
- pkgname.modulename1.acl_enforcer_func_for_test
- pkg1.module1.acl_enforcer_func_for_test
check the sample data and test cases inside the tokenleaderclient for better understanding. tokenleader server ( this repo) it self uses the tokenleader client for enforcing the rbac for many api routes , for example adding users , listing users etc. Check the tokenleader/app1/adminops/adminops_restapi.py file to get a better understanding or mail me your query at bhujay.bhatta@yahoo.com
decorator alos retrived work function context for the user from tokenleader and passed it to original route function acl_enforcer_func_for_test . The route function mandatorily to have a parameter called wfc as argument for the wfc , to get the value from the decorator.
now within the acl_enforcer_func_for_test funtion , wfc attributes like org, orgunit and department is used to display a message. They actually to be used for database query filtering so that based on the work function user is able to view only relevant information.
List of api routes and their rules
- /list/users acl rule name - tokenleader.adminops.adminops_restapi.list_users
use the follwoing :
To check the database objects from shell, and to see that the relational properties are working properly /microservice-tsp-billing/tokenleader$ flask shell
from app1.authentication import models
from app1.authentication.models import User, Role, Workfunctioncontext, Organization, OrgUnit, Department
r1 = Role.query.filter_by('role1').first()
r1 = Role.query.filter_by(rolename='role1').first()
r1
#<Role role1>
To generate token using curl :
curl -X POST -d '{"username": "admin", "password": "admin"}' \
-H "Content-Type: Application/json" localhost:5001/token/gettoken
what you get from tokenleader:
{'service_catalog': {
'microservice1': {'id': 1,
'name': 'microservice1',
'endpoint_url_external': 'localhost/5000',
'endpoint_url_admin': 'localhost/5000',
'endpoint_url_internal': 'localhost/5000'}},
'message': 'success',
'auth_token': 'eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzUxMiJ9.eyJpYXQiOjE1NDk4Njg5MDYsInN1YiI6eyJpZCI6MSwiZW1haWwiOiJ1MUBhYmMuY29tIiwicm9sZXMiOlsicm9sZTEiXSwid2ZjIjp7ImlkIjoxLCJuYW1lIjoid2ZjMSIsIm9yZyI6Im9yZzEiLCJvcmd1bml0Ijoib3UxIiwiZGVwYXJ0bWVudCI6ImRlcHQxIn0sInVzZXJuYW1lIjoidTEifSwiZXhwIjoxNTQ5ODcyNTA2fQ.BBtTUcu8kUz__sbHmC8sB111C4Yzk6Fth5DjOoLCCTygqDjj-gQOS3x6T7e8rpKmHtf0LrDWPWFCmhIIqD2I8DuK4U4b-Hk7gbKYIVsvqL3DksOVF2SSe_6v4nNbJR50Q8mYrYQz0yijj-KQHj0Gc1FVCaBSXeIbA-uAUmSpQKCBDRqJbayK85e4dSoILpKL_Q1_JT4qqM7OwnGq05akJrosohNGKxp46gBex9l5iTPkoRgvQk-p1H61MMTdLKZIr9CmjIReXBBzfla6LoX8Siur_Lb4o1r0PJUcok-w69h_QCEqLe9VX9e4zFWnXIpDj5nwKqnj0JRKNvMw5VTcHA',
'status': 'success'}
To verify token:
curl -H "X-Auth-Token:<paste toekn here>" localhost:5001/token/verify_token
How the verified toekn data looks like :
{
"message": "Token has been successfully decrypted",
"payload": {
"exp": 1549382308,
"iat": 1549378708,
"sub": {
"email": "u1@abc.com",
"id": 1,
"roles": [
"role1"
],
"username": "u1",
"wfc": {
"department": "dept1",
"id": 1,
"name": "wfc1",
"org": "org1",
"orgunit": "ou1"
}
}
},
"status": "Verification Successful"
}
for initial setup or when db model is changed
for db migration
flask db init
flask db migrate -m < COMMENT >
flask db upgrde
if there is a change in db structure, and a migration is done , commit and push the migration directory to the git
from the machine where migration was done.
For development machine with sqllite db , there are chalenges in migration due to lil8mitiaton of database
alter capabilities inherent to sqllite. So sometimes , delelting the migration folder and and recreating a
fresh migartion helped.
Deployment
sudo apt-get install -y apache2 apache2-dev
sudo su
source venv/bin/activate
pip install mod_wsgi
mod_wsgi-express module-config
mod_wsgi-express install-module
this will print the folowing lines : LoadModule wsgi_module "/usr/lib/apache2/modules/mod_wsgi-py35.cpython-35m-x86_64-linux-gnu.so" - copy this to wsgi.load WSGIPythonHome "/mnt/c/mydev/microservice-tsp-billing/tokenleader/venv" copy this to wsgi.conf
vi /etc/apache2/mods-available/wsgi.conf
vi /etc/apache2/mods-available/wsgi.load
cd /etc/apache2/mods-enabled/
ln -s ../mods-available/wsgi.conf wsgi.conf
ln -s ../mods-available/wsgi.load wsgi.load
sudo a2enmod ssl
sudo mkdir /etc/apache2/ssl
sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
-keyout /etc/apache2/ssl/tokenleader-apache-server.key \
-out /etc/apache2/ssl/tokenleader-apache-server.crt
apachectl configtest
download the copy of app.wsgi file and copy it in /var/www
download the tokenleader-apache.conf , place it in /etc/apache2/sites-enabled/ and modify the
directories and the username
start the apache service
sudo service apache2 start
https://pypi.org/project/mod_wsgi/
https://modwsgi.readthedocs.io/en/develop/user-guides/virtual-environments.html
important note :development
Testing
clone from git and then run
python -m unittest discover tests
to run single unit test
python -m unittest tokenleader.tests.unittests.test_admin_ops.TestAdminOps.test_abort_delete_admin_user_input_not_yes
for token generation and verification testing this is a useful test
python -m unittest tokenleader.tests.test_auth.TestToken.test_token_gen_n_verify_success_for_registered_user_with_role
to test the db operation :
(venv) bhujay@DESKTOP-DTA1VEB:/mnt/c/mydev/microservice-tsp-billing/tokenleader$ flask shell
from app1 import db
from app1.authentication.models import User, Role
r1 = Role(rolename='role1')
db.session.add(r1)
db.session.commit()
u = User(username='john', email='john@example.com')
db.session.add(u)
db.session.commit()
u = User.query.filter_by(username='john').first()
u
#<User john>
u.roles
#<sqlalchemy.orm.dynamic.AppenderBaseQuery object at 0x7fb94faa2e48>
u.roles=[r1]
db.session.commit()
u.roles
#<sqlalchemy.orm.dynamic.AppenderBaseQuery object at 0x7fb94fa8bf98>
for l in u.roles:
print(l.rolename)
#role1
1)operation scope filtering based on users org, div, dept details Todo: role and wfc shd not have any relation - done user can have only one wfc - done user to dict now gives wfc dictionary as well
tesing to be done/changes to that affect - upto verification unit test is passed
workcontext to be instantiated as a class
workcontext to be made avilable to api route function when required
ext important works:
- centralized catalogue for all microservice endpoints and
- client for tokenleader
change log
ver 1.5
tokenleader 0.70 and few fixes
ver1.3
- migration/version was missing
- Sample config files created under etc directory in source code
ver 1.1
- all configs are in /etc/tokenleader
- tlclient command changed to tokenleader
- tlconfig command changed to tokenleader-auth
ver 1.0
setting FLASK_APP during db init
ver 0.8 / 0.9
- added adminops initdb command for applying the changes in database schema
ver 0.7
- tokenleaderclient bug resolved in client version 0.64
ver 0.6
- check presence of required parameters in /etc/tokenleader/tokenleader_settings.ini while starting the service
ver 0.5
- introduction of /etc/tokenleader/tokenleader_settings.ini for hostname, port etc.
- tokenleader-start to start the service
- service can be started with ssl - although this will be mostly done by a nginx or apache in a production setup.