LazyCred
Minimalistic implementation of credential/secret storage using AWS S3 and KMS.
Installation
pip install git+git://github.com/2deviant/LazyCred.git
Use
Command Line
Set a value from a file:
~> lazycred put db_credentials db_creds.json
From standard input:
~> lazycred put db_credentials < db_creds.json
or
~> cat db_creds.json | lazycred put db_credentials
Get the value:
~> lazycred get db_credentials
Python
import lazycred
key = 'something'
val = 'Something Something Danger Zone'
lazycred.put(key, val)
print(lazycred.get(key))
Note
Secrets stored from inside Python are accessible to the command line, and vice versa, of coruse.
Configuration
In descending order of priority:
Inline
Only available inside Python:
import lazycred
lazycred.set_config({
"s3_path": "eFart/credentials/",
"key_alias": "eFart-key",
"region": "us-east-1"
})
-
s3_path
refers to the S3 bucketeFart
and foldercredentials/
. Folder is not require however,s3_path
must at least contain the bucket followed by/
. -
key_alias
is the alias of the KMS key. -
region
is self-explanatory.
File
.lazycred
:
{
"region": "us-east-1",
"s3_path": "eFart/credentials/",
"key_alias": "eFart-key",
}
This file is sought in the following manner:
- Check the current folder;
- If doesn't exist, go up a folder and try again;
- Continue until the root folder,
/
, is reached; - If not found, attempt to source from
$HOME
.
Environment Variables
LazyCred will attempt to source the values from environment variables if .lazycred
is not found:
LAZYCRED_S3_PATH
LAZYCRED_KEY_ALIAS
AWS_DEFAULT_REGION
Requirements
- AWS Account
- AWS S3 bucket in the nether region (regionless)
- AWS KMS key
Potential Issues
Speed
LazyCrypt relies on two remote, distinct services: S3 and KMS. S3 is optimized for bulk storage and fast downloads, not rock-bottom latency. Add to that the transit time for your request and payload return, add to that the same for KMS. What results is a function that is best not called from a tight loop. Recommended use is at the start of an application.
Blob Storage
To counteract low retrieval speed, one may store all of the secrets as one JSON blob.
Shop
Mechanism Of Action
# this is your secret
secret = 'The gold is at the post office.'
# this is a high quality random sequence of bytes
password = random_bytes()
# this is your secret encrypted with a high quality password
encrypted_secret = encrypt(secret, password)
# this is a high quality password encrypted with a black box
encrypted_key = aws_kms_encrypt(key)
# this is the data stored in S3
cipherblob = {
'data': encrypted_secret,
'key': encrypted_key
}
The secret
can be recovered:
secret = decrypt(
cipherblob['data'],
aws_kms_decrypt(cipherblob['key'])
)
Dependencies
Outside of Python's pre-packaged libraries:
- Boto
- cryptography
To install:
pip install boto cryptography
Boto
Boto v2 is implemented. AWS credentials are sourced from their default locations. For more information, see Boto Documentaiton.
Error Handling
In case of any error, be it configuration, communication, or cryptography, get returns None
, and put returns False
. In such case, error is logged with all available details via Python's logging
module:
> lazycred get SpaceBalls
ERROR:LazyCredLogger:Unable to get <Key: eFart,credentials/SpaceBalls>.
If your application has a log collector, it will sweep up LazyCred's errors.
Storage
Data is stored at an S3 path specified in the configuration suffixed with the key name. A typical record looks like so:
{
"data": "gAAAAABYOhZbt6reGldBveQ...",
"key": "jodC9cIM/FT7skOoWZrm0QAA..."
}
Where data
is the payload and key
is the KMS-encrypted random Fernet key. AWS KMS decrypts the key
, then the decrypted key decrypts the data
. One may note that key_alias
is not specified, that is because the reference to the KMS key is encoded in the key
cipherblob.
Intermediary Key
One may wonder why not encrypt the data directly with KMS. AWS KMS is limited to a 4KB payload. While most credential store items are less than 4KB, that limit is by no means guaranteed. A common practice is to encrypt an arbitrary amount of data with a random, fixed-size key, and then encrypt the key with KMS (same protocol as PGP). Both methods produce cryptographically equivalent results provided that both use equivalent encryption methods.
Code Penmanship
An astute coder notices that this small application encompasses the following four concepts:
- Key, as it pertains to key-value store nature of LazyCred;
- Key, a.k.a. everything after the bucket name in an S3 path;
- Key, a sequence of random characters used to encrypt data;
- Key, KMS key -- and an alias thereto -- used to encrypt the key from above.
A moderate effort has been made to contain the ambiguity however, some variable alliteration cannot be helped.