LordKey
It solves the problem of determining the combination in a sequence based on the some alphabet with a given length, and use combination to determine the iteration index.
The problem.
There are several elements to iterate - a, b and c. How many possible combinations for unique enumeration if key size is 3? Or which combination is at the tenth iteration? Or which iteration corresponds the acc combination?
For abc alphabet and 3 key size can be created the next iterations:
0. aaa 1. aab 2. aac 3. aba 4. abb 5. abc 6. aca 7. acb 8. acc 9. baa 10. bab 11. bac 12. bba 13. bbb 14. bbc 15. bca 16. bcb 17. bcc 18. caa 19. cab 20. cac 21. cba 22. cbb 23. cbc 24. cca 25. ccb 26. ccc
So, the maximum number of iterations - 27, for 10 iteration corresponds to baa combination and the acc combination - it is 7 iteration.
The theory.
Use the arbitrary alphabet (set of available characters to create a key) and size of key can be created a sequence of unique combinations, where each new combination has its unique numeric index (from 0 to N - where the N is maximum number of possible combinations or infinity).
If specify the index (for example an ID in the table of database) - will be returned the combination (key) for this index, and if specify combination - will be returned it index.
P.s. The sequence is not created - just calculate the data of a specified element. This algorithm allows you to quickly get the result.
Example:
# INITIAL DATA
# =================
# Alphabet | abcde
# ---------+-------
# Key size | 3
#
# SEQUENCE
# ==============================================
# ID | 0 | 1 | 2 | ... | 122 | 123 | 124
# ----+-----+-----+-----+-----+-----+-----+-----
# Key | aaa | aab | aac | ... | eec | eed | eee
lk = LordKey(alphabet='abcde', size=3)
lk.get_key_by_id(122) # eec
lk.get_id_by_key('ecc') # 122
The alphabet may be omitted then will be used value by default. If not set the size value - key size can be from one char to infinity.
Example:
# Size not specified. lk = LordKey(alphabet='abc') lk.get_key_by_id(1) # b lk.get_key_by_id(10) # bab lk.get_key_by_id(100) # bacab lk.get_key_by_id(1000) # bbabaab lk.get_key_by_id(10000) # bbbcabbab lk.get_key_by_id(100000) # bcaacabbcab lk.get_key_by_id(1000000) # bcbccbacacaab lk.get_key_by_id(10000000) # caacbbaabbacbab
Cache
The LordKey class can be expanded by adding caching capabilities. To do this, you must override a some of built-in methods:
- _cache_init(self, *args, **kwargs) - to initialize the cache settings.
- _cache_write(self, *, id, key) - to write the id and the corresponding sequence into cache.
- _cache_read(self, *, id=None, key=None) - to read data from the cache. Can specified only id or just key at an one moment of the time.
- _cache_is_active(self) - returns True if the cache is enabled.
Example - cache as dict on the class level.
See an example in the examples/classcache.py.
from lordkey import LordKey
class LordKeyClassCache(LordKey):
"""Support the cache at the class level."""
# Cache to accelerate the implementation of the same operations.
__GLOBAL_CLOUD = dict()
def __init__(self, alphabet=None, size=None, cache=False):
"""Advanced constructor - added the ability to manage the cache."""
super().__init__(alphabet=alphabet, size=size)
self._cache_init(cache=cache)
def _cache_init(self, *args, **kwargs):
"""Create a data structure of cache."""
# Cache cloud.
self._cache, self._cloud = kwargs.get('cache', False), None
if self._cache:
# Since the cache is implemented at the class level is necessary
# to ensure that the different cache to be used for different
# settings.
section = '{}:{}'.format(self._alphabet, self._size)
if section not in self.__GLOBAL_CLOUD:
CloudEntry = collections.namedtuple('CloudEntry', 'id key')
self.__GLOBAL_CLOUD[section] = CloudEntry(dict(), dict())
self._cloud = self.__GLOBAL_CLOUD[section]
def _cache_is_active(self):
"""Returns True if the cache is activated."""
return self._cache
def _cache_write(self, *, id, key):
"""Add to cache."""
self._cloud.id[id] = key
self._cloud.key[key] = id
Example - cache as dict on the object level.
See an example in the examples/objectcache.py.
from lordkey import LordKey
class LordKeyObjectCache(LordKey):
"""Support the cache at the class level."""
def __init__(self, alphabet=None, size=None, cache=False):
"""Advanced constructor - added the ability to manage the cache."""
super().__init__(alphabet=alphabet, size=size)
self._cache_init(cache=cache)
def _cache_init(self, *args, **kwargs):
"""Create a data structure of cache."""
# Cache cloud.
self._cache, self._cloud = kwargs.get('cache', False), None
if self._cache:
CloudEntry = collections.namedtuple('CloudEntry', 'id key')
self._cloud = CloudEntry(dict(), dict())
def _cache_is_active(self):
"""Returns True if the cache is activated."""
return self._cache
def _cache_write(self, *, id, key):
"""Add to cache."""
self._cloud.id[id] = key
self._cloud.key[key] = id
def _cache_read(self, *, id=None, key=None):
"""Read from cache."""
if id is not None:
return self._cloud.id.get(id, None)
elif key is not None:
return self._cloud.key.get(key, None)
Example - cache in a REDIS.
See an example in the examples/rediscache.py.
from lordkey import LordKey
# Redis settings.
REDIS_POLL = redis.ConnectionPool(host='localhost', port=6379, db=0)
class LordKeyRedisCache(LordKey):
"""Support the cache use Redis."""
def __init__(self, alphabet=None, size=None, cache=False):
"""Advanced constructor - added the ability to manage the cache."""
super().__init__(alphabet, size)
self._cache_init(cache=cache)
def _cache_init(self, *args, **kwargs):
"""..."""
"""Create a data structure of cache."""
self._cache = kwargs.get('cache', False)
self._cloud = self._dbkey = self._redis_cli = None
if self._cache:
self._redis_cli = redis.StrictRedis(connection_pool=REDIS_POLL)
self._dbkey = '{}:{}'.format(self._alphabet, self._size)
cloud = self._redis_cli.get(self._dbkey)
if not cloud:
CloudEntry = json.dumps({'id': {}, 'key': {}})
self._redis_cli.set(self._dbkey, CloudEntry)
cloud = self._redis_cli.get(self._dbkey)
# We use the virtual cache at the object level.
self._cloud = json.loads(cloud.decode('utf-8'))
def _cache_is_active(self):
"""Returns True if the cache is activated."""
return self._cache
def _cache_write(self, *, id, key):
"""Add to cache."""
# Attention.
# If each time to add the JSON data in Redis - it slows down the
# algorithm.
self._cloud['id'][str(id)] = key
self._cloud['key'][key] = id
def _cache_read(self, *, id=None, key=None):
"""Read from cache."""
if id is not None:
return self._cloud['id'].get(str(id), None)
elif key is not None:
return self._cloud['key'].get(key, None)
def cache_save(self):
"""Write virtual cache into Redis."""
if not self._redis_cli:
return
self._redis_cli.set(self._dbkey, json.dumps(self._cloud))
Best algorithm for caching.
For different tasks need to use different caching algorithms. You can experiment to see different speed caching algorithms in different tasks. See an example in the examples/speedtest.py.
The test results for different tasks:
If object is created and used repeatedly. 1. Object cache: 1.109303779026959 2. Class cache: 1.1138295580167323 3. Redis cache: 2.4269069299916737 4. Origin: 2.758108651963994
If for each new test used a new object. 1. Class cache: 1.1082238990347832 2. Object cache: 2.0088599160080776 3. Origin: 2.589507105993107 4. Redis cache: 3.3289793099975213
If use single call. 1. Class cache: 0.3119982249918394 2. Origin: 0.3376156840240583 3. Redis cache: 3.0777672510012053 4. Object cache: 16.18929815205047
