Base library for persistent key-value stores, 100% compatible with Python dict.
The idea behind this project is to have a common interface for a multitude of persistent key-value stores, easy to use and extend, with some extra built-in features as bonus. Inspired from Datastore and MemDown.
Features and Limitations
- the same API that you already know from Python dict
- thread safe updates
- excellent test coverage (>90%)
- 1st limitation: the keys can only be bytes. They are converted from str to bytes for compatibility.
- 2nd limitation: the values are actually also bytes, but a serializer (default is Pickle) converts the data structure into bytes
- there are several serializers available: Pickle, JSON and optional: cbor2, msgpack. You can easily add your own serializer, please check the documentation below.
- 3rd limitation: after the data was serialized, you have to continue using the same serializer. If for some reason, you want to switch the serializer, you can create a new Store with your new serializer and copy all the data from the old Store.
- the Memory Store and LMDB are available by default. Other key-value Stores used to be implemented (DBM, LevelDB, Redis), but are not included anymore.
$ poetry add stones # or $ pip install stones
from stones import stone, LmdbStore # Create a new persistent dictionary, backed by LMDB stor = stone('my-db', LmdbStore, serialize='pickle') # You can use it just like a normal Python dict, # with the limitation that keys are bytes, # and the values must be serializable stor[b's'] = b'whatever' stor[b'li'] = [True, False, None, -1] stor[b'tu'] = ('Yes', 'No', 3) b's' in stor # True len(stor) # 3 stor.keys() # [b'li', b's', b'tu'] stor.values() # [[True, False, None, -1], b'whatever', ('Yes', 'No', 3)] # The data is decoded in place -1 in stor[b'li'] # True 3 in stor[b'tu'] # True del stor[b'li'] stor.get(b'li', False) # False # The data is persisted del stor # Re-create the store, using the same encoder stor = stone('my-db', LmdbStore, serialize='pickle') len(stor) # 2 stor.keys() # [b's', b'tu']
More or less similar projects
- TinyDB - awesome lightweight document oriented database
- RocksDict - Key-value storage supporting any Python object
- uCache - Lightweight caching library for Python
- RaRe-Technologies/sqlitedict - Persistent dict backed by sqlite3 and pickle, multithread-safe
- Lukedeo/Cupboard - store things in a variety of NoSQL KV stores
- ShuhaoWu/Kvkit - unmaintained since 2014
- Datastore - unmaintained since 2014
- Persistent dict ActiveState recipe - from 2009
- lowDB - simple to use local JSON database. Powered by plain JS.
- Keyv - Key-value storage across multiple backends via storage adapters. Supports TTL based expiry, making it suitable as a cache or a persistent key-value store.
- MemDown - abstract LevelDown store for Node.js and browsers
- AppStorage - LocalStorage for Node.js, auto syncing objects with JSON files
- NeDB - embedded database for Node.js, nw.js, Electron and browsers
- FortuneJS - database abstraction layer with a common interface for databases
- PouchDB - pocket-sized database
- ToiletDB - flushes an object to a JSON file
- CubDB - Embedded key-value database written in the Elixir language. It runs locally, it is schema-less, and backed by a single file.
MIT © Cristi Constantin.