stones

Library for Persistent key-value containers, compatible with Python dict


Keywords
dict, persistent, python
License
MIT
Install
pip install stones==0.2

Documentation

🗿Stones

Build Status Codecov Python 3.10

Base library for persistent key-value stores, 100% compatible with Python dict.

Stones image

The idea behind this project is to have a common interface for a multitude of persistent key-value stores, easy to use and extend, with some extra built-in features as bonus. Inspired from Datastore and MemDown.

It plays nice with Object Querying libraries like: JMESPath, or JSONPath RW, or ObjectPath.

Features and Limitations

  • the same API that you already know from Python dict
  • thread safe updates
  • excellent test coverage (>90%)
  • 1st limitation: the keys can only be bytes. They are converted from str to bytes for compatibility.
  • 2nd limitation: the values are actually also bytes, but a serializer (default is Pickle) converts the data structure into bytes
  • there are several serializers available: Pickle, JSON and optional: cbor2, msgpack. You can easily add your own serializer, please check the documentation below.
  • 3rd limitation: after the data was serialized, you have to continue using the same serializer. If for some reason, you want to switch the serializer, you can create a new Store with your new serializer and copy all the data from the old Store.
  • the Memory Store and LMDB are available by default. Other key-value Stores used to be implemented (DBM, LevelDB, Redis), but are not included anymore.

Install

This project uses Python 3.9+ and Poetry, or pip.

$ poetry add stones
# or
$ pip install stones

Usage

from stones import stone, LmdbStore

# Create a new persistent dictionary, backed by LMDB
stor = stone('my-db', LmdbStore, serialize='pickle')

# You can use it just like a normal Python dict,
# with the limitation that keys are bytes,
# and the values must be serializable
stor[b's'] = b'whatever'
stor[b'li'] = [True, False, None, -1]
stor[b'tu'] = ('Yes', 'No', 3)

b's' in stor
# True
len(stor)
# 3
stor.keys()
# [b'li', b's', b'tu']
stor.values()
# [[True, False, None, -1], b'whatever', ('Yes', 'No', 3)]

# The data is decoded in place
-1 in stor[b'li']
# True
3 in stor[b'tu']
# True

del stor[b'li']
stor.get(b'li', False)
# False

# The data is persisted
del stor
# Re-create the store, using the same encoder
stor = stone('my-db', LmdbStore, serialize='pickle')

len(stor)
# 2
stor.keys()
# [b's', b'tu']

More or less similar projects

Python:

Node.js:

  • lowDB - simple to use local JSON database. Powered by plain JS.
  • Keyv - Key-value storage across multiple backends via storage adapters. Supports TTL based expiry, making it suitable as a cache or a persistent key-value store.
  • MemDown - abstract LevelDown store for Node.js and browsers
  • AppStorage - LocalStorage for Node.js, auto syncing objects with JSON files
  • NeDB - embedded database for Node.js, nw.js, Electron and browsers
  • FortuneJS - database abstraction layer with a common interface for databases
  • PouchDB - pocket-sized database
  • ToiletDB - flushes an object to a JSON file
  • whatDB - in-memory JavaScript database for Small Web applications that persists to JSON files

Other:

  • CubDB - Embedded key-value database written in the Elixir language. It runs locally, it is schema-less, and backed by a single file.

License

MIT © Cristi Constantin.