rivulet: A Redis-Based Message Broker for Python

Documentation: Read the docs.

Quickstart

$ pip install rivulet

Notes

Limitations

• No balanced consumers (yet)
• Without the proper management tools (see todos), managing messages is painful raw redis.

Todos

• Extend testing
  • Connection drops
  • Parallel producers, consumers (stressing the locking setup)
• Provies management functionality
  • list channels, delete channels
  • count, list, update subscribers
  • count, list, prune messages

Implementation details

Data model:

1. rvl:lock:<channel_id>: Used to maintain locks across multi-step redis calls (in particular when sending a message using a server-issued, monotonically increasing message seq id).
2. rvl:id:<channel_id>: A counter providing sequence ids for messages in channel channel_id
3. rvl:msg:<channel_id>: ZSET that maps a packed JSON string to the message sed id (allowing range queries on messages over seq ids).
4. rvl:channel:<channel_id>: ZSET that maps a client id to the last message id in channel channel_id seen by the clinet
5. rvl:index:<client_id>: ZSET that maps the channel_id to the last message id seen by client client_id