firestoreMQ

Firestore MQ python library


License
MIT
Install
pip install firestoreMQ==1.0.4

Documentation

Firestore MQ

Serverless firebase firestore message queue with javascript and python bindings and python CLI.

Work queues - Distributing tasks among workers (the competing consumers pattern)

Libs

Python

See libs/python. Install with pip3 install firestoreMQ

DB structure

New tasks are placed into the unassigned queue. Workers watch the queue (sorting by priority and created time) and assign takes to themselves - transferring them to the assigned queue.

Once a worker compltets a task it is removed. If a task cannot be completed it is moved into the errored queue.

If a worker dies while a task is assigned to it then the task will be released back to the unassigned queue after the TTL (time to live).

Queues: {
 <queue_id>: {
    unassigned: {
      task_id: {
        created: datetime,
        ttl: datetime,
        assigned_to: <worker_id>,
        data: JSON,
        priority: <number>
      }
      ...
    },
    assigned: {...}
    errored: {...}
 }
}