Django PubSubPull

django-pubsubpull is intended to help you decouple micro-services. It provides mechanisms that allow services to publish changes, subscribe to change notifications, and pull records.

In practice what we'll end up with is along the lines of:

• Use Django slumber and async to provide a load of basic infrastructure this is going to need.
• Provide new models for tracking database level changes to certain tables (models), and database triggers to ensure that they are properly recorded.
• All operations will be based on HTTP using GET, PUT and DELETE only.
• Django middleware to capture as much information about the request and the underlying changes we can and to tie that in to the records of database changes recorded there.
• Support for Django 1.0 through 1.7. We may support 1.8 and later if we can make our middleware subsume the functionality of the old transaction middleware.
• Only support Postgres 9.4 and later.

The use cases we envisage include:

• A service needs to locally cache a copy of some attributes of an object held in another micro-service, generally for performance reasons (for example, a billing service needing copies of a customer's location so that it can choose the correct rates).
• A service needs to record meta-data about user instructed changes for audit trail purposes (for example, any change to a customer email subscription is recorded with full details of who made the change and when it was made).
• A service needs to be notified of certain API calls so that it can in turn trigger other behaviours (for example, a comms service sending a welcome email when a new user is created).

The three parts are:

• Publish: A service will notify it's subscribers about any changes to the requested model instances. Ultimately the publish will hook into a database trigger to ensure that changes are always properly recorded no matter how it occurs.
• Subscribe: A service can request change notifications be sent to it from the publishing service. The subscriber is able to specify a URL that is used for a PUT request when changes are notified.
• Pull: A service can iterate over all of the instances of a particular model and then periodically iterate over new ones.