Pax Daemonica

Pax Daemonica is setting out to be a system for running applications with unified logging and monitoring, an extremely simple deploy and upgrade process, RPC/remote asynchronous execution. This readme will attempt to document what is currently implemented. I'll write blog posts about future plans and link to them in the appropriate sections so there is no confusion about what it does right now.

Requirements

I've attempted to minimize the number of external requirements. As of right now you need a redis server running on the same machine and the python redis library (https://github.com/andymccurdy/redis-py).

Starting Up/Basic Processes

To start paxd, run python paxd/rootserver.py. This will start up the server, the controller process, and a UI on port 22100. The UI's web server is actually a thin wrapper around a paxd application which you'll find in paxd/webuiapp/webui.py. The server also has a pseudo-app for the controller which handles the various commands.

Each paxd instance should have a unique instance id, although at present there is a value hard-coded into the root server. The unique id only matters when there are multiple instances of paxd reading from the same queue. It is being used to move "active" tasks into the "failed" task list when loading an application (since on application startup there can't be any active tasks, it is safe to assume that any tasks with the instance id of the current server were incomplete when the server shut down).

Applications

A paxd application is, at its simplest, just a python function. However, to load an application slightly more is needed. An application has several key attributes necessary to load it:

• path – this is added to sys.path in the processes where the application is run
• entry – this is the dot.delimiated.name of the function which should be called. It expects this to be true: entry.rpartition('.') == (module, '.', function)
• queue – (optional) this is where the application will get its data from. If there is no queue, all of the processes will run their commands constantly.

there are other attributes on the application objects, but these are the only ones which can be set for now. When an application is loaded a random ID is assigned to it. Multiple instances of the same application can run on the server.

Server Commands

The commands available on the server are:

• Loading an application. /load. See scripts/load_task.py.
• Loading several applications automatically. /load?autoconf=file&path=/path.
• pause an application. /pause?id=app_id
• unpause an application. /unpause?id=app_id
• remove an application. /remove?id=app_id

The bottom three commands are available in the web ui. Loading automatically is done via a file located in the root of the path given called pax.conf. The file is read in as JSON and the format so far is:

{

"APPS" : {

"dotted.entry.point.function" : {},

}, "ENVIRONMENT" : {

"KEY" : "VALUE"

}

}

The values inside of APPS are for future use in configuration. The function specified should be able to be imported based on the path given. ENVIRONMENT sets key-value pairs as environment variables.

RPC/Asynchronous execution

In paxd/client.py there is a task decorator which will add a "delay" attribute to a function. The attribute is a function which can be used (as in celery) to run a command asynchronously. It sends a pickled message to the server and returns a Promise object which can be used to retrieve the result if desired. In order for the requests to actually be processed, an application for the decorated function must be loaded (see scripts/load_task.py.

Web UI

The Web UI, started on port 22100, gives a list of the running applications and allows them to be paused, unpaused, and removed. The web UI itself cannot be paused or removed, as that would disable the API used to monitor/control the server. There is also a "trace" function which gives stack traces of all running processes in a group

The UI also gives counts for:

• Q - queued
• Q:a - active items
• Q:f - failed items
• Q:s - successes

Q:f can be clicked on to see the details of why the task failed. Additionally, failed tasks can be either removed or requeued via the web UI based on the particular needs of the application.

Messages

There are two classes for requests to be sent: one for JSON and one for python pickling. Their APIs are the same. A request requires a redis connection, the name of a queue, and the args/kwds that the target should be called with. Each request has a UUID which is the queue in redis which the response should go to (these queues will only ever have one item). When a message is send it is added to the given queue in redis. If there is a free processor in the application's pool it will:

• move an item from queue into queue:pending
• in a transaction, delete the item from queue:pending and set it in queue:active, a redis hash, with a value of the instance ID
• process the item
• in a transaction, delete the item from queue:active and write the response to the request's response queue.

The request's send method returns a Promise. If .get() is called on the promise it will either return the value or raise an exception (if there was an error).