sqs_event

Handle events from an AWS sqs queue.

introduction

sqs_event waits for events to arrive on a single sqs queue, processing each event to completion before returning to wait on the queue.

main event loop

Following is a representation of the main loop logic:

[1] for message in queue.recv():
[2]     event = normalize(message.body)
[3]     dispatch(event)
[4]     queue.delete(message)

[1]A message is read from an sqs queue and the Body is loaded as a json string into a python object. [2]The message body is then normalized into an sqs_event.Event object. [3]The event is dispatched to a configured handler, and, if everything goes well, [4]the message is deleted from the queue.

If there is a problem dealing with the message it will remain in the sqs queue to be received again, for as many times as the sqs queue configuration allows.

setup

the event file

At startup, sqs_event reads a setup file named event. The event file contains records specifying:

1. a handler routine for each type of event
2. custom setup code (optional)
3. custom configuration records (optional)
4. a custom normalize routine (optional)

The event file is treated like code, and is not meant to change during deployment.

Further detail on the event file

the config file

After reading the event file, sqs_event reads a file named config, containing settings which are expected to vary between deployments.

Further detail on the config file

an example

Suppose that we want to execute the following code, contained in a file named my_handler.py:

def hello(name):
    print('hello ' + name)

the event file

This event file tells sqs_event to pass the args and kwargs from any event named greetings to the hello function in the my_handler module.

EVENT greetings my_handler.hello

the event

Here is what a greetings event might look like:

{
    "name": "greetings",
    "args": ["world"],
    "kwargs": {},
}

the config file

In order to tell sqs_event where to find events to process, the name of the sqs queue can be supplied in a config file:

queue.name=some_queue_name

or by setting an environment variable named SQS_EVENT_QUEUE_NAME.

starting sqs_queue

Assuming sqs_queue is pip-installed or that ergaleia and sqs_queue are on the PYTHONPATH, the following command will wait on sqs for greeting events:

python -m sqs_event.main

You will see messages like this:

INFO:botocore.vendored.requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): queue.amazonaws.com
INFO:__main__:listening on queue: https://queue.amazonaws.com/999999999999/some_queue_name