ckanext-rq

This is a backport of the RQ-based background jobs code seen in CKAN 2.7, to enable earlier versions of CKAN to run rq-based background tasks.

It is based on the code by @torfsen, mainly here: https://github.com/ckan/ckan/pull/3165

NB:

You won't be able to use plugin.toolkit.enqueue_job. Use this fallback:

try:

enqueue_job = p.toolkit.enqueue_job

except KeyError:

from ckanext.rq.jobs import enqueue as enqueue_job

TODO:

• check Redis is available on startup
• setting ckan.redis.url from environment variable CKAN_REDIS_URL
• transfer the tests and RQTestBase

Requirements

For CKAN versions 2.0 to 2.6.x. (Must not be used with CKAN 2.7.0 or later)

Installation

You need Redis installed:

sudo apt-get install redis-server

To install ckanext-rq:

1. Activate your CKAN virtual environment, for example:

   . /usr/lib/ckan/default/bin/activate
   

2. Install the ckanext-rq Python package into your virtual environment:

   pip install ckanext-rq
   

3. Add rq to the ckan.plugins setting in your CKAN config file (by default the config file is located at /etc/ckan/default/production.ini).

4. Restart CKAN. For example if you've deployed CKAN with Apache on Ubuntu:

   sudo service apache2 reload
   

5. Start the worker:

   paster --plugin=ckanext-rq jobs worker --config=/etc/ckan/default/development.ini
   

6. To run the worker in a robust way, install and configure Supervisor: http://docs.ckan.org/en/latest/maintaining/background-tasks.html#using-supervisor

Config Settings

# URL to your Redis instance, including the database to be used.
ckan.redis.url = redis://localhost:6379/0

.. do::

Registering ckanext-rq on PyPI

ckanext-rq should be availabe on PyPI as https://pypi.python.org/pypi/ckanext-rq. If that link doesn't work, then you can register the project on PyPI for the first time by following these steps:

1. Create a source distribution of the project:

   python setup.py sdist
   

2. Register the project:

   python setup.py register
   

3. Upload the source distribution to PyPI:

   python setup.py sdist upload
   

4. Tag the first release of the project on GitHub with the version number from the setup.py file. For example if the version number in setup.py is 0.0.1 then do:

   git tag 0.0.1
   git push --tags
   

Releasing a New Version of ckanext-rq

ckanext-rq is availabe on PyPI as https://pypi.python.org/pypi/ckanext-rq. To publish a new version to PyPI follow these steps:

1. Update the version number in the setup.py file. See PEP 440 for how to choose version numbers.

2. Create a source distribution of the new version:

   python setup.py sdist
   

3. Upload the source distribution to PyPI:

   python setup.py sdist upload
   

4. Tag the new release of the project on GitHub with the version number from the setup.py file. For example if the version number in setup.py is 0.0.2 then do:

   git tag 0.0.2
   git push --tags