taskall

Taskall is a Python module which simplifies the chore of creating and executing tasks in parallel


Keywords
parallel, pool, multiprocessing
License
MIT
Install
pip install taskall==0.1.1

Documentation

Taskall

Introduction

Taskall is a Python module which simplifies the chore of creating and executing functions in parallel.

Usage

The taskall.parallel module contains two context managers which assist in creating functions which will execute in a separate process once executed.

from taskall import parallel


def slow_multiply(a, b):
    import time
    time.sleep(0.5)
    return a * b


# Execute a function in parallel (separate process)
with parallel.task() as tasker:

    # Convert our synchronous function into a parallel function
    parallel_slow_multiply = tasker.taskify(slow_multiply)
    future = parallel_slow_multiply(1, 2)

    # ... do stuff ...

    print future.result


# Send our data to our process pool and execute our
# function in parallel
with parallel.pool(pool_size=8) as pool:
    futures = pool.map(slow_multiply, range(5), range(5))

    # Retrieve the results as they are completed
    for result in futures:
        print result

In addition, taskall.parallel also contains two helper methods which can convert a standard function into a parallel function. These functions will now return taskall.future.Future objects and will become non-blocking.

@parallel.taskify
def slow_multiply(a, b):
    import time
    time.sleep(0.5)
    return a * b


@parallel.poolify(pool_size=2)
def slow_multiply(a, b):
    import time
    time.sleep(0.5)
    return a * b