Taskall is a Python module which simplifies the chore of creating and executing functions in parallel.
The taskall.parallel module contains two context managers which assist in creating functions which will execute in a separate process once executed.
from taskall import parallel def slow_multiply(a, b): import time time.sleep(0.5) return a * b # Execute a function in parallel (separate process) with parallel.task() as tasker: # Convert our synchronous function into a parallel function parallel_slow_multiply = tasker.taskify(slow_multiply) future = parallel_slow_multiply(1, 2) # ... do stuff ... print future.result # Send our data to our process pool and execute our # function in parallel with parallel.pool(pool_size=8) as pool: futures = pool.map(slow_multiply, range(5), range(5)) # Retrieve the results as they are completed for result in futures: print result
In addition, taskall.parallel also contains two helper methods which can convert a standard function into a parallel function. These functions will now return taskall.future.Future objects and will become non-blocking.
@parallel.taskify def slow_multiply(a, b): import time time.sleep(0.5) return a * b @parallel.poolify(pool_size=2) def slow_multiply(a, b): import time time.sleep(0.5) return a * b