Stream Tap

Capture data from an iterator as it gets pulled down stream.

Example

#!/usr/bin/env python
# -*- coding: utf-8 -*-


from stream_tap import Bucket
from stream_tap import stream_tap


def get_fruit(item):
    """Get things that are fruit.

    :returns: thing of item if it's a fruit"""
    if len(item) == 2 and item[1] == u"fruit":
        return item[0]


def get_metal(item):
    """Get things that are metal.

    :returns: thing of item if it's metal"""
    if len(item) == 2 and item[1] == u"metal":
        return item[0]


def certain_kind_tap(data_items):
    """
    :param data_items: A sequence of unicode strings
    """
    fruit_spigot = Bucket(get_fruit)
    metal_spigot = Bucket(get_metal)

    items = stream_tap((fruit_spigot, metal_spigot), data_items)

    # consume iterator.
    for item in items:
        print item

    return fruit_spigot.contents(), metal_spigot.contents()


def run():
    """
    Run the composition of csv_file_consumer and information tap
    with the csv files in the input directory, and collect
    the results from each file and merge them together,
    printing both kinds of results.

    """
    data_items = [
        [u"mushroom", u"fungus"],
        [u"tomato", u"fruit"],
        [u"topaz", u"mineral"],
        [u"iron", u"metal"],
        [u"dróżką", u"utf-8 sample"],
        [u"apple", u"fruit"],
        [u"cheese", u"dairy"],
        [u"peach", u"fruit"],
        [u"celery", u"vegetable"],
        [u"pear", u"fruit"],
        [u"ruby", u"mineral"],
        [u"titanium", u"metal"],
        [u"cat", u"animal"],
        [u"orange", u"fruit"],
        [u"WĄŻ", u"utf-8 sample"],
    ]

    results = certain_kind_tap(data_items)

    fruits, metals = results

    print("=== fruits ===")
    for fruit in fruits:
        print(fruit)

    print("=== metals ===")
    for metal in metals:
        print(metal)


if __name__ == "__main__":
    run()

Documentation

Read the docs: http://stream_tap.readthedocs.org/en/latest/

Contributing:

Submit any issues or questions here: https://github.com/johnwlockwood/stream_tap/issues.

Make pull requests to development branch of

https://github.com/johnwlockwood/stream_tap.

Documentation is written in reStructuredText and currently uses the

Sphinx style for field lists http://sphinx-doc.org/domains.html#info-field-lists

Check out closed pull requests to see the flow of development, as almost every change to master is done via a pull request on GitHub. Code Reviews are welcome, even on merged Pull Requests. Feel free to ask questions about the code.