avro-codec

An avro codec which exposes an API similar to the standard library's marshal, pickle and json modules


Keywords
avro, encode, decode, codec, data-pipeline
License
MIT
Install
pip install avro-codec==2.0.0

Documentation

avro_codec

A simple avro codec which exposes an API similar to the Python standard library's marshal, pickle and json modules

Example

from avro_codec import AvroCodec

USER_LOGGED_IN_SCHEMA = {
    "type": "record",
    "name": "UserLoggedInEvent",
    "doc": "This event describes a user logging into the system",
    "fields": [
        {"name": "user_id", "type": "string", "doc": "The id of the user who logged in"},
        {"name": "time", "type": "long", "doc": "Epoch timestamp indicating when the user logged in"},
        {"name": "channel",
         "type": {
             "type": "enum",
             "name": "channel_types",
             "symbols": ["web", "ios_app"]},
         "doc": "The channel through which the user logged in"}
    ]
}

# Construct an instance of the codec from an Avro schema, provided
# as a simple dict/list hierarchy
codec = AvroCodec(USER_LOGGED_IN_SCHEMA)

event = {
    "user_id": "abc123",
    "time": 1437676881,
    "channel": "web"
}


# Dump a python object to an Avro-encoded bytes object
avro_data = codec.dumps(event)
print avro_data
# => '\x0cabc123\xa2\xd5\x89\xdb\n\x00'

# Load an Avro-encoded bytes object into an object
decoded_event = codec.loads(avro_data)
print decoded_event
# => {u'user_id': u'abc123', u'channel': u'web', u'time': 1437676881}

Development

Running tests

python setup.py nosetests

Pull requests gratefully received.