Protowire

Write protobuf messages from the command line:

pw (field number) [data type] (value) > output.bin

where data type is one of the protobuf datatypes (or int = int32 = int64). If value is not given, it is read from STDIN. The field number can be left out and defaults to 1.

This enables creating protobuf messages for GRPC calls or other purposes without a protobuf compiler. The pw tool has no library dependencies (plain Python) and does not need the .proto files or any code generated from them.

Installation

sudo pip install protowire

Examples

To write a protobuf message conforming to, e.g., message Test1 { int32 a = 1; } such that a has the value 150, write

pw 1 int32 150 > message.bin

Then examine (cf. official docs):

$ hd /tmp/msg.bin
00000000  08 96 01                                          |...|
00000003

Another example: message Test2 { string b = 2; } with b = "testing":

pw 2 string testing

More complex examples (also from here) can be composed using standard UNIX tools

pw int 150 | pw 3 bytes

and

(pw fixed64 10 && pw 2 bool true) | pw 4 bytes

GRPC client

This tool also requires the grpcio Python package (pip install grpcio)

pw-grpc-client (-is) (-os) (--tag 1) host:port/path

Consider the following example:

syntax = "proto3";

message RequestThing {
    string query = 1;
}

message ResponseItem {
    int32 foo = 1;
    // ...
}

service MyService {
    rpc UnaryMethod(RequestThing) returns (ResponseItem) {}
    rpc ServerStream(RequestThing) returns (stream ResponseItem) {}
    rpc ClientStream(stream RequestThing) returns (ResponseItem) {}
    rpc BidirectionalStream(stream RequestThing) returns (stream ResponseItem) {}
}

// helper collections
message RequestCollection {
    repeated RequestThing things = 1;
}

message ResponseCollection {
    repeated ResponseItem items = 1;
}

An UnaryMethod call using a RequestThing with query = "hello" can be sent to a server running at localhost:8000 as follows:

pw string "hello" | \
    pw-grpc-client localhost:8000/MyService/UnaryMethod \
    > response_item.bin

which saves the obtained ResponseItem protobuf the file response_item.bin.

It is also possible to convert ServerStream responses to protobuf ResponseCollections using the -os/--stream_response flag:

pw string "hello" | \
    pw-grpc-client localhost:8000/MyService/ServerStream -os \
    > response_collection.bin

Similarly, ClientStream can be constructed from RequestCollections with -is/--stream_request:

pw string "singleton item" | pw 1 bytes | \
    pw-grpc-client localhost:8000/MyService/ClientStream -is \
    > response_item.bin

The flags -is and -os can be used simultaneously for BidirectionalStream

((pw string "first" | pw bytes) && (pw string "second" | pw bytes)) | \
    pw-grpc-client localhost:8000/MyService/BidirectionalStream -is -os \
    > response_collection.bin

In all -os-cases, there is also a --tag flag that can be used to change the field number in the response protobuf collection. For example message ResponseCollection { repeated ResponseItem items = 2; } would require:

pw string "hello" | \
    pw-grpc-client localhost:8000/MyService/ServerStream -os --tag 2 \
    > response_collection.bin

GRPC frames for low-level communication

This tool does not need any GRPC or protbuf packages, but can be combined with a HTTP/2 client like nghttp to make GRPC calls. Usage:

pw-grpc-frame [wrap|unwrap] (--stream) (--tag 1)

Wrap into a GRPC frame and unwrap the response to normal protobuf:

pw 1 string "my query" | pw-grpc-frame wrap > request.bin
nghttp -H ":method: POST" -H "Content-Type: application/grpc" -H "TE: trailers" \
    --data=request.bin \
    http://localhost:8000/MyService/UnaryMethod \
    | pw-grpc-frame unwrap > /tmp/output.bin

The option --stream can be used with both wrap and unwrap to convert protobuf collections to GRPC streams like in the other GRPC client example. The --tag option can be used to change the field number in the "unwrapped" protobuf collection.

Notice that older versions of nghttp (like 0.6.4 in Debian Jessie) cannot read STDIN with -d -.

Development

1. Create virtualenv
   • Python 2: virtualenv venvs/python2
   • Python 3: python3 -m venv venvs/python3
2. Activate virtualenv: source venvs/pythonNNN/bin/activate
3. Install locally pip install -e .[dev]
4. ./run-tests.sh
5. deactivate virtualenv