ngraph-onnx

nGraph™ Backend for ONNX.

This repository contains tools to run ONNX models using the Intel® nGraph™ library as a backend.

Installation

Prerequisites

Python 3.4 or higher is required.

Protocol Buffers

You will need Protocol Buffers v.2.6.1 or higher installed on your system to use ONNX.

On Ubuntu, for exmaple you can install protobuf using:

$ sudo apt-get install protobuf-compiler libprotobuf-dev

And on Mac OS you can install protobuf using Homebrew:

$ brew install protobuf

You can verify whether you have version >2.6.1 installed using the command:

$ protoc --version
libprotoc 3.4.0

nGraph

The other requirement is of course nGraph and nGraph's Python bindings. You can follow these instructions to build an ngraph Python wheel containing both:

https://github.com/NervanaSystems/ngraph/blob/master/python/README.md

Once the Python binary wheel file (ngraph-*.whl) is prepared you can install it using pip.

For example:

(your_venv) $ pip install -U build/dist/ngraph-0.0.1-cp35-cp35m-linux_x86_64.whl

You can check that ngraph is properly installed in your Python shell:

>>> import ngraph_api as ng
>>> ng.abs([[1, 2, 3], [4, 5, 6]])
<Abs: 'Abs_1' ([2, 3])>

If you don't see any errors, ngraph should be installed correctly.

Installing ngraph-onnx

You can install ngraph-onnx using pip:

 (your_venv) $ pip install git+https://github.com/NervanaSystems/ngraph-onnx/

Usage example

Importing the ONNX model

# Import ONNX and load an ONNX file from disk
>>> import onnx
>>> onnx_protobuf = onnx.load('/path/to/ResNet20_CIFAR10_model.onnx')

# Convert ONNX model to an ngraph model
>>> from ngraph_onnx.onnx_importer.importer import import_onnx_model
>>> ng_models = import_onnx_model(onnx_protobuf)

# The importer returns a list of ngraph models for every ONNX graph output:
>>> print(ng_models)
[{
    'name': 'Plus5475_Output_0',
    'output': <Add: 'Add_1972' ([1, 10])>,
    'inputs': [<Parameter: 'Parameter_1104' ([1, 3, 32, 32], float)>]
 }]

The output field contains the ngraph node corrsponding to the output node in the imported ONNX computational graph. The inputs list contains all input parameters for the computation which generates the output.

Running a computation

After importing the ONNX model, you can use it to generate and call a computation function.

# Using an ngraph runtime (CPU backend) create a callable computation
>>> import ngraph_api as ng
>>> ng_model = ng_models[0]
>>> runtime = ng.runtime(manager_name='CPU')
>>> resnet = runtime.computation(ng_model['output'], *ng_model['inputs'])

# Load an image (or create a mock as in this example)
>>> import numpy as np
>>> picture = np.ones([1, 3, 32, 32])

# Run computation on the picture:
>>> resnet(picture)
array([[ 1.312082 , -1.6729496,  4.2079577,  1.4012241, -3.5463796,
         2.3433776,  1.7799224, -1.6155214,  0.0777044, -4.2944093]],
      dtype=float32)

Supported ONNX operations

• Abs
• Add
• And
• AveragePool
• BatchNormalization
• Ceil
• Constant
• Conv
• Div
• Dot
• Elu
• Equal
• Exp
• Floor
• Gemm
• Greater
• LeakyRelu
• Less
• Log
• MatMul
• Max
• MaxPool
• Mean
• Min
• Mul
• Neg
• Not
• Or
• PRelu
• Reciprocal
• ReduceLogSumExp
• ReduceMax
• ReduceMean
• ReduceMin
• ReduceProd
• ReduceSum
• Relu
• Reshape
• Selu
• Sigmoid
• Softmax
• Sqrt
• Sub
• Sum
• Tanh
• Xor

Refer to ONNX docs for the complete operator list.