ml2rt - Utilities for taking ML to different runtimes
Machine learning utilities for model conversion, serialization, loading etc
- Free software: Apache Software License 2.0
Installation
pip install ml2rt
Documentation
ml2rt provides some convenient functions to convert, save & load machine learning models. It currently supports Tensorflow, PyTorch, Sklearn, Spark and ONNX but frameworks like xgboost, coreml are on the way.
Saving Tensorflow model
import tensorflow as tf
from ml2rt import save_tensorflow
# train your model here
sess = tf.Session()
save_tensorflow(sess, path, output=['output'])Saving PyTorch model
# it has to be a torchscript graph made by tracing / scripting
from ml2rt import save_torch
save_torch(torch_script_graph, path)Saving ONNX model
from ml2rt import save_onnx
save_onnx(onnx_model, path)Saving sklearn model
from ml2rt import save_sklearn
prototype = np.array(some_shape, dtype=some_dtype) # Equivalent to the input of the model
save_sklearn(sklearn_model, path, prototype=prototype)
# or
# some_shape has to be a tuple and some_dtype has to be a np.dtype, np.dtype.type or str object
save_sklearn(sklearn_model, path, shape=some_shape, dtype=some_dtype)
# or
# some_shape has to be a tuple and some_dtype has to be a np.dtype, np.dtype.type or str object
inital_types = utils.guess_onnx_tensortype(shape=shape, dtype=dtype)
save_sklearn(sklearn_model, path, initial_types=initial_types)Saving sparkml model
from ml2rt import save_sparkml
prototype = np.array(some_shape, dtype=some_dtype) # Equivalent to the input of the model
save_sparkml(spark_model, path, prototype=prototype)
# or
# some_shape has to be a tuple and some_dtype has to be a np.dtype, np.dtype.type or str object
save_sparkml(spark_model, path, shape=some_shape, dtype=some_dtype)
# or
# some_shape has to be a tuple and some_dtype has to be a np.dtype, np.dtype.type or str object
inital_types = utils.guess_onnx_tensortype(shape=shape, dtype=dtype)
save_sparkml(spark_model, path, initial_types=initial_types)Sklearn and sparkml models will be converted to ONNX first and then save to the disk. These models can be executed using ONNXRuntime, RedisAI etc. ONNX conversion needs to know the type of the input nodes and hence we have to pass shape & dtype or a prototype from where the utility can infer the shape & dtype or an initial_type object which is understood by the conversion utility. Frameworks like sparkml allows users to have heterogeneous inputs with more than one type. In such cases, use guess_onnx_tensortypes and create more than one initial_types which can be passed to save function as a list
Loading model & script
Loading function can load both single file models like freezed tensorflow model or torchscript model or onnx model as well as SavedModel from tensorflow
model = ml2rt.load_model(path)
script = ml2rt.load_script(script)
![dependabot[bot]](https://avatars.githubusercontent.com/u/49699333?size=120)