It is for eliminating repeat jobs of machine learning. Also it can makes your code more beautifully and Pythonic.

Table of Contents

Please see my several examples. It contains below networks using MNIST dataset:

- Logistic Regression
- Association Learning
- GAN: Generative Adversarial Network
- VAE: Variational Autoencoder
- AAE: Adversal Autoencoder

Data normalization and standardization,

```
train_xs = net.normalize (train_xs, normalize = True, standardize = True)
```

To show cumulative sum of explained_variance_ratio_ of sklearn PCA.

```
train_xs = net.normalize (train_xs, normalize = True, standardize = True, pca_k = -1)
```

Then you can decide n_components for PCA.

```
train_xs = net.normalize (train_xs, normalize = True, standardize = True, axis = 0, pca_k = 500)
```

Test dataset will be nomalized by factors of train dataset.

```
test_xs = net.normalize (test_xs)
```

This parameters will be pickled at your train directory named as *normfactors*. You can use this pickled file for serving your model.

For serving model,

```
import mydnn
net = mydnn.MyDNN ()
net.restore ('./checkpoint')
version = net.to_save_model (
'./export',
'predict_something',
inputs = {'x': net.x},
outputs={'label': net.label, 'logit': net.logit}
)
print ("version {} has been exported".format (version))
```

For testing your model,

```
from dnn import save_model
interpreter = save_model.load (model_dir, sess, graph)
y = interpreter.run (x)
```

You can serve the expoted model with TensorFlow Serving or tfserver.

Note: If you use net.normalize (train_xs), normalizing factors (mean, std, max and etc) willl be pickled and saved to model directory with tensorflow model. If you can use this file for normalizing new x data at real service.

```
from dnn import _normalize
def normalize (x):
norm_file = os.path.join (model_dir, "normfactors")
with open (norm_file, "rb") as f:
norm_factor = pickle.load (f)
return _normalize (x, *norm_factor)
```

- Required Tensorflow version 1.9*

For exporting tensorflow lite you should convert your model to save model first.

```
net.to_tflite (
"model.tflite",
save_model_dir
)
```

If you want to convert to quntized model, it will be needed additional parameters.

```
net.to_tflite (
"model.tflite",
save_model_dir,
True, # quantize
(128, 128), # mean/std stats of input value
(-1, 6) # min/max range output value of logit
)
```

For testing tflite model,

```
from dnn import tflite
interpreter = tflite.load ("model.tflite")
y = interpreter.run (x)
```

If your model is quantized, it need mean/std stats of input value,

```
from dnn import tflite
interpreter = tflite.load ("model.tflite", (128, 128))
y = interpreter.run (x)
```

If your input value range -1.0 ~ 1.0, its will be translated into 0 - 255 for qunatized model by mean and std parameters. So (128, 128) means your inout value range is -1.0 ~ 1.0. Then interpreter will qunatize x to uint8 by this parameter.

```
unit8 = (float32 x * std) + mean
```

And tflite will reverse this uinit8 to float value by,

```
float32 x = (uint8 x - mean) / std
```

There're several helper modules.

```
from dnn import costs, predutil
```

```
from dnn import split, vector
import dnn.video
import dnn.audio
import dnn.image
import dnn.text
```

You can override or add anything. If it looks good, contribute to this project please.

You should or could create these operations by overriding methods,

- train_op: create with 'make_optimizer'
- logit: create with 'DNN.make_logit'
- cost: create with 'DNN.make_cost'
- accuracy: create with 'DNN.calculate_accuracy'

- dropout_rate: if negative value, dropout rate will be selected randomly.
- is_training
- n_sample: Numner of x (or y) set. This value will be fed automatically, do not feed.

You can use predefined optimizers.

```
def make_optimizer (self):
return self.optimizer ("adam")
# Or
return self.optimizer ("rmsprob", mometum = 0.01)
```

Available optimizer names are,

- "adam"
- "rmsprob"
- "momentum"
- "clip"
- "grad"
- "adagrad"
- "adagradDA"
- "adadelta"
- "ftrl"
- "proxadagrad"
- "proxgrad"

see dnn/optimizers.py

- save
- restore
- to_save_model
- to_tflite
- reset_dir
- set_train_dir
- eval

- set_tensorboard_dir
- make_writers
- write_summary

- 0.3:
- remove trainale ()
- add set_learning_rate ()
- add argument to set_train_dir () for saving chcekpoit
- make compatible with tf 1.12.0

- 0.2
- add tensorflow lite conversion and interpreting

- 0.1: project initialized