Echo-AI
Python package containing all mathematical backend algorithms used in Machine Learning. The full documentation for Echo is provided here.
Table of Contents
About
Echo-AI Package is created to provide an implementation of the most promising mathematical algorithms, which are missing in the most popular deep learning libraries, such as PyTorch, Keras and TensorFlow.
Activation Functions
The package contains implementation for following activation functions (
| # | Function | Equation | Keras | PyTorch | TensorFlow-Keras | TensorFlow - Core |
|---|---|---|---|---|---|---|
| 1 | Weighted Tanh | |||||
| 2 | Swish | |||||
| 3 | ESwish | |||||
| 4 | Aria2 | |||||
| 5 | ELiSH | |||||
| 6 | HardELiSH | |||||
| 7 | Mila | |||||
| 8 | SineReLU | |||||
| 9 | Flatten T-Swish | |||||
| 10 | SQNL | |||||
| 11 | ISRU | |||||
| 12 | ISRLU | |||||
| 13 | Bent's identity | |||||
| 14 | Soft Clipping | |||||
| 15 | SReLU | |||||
| 15 | BReLU | |||||
| 16 | APL | |||||
| 17 | Soft Exponential | |||||
| 18 | Maxout | |||||
| 19 | Mish | |||||
| 20 | Beta Mish | |||||
| 21 | RReLU | ⬜ | ||||
| 22 | CELU | ⬜ | ||||
| 23 | ReLU6 | ⬜ | ||||
| 24 | HardTanh | ⬜ | ||||
| 25 | GLU | ⬜ | ||||
| 26 | LogSigmoid | ⬜ | ||||
| 27 | TanhShrink | ⬜ | ||||
| 28 | HardShrink | ⬜ | ||||
| 29 | SoftShrink | ⬜ | ||||
| 30 | SoftMin | ⬜ | ||||
| 31 | LogSoftmax | ⬜ | ||||
| 32 | Gumbel-Softmax | ⬜ |
Repository Structure
The repository has the following structure:
- echoAI # main package directory
| - Activation # sub-package containing activation functions implementation
| |- Torch # sub-package containing implementation for PyTorch
| | | - functional.py # script which contains implementation of activation functions
| | | - weightedTanh.py # activation functions wrapper class for PyTorch
| | | - ... # PyTorch activation functions wrappers
| |- Keras # sub-package containing implementation for Keras
| | | - custom_activations.py # script which contains implementation of activation functions
| |- TF_Keras # sub-package containing implementation for Tensorflow-Keras
| | | - custom_activation.py # script which contains implementation of activation functions
| - __init__.py
- Observations # Folder containing other assets
- docs # Sphinx documentation folder
- LICENSE # license file
- README.md
- setup.py # package setup file
- Smoke_tests # folder, which contains scripts with demonstration of activation functions usage
- Unit_tests # folder, which contains unit test scriptsSetup Instructions
To install echoAI package from PyPI run the following command:
$ pip install echoAI
Code Examples:
Sample scripts are provided in Smoke_tests folder. You can use activation functions from echoAI as simple as this:
# import PyTorch
import torch
# import activation function from echoAI
from echoAI.Activation.Torch.mish import Mish
# apply activation function
mish = Mish()
t = torch.tensor(0.1)
t_mish = mish(t)
