edunn

🧠 Learn how modern, modular Neural Networks work by implementing a PyTorch/Keras-like framework.

edunn is a very simple library that contains the basic elements to define, train and use modular Neural Networks using just Python and Numpy (full example).

import edunn as nn
x,y = load_data()
din,n_classes=4,10
layers = [nn.Linear(din,10),
          nn.Bias(10),
          nn.ReLU(),
          nn.Linear(10,n_classes),
          nn.Bias(n_classes),
          nn.Softmax()
          ]

model = nn.Sequential(layers)
print(model.summary())
error = nn.MeanError(nn.CrossEntropyWithLabels())
optimizer = nn.GradientDescent(lr=0.1,epochs=3000,batch_size=32)
history = optimizer.optimize(model,x,y,error)
nn.plot.plot_history(history)

We remove the key parts of edunn that deal with layers, initializers, error functions, backpropagation and optimizers. Then, through our guides, we help you reimplement the key parts of edunn. We also provide various automated tests along the way for you to check your solutions.

Supported languages:

• Spanish
• English (coming soon)

🌈 Why edunn?

There are many excellent courses and books that teach you how to program neural networks from the ground up such as CS231n: Convolutional Neural Networks for Visual Recognition, Andrew Ng's Machine Learning or Sebastian Rascha's Python Machine Learning. However, while these are great for understanding the basics using numpy, they use monolithic neural network models that make it difficult to understand how that code translates to other frameworks such as pytorch or tensorflow.

Alternatively, Andrew Ng's Deep Learning or FastAI build modular networks out of tensorflow or basic pytorch operators instead of numpy. This is great for building complex models quickly, but there's still a lot of magic under the hood. Both frameworks automatically derive backward methods and bring along other goodies.

Implementing edunn allows you to understand how modern neural networks frameworks work and fills the gap between those types of learning. You get to program a full neural network framework, without all the magic pytorch or tensorflow bring along. Of course, edunn guides help you out along the way and provide a clear path to complete the implementation of the library. Also, in this case you'd actually be reimplementing edunn, since we provide a reference implementation you can check when in doubt with your solutions.

➕ Pre-requisite knowledge

The guides are intended for learners with some experience with Python, Numpy and Neural Networks. We have included explanations of what you must implement, but learners still should be able to:

1. Read and write object-oriented python code, including subclassing and method overriding.
2. Understand Numpy, basic linear algebra and calculus and be able to translate mathematical operations to Numpy code.
3. Understand the basic ideas in modern modular Neural Network frameworks, such as models, layers, optimizers, batches, backward and forward passes, and specially backpropagation. It is also helpful but not necessary to have previous exposure to tensorflow/keras or pytorch.

🔧 Download and setup

Do not clone this repository if you want to use edunn as a learning exercise. Instead, follow these steps to get a working environment for the guides. These instructions assume a working installation of python3 (3.6 or greater), pip3 and virtualenv. The installations of those vary wildly with OS/distribution so it's up to you to get a setup working.

Afterward:

1. Download a release for your language. If you prefer the command line, use wget, replacing es with your preferred language (only es is currently supported):

   wget https://github.com/facundoq/edunn/blob/main/releases/edunn-es.zip

2. Unzip to edunn or other folder name you choose:

   unzip es.zip -d edunn && cd edunn

3. Make sure Python3.6 or greater, pip3 and venv are installed:

   python3 --version
   pip3 --version && 
   python3 -c "import venv" && echo "venv is installed"

4. Create a virtualenv environment and install the dependencies in the requirements file requirements.txt:

   python3 -m venv nnenv
   source nnvenv/bin/activate
   pip install -r requirements.txt

Alternatively, you can use your conda distribution or another such tool to create a virtual environment and install the required libraries for edunn (listed inside requirements.txt).

5. Run jupyter and follow the guides:

jupyter notebook

💊 Solutions and bugs 🐛

edunn is designed so that you can automatically test your implementations. However, it is intended to be used as a set of exercises in a course or seminar. Please address all questions regarding the material to your instructor.

This repository also has a reference implementation in the edunn folder of the library you can consult.

Alternatively, you may consult public forums such as stack overflow, r/neuralnetworks or r/MachineLearning

Please, only file an issue if there is an actual bug or feature request, not if you can't solve the exercises.

📦 Reference implementation and usage as library

You can pip install edunn to use the reference implementation of the library to train models and use them. Indeed, the API is very simple:

import edunn as nn

x, y, n_classes = ...  # load data 
n, din = x.shape

# Model definition
layers = [nn.Linear(din, 10),
          nn.Bias(10),
          nn.ReLU(),
          nn.Linear(10, 20),
          nn.Bias(20),
          nn.TanH(),
          nn.Linear(10, n_classes),
          nn.Bias(n_classes),
          nn.Softmax()
          ]

model = nn.Sequential(layers)
print(model.summary())

error = nn.MeanError(nn.CrossEntropyWithLabels())
optimizer = nn.StochasticGradientDescent(lr=0.001, epochs=1000, batch_size=32)
history = optimizer.optimize(model, x, y, error)

However, we do not recommend for serious projects since the library is very slow and not meant for neither research nor production environments. The reference implementation is mostly focused on being easy to understand so that learners can check their implementations.

👥 Contributing

We'd like your help to expand edunn with more guides/layers/optimizers. Check out our contribution guide!