candl

A tiny, pedagogical implementation of a neural network library with a pytorch-like API. The primary use of this library is for education. Use the actual pytorch for more serious deep learning business.

The implementation is complete with tensor-valued autodiff (~100 lines) and a neural network API built off of it (~80 lines).

Learning

This little project is actually the result of an article I wrote. Using it, you can learn more about how neural networks work and implement everything in candl yourself from scratch.

Installation

pip install candl

Usage

First, import candl.

import candl

Candl comes with two modules: nn and Tensor. The nn module contains tools like modules, layers, SGD, MSE, etc. Candl tensors are extensions of numpy ndarrays that can be used to represent data and compute derivatives.

To train a neural net (let's try to learn XOR), first we can create a model.

nn = candl.nn

model = nn.Sequential(nn.Linear(2, 64), 
                      nn.ReLU(), 
                      nn.Linear(64, 32), 
                      nn.ReLU(), 
                      nn.Linear(32, 1))
lr = 1e-3

loss_fn = nn.MSE()
optimizer = nn.SGD(model.parameters(), lr)

Then, we train:

data = [([0, 0], [0]), ([0, 1], [1]), ([1, 0], [1]), ([1, 1], [0])]

for epoch in range(100):
    for sample in data:
        """ 
        Note that we only allow batches of data, so the shape of the tensor must be n x m,
        where m is the dimensionality of the input for each batch.
        """
        x = candl.tensor([sample[0]]) 
        y = candl.tensor([sample[1]])
        loss = loss_fn(model.forward(x), y)
        loss.backward()
        # The `True` argument automatically zeroes the gradients after a step
        optimizer.step(True) 

Features

• Tensors built upon numpy's ndarrays
• Tensor-valued autograd
• Mean Squared Error Loss Function
• Stochastic Gradient Descent (SGD)
• Blocks (Modules) for putting together neural networks
• Built-in layers: Linear, ReLU, Sigmoid, Tanh