Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/facundoq/edunn

Learn how modern, modular Neural Networks work by implementing a PyTorch/Keras-like framework.
https://github.com/facundoq/edunn

framework keras neural-networks numpy pytorch

Last synced: about 1 month ago
JSON representation

Learn how modern, modular Neural Networks work by implementing a PyTorch/Keras-like framework.

Awesome Lists containing this project

README 

        



#  `edunn`



## :brain: Learn how modern, modular Neural Networks work by implementing a PyTorch/Keras-like framework.



`edunn` is a very simple library that contains the basic elements to define, train and use **modular** Neural Networks using just Python and Numpy ([full example](https://colab.research.google.com/github/facundoq/edunn/blob/main/samples/iris_classification.ipynb)).



````python

import edunn as nn

x,y = load_data()

din,n_classes=4,10

layers = [nn.Linear(din,10),

          nn.Bias(10),

          nn.ReLU(),

          nn.Linear(10,n_classes),

          nn.Bias(n_classes),

          nn.Softmax()

          ]



model = nn.Sequential(layers)

print(model.summary())

error = nn.MeanError(nn.CrossEntropyWithLabels())

optimizer = nn.GradientDescent(lr=0.1,epochs=3000,batch_size=32)

history = optimizer.optimize(model,x,y,error)

nn.plot.plot_history(history)

````



We _remove_ the key parts of `edunn` that deal with layers, initializers, error functions, backpropagation and optimizers. Then, through our guides, we help **_you_ reimplement** the key parts of `edunn`. We also provide various automated tests along the way for you to check your solutions. 



Supported languages:



* [Spanish](https://github.com/facundoq/edunn/blob/main/releases/edunn-es.zip)

* English (coming soon)



## :rainbow: Why `edunn`?



There are many excellent courses and books that teach you how to program neural networks from the ground up such as [*CS231n: Convolutional Neural Networks for Visual Recognition*](https://cs231n.github.io/), Andrew Ng's [*Machine Learning*](https://www.coursera.org/learn/machine-learning) or Sebastian Rascha's [Python Machine Learning](https://sebastianraschka.com/books/#python-machine-learning-3rd-edition). However, while these are great for understanding the basics using `numpy`, they use monolithic neural network models that make it difficult to understand how that code translates to other frameworks such as `pytorch` or `tensorflow`.



Alternatively, [Andrew Ng's *Deep Learning*](https://www.coursera.org/specializations/deep-learning) or [*FastAI*](https://course19.fast.ai/part2) build modular networks out of `tensorflow` or basic `pytorch` operators instead of numpy. This is great for building complex models quickly, but there's still a lot of magic under the hood. Both frameworks automatically derive `backward` methods and bring along other goodies. 



Implementing `edunn` allows you to understand how modern neural networks frameworks work and fills the gap between those types of learning. You get to program a full neural network framework, without all the magic `pytorch` or `tensorflow` bring along. Of course, `edunn` guides help you out along the way and provide a clear path to complete the implementation of the library. Also, in this case you'd actually be _reimplementing_ `edunn`, since we provide a reference implementation you can check when in doubt with your solutions.



## :heavy_plus_sign: Pre-requisite knowledge



The guides are intended for learners with some experience with Python, Numpy and Neural Networks. We have included explanations of what you must implement, but learners still should be able to:



1. Read and write object-oriented python code, including subclassing and method overriding.

2. Understand Numpy, basic linear algebra and calculus and be able to translate  mathematical operations to Numpy code.

3. Understand the basic ideas in modern modular Neural Network frameworks, such as models, layers, optimizers, batches, backward and forward passes, and specially backpropagation. It is also helpful but not necessary to have previous exposure to `tensorflow/keras` or `pytorch`.



## :wrench: Download and setup



**Do not clone** this repository if you want to use `edunn` as a learning exercise. Instead, follow these steps to get a working environment for the guides. These instructions assume a working installation of python3 (3.6 or greater), pip3 and virtualenv. The installations of those vary wildly with OS/distribution so it's up to you to get a setup working.



Afterward: 



1. Download a [release](https://github.com/facundoq/edunn/blob/main/releases) for your language. If you prefer the command line, use `wget`, replacing `es` with your preferred language (only `es` is currently supported):



    `wget https://github.com/facundoq/edunn/blob/main/releases/edunn-es.zip`



2. Unzip to `edunn` or other folder name you choose:



    `unzip es.zip -d edunn && cd edunn`



3. Make sure Python3.6 or greater, pip3 and venv are installed:



    ````bash

    python3 --version

    pip3 --version && 

    python3 -c "import venv" && echo "venv is installed"

    ````

   



4. Create a virtualenv environment and install the dependencies in the requirements file `requirements.txt`: 



    ````bash

    python3 -m venv nnenv

    source nnvenv/bin/activate

    pip install -r requirements.txt

    ````



Alternatively, you can use your `conda` distribution or another such tool to create a virtual environment and install the required libraries for `edunn` (listed inside `requirements.txt`). 



5. Run jupyter and follow the guides:



```bash

jupyter notebook

```



## :pill: Solutions and bugs :bug: 



`edunn` is designed so that you can automatically test your implementations. However, it is intended to be used as a set of exercises in a course or seminar. Please address all questions regarding the material to your instructor.



This repository also has a [reference implementation](https://github.com/facundoq/edunn/tree/main/edunn) in the `edunn` folder of the library you can consult. 



Alternatively, you may consult public forums such as [stack overflow](stackoverflow.com/), [r/neuralnetworks](https://www.reddit.com/r/neuralnetworks/) or [r/MachineLearning](https://www.reddit.com/r/MachineLearning)



Please, **only [file an issue](issues) if there is an actual bug** or feature request, not if you can't solve the exercises.  



## :package: Reference implementation and usage as library



You can `pip install edunn` to use the reference implementation of the library to train models and use them. Indeed, the API is very simple:



````python

import edunn as nn



x, y, n_classes = ...  # load data 

n, din = x.shape



# Model definition

layers = [nn.Linear(din, 10),

          nn.Bias(10),

          nn.ReLU(),

          nn.Linear(10, 20),

          nn.Bias(20),

          nn.TanH(),

          nn.Linear(10, n_classes),

          nn.Bias(n_classes),

          nn.Softmax()

          ]



model = nn.Sequential(layers)

print(model.summary())



error = nn.MeanError(nn.CrossEntropyWithLabels())

optimizer = nn.StochasticGradientDescent(lr=0.001, epochs=1000, batch_size=32)

history = optimizer.optimize(model, x, y, error)

````



However, we **do not recommend** for serious projects since the library is *very slow* and not meant for neither research nor production environments. The reference implementation is mostly focused on being easy to understand so that learners can check their implementations.



## :busts_in_silhouette: Contributing



We'd like your help to expand `edunn` with more guides/layers/optimizers. Check out our [contribution guide](CONTRIBUTE.md)!