{"id":24127032,"url":"https://github.com/arthurdjn/nets","last_synced_at":"2025-06-15T20:17:09.904Z","repository":{"id":57445686,"uuid":"249670230","full_name":"arthurdjn/nets","owner":"arthurdjn","description":"Recreating PyTorch from scratch, using Numpy. Supports FCN, CNN, RNN layers.","archived":false,"fork":false,"pushed_at":"2021-02-21T21:17:50.000Z","size":11250,"stargazers_count":32,"open_issues_count":0,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-11-06T13:50:41.894Z","etag":null,"topics":["autograd","back-propagation","convolutional-neural-networks","deep-learning","deep-learning-library","framework","fully-convolutional-networks","gradient-descent","machine-learning","machine-learning-projects","neural-network","numpy","numpy-library","python","recurrent-neural-networks"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/arthurdjn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-03-24T09:51:39.000Z","updated_at":"2024-10-29T08:24:01.000Z","dependencies_parsed_at":"2022-09-26T16:31:00.084Z","dependency_job_id":null,"html_url":"https://github.com/arthurdjn/nets","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arthurdjn%2Fnets","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arthurdjn%2Fnets/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arthurdjn%2Fnets/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arthurdjn%2Fnets/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/arthurdjn","download_url":"https://codeload.github.com/arthurdjn/nets/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":233401878,"owners_count":18670817,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autograd","back-propagation","convolutional-neural-networks","deep-learning","deep-learning-library","framework","fully-convolutional-networks","gradient-descent","machine-learning","machine-learning-projects","neural-network","numpy","numpy-library","python","recurrent-neural-networks"],"created_at":"2025-01-11T17:26:15.495Z","updated_at":"2025-01-11T17:26:16.111Z","avatar_url":"https://github.com/arthurdjn.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"![python](https://img.shields.io/badge/python-3.7+-blue.svg)\n[![pypi](https://img.shields.io/pypi/v/nets)](https://pypi.org/project/nets/)\n[![status](https://img.shields.io/pypi/status/nets)](https://pypi.org/project/nets/)\n[![docs](https://img.shields.io/readthedocs/lyricsfandom)](https://arthurdjn.github.io/nets/)\n[![website](https://img.shields.io/website?url=http%3A%2F%2Farthurdujardin.com%2Fprojects%2Fnets.html)](https://arthurdujardin.com/projects/nets.html)\n\n\u003cimg src=\"img/nets.png\" width=\"100%\"\u003e\n\n# 1. Table of Content\n\n\u003c!-- TOC --\u003e\n\n- [1. Table of Content](#1-table-of-content)\n- [2. Overview](#2-overview)\n    - [2.1. About](#21-about)\n    - [2.2. Requirements](#22-requirements)\n    - [2.3. Installation](#23-installation)\n- [3. Status](#3-status)\n- [4. Documentation](#4-documentation)\n- [5. Get Started](#5-get-started)\n    - [5.1. Computational Graph](#51-computational-graph)\n    - [5.2. Building a model](#52-building-a-model)\n- [6. Notebooks](#6-notebooks)\n- [7. References](#7-references)\n\n\u003c!-- /TOC --\u003e\n\n# 2. Overview\n\n## 2.1. About\n\n**NETS** is a light-weight Deep Learning **Python** package, made using only (mostly) **numpy**.\nThis project was first introduced as an assignment at the\n[University of Oslo](https://www.uio.no/studier/emner/matnat/ifi/IN5400/), which is similar to the second\nassignment from [Stanford University](http://cs231n.stanford.edu/syllabus.html).\n\nHowever, this project was pushed further to make it _OOP_ with an easier API.\nIn addition, the back-propagation and update rules where changed, using a custom **autograd** system.\n**NETS** was highly inspired from [PyTorch](https://pytorch.org/) and [TensorFlow](https://www.tensorflow.org/)\npackages.\n\n### But why ?\n\n**NETS** package has **NO CLAIMS** to shadow already well build deep learning packages like **PyTorch**\nor **TensorFlow**. Instead, this package was made to understand how all of these libraries work and handle\nforward / backward propagation by making one from scratch.\nAs I am going through this _deep_ understanding, I found interesting to share\nas much as possible my work, which I hope will help students or people who want to learn more about this subject.\n\n## 2.2. Requirements\n\nAll packages within **NETS** are made from scratch, using mainly **numpy**. However, some additional\npackages can offer a better experience if installed (saving checkpoints and models for example).\n\n-   **numpy**\n-   **json** (Optional)\n-   **time** (Optional)\n-   **pandas** (Optional)\n-   **scipy** (Optional)\n-   **sklearn** (Optional)\n\n## 2.3. Installation\n\nTo install this package from [PyPi](https://pypi.org)\n\n```css\n$ pip install nets\n```\n\nor from this repository\n\n```css\n$ git clone https://github.com/arthurdjn/nets\n$ cd nets\n$ pip install .\n```\n\n# 3. Status\n\n| Development                  | Status      | Feature                                                                |\n| ---------------------------- | ----------- | ---------------------------------------------------------------------- |\n| Autograd System              | finished    | \u003cul\u003e\u003cli\u003e[x] Tensor\u003c/li\u003e\u003cli\u003e[x] Parameter\u003c/li\u003e\u003c/ul\u003e                     |\n| Optimization                 | finished    | \u003cul\u003e\u003cli\u003e[x] SGD\u003c/li\u003e\u003cli\u003e[x] Adam\u003c/li\u003e\u003cli\u003e[x] RMSprop\u003c/li\u003e\u003c/ul\u003e         |\n| Loss                         | in progress | \u003cul\u003e\u003cli\u003e[x] MSE\u003c/li\u003e\u003cli\u003e[x] Cross Entropy\u003c/li\u003e\u003cli\u003e[ ] BCE\u003c/li\u003e\u003c/ul\u003e    |\n| Solver                       | finished    | \u003cul\u003e\u003cli\u003e[x] Train\u003c/li\u003e\u003cli\u003e[x] Eval\u003c/li\u003e\u003cli\u003e[x] Checkpoints\u003c/li\u003e\u003c/ul\u003e   |\n| Data                         | finished    | \u003cul\u003e\u003cli\u003e[x] Dataset\u003c/li\u003e\u003cli\u003e[x] Batch\u003c/li\u003e\u003cli\u003e[x] Iterator\u003c/li\u003e\u003c/ul\u003e   |\n| Dense Neural Network         | finished    | \u003cul\u003e\u003cli\u003e[x] Linear\u003c/li\u003e\u003cli\u003e[x] Sequential\u003c/li\u003e\u003c/ul\u003e                    |\n| Convolutional Neural Network | finished    | \u003cul\u003e\u003cli\u003e[x] Conv2d\u003c/li\u003e\u003cli\u003e[x] MaxPool2d\u003c/li\u003e\u003cli\u003e[x] Dropout\u003c/li\u003e\u003c/ul\u003e |\n| Recurrent Neural Network     | in progress | \u003cul\u003e\u003cli\u003e[x] RNN\u003c/li\u003e\u003cli\u003e[ ] LSTM\u003c/li\u003e\u003cli\u003e[ ] GRU\u003c/li\u003e\u003c/ul\u003e             |\n\n# 4. Documentation\n\nThe documentation and tutorials are in process and will be released soon.\nYou will find some tutorials and application on how to get started or build a similar package.\n\n# 5. Get Started\n\n**NETS** architecture follows the one from **PyTorch**.\nIt provides a basic neural network structure so you can create your own with numpy. You will need to\nwrap your arrays in a `Tensor` class to keep track of the gradients, just like in **PyTorch**.\n\n![NETS](img/xor.gif)\n\n## 5.1. Computational Graph\n\n**NETS** uses a forward \u0026 backward pass for gradient descent optimizations (NOTE: there are now other optimizers !).\n\nYou can also uses the autograd system (recommended). It behaves similarly as Pytorch, except it is entirely done with NumPy.\n\n```python\nimport nets\n\n\nt1 = nets.Tensor([1, 2, 3], require_grad=True)\nt2 = nets.Tensor([4, 5, 6])\n\nt3 = t1 + t2  \n# t3 now requires gradient\nt3 = t3.sum()\n# Compute the gradients for t1\nt3.backward()\n```\n\n## 5.2. Building a model\n\nA model is a `Module`subclass, where biases, weights and parameters transformations are computed.\nAll modules have a `forward` method, that MUST be overwritten.\nThis method will compute the forward propagation from an input tensor, and compute the transformation.\nIf using the `autograd` system, no back-propagation need to be added. However,\nif you prefer to manually compute the gradients, you will need to override the `backward` method.\n\nYour `Model` should inherits from the `Module` class and override\nthe `forward` method.\n\n```python\nimport nets\nimport nets.nn as nn\n\nclass Model(nn.Module):\n    \"\"\"\n    Create your own model.\n    The attributes should be your submodels used during the forward pass.\n    You don't have to necessary affect the activation function as an attribute,\n    unless you want to set a manual backward pass.\n    \"\"\"\n    def __init__(self, input_dim, hidden_dim, output_dim):\n        # Initialization\n        super().__init__() # Don't forget to add this line\n        self.layer1 = nn.Linear(input_dim, hidden_dim)\n        self.layer2 = nn.Linear(hidden_dim, hidden_dim)\n        self.layer3 = nn.Linear(hidden_dim, output_dim)\n\n    def forward(self, inputs):\n        # Forward pass\n        out1 = nets.tanh(self.layer1(inputs))\n        out2 = nets.tanh(self.layer2(out1))\n        return self.layer3(out2)\n\nmodel = Model(10, 100, 2)\n\n# Let's check the architecture\nmodel\n```\n\nOut:\n\n```pycon\nModel(\n   (layer1): Linear(input_dim=10, output_dim=100, bias=True)\n   (layer2): Linear(input_dim=100, output_dim=100, bias=True)\n   (layer3): Linear(input_dim=100, output_dim=2, bias=True)\n)\n```\n\nAgain, this is really similar to what **PyTorch** offers.\n\n# 6. Notebooks\n\n-   0 - [Getting Started with NETS](examples/notebooks/0_Getting_Started_with_NETS.ipynb)\n\n    This tutorial highlights the main part and modules of **NETS**.\n\n-   1 - [Build an Autograd System with NumPy]()\n\n    To be released\n\n-   2 - [Build a Feed Forward Neural Network with NumPy](examples/notebooks/1_Image_Classifier_with_PyTorch.ipynb)\n\n    Build a Fully Connected Neural Network for image classification.\n\n-   3 - [Build a Convolutional Neural Network with NumPy](examples/notebooks/2_Build_a_CNN_with_NETS.ipynb)\n\n    Build a Convolutional Neural Network for image classification.\n\n# 7. References\n\nHere is a list of tutorials and lectures/assignment that helped to develop **NETS**\n\n-   [PyTorch documentation](https://pytorch.org)\n-   [PyTorch autograd tutorial](https://pytorch.org/tutorials/beginner/blitz/autograd_tutorial.html)\n-   [Joel Grus autograd tutorial](https://github.com/joelgrus/autograd/tree/part06)\n-   [Joel Grus autograd live coding](https://www.youtube.com/watch?v=RxmBukb-Om4)\n-   [Stanford University cs231n 2nd assignment](http://cs231n.github.io/)\n-   [University of Oslo in5400 1st assignment](https://www.uio.no/studier/emner/matnat/ifi/IN5400/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farthurdjn%2Fnets","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Farthurdjn%2Fnets","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farthurdjn%2Fnets/lists"}