{"id":34628112,"url":"https://github.com/pranftw/neograd","last_synced_at":"2025-12-24T16:14:43.788Z","repository":{"id":62592977,"uuid":"527663026","full_name":"pranftw/neograd","owner":"pranftw","description":"A deep learning framework created from scratch with Python and NumPy","archived":false,"fork":false,"pushed_at":"2022-12-26T16:35:05.000Z","size":1250,"stargazers_count":237,"open_issues_count":0,"forks_count":9,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-09-04T23:14:46.554Z","etag":null,"topics":["ai","autograd","automatic-differentiation","deep-learning","deep-learning-framework","machine-learning","neural-network","numpy","python","pytorch-api","scratch-implementation"],"latest_commit_sha":null,"homepage":"https://neograd.readthedocs.io/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pranftw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-08-22T17:18:40.000Z","updated_at":"2025-06-04T15:40:12.000Z","dependencies_parsed_at":"2023-01-31T00:30:18.711Z","dependency_job_id":null,"html_url":"https://github.com/pranftw/neograd","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/pranftw/neograd","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pranftw%2Fneograd","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pranftw%2Fneograd/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pranftw%2Fneograd/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pranftw%2Fneograd/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pranftw","download_url":"https://codeload.github.com/pranftw/neograd/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pranftw%2Fneograd/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28004520,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-24T02:00:07.193Z","response_time":83,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","autograd","automatic-differentiation","deep-learning","deep-learning-framework","machine-learning","neural-network","numpy","python","pytorch-api","scratch-implementation"],"created_at":"2025-12-24T16:14:43.251Z","updated_at":"2025-12-24T16:14:43.775Z","avatar_url":"https://github.com/pranftw.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Neograd\n### A Deep Learning framework created from scratch with Python and NumPy\n\n\u003cbr\u003e\n\u003cdiv align=\"center\"\u003e\n  \u003cimg width=\"251\" alt=\"image\" src=\"https://github.com/pranftw/neograd/raw/main/ng.png\"\u003e\n\u003c/div\u003e\n\u003cbr\u003e\n\n[![Neograd Tests](https://github.com/pranftw/neograd/actions/workflows/python-app.yml/badge.svg)](https://github.com/pranftw/neograd/actions/workflows/python-app.yml)\n[![Documentation Status](https://readthedocs.org/projects/neograd/badge/?version=latest)](https://neograd.readthedocs.io/en/latest/?badge=latest)\n[![Downloads](https://static.pepy.tech/personalized-badge/neograd?period=total\u0026units=international_system\u0026left_color=blue\u0026right_color=orange\u0026left_text=Downloads)](https://pepy.tech/project/neograd)\n[![DOI](https://zenodo.org/badge/527663026.svg)](https://zenodo.org/badge/latestdoi/527663026)\n\n## Get started\n### Installation\n`pip install neograd`\n\n### PyPI\nhttps://pypi.org/project/neograd/\n\n### Documentation\nhttps://neograd.readthedocs.io/\n\n### Explore on Colab\nhttps://colab.research.google.com/drive/1D4JgBwKgnNQ8Q5DpninB6rdFUidRbjwM?usp=sharing\nhttps://colab.research.google.com/drive/184916aB5alIyM_xCa0qWnZAL35fDa43L?usp=sharing\n\n## Motivation\nI firmly believe that in order to understand something completely, you have to build it on your own from scratch. I used to do gradient calculation analytically, and thought that autograd was some kind of magic. So this was initially built to understand autograd but later on its scope was extended. You might be wondering, there are already many frameworks like TensorFlow and PyTorch that are very popular, and why did I have to create another one? The answer is that these have very complex codebases that are difficult to grasp. So I intend that this repository be used as an educational tool in order to understand how things work under the hood in these giant frameworks, with code that is intuitive and easily readable.\n\n## Features\n### Automatic Differentiation\n`autograd` offers automatic differentiation, implemented for the most commonly required operations for vectors of any dimension, with broadcasting capabilities\n```python\nimport neograd as ng\na = ng.tensor(3, requires_grad=True)\nb = ng.tensor([1,2,3], requires_grad=True)\nc = a+b\nc.backward([1,1,1])\nprint(a.grad)\nprint(b.grad)\n```\n### Custom autograd operations\nIf you wanted a custom operation to have `autograd` capabilities, those can be defined with very simple interface each having a forward method and a backward method\n```python\nclass Custom(Operation):\n  def forward(self):\n    pass\n  def backward(self):\n    pass\n```\n### Gradient Checking\nDebug your models/functions with Gradient Checking, to ensure that the gradients are getting propagated correctly\n### Highly customizable\nCreate your own custom layers, optimizers, loss functions which provides more flexibility to create anything you\ndesire\n### PyTorch like API\nPyTorch's API is one of the best and one the most elegant API designs, so we've leveraged the same\n### Neural Network module\n`nn` contains some of the most commonly used optimizers, activations and loss functions required to train a Neural Network\n### Save and Load weights, model\nTrained a model already? Then save the weights onto a file and load them whenever required or save the entire model, onto a file\n### Checkpoints\nLet's say you're training a model and your computer runs out of juice and if you'd waited until training was finished, to save the weights, then you'd lose all the weights. To prevent this, checkpoint your model with various sessions to save the weights during regular intervals with additional supporting data\n\n## Example\n```python\nimport neograd as ng\nfrom neograd import nn\nimport numpy as np\nfrom neograd.nn.loss import BCE\nfrom neograd.nn.optim import Adam\nfrom neograd.autograd.utils import grad_check\nfrom sklearn.datasets import make_circles\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import classification_report, accuracy_score\n\n# load dataset (binary classification problem)\nX, y = make_circles(n_samples=1000, noise=0.05, random_state=100)\nX_train, X_test, y_train, y_test = train_test_split(X,y)\n\nnum_train = 750 # number of train examples\nnum_test = 250 # number of test examples\nnum_iter = 50 # number of training iterations\n\n# convert data into tensors\nX_train, X_test = ng.tensor(X_train[:num_train,:]), ng.tensor(X_test[:num_test,:])\ny_train, y_test = ng.tensor(y_train[:num_train].reshape(num_train,1)), ng.tensor(y_test[:num_test].reshape(num_test,1))\n\n# define the structure of your neural net\nclass NN(nn.Model):\n  def __init__(self):\n    self.stack = nn.Sequential(\n      nn.Linear(2,100),\n      nn.ReLU(),\n      nn.Linear(100,1),\n      nn.Sigmoid()\n    )\n  \n  def forward(self, inputs):\n    return self.stack(inputs)\n\nmodel = NN() # initialize a model\nloss_fn = BCE() # initialize a loss function (Binary Cross Entropy)\noptim = Adam(model.parameters(), 0.05) # initialize an optimizer\n\n# training loop\nfor i in range(num_iter):\n  optim.zero_grad() # zero out the gradients in the tensors\n  outputs = model(X_train) # get the outputs by passing the training data to your model\n  loss = loss_fn(outputs, y_train) # calculate the loss\n  loss.backward() # initiate the backward pass to calculate the gradients\n  optim.step() # update the parameters\n  print(f\"iter {i+1}/{num_iter}\\nloss: {loss}\\n\")\n\nwith model.eval(): # put the model in evaluation mode\n  test_outputs = model(X_test) # get the outputs of the model on test data\n  preds = np.where(test_outputs.data\u003e=0.5, 1, 0) # make predictions\n\nprint(classification_report(y_test.data.astype(int).flatten(), preds.flatten()))\nprint(accuracy_score(y_test.data.astype(int).flatten(), preds.flatten()))\n\ngrad_check(model, X_train, y_train, loss_fn) # perform gradient checking in your model\n```\n\n## How is this any different from\n- **Andrej Karpathy's [micrograd](https://github.com/karpathy/micrograd)**\u003cbr\u003e\n  Natively only supports scalar values for computation, whereas we support scalars, vectors, matrices all compatible with NumPy broadcasting\n- **George Hotz's [tinygrad](https://github.com/geohot/tinygrad)**\u003cbr\u003e\n  Has an obligation to be under 1000 lines of code leading to cramped up code, therefore our implementation is so much more readable and easily understandable. Also, no dealing with C/C++ code used in tinygrad for GPU acceleration\n- **[pytorch](https://github.com/pytorch/pytorch), [tensorflow](https://github.com/tensorflow/tensorflow), etc**\u003cbr\u003e\n  Large messy codebases written mostly in C/C++ for efficiency making it impossible to find you're way around and understand stuff. We've a pure Python implementation making it easy to get started and understand what's going on under the hood\n  \n## Resources\n- A big thank you to Andrej Karpathy for his CS231n [lecture](https://youtu.be/i94OvYb6noo) on Backpropagation which was instrumental in helping me gain a good grasp of the basic mechanisms of autograd\n- Thanks to Terance Parr and Jeremy Howard for their [paper](https://arxiv.org/abs/1802.01528) The Matrix Calculus You Need For Deep Learning which helped me get rid of my fear for matrix calculus, that is beautifully written starting from the very fundamentals and slowly transitioning into advanced topics","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpranftw%2Fneograd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpranftw%2Fneograd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpranftw%2Fneograd/lists"}