{"id":28025443,"url":"https://github.com/visvav/micrograd-mlp---tiny-neural-network-from-scratch-using-autograd","last_synced_at":"2025-06-21T09:05:23.734Z","repository":{"id":291915110,"uuid":"979196577","full_name":"VisvaV/Micrograd-MLP---Tiny-Neural-Network-from-Scratch-using-Autograd","owner":"VisvaV","description":"This repository is a clean, educational reimplementation of Andrej Karpathy's micrograd project. It includes:  - A minimal autograd engine (`Value` class) built from scratch - Neuron, Layer, and MLP abstractions built directly on top of `Value` - Manual training loop using scalar-based backpropagation along with Visualization","archived":false,"fork":false,"pushed_at":"2025-05-07T18:29:39.000Z","size":3364,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-21T09:04:58.004Z","etag":null,"topics":["autograd","backpropagation","deep-learning","educational","micrograd","mlp","neural-networks","python","scalar-autograd"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/VisvaV.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-07T06:31:41.000Z","updated_at":"2025-05-07T18:34:59.000Z","dependencies_parsed_at":"2025-05-11T04:45:25.621Z","dependency_job_id":null,"html_url":"https://github.com/VisvaV/Micrograd-MLP---Tiny-Neural-Network-from-Scratch-using-Autograd","commit_stats":null,"previous_names":["visvav/micrograd","visvav/micrograd-mlp---tiny-neural-network-from-scratch-using-autograd"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/VisvaV/Micrograd-MLP---Tiny-Neural-Network-from-Scratch-using-Autograd","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VisvaV%2FMicrograd-MLP---Tiny-Neural-Network-from-Scratch-using-Autograd","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VisvaV%2FMicrograd-MLP---Tiny-Neural-Network-from-Scratch-using-Autograd/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VisvaV%2FMicrograd-MLP---Tiny-Neural-Network-from-Scratch-using-Autograd/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VisvaV%2FMicrograd-MLP---Tiny-Neural-Network-from-Scratch-using-Autograd/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/VisvaV","download_url":"https://codeload.github.com/VisvaV/Micrograd-MLP---Tiny-Neural-Network-from-Scratch-using-Autograd/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VisvaV%2FMicrograd-MLP---Tiny-Neural-Network-from-Scratch-using-Autograd/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261095307,"owners_count":23108782,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autograd","backpropagation","deep-learning","educational","micrograd","mlp","neural-networks","python","scalar-autograd"],"created_at":"2025-05-11T04:22:48.981Z","updated_at":"2025-06-21T09:05:18.715Z","avatar_url":"https://github.com/VisvaV.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Micrograd MLP - A Neural Network from Scratch Using Scalar Autograd\n\nThis repository is an educational implementation of a minimalist neural network framework, inspired by Andrej Karpathy’s [micrograd](https://github.com/karpathy/micrograd). It is written entirely in Python and operates on scalar values using a custom automatic differentiation engine. The purpose of this project is to provide a deep understanding of how forward and backward propagation work at the lowest level.\n\nThis implementation includes:\n- A scalar-based `Value` class supporting autodiff.\n- Neuron, Layer, and MLP classes to construct fully connected feedforward neural networks.\n- Manual training loop with gradient descent.\n- Graphviz-based visualization of the computation graph.\n\n## Table of Contents\n\n- [Project Structure](#project-structure)\n- [How It Works](#how-it-works)\n- [Installation](#installation)\n- [Usage](#usage)\n- [Computation Graph Visualization](#computation-graph-visualization)\n- [Requirements](#requirements)\n- [References](#references)\n- [License](#license)\n\n## Project Structure\n\n```\n.\n├── micrograd_engine.py      # Core autograd engine (Value class)\n├── mlp_model.py             # Neural network construction and training loop\n├── graph.py                 # Graphviz-based visualization of computation graph\n├── requirements.txt         # Python dependencies\n└── README.md                # This documentation file\n```\n\n## How It Works\n\n### 1. `Value` Class\n\nDefined in `micrograd_engine.py`, this is the core of the autodiff system.\n\n- Supports arithmetic operations: `+`, `-`, `*`, `/`, `**`\n- Tracks the computation graph dynamically by recording parent-child relationships between operations.\n- Uses backward mode autodiff to compute gradients via `backward()`.\n\n### 2. `Neuron`, `Layer`, and `MLP`\n\nLocated in `mlp_model.py`:\n\n- `Neuron`: Computes `tanh(w·x + b)` from input list `x`.\n- `Layer`: A fully connected layer consisting of multiple neurons.\n- `MLP`: A multi-layer perceptron composed of multiple layers.\n\nThese classes are designed to mimic a real neural network using only the scalar `Value` class.\n\n### 3. Training Loop\n\nThe model is trained on a toy dataset using mean squared error loss. Gradients are computed via `loss.backward()` and weights are updated using manual gradient descent.\n\n### 4. Visualization\n\nThe computation graph of the final loss is visualized using `graph.py`, which uses Graphviz. The result is saved as a `.png` file.\n\n## Installation\n\nClone the repository and install dependencies:\n\n```bash\ngit clone https://github.com/VisvaV/micrograd-mlp.git\ncd micrograd-mlp\npip install -r requirements.txt\n```\n\nYou also need to install Graphviz system package:\n\n- Windows: Download from https://graphviz.org/download/\n- Ubuntu/Debian: `sudo apt install graphviz`\n- macOS (brew): `brew install graphviz`\n\nMake sure the `dot` executable is in your system PATH.\n\n## Usage\n\nTo train the model and visualize the computation graph, run:\n\n```bash\npython mlp_model.py\n```\n\nThis will:\n\n- Train a 3-layer MLP on a small dataset.\n- Print loss at each step.\n- Render the computation graph of the final loss to `computation_graph.png`.\n\n## Computation Graph Visualization\n\nAt the end of training, a graph of the computation tree for the final scalar loss is rendered. This helps understand how values and gradients flow backward through the model.\n\nThe graph shows:\n\n- Nodes for every intermediate `Value`\n- Operations like `+`, `*`, `tanh`\n- Connections showing dependency and flow\n\nTo view the rendered graph:\n\n```bash\nopen computation_graph.png  # macOS\nstart computation_graph.png # Windows\nxdg-open computation_graph.png # Linux\n```\n\n## Requirements\n\nSee `requirements.txt`\n\n## References\n\n- Andrej Karpathy’s [micrograd](https://github.com/karpathy/micrograd)\n- Graphviz documentation: https://graphviz.org/\n\n## License\n\nThis project is open source under the MIT license.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvisvav%2Fmicrograd-mlp---tiny-neural-network-from-scratch-using-autograd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvisvav%2Fmicrograd-mlp---tiny-neural-network-from-scratch-using-autograd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvisvav%2Fmicrograd-mlp---tiny-neural-network-from-scratch-using-autograd/lists"}