{"id":15469040,"url":"https://github.com/imaddabbura/tiny-pytorch","last_synced_at":"2026-04-01T17:55:45.767Z","repository":{"id":65021057,"uuid":"578688795","full_name":"ImadDabbura/tiny-pytorch","owner":"ImadDabbura","description":"Tiny Deep Learning framework similar to PyTorch.","archived":false,"fork":false,"pushed_at":"2025-07-29T20:11:58.000Z","size":1337,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-30T12:07:20.217Z","etag":null,"topics":["autograd","deep-learning","deep-learning-framework","deep-learning-system","gpu","machine-learning","neural-network","python","pytorch","pytorch-implementation","tensor"],"latest_commit_sha":null,"homepage":"https://imaddabbura.github.io/tiny-pytorch/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ImadDabbura.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2022-12-15T16:41:39.000Z","updated_at":"2025-07-29T18:56:04.000Z","dependencies_parsed_at":"2025-04-08T14:31:22.577Z","dependency_job_id":"75362d0c-675c-40a7-a80c-2696edf46f38","html_url":"https://github.com/ImadDabbura/tiny-pytorch","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/ImadDabbura/tiny-pytorch","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ImadDabbura%2Ftiny-pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ImadDabbura%2Ftiny-pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ImadDabbura%2Ftiny-pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ImadDabbura%2Ftiny-pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ImadDabbura","download_url":"https://codeload.github.com/ImadDabbura/tiny-pytorch/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ImadDabbura%2Ftiny-pytorch/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274112721,"owners_count":25224329,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-07T02:00:09.463Z","response_time":67,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autograd","deep-learning","deep-learning-framework","deep-learning-system","gpu","machine-learning","neural-network","python","pytorch","pytorch-implementation","tensor"],"created_at":"2024-10-02T01:49:30.644Z","updated_at":"2026-04-01T17:55:45.761Z","avatar_url":"https://github.com/ImadDabbura.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Tiny-PyTorch 🧠\n\n## **Unravel the magic of modern deep learning by building a PyTorch-like framework from the ground up.**\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"https://raw.githubusercontent.com/ImadDabbura/tiny-pytorch/main/logo.png\" width=\"300\" height=\"200\"\u003e\n\u003c/p\u003e\n\n[![Python Versions](https://img.shields.io/badge/python-_3.13-blue.svg)](https://www.python.org/downloads/)\n[![PyPI version](https://img.shields.io/pypi/v/tiny-pytorch.svg)](https://pypi.org/project/tiny-pytorch/)\n[![PyPI downloads](https://img.shields.io/pypi/dm/tiny-pytorch.svg)](https://pypi.org/project/tiny-pytorch/)\n[![codecov](https://codecov.io/gh/imaddabbura/tiny-pytorch/branch/main/graph/badge.svg)](https://codecov.io/gh/imaddabbura/tiny-pytorch)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n\nTiny-PyTorch is an educational deep learning framework built entirely in Python. It demystifies the core machinery of libraries like PyTorch by providing a clean, focused, and from-scratch implementation of the essential components.\n\n---\n\n## Philosophy: Understanding by Building\n\nThe best way to truly understand how complex systems work is to build them yourself. Tiny-PyTorch is born from this philosophy. While production frameworks like PyTorch and TensorFlow provide powerful, high-level abstractions, their internal complexity can be a barrier to learning.\n\nThis project strips away those abstractions, allowing you to:\n\n- **See the Core Logic:** Grasp the fundamental algorithms and data structures that power deep learning, from the `Tensor` object to the backpropagation process.\n- **Connect Theory to Code:** Bridge the gap between the mathematical concepts of deep learning and their concrete implementation.\n- **Become a Better Practitioner:** Use high-level frameworks more effectively by understanding their internal mechanics, performance trade-offs, and potential pitfalls.\n\n---\n\n## ✨ Core Features\n\n- **Dynamic Computation Graph:** Tensors track their history, allowing for flexible model architectures.\n- **Reverse-Mode Automatic Differentiation:** An efficient gradient calculation engine (`autograd`) built from scratch.\n- **Extensible `nn.Module` System:** A familiar API for building complex neural network layers and models.\n- **Standard Optimizers:** Implementations of `SGD` and `Adam` to handle parameter updates.\n- **Hardware Acceleration:** A pluggable backend system supporting `NumPy`, custom `CPU` (C++), and `CUDA` (GPU) operations.\n- **Data Loading Utilities:** `Dataset` and `DataLoader` classes for efficient data pipelines.\n\n---\n\n## 🏗️ Project Architecture\n\nThe framework is built in a bottom-up fashion, where each layer of abstraction relies on the one below it. This mirrors the logical structure of major deep learning libraries.\n\n```mermaid\ngraph TD\n    subgraph \"High-Level API\"\n        C[nn.Module] --\u003e D[Optimizers]\n        B[Tensors \u0026 Autograd] --\u003e C\n    end\n    subgraph \"Low-Level Engine\"\n        A[NDArray] --\u003e B\n        subgraph \"Backends\"\n            direction LR\n            np[NumPy]\n            cpu[CPU]\n            gpu[CUDA]\n        end\n        Backend --\u003e A\n    end\n\n    style C fill:#f9f,stroke:#333,stroke-width:2px\n    style B fill:#ccf,stroke:#333,stroke-width:2px\n    style A fill:#cfc,stroke:#333,stroke-width:2px\n```\n\n1.  **Backends (NumPy, CPU, CUDA):** Perform the actual mathematical computations on flat arrays of data.\n2.  **NDArray:** A generic, strided N-dimensional array class that provides a unified interface over different backends.\n3.  **Tensor \u0026 Autograd:** The heart of the framework. A `Tensor` wraps an `NDArray` and builds a dynamic computation graph. The `autograd` engine traverses this graph to perform reverse-mode automatic differentiation.\n4.  **High-Level API (`nn`, `optimizer`):** Provides the familiar modules, layers, and optimization algorithms for building and training neural networks.\n\n---\n\n## 🚀 Quick Start\n\nTo install Tiny-PyTorch, you have two main options:\n\n### Install from PyPI (using pip)\n\nYou can install the latest stable version directly from PyPI using pip:\n\n```bash\npip install tiny-pytorch\n```\n\n### Install from Source (GitHub Repository)\n\nTo get the very latest development version or if you plan to contribute, you can install from the GitHub repository:\n\n1.  **Clone the repository:**\n    ```bash\n    git clone https://github.com/your-username/tiny-pytorch.git\n    ```\n2.  **Navigate to the project directory:**\n    ```bash\n    cd tiny-pytorch\n    ```\n3.  **Install in editable mode:** This allows you to make changes to the source code and have them reflected without reinstalling.\n    ```bash\n    pip install -e .\n    ```\n\nHere's a simple example of defining a model and running a forward/backward pass.\n\n```python\nimport tiny_pytorch as tp\nimport tiny_pytorch.nn as nn\n\n\n# 1. Define a simple model\nclass SimpleNet(nn.Module):\n    def __init__(self, in_features, out_features):\n        self.fc1 = nn.Linear(in_features, 64)\n        self.relu = nn.ReLU()\n        self.fc2 = nn.Linear(64, out_features)\n\n    def forward(self, x):\n        x = self.fc1(x)\n        x = self.relu(x)\n        return self.fc2(x)\n\n\n# 2. Initialize model, optimizer, and loss function\nmodel = SimpleNet(in_features=10, out_features=1)\noptimizer = tp.optim.Adam(model.parameters(), lr=0.001)\nloss_fn = nn.MSELoss()\n\n# 3. Create dummy data\nx_train = tp.randn(32, 10, requires_grad=True)\ny_true = tp.randn(32, 1)\n\n# 4. Perform a single training step\noptimizer.zero_grad()  # Reset gradients\ny_pred = model(x_train)  # Forward pass\nloss = loss_fn(y_pred, y_true)  # Compute loss\nloss.backward()  # Backward pass (autograd)\noptimizer.step()  # Update weights\n\nprint(f\"Loss: {loss.item():.4f}\")\n```\n\n---\n\n## 🗺️ Roadmap\n\nThe project is developed in two main phases. Our current progress is tracked below.\n\n- **Phase I: Core Framework (NumPy Backend)**\n  - [x] `Tensor`: The main multi-dimensional array with autograd support.\n  - [x] `Op`: The base class for all tensor operations.\n  - [x] `Automatic Differentiation`: Reverse-mode autograd engine.\n  - [x] `init`: Parameter initialization functions (`kaiming`, `xavier`, etc.).\n  - [x] `nn`: Core neural network layers (`Linear`, `ReLU`, `BatchNorm`, `Conv2d`).\n  - [x] `optimizer`: `SGD` and `Adam` optimizers.\n  - [x] `data`: `Dataset` and `DataLoader` for data handling.\n- **Phase II: Hardware Acceleration \u0026 Advanced Models**\n  - [x] `NDArray`: Generic, strided N-dimensional array.\n  - [x] NumPy Backend\n  - [x] CPU Backend (C++)\n  - [x] CUDA Backend (GPU)\n  - [x] Advanced CNN operations (e.g., `padding`, `dilation`).\n  - [x] ResNet implementation.\n  - [x] RNN and LSTM layers.\n  - [x] A simple Language Model.\n\n---\n\n## 📚 Documentation\n\nThe official documentation, including detailed API references and tutorials, is hosted at:\n**[https://imaddabbura.github.io/tiny-pytorch/](https://imaddabbura.github.io/tiny-pytorch/)**\n\n---\n\n## ⚠️ Limitations\n\nAs an educational project, Tiny-PyTorch has some intentional simplifications:\n\n- **Explicit Broadcasting:** Broadcasting for element-wise operations must be done manually if tensor shapes do not match.\n- **Single Data Type:** `NDArray` only supports the `float32` `dtype`.\n- **Contiguous Memory:** Operations on the underlying 1D array require a call to `compact()` to ensure the data is in a contiguous memory block.\n- **Limited Reductions:** Reduction operations (e.g., `sum`, `max`) can only be performed on a single axis or all axes at once.\n\n---\n\n## 🙏 Acknowledgements\n\nThis project was inspired by and built upon ideas from the following resources:\n\n- [Deep Learning Systems](https://dlsyscourse.org/) by J. Zico Kolter and Tianqi Chen\n- [micrograd](https://github.com/karpathy/micrograd) by Andrej Karpathy\n- [Machine Learning Systems](https://mlsyscourse.org/) by Tianqi Chen and Zhihao Jia\n\n---\n\n## License\n\nTiny-PyTorch is licensed under the Apache License 2.0. See the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fimaddabbura%2Ftiny-pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fimaddabbura%2Ftiny-pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fimaddabbura%2Ftiny-pytorch/lists"}