{"id":21864158,"url":"https://github.com/chanlumerico/lucid","last_synced_at":"2025-04-14T20:53:29.994Z","repository":{"id":261253499,"uuid":"883719291","full_name":"ChanLumerico/lucid","owner":"ChanLumerico","description":"Lumerico's Comprehensive Interface for Deep Learning","archived":false,"fork":false,"pushed_at":"2025-04-13T07:59:51.000Z","size":46003,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-13T08:18:19.767Z","etag":null,"topics":["autograd","deep-learning","from-scratch","numpy"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ChanLumerico.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-05T13:07:14.000Z","updated_at":"2025-04-13T07:59:54.000Z","dependencies_parsed_at":"2024-12-18T11:39:22.097Z","dependency_job_id":"ce02dd53-0444-42ce-a260-847b7379dabd","html_url":"https://github.com/ChanLumerico/lucid","commit_stats":{"total_commits":143,"total_committers":2,"mean_commits":71.5,"dds":"0.18181818181818177","last_synced_commit":"da9f484bdd34cdaaf12e96be5c0fdb9e64990fc3"},"previous_names":["chanlumerico/lucid"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChanLumerico%2Flucid","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChanLumerico%2Flucid/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChanLumerico%2Flucid/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChanLumerico%2Flucid/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ChanLumerico","download_url":"https://codeload.github.com/ChanLumerico/lucid/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248961048,"owners_count":21189990,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autograd","deep-learning","from-scratch","numpy"],"created_at":"2024-11-28T04:07:41.898Z","updated_at":"2025-04-14T20:53:29.985Z","avatar_url":"https://github.com/ChanLumerico.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Lucid 💎²\n\n![PyPI - Version](https://img.shields.io/pypi/v/lucid-dl?color=red)\n![PyPI - Downloads](https://img.shields.io/pypi/dm/lucid-dl)\n![PyPI - Total Downloads](https://img.shields.io/badge/total%20downloads-22.1k-yellow)\n![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/ChanLumerico/lucid)\n![Code Style](https://img.shields.io/badge/code%20style-black-000000.svg)\n![Lines of Code](https://img.shields.io/endpoint?url=https%3A%2F%2Floc-counter.onrender.com%2F%3Frepo%3DChanLumerico%2Flucid%26branch%3Dmain%26ignored%3Ddocs%26stat%3DlinesOfCode\u0026label=Lines%20of%20Code\u0026color=purple)\n\n**Lucid** is a minimalist deep learning framework built entirely from scratch in Python. It offers a pedagogically rich environment to explore the foundations of modern deep learning systems, including autodiff, neural network modules, and GPU acceleration — all while staying lightweight, readable, and free of complex dependencies.\n\nWhether you're a student, educator, or an advanced researcher seeking to demystify deep learning internals, Lucid provides a transparent and highly introspectable API that faithfully replicates key behaviors of major frameworks like PyTorch, yet in a form simple enough to study line by line.\n\n[📑 Lucid Documentation](https://chanlumerico.github.io/lucid/build/html/index.html)\n\n### 🔥 What's New\n\n- Pyramid Vision Transformer v2 (PVT-v2) added: `lucid.models.PVT_V2`\n\n## 🔧 How to Install\n\nLucid is designed to be light, portable, and friendly to all users — no matter your setup.\n\n### ▶️ Basic Installation\nLucid is available directly on PyPI:\n```bash\npip install lucid-dl\n```\n\nAlternatively, you can install the latest development version from GitHub:\n```bash\npip install git+https://github.com/ChanLumerico/lucid.git\n```\nThis installs all the core components needed to use Lucid in CPU mode using NumPy.\n\n### ⚡ Enable GPU (Metal / MLX Acceleration)\nIf you're using a Mac with Apple Silicon (M1, M2, M3), Lucid supports GPU execution via the MLX library.\n\nTo enable Metal acceleration:\n1. Install MLX:\n```bash\npip install mlx\n```\n2. Confirm you have a compatible device (Apple Silicon).\n3. Run any computation with `device=\"gpu\"`.\n\n### ✅ Verification\nHere's how to check whether GPU acceleration is functioning:\n```python\nimport lucid\nx = lucid.ones((1024, 1024), device=\"gpu\")\nprint(x.device)  # Should print: 'gpu'\n```\n\n\n## 📐 Tensor: The Core Abstraction\n\nAt the heart of Lucid is the `Tensor` class — a generalization of NumPy arrays that supports advanced operations such as gradient tracking, device placement, and computation graph construction.\n\nEach Tensor encapsulates:\n- A data array (`ndarray` or `mlx.array`)\n- Gradient (`grad`) buffer\n- The operation that produced it\n- A list of parent tensors from which it was derived\n- Whether it participates in the computation graph (`requires_grad`)\n\n### 🔁 Construction and Configuration\n```python\nfrom lucid import Tensor\n\nx = Tensor([[1.0, 2.0], [3.0, 4.0]], requires_grad=True, device=\"gpu\")\n```\n\n- `requires_grad=True` adds this tensor to the autodiff graph.\n- `device=\"gpu\"` allocates the tensor on the Metal backend.\n\n### 🔌 Switching Between Devices\nTensors can be moved between CPU and GPU at any time using `.to()`:\n```python\nx = x.to(\"gpu\")  # Now uses MLX arrays for accelerated computation\ny = x.to(\"cpu\")  # Moves data back to NumPy\n```\n\nYou can inspect which device a tensor resides on via:\n```python\nprint(x.device)  # Either 'cpu' or 'gpu'\n```\n\n\n## 📉 Automatic Differentiation (Autodiff)\n\nLucid implements **reverse-mode automatic differentiation**, which is commonly used in deep learning due to its efficiency for computing gradients of scalar-valued loss functions.\n\nIt builds a dynamic graph during the forward pass, capturing every operation involving Tensors that require gradients. Each node stores a custom backward function which, when called, computes local gradients and propagates them upstream using the chain rule.\n\n### 📘 Computation Graph Internals\nThe computation graph is a Directed Acyclic Graph (DAG) in which:\n- Each `Tensor` acts as a node.\n- Each operation creates edges between inputs and outputs.\n- A `_backward_op` method is associated with each Tensor that defines how to compute gradients w.r.t. parents.\n\nThe `.backward()` method:\n1. Topologically sorts the graph.\n2. Initializes the output gradient (usually with 1.0).\n3. Executes all backward operations in reverse order.\n\n### 🧠 Example\n```python\nimport lucid\n\nx = lucid.tensor([1.0, 2.0, 3.0], requires_grad=True)\ny = x * 2 + 1\nz = y.sum()\nz.backward()\nprint(x.grad)  # Output: [2.0, 2.0, 2.0]\n```\n\nThis chain-rule application computes the gradient $\\frac{\\partial z}{\\partial x} = \\frac{\\partial z}{\\partial y}\\cdot\\frac{\\partial y}{\\partial x} = [2, 2, 2]$.\n\n### 🔄 Hooks \u0026 Shape Alignment\nLucid supports:\n- **Hooks** for gradient inspection or modification.\n- **Shape broadcasting and matching** for non-conforming tensor shapes.\n\n\n## 🚀 Metal Acceleration (MLX Backend)\n\nLucid supports **Metal acceleration** on Apple Silicon devices using [MLX](https://github.com/ml-explore/mlx). This integration allows tensor operations, neural network layers, and gradient computations to run efficiently on the GPU, leveraging Apple’s unified memory and neural engine.\n\n### 📋 Key Features\n- Tensors with `device=\"gpu\"` are allocated as `mlx.core.array`.\n- Core mathematical operations, matrix multiplications, and backward passes use MLX APIs.\n- No change in API: switching to GPU is as simple as `.to(\"gpu\")` or passing `device=\"gpu\"` to tensor constructors.\n\n### 💡 Example 1: Basic Acceleration\n```python\nimport lucid\n\nx = lucid.randn(1024, 1024, device=\"gpu\", requires_grad=True)\ny = x @ x.T\nz = y.sum()\nz.backward()\nprint(x.grad.device)  # 'gpu'\n```\n\n### 💡 Example 2: GPU-Based Model\n```python\nimport lucid.nn as nn\nimport lucid.nn.functional as F\n\nclass TinyNet(nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.fc = nn.Linear(100, 10)\n\n    def forward(self, x):\n        return F.relu(self.fc(x))\n\nmodel = TinyNet().to(\"gpu\")\ndata = lucid.randn(32, 100, device=\"gpu\", requires_grad=True)\noutput = model(data)\nloss = output.sum()\nloss.backward()\n```\n\nWhen training models on GPU using MLX, **you must explicitly evaluate the loss tensor** after each forward pass to prevent the MLX computation graph from growing uncontrollably.\n\nMLX defers evaluation until needed. If you don’t force evaluation (e.g. calling `.eval()`), the internal graph may become too deep and lead to performance degradation or memory errors.\n\n### Recommended GPU Training Pattern:\n```python\nloss = model(input).sum()\nloss.eval()  # force evaluation on GPU\nloss.backward()\n```\nThis ensures that all prior GPU computations are flushed and evaluated **before** backward pass begins.\n\n\n## 🧱 Neural Networks with `lucid.nn`\n\nLucid provides a modular PyTorch-style interface to build neural networks via `nn.Module`. Users define model classes by subclassing `nn.Module` and defining parameters and layers as attributes.\n\nEach module automatically registers its parameters, supports device migration (`.to()`), and integrates with Lucid’s autodiff system.\n\n### 🧰 Custom Module Definition\n```python\nimport lucid.nn as nn\n\nclass MLP(nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.fc1 = nn.Linear(784, 128)\n        self.fc2 = nn.Linear(128, 10)\n\n    def forward(self, x):\n        x = self.fc1(x)\n        x = nn.functional.relu(x)\n        x = self.fc2(x)\n        return x\n```\n\n### 🧩 Parameter Registration\nAll parameters are registered automatically and can be accessed:\n```python\nmodel = MLP()\nprint(model.parameters())\n```\n\n### 🧭 Moving to GPU\n```python\nmodel = model.to(\"gpu\")\n```\nThis ensures all internal parameters are transferred to GPU memory.\n\n\n## 🏋️‍♂️ Training \u0026 Evaluation\n\nLucid supports training neural networks using standard loops, customized optimizers, and tracking gradients over batches of data.\n\n### ✅ Full Training Loop\n```python\nimport lucid\nfrom lucid.nn.functional import mse_loss\n\nmodel = MLP().to(\"gpu\")\noptimizer = lucid.optim.SGD(model.parameters(), lr=0.01)\n\nfor epoch in range(100):\n    preds = model(x_train)\n    loss = mse_loss(preds, y_train)\n    loss.eval()  # force evaluation\n\n    optimizer.zero_grad()\n    loss.backward()\n    optimizer.step()\n\n    print(f\"Epoch {epoch}, Loss: {loss.item()}\")\n```\n\n### 🧪 Evaluation without Gradients\n```python\nwith lucid.no_grad():\n    out = model(x_test)\n```\nPrevents gradient tracking and reduces memory usage.\n\n## 🧬 Educational by Design\n\nLucid is not a black box. It’s built to be explored. Every class, every function, and every line is designed to be readable and hackable.\n\n- Use it to build intuition for backpropagation.\n- Modify internal operations to test custom autograd.\n- Benchmark CPU vs GPU behavior on your own model.\n- Debug layer by layer, shape by shape, gradient by gradient.\n\nWhether you're building neural nets from scratch, inspecting gradient flow, or designing a new architecture — Lucid is your transparent playground.\n\n## 🧠 Conclusion\nLucid serves as a powerful educational resource and a minimalist experimental sandbox. By exposing the internals of tensors, gradients, and models — and integrating GPU acceleration — it invites users to **see, touch, and understand** how deep learning truly works.\n\n## 📜 Others\n\n**Dependencies**: `NumPy`, `MLX`, `openml`, `pandas`\n\n**Inspired By**:\n\n![](https://skillicons.dev/icons?i=pytorch)\n![](https://skillicons.dev/icons?i=tensorflow)\n![](https://skillicons.dev/icons?i=stackoverflow)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchanlumerico%2Flucid","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchanlumerico%2Flucid","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchanlumerico%2Flucid/lists"}