https://github.com/vdutor/picograd
Minimal Automatic Differentiation Engine in Julia. Inspired by micrograd.
https://github.com/vdutor/picograd
Last synced: 4 months ago
JSON representation
Minimal Automatic Differentiation Engine in Julia. Inspired by micrograd.
- Host: GitHub
- URL: https://github.com/vdutor/picograd
- Owner: vdutor
- Created: 2021-11-21T23:32:40.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2021-11-21T23:55:41.000Z (over 4 years ago)
- Last Synced: 2025-07-05T06:11:46.502Z (11 months ago)
- Language: Julia
- Size: 2.93 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# PicoGrad
A minimal auto-differentiation (AD) engine written in Julia. The soul purpose of the project is to learn how AD and Julia works. Inspired by Karpathy's [micrograd](https://github.com/karpathy/micrograd).
Example
```julia
include("src/ADEngine.jl")
using .ADEngine: Node, backward, relu
a = Node(-4.0)
b = Node(2.0)
c = a + b
d = a * b + b^3
c += c + 1
c += 1 + c + (-a)
d += d * 2 + relu(b + a)
d += 3 * d + relu(b - a)
e = c - d
f = e^2
g = f / 2.0
g += 10.0 / f
println("$(g.data)") # prints 24.7041, the outcome of this forward pass
@assert g.data ≈ 24.70408163265306
backward(g)
println("$(a.grad)") # prints 138.8338, i.e. the numerical value of dg/da
@assert a.grad ≈ 138.83381924198252
println("$(b.grad)") # prints 645.5773, i.e. the numerical value of dg/db
@assert b.grad ≈ 645.5772594752187
```