https://github.com/markoelez/minigrad
Autograd engine and neural net library
https://github.com/markoelez/minigrad
Last synced: 6 months ago
JSON representation
Autograd engine and neural net library
- Host: GitHub
- URL: https://github.com/markoelez/minigrad
- Owner: markoelez
- Created: 2021-07-04T21:18:36.000Z (almost 5 years ago)
- Default Branch: master
- Last Pushed: 2025-07-01T22:19:00.000Z (12 months ago)
- Last Synced: 2025-07-01T23:27:46.833Z (12 months ago)
- Language: Python
- Homepage:
- Size: 77.1 KB
- Stars: 2
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# minigrad
Simple vector-valued autodiff library.
--------------------------------------------------------------------
Inspired by [karpathy/micrograd](https://github.com/karpathy/micrograd) and [pytorch](https://github.com/pytorch/pytorch)
## Features
### Nueral Networks
See `eval/` for concrete implementations.
Example:
```python
class NN:
def __init__(self):
self.l1: Tensor = Tensor.uniform(4, 10)
self.l2: Tensor = Tensor.uniform(10, 3)
self.params = [self.l1, self.l2]
def forward(self, x):
x = x.dot(self.l1)
x = x.relu()
x = x.dot(self.l2)
x = x.softmax()
return x
def __call__(self, x):
return self.forward(x)
# input data
X_train, Y_train, X_test, Y_test = prepare(dataset)
model = NN()
optim = Adam(params=model.params, lr=0.001)
epochs = 1000
batch_size = 128
for _ in (t := trange(epochs)):
# reset gradients
optim.zero_grad()
# select batch
idx = np.random.choice(len(X_train), batch_size, replace=False)
# initialize tensors
x, y = Tensor(X_train[idx]), Tensor(Y_train[idx])
# forward pass
out = model(x)
# compute loss
loss = out.cross_entropy(y)
# backward pass
loss.backward()
# adjust weights
optim.step()
# eval
cat = np.argmax(out.numpy(), axis=-1)
accuracy = (cat == np.argmax(y.numpy(), axis=-1)).mean()
print(loss, accuracy)
```
Run with `DEBUG=1` to visualize the resulting computational graph, tensor operations, and gradient shapes:
Example:
```
********************************************************************************
[OP=]
---------------
input grad:
---------------
()
---------------
output grads:
---------------
(50, 3)
()
********************************************************************************
[OP=]
---------------
input grad:
---------------
(50, 3)
---------------
output grads:
---------------
(50, 3)
(50, 3)
********************************************************************************
[OP=]
---------------
input grad:
---------------
(50, 3)
---------------
output grads:
---------------
(50, 1)
********************************************************************************
[OP=]
---------------
input grad:
---------------
(50, 1)
---------------
output grads:
---------------
(50, 3)
********************************************************************************
[OP=]
---------------
input grad:
---------------
(50, 3)
---------------
output grads:
---------------
(50, 3)
********************************************************************************
```
Run with `DEBUG=2` to visualize the above in addition to computed gradients.