{"id":22603482,"url":"https://github.com/radenmuaz/slope-ad","last_synced_at":"2025-06-26T12:04:38.700Z","repository":{"id":156178116,"uuid":"606078532","full_name":"radenmuaz/slope-ad","owner":"radenmuaz","description":"A small automatic differentiation engine, supporting higher-order derivatives","archived":false,"fork":false,"pushed_at":"2024-06-09T11:03:06.000Z","size":4756,"stargazers_count":6,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-25T01:51:06.147Z","etag":null,"topics":["array","autograd","automatic-differentiation","cuda","gradient","iree","jvp","machine-learning","metal","mlir","onnx","onnxruntime","tensor","vjp"],"latest_commit_sha":null,"homepage":"https://radenmuaz.github.io/slope-ad/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/radenmuaz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-24T14:51:36.000Z","updated_at":"2025-01-07T02:37:38.000Z","dependencies_parsed_at":"2023-10-17T05:48:38.858Z","dependency_job_id":"e856f05b-4056-45ef-a00b-3febc4badccb","html_url":"https://github.com/radenmuaz/slope-ad","commit_stats":null,"previous_names":["radenmuaz/slope"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radenmuaz%2Fslope-ad","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radenmuaz%2Fslope-ad/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radenmuaz%2Fslope-ad/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radenmuaz%2Fslope-ad/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/radenmuaz","download_url":"https://codeload.github.com/radenmuaz/slope-ad/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247974731,"owners_count":21026742,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["array","autograd","automatic-differentiation","cuda","gradient","iree","jvp","machine-learning","metal","mlir","onnx","onnxruntime","tensor","vjp"],"created_at":"2024-12-08T13:06:30.260Z","updated_at":"2025-04-11T03:43:43.466Z","avatar_url":"https://github.com/radenmuaz.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"![logo](./assets/logo.jpeg)\n# SlopeAD\n\nSlope is a small automatic differentation (AD) engine, focused on machine learning (ML), supporting forward, reverse and higher-order AD.\n\nThis project is designed to be a **small**, **hackable** and **educational** AD engine focused on ML, yet able to do things **end-to-end from training to deployment**, instead of just some simple toy examples.\n\nTensor semantics are similar to Pytorch, functional API is similar to [JAX](https://github.com/google/jax), tensor operators code is heavily derived from [tinygrad](https://tinygrad.org/).\n\nExample:\n```python\nimport slope\n\ndef f(x):\n    y = x * 2.0\n    return y.sum()\n\nx = slope.tensor([1.,2.,3.])\ngf_x = slope.grad(f)(x)\nprint(f\"{gf_x=}\")\n```\n```\ngf_x=\u003cTensor: val=\n[2. 2. 2.]\nshape=(3,), dtype=float32, device='cpu:0'\u003e\n```\n\n\n# Install\n\n```\npip install slope-ad\n```\n\nor latest from main branch:\n\n```\ngit clone https://github.com/radenmuaz/slope-ad\ncd slope\npip install -e .\n```\n\nor you can just copy `src/slope` to your projects.\n\n\n# Features\n\n1. Small (?)\n    - \u003c3000 lines of core code [slope/core.py](./src/slope/core.py), after formatted with `black src --line-length 140`\n\n2. Functional API for forward-mode, reverse-mode, and higher-order AD, like in JAX:\n    - `grad vjp jvp jit vmap`\n    - `register_node tree_flatten tree_unflatten`\n\n3. Just-in-time compilation, where code is compiled to these supported backends running on either CPU, CUDA and Metal:\n    - [ONNX Runtime](https://onnxruntime.ai/) (ONNX graph)\n    - [OpenXLA IREE](https://iree.dev/) (StableHLO MLIR)\n    - NumPy (Python code)\n\n4. Training and inference, examples:\n    - [MLP on MNIST](examples/nn/mnist_mlp.py)\n    - [ResNet on CIFAR-10](examples/nn/cifar_resnet.py)\n    - [Export jitted function](examples/simple/export.py)\n\n5. Operators and procedures system\n    - 33 core operators defined in [slope/operators.py](./src/slope/operators.py)\n        - Unary: `exp log sin sqrt invert cast stop_gradient`\n        - Binary: `add mul sub div pow equal less greater maximum`\n        - Reduce: `sum max`\n        - Shape: `reshape expand permute slice pad flip cat`\n        - Init: `full arange random_normal random_uniform`\n        - GeneralReduce: `matmul conv gather_nd scatter_nd`\n    - Composite operators system with \"procedures\" [slope/procedures.py](./src/slope/procedures.py)\n        - For defining Tensor functions composed with core operators, e.g.\n          - `x.cos()`, where `def cos(x): return (math.pi/2 - x).sin()`\n          - `x.conv_transpose(w)`: where `def conv_transpose(x, w, ... ): ...` is a very long function.\n        - Procedures are exposed with `Tensor.procedure_name(*args)` syntax.\n        \n\n6. Extensible\n    - Add new backend by defining implementation translations [slope/backends](./src/slope/backends)\n    - Define new modules with NN module [slope/nn.py](./src/slope/nn.py)\n\n\n\n# Docs\n\nDocs are available online at [radenmuaz.github.io/slope-ad](https://radenmuaz.github.io/slope-ad)\nAPI reference: [radenmuaz.github.io/slope-ad/api](https://radenmuaz.github.io/slope-ad/api)\n\n## Tutorials\n\n[Quickstart](./docs/tutorials/quickstart.md): How Tensors work, how to write and jit compile functions and train something.\n\n[NN Training](./docs/tutorials/nn_training.md): Train MLP on MNIST with slope.nn module\n\n[Internals Walkthrough](./docs/tutorials/internals_walkthrough.md): Understand the core of SlopeAD (hint: like JAX). Useful if you want to start contributing to SlopeAD\n\n[Extending SlopeAD](./docs/tutorials/internals_walkthrough.md): Add new backend, operators, procedures. Modify the core functions.\n\n## API reference\n\n# Contributing\n\nOpen a PR, things on the roadmap below need to be done.\n\n# Roadmap\n\n- Docs\n- Symbolic shape inference \n- Dynamic shape jit\n- Optimizer filter frozen params\n- vmap vjp and jvp to compute jacobian and hessian\n- iree backend currently has fixed seed random, implement threefry and JAX-like random\n- make things fast\n- llama (gpt) training\n- whisper inference\n- core tests, operators tests on all Trace types","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fradenmuaz%2Fslope-ad","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fradenmuaz%2Fslope-ad","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fradenmuaz%2Fslope-ad/lists"}