{"id":22055266,"url":"https://github.com/marco-christiani/zigrad","last_synced_at":"2025-03-31T10:02:30.422Z","repository":{"id":220044220,"uuid":"726196720","full_name":"Marco-Christiani/zigrad","owner":"Marco-Christiani","description":"A deep learning framework built on an autograd engine with high level abstractions and low level control.","archived":false,"fork":false,"pushed_at":"2025-03-02T01:55:12.000Z","size":8465,"stargazers_count":93,"open_issues_count":11,"forks_count":3,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-21T15:55:55.923Z","etag":null,"topics":["autograd","deep-learning","machine-learning","neural-network","tensor","zig","zig-package","ziglang"],"latest_commit_sha":null,"homepage":"http://zigrad.datatician.io/","language":"Zig","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"lgpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Marco-Christiani.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":"docs/roadmap.norg","authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-01T18:38:16.000Z","updated_at":"2025-03-20T18:36:32.000Z","dependencies_parsed_at":null,"dependency_job_id":"31983301-8ac7-4959-bb5f-de7ce51351dd","html_url":"https://github.com/Marco-Christiani/zigrad","commit_stats":null,"previous_names":["marco-christiani/zigrad"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Marco-Christiani%2Fzigrad","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Marco-Christiani%2Fzigrad/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Marco-Christiani%2Fzigrad/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Marco-Christiani%2Fzigrad/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Marco-Christiani","download_url":"https://codeload.github.com/Marco-Christiani/zigrad/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245240915,"owners_count":20583101,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autograd","deep-learning","machine-learning","neural-network","tensor","zig","zig-package","ziglang"],"created_at":"2024-11-30T16:06:07.041Z","updated_at":"2025-03-24T09:05:31.017Z","avatar_url":"https://github.com/Marco-Christiani.png","language":"Zig","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./docs/zg-logo.svg\" width=350\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n\t\u003cimg src=\"https://img.shields.io/github/license/Marco-Christiani/zigrad?style=flat\u0026logo=opensourceinitiative\" alt=\"license\"\u003e\n\t\u003cimg src=\"https://img.shields.io/github/last-commit/Marco-Christiani/zigrad?style=flat\u0026logo=git\u0026logoColor=white\" alt=\"last-commit\"\u003e\n\t\u003cimg src=\"https://img.shields.io/github/languages/count/Marco-Christiani/zigrad?style=flat\" alt=\"repo-language-count\"\u003e\n\t\u003cimg src=\"https://img.shields.io/github/languages/top/Marco-Christiani/zigrad?style=flat\u0026color=F7A41D\" alt=\"repo-top-language\"\u003e\n\t\u003c!-- \u003cimg src=\"https://img.shields.io/badge/Zig-F7A41D.svg?style=flat\u0026logo=Zig\u0026logoColor=white\" alt=\"Zig\"\u003e --\u003e\n\t\u003c!-- 1325584101809324113 --\u003e\n\t\u003cimg alt=\"Discord\" src=\"https://img.shields.io/discord/1325584101809324113?style=flat\"\u003e\n\u003c/p\u003e\n\u003cbr\u003e\n\u003cp align=\"center\" class=\"markdown-heading\"\u003e\u003cstrong\u003e\u003ci\u003eSupporting AI innovation from ideation to results.\u003c/i\u003e\u003c/strong\u003e\u003c/p\u003e\n\nAI frameworks optimized for rapid research iteration do not seamlessly transition into the infrastructure required for large-scale training. This fragmented pipeline creates redundant engineering effort and slows iteration cycles. Zigrad provides a path to performance that preserves the natural development workflow researchers prefer; bridging research and engineering. Using Zigrad you can:\n\n  - Experiment using high-level, PyTorch-like abstractions\n  - Gradually opt into fine-grained control and performance optimizations\n  - Access low-level primitives and assert control--without switching frameworks, code translation, or building complex extensions.\n  - Quickly transition research to high performance training\n\n\nhttps://github.com/user-attachments/assets/3842aa72-9b16-4c25-8789-eac7159e3768\n\n**Fast**\n\u003c!-- benchmarks --\u003e\n\n2.5x+ speedup over a compiled PyTorch model on Apple Silicon, 1.5x on x86. Expect similar performance gains across more architectures and platforms as MKL/CUDA support improves and Zigrad's ML graph compiler is operational.\n\u003c!-- link to a benchmarking page --\u003e\n\u003c!-- only need one of the bm plots, probably fast vs fast since that requires the least explanation --\u003e\n\n\u003cpicture\u003e\n  \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"docs/zg_mnist_zg_torch_perf.svg\"\u003e\n  \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"docs/zg_mnist_zg_torch_perf_dark.svg\" \u003e\n  \u003cimg alt=\"Description of the image\" src=\"docs/zg_mnist_zg_torch_perf.svg\"\u003e\n\u003c/picture\u003e\n\u003c!-- ![](./docs/zg_mnist_zg_torch_perf_0_speedupzigrad_pytorch_plotly.svg) --\u003e\n\n\u003csub\u003e*Tensorflow excluded for scaling purposes (too slow).\u003c/sub\u003e\n\n**Flexible**\nZigrad supports research workflows with high level abstractions for rapid prototyping, and integrations like Tensorboard and Mujoco. Zigrad supports the transition of research code to training infrastructure. \n\nZigrad supports research through,\n\n- Easy to use torch-like ergonomics\n- A general purpose automatic differentiation system for n-dimensional data\n- Eager execution and dynamic computation graph by default\n- Computation graph tracing and visualization\n- A design that naturally allows for custom differentiable operations\n\nZigrad supports engineering through,\n\n- An architecture that enables deep control and customization through opt-in complexity,\n- Offering flexible tradeoffs between performance characteristics like latency vs throughput\n- Hardware-aware optimizations tailored to specific use cases and system requirements\n- Fine-grained memory management and allocation control\n- Cross-platform compatibility without compromising performance\n- A streamlined design that avoids abstraction layers or build systems that hinder aggressive optimizations\n\u003c!-- Scalar API --\u003e\n\n## Features\n\n### Trace the Computation Graph\n\n![](./docs/comp_graph_mnist_simple_noag.svg)\n\nAn example of tracing the computation graph generated by a fully connected neural network for MNIST.\n\n- *Input:* Batch of images 28x28 pixel samples.\n- **Flatten:** `28x28 -\u003e 784`\n- **FC1**: Linear layer `784 -\u003e 128`\n- **ReLU**\n- **FC2:** Linear layer `128 -\u003e 64`\n- **ReLU**\n- **FC3:** Linear layer `64 -\u003e 10`\n- *Output:* Value for each of the 10 classes\n\n\nWe did not have to use Zigrad's modules to write this network at all, as Zigrad is backed by a capable autograd engine. Even when using the autograd backend to dynamically construct the same neural network Zigrad can still trace the graph and render it.\n\n  \u003e Note: Since the graph is generated from the autograd information, we set the labels for the nodes by naming the tensors for the sake of the diagram.\n\n![](./docs/comp_graph_mnist_simple_ag.svg)\n\n## Getting Started\n\nOnly dependency is a BLAS library.\n\n### Linux\n\nOn linux (or intel mac) you have some options,\n\n- MKL (recommended for best performance)\n  - See https://www.intel.com/content/www/us/en/developer/tools/oneapi/onemkl-download.html\n  - Reccommend a system installation for simplicity although this can work with `conda` for example, just make sure you adjust the library paths as necessary.\n- OpenBLAS\n  - See https://github.com/OpenMathLib/OpenBLAS/wiki/Precompiled-installation-packages\n  - Likely available through your package manager as `libopenblas-dev` or `openblas-devel`\n\n### Apple Silicon\n\n- Nothing :)\n\n### Examples\n\nThe `examples/` directory has some standalone templates you can take and modify, the zon files are pinned to commit hashes.\n\nHello world example shows how to run a backward pass using the `GraphManager.` Note that in this very simple example, we do not need the `GraphManager` and the script could be simplified but this is designed to get you familiar with the workflow.\n\n```shell\ngit clone https://github.com/Marco-Christiani/zigrad/\ncd zigrad/examples/hello-world\nzig build run\n```\n\nRun the mnist demo\n\n```shell\ncd zigrad/examples/mnist\nmake help\nmake\n```\n\n## Roadmap\n\nA lot is planned and hoping for support from the Zig community so we can accomplish some of the more ambitious goals.\n\n- More comprehensive MKL and CUDA support (in progress)\n- Support for popular formats like ONNX and ggml.\n- Standardized benchmarking procedures (always an ongoing effort)\n- Lazy tensors\n- Static graph optimization\n- Dynamic graph compiler\n- MLIR\n- ZML translation for inference\n- Apache TVM integration. [Github](https://github.com/apache/tvm/) [Homepage](https://tvm.apache.org)\n- More examples like LLMs, physics and robotic control, etc.\n\n## Known Issues and Limitations\n\n- Documentation. As the API stabilizes more documentation will be added. For now, the examples are designed to be quickstart guides.\n- Effort has been directed towards performant primitives, not many layer types have been implemented\n  - e.g. conv, pooling, etc are test implementations for verification, they are slow and unoptimized, I would not use them\n\n## Contributing\n\n- [Join the discord](https://discord.gg/JWSSfWj3Uf) and into the dev channels\n- Any open issue is available for development, just leave a comment mentioning your interest and I can provide support to help get you started if necessary\n- Otherwise, **please open an issue first, before working on a PR**\n- If you are interested in contributing but do not know where to start then open an issue or leave a comment\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarco-christiani%2Fzigrad","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarco-christiani%2Fzigrad","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarco-christiani%2Fzigrad/lists"}