{"id":13763372,"url":"https://github.com/automata/mojograd","last_synced_at":"2025-04-25T00:32:39.062Z","repository":{"id":177704238,"uuid":"646918520","full_name":"automata/mojograd","owner":"automata","description":"Implementation of Karpathy's micrograd in Mojo :fire:","archived":false,"fork":false,"pushed_at":"2023-11-19T23:41:54.000Z","size":422,"stargazers_count":61,"open_issues_count":1,"forks_count":1,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-04-14T20:06:33.536Z","etag":null,"topics":["ai","autograd","micrograd","mojo","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/automata.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-05-29T16:32:25.000Z","updated_at":"2024-04-03T14:29:45.000Z","dependencies_parsed_at":"2024-01-13T03:15:14.996Z","dependency_job_id":null,"html_url":"https://github.com/automata/mojograd","commit_stats":null,"previous_names":["automata/mojograd"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/automata%2Fmojograd","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/automata%2Fmojograd/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/automata%2Fmojograd/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/automata%2Fmojograd/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/automata","download_url":"https://codeload.github.com/automata/mojograd/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250733515,"owners_count":21478380,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","autograd","micrograd","mojo","python"],"created_at":"2024-08-03T15:00:43.451Z","updated_at":"2025-04-25T00:32:38.726Z","avatar_url":"https://github.com/automata.png","language":"Python","funding_links":[],"categories":["🗂️ Libraries\u003ca id='libraries'\u003e\u003c/a\u003e"],"sub_categories":["AI"],"readme":"# 🔥grad\n\n\u003cbr /\u003e\n\u003cimg src=\"https://github.com/automata/mojograd/assets/49062/ca242b1e-e7d7-485d-8bb7-7e27ff0d69a8\" width=\"512px\" /\u003e\n\u003cbr /\u003e\n\n`mojograd` is a Mojo implementation of [micrograd](https://github.com/karpathy/micrograd),\na reverse-mode autodiff library with a PyTorch-like API.\n\nThe goal is to be as close as possible to micrograd, keeping a\npretty clean syntax to define computational graphs.\nLike micrograd, it only supports scalar values for now, but we\nplan to extend it to support Tensors in the near future.\n\nNote that `mojograd` is in WIP and relies on static register passable structures, so\nbackward pass copies values and can be really slow (Mojo traits support should\nimprove that, so please stay tuned!). However, even now with zero\noptimizations, forward pass is already ~40x faster than the original Python implementation\n(see benchmarks bellow).\n\n# Using\n\n`mojograd` dynamically builds a computational graph by overloading operators \non `Value` type, performing the forward pass. Just write your expression like\na normal (non-diff) equation and call `backward()` to perform the\nbackward pass:\n\n```python\nfrom mojograd import Value\n\nvar a = Value(2.0)\nvar b = Value(3.0)\nvar c: Float32 = 2.0\nvar d = b**c\nvar e = a + c\ne.backward()\n\na.print() # =\u003e \u003cValue data: 2.0 grad: 1.0 op:  \u003e\nb.print() # =\u003e \u003cValue data: 3.0 grad: 0.0 op:  \u003e\nd.print() # =\u003e \u003cValue data: 9.0 grad: 0.0 op: ** \u003e\ne.print() # =\u003e \u003cValue data: 4.0 grad: 1.0 op: + \u003e \n```\n\nFor a more complete example (a simple Multi-Layer Perceptron), please check\nthe `tests.mojo` file. You can run it with:\n\n```bash\nmojo tests.mojo\n```\n\n# Benchmarks\n\n## MLP binary classifier\n\nWhen compared to original Python implementation, `mojograd` is up to **~40 times faster\nin forward pass**.\n\n| # parameters | micrograd (Python) (sec) | mojograd (Mojo) (sec) | speed up |\n|--------------|--------------------------|-----------------------|----------|\n| 367          | 0.001                    | 0.00006               | x20      |\n| 1185         | 0.004                    | 0.0001                | x40      |\n| 4417         | 0.01                     | 0.0005                | x20      |\n| 17025        | 0.06                     | 0.002                 | x30      |\n\n# Changelog\n\n- 2023.11.19\n  - Benchmarking inference and comparing with micrograd\n- 2023.11.18\n  - Optimization pass through the code\n- 2023.11.14\n  - Rebuild the whole thing using pointer handling (dangerous) to register-passables\n  - Got the full micrograd implementation working!\n  - MLP example training and inference working!\n- 2023.09.05\n  - Starting from scratch based on suggestions from Jack Clayton\n  - Topological sort works but I'm messing something with memory handling,\n    the gradients are not getting updated\n- 2023.07.04\n  - Ported Neuron, Layer and MLP\n  - Back to use yakupc55's List (need `register_passable` data struct)\n- 2023.06.30\n  - Finally got it working! Only missing pow ops and review it","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fautomata%2Fmojograd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fautomata%2Fmojograd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fautomata%2Fmojograd/lists"}