{"id":19206900,"url":"https://github.com/stefanasandei/tiny-autograd","last_synced_at":"2025-11-14T10:02:50.351Z","repository":{"id":197616741,"uuid":"698862698","full_name":"stefanasandei/tiny-autograd","owner":"stefanasandei","description":"small autograd engine built from scratch in modern C++","archived":false,"fork":false,"pushed_at":"2023-10-01T15:15:26.000Z","size":45,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-04T15:35:30.490Z","etag":null,"topics":["autograd","cmake","cpp","deep-learning","neural-networks"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stefanasandei.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"License.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-10-01T07:46:19.000Z","updated_at":"2024-07-11T05:08:16.000Z","dependencies_parsed_at":"2023-10-01T19:41:22.689Z","dependency_job_id":null,"html_url":"https://github.com/stefanasandei/tiny-autograd","commit_stats":null,"previous_names":["stefanasandei/tiny-autograd"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stefanasandei%2Ftiny-autograd","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stefanasandei%2Ftiny-autograd/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stefanasandei%2Ftiny-autograd/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stefanasandei%2Ftiny-autograd/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stefanasandei","download_url":"https://codeload.github.com/stefanasandei/tiny-autograd/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240280825,"owners_count":19776414,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autograd","cmake","cpp","deep-learning","neural-networks"],"created_at":"2024-11-09T13:17:22.490Z","updated_at":"2025-11-14T10:02:50.301Z","avatar_url":"https://github.com/stefanasandei.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# tiny-autograd\n\nAn autograd engine calculates the partial derivite of each term with respect to the output one. To achieve this, it creates an in-memory graph of all the operations that lead up to the result. Each operation has an associated \"backward\" pass, which computes the gradient of the function. The main structure that makes this possible is the `Value` class, which wraps a floating point value and overloads the operators, while also defining common math operations, with their respective `backward` function.\n\n![image](./res/viz.png)\n\nThis visualization is done using the `graphviz` tool!\n\nBackpropagation works by first building a topological sorted set of the nodes, and then running the backward pass on each node. This will fill in the gradient for each node. Now, in a deep learning process, this can be applied on a loss function. A `step size` variable can be defined to be multiplied with the respective gradient of each node within a single learning step.\n\n## Example\n\n```cpp\nTinyAutograd::Value a = 2.0f;\nTinyAutograd::Value b = 3.0f;\n\nauto c = a * b;\nc.Backward();\n\nstd::cout \u003c\u003c c \u003c\u003c ' ' \u003c\u003c a.Gradient \u003c\u003c '\\n';\n// 6.0 and 3.0\n\nTinyAutograd::Visualize(c);\n// draws the operations graph in ./res/viz.png\n```\n\n## Resources\n\nResources I used to learn this:\n - Andrej Karpathy's lessons from YouTube: https://github.com/karpathy/micrograd\n - PyTorch's documentation: https://pytorch.org/docs/stable/notes/autograd.html\n - Stanford's C231N free lessons: https://cs231n.github.io/\n\nThis project is for learning purposes only, it's not intended to be used as a library.\n\n## License\n\nLicensed under GPLv3.\nCopyright 2023 Asandei Stefan-Alexandru. All rights reserved.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstefanasandei%2Ftiny-autograd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstefanasandei%2Ftiny-autograd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstefanasandei%2Ftiny-autograd/lists"}