{"id":15068089,"url":"https://github.com/andrewharabor/autograd","last_synced_at":"2026-01-21T08:11:16.950Z","repository":{"id":252446332,"uuid":"838109220","full_name":"andrewharabor/autograd","owner":"andrewharabor","description":"A C++ library for gradient computation via reverse-mode automatic differentiation","archived":false,"fork":false,"pushed_at":"2024-08-10T18:12:43.000Z","size":34,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-12T23:28:43.400Z","etag":null,"topics":["autograd","automatic-differentiation","cplusplus","cpp","cpp-templates","gradient"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/andrewharabor.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-05T00:42:08.000Z","updated_at":"2024-08-18T00:26:35.000Z","dependencies_parsed_at":"2024-11-22T05:15:26.885Z","dependency_job_id":null,"html_url":"https://github.com/andrewharabor/autograd","commit_stats":{"total_commits":18,"total_committers":2,"mean_commits":9.0,"dds":0.05555555555555558,"last_synced_commit":"e15ed5f2d5139f847b37b10c478995f4936ebc27"},"previous_names":["andrewharabor/autograd"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/andrewharabor/autograd","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andrewharabor%2Fautograd","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andrewharabor%2Fautograd/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andrewharabor%2Fautograd/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andrewharabor%2Fautograd/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/andrewharabor","download_url":"https://codeload.github.com/andrewharabor/autograd/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andrewharabor%2Fautograd/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28629922,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-21T04:47:28.174Z","status":"ssl_error","status_checked_at":"2026-01-21T04:47:22.943Z","response_time":86,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autograd","automatic-differentiation","cplusplus","cpp","cpp-templates","gradient"],"created_at":"2024-09-25T01:30:24.313Z","updated_at":"2026-01-21T08:11:16.928Z","avatar_url":"https://github.com/andrewharabor.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AutoGrad\n\nA C++ library for gradient computation via [reverse-mode automatic differentiation](https://en.wikipedia.org/wiki/Automatic_differentiation#Reverse_accumulation).\n\n## Description\n\nAutoGrad implements reverse-mode automatic differentiation (AD) using a tape-based representation. Specifically, the tape records mathematical operations (arithmetic and elementary functions) performed on variables bound to it and then accumulates the derivatives in reverse order to compute the gradient.\n\n[Forward-mode AD](https://en.wikipedia.org/wiki/Automatic_differentiation#Forward_accumulation) differs in that the derivatives are accumulated directly alongside the operations performed on the variables. Reverse-mode AD is a generalization of [backpropagation](https://en.wikipedia.org/wiki/Backpropagation) and is preferred in the context of [gradient-based optimization problems](https://en.wikipedia.org/wiki/Gradient_descent) in machine learning (over forward-mode AD, [numerical differentiation](https://en.wikipedia.org/wiki/Numerical_differentiation), and [symbolic differentiation](https://en.wikipedia.org/wiki/Computer_algebra)) due to its efficiency, flexibility, and numerical stability.\n\nMathematically, given a function $`f \\space : \\space \\mathbb{R}^{n} \\mapsto \\mathbb{R}^{m}`$, reverse-mode AD computes the partial derivatives of a scalar output variable with respect to all input variables with a single pass. This means that it takes on the order of $`O(m)`$ time in order to compute the full [Jacobian](https://en.wikipedia.org/wiki/Jacobian_matrix_and_determinant). For optimizing neural networks, there are usually a large number of input parameters and a scalar output that denotes the loss ($`f \\space : \\space \\mathbb{R}^{n} \\mapsto \\mathbb{R}`$). Thus, reverse-mode AD can efficiently compute the entire gradient with a single backward pass.\n\n## Usage\n\nIf you would like to use AutoGrad, follow these steps:\n\n1. Clone this GitHub repository with one of the following commands:\n\n``` bash\ngit clone https://github.com/andrewharabor/autograd.git\n```\n\n``` bash\ngit clone git@github.com:andrewharabor/autograd.git\n```\n\n2. Include the header file for AutoGrad in the C++ source file where you would like to use it:\n\n``` cpp\n#include \u003cautograd.hpp\u003e\n```\n\n3. Make sure to tell the compiler where to look for the header file. For example, with `g++`/`gcc`, pass the following flag:\n\n``` bash\n-I\u003cPATH TO CLONED AUTOGRAD REPOSITORY\u003e/src\n```\n\nNote that since the entirety of the AutoGrad library is templated, it is completely contained in `.hpp` files and so there is nothing to compile or link with. See [Limitations](#limitations) for more details.\n\n## Example\n\nThe file [example.cpp](/example/example.cpp) (copied below for convenience) shows a simple use case of the AutoGrad library.\n\n``` cpp\n#include \u003ciomanip\u003e\n#include \u003ciostream\u003e\n\n#include \"autograd.hpp\"\n\nint main() {\n  AutoGrad::Tape\u003cdouble\u003e tape;\n  AutoGrad::Variable\u003cdouble\u003e x = tape.variable(0.5);\n  AutoGrad::Variable\u003cdouble\u003e y = tape.variable(4.2);\n  AutoGrad::Variable\u003cdouble\u003e z = x * y + AutoGrad::sin(x);\n  AutoGrad::Gradient\u003cdouble\u003e grad = z.gradient();\n  std::cout \u003c\u003c std::setprecision(10);\n  std::cout \u003c\u003c \"z = \" \u003c\u003c z.value() \u003c\u003c std::endl; // z = 2.579425539\n  std::cout \u003c\u003c \"∂z/∂x = \" \u003c\u003c grad.withRespectTo(x) \u003c\u003c std::endl; // ∂z/∂x = 5.077582562\n  std::cout \u003c\u003c \"∂z/∂y = \" \u003c\u003c grad.withRespectTo(y) \u003c\u003c std::endl; // ∂z/∂y = 0.5\n}\n```\n\nHere, a `Tape` object is initialized and `Variable`s $`x = 0.5`$ and $`y = 4.2`$ are bound to it. The expression $`z = xy + sin(x)`$ and the partial derivatives $`\\frac{\\partial z}{\\partial x}`$ and $`\\frac{\\partial z}{\\partial y}`$ are computed. Notice how the gradient is computed once and stored in a `Gradient` object. This allows the partial derivative of the output variable with respect to any input variable to be retrieved in constant time.\n\nTo run the example yourself, clone the repository as described in [Usage](#usage) and `cd` into the `autograd` directory. Then run the following commands:\n\n``` bash\nmake\n./build/main\n```\n\nTo clean up the output directory `build`, run:\n\n``` bash\nmake clean\n```\n\n## Limitations\n\nWhile AutoGrad is a complete library, there are some areas in which it could use some improvements:\n\n- **There is only support for reverse-mode AD and first-order derivatives.**\n- **No direct support for linear algebra operations or Jacobians.** This means that the user would have to create their own `Matrix`/`Tensor` class that correctly interfaces with the AutoGrad library and that implements a way to directly compute Jacobians.\n- **None of the mathematical functions implemented by AutoGrad do any domain checking.** This leads to cases where evaluating a function is undefined but the derivative seems reasonable even though it should be invalid. For example, computing $`log(-2)`$ results in `-nan` but AutoGrad reports the gradient as $`-0.5`$ (since the derivative of $`log(x)`$ is $`\\frac{1}{x}`$) when really it should also be undefined. It is deemed the responsibility of the user to ensure this doesn't happen and handle it accordingly.\n- **The entirety of AutoGrad is contained solely in `.hpp` header files.** Because the C++ compiler needs access to an entire template definition in order to instantiate it at compile-time, templates cannot be declared and defined separately (see [this](https://stackoverflow.com/questions/495021/why-can-templates-only-be-implemented-in-the-header-file)). Of course, there are workarounds (see [this](https://stackoverflow.com/questions/44774036/why-use-a-tpp-file-when-implementing-templated-functions-and-classes-defined-i)) but since AutoGrad significantly relies on `friend` classes and functions, it would lead to even more boilerplate code and bloat than already exists. Furthermore, this means that there is some compile-time overhead from including entire class definitions and that users implicitly gain access to headers like `\u003ccmath\u003e` that AutoGrad includes for internal use. On the upside, we don't have to go through the trouble of dealing with the C/C++ linker!\n\n## References\n\n- [Reverse-mode automatic differentiation: a tutorial](https://rufflewind.com/2016-12-30/reverse-mode-automatic-differentiation) (article) and [revad](https://github.com/Rufflewind/revad) (GitHub repository)\n\n- [What's Automatic Differentiation?](https://huggingface.co/blog/andmholm/what-is-automatic-differentiation) (article)\n\n- [What is Automatic Differentiation?](https://www.youtube.com/watch?v=wG_nF1awSSY) (YouTube video)\n\n- [Automatic Differentiation: Differentiate (almost) any function](https://www.youtube.com/watch?v=4wgXBr7fnQg) (YouTube video)\n\n- [Automatic Differentiation Explained with Example](https://www.youtube.com/watch?v=jS-0aAamC64) (YouTube video)\n\n- [simple-autodiff](https://github.com/gtoubassi/simple-autodiff) (GitHub repository)\n\n- [Automatic Differentiation](https://en.wikipedia.org/wiki/Automatic_differentiation) (Wikipedia article)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fandrewharabor%2Fautograd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fandrewharabor%2Fautograd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fandrewharabor%2Fautograd/lists"}