{"id":16705034,"url":"https://github.com/mcabbott/tensorgrad.jl","last_synced_at":"2026-01-02T10:05:03.021Z","repository":{"id":117871521,"uuid":"201618185","full_name":"mcabbott/TensorGrad.jl","owner":"mcabbott","description":null,"archived":false,"fork":false,"pushed_at":"2021-03-20T00:05:45.000Z","size":14,"stargazers_count":4,"open_issues_count":1,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-10-13T19:28:30.620Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Julia","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mcabbott.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-08-10T11:01:26.000Z","updated_at":"2024-02-12T02:19:00.000Z","dependencies_parsed_at":"2023-04-07T13:24:35.371Z","dependency_job_id":null,"html_url":"https://github.com/mcabbott/TensorGrad.jl","commit_stats":{"total_commits":10,"total_committers":2,"mean_commits":5.0,"dds":"0.30000000000000004","last_synced_commit":"81ce3bba8bf8b22a19959d6174f435f79700c16a"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcabbott%2FTensorGrad.jl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcabbott%2FTensorGrad.jl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcabbott%2FTensorGrad.jl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcabbott%2FTensorGrad.jl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mcabbott","download_url":"https://codeload.github.com/mcabbott/TensorGrad.jl/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243561199,"owners_count":20311052,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-12T19:28:26.019Z","updated_at":"2026-01-02T10:05:02.955Z","avatar_url":"https://github.com/mcabbott.png","language":"Julia","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TensorGrad.jl\n\n[![Build Status](https://travis-ci.org/mcabbott/TensorGrad.jl.svg?branch=master)](https://travis-ci.org/mcabbott/TensorGrad.jl)\n\nThis package adds gradient definitions for [Zygote.jl](https://github.com/FluxML/Zygote.jl) \nto most calculations using [TensorOperations.jl](https://github.com/Jutho/TensorOperations.jl),\nand some using [Einsum.jl](https://github.com/ahwillia/Einsum.jl).\nIt exports a macro `@grad` which rewrites an expression like\n```julia\n@grad @tensor A[i,k] := B[i,j] * C[j,k] * D[l,l]\n```\ninto something equivalent to this:\n```julia\nfun(b,c,d) = @tensor a[i,k] := b[i,j] * c[j,k] * d[l,l]  # define a function\n\n@adjoint function fun(b,c,d)\n    fwd = @tensor a[i,k] := b[i,j] * c[j,k] * d[l,l]     # forward pass\n    function back(Δa)\n        @tensor Δb[i,j] := Δa[i,k] * c[j,k] * d[l,l]     # reverse pass\n        @tensor Δc[j,k] := b[i,j] * Δa[i,k] * d[l,l]\n        δ = Diagonal(ones(size(d,1)))\n        @tensor Δd[l,l′] := b[i,j] * c[j,k] * Δa[i,k] * δ[l,l′]\n        return (Δb, Δc, Δd)\n    end\n    return (fwd, back)\nend\n\nA = fun(B,C,D)                                           # apply this to B, C, D\n```\nYou may also write `@grad B C @tensor A[i,k] := B[i,j] * C[j,k] * D[l,l]` to specify that\nonly sensitivities for `B` and `C` are needed, this will remove the calculation \nof `Δd` above. \n\nTo see what is being defined, call `TensorGrad.verbose(true)` before the macro \n(rather than using `@macroexpand1`).\n\nIf [Tracker.jl](https://github.com/FluxML/Tracker.jl) is loaded, then it will now\ndefine the same gradients for `B::TrackedArray` etc. \n\nNote that this is a fairly crude experiment, probably not something to rely on.\n\n### Limitations:\n\n1. The expression must be one term, and scalar factors are not handled yet.\n2. It makes no attempt to cache intermediate contractions for re-use, \n  and thus if there are many tensors it will do the same work several times\n  (like `b[i,j] * c[j,k]` above, done twice).\n3. Requires you to add `@grad` everywhere, so won't work in other people's code.\n\nI can solve 1. But 2 seems hard to solve with this design.\n\nIt now understands other macros like `@einsum` which share the same syntax. \nThis allows it to treat non-Einstein contractions, such as batched matrix multiplication:\n```julia\n@grad x @einsum z[i,k,b] := x[i,j,b] * y[j,k,b]\n```\nThose are also handled by `@ein` from [OMEinsum.jl](https://github.com/under-Peter/OMEinsum.jl),\nwhich may be pointless as that has its own gradients built-in. \nProbably you should use that instead! \n\nAn earlier attempt is now [TensorTrack.jl](https://github.com/mcabbott/TensorTrack.jl), which works at the level of \nfunctions `contract!` etc, and thus gets some re-use, 4. \nBut is completely limited by 2, being deeply plugged into TensorOperations.\n\nFinally, note also that [TensorCast.jl](https://github.com/mcabbott/TensorCast.jl) should \nbe almost fully differentiable (although focused on operations other than contractions).\n\n--- Michael Abbott, August 2019\n\n### Update:\n\nEssentially the same code has been bolted onto [Tullio.jl](https://github.com/mcabbott/Tullio.jl) \noriginally in [PR#6](https://github.com/mcabbott/Tullio.jl/pull/6), and moved to `@tensor` in [PR#92](https://github.com/mcabbott/Tullio.jl/pull/92). It has the same limitations as above.\n(But it avoids `eval` by attaching gradients to a callable struct `Eval` always, not to the newly defined functions.)\n\nThe package [TensorRules.jl](https://github.com/ho-oto/TensorRules.jl) has a macro `@∇` which performs\nmanipulations of `@tensor` expressions, acting on whole functions containing them.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmcabbott%2Ftensorgrad.jl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmcabbott%2Ftensorgrad.jl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmcabbott%2Ftensorgrad.jl/lists"}