{"id":18711199,"url":"https://github.com/rocm/aotriton","last_synced_at":"2026-04-17T22:08:59.923Z","repository":{"id":226689411,"uuid":"706046142","full_name":"ROCm/aotriton","owner":"ROCm","description":"Ahead of Time (AOT) Triton Math Library","archived":false,"fork":false,"pushed_at":"2025-04-09T20:46:10.000Z","size":69645,"stargazers_count":57,"open_issues_count":15,"forks_count":20,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-04-09T21:38:02.625Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ROCm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-17T07:48:23.000Z","updated_at":"2025-04-09T20:46:14.000Z","dependencies_parsed_at":"2024-03-09T02:29:41.429Z","dependency_job_id":"1c12a489-802f-417c-885d-df909867f6e4","html_url":"https://github.com/ROCm/aotriton","commit_stats":null,"previous_names":["rocm/aotriton"],"tags_count":17,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ROCm%2Faotriton","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ROCm%2Faotriton/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ROCm%2Faotriton/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ROCm%2Faotriton/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ROCm","download_url":"https://codeload.github.com/ROCm/aotriton/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248562841,"owners_count":21125154,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T12:37:44.275Z","updated_at":"2026-04-17T22:08:59.917Z","avatar_url":"https://github.com/ROCm.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Build Instructions\n\n```\npip install -r requirements.txt\nmkdir build\ncd build\nexport PKG_CONFIG_PATH=\"${PKG_CONFIG_PATH}:${CONDA_PREFIX}/lib/pkgconfig\"\ncmake .. -DCMAKE_INSTALL_PREFIX=./install_dir -DCMAKE_BUILD_TYPE=Release -DAOTRITON_GPU_BUILD_TIMEOUT=0 -G Ninja\n# Use ccmake to tweak options\nninja install/strip  # Use `ninja install` to keep symbols\n```\n\nThe library and the header file can be found under `build/install_dir` afterwards.\nYou may ignore the `export PKG_CONFIG_PATH` part if you're not building with conda\n\nNote: do not run `ninja` separately, due to the limit of the current build\nsystem, `ninja install` will run the whole build process unconditionally.\n\n### Prerequisites\n\n* `python \u003e= 3.10`\n  - For `python \u003e= 3.14`, Triton needs patch https://github.com/triton-lang/triton/commit/c44b870bdd9e1ea8933fd4057b6b59a5e6e5407b\n    to fix `ast.Num` error.\n* `gcc \u003e= 8` or `clang \u003e= 10`\n  - For Designated initializers, but only gcc \u003e= 9 is tested.\n  - The binary delivery is compiled with gcc13\n* `cmake \u003e= 3.26`\n  - Only `cmake \u003e= 3.30` is tested\n* `ninja`\n  - Only `ninja \u003e= 1.11` is tested\n  - `ninja \u003e= 1.13.1` on Windows due to https://github.com/ninja-build/ninja/issues/2616\n* `liblzma`\n  - Common names are `liblzma-dev` or `xz-devel`.\n* `pkg-config`\n  - Required by the build system to find `liblzma`.\n  - On RHEL and its derivatives this dependency is ensured by `xz-devel`.\n  - On Debian and its derivatives this dependency is met by installing `pkgconf`\n    or `pkg-config` in older releases.\n* [`dlfcn-win32`](https://github.com/dlfcn-win32/dlfcn-win32) (**WINDOWS ONLY**)\n  - Windows version of the `dl` library.\n\n## Generation\n\nThe kernel definition for generation is done in\n[rules.py](https://github.com/ROCm/aotriton/blob/main/python/rules.py). Edits\nto this file are needed for each new kernel, but it is extensible and generic.\n\nInclude files can be added in\n[this](https://github.com/ROCm/aotriton/tree/main/include/aotriton) directory.\n\nThe final build output is an archive object file any new project may link\nagainst.\n\nThe archive file and header files are installed in the path specified by\n`CMAKE_INSTALL_PREFIX`.\n\n## Kernel Support\n\nCurrently the first kernel supported is FlashAttention as based on the\n[algorithm from Tri Dao](https://github.com/Dao-AILab/flash-attention).\n\n## PyTorch Consumption \u0026 Compatibility\n\nAOTriton is consumed in PyTorch through\nthe [SDPA kernels](https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/native/transformers/hip/flash_attn/aot/mha_all_aot.hip).\nThe precompiled binaries will be downloaded and shipped with PyTorch during [builds](https://github.com/pytorch/pytorch/blob/main/cmake/External/aotriton.cmake).\n\nCAVEAT: As a fast moving target, AOTriton's FlashAttention API changes over\ntime. Hence, a specific PyTorch release is only compatible with a few versions\nof AOTriton. The compatibility matrix is shown below\n\n|  PyTorch Upstream     |           AOTriton Feature Release              |\n|-----------------------|-------------------------------------------------|\n|        2.9 (upcoming) |    0.11b, 0.10b\u003csup\u003e(1)\u003c/sup\u003e                   |\n|        2.8            |    0.10b\u003csup\u003e(2)\u003c/sup\u003e, 0.9b\u003csup\u003e(2)\u003c/sup\u003e      |\n|        2.7            |    0.9b\u003csup\u003e(4)\u003c/sup\u003e, 0.10b\u003csup\u003edrop-in only(3)\u003c/sup\u003e      |\n|        2.6            |                   0.8b\u003csup\u003e(5)\u003c/sup\u003e            |\n|        2.5            |                   0.7b, 0.8b\u003csup\u003e(6)\u003c/sup\u003e      |\n|        2.4            |                   0.6b                          |\n|        2.3            |                   0.4b                          |\n|  2.2 and earlier      |               N/A, no support                   |\n\n1. Using AOTriton 0.10b on PyTorch 2.9 will re-introduce Context Parallelism Bug:\n   [Issue 156012](https://github.com/pytorch/pytorch/issues/156012)\n2. PyTorch 2.8 will lose sliding window attention (SWA) support if built with\n   0.9b since this feature is newly added in 0.10b. In addition,\n   https://github.com/pytorch/pytorch/pull/159773 is needed to properly\n   integrate SWA into PyTorch.\n3. 0.10b is backward compatible with 0.9b's API. Hence it can be used as a drop-in\n   replacement for installed PyTorch wheels by symlinking\n   `libaotriton_v2.so.0.9.2` to `libaotriton_v2.so.0.10.0`. However, 0.10b\n   cannot be built with PyTorch 2.7 due to the integrity check in the\n   integration code.\n4. To be specific, it is shipped with 0.9.2b. Other versions like 0.9b and 0.9.1b\n   should not be used in order to avoid linking issues, and also avoid\n   confusion about version strings.\n5. PyTorch 2.6 requires some 0.8b-only features. Hence even if PyTorch 2.6\n   can compile with 0.7b due to API compatibility, the end product will\n   suffer from runtime errors.\n6. 0.8b's API is backward compatible with 0.7b, but the packaging scheme\n   has changed drastically.\n\nROCm's PyTorch release/\\\u003cversion\\\u003e branch is slightly different from PyTorch\nupstream and may support more recent version of AOTriton\n\n|  PyTorch ROCm Fork    |           AOTriton Feature Release              |\n|-----------------------|-------------------------------------------------|\n|        2.9 (upcoming) |                   0.11b                         |\n|        2.8            |                   0.10b (backported)            |\n|        2.7            |                   0.9b (backported)             |\n|        2.6            |                   0.9b (backported)             |\n|        2.5            |                   0.9b (backported)             |\n|        2.4            |                   0.10b (backported)            |\n|        2.3            |                   0.4b                          |\n|  2.2 and earlier      |               N/A, no support                   |\n\n### Point Release\n\nAOTriton's point releases maintain ABI compatibility and can be used as drop-in\nreplacement of their corresponding feature releases.\n\n### Windows Limitations\n\n1. AOTriton on Windows currently isn't able to build kernel images by itself.\n   This is because triton is not officially available on Windows yet.\n2. To build on Windows, set AOTRITON_NOIMAGE_MODE and use the `aotriton.images`\n   folder from a Linux build.\n3. The Windows version uses dlfcn-win32 which doesn't support file paths with\n   unicode characters in them. A fix for this is planned.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frocm%2Faotriton","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frocm%2Faotriton","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frocm%2Faotriton/lists"}