{"id":20121236,"url":"https://github.com/iree-org/iree-torch","last_synced_at":"2025-05-06T15:30:41.515Z","repository":{"id":38300218,"uuid":"471540385","full_name":"iree-org/iree-torch","owner":"iree-org","description":"Torch Frontend for IREE","archived":true,"fork":false,"pushed_at":"2023-12-21T23:31:27.000Z","size":135,"stargazers_count":25,"open_issues_count":6,"forks_count":10,"subscribers_count":16,"default_branch":"main","last_synced_at":"2025-04-26T09:45:19.369Z","etag":null,"topics":["compiler","iree","machine-learning","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/iree-org.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-03-18T23:05:00.000Z","updated_at":"2024-06-27T09:28:09.000Z","dependencies_parsed_at":"2023-12-22T02:09:52.530Z","dependency_job_id":"77a0e9d2-e5cd-4aa4-a1a7-a5dc187732a4","html_url":"https://github.com/iree-org/iree-torch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iree-org%2Firee-torch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iree-org%2Firee-torch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iree-org%2Firee-torch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iree-org%2Firee-torch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/iree-org","download_url":"https://codeload.github.com/iree-org/iree-torch/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252712822,"owners_count":21792377,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["compiler","iree","machine-learning","pytorch"],"created_at":"2024-11-13T19:26:50.545Z","updated_at":"2025-05-06T15:30:41.229Z","avatar_url":"https://github.com/iree-org.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Inactivity Notice\n\n\u003e **Note**\n\u003e We haven't been able to invest as much time into this as we'd like lately.  For a more actively supported packaged solution combining torch-mlir and IREE, see [SHARK](https://github.com/nod-ai/SHARK).\n\n# Torch Frontend for IREE\n\nThis project provides end-to-end flows supporting users of PyTorch that want to target [IREE](https://iree-org.github.io/iree/) as a compiler backend, which offers a [number of benefits](https://iree-org.github.io/iree/#key-features). We use the [Torch-MLIR](https://github.com/llvm/torch-mlir) project to provide our PyTorch frontend.\n\nThis project is under active development and is subject to frequent changes.\n\n# Example Usage\n\n## Training \u0026 Inference (`functorch`-based)\n\nAn end-to-end example of training a PyTorch basic regression model on IREE can be found in [this script](https://github.com/iree-org/iree-torch/blob/main/examples/regression.py).  This script uses `functorch` to define the model's forward and backward pass.\n\n## Inference (`nn.Module`-based)\n\nAn end-to-end example of compiling an `nn.Module`-based PyTorch BERT model to IREE can be found in [this notebook](https://github.com/iree-org/iree-torch/blob/main/examples/bert.ipynb). The notebook also demonstrates the significantly smaller runtime size of the compiled model when compared to PyTorch (**~4MB versus ~700MB**).\n\n## Native, On-device Training\n\nA small (~100-250KB), self-contained binary can be built for deploying to resource-constrained environments. An example illustrating this can be found in [this example](https://github.com/iree-org/iree-torch/tree/main/examples/native_training). This binary runs a model without a Python interpreter.\n\n# Planned features\n\n- Python (or, if absolutely necessary, C++) code that pulls in the bindings from both projects into an end-to-end flow for users.\n- Docker images for users to be able to quickly get started\n- CI of the Torch-MLIR end-to-end tests, with IREE plugged in as a backend\n- User examples:\n  - Jupyter notebooks using the above to demonstrate interactive use of the tools\n  - Standalone user-level Python code demonstrating various deployment flows (mobile, embedded).\n\n# Running end-to-end correctness tests\n\nSetup the venv for running:\n\n```bash\n# Create a Python virtual environment.\n$ python -m venv iree-torch.venv\n$ source iree-torch.venv/bin/activate\n\n# Option 1: Install Torch-MLIR and IREE from nightly packages:\n(iree-torch.venv) $ python -m pip install -r \"${IREE_TORCH_SRC_ROOT}/requirements.txt\"\n\n# Option 2: For development, build from source and set `PYTHONPATH`:\nninja -C \"${TORCH_MLIR_BUILD_ROOT}\" TorchMLIRPythonModules\nninja -C \"${IREE_BUILD_ROOT}\" IREECompilerPythonModules bindings_python_iree_runtime_runtime\nexport PYTHONPATH=\"${IREE_BUILD_ROOT}/runtime/bindings/python:${IREE_BUILD_ROOT}/compiler/bindings/python:${TORCH_MLIR_BUILD_ROOT}/tools/torch-mlir/python_packages/torch_mlir:${PYTHONPATH}\"\n```\n\nRun the Torch-MLIR TorchScript e2e test suite on IREE:\n```bash\n# Run all the tests on the default backend (`llvm-cpu`).\n(iree-torch.venv) $ tools/e2e_test.sh\n# Run all tests on the `vmvx` backend.\n(iree-torch.venv) $ tools/e2e_test.sh --config vmvx\n# Filter the tests (with a regex) and report failures with verbose error messages.\n# This is good for drilling down on a single test as well.\n(iree-torch.venv) $ tools/e2e_test.sh --filter Elementwise --verbose\n# Shorter option names.\n(iree-torch.venv) $ tools/e2e_test.sh -f Elementwise -v\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Firee-org%2Firee-torch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Firee-org%2Firee-torch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Firee-org%2Firee-torch/lists"}