{"id":19001205,"url":"https://github.com/oneflow-inc/oneflow-xrt","last_synced_at":"2025-08-26T06:38:02.459Z","repository":{"id":38844725,"uuid":"486130573","full_name":"Oneflow-Inc/oneflow-xrt","owner":"Oneflow-Inc","description":null,"archived":false,"fork":false,"pushed_at":"2023-04-25T09:16:06.000Z","size":282,"stargazers_count":23,"open_issues_count":0,"forks_count":3,"subscribers_count":35,"default_branch":"master","last_synced_at":"2025-07-21T11:45:36.202Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Oneflow-Inc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-04-27T09:35:36.000Z","updated_at":"2024-11-21T07:25:17.000Z","dependencies_parsed_at":"2024-01-16T01:55:47.586Z","dependency_job_id":"93450a43-6fc3-4678-ba74-80dae0de1a2d","html_url":"https://github.com/Oneflow-Inc/oneflow-xrt","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Oneflow-Inc/oneflow-xrt","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Oneflow-Inc%2Foneflow-xrt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Oneflow-Inc%2Foneflow-xrt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Oneflow-Inc%2Foneflow-xrt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Oneflow-Inc%2Foneflow-xrt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Oneflow-Inc","download_url":"https://codeload.github.com/Oneflow-Inc/oneflow-xrt/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Oneflow-Inc%2Foneflow-xrt/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":272186211,"owners_count":24888333,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-26T02:00:07.904Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-08T18:10:21.105Z","updated_at":"2025-08-26T06:38:02.442Z","avatar_url":"https://github.com/Oneflow-Inc.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"## OneFlow-XRT\n\nOneFlow-XRT is an OneFlow extension that provides an easy to use, flexible and unified way to integrate third-party computing engines in OneFlow.\n\nOneFlow-XRT has support multiple third-party computing engines, such as XLA and TensorRT. Different engines support different backend hardware. For the same computing graph, XRT allows multiple computing engines to be used in combination to obtain better acceleration effects.\n\n| engine       | device         | inference | training                    |\n| ------------ | -------------- | --------- | --------------------------- |\n| XRT-XLA      | X86 CPU + CUDA | \u0026#10004;  | \u0026#10004;                    |\n| XRT-TensorRT | CUDA           | \u0026#10004;  | \u0026#10004; only no weights op |\n| XRT-OpenVINO | X86 CPU        | \u0026#10004;  | \u0026#10004; only no weights op |\n| XRT-TVM      | -              | -         | -                           |\n\n\n## Architecture\n\n\u003cimg width=\"600\" alt=\"XRT Architecture\" src=\"https://user-images.githubusercontent.com/13991173/178210972-7e2edf05-8078-4045-8f72-290e78949f40.png\"\u003e\n\n\n## Installation\n\n### pip\n\nTo install OneFlow-XRT via pip, use the following command:\n\n```shell\n# TODO\n# pip3 install oneflow_xrt\n\n# run the following commands according to your needs\n# pip3 install oneflow_xrt_xla\n# pip3 install oneflow_xrt_tensorrt\n# pip3 install oneflow_xrt_openvino\n```\n\n### Building From Source\n\n#### Prerequisites\n\n- install cmake\n- install oneflow\n- install CUDA if oneflow supports CUDA device or building TensorRT\n- install bazel if building XLA\n- download and unzip TensorRT if building TensorRT\n- download and unzip OpenVINO runtime if building OpenVINO\n\n#### Get the OneFlow-XRT Source\n\n```shell\ngit clone https://github.com/Oneflow-Inc/oneflow-xrt\n```\n\n#### building\n\nInside OneFlow-XRT source directory, then run the following command to install `oneflow_xrt`:\n\n```shell\npython3 setup.py install\n```\n\nThe following components are optional, run the command to install it according to your needs,\n\n- `oneflow_xrt_xla`\n\n```shell\nBUILD_XLA=ON python3 setup.py install\n```\n\n- `oneflow_xrt_tensorrt`\n\n```shell\nBUILD_TENSORRT=ON TENSORRT_ROOT=/home/TensorRT-8.4.0.6 python3 setup.py install\n```\n\n- `oneflow_xrt_openvino`\n\n```shell\nBUILD_OPENVINO=ON OPENVINO_ROOT=/home/intel/openvino_2022.1.0.643/runtime python3 setup.py install\n```\n\n## Run A Toy Program\n\n```python\n# python3\n\n\u003e\u003e\u003e import oneflow as flow\n\u003e\u003e\u003e import oneflow_xrt as ofrt\n\u003e\u003e\u003e m = flow.nn.Linear(3, 4).to(\"cuda\")\n\u003e\u003e\u003e m = ofrt.XRTModule(m, engine=[\"tensorrt\"])\n\u003e\u003e\u003e x = flow.randn(4, 3, device=\"cuda\")\n\u003e\u003e\u003e y = m(x)\n\u003e\u003e\u003e print(y)\ntensor([[ 0.2404,  0.7121,  0.4473,  0.4782],\n        [-0.8697,  1.5353,  0.2829,  0.4772],\n        [-0.3865, -1.2719,  1.0911,  0.1179],\n        [ 0.3779,  0.7363,  0.5319,  0.3167]], device='cuda:0', dtype=oneflow.float32)\n```\n\n\n\n## Documentation\n\n- [OneFlow XRT overall architecture and introduction](https://github.com/Oneflow-Inc/oneflow-xrt/wiki/OneFlow-XRT整体架构及简介)\n- [Summary of operators in OneFlow XRT](https://github.com/Oneflow-Inc/oneflow-xrt/wiki/Summary-of-operators-in-OneFlow-XRT)\n- [How to use low precision in OneFlow XRT](https://github.com/Oneflow-Inc/oneflow-xrt/wiki/OneFlow-XRT如何使用低精度计算)\n- [How to extend custom engine components](https://github.com/Oneflow-Inc/oneflow-xrt/wiki/如何在XRT框架下添加自定义的后端引擎)\n\n\n## Roadmap\n\nTODO\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foneflow-inc%2Foneflow-xrt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foneflow-inc%2Foneflow-xrt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foneflow-inc%2Foneflow-xrt/lists"}