{"id":19001285,"url":"https://github.com/oneflow-inc/oneflow-lite","last_synced_at":"2025-04-22T17:27:28.202Z","repository":{"id":103034307,"uuid":"561354520","full_name":"Oneflow-Inc/oneflow-lite","owner":"Oneflow-Inc","description":null,"archived":false,"fork":false,"pushed_at":"2024-01-01T11:20:12.000Z","size":169,"stargazers_count":18,"open_issues_count":2,"forks_count":2,"subscribers_count":31,"default_branch":"master","last_synced_at":"2025-04-17T07:17:47.600Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Oneflow-Inc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-11-03T14:08:12.000Z","updated_at":"2024-04-03T18:42:14.000Z","dependencies_parsed_at":null,"dependency_job_id":"c61a346f-a12a-49a9-a41b-fc0ddfafaea3","html_url":"https://github.com/Oneflow-Inc/oneflow-lite","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Oneflow-Inc%2Foneflow-lite","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Oneflow-Inc%2Foneflow-lite/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Oneflow-Inc%2Foneflow-lite/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Oneflow-Inc%2Foneflow-lite/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Oneflow-Inc","download_url":"https://codeload.github.com/Oneflow-Inc/oneflow-lite/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250286547,"owners_count":21405467,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-08T18:10:36.914Z","updated_at":"2025-04-22T17:27:28.175Z","avatar_url":"https://github.com/Oneflow-Inc.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"oneflow-lite是一个轻量化的模型部署框架，支持在MCU、移动端、边缘端和服务端等各种应用场景下执行高效的推理任务。\n\noneflow-lite由compiler和runtime两部分组成。compiler负责对oneflow训练保存的checkpoint进行编译优化，最终转换成oneflow-lite runtime所支持的二进制文件格式，runtime提供该二进制文件的运行时环境，驱动硬件执行计算过程，并向上提供合适的用户接口。\n\nCompiler支持对计算图进行一序列优化，包括计算融合、冗余消除、Layout变换等中端图优化，也包括内存规划、算子放置、计算图切分和后端代码生成等优化。\n\nRuntime由纯C语言开发，核心框架二进制大小只有50KB左右，包含硬件抽象层、执行的上下文和支撑库，支持以非常简洁的方式集成各类硬件模块，比如X86 CPU、英伟达CUDA GPU、华为Ascend NPU等。\n\n\n\n### 编译\n\n- Compiler\n\n  1. 下载/更新子模块\n\n     ```shell\n     git submodule init \u0026\u0026 git submodule update\n     ```\n\n  2. 编译\n\n     - 目标硬件为CUDA\n\n       ```shell\n       cd compiler \u0026\u0026 mkdir build\n       cmake .. -DLITE_USE_CUDA=ON \u0026\u0026 make\n       ```\n\n     - 目标硬件为Ascend NPU\n\n       安装华为Ascend Toolkit，到[华为Ascend网站](https://www.hiascend.com/software/cann/commercial)上下载Ascend-cann-toolkit安装包，完成安装之后执行下面的命令更新一下环境变量。\n\n       ```shell\n       source /Ascend-Toolkit-Install-Path/ascend-toolkit/set_env.sh\n       ```\n\n       接着执行下面命令完成编译。\n\n       ```shell\n       cd compiler \u0026\u0026 mkdir build\n       cmake .. -DLITE_USE_ASCEND_NPU=ON \u0026\u0026 make\n       ```\n\n  3. 检查编译结果\n\n     在bin目录下检查是否生成了编译工具oneflow-lite-compile。\n\n  注意：Compiler组件不需要在目标硬件平台上就可以编译。\n\n- Runtime\n\n  - 在目标硬件平台上编译\n\n    比如目标平台为X86 CPU + Ascend NPU，编译命令如下，\n\n    ```shell\n    cd runtime \u0026\u0026 mkdir build\n    cmake .. -DBUILD_X86=ON -DBUILD_ASCEND_NPU=ON\n    ```\n\n  - 交叉编译\n\n    暂时不支持\n\n### 使用教程\n\n- 优化和转换模型\n\n  1. 使用graph模式导出mlir模型\n\n     ```python\n     class MyGraph(nn.Graph):\n         def __init__(self, model):\n             super().__init__()\n             self.model = model\n     \n         def build(self, *input):\n             return self.model(*input)\n     \n     model = Resnet50Module()\n     model.eval()\n     graph = MyGraph(model)\n     flow.save(graph, \"./resnet50_model/\")\n\n  2. 使用`oneflow-lite-compile`工具将模型编译成`oneflow-lite`的模型文件格式（以华为Ascend NPU为例）\n\n     ```shell\n     oneflow-lite/compiler/build/bin/oneflow-lite-compile ./resnet50_model -o resnet50_lite.bin --targets=ascend\n     ```\n\n- 在目标硬件平台上执行（以华为Ascend NPU为例）\n\n  ```shell\n  source /Ascend-Toolkit-Install-Path/ascend-toolkit/set_env.sh\n  oneflow-lite/runtime/build/oneflow-lite/tests/test-resnet50 resnet50_lite.bin\n  ```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foneflow-inc%2Foneflow-lite","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foneflow-inc%2Foneflow-lite","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foneflow-inc%2Foneflow-lite/lists"}