{"id":13642590,"url":"https://github.com/google/ml-compiler-opt","last_synced_at":"2025-04-20T16:32:47.143Z","repository":{"id":37048261,"uuid":"267719344","full_name":"google/ml-compiler-opt","owner":"google","description":"Infrastructure for Machine Learning Guided Optimization (MLGO) in LLVM.","archived":false,"fork":false,"pushed_at":"2025-04-14T21:18:22.000Z","size":3084,"stargazers_count":676,"open_issues_count":55,"forks_count":99,"subscribers_count":19,"default_branch":"main","last_synced_at":"2025-04-15T19:53:50.848Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/google.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"docs/contributing.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-05-28T23:32:39.000Z","updated_at":"2025-04-15T00:58:59.000Z","dependencies_parsed_at":"2023-02-11T21:01:06.600Z","dependency_job_id":"fa848c30-41d1-4e1a-899e-5e3195140ee4","html_url":"https://github.com/google/ml-compiler-opt","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google%2Fml-compiler-opt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google%2Fml-compiler-opt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google%2Fml-compiler-opt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google%2Fml-compiler-opt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/google","download_url":"https://codeload.github.com/google/ml-compiler-opt/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249926575,"owners_count":21346587,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T01:01:33.572Z","updated_at":"2025-04-20T16:32:47.134Z","avatar_url":"https://github.com/google.png","language":"Python","readme":"# Infrastructure for MLGO - a Machine Learning Guided Compiler Optimizations Framework.\n\nMLGO is a framework for integrating ML techniques systematically in LLVM. It\nreplaces human-crafted optimization heuristics in LLVM with machine learned\nmodels. The MLGO framework currently supports two optimizations:\n\n1.  inlining-for-size([LLVM RFC](https://lists.llvm.org/pipermail/llvm-dev/2020-April/140763.html));\n2.  register-allocation-for-performance([LLVM RFC](https://lists.llvm.org/pipermail/llvm-dev/2021-November/153639.html))\n\nThe compiler components are both available in the main LLVM repository. This\nrepository contains the training infrastructure and related tools for MLGO.\n\nWe currently use two different ML algorithms: Policy Gradient and Evolution\nStrategies to train policies. Currently, this repository only support Policy\nGradient training. The release of Evolution Strategies training is on our\nroadmap.\n\nCheck out this [demo](docs/inlining-demo/demo.md) for an end-to-end demonstration of how\nto train your own inlining-for-size policy from the scratch with Policy\nGradient, or check out this [demo](docs/regalloc-demo/demo.md) for a demonstration of how\nto train your own regalloc-for-performance policy.\n\nFor more details about MLGO, please refer to our paper\n[MLGO: a Machine Learning Guided Compiler Optimizations Framework](https://arxiv.org/abs/2101.04808).\n\nFor more details about how to contribute to the project, please refer to\n[contributions](docs/contributing.md).\n\n## Pretrained models\n\nWe occasionally release pretrained models that may be used as-is with LLVM.\nModels are released as github releases, and are named as\n[task]-[major-version].[minor-version].The versions are semantic: the major\nversion corresponds to breaking changes on the LLVM/compiler side, and the minor\nversion corresponds to model updates that are independent of the compiler.\n\nWhen building LLVM, there is a flag `-DLLVM_INLINER_MODEL_PATH` which you may\nset to the path to your inlining model. If the path is set to `download`, then\ncmake will download the most recent (compatible) model from github to use. Other\nvalues for the flag could be:\n\n```sh\n# Model is in /tmp/model, i.e. there is a file /tmp/model/saved_model.pb along\n# with the rest of the tensorflow saved_model files produced from training.\n-DLLVM_INLINER_MODEL_PATH=/tmp/model\n\n# Download the most recent compatible model\n-DLLVM_INLINER_MODEL_PATH=download\n```\n\n## Prerequisites\n\nCurrently, the assumptions for the system are:\n\n*   Recent Ubuntu distro, e.g. 20.04\n*   python 3.10.x/3.11.x\n*   for local training, which is currently the only supported mode, we recommend\n    a high-performance workstation (e.g. 96 hardware threads).\n\nTraining assumes a clang build with ML 'development-mode'. Please refer to:\n\n*   [LLVM documentation](https://llvm.org/docs/CMake.html)\n*   the build\n    [bot script](https://github.com/google/ml-compiler-opt/blob/main/buildbot/buildbot_init.sh)\n\nThe model training - specific prerequisites are:\n\nPipenv:\n```shell\npip3 install pipenv\n```\n\nThe actual dependencies:\n```shell\npipenv sync --system --categories \"packages dev-packages ci\"\n```\nNote that the above command will only work from the root of the repository\nsince it needs to have `Pipfile.lock` in the working directory at the time\nof execution.\n\nThe above command will also install all the packages, including development\npackages (the `dev-packages` category), and packages only needed in CI (the\n`ci` category). If you do not need those, you can omit them from the categories\noption.\n\nOptionally, to run tests (run_tests.sh), you also need:\n\n```shell\nsudo apt-get install virtualenv\n```\n\nNote that the same tensorflow package is also needed for building the 'release'\nmode for LLVM.\n\n## Docs\n\nAn end-to-end [demo](docs/inlining-demo/demo.md) using Fuchsia as a codebase from which\nwe extract a corpus and train a model.\n\n[How to add a feature](docs/adding_features.md) guide.\n[Extensibility model](docs/extensibility.md).\n","funding_links":[],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle%2Fml-compiler-opt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoogle%2Fml-compiler-opt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle%2Fml-compiler-opt/lists"}