{"id":19646730,"url":"https://github.com/onnx/turnkeyml","last_synced_at":"2025-05-16T06:02:22.649Z","repository":{"id":210480003,"uuid":"725429681","full_name":"onnx/turnkeyml","owner":"onnx","description":"Local LLM Server with NPU Acceleration","archived":false,"fork":false,"pushed_at":"2025-05-13T20:45:49.000Z","size":2021,"stargazers_count":180,"open_issues_count":11,"forks_count":23,"subscribers_count":14,"default_branch":"main","last_synced_at":"2025-05-13T21:45:02.794Z","etag":null,"topics":["ai","amd","benchmark","gpu","igpu","llm","llm-server","local-server","npu","onnx","onnxruntime-genai","toolchain"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/onnx.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-11-30T05:51:52.000Z","updated_at":"2025-05-13T20:45:06.000Z","dependencies_parsed_at":"2024-01-16T23:57:29.746Z","dependency_job_id":"cc17d581-fe48-42ac-8ad0-89602b0e9470","html_url":"https://github.com/onnx/turnkeyml","commit_stats":null,"previous_names":["onnx/turnkeyml"],"tags_count":53,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/onnx%2Fturnkeyml","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/onnx%2Fturnkeyml/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/onnx%2Fturnkeyml/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/onnx%2Fturnkeyml/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/onnx","download_url":"https://codeload.github.com/onnx/turnkeyml/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254039982,"owners_count":22004408,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","amd","benchmark","gpu","igpu","llm","llm-server","local-server","npu","onnx","onnxruntime-genai","toolchain"],"created_at":"2024-11-11T14:40:04.217Z","updated_at":"2025-05-16T06:02:22.639Z","avatar_url":"https://github.com/onnx.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Welcome to ONNX TurnkeyML\n\n[![Lemonade tests](https://github.com/onnx/turnkeyml/actions/workflows/test_lemonade.yml/badge.svg)](https://github.com/onnx/turnkeyml/tree/main/test \"Check out our tests\")\n[![Turnkey tests](https://github.com/onnx/turnkeyml/actions/workflows/test_turnkey.yml/badge.svg)](https://github.com/onnx/turnkeyml/tree/main/test \"Check out our tests\")\n[![OS - Windows | Linux](https://img.shields.io/badge/OS-windows%20%7C%20linux-blue)](https://github.com/onnx/turnkeyml/blob/main/docs/install.md \"Check out our instructions\")\n[![Made with Python](https://img.shields.io/badge/Python-3.8,3.10-blue?logo=python\u0026logoColor=white)](https://github.com/onnx/turnkeyml/blob/main/docs/install.md \"Check out our instructions\")\n\nWe are on a mission to make it easy to use the most important tools in the ONNX ecosystem. TurnkeyML accomplishes this by providing a full SDK for LLMs with the Lemonade SDK, as well as a no-code CLIs for general ONNX workflows with `turnkey`.\n\n## 🍋 Lemonade SDK: Quickly serve, benchmark and deploy LLMs\n\nThe [Lemonade SDK](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/README.md) is designed to make it easy to serve, benchmark, and deploy large language models (LLMs) on a variety of hardware platforms, including CPU, GPU, and NPU. \n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://download.amd.com/images/lemonade_640x480_1.gif\" alt=\"Lemonade Demo\" title=\"Lemonade in Action\"\u003e\n\u003c/div\u003e\n\nThe [Lemonade SDK](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/README.md) is comprised of the following:\n\n- 🌐 **Lemonade Server**: A server interface that uses the standard Open AI API, allowing applications to integrate with local LLMs.\n- 🐍 **Lemonade Python API**: Offers High-Level API for easy integration of Lemonade LLMs into Python applications and Low-Level API for custom experiments.\n- 🖥️ **Lemonade CLI**: The `lemonade` CLI lets you mix-and-match LLMs, frameworks (PyTorch, ONNX, GGUF), and measurement tools to run experiments. The available tools are:\n  - Prompting an LLM.\n  - Measuring the accuracy of an LLM using a variety of tests.\n  - Benchmarking an LLM to get the time-to-first-token and tokens per second.\n  - Profiling the memory usage of an LLM.\n\n### [Click here to get started with Lemonade.](https://github.com/onnx/turnkeyml/blob/main/docs/lemonade/README.md)\n\n## 🔑 Turnkey: A Complementary Tool for ONNX Workflows\n\nWhile Lemonade focuses on LLMs, [Turnkey](https://github.com/onnx/turnkeyml/blob/main/docs/turnkey/README.md) is a no-code CLI designed for general ONNX workflows, such as exporting and optimizing CNNs and Transformers.\n\nTo see the list of supported tools, using the following command:\n\n```bash\nturnkey -h\n```\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://download.amd.com/images/tkml_640x480_1.gif\" alt=\"Turnkey Demo\" title=\"Turnkey CLI\"\u003e\n\u003c/div\u003e\n\n### [Click here to get started with `turnkey`.](https://github.com/onnx/turnkeyml/blob/main/docs/turnkey/README.md)\n\n## Contributing\n\nWe are actively seeking collaborators from across the industry. If you would like to contribute to this project, please check out our [contribution guide](https://github.com/onnx/turnkeyml/blob/main/docs/contribute.md).\n\n## Maintainers\n\nThis project is sponsored by the [ONNX Model Zoo](https://github.com/onnx/models) special interest group (SIG). It is maintained by @danielholanda @jeremyfowers @ramkrishna @vgodsoe in equal measure. You can reach us by filing an [issue](https://github.com/onnx/turnkeyml/issues) or emailing `turnkeyml at amd dot com`.\n\n## License\n\nThis project is licensed under the [Apache 2.0 License](https://github.com/onnx/turnkeyml/blob/main/LICENSE).\n\n## Attribution\n\nTurnkeyML used code from other open source projects as a starting point (see [NOTICE.md](NOTICE.md)). Thank you Philip Colangelo, Derek Elkins, Jeremy Fowers, Dan Gard, Victoria Godsoe, Mark Heaps, Daniel Holanda, Brian Kurtz, Mariah Larwood, Philip Lassen, Andrew Ling, Adrian Macias, Gary Malik, Sarah Massengill, Ashwin Murthy, Hatice Ozen, Tim Sears, Sean Settle, Krishna Sivakumar, Aviv Weinstein, Xueli Xao, Bill Xing, and Lev Zlotnik for your contributions to that work.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fonnx%2Fturnkeyml","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fonnx%2Fturnkeyml","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fonnx%2Fturnkeyml/lists"}