{"id":16615704,"url":"https://github.com/shishirpatil/poet","last_synced_at":"2025-06-11T09:02:33.578Z","repository":{"id":47350840,"uuid":"515699139","full_name":"ShishirPatil/poet","owner":"ShishirPatil","description":"ML model training for edge devices ","archived":false,"fork":false,"pushed_at":"2023-09-29T03:26:21.000Z","size":175,"stargazers_count":162,"open_issues_count":9,"forks_count":19,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-03-16T06:11:16.861Z","etag":null,"topics":["compiler","deep-learning","edge-computing","ilp","machine-learning","memory-management"],"latest_commit_sha":null,"homepage":"https://shishirpatil.github.io/poet/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ShishirPatil.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2022-07-19T18:28:03.000Z","updated_at":"2025-03-08T07:51:49.000Z","dependencies_parsed_at":"2023-02-15T04:32:02.345Z","dependency_job_id":"70696b94-3eae-472c-98a4-abb6d11e5514","html_url":"https://github.com/ShishirPatil/poet","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShishirPatil%2Fpoet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShishirPatil%2Fpoet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShishirPatil%2Fpoet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShishirPatil%2Fpoet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ShishirPatil","download_url":"https://codeload.github.com/ShishirPatil/poet/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243936503,"owners_count":20371523,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["compiler","deep-learning","edge-computing","ilp","machine-learning","memory-management"],"created_at":"2024-10-12T02:10:25.036Z","updated_at":"2025-03-16T21:31:10.292Z","avatar_url":"https://github.com/ShishirPatil.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# POET\nBy Shishir G. Patil, Paras Jain, Prabal Dutta, Ion Stoica, and Joseph E. Gonzalez  ([Project Website](https://poet.cs.berkeley.edu/))\n\n![](https://github.com/ShishirPatil/poet/blob/gh-pages/assets/img/logo.png)\n\n_See the paper!_ [https://arxiv.org/abs/2207.07697](https://arxiv.org/abs/2207.07697)\n\n`POET` enables the training of state-of-the-art memory-hungry ML models on smartphones and other edge devices. POET (Private Optimal Energy Training) exploits the twin techniques of integrated tensor rematerialization, and paging-in/out of secondary storage \\(as detailed in our paper at ICML 2022\\) to optimize models for training with limited memory. POET's Mixed Integer Linear Formulation (MILP) ensures the solutions are provably optimal!\n\nWith POET, we are the first to demonstrate how to train memory-hungry SOTA ML models such as BERT and\nResNets on smartphones and tiny ARM Cortex-M devices :muscle:\n\nReach out to us at [sgp@berkeley.edu](mailto:sgp@berkeley.edu), if you have large models that you are trying to train - be it on GPUs, or your commodity edge devices such as laptops, smartphones, raspberry-pis, ARM Cortex M and A class, fitbits, etc.\n\n\n## Get Started [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1iup_edJd9zB1tfVBHXLmkWOT5yoSmXzz?usp=sharing)\n\n### Installation\n\nClone the repository and install POET:\n\n```bash\ngit clone https://github.com/ShishirPatil/poet.git\ncd poet/\npip install -e .\n```\n\n### Setting Up Gurobi (Recommended)\n\nThe ILP solver defaults to using the COIN-OR CBC solver when Gurobi isn't available. However, since Gurobi is much faster and presents solutions with tighter constraints, it is recommended to install it when possible.\n\n#### Acquiring a Free Academic Gurobi Web License\n\nIf you are affiliated with an academic institution, you can acquire a free Gurobi Web License:\n\n1. Create a free Gurobi account [here](https://pages.gurobi.com/registration). Make sure to specify the Academic user option.\n\n    \u003cimg width=\"602\" alt=\"image\" src=\"https://user-images.githubusercontent.com/52852612/206888332-cefa3d3e-9514-49f1-8bd1-82516a16ca08.png\"\u003e\n\n2. Complete the rest of the Gurobi account creation process, which will include creating a password and verifying your email address.\n3. Login to the Gurobi [Web License Manager](https://license.gurobi.com/) using your new account.\n4. Create and download a new Web License file. It will be called `gurobi.lic`.\n\n    \u003cimg width=\"493\" alt=\"image\" src=\"https://user-images.githubusercontent.com/52852612/206888423-4b3588bb-9724-4f38-96c3-778a8fff15af.png\"\u003e\n\n5. Move the `gurobi.lic` file to your home directory (i.e. to `~/gurobi.lic` on MacOS/Linux, or `C:\\Users\\YOUR_USERNAME\\gurobi.lic` on Windows).\n\n### Running the Solver via the Command Line\n\nOnce you have installed POET and optionally configured Gurobi, you can run the solver via the command line. Here's an example:\n\n```bash\npython poet/solve.py --model resnet18_cifar --platform a72 --ram-budget 3000000 --runtime-budget 7.6\n```\n\n### Using the Solver API Directly\n\nIf you'd like to use the solver API directly, you can do so as follows:\n\n```python\nfrom poet import solve\n\n# ... use the solver API here\nsolve(\n    model=\"resnet18_cifar\",\n    platform=\"m4\",\n    ram_budget=737719,\n    runtime_budget=1.5,\n    time_limit_s=400,\n    solve_threads=16,\n)\n```\n\n\n## Key ideas\n\nFrom our [paper at ICML 2022](https://arxiv.org/abs/2207.07697):\n```text\nIn this work, we show that paging and rematerialization are highly complementary.\nBy carefully rematerializing cheap operations while paging results of expensive operations\nto auxiliary memory such as a flash or an SD card, we can scale effective memory capacity\nwith minimal energy over- head. By combining these two methods, we demonstrate it is\npossible to train models like BERT on mobile-class edge devices. By framing edge training\nas an optimization problem, we discover optimal schedules with provable minimal energy\nconsumption at a given memory budget. While the focus of this paper is edge deployments,\nthe energy objective is increasingly becoming relevant even for cloud deployments!\n```\n\n## Citation\n\nIf you use POET in your work, please cite us with:\n\n```text\n@inproceedings{patil2022poet,\n  title={POET: Training Neural Networks on Tiny Devices with Integrated Rematerialization and Paging},\n  author={Patil, Shishir G and Jain, Paras and Dutta, Prabal and Stoica, Ion and Gonzalez, Joseph},\n  booktitle={International Conference on Machine Learning},\n  pages={17573--17583},\n  year={2022},\n  organization={PMLR}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshishirpatil%2Fpoet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshishirpatil%2Fpoet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshishirpatil%2Fpoet/lists"}