{"id":19382859,"url":"https://github.com/locuslab/e2e-model-learning","last_synced_at":"2025-04-23T20:32:31.206Z","repository":{"id":38375848,"uuid":"84876374","full_name":"locuslab/e2e-model-learning","owner":"locuslab","description":"Task-based end-to-end model learning in stochastic optimization","archived":false,"fork":false,"pushed_at":"2020-11-29T02:51:58.000Z","size":1240,"stargazers_count":203,"open_issues_count":2,"forks_count":62,"subscribers_count":13,"default_branch":"master","last_synced_at":"2025-04-02T20:11:21.908Z","etag":null,"topics":["deep-learning","machine-learning","optimization","paper","pytorch","stochastic-optimizers"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/locuslab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-03-13T21:28:48.000Z","updated_at":"2025-02-26T02:44:20.000Z","dependencies_parsed_at":"2022-08-26T15:51:18.599Z","dependency_job_id":null,"html_url":"https://github.com/locuslab/e2e-model-learning","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/locuslab%2Fe2e-model-learning","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/locuslab%2Fe2e-model-learning/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/locuslab%2Fe2e-model-learning/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/locuslab%2Fe2e-model-learning/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/locuslab","download_url":"https://codeload.github.com/locuslab/e2e-model-learning/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250509866,"owners_count":21442514,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","machine-learning","optimization","paper","pytorch","stochastic-optimizers"],"created_at":"2024-11-10T09:23:37.301Z","updated_at":"2025-04-23T20:32:29.741Z","avatar_url":"https://github.com/locuslab.png","language":"Python","readme":"# Task-based End-to-end Model Learning in Stochastic Optimization\n\nThis repository is by \n[Priya L. Donti](https://www.priyadonti.com),\n[Brandon Amos](http://bamos.github.io),\nand [J. Zico Kolter](http://zicokolter.com)\nand contains the [PyTorch](https://pytorch.org) source code to\nreproduce the experiments in our paper\n[Task-based End-to-end Model Learning in Stochastic Optimization](https://arxiv.org/abs/1703.04529).\n\nIf you find this repository helpful in your publications,\nplease consider citing our paper.\n\n```\n@inproceedings{donti2017task,\n  title={Task-based end-to-end model learning in stochastic optimization},\n  author={Donti, Priya and Amos, Brandon and Kolter, J Zico},\n  booktitle={Advances in Neural Information Processing Systems},\n  pages={5484--5494},\n  year={2017}\n}\n```\n\n# Introduction\n\nAs machine learning techniques have become more ubiquitous, it has become \ncommon to see machine learning prediction algorithms operating within some \nlarger process. However, the criteria by which we train machine learning \nalgorithms often differ from the ultimate criteria on which we evaluate them.\n\nThis repository demonstrates an end-to-end approach for learning probabilistic \nmachine learning models within the context of stochastic programming, in a \nmanner that directly captures the ultimate task-based objective for which they \nwill be used. Specifically, we evaluate our approach in the context of\n(a) a generic inventory stock problem and (b) an electrical grid scheduling\ntask based on over eight years of data from PJM.\n\nPlease see our paper [Task-based End-to-end Model Learning in Stochastic Optimization](https://arxiv.org/abs/1703.04529)\nand the code in this repository ([locuslab/e2e-model-learning](https://github.com/locuslab/e2e-model-learning))\nfor more details about the general approach proposed and our initial\nexperimental implementations.\n\n\n## Setup and Dependencies\n\n+ Python 3.x/numpy/scipy\n+ [cvxpy](http://www.cvxpy.org/en/latest/) 1.x\n+ [PyTorch](https://pytorch.org) 1.x\n+ [qpth](https://github.com/locuslab/qpth) 0.0.15:\n  *A fast QP solver for PyTorch released in conjunction with the paper \n  [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://arxiv.org/abs/1703.00443).*\n+ [bamos/block](https://github.com/bamos/block):\n  *An intelligent block matrix library for numpy, PyTorch, and beyond.*\n+ pandas/matplotlib/seaborn\n+ Optional: [bamos/setGPU](https://github.com/bamos/setGPU):\n  A small library to set `CUDA_VISIBLE_DEVICES` on multi-GPU systems.\n+ Optional: setproctitle: A library to set process names.\n\n# Inventory Stock Problem (Newsvendor) Experiments\n\nExperiments considering a \"conditional\" variation of the inventory stock problem. \nProblem instances are generated via random sampling.\n\n```\nnewsvendor\n├── main.py - Run inventory stock problem experiments. (See arguments.)\n├── task_net.py - Functions for our task-based end-to-end model learning approach.\n├── mle.py - Functions for linear maximum likelihood estimation approach.\n├── mle_net.py - Functions for nonlinear maximum likelihood estimation approach.\n├── policy_net.py - Functions for end-to-end neural network policy model.\n├── batch.py - Helper functions for minibatched evaluation.\n├── plot.py - Plot experimental results.\n└── constants.py - Constants to set GPU vs. CPU.\n```\n\n# Load Forecasting and Generator Scheduling Experiments\n\nExperiments considering a realistic grid-scheduling task, in which\nelectricity generation is scheduled based on some (unknown) distribution\nover electricity demand. Historical load data for these experiments were obtained from\n[PJM](http://www.pjm.com/markets-and-operations/ops-analysis/historical-load-data.aspx).\n\n```\npower_sched\n├── main.py - Run load forecasting problem experiments. (See arguments.)\n├── model_classes.py - Models used for experiments.\n├── nets.py - Functions for RMSE, cost-weighted RMSE, and task nets.\n├── plot.py - Plot experimental results.\n├── constants.py - Constants to set GPU vs. CPU.\n└── pjm_load_data_*.txt - Historical load data from PJM.\n```\n\n# Price Forecasting and Battery Storage Experiments\n\nExperiments considering a realistic battery arbitrage task, in which\na power grid-connected battery generates a charge/discharge schedule \nbased on some (unknown) distribution\nover energy prices. Historical energy price data for these experiments were obtained from\n[PJM](http://www.pjm.com/markets-and-operations/energy/real-time/monthlylmp.aspx).\n\n```\nbattery_storage\n├── main.py - Run battery storage problem experiments. (See arguments.)\n├── model_classes.py - Models used for experiments.\n├── nets.py - Functions for RMSE and task nets.\n├── calc_stats.py - Calculate experimental result stats.\n├── constants.py - Constants to set GPU vs. CPU.\n└── storage_data.csv - Historical energy price data from PJM.\n```\n\n### Acknowledgments\n\nThis material is based upon work supported by the \nNational Science Foundation Graduate Research Fellowship Program under\nGrant No. DGE1252522. \n\n# Licensing\n\nUnless otherwise stated, the source code is copyright\nCarnegie Mellon University and licensed under the\n[Apache 2.0 License](./LICENSE).\n","funding_links":[],"categories":["Paper implementations｜论文实现","Paper implementations"],"sub_categories":["Other libraries｜其他库:","Other libraries:"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flocuslab%2Fe2e-model-learning","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flocuslab%2Fe2e-model-learning","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flocuslab%2Fe2e-model-learning/lists"}