{"id":24770823,"url":"https://github.com/lmarzocchetti/inventorysystem","last_synced_at":"2026-04-09T21:15:10.893Z","repository":{"id":273907011,"uuid":"866616587","full_name":"lmarzocchetti/InventorySystem","owner":"lmarzocchetti","description":"Inventory System with simpy, with a Reinforcement Learning Agent","archived":false,"fork":false,"pushed_at":"2025-05-27T11:23:40.000Z","size":124,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-29T00:42:47.006Z","etag":null,"topics":["python","pytorch","reinforcement-learning","simpy","simulation"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lmarzocchetti.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-10-02T15:19:14.000Z","updated_at":"2025-05-27T11:24:34.000Z","dependencies_parsed_at":"2025-05-12T14:53:58.242Z","dependency_job_id":"849b2611-8a45-4a53-bb96-60670dda3d7e","html_url":"https://github.com/lmarzocchetti/InventorySystem","commit_stats":null,"previous_names":["lmarzocchetti/inventorysystem"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/lmarzocchetti/InventorySystem","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmarzocchetti%2FInventorySystem","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmarzocchetti%2FInventorySystem/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmarzocchetti%2FInventorySystem/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmarzocchetti%2FInventorySystem/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lmarzocchetti","download_url":"https://codeload.github.com/lmarzocchetti/InventorySystem/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmarzocchetti%2FInventorySystem/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":272192669,"owners_count":24889452,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-26T02:00:07.904Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["python","pytorch","reinforcement-learning","simpy","simulation"],"created_at":"2025-01-29T03:56:41.462Z","updated_at":"2026-04-09T21:15:10.850Z","avatar_url":"https://github.com/lmarzocchetti.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# InventorySystem\nInventory System with simpy, with Reinforcement Learning Agent (DQN and REINFORCE)\n\n## Problem Description\nWarehouse wants to stock and sell 2 different Products.\nThe simulation's parameters are:\n- demand_inter_arrival_mean_time = 0.1,\n- order_setup_cost = 10,\n- order_incremental_cost = 3,\n- holding_cost = 1,\n- shortage_cost = 7\n\nFirst product's parameters are:\n- name = \"First Product\",\n- demand_distribution = ([1, 2, 3, 4], [1/6, 1/3, 1/3, 1/6]),\n- lead_time_min = 0.5,\n- lead_time_max = 1.0\n\nSecond product's paramters are:\n- name = \"Second Product\",\n- demand_distribution = ([5, 4, 3, 2], [1/8, 1/2, 1/4, 1/8]),\n- lead_time_min = 0.2,\n- lead_time_max = 0.7\n\n## Choosing actions\nObviously we need to control the simulation, so the warehouse need to order\nevery day (or not), some quantity for the each Product.\n\n### S-min S-max Policy\nSimple policy that states: If the stocked quantity of a product is less than\nS-min, then order products up to S-max\n\n### Reinforcement Learning\nI have implemented from scratch these two methods:\n\n#### DQN\nClassic Deep Q Network, so a Value-based approach with Replay Buffer, Temporal Difference\n\n#### REINFORCE\nAnother classic algorithm, Policy-based approach, Monte-carlo\n\n## Performance\nI have optimized each methods to act, empirically, the best that they could. Then i have tested\nthese methods, to see which is the best (basically minimizing the total_cost after a year):\n1. DQN: 248427\n2. REINFORCE: 364943\n3. S-min S-max: 703995\n\n## Usage\nI used Python 3.13.3, but for sure python 3.11 and 3.12 will still works.\n```\n$ pip install -r requirements.txt\n```\n\n### Train\n```\n$ python main.py train {dqn,reinforce} --save_path {folder}\n```\n\n### Test\nIn case of smin_smax you don't need to specify the load path\n```\n$ python main.py test {dqn,reinforce,smin_smax} [--load_path {path_to_pt_file}]  \n```\n\n### Changing Hyper Parameters\nAs of right now you need to modify them inside the code, maybe i can introduce others CLI flags to specify them.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flmarzocchetti%2Finventorysystem","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flmarzocchetti%2Finventorysystem","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flmarzocchetti%2Finventorysystem/lists"}