{"id":25776995,"url":"https://github.com/siemens/industrialbenchmark","last_synced_at":"2025-08-04T12:05:20.547Z","repository":{"id":51887282,"uuid":"70696374","full_name":"siemens/industrialbenchmark","owner":"siemens","description":"Industrial Benchmark","archived":false,"fork":false,"pushed_at":"2023-06-14T22:29:48.000Z","size":4439,"stargazers_count":131,"open_issues_count":9,"forks_count":25,"subscribers_count":11,"default_branch":"master","last_synced_at":"2025-07-28T18:57:14.323Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/1709.09480","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/siemens.png","metadata":{"files":{"readme":"README.markdown","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2016-10-12T12:11:48.000Z","updated_at":"2025-07-18T22:30:32.000Z","dependencies_parsed_at":"2025-06-29T09:38:18.314Z","dependency_job_id":"326a0380-0bfb-422b-9d15-21e938987460","html_url":"https://github.com/siemens/industrialbenchmark","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/siemens/industrialbenchmark","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/siemens%2Findustrialbenchmark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/siemens%2Findustrialbenchmark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/siemens%2Findustrialbenchmark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/siemens%2Findustrialbenchmark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/siemens","download_url":"https://codeload.github.com/siemens/industrialbenchmark/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/siemens%2Findustrialbenchmark/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":268692283,"owners_count":24291377,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-04T02:00:09.867Z","response_time":79,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-27T06:01:34.026Z","updated_at":"2025-08-04T12:05:20.422Z","avatar_url":"https://github.com/siemens.png","language":"Java","funding_links":[],"categories":["测试"],"sub_categories":[],"readme":"# Industrial Benchmark \r\n \r\nThe \"Industrial Benchmark\" is a realistic benchmark for offline RL and online RL, used to find those RL algorithms that are best suited for real-world applications.\r\n\r\nThe Industrial Benchmark includes a variety of aspects that we have identified as essential in industrial applications. It is designed to have the same difficulty and complexity as real RL applications. State- and action-space are continuous, the state-space is rather high-dimensional and only partially observable. The actions consist of three continuous components and act on three steerings. There are delayed effects.\r\n\r\nThe optimization task is multi-criterial in the sense that there are two reward components, which have opposing dependencies on the actions. The dynamical behavior is heteroskedastic with state-dependent observation noise and state-dependent probability distributions, based on latent variables. The industrial benchmark is designed in such a way that the optimal policy does not approach a fixed operating point in the three steerings.\r\nEach specific choice is based on our experience with industrial challenges.\r\n\r\n\r\nRequires: Java 8 and Apache Maven 3.x or Python 3.7\r\n\r\nFor the Python Version, the industrial benchmark environment is contained in industrial_benchmark_python/IDS.py, and there is an OpenAI Gym compliant wrapper in industrial_benchmark_python/IBGym.py\r\n\r\nYou can install the Benchmark as a package after cloning, using:\r\n\r\n\tpip install dist/industrial_benchmark_python-2.0-py3-none-any.whl\r\n\r\nOr directly from PyPI:\r\n\r\n\tpip install industrial_benchmark_python\r\n\r\nTo test whether it works you can run\r\n\r\n    cd industrial_benchmark_python\r\n    python test_gym_wrapper.py\r\n\r\nand to check out how current RL methods implemented in the stable_baselines package do on the benchmark (stable baselines needs to be installed):\r\n\r\n\tpython industrial_benchmark_python/test_baselines.py\r\n\r\nDocumentation: The documentation is available online at: https://arxiv.org/abs/1709.09480\r\n\r\n\tSource: D. Hein, S. Depeweg, M. Tokic, S. Udluft, A. Hentschel, T.A. Runkler, and V. Sterzing. \r\n\t\t\"A benchmark environment motivated by industrial control problems,\" in 2017 IEEE \r\n\t\tSymposium Series on Computational Intelligence (SSCI), 2017, pp. 1-8. \r\n\r\n## Citing Industrial Benchmark\r\n\r\nTo cite Industrial Benchmark, please reference:\r\n\r\n\tD. Hein, S. Depeweg, M. Tokic, S. Udluft, A. Hentschel, T.A. Runkler, and V. Sterzing. \"A \r\n\t\tbenchmark environment motivated by industrial control problems,\" in 2017 IEEE Symposium \r\n\t\tSeries on Computational Intelligence (SSCI), 2017, pp. 1-8. \r\n\r\nAdditional references using Industrial Benchmark:\r\n\t\r\n\tS. Depeweg, J. M. Hernández-Lobato, F. Doshi-Velez, and S. Udluft. \"Learning and\r\n\t\tpolicy search in stochastic dynamical systems with Bayesian neural networks.\" arXiv\r\n\t\tpreprint arXiv:1605.07127, 2016.\r\n\r\n\tD. Hein, S. Udluft, M. Tokic, A. Hentschel, T.A. Runkler, and V. Sterzing. \"Batch reinforcement \r\n\t\tlearning on the industrial benchmark: First experiences,\" in 2017 International Joint \r\n\t\tConference on Neural Networks (IJCNN), 2017, pp. 4214–4221.\r\n\r\n\tS. Depeweg, J. M. Hernández-Lobato, F. Doshi-Velez, and S. Udluft. \"Uncertainty decomposition \r\n\t\tin Bayesian neural networks with latent variables.\" arXiv preprint arXiv:1605.07127, \r\n\t\t2017.\r\n\t\t\r\n\tD. Hein, A. Hentschel, T. A. Runkler, and S. Udluft. \"Particle Swarm Optimization for Model \r\n\t\tPredictive Control in Reinforcement Learning Environments,\" in Y. Shi (Ed.), Critical \r\n\t\tDevelopments and Applications of Swarm Intelligence, IGI Global, Hershey, PA, USA, \r\n\t\t2018, pp. 401–427.\r\n\t\t\r\n\tS. Depeweg, J. M. Hernandez-Lobato, F. Doshi-Velez, and S. Udluft. \"Decomposition of \r\n\t\tUncertainty in Bayesian Deep Learning for Efficient and Risk-sensitive Learning.\" \r\n\t\t35th International Conference on Machine Learning, ICML 2018. Vol. 3. 2018.\r\n\t\r\n\tD. Hein, S. Udluft, and T.A. Runkler. \"Interpretable policies for reinforcement learning by \r\n\t\tgenetic programming.\" Engineering Applications of Artificial Intelligence, 76, 2018, \r\n\t\tpp. 158-169.\r\n\t\r\n\tD. Hein, S. Udluft, and T.A. Runkler. \"Generating interpretable fuzzy controllers using \r\n\t\tparticle swarm optimization and genetic programming,\" in Proceedings of the Genetic \r\n\t\tand Evolutionary Computation Conference Companion, ACM, 2018, pp. 1268-1275.\r\n\t\r\n\tN. Di Palo, and H. Valpola. \"Improving Model-Based Control and Active Exploration with \r\n\t\tReconstruction Uncertainty Optimization.\" arXiv preprint arXiv:1812.03955, 2018.\r\n\t\r\n\tF. Linker. \"Industrial Benchmark for Fuzzy Particle Swarm Reinforcement Learning.\" \r\n\t\thttp://felixlinker.de/doc/ib_fpsrl.pdf, 2019\r\n\t\r\n\tH. Zhang, A. Zhou, and X. Lin. \"Interpretable policy derivation for reinforcement learning \r\n\t\tbased on evolutionary feature synthesis.\" Complex \u0026 Intelligent Systems, 2020. \r\n\t\tpp. 1-13.\r\n\t\r\n\tP. Swazinna, S. Udluft, and T.A. Runkler. \"Overcoming Model Bias for Robust Offline Deep \r\n\t\tReinforcement Learning.\" arXiv preprint arXiv:2008.05533, 2020.\r\n\t\r\n\tA. Kumar. \"Policy Optimization for Industrial Benchmark Using Deep Reinforcement Learning.\"\r\n\t\tPhD Thesis, Colorado State University, 2020.\r\n\t\r\n\tR. Qin, S. Gao, X. Zhang, Z. Xu, S. Huang, Z. Li, Z. Weinan, and Y. Yu. \"NeoRL: A Near \r\n\t\tReal-World Benchmark for Offline Reinforcement Learning.\" arXiv preprint \r\n\t\tarXiv:2102.00714, 2021.\r\n\t\r\n\tP. Swazinna, S. Udluft, D. Hein, and T.A. Runkler. \"Behavior Constraining in Weight \r\n\t\tSpace for Offline Reinforcement Learning.\" arXiv preprint arXiv:2107.05479, 2021.\r\n\t\t\r\n\tV. Kurenkov, and S. Kolesnikov. \"Showing Your Offline Reinforcement Learning Work: Online \r\n\t\tEvaluation Budget Matters.\" arXiv preprint arXiv:2110.04156, 2021.\r\n\t\r\n\tP. Swazinna, S. Udluft, and T.A. Runkler. \"Measuring Data Quality for Dataset Selection in \r\n\t\tOffline Reinforcement Learning,\" in 2021 IEEE Symposium Series on Computational \r\n\t\tIntelligence (SSCI), 2021, pp. 1-8.\r\n\t\r\n\tP. Swazinna, S. Udluft, D. Hein, and T.A. Runkler. \"Comparing Model-free and Model-based \r\n\t\tAlgorithms for Offline Reinforcement Learning.\" arXiv preprint arXiv:2201.05433, 2022.\r\n\t\r\n\tC. Feng, and G. Jinyan. \"Reliable Offline Model-based Optimization for Industrial Process \r\n\t\tControl.\" arXiv preprint arXiv:2205.07250, 2022.\r\n\t\t\r\n\tP. Swazinna, S. Udluft, and T.A. Runkler. \"User-Interactive Offline Reinforcement Learning.\" \r\n\t\tarXiv preprint arXiv:2205.10629, 2022.\r\n\r\nAdditional references mentioning Industrial Benchmark:\r\n\r\n\tY. Li. \"Deep reinforcement learning: An overview.\" arXiv preprint arXiv:1701.07274, 2017.\r\n\t\r\n\tD. Ha, and J. Schmidhuber. \"Recurrent world models facilitate policy evolution,\" in Advances \r\n\t\tin Neural Information Processing Systems, 2018, pp. 2450-2462.\r\n\t\r\n\tM. Schaarschmidt, A. Kuhnle, B. Ellis, K. Fricke, F. Gessert, and E. Yoneki. \"Lift: \r\n\t\tReinforcement learning in computer systems by learning from demonstrations.\" arXiv \r\n\t\tpreprint arXiv:1808.07903, 2018.\r\n\t\r\n\tM. Kaiser, C. Otte, T.A. Runkler, and C.H. Ek. \"Data Association with Gaussian Processes.\" \r\n\t\tarXiv preprint arXiv:1810.07158, 2018.\r\n\t\r\n\tD. Lee, and J. McNair. \"Deep reinforcement learning agent for playing 2D shooting games.\" Int. \r\n\t\tJ. Control Autom, 11, 2018, pp. 193-200.\r\n\t\r\n\tD. Marino, and M. Manic. \"Modeling and planning under uncertainty using deep neural networks.\" \r\n\t\tIEEE Transactions on Industrial Informatics, 2019.\r\n\t\r\n\tJ. Fu, A. Kumar, O. Nachum, G. Tucker, and S. Levine. \"Datasets for Data-Driven Reinforcement \r\n\t\tLearning.\" arXiv preprint arXiv:2004.07219, 2020.\r\n\t\r\n\tM. Schaarschmidt. \"End-to-end deep reinforcement learning in computer systems.\" PhD Thesis, \r\n\t\tUniversity of Cambridge, 2020.\r\n\t\r\n\tT. Gangwani, Y. Zhou, and J. Peng. \"Learning Guidance Rewards with Trajectory-space Smoothing.\" \r\n\t\tAdvances in Neural Information Processing Systems 33, 2020.\r\n\t\r\n\tA. Zubow, S. Rösler, P. Gawłowicz, F. Dressler. \"GrGym: When GNU Radio goes to (AI) Gym,\" in \r\n\t\tProceedings of the 22nd International Workshop on Mobile Computing Systems and \r\n\t\tApplications, 2021, pp. 8-14.\r\n\t\t\r\n\tJ. McLeod, H. Stojic, V. Adam, D. Kim, J. Grau-Moya, P. Vrancx, and F. Leibfried. \"Bellman: \r\n\t\tA Toolbox for Model-Based Reinforcement Learning in TensorFlow.\" arXiv preprint \r\n\t\tarXiv:2103.14407, 2021.\r\n\t\t\r\n\tT. Xu, and Y. Liang. \"Provably Efficient Offline Reinforcement Learning with Trajectory-Wise \r\n\t\tReward.\" arXiv preprint arXiv:2206.06426, 2022.\r\n\t\t\r\n\tF.M. Luo, T. Xu, H. Lai, X.H. Chen, W. Zhang, and Y. Yu. \"A survey on model-based reinforcement \r\n\t\tlearning.\" arXiv preprint arXiv:2206.09328, 2022.\r\n\r\n\tB. Han, Z. Ren, Z. Wu, Y. Zhou, and J. Peng. \"Off-Policy Reinforcement Learning with \r\n\t\tDelayed Rewards.\" arXiv preprint arXiv:2106.11854, 2021.\r\n\t\t\r\n\tM. Kaiser. \"Structured Models with Gaussian Processes.\" Doctoral dissertation, Technische \r\n\t\tUniversität München, 2021.\r\n\t\t\r\n\tM. Videau. \"Découverte de Politiques Interprétables pour l'Apprentissage par Renforcement \r\n\t\tvia la Programmation Génétique.\" Doctoral dissertation, Université Paris Dauphine-PSL,\r\n\t\t2021.\r\n\t\r\n\tZ. Ren, R. Guo, Y. Zhou, and J. Peng. \"Learning Long-Term Reward Redistribution via Randomized \r\n\t\tReturn Decomposition.\" arXiv preprint arXiv:2111.13485, 2021.\r\n\t\r\n\tM. Videau, A. Leite, O. Teytaud, and M. Schoenauer. \"Multi-objective Genetic Programming for \r\n\t\tExplainable Reinforcement Learning,\" in European Conference on Genetic Programming \r\n\t\t(Part of EvoStar), Springer, Cham, 2022, pp. 278-293.\r\n\t\r\n\tX.-Y. Liu, Z. Xia, J. Rui, J. Gao, H. Yang, M. Zhu, C.D. Wang, Z. Wang, and J. Guo. \r\n\t\t\"FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial \r\n\t\tReinforcement Learning.\" arXiv preprint arXiv:2211.03107, 2022.\r\n\t\t\r\n\tT. Xu. \"Towards the Understanding of Sample Efficient Reinforcement Learning Algorithms.\" \r\n\t\tDoctoral dissertation, The Ohio State University, 2022.\r\n\t\r\n\tM. Schlappa, J. Hegemann, and S. Spinler. \"Optimizing Control of Waste Incineration Plants \r\n\t\tUsing Reinforcement Learning and Digital Twins.\" IEEE Transactions on Engineering \r\n\t\tManagement, 2022.\r\n\t\t\r\n\tF. Vignat, N. Béraud, and T.T.D. Montcel. \"Toolpath Calculation Using Reinforcement Learning \r\n\t\tin Machining,\" in International Joint Conference on Mechanics, Design Engineering \u0026 \r\n\t\tAdvanced Manufacturing, Springer, Cham, 2023, pp. 1149-1158.\r\n\t\t\r\n\tF. Huang, J. Xu, D. Wu, Y. Cui, Z. Yan, W. Xing, and X.Zhang. \"A general motion controller \r\n\t\tbased on deep reinforcement learning for an autonomous underwater vehicle with \r\n\t\tunknown disturbances.\" Engineering Applications of Artificial Intelligence 117, 2023.\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsiemens%2Findustrialbenchmark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsiemens%2Findustrialbenchmark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsiemens%2Findustrialbenchmark/lists"}