{"id":21683371,"url":"https://github.com/matamegger/reinforced-pid-parameter","last_synced_at":"2026-04-30T06:31:29.770Z","repository":{"id":187077446,"uuid":"433154026","full_name":"matamegger/reinforced-pid-parameter","owner":"matamegger","description":null,"archived":false,"fork":false,"pushed_at":"2021-12-10T16:12:02.000Z","size":1194,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-20T11:25:10.396Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/matamegger.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2021-11-29T18:28:34.000Z","updated_at":"2021-12-10T16:12:07.000Z","dependencies_parsed_at":"2023-08-08T22:25:53.993Z","dependency_job_id":null,"html_url":"https://github.com/matamegger/reinforced-pid-parameter","commit_stats":null,"previous_names":["matamegger/reinforced-pid-parameter"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/matamegger/reinforced-pid-parameter","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matamegger%2Freinforced-pid-parameter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matamegger%2Freinforced-pid-parameter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matamegger%2Freinforced-pid-parameter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matamegger%2Freinforced-pid-parameter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/matamegger","download_url":"https://codeload.github.com/matamegger/reinforced-pid-parameter/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matamegger%2Freinforced-pid-parameter/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261373597,"owners_count":23148918,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-25T16:11:43.448Z","updated_at":"2026-04-30T06:31:24.750Z","avatar_url":"https://github.com/matamegger.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Reinforcement Learning to find PID Parameter\n\nThis project represents a basic showcase of Python bindings for Simulink model binaries in computational demanding tasks.\n\nA Simulink model of a Spring-Mass-Damper system controlled by a PID controller is the base of the showcase. Using [SliM-PyB](https://github.com/matamegger/slim-pyb), the model is converted to native binaries and the according Python bindings.\n\nThe model is then applied in a reinforcement learning environment to find the optimal controller parameters for specific scenarios.\n\n## Reinforcement Learning Environment\n\nThe setup of the ML environment shouldn't be seen as a best practice guide. It was created with limited knowledge and no previous experience.\n\nIn the environment the agent can give only one action (i.e. static PID parameters) for the simulation to be finished. The action consists of the three control parameters of the PID controller (Kp, Ki, Kd).\nUsing the provided parameters a 50 seconds long simulation of the controlled Spring-Mass-Damper system is executed.\nAfter the simulation the inverse normalized square of the error is used as the reward function of the agent.\n\n![](pictures/reward_function.png)\n\n_(Where `e` is the error [difference between input and output signal] in the simulation step `i` and `T` the absolute number of steps)_\n\n\nThe inverse was needed, because otherwise the reward would sum up to very high numbers, which the RL library could not handle.\n\n## Trained Model\n\nThe model in the project is trained with 150k `timesteps` on mixed control environments (a `step` and a `sinus` input signal).\n\n## Prerequisits\n\nPython3 as well as [`pipenv`](https://pypi.org/project/pipenv/) must be installed. The remaining dependencies should be automatically handled with the `Pipfile`s.\n\n## Usage\n_The pre-trained model `PID-Parameter-Model` will always be loaded if it exists in the filepath. Else a new model is created._\n\n```\npython3 main.py\n```\nUses the model to get the PID parameters and plots a step response of the system.\n\nProviding `sin` as an argument (`python3 main.py sin`) will use a sinus input function instead of the step.\n\nTo also train the model before showing a system response `-t` must be provided as a command line argument.\nTrainings always run for 25k timesteps and will override the model on disk.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatamegger%2Freinforced-pid-parameter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmatamegger%2Freinforced-pid-parameter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatamegger%2Freinforced-pid-parameter/lists"}