{"id":50666078,"url":"https://github.com/dnv-opensource/crane-controller","last_synced_at":"2026-06-08T06:30:44.317Z","repository":{"id":352847147,"uuid":"1117500075","full_name":"dnv-opensource/crane-controller","owner":"dnv-opensource","description":"Control a crane using AI-based controllers","archived":false,"fork":false,"pushed_at":"2026-06-01T07:22:16.000Z","size":1084,"stargazers_count":0,"open_issues_count":2,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-01T09:18:25.326Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dnv-opensource.png","metadata":{"files":{"readme":"README.rst","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-16T11:57:59.000Z","updated_at":"2026-04-30T08:39:06.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/dnv-opensource/crane-controller","commit_stats":null,"previous_names":["dnv-opensource/crane-controller"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/dnv-opensource/crane-controller","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dnv-opensource%2Fcrane-controller","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dnv-opensource%2Fcrane-controller/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dnv-opensource%2Fcrane-controller/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dnv-opensource%2Fcrane-controller/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dnv-opensource","download_url":"https://codeload.github.com/dnv-opensource/crane-controller/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dnv-opensource%2Fcrane-controller/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34051768,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-08T02:00:07.615Z","response_time":111,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-08T06:30:39.242Z","updated_at":"2026-06-08T06:30:44.310Z","avatar_url":"https://github.com/dnv-opensource.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"|pypi| |versions| |license| |ci| |docs|\n\nIntroduction\n============\n\nThe package provides AI-based control of a crane system using reinforcement learning, developed at DNV AS.\n\nThe primary goal is to solve the **anti-pendulum problem**: training an agent to dampen (or start)\nthe swing of a load hanging from a mobile crane, using only horizontal crane acceleration as the control input.\n\nGetting Started\n---------------\n\nEnvironments\n^^^^^^^^^^^^\n\n``AntiPendulumEnv``\n    The main environment. A mobile crane with a swinging load modelled via real crane physics\n    (``crane-controller`` library). The agent controls horizontal crane acceleration and must either\n    start or stop the pendulum motion.\n\n    - **Observation**: crane x-position, crane x-velocity, load polar angle, load x-velocity\n    - **Actions**: Discrete(3) — accelerate left / coast / accelerate right\n    - **Modes**: *start* (build pendulum energy) or *stop* (dampen swing)\n\n``ControlledCraneEnv``\n    A more general mobile crane environment for future work.\n\nAlgorithms\n^^^^^^^^^^\n\nThree RL algorithms are implemented, each as a self-contained agent class:\n\n- **PPO** (``ppo_agent.py``) — Proximal Policy Optimization via ``stable-baselines3``. Supports\n  vectorized environments for faster training. Models saved as ``.zip`` files.\n\n- **Q-Learning** (``q_agent.py``) — Tabular Q-learning with epsilon-greedy exploration.\n  Uses a discretized observation space. Q-tables saved/loaded as JSON for incremental training.\n\n- **AlgorithmAgent** (``algorithm.py``) — Brute-force search over all 81 handcoded strategies\n  (3\\ :sup:`4` combinations). Useful as a baseline.\n\nWrappers\n^^^^^^^^\n\nGeneric Gymnasium wrappers (from the Farama Foundation examples) are included for reference:\n\n- ``ClipReward`` — clips immediate rewards to a valid range\n- ``DiscreteActions`` — restricts the action space to a finite subset\n- ``RelativePosition`` — computes relative position between agent and target\n- ``ReacherRewardWrapper`` — weights multiple reward terms\n\nLearning Examples\n^^^^^^^^^^^^^^^^^\n\nTwo classic Gymnasium environments were used as stepping stones when developing this project:\n\n- **GridWorldEnv** — minimal grid navigation, ideal for learning the Gymnasium API.\n  See the `environment creation tutorial \u003chttps://gymnasium.farama.org/tutorials/gymnasium_basics/environment_creation/\u003e`_\n  and the `Gymnasium examples repo \u003chttps://github.com/Farama-Foundation/gymnasium-env-template\u003e`_.\n- **CartPoleEnv** — cart-pole balancing, useful for verifying RL algorithms before applying them\n  to the crane. Available via ``gymnasium.make(\"CartPole-v1\")``.\n\nInstallation\n------------\n\n.. code-block:: shell\n\n   pip install crane-controller\n\n\nUsage\n-----\n\nRunning\n^^^^^^^\n\nInstall dependencies and run the test suite with ``uv``:\n\n.. code-block:: shell\n\n   uv run pytest tests/ -v\n\nTest files are organised by algorithm:\n\n- ``tests/test_environment.py`` -- environment and observation space tests\n- ``tests/test_algorithm.py`` -- brute-force algorithm tests\n- ``tests/test_q.py`` -- Q-learning smoke and analysis tests\n- ``tests/test_ppo.py`` -- PPO training, VecNormalize, and inference tests\n\nTests are suitable for CI/CD — no plot windows are produced.\n\nTraining\n^^^^^^^^\n\n**PPO:**\n\n.. code-block:: shell\n\n   uv run python scripts/train_ppo.py\n\nKey options:\n\n- ``--steps N`` — total training timesteps (default: 100 000)\n- ``--n-envs N`` — number of parallel environments (default: 4)\n- ``--save-path PATH`` — where to write the trained model (default: ``models/ppo_AntiPendulumEnv.zip``)\n- ``--resume-from PATH`` — continue training from a saved checkpoint; preserves VecNormalize statistics and learning rate schedule\n- ``--dry-run`` — run 1 000 steps with a live reward-tracking plot and no model saved\n\n**Q-learning:**\n\n.. code-block:: shell\n\n   uv run python scripts/train_q.py\n\nKey options:\n\n- ``--episodes N`` — total training episodes (default: 10 000)\n- ``--v0 F`` — initial crane speed; negative = stop mode, positive = start mode (default: ``-1.0``)\n- ``--reward-limit F`` — per-episode termination threshold (default: ``-0.05``)\n- ``--save-path PATH`` — where to write the Q-table (default: ``models/q_AntiPendulumEnv.json``)\n- ``--trained PATH`` — continue training from an existing Q-table JSON\n- ``--intervals N`` — run interval training: N rounds of 10 episodes each\n- ``--dry-run`` — run 50 episodes with a reward plot and no model saved\n\nPlaying\n^^^^^^^\n\nRun a trained agent visually. Both scripts accept ``--render-mode`` with the following options:\n\n- ``plot`` — 4-panel figure per episode (load angle, crane position/speed, rewards)\n- ``play-back`` — animated crane trajectory after each episode\n- ``reward-tracking`` — live reward line plot updating every step\n\n**PPO** (default render-mode: ``play-back``):\n\n.. code-block:: shell\n\n   uv run python scripts/play_ppo.py --model-path models/ppo_AntiPendulumEnv.zip\n   uv run python scripts/play_ppo.py --model-path models/ppo_AntiPendulumEnv.zip --render-mode plot --episodes 3\n\n**Q-learning** (default render-mode: ``plot``):\n\n.. code-block:: shell\n\n   uv run python scripts/play_q.py --model-path models/q_AntiPendulumEnv.json\n   uv run python scripts/play_q.py --model-path tests/anti-pendulum.json --render-mode play-back --episodes 3\n\nAnalysing\n^^^^^^^^^\n\nInspect a trained Q-table without running the environment:\n\n.. code-block:: shell\n\n   uv run python scripts/analyse_q.py --model-path tests/anti-pendulum.json\n\nPrints per-pos/speed average Q-values for a quick sanity check. To drill into\nspecific states, use ``--obs`` with 5 integers (use ``-1`` as a wildcard):\n\n.. code-block:: shell\n\n   uv run python scripts/analyse_q.py --model-path tests/anti-pendulum.json --obs -1 0 0 -1 -1\n\nThe five observation dimensions are: ``[energy, pos, speed, distance, sector]``.\n\nDevelopment Setup\n-----------------\n\n1. Install uv\n^^^^^^^^^^^^^\nThis project uses `uv` as package manager.\n\nIf you haven't already, install `uv \u003chttps://docs.astral.sh/uv/\u003e`_, preferably using it's `\"Standalone installer\" \u003chttps://docs.astral.sh/uv/getting-started/installation/#__tabbed_1_2/\u003e`_ method:\n\n..on Windows:\n\n.. code:: sh\n\n   powershell -ExecutionPolicy ByPass -c \"irm https://astral.sh/uv/install.ps1 | iex\"\n\n..on MacOS and Linux:\n\n.. code:: sh\n\n   curl -LsSf https://astral.sh/uv/install.sh | sh\n\n(see `docs.astral.sh/uv \u003chttps://docs.astral.sh/uv/getting-started/installation//\u003e`_ for all / alternative installation methods.)\n\nOnce installed, you can update `uv` to its latest version, anytime, by running:\n\n``uv self update``\n\n2. Clone the repository\n^^^^^^^^^^^^^^^^^^^^^^^\nClone the crane-controller repository into your local development directory:\n\n.. code:: sh\n\n   git clone https://github.com/dnv-opensource/crane-controller path/to/your/dev/crane-controller\n\nChange into the project directory after cloning:\n\n.. code:: sh\n\n   cd crane-controller\n\n3. Install dependencies\n^^^^^^^^^^^^^^^^^^^^^^^\nRun ``uv sync -U`` to create a virtual environment and install all project dependencies into it:\n\n.. code:: sh\n\n   uv sync -U\n\n..\n\n   **Note**: Using ``--no-dev`` will omit installing development\n   dependencies.\n\n   **Explanation**: The ``-U`` option stands for ``--update``. It forces\n   ``uv`` to fetch and install the latest versions of all dependencies,\n   ensuring that your environment is up-to-date.\n\n..\n\n   **Note**: ``uv`` will create a new virtual environment called\n   ``.venv`` in the project root directory when running ``uv sync -U``\n   the first time. Optionally, you can create your own virtual\n   environment using e.g. ``uv venv``, before running ``uv sync -U``.\n\n4. (Optional) Activate the virtual environment\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nWhen using ``uv``, there is in almost all cases no longer a need to manually activate the virtual environment.\n\n``uv`` will find the ``.venv`` virtual environment in the working directory or any parent directory, and activate it on the fly whenever you run a command via `uv` inside your project folder structure:\n\n.. code:: sh\n\n   uv run \u003ccommand\u003e\n\nHowever, you still *can* manually activate the virtual environment if needed.\nWhen developing in an IDE, for instance, this can in some cases be necessary depending on your IDE settings.\nTo manually activate the virtual environment, run one of the \"known\" legacy commands:\n\n..on Windows:\n\n.. code:: sh\n\n   .venv\\Scripts\\activate.bat\n\n..on Linux:\n\n.. code:: sh\n\n   source .venv/bin/activate\n\n6. Install pre-commit hooks\n^^^^^^^^^^^^^^^^^^^^^^^^^^^\nThe ``.pre-commit-config.yaml`` file in the project root directory contains a configuration for pre-commit hooks.\nTo install the pre-commit hooks defined therein in your local git repository, run:\n\n.. code:: sh\n\n   uv run pre-commit install\n\nAll pre-commit hooks configured in ``.pre-commit-config.yam`` will now run each time you commit changes.\n\npre-commit can also manually be invoked, at anytime, using:\n\n.. code:: sh\n\n   uv run pre-commit run --all-files\n\nTo skip the pre-commit validation on commits (e.g. when intentionally\ncommitting broken code), run:\n\n.. code:: sh\n\n   uv run git commit -m \u003cMSG\u003e --no-verify\n\nTo update the hooks configured in ``.pre-commit-config.yaml`` to their\nnewest versions, run:\n\n.. code:: sh\n\n   uv run pre-commit autoupdate\n\n7. Test that the installation works\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nTo test that the installation works, run pytest in the project root folder:\n\n.. code:: sh\n\n   uv run pytest\n\nMeta\n----\nCopyright (c) 2026 `DNV \u003chttps://www.dnv.com/\u003e`_ AS. All rights reserved.\n\nSiegfried Eisinger - `@LinkedIn \u003chttps://www.linkedin.com/in/siegfried-eisinger-a337638b\u003e`_ - siegfried.eisinger@dnv.com\n\nAleksandar Babic - `@LinkedIn \u003chttps://www.linkedin.com/in/aleksandar-babic-no\u003e`_ - aleksandar.babic@dnv.com\n\nDistributed under the MIT license. See `LICENSE \u003cLICENSE.md/\u003e`_ for more information.\n\n`https://github.com/dnv-opensource/crane-controller \u003chttps://github.com/dnv-opensource/crane-controller/\u003e`_\n\nContributing\n------------\n\n1. Fork it `\u003chttps://github.com/dnv-opensource/crane-controller/fork/\u003e`_\n2. Create an issue in your GitHub repo\n3. Create your branch based on the issue number and type (``git checkout -b issue-name``)\n4. Evaluate and stage the changes you want to commit (``git add -i``)\n5. Commit your changes (``git commit -am 'place a descriptive commit message here'``)\n6. Push to the branch (``git push origin issue-name``)\n7. Create a new Pull Request in GitHub\n\nFor your contribution, please make sure you follow the `STYLEGUIDE \u003cSTYLEGUIDE.md/\u003e`_ before creating the Pull Request.\n\n.. |pypi| image:: https://img.shields.io/pypi/v/crane-controller.svg?color=blue\n   :target: https://pypi.python.org/pypi/crane-controller\n.. |versions| image:: https://img.shields.io/pypi/pyversions/crane-controller.svg?color=blue\n   :target: https://pypi.python.org/pypi/crane-controller\n.. |license| image:: https://img.shields.io/pypi/l/crane-controller.svg\n   :target: https://github.com/dnv-opensource/crane-controller/blob/main/LICENSE\n.. |ci| image:: https://img.shields.io/github/actions/workflow/status/dnv-opensource/crane-controller/.github%2Fworkflows%2Fnightly_build.yml?label=ci\n.. |docs| image:: https://img.shields.io/github/actions/workflow/status/dnv-opensource/crane-controller/.github%2Fworkflows%2Fpush_to_release.yml?label=docs\n   :target: https://dnv-opensource.github.io/crane-controller/README.html\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdnv-opensource%2Fcrane-controller","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdnv-opensource%2Fcrane-controller","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdnv-opensource%2Fcrane-controller/lists"}